Is it (F)ake?!

Image Classification using TensorFlow.js and Elasticsearch

This repo documents my exciting journey into learning machine learning practices for the web. It is featured in the talk of the same name at Devoxx UK 2024 and HalfStack London 2024.

You can play the game and pit yourself against the models at https://is-it-fake.netlify.app/.

Structure

This repo contains two applications:

1. Is it (F)ake?! game, located in the cake-app folder where players can guess if the image is cake or not cake. This application is written using React, Elasticsearch Node.js client and Netlify functions.
2. The Model Classification application is written in vanilla JavaScript, HTML and CSS. It is present in the model-playground folder and contains several useful elements:

• A sample webpage showing predictions generated using the COCO-SD and MobileNet model on a sample image.
• The classify-images.js script includes a Node.js implementation generating image classifications of a set of source images for MobileNet and COCO-SSD. These results are persisted in Elasticsearch.
• A custom sequential image classification model is created and executed in custom-model.js. These results are also persisted in Elasticsearch, and the model is saved in the model folder.
• The script transfer-learning.js includes an example approach combining MobileNet with an additional classification head to determine the presence of cake in our image set. Just like our other scripts, the generated classifications are persisted in Elasticsearch.

How to run

See below for how to run the applications and tests.

cake-game

You need an Elasticsearch cluster to pull the data from. Access is granted using environment variables ELASTIC_URL and ELASTIC_API_KEY, as per the .env-example file at the top of the project:

ELASTIC_URL=https://my-elastic-deployment:port
ELASTIC_API_KEY=my-api-key

Start the application on http://localhost:5173/, ensuring you have Netlify CLI installed:

cd ./cake-game
npm install
npm install netlify-cli -g
netlify dev

Execute the component tests:

cd ./cake-game
npm install
npm run test-ct

model-classification-app

To start the model application on http://localhost:3000:

cd ./model-classification-app
npm install
npm run start

Each of the scripts for performing different types of image classification should be executed from the model-classification-app folder. To persist the classifications please ensure you have a simple Elasticsearch cluster available to persist the results, with environment variables ELASTIC_URL, ELASTIC_API_KEY for Elasticsearch and UNSPLASH_ACCESS_KEY for your Unsplash API key exposed to your service as per the .env-example file as the top of the project.

ELASTIC_URL=https://my-elastic-deployment:port
ELASTIC_API_KEY=my-api-key
UNSPLASH_ACCESS_KEY=my-access-unsplash-key

Once the Elasticsearch and Unsplash API prerequisite has been met you can run the scripts in turn similar to the below:

cd ./model-classification-app
npm install

# Node 22 required for TensorFlow (as of 2025)
nvm install v22.1.0
nvm use v22.1.0

node scripts/classify-images.js
node scripts/custom-model.js
node scripts/grayscale-custom-model.js
node scripts/transfer-learning.js

Alternatively, to persist your results to an alternative store you need to override the respective functions in elasticsearch-util.js to send your classifications to the right place.

Learning Resources

1. TensorFlow.js API Reference
2. MobileNet
3. COCO-SSD
4. Kaggle Models

Machine Learning Explained

1. IBM: What are convolutional neural networks?
2. Kaggle: Basic Terminologies of Convolutional Neural Networks
3. National High Magnetic Field Laboratory: Convolution Kernels
4. Deeplizard: Max Pooling in Convolutional Neural Networks
5. Govinda Dumane: Introduction to Convolutional Neural Network (CNN) using Tensorflow
6. AI Wiki: Accuracy and Loss

Tutorials

1. Classification Model On Custom Dataset Using Tensorflow.js Made Simple: Here's What You Need to Know | Garima Nisha
2. Machine Learning for Web Developers (WebML) | Google for Developers
3. Training an image model with TensorFlow in Node.js | ????
4. TensorFlow.js: Make your own "Teachable Machine" using transfer learning with TensorFlow.js | Google Codelab
5. Google: Google AI for JavaScript developers with TensorFlow.js

Common Issues

1. CORS Issue with TensorFlow.js
2. TensorFlow incorrect tensor shape sample error
3. Tensor has disposed
4. TensorFlow.js TypeError: Cannot read property 'backend' of undefined
5. TensorFlow.js Cannot read property 'backend' of undefined in NodeJS
6. TensorFlow.js TypeError: Cannot read property 'backend' of undefined - tensor is already disposed when moveData is called

Sources

Thanks go to the fantastic bakers out there making hyper-realistic cakes that inspired this talk and have given me great family memories. Specific thanks go to these bakers whose images I used for training and classification:

1. https://www.thelondonbaker.com
2. https://hemussweetsensation.com
3. https://www.everythingjustbaked.com/
4. https://ninamariacharles.com
5. https://www.thesugardreams.net
6. https://juliemcallistercakes.com
7. https://www.thesweetstopofrva.com
8. https://apriljulian.com/portfolio
9. https://cakeyall.com/
10. https://www.layerscakestudio.com/
11. https://www.apinchofspirit.com/gallery
12. https://www.mycakebuds.com
13. https://mannandcobakeshop.com/
14. https://www.thesculptedslice.com/

Thanks also to Unsplash and their amazing contributors who allowed me to find non-cake images through the JavaScript wrapper unsplash-js of the Unsplash API.