Image Classifier: Doodle Recognition with Convolutional Neural Network, Part 1 – Project Setup

Part 1 of the step by step video tutorial series on making a game like “Quick, Draw!“, an image classifier powered by Machine Learning.

In this part, we will start developing our own game called Doodle Predictor that runs directly in the browser and recognizes doodles.

So let’s get started with creating the main programming structure and basics of the user interface.

 

1. Video

To find out what we’re going to do in this part, watch this video:

 

 

2. Libraries

As we already said in the introduction, we will use Phaser framework to create all the necessary game objects, build the game logic, develop the user interface and so on.

Later, we will use Tensorflow.js library to implement machine learning.

So we need the following libraries:

• Phaser.js
• a free 2D game framework for making HTML5 games for desktop and mobile
• note: the game uses old Phaser 2 because at the time of development that was the current stable version!

• Tensorflow.js
• a JavaScript library for training and deploying Machine Learning models in the browser

 

 

3. Main Program (main.js)

The main program is consisted of two major sections:

1. Game Setup – used to create a new Phaser Game object.
2. Main State – used to implement the game logic.

 

3.1. Game Setup

In Phaser, the first thing we need to do is to set up the game as follows:

1. on window load event create a new game object as an instance of the Phaser.Game
2. create all states and attach them to the previously created game object
3. start the default state

 

3.2. Main State

In Phaser, all the action occurs around Phaser States.

To keep things simple, we will add only the Main State and implement the following three Phaser.State methods:

• preload()
• the first method called to load all assets

• create()
• this method is called only once immediately after all assets are loaded in memory
• used to create all game objects such as sprites, graphics, text fields and so on

• update()
• after all objects are created, this method will be called on every game tick
• used as the main game loop so the entire game logic goes here

To keep the source code clearer and easier to read when it later becomes bigger, it’s a good idea to break the Main State into smaller parts using a simple finite state machine.

So we can define the following sub-states inside the Main State:

3.3. Pseudo Code of the Main Program

Setup Game on Window Load {
	create a new game using Phaser.Game
	add MainState to the game 
	start MainState
}

MainState {
	preload(){
		load all assets
	}
	
	create(){
		create all objects
	}
	
	update(){
		on MODE_INIT: 
			initialize the game
			
		on MODE_OPEN_FILE: 
			start loading dataset
			
		on MODE_LOAD_FILE: 
			wait to load dataset
			
		on MODE_START_TRAIN: 
			start training model
			
		on MODE_DO_TRAIN: 
			wait to train model
			
		on MODE_START_PREDICT: 
			start predicting the samples
			
		on MODE_DO_PREDICT:
			wait to predict all samples
			
		on MODE_DRAW:
			draw a doodle and predict it
	}
}

 

4. User Interface (ui.js)

To allow users an interaction with the game we need to create an User Interface (UI) with buttons, messages, head-up display and so on.

For now, we will only create a new UI class with the following objects:

1. “Play More Game” button which opens this Web Site
2. Text for showing messages in the status bar

As we develop the game, we will gradually add all the other elements of the user interface.

 

5. Playing Doodle Predictor (Part 1)

After completing the part 1, the game looks pretty miserable, but this is just beginning!

Play in Full Screen

 

 

6. Source Code

The complete source code for Doodle Predictor is available on Github:
https://github.com/ssusnic/Machine-Learning-Doodle-Recognition

In the next part, we will see how to get and prepare data. So stay tuned!

 

---