# Machine Learning Algorithm for Flappy Bird using Neural Network and Genetic Algorithm

Welcome to a complete HTML5 tutorial with demo of a machine learning algorithm for the Flappy Bird video game. The aim of this experiment is programming an artificial intelligence game controller using neural networks and a genetic algorithm.

Hence, we want to create an AI robot which can learn how to optimally play the Flappy Bird game. As a result, our little bird should be able to fly safely through a number of barriers. In the best scenario, it should never die.

When you read all theory behind this project, you can download the source code at the end of this tutorial. All code is written in HTML5 using Phaser framework.  Also, we used Synaptic Neural Network library to implement neural network instead of making it from scratch.

## The Demo

First of all, let’s start with the live demo to see the algorithm in action:

## The Video Presentation

Additionally to the previous live demo, here you can watch a short video with a simple presentation of the algorithm. So this should be great for those who like to fast forward things!

## What Is Machine Learning Algorithm

According to Arthur Samuel in 1959, machine learning is the science of getting computers to act without being explicitly programmed. Generally speaking, this is a fine tuning process of learning that incrementally improves an initial random system.

Therefore, here is the goal to achieve an artificial intelligence which can find a proper solution from a bad system by fine tuning model parameters. To do that, machine learning algorithm uses a number of different approaches.

Specifically for this project, the main approach of machine learning algorithm (ML) is based on the NeuroEvolution (or neuro-evolution). This form of machine learning uses evolutionary algorithms such as a genetic algorithm (GA) to train artificial neural networks (ANN).

So for this example, we can say ML = GA + ANN.

## Artificial Neural Network

An artificial neural network is a subset of machine learning algorithm. It is inspired by the structure and functions of biological neural networks. These networks are made out of many neurons which send signals to each other.

Therefore, to create an artificial brain we need to simulate neurons and connect them to form a neural network.

A generic artificial neural network consists of an input layer, one or more hidden layers and an output layers. Each layer has a number of neurons. Input and output neurons are connected directly to an external environment. Finally, hidden neurons are connected between them.

In this project, each unit (bird) has its own neural network used as its AI brain for playing the game. It consists of 3 layers as follows:

• 1) an input layer with 2 neurons representing what a bird sees:

• horizontal distance to the closest gap
• height difference to the closest gap

• 2) a hidden layer with 6 neurons
• 3) an output layer with 1 neuron which provides an action as follows:

• if output > 0.5 then flap else do nothing

The picture below shows the neural network architecture for this example:

## Genetic Algorithm

When we talked about machine learning algorithm, we said that a genetic algorithm is used to train and improve neural networks.

Genetic algorithm is a search-based optimization technique inspired by the process of natural selection and genetics. It uses the same combination of selection, crossover and mutation to evolve initial random population.

Here are the main steps of our genetic algorithm implementation:

• 1. create initial population of 10 units (birds) with random neural networks
• 2. let all units play the game simultaneously by using their own neural networks
• 3. for each unit calculate its fitness function to measure its quality
(for more details see fitness function below)
• 4. when all units died, evaluate the current population to the next one by using genetic operators
(for more details see replacement strategy below)
• 5. go back to the step 2

### Fitness Function

In addition to the genetic algorithm (step 3), here we go with more details about fitness function – what it is and how to define it.

Since we want to evolve a population by using the best units, we need to define a fitness function.

Generally, the fitness function is the metrics to measure quality of an object. While we have a quality measure for each bird, we can select the fittest units and use them to reproduce the next population.

In this project, we reward a bird equally to its travelled distance. Also, we penalize it by its current distance to the closest gap. So in that way, we are making a difference between birds which travelled the same distance.

To conclude, our fitness function is the difference between the total distance covered by a bird and its current distance to the closest gap.

### Replacement Strategy

In addition to the genetic algorithm (step 4), here are the steps for applying natural evolution on dying population. Basically, the best units survive and their children replace the worst units in this way:

• 1. sort the units of the current population by their fitness ranking
• 2. select the top 4 units (winners) and pass them directly on to the next population
• 3. create 1 offspring as a crossover product of two best winners
• 4. create 3 offsprings as a crossover products of two random winners
• 5. create 2 offsprings as a direct copy of two random winners
• 6. apply random mutations on each offspring to add some variations

## The Source Code

https://github.com/ssusnic/Machine-Learning-Flappy-Bird

## Machine Learning Algorithm for Flappy Bird – Conclusion

In this tutorial we have successfully implemented AI robot for learning how to play the Flappy Bird game. As a result of several iterations, we can get an almost invincible player. To achieve that goal we have used two approaches of machine learning algorithms: artificial neural networks and genetic algorithm.

As a future work, you can try to change some parameters in code and see what happens. For instance, you can change the number of neurons in hidden layer or number of units in population. Also, you can try to change the fitness function somehow. Furthermore, change some physical parameters such as the distance between barriers, gravity and so on!

Also, try to apply the same idea of evolution to some other games!

## 21 thoughts on “Machine Learning Algorithm for Flappy Bird using Neural Network and Genetic Algorithm”

1. Anees ul Mehdi says:

Hi

Nice article. But what I didn’t get is how did you train your ANN using GA? If I am not mistaken, did you use the outcome of GA as training set for ANN?

1. Each unit starts with its own random ANN. When all units from the current population are killed we are using GA to select the best ones and make crossover to get the offsprings from the best parent. Here, parents and their offsprings are improved ANN in fact. So we are using GA operators to produce a new population with units which have better ANN than the units from the previous population.

1. Montana Burr says:

So, you have a bunch of bird with their own neural networks. My question is, what exactly does your genetic algorithm evolve – weights? number of hidden nodes?

1. It evolves weights of connections and bias information.

2. mayank satnalika says:

Can you explain what is the mutation done between? For example if there are 2 top neural networks each having a set of weights. Now if I a to form a new network from these two, how should I select the weight? Is it average of the weights of the original top 2 performers or say I select the weights of 1st layer from first network, and others from thee 2nd network or something else. Thanks

1. Inside the genetic algorithm, there is defined mutation rate to always take about 20% offsprings from the new population for mutation. To mutate these units, their connection weights are just a little bit changed using some random value.

3. Nelson says:

I love what you have shown us , the application of machine learning in playing games.

But I wonder how do you get the needed data from the game, like the distance or game states, and feed them into your programme?

Actually can you also tell me other than this particular game, what are the usual ways for people to get data from a flash game or video game or html game? Is it from screen capture followed by examination into the pixels? or there are kind of backdoor APIs to get the data?

Thanks!

1. No, there is nothing like getting data from screen capture. The neural networks need only two input parameters: current distance to the closest gap (gap_position_X – bird_position_X) and height difference to the closest gap (gap_position_Y – bird_position_Y).

4. Jay says:

How do I get the program running?

1. Here is a quick guide (probably you will need to do some more configuration steps after installing a web-server):

1. Install some web-server on your machine (Xampp or Wamp)
2. copy and paste all files in a web-server folder with “permits public access” (it’s \htdocs folder on Xampp so you can put all files in Xampp\htdocs\flappy folder)
3. Start web-server (click on Start Apache button in Xampp Control Panel)
4. open a web browser and enter the link http://localhost/flappy to run the application

Or you can try without a web server by copy&paste all files in some folder on your disk and then open a web browser and just drag and drop index.html in it (I found Firefox is the best choice if you are using this method, it seems other browsers don’t want to start html5 apps in that way)

5. Moh says:

Hi, can you please tell me why you used 6 hidden neurons, why not 4, why not 8? what’s special about 6?

1. Well, it’s a bit of a mix between art and science. You can use less or more than 6 neurons, it is your choice. I used 6 neurons because I found it is an optimal number of hidden neurons for this example – with less neurons the system needs more time to produce a good population, with more neurons the system needs to do more calculations in each iteration.

6. Bob Jones says:

What’s the crossover strategy?

1. The crossover algorithm swaps ‘bias’ information between both parents as follows:
1. get a random crossover cutting point
2. copy from the first parent its left side to the crossover point
3. copy from the second parent its right side after the crossover point

1. Any specific reason why you swap only the bias weights and not all weights?

1. Well, during the development, I noticed it works fine with swapping only the bias weights so I used only that.

7. can you make a video doing this in real time? please! 🙂 to learn even how to install the framework. Thanks!

1. Just download all files from Github and put them to a local webserver on your machine (for instance you can use Xampp or Wamp server). The project contains index.html file and a number of javascript files including the source code and frameworks. To run the application start the webserver and then open a proper link pointing to the index.html in your web browser.

8. Zekraim says:

How exactly does the input get changed by the connections and become the hidden layer, and how do the hidden layers become the output? What operations are used, and in what way? Thank you, and great article.

1. In this example, the neural network is implemented by using Synaptic Neural Network library. So I didn’t make this part from the scratch but used a complete neural network as a black box. I am just sending inputs to it and reading its output.

9. I muѕt thank you for the efforts you have put in writing this site.
I really hope to check out the same high-grade content by you later on as well.
Іn fact, your creativе writing abilities has motivated me to get my very own website now ;
)