Usage

1. Donwload and run executable (for your OS) in Releases.
2. Application is hosted at localhost:3000.
3. Click start
4. Wait until some training is done (Calculations done is recommended to be at least 20,000)
5. Click stop
6. Click play to see results (game is played between two best bots)
7. Training can be resumed by clicking Start again

compiling

requirements

• node installed
• yarn installed

Building executable

• Run yarn dist
• Executable is located inside dist folder

dev

• yarn ci or yarn install (if not called already)
• yarn dev

Application

This project was changed from racing game into duel game of two bots. (See old).

UI

Left side contains list of cards with best bots. Left side contains name of bot, blue bar with bonus and red bar with health. Right sides contains number of games bot played, number of bots wins, number of childrens (mutated compies of this bot) and steps from it's last game. Right side contains player which replays duel of best two bots.

Evolutionary algorithm

First version of this project used game for one player. Newer version introduces duel game, which cause problems with evolving bots. Instead of incrementaly improving bots fitness and approximating some theoretical fitness boundary, bots was oscilating in low and high values. It was probably caused by switching between two (or possibly more) strategies, instead of improving existing ones.

Problem with oscilating was so unexpected for me and I didn't manage to find online solution for this kind of problem using neural nets and evolutinary algorithms. So I have used method which does not use generation but is pruning and creating new bot's in one same population (method probably called live learning or similar).

Despite the fact of using method of learning that better suits this particular game problem, training works best with used super parameters inside source of this project and even minor tweak in value of some of super parameters can make learning process fail. Suggested next step to improve learning process would be using some other machine learning method as meta learning to find best super parameters for this project.

Bot has sensor*2 + 2 neural net input values. Each sensor has type of detected object and range of that detection. Last two inputs are bots health and cooldown for shooting weapon.

Algorithm

Every bot has it's own health and bonus score. Bot's health is increased proportionally to health at end of each game. If bots health exceedes max-health, exceeding healths is added into bonus score instead. score is also increased when population is low (and lowered when population is high), so population can be kept in comfort range for training.

Algorithm step

1. Select two random bots ramdomly. Using weighted random -> weight is increased by: steps from last game, wins and other bot properties.
2. Play game with selected bots
3. Increase/Lower their health base on game result.
4. Transfer exceeding health into bonus.
5. Remove bots with no health.
6. Reset bonus metter and create mutated copy of bot for bots with maxed bonus

Game

Each player is in his corner facing into center of map at start of each game. Bots has sensor attached asi inputs to their neural nets. Players health decreases each game step (motivation for bots to play aggressive -> faster the game is, lower the damage bot gets from game itself). Player can walk straight, rotate and shoot. Bullets can bounce and their damage is proportional to their speed. Motionless bullets are destroyed.

old (racing) version documentation

Code documented in this section is already deleted. This documetation is only for comparisson with newer version of bot.

Learning steering car inside 2D using genetic algorithm

Every cas has:

• neural net.
• 3 inputs: Left, Right, Forward.
• 5 sensors measuring distance from nearest obstacle

Every neural net has it's score based on checkpoints reached. Car that manages reach goal has score 100.

controls:

• Maximum generation: max generation if no winner found
• Population size: car count
• Mutation rate: chance of change inside neural net
• Weighted/Percent: type of selecting adepts to pass genes to next generation
  • weighted:
  • Percent:
• Hidden layers: every number represents number of neurons inside layer

Pressing Start will start algorithmic calculations (can take some time). Plot wiht maximal fitnesses is shown when algorithm is done running.