nicholas mifsud. behaviour trees (bt) proposed as an improvement over finite state machines (fsm) ...
TRANSCRIPT
Evolving Behaviour Trees for different AI-
bots Nicholas Mifsud
Behaviour Trees (BT) proposed as an improvement over Finite State Machines (FSM)
BTs are simple to design, implement, easily scalable, modular and reusable
BTs provide a hierarchical way of organising behaviours in a descending order of complexity (broad tasks on top, sub-tasks at the bottom)
Each tree has one defined high level behaviour, multiple trees are used to define an AI-bot
Introduction
Nodes are divided in two categories – ◦ Control Nodes
Drive execution flow through the tree, deciding which nodes to execute next
Sequence – execute children left to right until one fails (logical AND)
Selector – execute children until one fails (logical OR)◦ Leaf Nodes
Conditions – query the game state (left most) Actions – carry out specific tasks (sequence of
actions follow conditions)
Behaviour Trees
Lim et al. investigate feasibility of applying evolutionary techniques to develop BTs for competitive AI-bots
Authors give a brief outline of ◦ DEFCON◦ Relevance of BTs◦ Four different fitness functions◦ Experiments and Results◦ Future Work
BTs for DEFCON
Multiplayer Real-Time Strategy game Six world territories Players have set of units and structures at
their disposal Default AI-bot uses a deterministic FSM
consisting of 5 sets and follows them in sequence
AI-bot was given all functionalitites to play the game (makes use of DEFCON API)
Decisions performed randomly
DEFCON
Original set of trees were hand defined in order to cater for all possible actions
Handcrafted trees then used as a basis to produce more complex behaviours
Develop AI-bot behaviour by following an evolution algorithm (genetic operators + fitness functions)
Evolved behaviour trees for individual behaviours and combined the best performing trees
DEFCON BTs
Tree structures naturally allow genetic operators to modify the behaviours
Crossover on branches Mutation on nodes
◦ Random mutation ◦ Incremental mutation
Add a new branch to a behaviour◦ Point mutation
Change the spawn point of a structure/unit
Genetic Operators
Defence potential – total number of air units destroyed in a given game
Uncovered enemy units – number of enemy sea units uncovered
Fleet Actions – number of enemy buildings uncovered, buildings destroyed and number of units destroyed
Timing of attacks – difference between final end scores as an overall tally (large difference would mean convincing win)
Fitness Functions
Four different experiments run each to develop a different fitness described above
Each population contained 100 individuals
Each experiment evolved between 80 to 250 generations
Mutation rate set to 5%
Experiment
All results indicate the mean of each fitness function increased as more generations were produced
Higher percentage win rate
Results
AI-bot constructed with a controller that used the best trees evolved for each of the four behaviours
Played 200 games against the default AI-bot Won 55% of the time Games that it lost, was by a very low
margin Before evolving the behaviour trees it only
won 3%
Results
By using evolutionary computing and BTs, able to defeat a handcrafted AI-bot
Mean fitness reached a plateau indicating more generations may be redundant
Raises question whether other techniques could be coupled with this process
Not all possibilities of game explored (only two territories and only 4 tasks were considered)
Conclusions and Future Work
Perez et al. also investigate the applicability of Genetic Programming to evolve BTs
Authors make reference to DEFCON paper discussed above and state that their work may be extended by using Grammar Based Genetic Programming systems such as Grammatical Evolution
Develop AI agent for the Mario AI Benchmark following Grammatical Evolution
BTs using Grammatical Evolution
Specify syntax of possible solutions through a context free grammar
Variable length integer strings are evolved following a genetic algorithm and these then choose the production rules from the grammar until all symbols are mapped
Grammatical Evolution
Open source software based on the Mario World
All information given in two matrices (21x21)
Data about geometry of level and enemies that populate it is given
Information about current state of game is also given (position, status, mode etc)
Mario has 6 effectors (up, down, left, right, jump and fire and jump)
Mario AI Benchmark
At every cycle a button needs to be pressed in order to move Mario – impacts execution of BT
Control Nodes◦ Sequence – logical AND ◦ Selector – logical OR◦ Filters - added to add loops
Leaf Nodes ◦ Conditions – use level info to perform checks◦ Actions – Mario’s possible movements◦ Sub-Trees – designed to solve specific problems
(long jumps)
Behaviour Trees for Mario
The BT syntax was coded into the grammar◦ 30 conditions◦ 8 actions◦ 19 sub-trees◦ 4 filters
Evolution combines these as long as syntax is respected
Had to limit certain rule combinations through the grammar since some trees were impossible to read and too demanding to execute
Incorporation into GE
Trees can have variable length but follow an and-or structure
Trees consist of a root node and number of sub trees (behaviour blocks)
Each block consists of one or more conditions followed by a sequence of actions
Tree has a default sub tree that is selected if no conditions are satisfied
If no default then it could be the case that Mario does not move
Incorporation into GE
Each behaviour block is self contained and hence it allows for individuals to exchange these between them in order to produce different behaviours
Use two point crossover Allow for sub tree swap operation also
(internal crossover)
Genetic Operators
Each individual evaluated on 18 levels Elitism ensures % of population kept from
one generation to the next At the end of all runs, all best individuals
were evaluated on 600 unseen maps
Experiments
Best BT generated comprised of four behaviour blocks
Results
BT sent to Mario AI competition and managed to score in fourth place, proving BTs validity in the field of AI (relatively high kill count)
First and third place used A* variants Second place used a neural network
Competition
Use of Grammar simplifies the task of encoding the syntax of the BTs
Remarkable reactive behaviour capabilities but does not excel at planning
Hybrid approach under construction to aid in path planning
Conclusion and Future Work
DEFCON starts with a set of hand crafted trees, encoding feasible behaviours for each of the game four parts
Separate genetic programs were run for each part, creating new predefined behaviours from the original set
Main difference is that the BTs used in the Mario agent were encoded into a context free grammar in order to reduce complexity and create the behaviour blocks
Mario BTs also have loops defined in structure
Comparison of BTs Approach