herding - cpp · 2016. 2. 9. · herding sheep often predator/prey scenarios involve: use of...
TRANSCRIPT
![Page 1: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/1.jpg)
Herding
Dr. Daisy Tang
http://www.youtube.com/watch?v=wJUVULU7GDM
![Page 2: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/2.jpg)
Objectives
� Understand how herding behaviors can be generated
� Understand engineered solutions vs. learned solutions
� Understand line-of-sight herding
![Page 3: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/3.jpg)
Introduction to Herding
� What is herding?
� Has different meanings to different researchers:
� Common uses of the term “herding”:
� Herding = coming together as a group(often, to avoid predators)
� Herd of African Antelope
� Herding = surrounding targets and moving them to some destination
� Herding sheep
� Often predator/prey scenarios involve:
� Use of herding to surround targets, then declaring victory (prey are eaten)
![Page 4: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/4.jpg)
Engineered vs. Learned Solutions
� Two approaches to designing complex cooperative behavior:
� Engineer solution “by hand”
� Create learning environment where cooperative behavior can be learned or evolved
� How do you decide which to use?
![Page 5: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/5.jpg)
Evolution of Herding Behavior in Artificial Animals
Gregory M. Werner and Michael G. Dyer
AI Lab @ UCLA, 1993
![Page 6: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/6.jpg)
Abstract
� Simulation of evolution of herding behavior in prey animals
� A population of preys were placed with a population of predators in an environment
� Biot is controlled by a neural networkspecified by its individual genome
� Evolving herding behavior in simple artificial animals (“biots”) that do not originally herd
![Page 7: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/7.jpg)
Herding, Flocking and Schooling
� Many types of animals seek the company of others
� safety from predators
� access to mates
� help in finding food
![Page 8: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/8.jpg)
Motivations for Herding
� Herding provides safety for individual� increases vigilance of individual
� no need to have everyone watch out; less likely a danger will be overlooked
� confuses or intimidates a predator� musk oxen form a circle
� baboons mob leopards
� makes it less likely for predators to find prey
� selfish-herding
� desire to remain in the center vs. greater resources at the periphery
� increases feeding
� brings together mates
� etc.
![Page 9: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/9.jpg)
This Approach
� Objective: simulate the evolution of herding behavior in prey animals that do not originally herd
� A bottom-up exploration of the evolution of cooperative behavior and communication
� BioLand contains information relevant to the life of simple animals (biots)
� The biots should evolve cooperative skills similar to those in nature
![Page 10: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/10.jpg)
Approach: Environment (1)
� 2-D environment with 2 species “deer” and “wolves” and 2 non-animal objects “plants” and “trees”� Deer increase energy by eating plants
� Wolves increase energy by eating deer
� Both expend energy in eating, mating, existence, etc.
� Wrap-around simulation map� Animals that go up through the top appear at the
bottom
� Effectively no boundaries
![Page 11: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/11.jpg)
Approach: Environment (2)
� Populate with trees and edible plants
� Trees cannot be eaten, simply add complexity to map
� Plants tend to cluster, encouraging aggregation in “deer”
� Supply with initial population of plants to provide energy
� Plants added at constant rate
� They are placed near other existing plants
![Page 12: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/12.jpg)
Approach: Environment (3)
� Each object has a unique “smell”� Smell strength decays exponentially with
distance
� Can be sensed by a biot
� Non-animal entities (trees, plants) produce smells of constant strength
� Animals have characteristic “sounds”� Volume of sound associated with movement is
proportional to speed
� Mating, eating have different sounds (frequency, louderness)
� Sounds are used as signals to others animals
![Page 13: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/13.jpg)
Costs and Gain for Each Action
Parameters need to be reasonably set
![Page 14: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/14.jpg)
Modeling Biots (1)
� Behavior of each biot is determined by a neural network� Specific sensory neurons detect “smells” and
“sounds” (wolf, plant, tree, etc.), as well as other indicators (energy level, being eating, etc.)
� Motor neurons control movement, mating impulse, eating, etc.
� The above are predetermined
� Connections, hidden neurons are allowed to evolve
![Page 15: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/15.jpg)
Modeling Biots (2)
� Sensory neurons placed on two distinctly located antennae
� One on right, one on left
� Difference between the two allows biots to determine directional gradients, locatingobjects by sound or smell
![Page 16: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/16.jpg)
Modeling Biots (3)
� Genetic algorithm determines neural network architecture for individual biots
� Genetic string encodes synaptic strength, source and destination
� When animals mate, the genomes of parents are combined using crossover and mutation, which is translated into the neural network that controls the biot
![Page 17: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/17.jpg)
Experiment
� Initial population:
� 8,000 genetically random wolves
� 8,000 genetically random deer
� 1,000 by 1,000 length environment, with initial clusters of plants
plant on left
plant on right
deer on left
deer on right
female deer on left
female deer on right
forward
backward
go left
go right
……
……
Randomly connected initially
![Page 18: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/18.jpg)
Experiment
� Behaviors of populations checked throughout run
� Genome sample taken every 20 generations
� Genomic sample analyzed to determine what biot behavior is implemented in the network
![Page 19: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/19.jpg)
Experimental Results
� In early generations, biot behavior is random
� Most biots die off, unable to find food
� Population drops to around 400
� Remainder have the few necessary sensory/motor connections to eat and mate
![Page 20: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/20.jpg)
Results, Cont.
� At around 30 generations, biots evolve to home toward food, away from predators
� This tendency causes an elementary aggregation around food sources, safe locations
![Page 21: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/21.jpg)
Results, Cont.
� Elementary aggregation around food sources and safety is not herding
� Biots being pushed together by environmental factors, not yet seeking out members of their own species and moving toward them
![Page 22: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/22.jpg)
Results, Cont.
� Biots do begin herding
� At around 80-100 generations, biots evolve to home toward members of same species
� Neural connections that produce this behavior have evolved in biots’ neural network
![Page 23: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/23.jpg)
3 Pressures
� 3 Pressures cause biots to begin herding� Pressure to find a mate
� Herding confuses predators’ sense of gradient toward prey
� Finding food� herd as a distributed sensor network where
nodes implicitly communicate locations of food sources by moving towards them, or by sound of eating
![Page 24: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/24.jpg)
![Page 25: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/25.jpg)
Why is This Model Successful?
� Plenty of food to begin with (8000 plants)
� Massive death of initial biot population leaves plenty of food for the rest
� Biots do not die through aging
� The input neuron, “always on”, represents a kind of internal “drive”
� Biots don’t have to have perfect connections to survive
![Page 26: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/26.jpg)
Learned Solution to Herding
� Set up neural-net framework for learning:
� Inputs are “percepts”:
� Plant on left, Deer on left, Sound #4 on right, …
� Outputs are “actions”:
� forward, backward, turn left, eat, …
� Note that these inputs and outputs are pre-defined
� To learn: connections and weights between inputs and outputs
![Page 27: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/27.jpg)
Learned Solution to Herding (con’t.)
� Approach:� Two types of populations: predator and prey
� Use Genetic Algorithms to evolve populations of agents
� Members of population are controlled by randomly generated connections between inputs and outputs
� “Fitness” of population members determined by energy levels
� Different costs and gains for predator and prey
� Results:� Prey avoid predators; predators approach prey
� Clumping with like-animals
� Move toward like-animals – herding� Happens due to 1) need to find mate, 2) pressure from
predators, 3) pressure to find food
![Page 28: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/28.jpg)
Engineered Solution to Herding
� Exercise:
� Assume
� 2 sets of robots – sheep and dogs
� The sheep just want to wander around
� The dogs want to get the sheep into a corral
� How would you write (non-learning) code to generate the sheep herding behavior?
![Page 29: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/29.jpg)
One Suggested Approach to Engineered Herding Behavior…
� Similar to ideas we discussed earlier in flocking/schooling
� Recall Mataric’s paper on basis behaviors:
� Proposed method for generating herding:
![Page 30: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/30.jpg)
Heterogeneous Mobile Sensor Net Deployment Using Robot Herding and Line-of-Sight Formations
by Parker, et al., IROS’03
Presented by Kenny Iraheta
![Page 31: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/31.jpg)
Introduction
� Address issues of robot team heterogeneity in context of mobile sensor net deployment
� Heterogeneous team:
� Some robots are highly capable
� Some are less capable
� A typical dispersion algorithm does not work
![Page 32: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/32.jpg)
Herding in Heterogeneous Team
� Mobile sensor node
� Leader helper
� Follower helper
� Herding process
![Page 33: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/33.jpg)
Approach Overview
� A plan for how to deploy entire set of sensor nodes is generated
� Leader helper follows deployment path
� All robots within a herd move into their formation
� Maintain visibility
� The herd moves together
� Sensor nodes deployed via teleoperation through follower helper
![Page 34: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/34.jpg)
Robot Detection
� Follower helper needs to:
� Identify sensor nodes
� Identify leader helper
� Color cylindrical marker
� Extract ID, orientation, angle, distance
![Page 35: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/35.jpg)
Final-Version of Color Marker
![Page 36: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/36.jpg)
Line-Of-Sight Formation Keeping
� Generate a formation that satisfies the line-of-sight constraints
� Φx(n-1) ≤ θ
![Page 37: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/37.jpg)
Pictures & Videos
� http://dilab.eecs.utk.edu/DILpicmovie.html
![Page 38: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/38.jpg)
Shepherding Deployment Simulation
![Page 39: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/39.jpg)
Deployment
![Page 40: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/40.jpg)
![Page 41: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/41.jpg)
Important Points Regarding Herding
� Complex cooperative behaviors can either be engineered or learned
� One approach to learning herding involves two populations simultaneously, with different goals
� Herding involves complex interaction between two types of agents with different goals
� Herding requires animals of similar types work together to be successful
![Page 42: Herding - CPP · 2016. 2. 9. · Herding sheep Often predator/prey scenarios involve: Use of herding to surround targets, then declaring victory (prey are eaten) Engineered vs. Learned](https://reader035.vdocuments.net/reader035/viewer/2022071609/6147dec5a830d0442101b723/html5/thumbnails/42.jpg)
Student Presentation
� Multi-Robot Caravanning
� Presented by Fariba Derakhshan