Multi-Agent Exploration in Unknown Environments

Changchang WuNov 2, 2006

Outline

• Why multiple robots• Design issues• Basic approaches

– Distributed– Centralized– Market-based

Why Multiple Robots

• Some tasks require a robot team

• Have potential to finish tasks faster

• Increase robustness w/ redundancy

• Compensate sensor uncertainty by merging overlapping information

• Multiple robots allow for more varied and creative solutions

A Good Multi-Robot System Is:

• Robust: no single point of failure

• Optimized, even under dynamic conditions

• Quick to respond to changes

• Able to deal with imperfect communication

• Able to avoid robot interference

• Able to allocate limited resources

• Heterogeneous and able to make use of different robot skills

Basic Approaches

• Distributed – Every robot goes for itself

• Centralized– Globally coordinate all robots

• Market-based– Analogy To Real Economy

Distributed Methods

• Planning responsibility spread over team

• Each robot basically act independently• Robots use locally observable

information to coordinate and make their plans

Example: Frontier-Based Exploration Using Multiple Robots (Yamauchi 1998)

• A highly distributed approach

• Simple idea: To gain the most new information about the world, move to the boundary between open space and uncertainty territory

• Frontiers are the boundaries between open space and unexplored space

Occupancy Grid

• World is represented as grid• Each cell in the grid is assigned with a

probability of being already occupied/observed• The initial probability is all set to .5• Cell status can be Open (<0.5), Unknown

(=0.5) or Occupied (>0.5)• Bayesian rule is used to update cells by

merging information from each sensor reading (sonar)

Frontier Detection• Frontier = Boundary between open and unexplored

space.

• Any open cell adjacent to unknown cell is frontier edge cell.

• Frontier cells grouped into frontier regions based on adjacency.

• Accessible frontier = Robot can pass through opening.

• Inaccessible frontier = Robot cannot pass through opening.

Multi-Robot Navigation

• Simple algorithm: Each robot goes along the shortest obstacle free path to a frontier region

• Robots share a common map: All information obtained by any robot is available to all robots

• Robots are planning path independently • Use reactive strategy to avoid collisions• Robots may waste time for the same frontiers

An Exploration Sequence

Distributed Methods: Pros & Cons

• Pros– Very robust. No single point failure– Fast response to dynamic conditions– Little or no communication is required– Easy….Little computation required

• Cons– Plans only based on local information– Solutions are often sub-optimal

Centralized Methods

• Robot team treated as a single “system” with many degrees of freedom

• A single robot or computer is the “leader”

• Leader plans optimal tasks for groups

• Group members send information to leader and carry out actions

Example: Arena (Jia 2004)

• Robots share a common map and only communicate with a leader

• Robots compete for resources by their efficiency

• leader greedily assigns the most efficient tasks

• Leader coordinate robots to handle interference

Background

• World representation– Occupancy grid

• Cost unit– Moving forward one step = Turning 45

degrees

• Cost overflow– Similar to minimum cost spanning tree– Easy to compute the shortest path– Easy to handle obstacle

Cost Overflow

Cost of 45° turning ＝ Cost of one cell’s step

1 2

2 3

3

3

3

4

4

4

44

4

5

5 5 5 5

5

5

5

Direction

priority

6

6

6 6 6 6 6

7

7

7

77

8

8

8

8

88

8

9 9

9

9

9

9

9

10

10

10

10

10

10

11

11

11

11

11

11

11

12

12

12

12

12

12

12

12

Goal Candidates Detection

• A goal point P should satisfyi. P is passable (Mark the cells in warning range or

obstacles/Wall/Unknown cells as impassable)ii. Some unexplored cells lie in the circle with P as the

center and (R + K) as the radium, where R is the warning radius and K is usually 1

Robot cellpaths cellobservation cell

candidate goal

Goal Resource

• Reserved goal candidates– Robots obtained by competition

• Recessive goal candidates– The goal points in a given range

to a reserved goal point– This distance can be adjusted

Goal candidates

Recessive goals candidates

Path Resource

• Path resource is a time-space term

• For a given time, the cells close to any robot are marked off for safety

• Looks just like a widened path

• Basically a reactive strategy

goal

path

resource

Revenue and Utility

• Revenue – The expected gain of information that

robots observe at a goal point

• Utility used by many other approaches– Utility = revenue – cost

• Utility in this paper– Utility = Revenue / Cost– Better connected to purpose of smallest

cost– No need to care about unit conversion

Greedy Goal Selection

• Try to maximize the global utility

• Coordination: robots obtain goal and path resources exclusively

• Competition: repetitively select the pair of free agent and goal with highest utility

• Sub-optimal

Simple Algorithm

• Repeat until map is complete– Repeat #free robots times

1. Cost computation (Also make sure no interference with the busy robots)

2. Select the highest utility task (Compete)3. Mark off the associated robot and goal

points, and nearby goal points

1st Competition:

2

2 2

2

2

2

2

2

2

2 2 2 2

2

2

2

2

2

2

3 3 3

3

3

3

3

3

3

3 3 3 3 3 3

3

3

3

3 3 34 4 4

4

4

4

4 4 4 4 4 4 4 4

1

1

1 1

1

1

1

1

1

11

55

5 55 5 5 5 55

6 6 6 6 6

6

6

6

6

6

Interval = 3 Competitor:

1st Competition Result:

4 4

6 6

5 5

3

2

2

1

4

4

1

2 2

4 4

3 3

2 2

4

Interval = 3 Competitor:

2nd Competition

1

2

1

2 2

3 3 3

4 4

4

4 4 4

44

1

11

1

1 1

Interval = 3 Competitor:

Satisfied:

2 2 2

2

2

2

2

2 2 2

3 3 3 3

3

3

3

3

3

3

3 3 3 3444 4 4 4

4

4

4

4

4

4

4 4 4 4 4

5 5 5 5 5

5 5 5

5

5

5 5

6

6

6

6 6 66 6

6

6

6

6

2nd Competition Result

1

2

1

2 2

3 3 3

4 4

4

4 4 4

44

4

1

2

3

4

5

1

1

2

2

22

2

333

4 44

5 5 5

6 6 6

6

666

6

Interval = 3 Competitor:

Satisfied:

3rd Competition

1

2

1

2 2

3 3 3

4 4

4

4 4 4

44

4

1

2

3

4

5

1

2

2

22

2

333

4 44

5 5 5

6 6 6

6

666

6 6

Competitor:

Satisfied:

Interval = 3

1

1 1

2

2

222

3 3 3 3

3

3

3

4 4 4 4 4

4

4

5 5 5 5 5

6 6

6

7 7

7

7

7

7

7

7

7

8

8

8

8

8

8

8

9

9

9

9

9

9

9

10

10

10

10

10

10

10

10

11 11

11

11

11

11

11

11

11

11 12

12

12

12

12

12

12

12

13

13

13

13

13

13

13

3rd Competition Result

1

3 2

4

4

1

2

3

4

5

1

2

3

4

6 6 6

Competitor:

Satisfied:

Interval = 3

1

2

3

4 7

8

9 10 11 12

5 6

13

4

6

Planning Issues

• Do not transfer a reserved goal point to another free agent (unless necessary). Frequent change of tasks can cause localization error.

• Quit an assigned task when the goal point is unexpectedly observed by other robots

• Schedule at most one task for each agent

Possible Variations• Still keep busy agents in competition.

Remove the goal resources they win from competition.– This prevents those goal resources being

assigned to other agents– It is too early to burden a new task on a

robot who has not achieved it current task• No need to schedule them.

– New resources probably will be found when they reach the goals

Handling Failure of Planning

• It may fail to plan safe paths– When some robot get to a place where

• it is almost too close to other robot • it has no good space to detour

– And it choose to just wait there for other robots to move away, which is not known by other robots

• Avoidance of unexpected obstacle– Robots have simple reactive mechanism– Release resources and try to gain new task

Fail to plan safe paths

Competitor:

Satisfied:

Interval = 3

1

2

3 4 5 6 7 8

9

10

1

2

3410111213141516

collision

Reactive Mechanism

Competitor:

Satisfied:

Interval = 3

Exchange Tasks

Competitor:

Satisfied:

Interval = 3

1234

5

1

2

3

Some Statistics

Demo

Centralized Methods : Pros

• Leader can take all relevant information into account for planning

• Optimal s islution possible! • One can try different approximate

solutions to this problem

Centralized Methods: Cons

• Optimal solution is computationally hard– Intractable for more than a few robots

• Makes unrealistic assumptions:– All relevant info can be transmitted to leader

– This info doesn’t change during plan construction

• Vulnerable to malfunction of leader

• Heavy communication load for the leader

Market-Based Methods

• Based on market architecture• Each robot seeks to maximize individual

“profit”• Robots can negotiate and bid for tasks• Individual profit helps the common good• Decisions are made locally but effects

approach optimality– Preserves advantages of distributed

approach

Why Is This Good?

• Robust to changing conditions– Not hierarchical– If a robot breaks, tasks can be re-bid to others

• Distributed nature allows for quick response• Only local communication necessary• Efficient resource utilization and role adoption• Advantages of distributed system with

optimality approaching centralized system

Architecture• World is represented as a grid

– Squares are unknown (0), occupied (+), or empty (-)• Goals are squares in the grid for a robot to explore

– Goal points to visit are the main commodity exchanged in market

• For any goal square in the grid:– Cost based on distance traveled to reach goal– Revenue based on information gained by reaching goal

• R = (# of unknown cells near goal) x (weighting factor)

• Team profit = sum of individual profits– When individual robots maximize profit, the whole team

gains

Example World

Goal Selection Strategies

• Possible strategies:– Randomly select points, discard if

already visited– Greedy exploration:

•Choose goal point in closest unexplored region

– Space division by quadtree

Exploration Algorithm

Algorithm for each robot:1. Generate goals (based on goal selection

strategy)2. If OpExec (human operator) is reachable,

check with OpExec to make sure goals are new to colony

3. Rank goals greedily based on expected profit

4. Try to auction off /bid goals to each reachable robot

– If a bid is worth more than you would profit from reaching the goal yourself (plus a markup), sell it

Exploration Algorithm

5. Once all auctions are closed, explore highest-profit goal

6. Upon reaching goal, generate new goal points

– Maximum # of goal points is limited

7. Repeat this algorithm until map is complete

Bidding Example

• R1 auctions goal to R2

Expected vs. Real

• Robots make decisions based on expected profit– Expected cost and revenue based on

current map

• Actual profit may be different– Unforeseen obstacles may increase cost

• Once real costs exceed expected costs by some margin, abandon goal– Don’t get stuck trying for unreachable goals

Information Sharing

• If an auctioneer tries to auction a goal point already covered by a bidder:– Bidder tells auctioneer to update map– Removes goal point

• Robots can sell map information to each other– Price negotiated based on information gained– Reduces overlapping exploration

• When needed, OpExec sends a map request to all reachable robots– Robots respond by sending current maps– OpExec combines the maps by adding up cell

values

Advantages of Communication

• Low-bandwidth mechanisms for communicating aggregate information

• Unlike other systems, map info doesn’t need to be communicated repeatedly for coordination

What Is a Robot Doing

• Goal generation and exploration• Sharing Information with other

robots• Report information to OpExec at

some frequency

Experimental Setup

• 4 or 5 robots– Equipped with

fiber optic gyroscopes

– 16 ultrasonic sensors

Experimental Setup

• Three test environments– Large room cluttered with obstacles– Outdoor patio, with open areas as well as walls and tables– Large conference room with tables and 100 people

wandering around• Took between 5 and 10 minutes to map areas

Experimental Results

Experimental Results

Experimental Results

• Successfully mapped regions• Performance metric (exploration efficiency):

– Area covered / distance traveled [m2 / m]– Market architecture improved efficiency over no

communication by a factor of 3.4

Conclusion

• Market-based approach for multi-robot coordination is promising– Robustness and quickness of distributed system– Approaches optimality of centralized system– Low communication requirements

• Probably not perfect– Cost heuristics can be inaccurate– Much of this approach is still speculative

• Some pieces, such as leaders, may be too hard to do

In Sum

• Distributed vs. centralized mapping• Distributed vs. centralized planning• Revenue/Cost vs. Revenue – Cost

• Often sub-optimal solutions• No common evaluation system for

comparisons

References

• Yamauchi, B., "Frontier-Based Exploration Using Multiple Robots," In Proc. of the Second International Conference on Autonomous Agents (Agents98), Minneapolis, MN., 1998.

• Menglei Jia , Guangming Zhou ,Zonghai Chen, "Arena—an Architecture for Multi-Robot Exploration Combining Task Allocation and Path Planning,“ 2004

• Zlot, R., Stentz, A., Dias, M. B., and Thayer, S. “Multi-Robot Exploration Controlled By A Market Economy.” Proceedings of the IEEE International Conference on Robotics and Automation, 2002.

• http://voronoi.sbp.ri.cmu.edu/presentations/motionplanning2001Fall/FrontierExploration.ppt

• http://www.ai.mit.edu/courses/16.412J/lectures/advanced%20lecture_11.6.ppt

• http://mail.ustc.edu.cn/~jml/jml.files/Arena.ppt