l1 intro 9 rpdspace.mit.edu/bitstream/handle/1721.1/36896/16...•• the promise of authe promise...

Brian C. Williams

16.410/16.413

September 3rd, 2003

Introduction to Principles of Autonomy and Decision Making

1

Brian C. Williams

16.410/16.413

September 3rd, 2003

Today’s AssignmentToday’s Assignment

•• RRead Chapters 1 and 2 of AIMAead Chapters 1 and 2 of AIMA–– ““ArtificiaArtificial Intelligencl Intelligencee: A Modern Approach”: A Modern Approach”

by Stuart Russell and Peter Norvigby Stuart Russell and Peter Norvig

– 2nd Edition (not 1st Edition!!)– 2nd Edition (not 1st Edition!!)

–– AIMA is available at the CoopAIMA is available at the Coop

1

OutlineOutline

•• The promise of auThe promise of autonomous explorerstonomous explorers•• The challenge of autonomous explorersThe challenge of autonomous explorers•• Agents great and sAgents great and smmallall•• Course objectiCourse objectivve 1 (e 1 (116.410/13):6.410/13):

–– PrinciplePrinciples for Building Agentss for Building Agents

•• CCourse objectiourse objectivve 2 (e 2 (116.413):6.413):– Building auilding ann Agent:Agent:– B

The Mars exploration rover (MER) project.The Mars exploration rover (MER) project.

2

.

Courtesy NASA/JPL-Caltech. http://www.jpl.nasa.gov

A

Image taken from NASA's website: http://www.nasa.gov Courtesy of Kanna Rajan.

Frontier . . .

spacecraft.’’ establish a virtual presence, in space, on planets, in aircraft and ``Our vision in NASA is to open the Space We must

- Daniel S. Goldin, NASA Administrator, May 29, 1996

3

cc_ababba

Text Box

Courtesy of Kanna Rajan.

Motive: Follow the water ….

• to find evidence of past life on Mars.

• to study the forces that shaped Mars.

• to develop future life on Mars.

Courtesy NASA/JPL-Caltech. http://www.jpl.nasa.gov Courtesy of Kanna Rajan.

MarsMars

Motive: Follow the water ….

• to find evidence of past life on Mars.

• to study the forces that shaped Mars.

• to develop future life on Mars.

Courtesy U.S. Geological Survey.

4

Inner and Outer Planets Missions

MESSENGER mission to Mercury

MESSENGERmission to Mercury

Venus Sample Return

VenusSample Return

Comet Nucleus Sample Return

Primitive Bodies MissionsPrimitive Bodies Missions

Pluto/Kuiper Express

Europa Orbiter

Europa Lander

Neptune Orbiter

Titan Explorer

Cryobot & Hydrobot

Motive: life underMotive: life under EuropaEuropa??


5

MIT SPHERES Image taken from NASA's website. http://www.nasa.gov. Adapted. Courtesy of Kanna Rajan.

6

Cooperative Exploration

Distributed Planning Group, JPL Model-based Embedded

and Robotic Systems Group, MIT


OutlineOutline

•• The promise of autonomous explorersThe promise of autonomous explorers•• The challenge of autonomous explorersThe challenge of autonomous explorers •• Agents great and smallAgents great and small •• Course objective 1 (16.410/13):Course objective 1 (16.410/13):

–– Principles for Building AgentsPrinciples for Building Agents

•• Course objective 2 (16.413):Course objective 2 (16.413): – Building n Agent:– Building an Agent:


7

A Capable Robotic Explorer: Cassini • 7 year cruise Faster, Better, Cheaper

• ~ 150 - 300 ground

operators

•~ 1 billion $ •150 million $

• 7 years to •2 year build build • 0 ground

ops

Four launches in 7 months

Mars Climate Orbiter: 12/11/98 /99

Stardust: 2/7/99 QuickSCAT: 6/19/98

Mars Polar Lander: 1/3

courtesy of JPL

8

Mars Polar Lander

L

Spacecraft require a good physical commonsense…

Launch: 1/3/99

Traditional spacecraft commanding

2; 6;

,

Whats a better paradigm?

GS,SITURN,490UA,BOTH,96-355/03:42:00.000; CMD,7GYON, 490UA412A4A,BOTH, 96-355/03:47:00:000, ON; CMD,7MODE, 490UA412A4B,BOTH, 96-355/03:47:02:000, INT; CMD,6SVPM, 490UA412A6A,BOTH, 96-355/03:48:30:000, CMD,7ALRT, 490UA412A4C,BOTH, 96-355/03:50:32:000, CMD,7SAFE, 490UA412A4D,BOTH, 96-355/03:52:00:000, UNSTOW; CMD,6ASSAN, 490UA412A6B,BOTH, 96-355/03:56:08:000, GV,153,IMM,231,

GV,153; CMD,7VECT, 490UA412A4E,BOTH, 96-355/03:56:10.000, 0,191.5,6.5,

0.0,0.0,0.0, 96-350/ 00:00:00.000,MVR;

SEB,SCTEST, 490UA412A23A,BOTH, 96-355/03:56:12.000 SYS1,NPERR; CMD,7TURN, 490UA412A4F,BOTH, 96-355/03:56:14.000, 1,MVR; MISC,NOTE, 490UA412A99A,, 96-355/04:00:00.000, ,START OF TURN;,

CMD,7STAR, 490UA412A406A4A,BOTH 96-355/04:00:02.000, 7,1701, 278.813999,38.74;

CMD,7STAR, 490UA412A406A4B,BOTH, 96-355/04:00:04.000, 8,350,120.455999, -39.8612;

CMD,7STAR, 490UA412A406A4C,BOTH, 96-355/04:00:06.000, 9,875,114.162, 5.341;

CMD,7STAR, 490UA412A406A4D,BOTH, 96-355/04:00:08.000, 10,159,27.239, 89.028999;

CMD,7STAR, 490UA412A406A4E,BOTH, 96-355/04:00:10.000, 11,0,0.0,0.0; CMD,7STAR, 490UA412A406A4F,BOTH, 96-355/04:00:12.000, 21,0,0.0,0.0;

9

OutlineOutline

•• TThe promise of auhe promise of autonomous explorerstonomous explorers•• TThe challenge of autonomous explorershe challenge of autonomous explorers•• AAgents great and sgents great and smmallall•• CCourse objectiourse objectivve 1 (e 1 (116.410/13):6.410/13):


•• CCourse objectiourse objectivve 2 (e 2 (116.413):6.413):–– BBuilding auilding ann Agent:Agent:


Agents and IntelligenceAgents and Intelligence

Adaoted from J. Malik, U.C. Berkeley

10

Reflex agentsReflex agents


Goal-oriented agentGoal-oriented agent


11

Utility-based agentUtility-based agent


OutlineOutline





12

Course Objective 1:Course Objective 1: Principles of AgentsPrinciples of Agents

16.410/13: To learn the modeling and16.410/13: To learn the modeling and algorithmic building blocks for creatingalgorithmic building blocks for creating reasoning, learning agents:reasoning, learning agents:

•• TTo formulate reasoning problems.o formulate reasoning problems.•• TTo describe, analyze and demonstrateo describe, analyze and demonstrate

reasoning algorithms.reasoning algorithms.•• TTo model and encodeo model and encode knowledge used byknowledge used by

reasoning algorithms.reasoning algorithms.

Agent ParadigmsAgent Paradigms

• Extensive Reasoning

• Extensive Learning

• Extensive Optimization

• Extensive Reasoning

• Extensive Learning

• Extensive Optimization

13

Extensive Reasoning: Houston, we have a problem ...

courtesy of NASA

• Quintuple fault occurs (three shorts, tank-line and pressure jacket burst, panel flies off).

• Mattingly works in ground simulator to identify new sequence handling severe power limitations.

• Mattingly identifies novel reconfiguration, exploiting LEM batteries for power.

• Swaggert & Lovell work on Apollo 13 emergency rig lithium hydroxide unit.

Image taken from NASA's website: http://www.nasa.gov. Courtesy of Kanna Rajan.

Example of a Model-based Agent:

• Goal-directed

• First time correct

• projective • reactive

• Commonsense models

• Heavily deductive

Scripts

component models

GoalsGoals

DiagnosisDiagnosis& Repair& Repair

MissionMissionManagerManager ExecutiveExecutive

Planner/Planner/SchedulerScheduler

Remote AgentRemote Agent

Mission-level actions & resources

14

Reasoning methods thatReasoning methods that support the creation of agentssupport the creation of agents

•• Rule-bRule-based systemsased systems–– ForwaForward chainingrd chaining–– GoalGoal-directeddirected-

•• Search and roadmSearch and roadmap path planningap path planning–– Blind SearchBlind Search –– InformInformed Searched Search–– AdversaAdversarial (Game) Searchrial (Game) Search

•• Planning and Acting in the WPlanning and Acting in the Worldorld•• Constraints and SchedulingConstraints and Scheduling•• Model-based DiagnosisModel-based Diagnosis•• Logic and DeductionLogic and Deduction

Extensive Learning: TD-Gammon [Tesauro, 1995]

Extensive Learning: TD-Gammon [Tesauro, 1995]

Learns to play Backgammon

Situations: • Board configurations (1020)

Actions: • Moves

Rewards: – +100 if win – - 100 if lose – 0 for all other states

• Trained by playing 1.5 million games against self.

Î Currently, roughly equal to best human player.

Learns to play Backgammon

Situations:• Board configurations (1020)

Actions:• Moves

Rewards:– +100 if win– - 100 if lose– 0 for all other states

• Trained by playing 1.5 million games against self.

Î Currently, roughly equal to best human player.

15

Learning methods that supportLearning methods that support the creation of agentsthe creation of agents

•• LLearning through reinforcementearning through reinforcement

•• LLearearnining decision treesng decision trees

•• NNeural net learningeural net learning

Extensive OptimizationExtensive Optimization

16

Cooperative Path Planning: MILP Encoding: Fuel Equation

Cooperative Path Planning:MILP Encoding: Fuel Equation

min = JT = min 6 q’wi + 6 r’vi + p’wNmin = JT = min 6 q’wi + 6 r’vi + p’wNwi, vi wi, vi i=1

N-1

i=1

N-1

slack control vector weighting vectors

slack state vector

past-horizon terminal cost term

total fuel calculated over all time instants i

Cooperative Path Planning:Cooperative Path Planning:MILP Encoding: ConstraintsMILP Encoding: Constraints

• s• sijij <= w<= wij, etc.ij, etc. State Space ConstraintsState Space Constraints

•• ssii+1 = As+1 = Asii + Bu+ Buii State Evolution EquationState Evolution Equation

-x-x• x• xii <= xmi<= xmi n + Mti1+ Mti1n

ii <= -x<= -x + Mti2+ Mti2mama xx

-y-yyyii <= ymi<= ymi n + Mti3+ Mti3n Obstacle AvoidanceObstacle Avoidance

ii <= -yma<= -yma xx + Mti4+ Mti4 (for all time i)(for all time i)

6 tik6 tik <= 3<= 3 ((t introduce IP element)t introduce IP element)

•• SimSimilar equation for Collision Avoidance (for all pairs ofilar equation for Collision Avoidance (for all pairs of vehicles)vehicles)

17

OutlineOutline





Optimization methods thatOptimization methods that support the creation of agentssupport the creation of agents

Modeling Frameworks:Modeling Frameworks:– Markov Decision Processes– Markov Decision Processes

Mixed Integer Linear Programminging–– Mixed Integer Linear Programm

Solution Methods:Solution Methods:– Dyna– Dynamic Programmingmic Programming– Simplex– Simplex – Branch and Bound– Branch and Bound

18

Course Objective 2:Course Objective 2: Building AgentsBuilding Agents

16.413: To appreciate the challenges of building a16.413: To appreciate the challenges of building a state of the art autonomous explorer:state of the art autonomous explorer:

•• To model and encode knowledge needed to solveodel and encode knowledge needed to solveTo m a state of the art challenge.a state of the art challenge.

•• To work through theTo work through the process of autonomy systemssprocess of autonomy system integration.integration.

•• To assess the promise, frustrations and challengesise, frustrations and challengesTo assess the prom of using (b)leading art technologies.of using (b)leading art technologies.

Mars Exploration Rovers – Jan. 2004Mars Exploration Rovers – Jan. 2004

Mission Objectives:

• Learn about ancient water and climate on Mars.

• For each rover, analyze a total of 6-12 targets – Targets = natural rocks, abraded rocks, and soil

• Drive 200-1000 meters per rover

• Take 1-3 panoramas both with Pancam and mini-TES

• Take 5-15 daytime and 1-3 nightime sky observations with mini-TES

Mission Objectives:

• Learn about ancient water and climate on Mars.

• For each rover, analyze a total of 6-12 targets– Targets = natural rocks, abraded rocks, and soil

• Drive 200-1000 meters per rover

• Take 1-3 panoramas both with Pancam and mini-TES

• Take 5-15 daytime and 1-3 nightime sky observations with mini-TES

Mini-TES Pancam

Navcam

Rock Abrasion Tool Microscopic Imager

Mossbauer spectrometer APXS

Courtesy NASA/JPL-Caltech. http://www.jpl.nasa.gov. Courtesy of Kanna Rajan.

19

Surface Operations Scenario

Day 4 Day 1

Day 2

Board Navigation Changes, as needed

Day 3

Mars Exploration Rover

Target

During the Day Science Activities

Long-Distance Traverse (<20-50 meters)

Initial Position; Followed by “Close Approach”

During the Day Autonomous On-

Day 2 Traverse Estimated Error Circle

Science Prep (if Required)

Day 2 Traverse Estimated Error Circle

Courtesy of Kanna Rajan, NASA Ames. Used with permission.

Activity Name Durati

on 10 11 12 13 14 15 16 17 18 19 20 21 22 23 0 1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 0 1 2 3 4 5 6 7 8 9

DTE 4.50 0.75

DTE period DFE

Night Time Rover Operations 16.97 Night Time Rover OperationsSleep Wakeup

Pre-Comm Session Sequence Plan Review

Current Sol Sequence Plan Review 1.50 1.50

Current Sol Sequence Plan Review

Prior Sol Sequence Plan Review 2.00 Prior Sol Sequence Plan Review

Real-TIme Monitoring 4.50 0.75

Real-TIme Monitoring Real-TIme Monitoring

2.75 Downlink Product Generation

Tactical Science Assessment/Observation Planning

5.00 Tactical Science Assessment/Observation Planning

Science DL Assessment Meeting 1.00 Science DL Assessment Meeting

Payload DL/UL Handoffs 0.50 Payload DL/UL Handoffs

Tactical End-of-Sol Engr. Assessment & Planning

5.50 Tactical End-of-Sol Engr. Assessment & Planning

DL/UL Handover Meeting 0.50 DL/UL Handover Meeting

Skeleton Activity Plan Update 2.50 Skeleton Activity Plan Update

SOWG Meeting 2.00 SOWG Meeting

Uplink Kickoff Meeting 0.25 Uplink Kickoff Meeting

Activity Plan Integration & Validation 1.75 Activity Plan Integration & Validation

Activity Plan Approval Meeting 0.50 Activity Plan Approval Meeting

Build & Validate Sequences 2.25 Build & Validate Sequences

UL1/UL2 Handover 1.00 UL1/UL2 Handover

Complete/Rework Sequences 2.50 Complete/Rework Sequences

Margin 1 0.75 Margin 1

Command & Radiation Approval 0.50 Command & Radiation Ap

Margin 2 1.25 Margin 2

Radiation 0.50 Radiation

MCT Team 7.00 4.00

One day in the life of a Mars rover

Science Planning Sequence Build/Validation Uplink

Downlink Product Generation...

Courtesy: Jim Erickson

Downlink Assessment


20

EUROPAAutomated

Planning System

EUROPA Automated

Planning System

Science

Navigation

Engineering

Resource Constraints

DSN/Telcom

Flight Rules

Science Team

Sequence Build

MAPGEN: Automated Science Planning for MER

Planning Lead: Kanna Rajan (ARC)


Course ChallengeCourse Challenge

• What would it be like to operate MER if it was fully autonomous?

Course project:

• Demonstrate an autonomous MER mission in simulation, and in the MIT rover testbed.

• What would it be like to operate MER if it was fully autonomous?

Course project:

• Demonstrate an autonomous MER mission in simulation, and in the MIT rover testbed.

21

Next Challenge: Mars Smart Lander (2009)

Next Challenge: Mars Smart Lander (2009)

Mission Duration: 1000 days Total Traverse: 3000-69000 meters Meters/Day: 230-450 Science Mission: 7 instruments, sub-surface science package (drill, radar), in-situ sample “lab”

Technology Demonstration: (2005).

Courtesy NASA/JPL-Caltech. http://www.jpl.nasa.gov. Courtesy of Kanna Rajan.

22

l1 intro 9 rpdspace.mit.edu/bitstream/handle/1721.1/36896/16...•• the promise of authe promise...

Documents