Robotic Space Explorers

Brian C. Williams

Space Systems Lab &

Artificial Intelligence Lab, MIT

Marskokhod at NASA Ames research center

Smart Buildings at CMU & Xerox PARC

Ecological Life SupportFor Mars Exploration

Courtesy of Yuri Gawdiak, NASA Ames

Robotic Webs

Unmanned Air Vehicles

Intelligence Embedded at all Levels

• Behavior-based robotics: Subsumption

• Reinforcement learning and MDPS

• Classical planning and execution

• Model-based diagnosis and execution

• Mission-level planning

• Robotic path planning

• Probabilistic monitoring and

• Decision-theoretic planning

• Multi-agent coordination

Increased Reasoning

To Boldly Go Where No AI System Has Gone Before

A Story of Survival

courtesy JPL

Started: January 1996Launch: Fall 1998

Remote Agent Team Members

• Douglas Bernard JPL

• Steve Chien JPL

• Greg Dorais Ames

• Julia Dunphy JPL

• Dan Dvorak JPL

• Chuck Fry Ames

• Ed Gamble JPL

• Erann Gat JPL

• Othar Hansson Thinkbank

• Jordan Hayes Thinkbank

• Bob Kanefsky Ames

• Ron Keesing Ames

• James Kurien Ames

• Bill Millar Ames

• Sunil Mohan Formida

• Paul Morris Ames

• Nicola Muscettola Ames

• Pandurang Nayak Ames

• Barney Pell Ames

• Chris Plaunt Apple

• Gregg Rabideau JPL

• Kanna Rajan Ames

• Nicolas Rouquette JPL

• Scott Sawyer LMMS

• Rob Sherwood JPL

• Reid Simmons CMU

• Ben Smith JPL

• Will Taylor Ames

• Hans Thomas Ames

• Michael Wagner 4th Planet

• Greg Whelan CMU

• Brian C. Williams Ames

• David Yan Stanford

I am a HAL 9000 computer production number three. I became operational at the H.A.L. plant in Urbana, Illinois on January 12, 1997.

International Space Station 1998-2002

courtesy NASA

``Our vision in NASA is to open the Space Frontier. When people think of space, they think of rocket plumes and the space shuttle. But the future of space is in information technology. We must establish a virtual presence, in space, on planets, in aircraft and spacecraft.’’

- Daniel S. Goldin, NASA AdministratorSacramento, California, May 29, 1996

Motive: Astrobiology & Origins Programs

Means: New Millennium Program

Smarts: Autonomous Reasoning

1997: Mars Pathfinder and Sojourner

Motive: Primitive life on Early Mars?

courtesy JPL

Mars Pathfinder, 1997 courtesy JPL

New Means

courtesy NASA Ames courtesy NASA Lewis

New Means: Mars Airplane

Cryobot & Hydrobot courtesy JPL

Motive: life under Europa?Motive: life under Europa?

Formation Flying Optical Interferometer (ST3) courtesy JPL

Motive: Earth-like Planets Around Motive: Earth-like Planets Around Other Stars?Other Stars?

Four launches in 7 months

Mars Climate Orbiter: 12/11/98Mars Polar Lander: 1/3/99

Stardust: 2/7/99 QuickSCAT: 6/19/98

courtesy of JPL

Traditional spacecraft commanding

GS,SITURN,490UA,BOTH,96-355/03:42:00.000; CMD,7GYON, 490UA412A4A,BOTH, 96-355/03:47:00:000, ON; CMD,7MODE, 490UA412A4B,BOTH, 96-355/03:47:02:000, INT; CMD,6SVPM, 490UA412A6A,BOTH, 96-355/03:48:30:000, 2; CMD,7ALRT, 490UA412A4C,BOTH, 96-355/03:50:32:000, 6; CMD,7SAFE, 490UA412A4D,BOTH, 96-355/03:52:00:000, UNSTOW; CMD,6ASSAN,490UA412A6B,BOTH, 96-355/03:56:08:000, GV,153,IMM,231,

GV,153; CMD,7VECT, 490UA412A4E,BOTH, 96-355/03:56:10.000, 0,191.5,6.5,

0.0,0.0,0.0,96-350/00:00:00.000,MVR;

SEB,SCTEST,490UA412A23A,BOTH, 96-355/03:56:12.000, SYS1,NPERR; CMD,7TURN, 490UA412A4F,BOTH, 96-355/03:56:14.000, 1,MVR; MISC,NOTE, 490UA412A99A,, 96-355/04:00:00.000, ,START OF TURN;, CMD,7STAR, 490UA412A406A4A,BOTH 96-355/04:00:02.000, 7,1701,

278.813999,38.74; CMD,7STAR, 490UA412A406A4B,BOTH,96-355/04:00:04.000, 8,350,120.455999,

-39.8612; CMD,7STAR, 490UA412A406A4C,BOTH,96-355/04:00:06.000, 9,875,114.162,

5.341; CMD,7STAR, 490UA412A406A4D,BOTH,96-355/04:00:08.000, 10,159,27.239,

89.028999; CMD,7STAR, 490UA412A406A4E,BOTH,96-355/04:00:10.000, 11,0,0.0,0.0; CMD,7STAR, 490UA412A406A4F,BOTH,96-355/04:00:12.000, 21,0,0.0,0.0;

How Will They Survive?

Vanished:• Mars Observer• Mars Polar Lander

Miscommanded• Clementine• Mars Climate Orbiter

courtesy of JPL

STS-93 Hydrogen Leak

Symptoms:• Engine temp sensor high• LOX level low• GN&C detects low thrust• H2 level low (???)

Problem: Liquid hydrogen leakEffect:

• LH2 used to cool engine• Engine runs hot• Consumes more LOX

Cassini Maps Titan courtesy JPL

• 7 year cruise

• ~ 150 - 300 ground operators

•~ 1 billion $

• 7 years to build

Intelligent Embedded Systems: Cassini

•150 million $

•2 year build

• 0 ground ops

Faster, Better, Cheaper

QuickTime™ and aPhoto - JPEG decompressor

are needed to see this picture

courtesy JPL

Ames-JPL NewMaap: New Millennium Advanced Autonomy Prototype

July - November, 1995July - November, 1995• no Earth Comm

• ~ 1 hr insertion window

• engines idle for several years

• moves through ring plane

Reconfiguring for a Failed Engine

Fire backupengine

Valve failsstuck closed

Open fourvalves

Fuel tankFuel tankOxidizer tankOxidizer tank

AI in the pre-90’s: Reservations about Embedded Systems being Intelligent

• “[For reactive systems] proving theorems is out of the question” [Agre & Chapman 87]

• ``Diagnostic reasoning from a tractable model is largely well understood. [However] we don’t know how to model complex behavior...’’ [Davis & Hamscher 88]

• “[Commonsense] equations are far too general for practical use.” [Sacks & Doyle 91]

Houston, We have a problem ...

courtesy of NASA

• Quintuple fault occurs (three shorts, tank-line and pressure jacket burst, panel flies off).

• Mattingly works in ground simulator to identify new sequence handling severe power limitations.

• Mattingly identifies novel reconfiguration, exploiting LEM batteries for power.

• Swaggert & Lovell work on Apollo 13 emergency rig lithium hydroxide unit.

Challenge: Thinking Through Interactions

Programmers must reason through system-wide interactions to generate codes for:

• command command confirmationconfirmation

• goal trackinggoal tracking• detecting anomaliesdetecting anomalies• isolating faultsisolating faults• diagnosing causesdiagnosing causes

• hardware reconfighardware reconfig• fault recoveryfault recovery• safingsafing• fault avoidancefault avoidance• control control

coordinationcoordination

Equally problematic at mission operations levelEqually problematic at mission operations level

Programmers generate breadth of functions from commonsense models in light of mission goals.

• Model-based Programming• Program by specifying commonsense, compositional

declarative models.

• Model-directed Planning & Execution• Provide services that reason through each type of

system interaction from models.

• on the fly reasoning requires significant search & deduction within the reactive control loop.

Model-based Autonomy

Transition Systems + Constraints + Probabilities

ClosedClosed

ValveValve

OpenOpen StuckStuckopenopen

StuckStuckclosedclosed

OpenOpen CloseClose

0. 010. 01

0. 010. 01

0.010.01

0.010.01

inflow = outflow = 0

Towards a Unified Model

VDECU Op_State

On

Engine Op State

Burn_Ignition

BurnBurn_Termination

Shut_down

Off

Early_Prep Wait

Late_Prep

Mission Operations ModelHardware Commanding & Failure Model

Fast Search: Deep Blue beats Kasparov by brute force.

Phase Transition for 3-SAT, N = 12 to 100 Data Rescaled Using c = 4.17, = 1.5 (Kirkpatrick and Selman, Science, May 1994)

100

244050

12

UNSAT Phase

SAT Phase

20

Fra

cti

on

of

Form

ula

e U

nsati

sfi

ed

0.0-10 0

0.2

0.6

0.4

0.8

1.0

03

0.2

1.0

0.4

0.6

0.8

M/N

10 20

4 5 6 7

Many problems aren’t so hard

0 100 200 3000

50

100

150

200

FOUNDUNSOLVABLE

SOLUTIONFOUND

Average constraints per variable

Phase Transition

4-sat cost

25 var.

Many problems aren’t so hard

generatesuccessor

generatesuccessor

AgendaAgenda

TMSTMS

conflictdatabase

conflictdatabase

RISC-like, deductive kernel

General Deduction Can Achieve Reactive Time Scales

• Quintuple fault occurs (three shorts, tank-line and pressure jacket burst, panel flies off).

• Mattingly works in ground simulator to identify new sequence handling severe power limitations.

• Mattingly identifies novel reconfiguration, exploiting LEM batteries for power.

• Swaggert & Lovell work on Apollo 13 emergency rig lithium hydroxide unit.

• Multiple fault diagnosis of unexperienced failures.

• Mission planning and scheduling

• Hardware reconfiguration

• Scripted execution

Services For Thinking Through Interactions

Real-Time Real-Time ExecutionExecution

Real-Time Real-Time ExecutionExecution

Flight Flight H/WH/W

Flight Flight H/WH/WFault Fault

MonitorsMonitorsFault Fault

MonitorsMonitorsPlanning Experts Planning Experts (incl. Navigation)(incl. Navigation)Planning Experts Planning Experts (incl. Navigation)(incl. Navigation)

Remote Agent Architecture

Ground Ground SystemSystemGround Ground SystemSystem

RAX ManagerRAX Manager

Diagnosis Diagnosis & &

ReconfigReconfig

Mission Mission ManagerManager

Scripted Scripted ExecutiveExecutive

Planner/Planner/SchedulerScheduler

Remote AgentRemote AgentRAX_STARTRAX_START

QuickTime™ and aPhoto - JPEG decompressor

are needed to see this picture

courtesy JPL

Ames-JPL NewMaap: New Millennium Advanced Autonomy Prototype

July - November, 1995

Thrust Goals

Attitude Point(a)

Engine Off

Off

Delta_V(direction=b, magnitude=200)

Power

Mission Manager Sets Goals

Thrust Goals

Attitude Turn(a,b)Point(a) Point(b) Turn(b,a)

Engine Thrust (b, 200) Off

Off

Delta_V(direction=b, magnitude=200)

Power

Warm Up

Plan!Plan!

Planner Models

• Objects

– state-variables

– tokens

• Constraints

– compatibilities

– functional dependencies

VDECU Op_State

On

Engine Op State

Burn_Ignition

BurnBurn_Termination

Shut_down

Off

Early_Prep Wait

Late_Prep

CompatibilityCompatibility

Thrust Goals

Attitude

Engine Thrust (b, 200)

Delta_V(direction=b, magnitude=200)

Power

contains

Thrust Goals

Attitude Point(b)

Engine Thrust (b, 200) Off

Delta_V(direction=b, magnitude=200)

Power

Warm Up

meets met_by

contained_by

contained_by

equals

CompatibilityCompatibility

Planning/Scheduling Cycle

Plan has flaws

Instantiate compatibility

Schedule token

Plan is consistent

Backtrack

YES

Uninstantiated compatibility

.

.

.

.

.

.

NO

NO

PLAN

Heuristics

Types of Plan Flaws

• Un-instantiated compatibilities– subgoaling

• Un-inserted tokens• Under-constrained parameter• Gaps in scheduling horizon

Plan Generates A Simple Temporal Constraint Network

Executing Temporal Plans

[130,170]]

<0, 0>

[0, 300]

• Propagate time• Select enabled events• Terminate preceding tokens• Run next tokens

EXECUTIVE

CONTROLLED SYSTEM

Time Propagation Can Be Costly

EXECUTIVE

CONTROLLED SYSTEM

Compile to Efficient Network

Programmers and operators must reason through system-wide interactions to generate codes for:

Identifying Modes Reconfiguring Modes

• monitoringmonitoring• tracking goalstracking goals• confirming confirming

commandscommands• detecting anomaliesdetecting anomalies• diagnosing faults diagnosing faults

• reconfiguring reconfiguring hardwarehardware

• coordinating control coordinating control policies policies

• recovering from faultsrecovering from faults• avoiding failuresavoiding failures

Model-based Execution of Tokens

Model-based Execution asStochastic Optimal Control

Controller

Plant

modeidentification

mode reconfiguration

s’(t)

(t)

ffs (t)

gg

o(t)

Model

Livingstone

Goals

Closed

Open Stuckopen

Stuckclosed

OpenCloseCost 5

Prob .9

Models• modes engage physical processes• probabilistic automata for dynamics

Vlv = closed => Outflow = 0;

vlv=open => Outflow = Mz

+(inflow);

vlv=stuck open => Outflow = Mz

+(inflow);

vlv=stuck closed=> Outflow = 0;

Model-based programming language

(defcomponent valve (?name)

:attributes ((sign-values (flow (input ?name)))

(sign-values (flow (output ?name)))

...)

...

(closed

:model (and (= (flow (input ?name)) zero)

(= (flow (output ?name)) zero))

:transitions ((open-valve :when (open (cmd-in ?name)) :next open)

(:otherwise :persist)))

...)

Mode Identification and Diagnosis

Observe“no thrust”

Find most likely reachable states consistent with observations.

Goal: Achieve Thrust

Mode Reconfiguration

Models Compile to Propositional Logic

• Specifying Variables mode ranges over {open, closed, stuck-open, stuck-closed}

cmd ranges over {open, close, no-cmd}

fin, and fout range over {positive, negative, zero}

pin, and pout ranging over {high, low, nominal}

• Specifying Mode Behaviors mode = open (pin = pout) (fin = fout)

mode = closed (fin = zero) (fout = zero)

mode = stuck-open (pin = pout) (fin = fout)

mode = stuck-closed (fin = zero) (fout = zero)

• Specifying nominal transitionsmode = closed cmd = open next (mode = open)

mode = closed cmd ≠ open next (mode = closed)

mode = open cmd = close next (mode = closed)

mode = open cmd ≠ close next (mode = open)

mode = stuck-open next (mode = stuck-open)

mode = stuck-closed next (mode = stuck-closed)

• Specifying failure transitions 1 : mode = closed next (mode = stuck-closed)

2 : mode = closed next (mode = stuck-open)

3 : mode = open next (mode = stuck-open)

4 : mode = open next (mode = stuck-closed)

Mode identification and reconfiguration performed by OPSAT

generatebest

implicants

generatebest

implicants

Best-first AgendaBest-first Agenda Check ConstraintsCheck ConstraintsOptimalOptimalfeasiblefeasiblemodesmodes

ConflictsConflicts(infeasible(infeasible modes)modes)

CheckedCheckedmodesmodes

DPLL SATWith ITMS

DPLL SATWith ITMS

conflictdatabase

conflictdatabase

Combinatorial optimization w propositional constraints

A* + Conflict-directed Search + DPLL + TMS

MI and MR performanceNumber of components: 80Number of clauses: 11101

Failurescenario

MI time(Sparc 5 in sec)

MR time(Sparc 5 in sec)

MI time(est Sparc 5 sec)

MR time(est Sparc 5 sec)

EGApreaim

2.2 1.7 .07 .17

BPLVD 2.7 2.9 .18 .29IRU 1.5 1.6 .02 .03

EGA burn 2.2 3.6 .08 .03ACC 2.5 1.9 .04 .02

ME toohot

2.4 3.8 .18 .78

Acc low 5.5 6.1 .25 .05

no TMS LTMS

courtesy JPL

Started: January 1996Launch: Fall 1998

Remote Agent Experiment

May 17-18th experiment• Generate plan for course correction and thrust

• Diagnose camera as stuck on

– Power constraints violated, abort current plan and replan

• Perform optical navigation• Perform ion propulsion thrust

May 21th experiment.• Diagnose faulty device and

– Repair by issuing reset.

• Diagnose switch sensor failure.

– Determine harmless, and continue plan.

• Diagnose thruster stuck closed and

– Repair by switching to alternate method of thrusting.

• Back to back planning

See rax.arc.nasa.gov

Intelligent embedded systems research has addressed each of AI’s concerns

• “[For reactive systems] proving theorems is out of the question” [Agre & Chapman 87]

• ``Diagnostic reasoning from a tractable model is largely well understood. [However] we don’t know how to model complex behavior...’’ [Davis & Hamscher 88]

• “[Commonsense] equations are far too general for practical use.” [Sacks & Doyle 91]

“With autonomy we declare that no sphere is off limits. We will send our spacecraft to search beyond the horizon, accepting that we cannot directly control them, and relying on them to tell the tale.”

Bob Rasmussen ArchitectJPL MissionData System