synthesizing robust plans under incomplete domain models tuan nguyen and subbarao kambhampati...

Synthesizing Robust Plans under Incomplete Domain Models

Tuan Nguyen and Subbarao Kambhampati

Department of CSE, Arizona State University

Minh Do

Palo Alto Research Center

Acknowledgement: Funding from ONR grants N00014-09-1-0017, N00014-07-1-1049, NFS grant IIS-0905672, and by DARPA and the U.S. Army Research Laboratory under contract W911NF-11-C-0037.

Planning – The Traditional View

PLANNER

Problem instance

Domain model

a1

a3

a2

Deterministicactions

Stochasticnon-deterministic

actions

“Valid” plan

“Probabilistic” plan/policy

add “p”delete “q”

0.4 : add “p”0.6 : delete “q”

COMPLETE/FULL MODEL

Laborious and error-prone!!!

Model-lite Planning as Generalized Planning

Model-lite PlanningA Domain Incompleteness View

Missing some preconditions/effects of actions(e.g. Garland & Lesh, AAAI-05)

I/O typesTask dependency

(e.g. workflows management, web service composition)

Deterministicactions

Stochastic,non-deterministic

actions

There are known knowns; there are things we know that we know. There are known unknowns; that is to say, there are things that we now know we don’t know. But there are also unknown unknowns; there are things we do not know we don’t know.

Problem Formulation

Incomplete domain Proposition set Action

D=⟨F , A ⟩

F={p1 , p2 , ... , pn}a∈A

a

p

p '

q

r

q '

r '

-

-

Pre(a)

Pre (a)

Add (a)

Del (a)

Add (a )

Del (a )

wapre( p ' )

wadel (r ' )

wadel (q ' )

Different from stochastic effects!

Problem Formulation

Transition function

D

D1 D2 D2K

Completion set

⟨ ⟨ D ⟩ ⟩

γ(π , I , D)= UDi∈⟨⟨ D ⟩⟩

γ(π , I , Di)

I π?

I

D

πDi

πD j

Assumption: “Inapplicable” actioncauses state unchanged

Challenges

Language for domain incompleteness A robustness measure for plans Generating robust plans

Incompleteness Annotation: Modeling Issues

Incompleteness annotations can be at Schema level

Grounded level Or in between

Incompleteness Annotation: Modeling Issues

Restriction on variable values

p (x3)

p ' (C 1 , x2)

q( x1)

r (x2)

q ' (x1 ,C 3)

r ' (x1 ,C 2)

-

a ( x1 , x2 , x3)

Possible precondition:

p ' ( x1 , x 2):when ( x 1=C 1)

-

Possible add:

q ' ( x1 , x3) : when (x 3=C 3)

Possible delete:

r ' (x 1 , x 2) : when ( x 2=C 2)The domain writer knows thatis NOT a precondition ofwhen , and may be in other cases

p ' ( x1 , x 2)a ( x1 , x2 , x3)

x1≠C 1

A tourist planning to have food in a small town is not sure if she needs to have cash. Her action have_food(M: Meals, C: Town) has possible precondition need_cash(M: Meals, C:Town): when (C=the town)

Incompleteness Annotation: Modeling Issues

Restriction on variables: Possible preconditions/effects depending on

values of some variables, but such values are unknown!

p (x3)

p ' (x1 , x2)

q( x1)

r (x2)

q ' (x1 , x3)

r ' (x1 , x2)

-

a ( x1 , x2 , x3)

p ' ( x1 , x 2) : depends x1

-

Possible add:

q ' ( x1 , x3) : depends x 3

Possible delete:

r ' (x 1 , x 2) : depends x 2

Possible precondition:

The domain writer knows thatis a possible precondition ofwhen has some specific value, but unknown.

p ' ( x1 , x 2)a ( x1 , x2 , x3)

x1

p ' ( x1 , x 2) : depends x1

have_food(M: Meals, C: Town) has possible precondition need_cash(M: Meals, C:Town): depends C

Incompleteness Annotation: Modeling Issues

Correlated incompleteness

a

p

p '

q

r

q '

r '

-

-

Pre(a)

Pre (a)

Add (a)

Del (a)

Add (a )

Del (a )

wapre( p ' )

wadel (r ' )

wadel (q ' )

If p' is realized as a precondition of a, then more likely that r' will be delete effect of the action.

Challenges

Language for domain incompleteness A robustness measure for plans Generating robust plans

A Robustness Measure for Plans

A plan in may fail or succeed in reaching goal states

DD

D1 D2 D2K

Completion set

⟨ ⟨ D ⟩ ⟩Plan execution reaches goal

state? yes no no

Plan robustness: Cummulative probability mass of complete models under which the plan

succeeds.

A Spectrum of Robust Planning Problems

Robustness assessment Maximally robust plan generation Generating plans with desired level of robustness Cost sensitive robust plan generation Incremental robustification

Challenges

Language for domain incompleteness A robustness measure for plans Generating robust plans

to Conformant Probabilistic Planning Problem

Conformant Probabilistic Planning Problem

Domain Proposition set

Action

Preconditions

Conditional effects

Problem

D'=⟨F ' , A ' ⟩

F '

a '∈A'

Pre (a ' )⊆F '

e=(cons(e ) , O (e)={(Pr (ϵ) , add (ϵ) , del (ϵ))})

Mutually exclusive and

exhaustive

P '=⟨D ' ,b I ,G ' ,ρ⟩

0.7

0.3

a' 0.2

0.8

Compilation Approach: An Example

Compilation Example

Compiled “pick-up”

Correctness of the compilation

Experimental Results

Logistics Two cities and each with a downtown and airport. Heavy packages at the downtown areas Robots at the airport of the city

Source of incompleteness: robots were made

from the same manufacturer, having possible

precondition that packages should not be heavy to pick. Goals: move packages from to and vice versa.

C 1 C 2

Ri ,1 , ... , Ri ,m C i

R1, j , R2, j

C 1 C 2C 1

Plans can be made more robust by using robots from different manufacturers after moving them into the downtown area, with the cost of increasing the plan length.

Experimental Results

Satellite Two satellites and orbiting the planet Earth Imagers installed on

Source of incompleteness: lense of were

made from the type of material and can produce

possible effect that images taken are mangled. Goals: images taken in some mode at some direction.

S1 S 2

Li ,1 , ... , Li ,m

L1, j , L2, j

Plans can be made more robust by using additional instruments, which might be in different satellites, but should be of different types of material and can also take an

image of the interested mode at some direction.

S i

M j

Experimental ResultsLogistics Satellite

Observations

Fixed the number of models:

Plan tends to be longer with increasing robustness threshold

Fixed the robustness threshold:

The maximal robustness value of plans that can be returned increases with higher number of manufacturers.

Number of models: 2m

Related work K-faults plans (Jensen et al 2004) Plan evaluation with incomplete models

(Garland & Lesh, 2002) Planning and Acting in Incomplete Domains

(Bryce & Weber, 2011) Robust temporal planning (Fox 2006) Handling incompleteness at the atomic level:

MDP with uncertain transition probabilities (Satia & Lave 1973; Delago & Sanner 2009)

Bounded parameter MDP (Givan, Leach, Dean 2000)

Conclusion & Future work Introduce planning with incomplete models Incompleteness annotations Robustness measure for plans A spectrum of robust planning problems Finding a plan with at least a robustness

value: compilation approach Future work:

Heuristic approach utilizing annotations Plan robustification, and other problems in

the spectrum.

Thank you!Q&A

Backup

Uniform distribution: 6/8 robustness

0.7

0.3

Belief state b

a' 0.2

0.8

Resulting belief state b'

R(π , P)= ∑D i∈⟨⟨ D ⟩ ⟩ ,γ(π , I ,G )∣%equal G

Pr (Di)

Problem Formulation

a

p

p '

q

r

q '

r '

-

-

Pre(a)

Problem Formulation

a

p

p '

q

r

q '

r '

-

-

Pre(a)

Pre (a)

Add (a)

Del (a)

Add (a )

Del (a )

wapre( p ' )

wadel (p' )

wadel (p' )