atai17 4 conditional planning-1 · *hudog 6whlqedxhu,qvwlwxwh iru 6riwzduh 7hfkqrorj\ $gydqfhg...

$: atai17 4 conditional planning-1 · *hudog 6whlqedxhu,qvwlwxwh iru 6riwzduh 7hfkqrorj\ $gydqfhg 7rslfv ri $, ±&rqglwlrqdo 3odqqlqj *hudog 6whlqedxhu,qvwlwxw i u 6riwzduhwhfkqrorjlh$
1

Gerald Steinbauer

Institute for Software Technology

1

Advanced Topics of AI – Conditional Planning

Gerald Steinbauer

Institut für Softwaretechnologie

Inffeldgasse 16b/2

A-8010 Graz

Austria

Advanced Topics of Artificial Intelligence- Conditional Planning -

Gerald Steinbauer


2


Motivation• so far: fully observable, static and deterministic domains

• agent can plan first and then execute plan with eyes closed

• uncertain environment: • incomplete (partially observable and/or nondeterministic)• incorrect (differences between world and model) information• use sensing• adapt plan when necessary

• degree of uncertainty defined by indeterminacy• bounded: actions can have unpredictable effects, yet can be listed in

action description axioms• unbounded: preconditions and effects unknown or too large to

enumerate

Gerald Steinbauer


3


Handling indeterminacy

• sensorless planning (conformant planning)• find plan that achieves goal in all possible circumstances (regardless

of initial state and action effects)

• conditional planning (contingency planning)• Construct conditional plan with different branches for possible

contingencies

• execution monitoring and replanning• while executing plan judge whether plan requires revision

• continuous planning • planning active for a life time: adapt to changed circumstances and

reformulate goals if necessary

Gerald Steinbauer


4


Abstract example• initial state = <chair, table, cans of paint, unknown colors>

goal state = <color(table) = color(chair)>

• sensorless planning (conformant planning)• open any can of paint and apply it to both chair and table

• conditional planning (contingency planning)• sense color of table and chair, if they are the same then finish;

else sense label’s paint; if color(label) = color(Furniture) then apply color to other piece; else apply color to both

• execution monitoring and replanning• same as conditional and can fix errors (missed spots)

• continuous planning • can revise goal when we want to first eat before painting the table and the chair

2

Gerald Steinbauer


5


Vacuum Cleaner

3 state variables (at_r, at_l, clean_r, clean_l)

3 actions (right, left, suck)

Gerald Steinbauer


6


Murphy Vacuum Cleaner

• single murphy vacuum cleaner• non-deterministically the cleaner dumps dust on the location it

moves to if location is clean

• double murphy vacuum cleaner+ non-deterministically the cleaner dumps dust on the location if it

sucks and location is clean

• triple murphy vacuum cleaner+ non-deterministically the cleaner does not move if it is command to

Gerald Steinbauer


7


Sensorless Planning (1)

• assume• partial observability/incomplete knowledge

• deterministic actions

• state space becomes a belief space• 8 states

• 2 beliefs (collections of possible states)

• in general 2

Gerald Steinbauer


8


Sensorless Planning (2)

brave vs. cautious execution

3

Gerald Steinbauer


9


Prediction of Beliefs

• still no sensing

• deterministic vs. non-deterministic actions

• belief space allows to represent both

deterministic right non-deterministic rightGerald Steinbauer


10


Different Plans

• weak plan (optimistic)• the plan has a chance of success, i.e. some of its executions

achieve the goal

• strong plan (pessimistic)• the plan is guaranteed to achieve the goal for all possible

executions

• strong cyclic plan (reasonably optimistic)• the plan reaches the goal with an iterative trial-and-error strategy,

such that all the associated executions always have a possibility of terminating and, when they do, they achieve the goal

• sensorless (conformant) planning looks for a strong plan without observability

Gerald Steinbauer


11


Conditional Planning (1)

• deal with uncertainty by checking the environment to see what is really happening

• used in fully observable and nondeterministic environments: • the outcome of an action is non-deterministic• conditional steps will check the state of the environment• how to construct a conditional plan?

Gerald Steinbauer


12


Conditional Planning (2)

• actions: left, right, suck• propositions to define states: 𝑎𝑡_𝑟, 𝑎𝑡_𝑙, 𝑐𝑙𝑒𝑎𝑛_𝑙,

𝑐𝑙𝑒𝑎𝑛_𝑟

• how to include indeterminism?• actions can have more than one effect

• e.g., moving left sometimes fails• Action(Left, PRECOND: AtR, EFFECT: AtL)• Becomes: Action(Left, PRECOND: AtR, EFFECT: AtL AtR)

• actions can have conditional effects (vacuum cleaner sometimes dumps dirt when moved)

• Action(Left, PRECOND: AtR,EFFECT: AtL (AtL when cleanL: ¬cleanL)

• both disjunctive and conditional (“when cleanL” refers to before the action is taken, “¬cleanL” refers to after action)

4

Gerald Steinbauer


13


Conditional planning (3)

• conditional plans require conditional steps:• if <test> then plan_A else plan_B

• if 𝑎𝑡_𝑙𝑐𝑙𝑒𝑎𝑛_𝑙 then 𝑟𝑖𝑔ℎ𝑡 else 𝑠𝑢𝑐𝑘

• plans become trees

• games against nature:• find conditional plans that work regardless of which action

outcomes actually occur

• assume vacuum-world• initial state = 𝑎𝑡 𝑐𝑙𝑒𝑎𝑛_𝑙 𝑐𝑙𝑒𝑎𝑛_𝑟

• would be easy, except that we have a double murphy vacuum cleaner: possibility of depositing dirt when moving to clean other square, and possibility of depositing dirt when action is suck in a clean square

Gerald Steinbauer


14


Game tree

state node

chance node

[left, if at_l clean_l clean_r then [ ] else suck]

Gerald Steinbauer


15


Solution of games against Nature

• solution is a subtree that• has a goal node at every leaf• specifies one action at each of its state nodes• oncludes every outcome branch at each of the chance nodes

• in previous example:• [left, if at_l clean_l clean_r then [ ] else suck]

• for exact solutions: use minimax algorithm with 2 modifications:• max and min nodes become OR and AND nodes• algorithm returns conditional plan instead of single move

• at an OR node, the plan is the action selected• at an AND node, the plan is a nested series of if-then-

else steps (or “cases”) specifying subplans for each outcome (with tests being complete state descriptions)

Gerald Steinbauer


16


Search And-Or Graph Search (1)

“discards” repeated states in graph, like the “Loop” state in vacuum cleaner game tree

5

Gerald Steinbauer


17


• how does it deal with cycles?• when a state already on the path appears, return failure

• if there is a non-cyclic solution, it is reachable from the earlier incarnation of the state

• ensures algorithm termination in every finite state space, since every path must reach a goal, a dead end, or a repeated state

• but the algorithm does not check whether some state is already on some other path from the root


Gerald Steinbauer


18


• sometimes only a cyclic solution exists • e.g., triple Murphy: sometimes the move is not performed

• [Left, if CleanL then [] else Suck] is not a solution

• our And-Or-search algorithm would fail• modification: use label to repeat parts of plan (but now there may be infinite

loops)• [L1: left, if at_r then L1 else if clean_l then [] else suck]

search graph for the “triple murphy” world,

left doesn’t always move the vacuum

cleaner


Gerald Steinbauer


19


Conditional Planning and Partially Observable Environments

• fully observable: conditional tests can ask any question and get an answer

• partially observable• the agent has limited information about the environment• modeled by a state-set = belief states• e.g., assume vacuum agent which cannot sense presence or absence

of dirt in squares other than the one it’s on• alternate double murphy: dirt can be left behind when agent leaves a clean

square• solution in fully observable world: keep moving left and right, sucking dirt

whenever it appears until both squares are clean and I’m in square left• but “both squares a clean” can’t be tested now…

Gerald Steinbauer


20


Incomplete Knowledge

• the agent knows: 𝑐𝑙𝑒𝑎𝑟 𝐴 , 𝑐𝑙𝑒𝑎𝑟 𝐶 , 𝑐𝑙𝑒𝑎𝑟 𝐷 , 𝑜𝑛𝑇𝑎𝑏𝑙𝑒 𝐶 , ¬𝑜𝑛 𝐴, 𝐶 ,

¬𝑜𝑛 𝐴, 𝐷 , 𝑜𝑛 𝐴, 𝐵 ∨ 𝑜𝑛𝑇𝑎𝑏𝑙𝑒 𝐴 ,∀𝑥. ¬𝑜𝑛 𝑥, 𝐵 → 𝑜𝑛𝑇𝑎𝑏𝑙𝑒 𝑥

• the agent doesn’t know: 𝑐𝑙𝑒𝑎𝑟 𝐵 , ¬𝑐𝑙𝑒𝑎𝑟 𝐵 , 𝑜𝑛𝑇𝑎𝑏𝑙𝑒 𝐴 , ¬𝑜𝑛𝑇𝑎𝑏𝑙𝑒(𝐴)

6

Gerald Steinbauer


21


Multiple Worlds

• the agent believes to be in multiple worlds

• all words evolve separately

• the agent knows at the end: 𝑜𝑛 𝐷, 𝐶 , 𝑜𝑛 𝐴, 𝐵 ∨𝑜𝑛𝑇𝑎𝑏𝑙𝑒 𝐴 , ∀𝑥. 𝑜𝑛𝑇𝑎𝑏𝑙𝑒 𝑥 → 𝑥 = 𝐴 ∨ 𝑥 = 𝐵

Gerald Steinbauer


22


Defining Knowledge

• we define a new predicate 𝐾 𝑠, 𝑠• called accessibility relation

• expresses that for an agent it is the same to be in situation 𝑠 or 𝑠’

• we can express that an agent knows 𝜙 in situation 𝑆• ∀𝑠. 𝐾 𝑠, 𝑆 → 𝜙[𝑠]

• e.g. ∀𝑠. 𝐾(𝑠, 𝑆 ) → ∀𝑥. 𝑜𝑛𝑇𝑎𝑏𝑙𝑒 𝑥, 𝑠 → 𝑐𝑙𝑒𝑎𝑟 𝑥, 𝑠

• logical omniscience:• if ∀𝑠. 𝜙(𝑠) → 𝜓(𝑠) and ∀𝑠, 𝑠 . 𝐾 𝑠, 𝑠 → 𝜙 𝑠

• then ∀𝑠, 𝑠 . 𝐾 𝑠, 𝑠 → 𝜓 𝑠

• we can now define what an agent knows• 𝐾𝑛𝑜𝑤𝑠 𝜙, 𝑠 = ∀𝑠 𝐾(𝑠 , 𝑠) → 𝜙[𝑠′]

• 𝐾𝑊ℎ𝑒𝑡ℎ𝑒𝑟 𝜙, 𝑠 = 𝐾𝑛𝑜𝑤𝑠 𝜙, 𝑠 ∨ 𝐾𝑛𝑜𝑤𝑠(¬𝜙, 𝑠)

Gerald Steinbauer


23


Incomplete Knowledge and Sensing

• situations are discarded that does not match the sensing result, e.g. 𝐹(𝑠)

Gerald Steinbauer


24


Partially Observable:Alternate Double Murphy

this relates back to our

possible worlds formalization of

knowledge; e.g., at top,

agent might be in two possible

worlds, because all it knows is 𝑎𝑡_𝑟and 𝑐𝑙𝑒𝑎𝑛_𝑟

brave vs. cautious execution

7

Gerald Steinbauer


25


Belief states

• three choices for representation• sets of full state descriptions

• {{at_r,clean_r,clean_l},{{at_r,clean_r}}

• logical sentences that capture the set of possible worlds in the belief state (Open World Assumption, unknown propositions are left unspecified)

• 𝑎𝑡 ∨ 𝑐𝑙𝑒𝑎𝑛_𝑟

• knowledge propositions describing the agent’s knowledge (Closed World Assumption, unknown knowledge propositions are assumed false)

• 𝐾(𝑎𝑡_𝑟) 𝐾(𝑐𝑙𝑒𝑎𝑛_𝑟)

Gerald Steinbauer


26


Belief states• choices 2 and 3 are roughly equivalent (let’s continue with

3)• symbols can appear in three ways: positive, negative or

unknown: 3𝑛 possible belief states for 𝑛 proposition symbols• Yet, set of belief sets is a powerset (set of all subsets) of the

physical states: 2𝑛 physical states, so 22𝑛 belief states --- which is

much larger than 3𝑛

• hence 3 is restricted as representation• any scheme capable of representing every possible belief state will

require 𝑂(2𝑛) bits to represent each one in the worst case • the current scheme only requires 𝑂(𝑛) --- trading expressiveness for

compactness• STRIPS closure

Gerald Steinbauer


27


Sensing in Conditional Planning

• how does it work?• automatic sensing

• At every time step the agent gets all available percepts• active sensing

• percepts are obtained through the execution of specific sensory actions.• checkDirt and checkLocation

• given the representation and the sensing, action descriptions for checkDirt and checkLocation can now be formulated

Gerald Steinbauer


28


Contingent-FF (1)

• implementation of a conditional planner

• by Jörg Hoffmann and Ronen Brafman

• supports incomplete knowledge and conditional/nondeterministic effects

• understands PDDL plus neccesary extensions

• is limited to acyclic conditional plans and restricts the use of nondeterminism

8

Gerald Steinbauer


29


Contingent-FF (2)

• extensions to PDDL• (nondet literal)

• the statement can be used to declare an effect non-deterministic

• supports only one literal

• supports only one non-deterministic effect part per action

• (unknown atom)• declares an atom to be unknown in the initial state

• both possibility will be consider in planning

• (oneof literal_1 literal_2 … literal_n)• allows to specify that exactly one literal holds in the initial state

• (or literal_1 literal_2 … literal_n)• allows to specify that some literals holds in the initial state

• statements are conjunctive -> allows CNF

Gerald Steinbauer


30


Literature

• Stuart J. Russell and Peter Norvig. 2003. Artificial Intelligence: A Modern Approach. Pearson Education.

• Malik Ghallab, Dana Nau and Paolo Traverso. Automated Planning – Theory and Practice. Morgan Kaufman. 2005.

• Malik Ghallab, Dana Nau, and Paolo Traverso. 2016. Automated Planning and Acting. Cambridge University Press, New York, NY, USA.

• Jörg Hoffmann and Ronen I. Brafman. 2005. Contingent planning via heuristic forward search with implicit belief states. In Proceedings of the Fifteenth International Conference on International Conference on Automated Planning and Scheduling (ICAPS'05). AAAI Press 71-88.

• This slides reuse material from publicly available lectures by Dana Nau, Jeff Rosenschein and Jörg Hoffmann.

Gerald Steinbauer


31


Thank you!

atai17 4 conditional planning-1 · *hudog 6whlqedxhu,qvwlwxwh iru 6riwzduh 7hfkqrorj\ $gydqfhg...

Documents