reasoning in discrete event systemsusers.cecs.anu.edu.au/~patrik/lss/lss13.pdfreasoning in discrete...
TRANSCRIPT
Reasoning In Discrete Event Systems
P@trik Haslum
Logic Summer School 2013
Reasoning In Discrete Event Systems
1. Introduction & basic concepts.2. Planning, verification & diagnosis.3. State-space exploration.4. Relaxations and heuristics.5. What about logic?6. Infinite-state systems.
Introduction
Example: The Optical Telegraph Protocol
* Communications Protocol:- Message vocabulary.- Rules govern behaviour of
protocol participants.
* Model protocol participants.
* Model system bycomposition.
Introduction
Idle
Take CtlSend Start
Recv Attn
Send Data Recv Data
Send Stop
Recv Stop
Release Ctl
Recv StartTake Ctl
Send Attn
Recv DataSend Data
Recv Stop
Send Stop
Release Ctl
Discrete Event Model of a Telegraph Operator
Introduction
Synchronised Composition
* Discrete event system: S = (Q,L,qI ,T ), where- Q is the set of states;- L is the set of event labels (alphabet);- qI ∈ Q is the initial state; and- T ⊆ Q × L×Q the transition relation.
* S1 × S2 = (Q1 ×Q2,L1 ∪ L2, (qI1,qI2),T ), where- T ((q1,q2), (a,b), (q′1,q
′2)) iff T1(q1,a,q′1), T2(q2,b,q′2)
and aRb;- T ((q1,q2), (a, ), (q′1,q2)) iff T1(q1,a,q′1) and¬∃b ∈ L2 : aRb;
- T ((q1,q2), ( ,b), (q1,q′2)) iff T2(q2,b,q′2) and¬∃a ∈ L1 : aRb; and
- R ⊆ L1 × L2 is the label synchronisation relation.
Introduction
0 1
S1.a
S1.c
2S1.dS1.b
×0 1
S2.a
2
S2.bS2.e
3S2.a
S2.c
=(0, 0) (1, 1)
(a,a)(2, 1)
(d , ·)
(0, 2)
(b,b)(·,e)
(1, 3)(a,a)
(c, c)
(1, 0) (2, 0)(d , ·)
(1, 2)
(·,e)
(2, 2)(d , ·)
(·,e)
(0, 1) (0, 3) (2, 3)
Introduction
24 states
215 states
1911 states
16984 states
150945 states
Planning, Verification & Diagnosis
Planning
“Planning is the art and practice of thinking before acting.”
* Given a world model, and initial world state, and a desiredgoal condition, find a (cheapest, fastest, etc.) course ofactions that transforms the world from the initial state to astate where the goal condition holds.
* The Classical Planning Assumptions:- The (model of the) world is finite, discrete and
deterministic.- The world model complete and correct.- There is no interference.
* In what sitation could such assumptions possibly hold?
Planning, Verification & Diagnosis
Puzzles
Planning, Verification & Diagnosis
(Ruml, Do, Zhou & Fromherz 2011)
(c) MYDATA automation AB
(Helmert & Lasinger 2010)
Industrial Automation
Planning, Verification & Diagnosis
(Hatzack & Nebel 2001)
(Trug, Hoffmann & Nebel 2004)
Airport Ground Traffic Control
Planning, Verification & Diagnosis
S4
G11S6S7
S3
S1
−G84 S5
S2
→ S4
−S7
−S6G11
S3
S1
−G84 S5
S2
→ S4
−S7
−S3−S6
G11
S1
−G84 S5
S2
→ S4
−S7
−S3S6
G11
S1
−G84 S5
S2
(Blanchette et al. 1999)
Genome Edit Distance Computation
Planning, Verification & Diagnosis
The Classical Planning World Model
* States are assignments of truth values to a finite set ofatomic propositions (V ).
* Transitions are instances of actions, characterised by:- Preconditions: Must hold in the state for action to be
applicable.- Effects: Will hold in the state after action application.- Assumption of inertia: Atoms not affected by the action
remain as they were.
* There is a single initial state.
* The set of goal states is a defined by a formula over V .
Planning, Verification & Diagnosis
Example: Sokoban
(empty s22)(empty s23)(empty s32)(empty s33)(box s42) (man s43) (empty s44)(empty s45)(empty s52)(empty s53)(box s54) (empty s55)(empty s62)(empty s63)
Planning, Verification & Diagnosis
Verification
* Given a system model, decide if the system meetsspecified requirements:- Safety.- Liveness (absence of deadlock states).- Progress / responsiveness.
* Distributed & concurrent systems:- Set of finite-state processes, may be dynamic.- Inter-processes communication: shared memory;
synchronised transitions; or message queues (pipes).- Non-deterministic: unpredictable relative speeds; or
abstract model.
Planning, Verification & Diagnosis
Protocols & Sequential Circuits
Planning, Verification & Diagnosis
Diagnosis & Diagnosability
* Given a system model, some knowledge of the initialsystem state and a series of observations of events fromthe running system, decide- what are possible current states of the system?- what unobservable events can / cannot have happened?- is system behaving according to the model?
* Given a system model, decide- what unobservable (failure) events, if any, can go
undiagnosed infinitely?- where to place additional sensors to make important
(e.g., failure) events diagnosable, at minimum cost?
Planning, Verification & Diagnosis
Example: The Vadstena Water Plant
Planning, Verification & Diagnosis
Industrial Automation
Planning, Verification & Diagnosis
(Williams & Nayak 1996)
(c) NASA
Autonomous Systems
Planning, Verification & Diagnosis
(Sohrabi, Udrea & Riabov 2013)
client1.connect(server)client1.high NX volumeserver.connect(client1)server.port scanning(client2)server.port scanning(client3)client3.connect(server)client2.connect(server)client2.many requests to one domainclient3.many requests to one domain
Security
Planning, Verification & Diagnosis
Example: WITAS UAV
10877070000 NAV.0.0 Created { MissionExecutor { { Position { 265.0 -66.0 20.0 ...10893800000 NAV.0.0 CheckPt 4 { { SegmentSeq { { { { 289.7897914429 -72.7260848284 ...10895050000 F3D.0.0 Created { NAV.0.0 { { { SegmentSeq { { { { 289.7897914429 ...10895770000 F3D.0.0 CheckPt 11 { { { double 19.1655151302 } } } }10896280000 F3D.0.0 CheckPt 2110990120000 Heli 0 FINISHED10990270000 F3D.0.0 CheckPt 2410990380000 F3D.0.0 Mode:done10990660000 F3D.0.0 Destroyed
Planning, Verification & Diagnosis
1
4
4
4
4
4
4
3
4
2
2
4
2
2
2 2
4
4
4
Planning, Verification & Diagnosis
1
4
4
4
4
4
4
3
4
2
2
4
2
2
2 2
4
4
4
Planning, Verification & Diagnosis
1
4
4
4
4
4
4
3
4
2
2
4
2
2
2 2
4
4
4
Planning, Verification & Diagnosis
Diagnosability
* Let Mod be the set of all event sequences allowed by thesystem model.
* A fault f is diagnosable iff ∃n∀s ∈ Mod : f ∈ s∀s′ ∈ Mod : s′ = s, t ; |obs(t)| ≥ n∀s′′ ∈ Mod : obs(s′′) = obs(s′)f ∈ s′′,
i.e., f is detectable after at most n observations.
* Twin-plant construction:- Compose S with a copy S′ of itself, synchronising on
observations.- If no infinite (i.e., looping) trajectory in S × S′ such that f
occurs in S but not in S′ exists, f is diagnosable.
Planning, Verification & Diagnosis
Problem Reductions
* Planning-as-model checking- Create a deadlock when goal achieved.
* Diagnosis-as-planning- Goal is to generate the given observations.- Need 0/1 action costs to minimise fault events.
* Planning-as-diagnosis- Explain observation “goal achieved”, with/without
“cheating” (fault).
* Diagnosability-as-planning- “Mark & return” for looping acceptance path.
* Model checking-as-planning- Synchronise with Buchi automaton.
Planning, Verification & Diagnosis
Domain Independence
* System models required for reasoning typically given in aformal description language.
* A reasoner/algorithm is domain-independent if it isapplicable to a class of models that is defined by what thereasoners input language can express.
* Why, or why not, domain-independence?- Generality of algorithms & underlying principles.- Improve reuse, rapid prototyping.- Opportunities for improvement (efficiency, reasoning
capability) by specialisation to a domain / application.
* No reasoning systems are truly domain-independent.
Planning, Verification & Diagnosis
0 : (move-up g)1 : (move-right h)2 : (move-down f)3 : (move-down f)4 : (move-right b)5 : (move-right b)6 : (move-up d)7 : (move-up d)8 : (move-right a)9 : (move-down c)
10 : (move-down c)11 : (move-down c)12 : (move-left a)13 : (move-down d)14 : (move-down d)15 : (move-left b)16 : (move-left b)17 : (move-up f)18 : (move-up f)19 : (move-left h)20 : (move-left b)21 : (move-up d)22 : (move-left h)23 : (move-left h)24 : (move-down f)25 : (move-down f)
26 : (move-down c)27 : (move-left h)28 : (move-down d)29 : (move-right b)30 : (move-right b)31 : (move-right b)32 : (move-up d)33 : (move-right h)34 : (move-up c)35 : (move-up d)36 : (move-right a)37 : (move-up c)38 : (move-up c)39 : (move-left h)40 : (move-up e)41 : (move-left i)42 : (move-left i)43 : (move-left i)44 : (move-down f)45 : (move-down g)46 : (move-right a)47 : (move-right a)48 : (move-down g)49 : (move-down g)50 : (move-right a)
* Trying to find a solution for ∼ 3 days.
* Writing a PDDL model took ∼ 45 minutes.
* Planner to find an optimal plan: 0.1 seconds.
State-Space Exploration
State-Space Exploration
* Given an implicit representation of S = S1 × . . .× Sn,generate (and store) only as much of S as is needed tosolve the problem.
* Forward & Regression search.
* Graph search algorithms:- Best-First Search.- Memory-limited search.
State-Space Exploration
......
......
...
......
State-Space Exploration
Microban #1
State-Space Exploration
Regression (backward search)
��� � ����� ��� ��� ���������������� ������ ����� � �
!"���������������� ������ �����#� �
!$�
% '&�� &(� & �$��� &����) ��*�� � �!"�
��) ��*�� � �!$�
��+ � � ��, � ��� ��+ � � ��,-� ��� ��+ � � ��,-� ��� ��+ � � ��, � ���
* φ holds in apply(a, s) iff regress(φ,a) holds in s.- apply(a, s) |= ϕ iff s |= regress(ϕ,a).
* In STRIPS notation:
regress(c,a) =
{(c − add(a) ∪ pre(a) if del(a) ∩ c = ∅FALSE otherwise
State-Space Exploration
(box s42),(box s23)
(push-down s22 s32 s42)
(empty s42),(man s22),(box s32),(box s23)
(move-down s42 s52)
(empty s52),(man s42),(man s22),(box s32),(box s23)
(move-up s42 s32)
(man s42),(empty s32),(man s22),(box s32),(box s23)
(move-up s32 s22)
(empty s42),(man s32),(empty s22),(box s32),(box s23)
. . .
(push-up s62 s52 s42)
(man s62),(empty s42),(box s52),(box s23)
. . .
(push-up s43 s33 s23)
(man s43),(empty s23),(box s42),(box s33)
. . . . . .
(push-left s44 s43 s42)
(man s44),(empty s42),(box s43),(box s23)
. . .
State-Space Exploration
(box s42),(box s23)
(push-down s22 s32 s42)
(empty s42),(man s22),(box s32),(box s23)
(move-down s42 s52)
(empty s52),(man s42),(man s22),(box s32),(box s23)
(move-up s42 s32)
(man s42),(empty s32),(man s22),(box s32),(box s23)
(move-up s32 s22)
(empty s42),(man s32),(empty s22),(box s32),(box s23)
. . .
(push-up s62 s52 s42)
(man s62),(empty s42),(box s52),(box s23)
. . .
(push-up s43 s33 s23)
(man s43),(empty s23),(box s42),(box s33)
. . . . . .
(push-left s44 s43 s42)
(man s44),(empty s42),(box s43),(box s23)
. . .
State-Space Exploration
(box s42),(box s23)
(push-down s22 s32 s42)
(empty s42),(man s22),(box s32),(box s23)
(move-down s42 s52)
(empty s52),(man s42),(man s22),(box s32),(box s23)
(move-up s42 s32)
(man s42),(empty s32),(man s22),(box s32),(box s23)
(move-up s32 s22)
(empty s42),(man s32),(empty s22),(box s32),(box s23)
. . .
(push-up s62 s52 s42)
(man s62),(empty s42),(box s52),(box s23)
. . .
(push-up s43 s33 s23)
(man s43),(empty s23),(box s42),(box s33)
. . . . . .
(push-left s44 s43 s42)
(man s44),(empty s42),(box s43),(box s23)
. . .
State-Space Exploration
(box s42),(box s23)
(push-down s22 s32 s42)
(empty s42),(man s22),(box s32),(box s23)
(move-down s42 s52)
(empty s52),(man s42),(man s22),(box s32),(box s23)
(move-up s42 s32)
(man s42),(empty s32),(man s22),(box s32),(box s23)
(move-up s32 s22)
(empty s42),(man s32),(empty s22),(box s32),(box s23)
. . .
(push-up s62 s52 s42)
(man s62),(empty s42),(box s52),(box s23)
. . .
(push-up s43 s33 s23)
(man s43),(empty s23),(box s42),(box s33)
. . . . . .
(push-left s44 s43 s42)
(man s44),(empty s42),(box s43),(box s23)
. . .
Relaxations and Heuristics
Relaxations and Heuristics
* A relaxation is a problem transformation that preservessolution existence and cost.
* If the relaxed problem is easier to solve (e.g., polynomial),optimal solution cost for the relaxed problem is usable asan admissible heuristic for the original problem.
* Relaxations of the reachability problem:- Abstraction.- Monotonic relaxation.
Relaxations and Heuristics
Abstraction
* An abstraction is a mapping from (states of) S to someabstract space S′, which preserves labelled paths andgoal states.
Relaxations and Heuristics
Monotonic Relaxation
* State variables/components accumulate values.
* Propositionally, facts, once true, never become false.- In planning, this is known as the delete relaxation (P+).
* Relaxation: any plan for P is also valid for P+.
* Solving P+ is in P.- Applying an action (transition) cannot disable any other
action.- No action needs to be applied more than once.
* Solving P+ optimally is NP-hard.
Relaxations and Heuristics
Example: Monotonic Relaxation
B
A
C
(on-table A)(on-table B)(on-table C)(on A B)(on A C)(on B A)(on B C)(on C A)(on C B)(clear A)(clear B)(clear C)(MoveToT+ A B)pre: (on A B),
(clear A)add: (on-table A),
(clear B)
A
A
B C
(on-table A)(on-table B)(on-table C)(on A B)(on A C)(on B A)(on B C)(on C A)(on C B)(clear A)(clear B)(clear C)(MoveFromT+ B C)pre: (on-table B),
(clear B), (clear C)add: (on B C)
A
B
A
A C
B
(on-table A)(on-table B)(on-table C)(on A B)(on A C)(on B A)(on B C)(on C A)(on C B)(clear A)(clear B)(clear C)
Logical Reasoning About Discrete Event Systems
Using Logic
* Proving invariance.
* Bounded reachability and propositional logic.
* Tense-modal logic- Logic as a specification language.- LTL and LTL resolution.
* Proving non-reachability.
Logical Reasoning About Discrete Event Systems
Invariants (for Planning)
* A formula ϕ is an invariant iff s |= ϕ and enabled(a, s)implies apply(a, s) |= ϕ, for s and a.- If ϕ is invariant and holds in s0, ϕ holds in any reachable
state.
* Testing invariance w.r.t. action schemas:- Regression: apply(a, s) |= ϕ iff s |= regress(ϕ,a).- ϕ is invariant iff for each a
∀~x(ϕ ∧ pre(a(~x))→ regress(ϕ,a(~x))
is a tautology.
Logical Reasoning About Discrete Event Systems
Regression over Action Schemas
* Regressing an atomic formula: apply(a(~y), s) |= p(~x) iff- p(~z) ∈ effects(a(~y)) and ~x = ~z; or- s |= p(~x) and for any ¬p(~z) ∈ effects(a(~y)), ~x 6= ~z
* Regressing any formula: substitute condition for atoms.
* move-right(f , t)precond: right-of(f , t),man(f ),empty(t)effects: ¬man(f ),empty(f ),man(t),¬empty(t)
* regress(man(x) ∨ box(x) ∨ empty(x),move-right(f , t))= ((x = t) ∨ (man(x) ∧ x 6= f )) ∨ box(x)∨
((x = f ) ∨ (empty(x) ∧ x 6= t)).
Logical Reasoning About Discrete Event Systems
Example: Invariants (?)
1. ∀x(man(x) ∨ box(x) ∨ empty(x))
2. ∀x((¬man(x) ∧ ¬box(x)) ∨ (¬man(x) ∧ ¬empty(x))∨(¬box(x) ∧ ¬empty(x)))
3. ∀x , y(man(x) ∧ man(y)→ (x = y))
4. ∀x , y , z(box(x) ∧ box(y) ∧ box(z)→((x = y) ∨ (x = z) ∨ (y = z)))
5. ∃x , y(box(x) ∧ box(y) ∧ (x 6= y))
Logical Reasoning About Discrete Event Systems
Horizon-k Planning as SAT
p1@[email protected]@0
a1@[email protected]@1
p1@[email protected]@1
a1@[email protected]@2
p1@[email protected]@2
. . .
a1@[email protected]@k
p1@[email protected]@k
* The set of valid plans up to (parallel) length k can beencoded as a propositional logic formula φ.
Logical Reasoning About Discrete Event Systems
* a@i →∧
p∈pre(a) p@(i − 1).- If a takes place at i , pre(a) must hold at i − 1.
* a@i →∧
p∈add(a) p@i .
* a@i →∧
q∈del(a) ¬q@i .- If a takes place at i , it’s effects hold at i .
* (p@i ∧ ¬p@(i − 1))→∨{a | p∈add(a)} a@i .
* (¬p@i ∧ p@(i − 1))→∨{a | p∈del(a)} a@i .
- If p changes from i − 1 to i , some action must havecaused the change.
Logical Reasoning About Discrete Event Systems
* p@0 if p ∈ s0.
* ¬p@0 if p 6∈ s0.- The first atom layer is the initial state.
* p@k if p ∈ G.- The last atom layer satisfies the goal.
* ¬a@i ∨ ¬a′@i if del(a) ∩ (pre(a′) ∪ add(a′)) 6= ∅ ordel(a′) ∩ (pre(a) ∪ add(a)) 6= ∅.- More than one action can take place at i as long as they
do not interfere.- Weaker requirement: actions at i can be performed in
some sequence.
Logical Reasoning About Discrete Event Systems
SAT-based planning
* How to find the right k?
* Test Φk , k = 1,2, . . .,until a satisfyingassignment (plan) isfound.
* Can be very inefficient.
* This is what SATPLANdoes.
(Rintanen, Heljanko & Niemela, 2006)
Logical Reasoning About Discrete Event Systems
SAT-based planning
* Try many values inparallel, allocatinggeometrically less timeto higher values.
* As SAT formulasproved unsolvable, thefrontier moves up.
* Limited by memory.(Rintanen, Heljanko & Niemela, 2006)
Logical Reasoning About Discrete Event Systems
Tense-Modal Logic
* LTL, CTL and CTL* are tense modal logics.- LTL is interpreted over paths (sequences of states).- CTL and CTL* are interpreted over trees.
* Property specification language:- Safety: �¬deadlock (LTL), ∀�¬deadlock (CTL)- Responsiveness: �(p → ♦q) (LTL), ∀�(p → ∀♦q) (CTL)- Possibility: ∀�∃♦goal (CTL)- ♦�p (LTL)
Logical Reasoning About Discrete Event Systems
Linear Temporal Logic (LTL)
* LTL is interpreted over infinite sequences of states.
* w = s0, s1, . . .:- w |= p iff s0 |= p.- Interpretation of logical connectives (¬, ∧, ∨) is standard.- w |=©α iff w1 = s1, s2, . . . |= α.- w |= (αU β) iff ∃k .wk |= β and ∀0 ≤ i < k .w i |= α.
* Common abbreviations;- ♦α ≡ (TRUE U α)- �α ≡ (αU FALSE)- αWβ ≡ (αU β) ∨�α
Logical Reasoning About Discrete Event Systems
Planning as LTL-SAT
* The set of valid plans can be encoded as an LTL formula:- �(a→
∧p∈pre(a) p) (action implies prec.)
- �(a→∧
p∈add(a)©p)
- �(a→∧
p∈del(a)©¬p) (action implies eff.)- �(¬p ∧
∧{a | p∈add(a)} ¬a→©¬p)
- �(p ∧∧{a | p∈del(a)} ¬a→©p) (frame axioms)
- �(¬a ∨ ¬a′) (single action per step)- p if p ∈ s0, ¬p if p 6∈ s0 (init state)- ♦
∧p∈G p (eventually goal)
Logical Reasoning About Discrete Event Systems
LTL Resolution
* Clausal normal form:- �(L1 ∨ . . . ∨ Lm) (state clause)- �(A→©B) (step clause)- �(A→©♦L) (eventuality clause)- (L1 ∨ . . . ∨ Lm) (initial clause)(A conjunction of literals, B disjunction of literals).
* State and step resolution:- Resolve state/step clauses with conflict literals of
“matching” modality.
* ♦-resolution.- (∨
i Ai)→©�¬L, C →©♦L⇒ C → (∧
i ¬Ai)WL- (∨
i Ai)→©�¬L is constructed from a loop in ¬L.
Logical Reasoning About Discrete Event Systems
* A merged step clause is a conjunction of step clauses,∧j �(Aj →©Bj), rewritten as�(Aj ∧ . . . ∧ Am →©(B1,1 ∧ . . . ∧ Bk ,1) ∨ . . . ∨©(B1,m ∧ . . . ∧ Bk ,m)).
* A set of merged step clauses �(Ai →©Bi) such that- Bi |= ¬L, and- Bi |=
∨i Ai ,
is a loop in ¬L.
* A loop is a conditional invariant: (∨
i Ai)→©�¬L.
Logical Reasoning About Discrete Event Systems
Proving Unreachability
* Exhaustive state-space search.
* Bounded SAT-encoding combined with induction.
* Constructive:- LTL-Resolution.- Relaxed regression.
* Constructive proof search can be guided.
Searching for something that exists is usually easier thansearching for something that does not exist.
Infinite-State Systems
Infinite-State Systems
* The undecidability problem.- Counter machines.
* Petri nets.
* Timed & hybrid automata.
Infinite-State Systems
k -Counter Machines
* Deterministic finite program with kcounters of infinite capacity.
* Typical instruction set:- Set ci to 0.- Increment or decrement ci by 1.- goto n.- if ci = 0 goto n else goto n′.- Halt.
* Halting of k -CMs is undecidable fork > 2.
1 clear c22 if c1 = 0 goto 73 dec c14 inc c25 inc c26 goto 27 if c2 = 0 goto 118 dec c29 inc c1
10 goto 711 ...
Infinite-State Systems
Petri Nets* Directed bipartite graph of places and transitions.- Places are counters: A marking assigns each place
n ∈ N tokens.- Transition t applicable at marking n iff ∀p ∈ •t ,m(p) > 0
and leads to marking m′ = m − •t + t•.
* Marking reachability for Petri nets is decidable.
p1
t1
p2
p3t2
t3
t1−→p1
t1
p2
p3t2
t3
t3−→p1
t1
p2
p3t2
t3
t1−→p1
t1
p2
p3t2
t3
t2−→p1
t1
p2
p3t2
t3
t2−→p1
t1
p2
p3t2
t3
Infinite-State Systems
Timed Automata
* Non-deterministic finite automatawith real-valued clocks.- Transition guards: Linear
conditions on clocks and clockdifferences.
- Transitions can reset clocks.- Clocks evolve at uniform rate.
* Reachability is PSPACE-completefor a single automaton.