symbolic synthesis of masking fault-tolerant distributed programs borzoo bonakdarpour workshop...
TRANSCRIPT
Symbolic Synthesis ofSymbolic Synthesis ofMasking Fault-Tolerant Masking Fault-Tolerant Distributed ProgramsDistributed Programs
Borzoo BonakdarpourBorzoo BonakdarpourWorkshop APRETAFWorkshop APRETAFJanuary 23, 2009January 23, 2009
Joint work with Sandeep Kulkarni
2
Motivation
• The most important goal of formal methods is achieving correctness in computing systems (programs).– Correct-by-verification
• A program is built manually
• The correctness of the program is verified by a model checker or a theorem prover.
– Correct-by-construction• A program is constructed so that its correctness is guaranteed.
Manual Design
Verification
3
Motivation
• Automated synthesis from temporal logic specifications– Pros:
• Ability to start from a null program
• Capability to handle highly expressive specifications
– Cons:• Highly complex decision procedures
• Limited to no reusability
• Automated program revision– An existing program is revised with respect to a property
4
• Question:
– Is it possible to revise the program automatically such that it satisfies the failed property while ensuring satisfaction of existing properties?
• bugs
• incomplete specification
• change of environment
CounterexampleCounterexamplePropertyProperty
ModelCheckerModel
Checker
ProgramProgram
Motivation
5
Revised Revised programprogramPropertyProperty
RevisionRevisionAlgorithmAlgorithmRevisionRevision
AlgorithmAlgorithm
ProgramProgram
Motivation
6
Motivation
A one-lane bridge is controlled by two traffic signals at the two ends of the bridge.
(sig1 = G) (1 ≤ x1 ≤ 10) sig1 := Y;
[]
(1(1))
(2(2))
(sig1 = Y) (1 ≤ y1 ≤ 2) sig1 := R ;
[]
(sig2 = R) (z1 ≤ 1) sig2 := G ;
[]
{y1}
{z1}
{x2}
Controller Program:Controller Program:
((sig1 = G) (x1 ≤ 10)) ((sig1 = Y ) (y1 ≤ 2)) ((sig1 = R) (z2 ≤ 1)) wait;
SPECSPECbtbt = = {({(00, , 11) | ) | sigsig11 ( (11) ≠ ) ≠ RR sigsig22 ( (11) ≠ ) ≠ RR}}
7
true skip; {z2}
Traffic Controller Fault Action: Traffic Controller Fault Action:
(1(1))
(2(2))
1. (sig1 = sig2 = R) (z1 ≤ 1) (z2 > 1)
2. (sig1 = sig2 = R) (z1 ≤ 1) (z2 = 0)
3. (sig1 = G) (sig2 = R) (z1 ≤ 1) (z2 = 0)
4. (sig1 = G) (sig2 =G) (z1 ≤ 1) (z2 = 0)
Motivation
8
19821982 20002000
19861986
19891989
Vardi and Wolper introduce
automata-theoretic verfication and synthesis
Vardi and Wolper introduce
automata-theoretic verfication and synthesis
2005200519921992
Alur and Henzinger propose verification and synthesis of
real-time systems
Alur and Henzinger propose verification and synthesis of
real-time systems
Clarke, Emerson, Sifakis, and Queille invent model checking
Clarke, Emerson, Sifakis, and Queille invent model checking
19811981
Emerson and Clarke propose synthesis from CTL properties
Emerson and Clarke propose synthesis from CTL properties
McMilan et al. intorduce BDD-based model
checking (1020 reachable states) and find bugs in
IEEE futurebus+
McMilan et al. intorduce BDD-based model
checking (1020 reachable states) and find bugs in
IEEE futurebus+
19931993
Wonham and Ramadge introduce
controller synthesis
Wonham and Ramadge introduce
controller synthesis
Intel reports bug in floating point operations in
Pentium processors
Intel reports bug in floating point operations in
Pentium processors
19941994
Clarke and Grumberg introduce counterexample guided abstraction-refinement (CEGAR), 101000 reachable states
Clarke and Grumberg introduce counterexample guided abstraction-refinement (CEGAR), 101000 reachable states
19991999
Kulkarni and Arora introduce automated addition of fault-tolerance to fault-intolerant programs
Kulkarni and Arora introduce automated addition of fault-tolerance to fault-intolerant programs
Biere and Clarke invent SAT-based model
checking (10500 reachable states)
Biere and Clarke invent SAT-based model
checking (10500 reachable states)
Bonakdarpour, Kulkarni, and Ebnenasir, and, Jobstmann and Bloem independently introduce program revision (repair) techniques
Bonakdarpour, Kulkarni, and Ebnenasir, and, Jobstmann and Bloem independently introduce program revision (repair) techniques
20072007
Bonakdarpour and Kulkarni synthesize distributed programs of size 1050
Bonakdarpour and Kulkarni synthesize distributed programs of size 1050
Clarke, Emerson, Sifakis, and Queille invent model checking
Clarke, Emerson, Sifakis, and Queille invent model checking
McMilan et al. intorduce BDD-based model
checking (1020 reachable states) and find bugs in
IEEE futurebus+
McMilan et al. intorduce BDD-based model
checking (1020 reachable states) and find bugs in
IEEE futurebus+
Mohamed Gouda:Mohamed Gouda:
When does your “12 years” end?!When does your “12 years” end?!
Mohamed Gouda:Mohamed Gouda:
When does your “12 years” end?!When does your “12 years” end?!20082008
Motivation
9
InvariantInvariantInvariantInvariant
f f
f
Fault-SpanFault-Span
State spaceState space
p
p
p
p
p
pp
pp
p
f
f
The Synthesis Problem
10
The Issue of Distribution
• Modeling distributed programs:– A program consists of a set of processes. Each process p is
specified by:• A set Vp of variables,
• A set Tp of transitions,
• A set Rp Vp of variables that p is allowed to read,
• A set Wp Rp of variable that p is allowed to write.
• Write restrictionsa = 0b = 1
a = 0b = 1
a = 1b = 1
a = 1b = 1
a Wp
a = 0b = 1
a = 0b = 1
a = 1b = 1
a = 1b = 1
a Wp
Such transitions cannot be executed by process p.
11
• Read restrictions
a = 1 b = 0
a = 1 b = 0
a = 0 b = 0
a = 0 b = 0
b Rp
a = 1 b = 1
a = 1 b = 1
a = 0b = 1
a = 0b = 1
a = 1 b = 0
a = 1 b = 0
a = 0 b = 0
a = 0 b = 0
b Rp
a = 1 b = 1
a = 1 b = 1
a = 0b = 1
a = 0b = 1
– Such set of transitions form a group.– Addition and removal of any transition must
occur along with its entire group.
The Issue of Distribution
12
What Is DifficultAbout Program Revision?
• Space complexity– The state explosion problem
• Time complexity– NP-completeness
• Identifying the complexity hierarchy of the problem• The need for designing efficient heuristics• Proofs are often helpful in identifying bottlenecks of the problem
The combination of the above complexitiesThe combination of the above complexitiesis the worst nightmare!is the worst nightmare!
13
Daniel MosDaniel Moséé::
As that wise man said “bridging theAs that wise man said “bridging the
gap between theory and practice isgap between theory and practice is
easier in theory than in practice!”easier in theory than in practice!”
Daniel MosDaniel Moséé::
As that wise man said “bridging theAs that wise man said “bridging the
gap between theory and practice isgap between theory and practice is
easier in theory than in practice!”easier in theory than in practice!”
What Is DifficultAbout Program Revision?
14
The Byzantine Agreement Problem
Decision
d.g {0, 1}
(d.j = ) ( f.j = false) d.j := d.g
(d.j ) ( f.j = false) f.j := true
d.j
d.k {0, 1, }
d.l
Decision
f.j
f.k {false, true}
f.l
Final?
GENERAL
NON-GENERALS
Program:
15
The Byzantine Agreement Problem
Byzantine?
b.g {false, true}
b.j
b.k {false, true}
b.l
Byzantine?
(b.j , b.k , b.l , b.g = false) b.j := true
(b.j := true) d.j := 0|1Faults:
16
• Experimental results with enumerative (explicit) state space (the tool FTSyn)– Byzantine agreement - 3 processes
• 6912 states
• Time: 10s
– Byzantine agreement - 4 processes • 82944 states
• Time: 15m
– Byzantine agreement - 5 processes• 995328 states
• Out of memory!
What Is DifficultAbout Program Revision?
17
Polynomial -Time Heuristics
Identify the state predicatems from where faults
alone violate the safety;
S := S ms
ms
SPEC
ff
f f
18
Identify the state predicatems from where faults
alone violate the safety;
S := S ms
Polynomial -Time Heuristics
Re-compute theRe-compute thefault-spanfault-span
ms
Inv.Inv.
Fault-Span
f
f
f
f
p
p
f
p
BDD frontier = Invariant;BDD current = mgr -> bddZero();BDD FaultSpan = Invariant;
while (FaultSpan != current){ current = FaultSpan; BDD image = frontier * (P + F); // -FaulSpan frontier = Unprime(image); FaultSpan = current + frontier; }
BDD frontier = Invariant;BDD current = mgr -> bddZero();BDD FaultSpan = Invariant;
while (FaultSpan != current){ current = FaultSpan; BDD image = frontier * (P + F); // -FaulSpan frontier = Unprime(image); FaultSpan = current + frontier; }
19
Polynomial -Time Heuristics
Yes
No
Identify the state predicatems from where faults
alone violate the safety;
S := S ms
Re-compute theRe-compute thefault-spanfault-span
Identify transitions in the fault-intolerant programIdentify transitions in the fault-intolerant programthat may be included in the fault-tolerant programthat may be included in the fault-tolerant program
Fixpoint?Fixpoint?
Resolve deadlock statesResolve deadlock states
InvariantInvariant
Fault-Span
ffpp
p
Re-computing state predicates or transitions predicates do not occur often in model checking, but it does happen quite often during synthesis.
Re-computing state predicates or transitions predicates do not occur often in model checking, but it does happen quite often during synthesis.
s0
s1
20
Experimental Results
• Polynomial-time sound BDD-based heuristics– The tool SYCRAFT (http://www.cse.msu.edu/~borzoo/sycraft)– C++– CuDD (Colorado University Decision Diagram Package)
• Platform– Dedicated PC – 2.2GHz AMD Opteron processor– 1.2GB RAM
21
Experimental Results
• Goal:– Identifying various bottlenecks of our synthesis heuristics
• Fault-span generation
• Deadlock resolution– Adding recovery– State elimination
• Cycle detection and resolution
• Memory usage
• Total synthesis time
22
Experimental Results
23
Experimental Results
24
Performance of synthesizing the
Byzantine Byzantine agreementagreement
program
Experimental Results
25
• Observations
– 1050 reachable states– State elimination (deadlock resolution) is the most serious
bottleneck– We run of time before we run out of space– Size of state space by itself is not a bottleneck
Experimental Results
26
-----------------------------------------------------------------------------------------------------UNCHANGED ACTIONS:UNCHANGED ACTIONS:-----------------------------------------------------------------------------------------------------1- (d.j==2) & !(f.j==1) & !(b.j==1) (d.j:=dg)-----------------------------------------------------------------------------------------------------REVISED ACTIONS:REVISED ACTIONS:-----------------------------------------------------------------------------------------------------2- (b.j==0) & (d.j==1) & (d.k==1) & (f.j==0) (f.j:=1)3- (b.j==0) & (d.j==0) & (d.l==0) & (f.j==0) (f.j:=1)4- (b.j==0) & (d.j==0) & (d.k==0) & (f.j==0) (f.j:=1)5- (b.j==0) & (d.j==1) & (d.l==1) & (f.j==0) (f.j:=1)-----------------------------------------------------------------------------------------------------NEW RECOVERY ACTIONS:NEW RECOVERY ACTIONS:-----------------------------------------------------------------------------------------------------6- (b.j==0) & (d.j==0) & (d.l==1) & (d.k==1) & (f.j==0) (d.j:=1)7- (b.j==0) & (d.j==1) & (d.l==0) & (d.k==0) & (f.j==0) (d.j:=0)8- (b.j==0) & (d.j==0) & (d.l==1) & (d.k==1) & (f.j==0) (d.j:=1), (f.j:=1)9- (b.j==0) & (d.j==1) & (d.l==0) & (d.k==0) & (f.j==0) (d.j:=0), (f.j:=1)------------------------------------------------------------------------------------------
Experimental Results
27
The effect of exploiting
human knowledgehuman knowledge(Each non-general process is allowed to finalize its decision if no two non-generals are undecided.)
Experimental Results
28
Experimental Results
Performance of synthesizing
token ringtoken ring mutual exclusion with
multi-step recovery
29
Experimental Results
Multi-step vs. single-step single-step
recoveryrecovery for synthesizing
token ring mutual exclusion
30November 7, 2008 Doctoral Dissertation Defense
Open Problems
• Exploiting techniques from model checking
– State space generation (e.g., clustering and partitioning)
– Symmetry reduction
– Counter-example guided abstraction-refinement (CEGAR)
– SMT/QBF-based methods
– Distributed/parallel techniques
31November 7, 2008 Doctoral Dissertation Defense
Open Problems
• Multidisciplinary research problems
– Revising hybrid systems
– Synthesizing programs with multiple concerns (e.g., security, communication, real-time, fault-tolerance, distribution) in epistemic logic
– Program synthesis using graph mining and machine learning techniques
– Biologically-inspired revision/synthesis techniques
THANK YOU!