Model Checking Large-Scale Software
Natasha Sharygina
Carnegie Mellon University
Software Engineering Institute
2
Outline
• Model Checking Software
• Integrated Design and Verification Approach
• Integrated State Space Reduction Approach
• Conclusions
• Directions for Future Work
3
Formal Verification by Model Checking
Continuously operating concurrent systems (e.g. operating systems, hardware controllers and network protocols)
• Ongoing, reactive semantics• Non-terminating, infinite computations• Manifest non-determinism
Temporal Logic Model checking [Clarke,Emerson 81][Queille,Sifakis 82]:• Formal model, M
(state transition system - finite state machines)
• Specification, P(temporal logic)
• Algorithm to check if M satisfies P, M |= P• exhaustive enumeration of all states reachable by the system • when the design fails to satisfy a desired property, a counterexample is
generated
4
State Space Explosion
Problem: Size of the state graph can be exponential in size of the program
text (both in the number of the program variables and the number of
program components)
M = M1 || … || Mn
If each Mi has just 2 local states, potentially 2n global states
Research Directions: State space reduction
Principal Approaches:
• Abstraction (elimination of details irrelevant to verification)
• Compositional reasoning (reasoning about parts of the system)
5
Application of Model Checking to Hardware Verification
• Simple data structures are used
• Systems are modular
• Mostly finite-state systems
• System components have well defined interfaces
• Mostly synchronous execution
6
Application of Model Checking to Software Verification
• Complex data structures are used
• Procedural or OO design
• Non-finite state systems
• System components do not have well defined interfaces
• Complex coordination between SW components
• Synchronous or asynchronous execution
7
Model Checking Software(current practice)
1. Design/Implementation/Testing
2. Modeling/Property Specification• Finite-state model extraction• Simplifications• Restrictions
3. Verification• Abstractions
• Divide-and-conquertechniques (when applicable)
Limitations• Final (expensive) stage in the program development • Consistency between code and model• Limited to simplified systems
M odel
N o
M odel C hecker
E rror track
C ode
Y es
P roperty
O U T of M /T
R e-design
8
Outline
• Model Checking Software
• Integrated Design and Verification Approach
• Integrated State Space Reduction Approach
• Conclusions
• Directions for Future Work
9
Model Checking Software (UT Austin – Bell Labs Project)
3. Verification
• State space reduction techniques
D e s ig n M o d e l
N o
M o d e l C h e c k e r
E r r o r tr a c k
C o d e
Y e s
P r o p e r ty
O U T o f M /T
F o r m a l M o d e l
T e s t in g T o o l
R e -d e s ig n
Advantages:• Earlier bug detection• Direct translation of informal program into formal syntax (no simplifications)• Separation of concerns: abstraction of control from data• Domain-specific property specification
4. Code Generation (last stage)
1. Executable Design Specifications
•Abstraction from low-level to
high-level operations
2. Modeling/Property Specification•Finite-state model extraction
10
xUML: An Executable OO Modeling Language
Design Specification
Class Information
Diagrams
Class Collaboration
State Transition
Static model
Subsystem Diagrams
Dynamic model
xUML Model
Code
Code Generation
Requirements
Diagrams
Diagrams
• Executable dialect of UML
• A system consists of interacting sequential programs (class instances)
• System and class hierarchies
• Class instances communicate mainly through asynchronous event passing with buffering
• Behavior of class instances is defined as State Models
• State Models are extended with state actions• Each action is run-to-completion• State transitions are enabled by events
11
Executable OO Modeling Language(textual xUML)
An xUML system is a parallel composition of individual sequential program,
P = p1 || … || pn,
Sample commands of xUML programs:
Assignments: x: = exp | x := any{exp1 , …, expn}
Communication: : Generate ei(ID,exp) - Event generation
Receive ei(ID,x) - Event consumption
Compounds: if then else, while do od, switch
13
Model Checking xUML Programs(UT-Bell Labs Project)
xUML Model
xUML Query
xUML-to-S/R Translation
S/R Model S/R Query
Model Checking with COSPAN Model Checker
S/R Query COSPAN Error Track
Error Report Generation
xUML Level Error Report
Data Process Input OutputLegend:
14
COSPAN Model Checker and S/R Automaton Language
• COSPAN is an - automata-based model checker and input models and queries are formulated in S/R
• In S/R, a system is a synchronous parallel composition of processes
15
xUML-to-S/R Model Translation
•Maps xUML class instances to S/R processes
•Models asynchrony with synchrony:
–An S/R process as global execution scheduler
–Message buffers by separate S/R processes
•Simulates dynamic creation of class instances
•Bounds infinite state spaces of xUML models
•Reuses modules from SDLCheck [Kurshan, Levin’01], such as optimization modules that conduct Static Partial Order Reduction
16
xUML Level Query Formulation
• DECLARE Joint_2_in_Move_EE <<Joint 2>> $Move_EE;• DECLARE Recovery_Called <<Recovery 1>>
recovery_status = 1;
• NEVER (Joint_2_in_Move_EE AND Recovery_Called);
Proposition
Instantiation of Temporal Template
Semantic Constructs of xUML Model
17
Demonstration
Property Specification Interface xUML IDE Error Visualizer
xUML-to-S/R Translator Error Report Generator
COSPAN Model Checker
S/R ModelS/R Query
Error Report
Error Track
Designer
xUML ModelProperty
18
Step-by-Step Demonstration
Designer
Property Specification Interface xUML IDE Error Visualizer
Error ReportxUML ModelProperty
xUML-to-S/R Translator Error Report Generator
Error TrackS/R ModelS/R Query
COSPAN Model Checker
25
Step-by-Step Demonstration
Designer
Property Specification Interface xUML IDE Error Visualizer
Error ReportxUML ModelProperty
xUML-to-S/R Translator Error Report Generator
Error TrackS/R ModelS/R Query
COSPAN Model Checker
27
Step-by-Step Demonstration
Designer
Property Specification Interface xUML IDE Error Visualizer
Error ReportxUML ModelProperty
xUML-to-S/R Translator Error Report Generator
Error TrackS/R ModelS/R Query
COSPAN Model Checker
30
Step-by-Step Demonstration
Designer
Property Specification Interface xUML IDE Error Visualizer
Error ReportxUML ModelProperty
xUML-to-S/R Translator Error Report Generator
Error TrackS/R ModelS/R Query
COSPAN Model Checker
32
Step-by-Step Demonstration
Designer
Property Specification Interface xUML IDE Error Visualizer
Error ReportxUML ModelProperty
xUML-to-S/R Translator Error Report Generator
Error TrackS/R ModelS/R Query
COSPAN Model Checker
34
Step-by-Step Demonstration
Designer
Property Specification Interface xUML IDE Error Visualizer
Error ReportxUML ModelProperty
xUML-to-S/R Translator Error Report Generator
Error TrackS/R ModelS/R Query
COSPAN Model Checker
38
NASA Robot Controller System
vEEFD y n a m ics
Fo rcesT o rq u esIn e rtia
C riter ia C o m p lia n ce
W
O p era tio n a l S o ftw a reC o m p o n en ts
T o S im u la tio n
K in em a tic s
R ea l-T im e C o n tro lC o m p o n en ts
P e r f o r m a n c e
A c tu a to r C o n t r o l
R eso u rceA llo ca tio n
O p era to r P r io r ity S e ttin g
39
NASA Robot Controller System
• Actual robot control system deployed in space (Canadian arm)• Complex functionality (kinematics control, obstacle avoidance, fault tolerance,
performance evaluation, etc.)• Complex coordination (hand shaking) between functional units • Stringent reliability requirements• Distributed control
• xUML framework for design and validation of robot control algorithms • Discrete real-time modeling• ~800K LOC of textual xUML• Abstract interfaces to the computational libraries
• Computational libraries (C++)• ~300K LOC, 150+ classes, 500+ methods of C++
• 34 functional properties, 20 implementation properties
• 6 logical errors were found and reported to NASA
vEEF
40
Class Information Model of the Robot Controller Systems
Joint motion is controlled by Controller
is checked by
consists of
defines position of
specifies
is a part of
R15
Computational Subsystem
….
R1
R11
Follows to the specified
R2
R5
R4
Inverse Kinematics . new_angle
R7
R6
TrajectoryDB(T) * TJ_ID . position . ee_reference . final_position . obstacle . EE_ID(R3)
TrajectoryPoint(TP) *TP_ID . TJ_ID (R4) . point
R3
R9
R10
has
can have different
can have a number of
Is A
is a component of
requires
OSCAR Interfaces
* OS_ID . TC_ID (R10) . status * link-ID
R8
Forward Kinematics . position
Arm (A) * Arm_ID . arm_status . dof
is configured by
End-Effector (EF) * EE_ID . Current position . Limit_max . Limit_min . status . obstacle_avoidance . end_position . undesired_position . forward_kinematics
JointConfiguration (JC) *JC_ID . Joint_ID(R7) . direction . trial_angle . type . NoinSet . TS_counter . Status
3
Joint (J) * Joint ID . current_angle . limit_max . limit_min . EE_ID (R2) . reference . Arm_ID (R1) . O_ID (R11) . joint_status
is interfaced with
TrialConfiguration (TC) *TC_ID . DT_ID(R8) . configuration . validSolutions . SS_ID(R13) . status . LockConfiguraion . TP_ID(R5) . JC_ID(R6)
Is evaluated by DecisionTree
R13 Is a component of Search Space
R14
Checker (Ch) * CH_ID . counter . recovery_status . recovery_counter
Controller * C_ID . DT_ID (R11) . status . abort_var
Control is optimized by the DecisionTree
R16
Optimization Subsystem
R15
R14 = R15+R16
Is A
R19
Performance Criterion (PC)
* PC ID . average value . status . FC_ID (R16) . scale . PC_name .TS_ID (R15)
Kinetic energy distortion Criterion
* PC ID (R19) *Criteria name . criterion value
System Compliance Criterion
* PC ID (R19) *Criteria name . criterion value
Inertia Criterion * PC ID (R19) *Criteria name . criterion value
Geometric Criterion * PC ID (R19) *Criteria name .criterion value
Constraint Criterion * PC ID (R19) * Criteria name . Error . criterion value
R17
DecisionTree(DT) *DT_ID . optimal_solution . status
R12
R13
R18
has
has
has
creates
Search Space (SS) * SS_ID . SS_size . DT_ID (R12)
FactorialSearch Space (FSS) * FSS_ID . SS_ID (R17) . condition
9
SimpleSearch Space(SSS) * SSS_ID . SS_ID (R18) . condition
9
Trial configuration is characterized by
Fused Criterion (FC) * FC ID . PI . TS_ID (R14)
Consists of a number of trial configurations
Evaluates a trial configuration
R11 Controls motion of each joint
R8
Is used for evaluation of a trial configuration
41
xUML Modeling of the NASA Robot Controller System
MovingJoints
stopped
Valid Not_Valid
A1:Valid(Arm_ID)
A3:toNotValidState(Arm_ID)
A2: NotValidConfiguration(Arm_ID)
A4:toValidState(Arm_ID)
A5:stop(Arm_ID)
abort_var=1;
Foreach Joint{ Generate J1:Configure(Joint(Joint_ID).Joint_ID);} arm_status=0;
A6:terminate(Arm_ID)
arm_status=0;
arm_status=1;
EndEffector xUML programArm xUML program
Idle
Checking constraints
Initial positioning
Following Desired
Trajectory
EE2: CheckLimits(EE_ID)
EE3: BacktoIdle(EE_ID)
EE6: MoveEndEffector(EE_ID)
EE5: back(EE_ID)
EE4: CheckConstraints(EE_ID)
ee_reference=0; end_position=0;
ee_reference=1; ….. if(Current_position>=final_point) end_position=1; ……
For (int i=0;i<6;i++){ if (Current_position[i]>Limit[i]{ End_position=1;}} ……
||
42
Examples of the Robot Control Properties
• Control Termination: Eventually the robot control terminates
EventuallyAlways(abort_var=1)
• Safety Operation: If the EndEffector reaches an undesired position than the program terminates prior to a new move of the EndEffector
AfterAlwaysUntil(undesired_position =1,ee_reference=1,abort_var=1)
• Configuration Validity Check: If an instance of the EndEffector class is in the “FollowingDesiredTrajectory” state than the instance of the corresponding Arm class is in the ‘Valid” state
Always((ee_reference=1) ->(arm_status=1))
• Proper Program Termination: The program terminates when it either completes the task or reaches the state where there is no solution for the fault recovery
AlwaysUntil(abort_var=0,end_position=1 OR (recovery_status=1 AND number_joints=1))
43
Outline
• Model Checking Software
• Integrated Design and Verification Approach
• Integrated State Space Reduction Approach
• Conclusions
• Directions for Future Work
44
Efficient Model Checking of xUML Programs
State Space Reduction Approach:
– Exploit design modularity for modular verification
– Reduce complexity of the program prior to generation of state graph (by syntactic analysis and program transformation)
– Use domain specific knowledge to derive abstractions and other reduction techniques
– Use existing state space reduction techniques
45
Integrated State Space Reduction Approach
xUML Model (Validated Designs)
Syntactic Analysis
Domain-Specific Bounding
xUML Component
Design for Verifiability
xUML Component
xUML
Component
xUML2SR
SR Model
xUML2SR + Partial-order
Reduction
Assume-Guarantee Reasoning
Data Abstraction (Predicate Abstraction)
Symbolic Verification
Syntactic Analysis
transformation
Data Abstraction (Loop Abstraction)
debug
Methodological Approaches
Algorithmic Approaches
46
Assume-Guarantee Compositional Reasoning
In the assume-guarantee paradigm: each component guarantees properties based on assumptions about other components via proof rules
M1 || T2 |= T1M2 || T1 |= T2
* Circularity during verification of different blocks is broken by induction over time
[AbadiLamport95][AlurHenzinger96][HenzingerQadeerRajamani99][Kurshan94][McMillan98][Stark85]
A trace is a sequence of states
- Composition of FSM (||) is the intersection of traces
- Consistency check (|=) is the trace containment check
M1 || M2 |= T1 || T2*
Constraining environment
47
Scalability of Multi-Process Model Checking
Limitation: Properties when used as constraints commonly do not give sufficient details about the verifiable processes
Approach: Add abstraction constraints
Abstraction constraints (cf. Kurshan 94, Henzinger 98) are added to the property specifications:
M1 || T2 |= T1 || T1abs || …|| Tn
abs
T1abs , …, Tk
abs – abstraction constraints
Abstraction constraints specification: - temporal logic formulae constraining the external variables of the verifiable processes - translation of the formulae to processes
48
LimitationsObservation:Conventional software structure and behaviors preclude system decomposition and applicability of assume/guarantee reasoning
Hardware (Spatial Modularity):• Natural division into components
• System components have well defined interfaces• Precisely defined communication protocols
Software (Not Spatially Modular):• Monolithic designs (even for OO programs)
• Difficulties in property decomposition
• No clean interfaces• Complex interaction between SW elements
• Difficulties in identifying environment assumptions• Inability to specify meaningful abstraction constraints
49
Design for Verification
Approach Enforce software design discipline: • apply design constraints analogous to the physical constraints of hardware designs
Principle 1: Perform encapsulation at the component level (not an object)
Principle 2: Design software components to spatial modularity
What to do?
• Make state spaces of components rigorously disjoint• Channel communication through precisely defined interfaces• Specify precise communication protocols
50
Assume/Guarantee Reasoning of Software Spatial Designs
T4 T2
T3
Component 3: Design model
GATE
Component 1: formal model
T1
|= ?
Component 1: Design model
Component 4: Design model
Component 2: Design model
GATE GATE GATE
Property decomposition:Temporal logic specification
M1 || T2 || T2 || T3 |= T1
Environment constraints
51
Assume/Guarantee Reasoning of Software Spatial Designs
Component 2: Design model
GATE
T2abs Component 1: formal model, M1
T1
|= ?
Component 2: formal model, M2
T2
|= ?
Component 1: Design model
GATE
T1abs
Abstraction Constraints
High-level environment specs
environment constraints
abstraction constraints
M1 || M2 |= T1 || T2
M1 || T2 |= T1
M2 || T1 |= T2
|| T2abs || T1abs
|| T2abs || T1abs
52
Verification Statistics: Robot Controller System
i(M) Monolithic verification, i – number of joints
i(C) Compositional verification of Kinematics component
Platform: HP9000 (440MHz) with 6144 MB RAM
1.00E+061.00E+081.00E+101.00E+121.00E+141.00E+161.00E+181.00E+201.00E+221.00E+24
ControlTermination
SafetyOperation
2(M)
2(C)
3(M)
3(C)
4(M)
4(C)
Memory/Timeexhaustion
Memory, B
Models
53
Abstractions in Model Checking
Data Abstraction (abstraction of details irrelevant to verification of a property)
Approach: Mabs T Mconc T
To Prove: soundness and completeness
Systematic Construction of Abstractions (Predicate Abstraction) [Saidi,Graf 97] :• Define an abstraction function as a predicate over concrete data • Specify decision procedures to compute a set of abstraction predicates• Demonstrate the soundness and completeness of the abstraction
Refinement–based abstraction [MSR SLAM Project],[Clarke et. al. ‘00], [Saidi 00],
[Visser et. al 00]
54
Abstractions in Model Checking:Limitations
Abstraction might be expensive! • Complex computation procedures to define a set of the abstraction predicates
• Memory exhaustion during computation of predicates for large systems and complex properties
• Behavioral over-approximations caused by the abstraction might lead to the state space explosion
Can we do better?• Abstract and verify selectively
• Example: Abstract with respect to a subset of properties (not a single property)• Simplify decision procedures
• Example: Eliminate some predicates from consideration
55
Model Checking Programs with Loops
Control intensive systems typically execute loops of control functions.
Each decision point in the loop typically depends on a small number of variables and each possible branch is commonly taken for a wide range of data values of these variables.
Let Xcontrol X be the control effect variables (variables that determine the control flow).
The global state transition graph incorporates all of the loops for all values of the control effect variables.
ek
ej
ej ej ej ej ej ei
en
e1
Control Node (node that receives an event)
Control Loop
Data Node
56
Example of a Program with Loops
MovingJoints
stopped
Valid Not_Valid
A1:Valid(Arm_ID)
A3:toNotValidState(Arm_ID)
A2: NotValidConfiguration(Arm_ID)
A4:toValidState(Arm_ID)
A5:stop(Arm_ID)
abort_var=1;
Foreach Joint{ Generate J1:Configure(Joint(Joint_ID).Joint_ID);} arm_status=0;
A6:terminate(Arm_ID)
arm_status=0;
arm_status=1;
EndEffector xUML program Arm xUML program
Idle
Checking constraints
Initial positioning
Following Desired
Trajectory
EE2: CheckLimits(EE_ID)
EE3: BacktoIdle(EE_ID)
EE6: MoveEndEffector(EE_ID)
EE5: back(EE_ID)
EE4: CheckConstraints(EE_ID)
ee_reference=0; end_position=0;
ee_reference=1; LOOP MARKER ….. if(*Current_position>=final_point) end_position=1; ……
For (int i=0;i<6;i++){ if (Current_position[i]>Limit[i]{ End_position=1;}} ……
vEEF
||
57
Control Abstraction
Goal: Verify Control Properties (properties defined over states that input events) of programs with loops
Approach: Abstract execution of loops from valuation of data
- Construct an abstract program which contains all of the control flow paths in the concrete program
Atomicity of Program Structure:
Basic Block is a sequence of statements which can
be entered by an event only at the head statement
and which always run to completion.
statements
e2 e3
e1
e2
e4 e1
e3
e4
e3
e5
e1
Event queue of process I
Event queue of process J
Event queue of process K
58
Key Ideas
1. Exploit atomicity of the program structure (not predicates) to identify control flow sequences
2. Compute output ranges of each basic block that has loop predicates
3. Transform control flow predicates of each basic block into a multi-way selector
expression (non-deterministic choice) that controls all possible sequences of events.
Each output of an abstract basic block is controlled by a single value
4. Perform dependency analysis and source-to-source transformation
59
Loop Abstraction for Model Checking of Control Properties
TRUE
OB_xUML Model (graphical designs)
OB_xUML Model (textual file)
SR Model
Refinements Constraints
Fairness Constraints
CONTROL PROPERTY
AUTOMATA
Abstraction Tool (Loop_Abstraction)
Loop Detection
Labeling
Translation GOB2TOB
Transformation
Syntactic Analysis
Formalization and Translation TOB2SR
Re-design
Consistency Check
Refinement
FALSE NEGATIVE
REAL ERROR
Testing Tool (OB Simulator)
Model Checker (COSPAN)
60
Verification Statistics: Robot Controller
1.00E+041.00E+061.00E+081.00E+101.00E+121.00E+141.00E+161.00E+181.00E+201.00E+221.00E+24
ControlTermination
SafetyCoordination
2(C)
2(A)
3(C)
3(A)
4(C)
4(A)
5(C)
5(A)
6(C)
6(A)
Memory/Timeexhaustion
6(C)5(C)
i(C) Concrete program, i – number of joints
i(A) Abstract program
Platform: HP9000 (440MHz) with 6144 MB RAM
Memory
Models
61
Lessons Learned• Integration design and verification environment enables verification of non-
trivial software systems
– Translation of software into a formal language of model checkers is simplified
– Model checker is used as a debugging tool during software design
– Software design management enhances the applicability of the existing state space reduction techniques
– Testing of design-level specifications is helpful for specification of useful abstractions
• Integrated state space reduction supports verifying larger systems
– State space reduction techniques conducted by the syntactic program transformation are applicable without a change to the model checking tools and can be combined with the existing reduction techniques
– Effectiveness of reduction algorithms depends on interaction (combination, order of application, etc.) among reduction algorithms
62
Some Ideas for Future Work
• Integration of abstraction techniques• Combined abstraction approach: syntactic program transformation and reductions of the state
transition graphs• Systematic design and analysis of refinement procedures for design-level specifications
• Predictable component assembly• Abstraction techniques to check consistency of the component interfaces
• Integration of model checking and testing techniques• Model checking to support interactive simulation during analysis of errors
- To track multiple processes- To allow analysis backwards and forward during the execution
• Testing of data dependent parts of the code which can not be handled by finite-state model checking techniques
• Engineering of software systems to enable symbolic verification to be efficient