model checking large-scale software natasha sharygina carnegie mellon university software...

Model Checking Large-Scale Software

Natasha Sharygina

Carnegie Mellon University

Software Engineering Institute

2

Outline

• Model Checking Software

• Integrated Design and Verification Approach

• Integrated State Space Reduction Approach

• Conclusions

• Directions for Future Work

3

Formal Verification by Model Checking

Continuously operating concurrent systems (e.g. operating systems, hardware controllers and network protocols)

• Ongoing, reactive semantics• Non-terminating, infinite computations• Manifest non-determinism

Temporal Logic Model checking [Clarke,Emerson 81][Queille,Sifakis 82]:• Formal model, M

(state transition system - finite state machines)

• Specification, P(temporal logic)

• Algorithm to check if M satisfies P, M |= P• exhaustive enumeration of all states reachable by the system • when the design fails to satisfy a desired property, a counterexample is

generated

4

State Space Explosion

Problem: Size of the state graph can be exponential in size of the program

text (both in the number of the program variables and the number of

program components)

M = M1 || … || Mn

If each Mi has just 2 local states, potentially 2n global states

Research Directions: State space reduction

Principal Approaches:

• Abstraction (elimination of details irrelevant to verification)

• Compositional reasoning (reasoning about parts of the system)

5

Application of Model Checking to Hardware Verification

• Simple data structures are used

• Systems are modular

• Mostly finite-state systems

• System components have well defined interfaces

• Mostly synchronous execution

6

Application of Model Checking to Software Verification

• Complex data structures are used

• Procedural or OO design

• Non-finite state systems

• System components do not have well defined interfaces

• Complex coordination between SW components

• Synchronous or asynchronous execution

7

Model Checking Software(current practice)

1. Design/Implementation/Testing

2. Modeling/Property Specification• Finite-state model extraction• Simplifications• Restrictions

3. Verification• Abstractions

• Divide-and-conquertechniques (when applicable)

Limitations• Final (expensive) stage in the program development • Consistency between code and model• Limited to simplified systems

M odel

N o

M odel C hecker

E rror track

C ode

Y es

P roperty

O U T of M /T

R e-design

8

Outline




• Conclusions


9

Model Checking Software (UT Austin – Bell Labs Project)

3. Verification

• State space reduction techniques

D e s ig n M o d e l

N o

M o d e l C h e c k e r

E r r o r tr a c k

C o d e

Y e s

P r o p e r ty

O U T o f M /T

F o r m a l M o d e l

T e s t in g T o o l

R e -d e s ig n

Advantages:• Earlier bug detection• Direct translation of informal program into formal syntax (no simplifications)• Separation of concerns: abstraction of control from data• Domain-specific property specification

4. Code Generation (last stage)

1. Executable Design Specifications

•Abstraction from low-level to

high-level operations

2. Modeling/Property Specification•Finite-state model extraction

10

xUML: An Executable OO Modeling Language

Design Specification

Class Information

Diagrams

Class Collaboration

State Transition

Static model

Subsystem Diagrams

Dynamic model

xUML Model

Code

Code Generation

Requirements

Diagrams

Diagrams

• Executable dialect of UML

• A system consists of interacting sequential programs (class instances)

• System and class hierarchies

• Class instances communicate mainly through asynchronous event passing with buffering

• Behavior of class instances is defined as State Models

• State Models are extended with state actions• Each action is run-to-completion• State transitions are enabled by events

11

Executable OO Modeling Language(textual xUML)

An xUML system is a parallel composition of individual sequential program,

P = p1 || … || pn,

Sample commands of xUML programs:

Assignments: x: = exp | x := any{exp1 , …, expn}

Communication: : Generate ei(ID,exp) - Event generation

Receive ei(ID,x) - Event consumption

Compounds: if then else, while do od, switch

12

A Sample xUML State Model

State Action

State

State Transition

Message Type

13

Model Checking xUML Programs(UT-Bell Labs Project)

xUML Model

xUML Query

xUML-to-S/R Translation

S/R Model S/R Query

Model Checking with COSPAN Model Checker

S/R Query COSPAN Error Track

Error Report Generation

xUML Level Error Report

Data Process Input OutputLegend:

14

COSPAN Model Checker and S/R Automaton Language

• COSPAN is an - automata-based model checker and input models and queries are formulated in S/R

• In S/R, a system is a synchronous parallel composition of processes

15

xUML-to-S/R Model Translation

•Maps xUML class instances to S/R processes

•Models asynchrony with synchrony:

–An S/R process as global execution scheduler

–Message buffers by separate S/R processes

•Simulates dynamic creation of class instances

•Bounds infinite state spaces of xUML models

•Reuses modules from SDLCheck [Kurshan, Levin’01], such as optimization modules that conduct Static Partial Order Reduction

16

xUML Level Query Formulation

• DECLARE Joint_2_in_Move_EE <<Joint 2>> $Move_EE;• DECLARE Recovery_Called <<Recovery 1>>

recovery_status = 1;

• NEVER (Joint_2_in_Move_EE AND Recovery_Called);

Proposition

Instantiation of Temporal Template

Semantic Constructs of xUML Model

17

Demonstration

Property Specification Interface xUML IDE Error Visualizer

xUML-to-S/R Translator Error Report Generator

COSPAN Model Checker

S/R ModelS/R Query

Error Report

Error Track

Designer

xUML ModelProperty

18

Step-by-Step Demonstration

Designer


Error ReportxUML ModelProperty


Error TrackS/R ModelS/R Query


25


Designer






27


Designer






30


Designer






32


Designer






34


Designer






38

NASA Robot Controller System

vEEFD y n a m ics

Fo rcesT o rq u esIn e rtia

C riter ia C o m p lia n ce

W

O p era tio n a l S o ftw a reC o m p o n en ts

T o S im u la tio n

K in em a tic s

R ea l-T im e C o n tro lC o m p o n en ts

P e r f o r m a n c e

A c tu a to r C o n t r o l

R eso u rceA llo ca tio n

O p era to r P r io r ity S e ttin g

39

NASA Robot Controller System

• Actual robot control system deployed in space (Canadian arm)• Complex functionality (kinematics control, obstacle avoidance, fault tolerance,

performance evaluation, etc.)• Complex coordination (hand shaking) between functional units • Stringent reliability requirements• Distributed control

• xUML framework for design and validation of robot control algorithms • Discrete real-time modeling• ~800K LOC of textual xUML• Abstract interfaces to the computational libraries

• Computational libraries (C++)• ~300K LOC, 150+ classes, 500+ methods of C++

• 34 functional properties, 20 implementation properties

• 6 logical errors were found and reported to NASA

vEEF

40

Class Information Model of the Robot Controller Systems

Joint motion is controlled by Controller

is checked by

consists of

defines position of

specifies

is a part of

R15

Computational Subsystem

….

R1

R11

Follows to the specified

R2

R5

R4

Inverse Kinematics . new_angle

R7

R6

TrajectoryDB(T) * TJ_ID . position . ee_reference . final_position . obstacle . EE_ID(R3)

TrajectoryPoint(TP) *TP_ID . TJ_ID (R4) . point

R3

R9

R10

has

can have different

can have a number of

Is A

is a component of

requires

OSCAR Interfaces

* OS_ID . TC_ID (R10) . status * link-ID

R8

Forward Kinematics . position

Arm (A) * Arm_ID . arm_status . dof

is configured by

End-Effector (EF) * EE_ID . Current position . Limit_max . Limit_min . status . obstacle_avoidance . end_position . undesired_position . forward_kinematics

JointConfiguration (JC) *JC_ID . Joint_ID(R7) . direction . trial_angle . type . NoinSet . TS_counter . Status

3

Joint (J) * Joint ID . current_angle . limit_max . limit_min . EE_ID (R2) . reference . Arm_ID (R1) . O_ID (R11) . joint_status

is interfaced with

TrialConfiguration (TC) *TC_ID . DT_ID(R8) . configuration . validSolutions . SS_ID(R13) . status . LockConfiguraion . TP_ID(R5) . JC_ID(R6)

Is evaluated by DecisionTree

R13 Is a component of Search Space

R14

Checker (Ch) * CH_ID . counter . recovery_status . recovery_counter

Controller * C_ID . DT_ID (R11) . status . abort_var

Control is optimized by the DecisionTree

R16

Optimization Subsystem

R15

R14 = R15+R16

Is A

R19

Performance Criterion (PC)

* PC ID . average value . status . FC_ID (R16) . scale . PC_name .TS_ID (R15)

Kinetic energy distortion Criterion

* PC ID (R19) *Criteria name . criterion value

System Compliance Criterion

* PC ID (R19) *Criteria name . criterion value

Inertia Criterion * PC ID (R19) *Criteria name . criterion value

Geometric Criterion * PC ID (R19) *Criteria name .criterion value

Constraint Criterion * PC ID (R19) * Criteria name . Error . criterion value

R17

DecisionTree(DT) *DT_ID . optimal_solution . status

R12

R13

R18

has

has

has

creates

Search Space (SS) * SS_ID . SS_size . DT_ID (R12)

FactorialSearch Space (FSS) * FSS_ID . SS_ID (R17) . condition

9

SimpleSearch Space(SSS) * SSS_ID . SS_ID (R18) . condition

9

Trial configuration is characterized by

Fused Criterion (FC) * FC ID . PI . TS_ID (R14)

Consists of a number of trial configurations

Evaluates a trial configuration

R11 Controls motion of each joint

R8

Is used for evaluation of a trial configuration

41

xUML Modeling of the NASA Robot Controller System

MovingJoints

stopped

Valid Not_Valid

A1:Valid(Arm_ID)

A3:toNotValidState(Arm_ID)

A2: NotValidConfiguration(Arm_ID)

A4:toValidState(Arm_ID)

A5:stop(Arm_ID)

abort_var=1;

Foreach Joint{ Generate J1:Configure(Joint(Joint_ID).Joint_ID);} arm_status=0;

A6:terminate(Arm_ID)

arm_status=0;

arm_status=1;

EndEffector xUML programArm xUML program

Idle

Checking constraints

Initial positioning

Following Desired

Trajectory

EE2: CheckLimits(EE_ID)

EE3: BacktoIdle(EE_ID)

EE6: MoveEndEffector(EE_ID)

EE5: back(EE_ID)

EE4: CheckConstraints(EE_ID)

ee_reference=0; end_position=0;

ee_reference=1; ….. if(Current_position>=final_point) end_position=1; ……

For (int i=0;i<6;i++){ if (Current_position[i]>Limit[i]{ End_position=1;}} ……

||

42

Examples of the Robot Control Properties

• Control Termination: Eventually the robot control terminates

EventuallyAlways(abort_var=1)

• Safety Operation: If the EndEffector reaches an undesired position than the program terminates prior to a new move of the EndEffector

AfterAlwaysUntil(undesired_position =1,ee_reference=1,abort_var=1)

• Configuration Validity Check: If an instance of the EndEffector class is in the “FollowingDesiredTrajectory” state than the instance of the corresponding Arm class is in the ‘Valid” state

Always((ee_reference=1) ->(arm_status=1))

• Proper Program Termination: The program terminates when it either completes the task or reaches the state where there is no solution for the fault recovery

AlwaysUntil(abort_var=0,end_position=1 OR (recovery_status=1 AND number_joints=1))

43

Outline




• Conclusions


44

Efficient Model Checking of xUML Programs

State Space Reduction Approach:

– Exploit design modularity for modular verification

– Reduce complexity of the program prior to generation of state graph (by syntactic analysis and program transformation)

– Use domain specific knowledge to derive abstractions and other reduction techniques

– Use existing state space reduction techniques

45

Integrated State Space Reduction Approach

xUML Model (Validated Designs)

Syntactic Analysis

Domain-Specific Bounding

xUML Component

Design for Verifiability

xUML Component

xUML

Component

xUML2SR

SR Model

xUML2SR + Partial-order

Reduction

Assume-Guarantee Reasoning

Data Abstraction (Predicate Abstraction)

Symbolic Verification

Syntactic Analysis

transformation

Data Abstraction (Loop Abstraction)

debug

Methodological Approaches

Algorithmic Approaches

46

Assume-Guarantee Compositional Reasoning

In the assume-guarantee paradigm: each component guarantees properties based on assumptions about other components via proof rules

M1 || T2 |= T1M2 || T1 |= T2

* Circularity during verification of different blocks is broken by induction over time

[AbadiLamport95][AlurHenzinger96][HenzingerQadeerRajamani99][Kurshan94][McMillan98][Stark85]

A trace is a sequence of states

- Composition of FSM (||) is the intersection of traces

- Consistency check (|=) is the trace containment check

M1 || M2 |= T1 || T2*

Constraining environment

47

Scalability of Multi-Process Model Checking

Limitation: Properties when used as constraints commonly do not give sufficient details about the verifiable processes

Approach: Add abstraction constraints

Abstraction constraints (cf. Kurshan 94, Henzinger 98) are added to the property specifications:

M1 || T2 |= T1 || T1abs || …|| Tn

abs

T1abs , …, Tk

abs – abstraction constraints

Abstraction constraints specification: - temporal logic formulae constraining the external variables of the verifiable processes - translation of the formulae to processes

48

LimitationsObservation:Conventional software structure and behaviors preclude system decomposition and applicability of assume/guarantee reasoning

Hardware (Spatial Modularity):• Natural division into components

• System components have well defined interfaces• Precisely defined communication protocols

Software (Not Spatially Modular):• Monolithic designs (even for OO programs)

• Difficulties in property decomposition

• No clean interfaces• Complex interaction between SW elements

• Difficulties in identifying environment assumptions• Inability to specify meaningful abstraction constraints

49

Design for Verification

Approach Enforce software design discipline: • apply design constraints analogous to the physical constraints of hardware designs

Principle 1: Perform encapsulation at the component level (not an object)

Principle 2: Design software components to spatial modularity

What to do?

• Make state spaces of components rigorously disjoint• Channel communication through precisely defined interfaces• Specify precise communication protocols

50

Assume/Guarantee Reasoning of Software Spatial Designs

T4 T2

T3

Component 3: Design model

GATE

Component 1: formal model

T1

|= ?




GATE GATE GATE

Property decomposition:Temporal logic specification

M1 || T2 || T2 || T3 |= T1

Environment constraints

51

Assume/Guarantee Reasoning of Software Spatial Designs


GATE

T2abs Component 1: formal model, M1

T1

|= ?

Component 2: formal model, M2

T2

|= ?


GATE

T1abs

Abstraction Constraints

High-level environment specs

environment constraints

abstraction constraints

M1 || M2 |= T1 || T2

M1 || T2 |= T1

M2 || T1 |= T2

|| T2abs || T1abs

|| T2abs || T1abs

52

Verification Statistics: Robot Controller System

i(M) Monolithic verification, i – number of joints

i(C) Compositional verification of Kinematics component

Platform: HP9000 (440MHz) with 6144 MB RAM

1.00E+061.00E+081.00E+101.00E+121.00E+141.00E+161.00E+181.00E+201.00E+221.00E+24

ControlTermination

SafetyOperation

2(M)

2(C)

3(M)

3(C)

4(M)

4(C)

Memory/Timeexhaustion

Memory, B

Models

53

Abstractions in Model Checking

Data Abstraction (abstraction of details irrelevant to verification of a property)

Approach: Mabs T Mconc T

To Prove: soundness and completeness

Systematic Construction of Abstractions (Predicate Abstraction) [Saidi,Graf 97] :• Define an abstraction function as a predicate over concrete data • Specify decision procedures to compute a set of abstraction predicates• Demonstrate the soundness and completeness of the abstraction

Refinement–based abstraction [MSR SLAM Project],[Clarke et. al. ‘00], [Saidi 00],

[Visser et. al 00]

54

Abstractions in Model Checking:Limitations

Abstraction might be expensive! • Complex computation procedures to define a set of the abstraction predicates

• Memory exhaustion during computation of predicates for large systems and complex properties

• Behavioral over-approximations caused by the abstraction might lead to the state space explosion

Can we do better?• Abstract and verify selectively

• Example: Abstract with respect to a subset of properties (not a single property)• Simplify decision procedures

• Example: Eliminate some predicates from consideration

55

Model Checking Programs with Loops

Control intensive systems typically execute loops of control functions.

Each decision point in the loop typically depends on a small number of variables and each possible branch is commonly taken for a wide range of data values of these variables.

Let Xcontrol X be the control effect variables (variables that determine the control flow).

The global state transition graph incorporates all of the loops for all values of the control effect variables.

ek

ej

ej ej ej ej ej ei

en

e1

Control Node (node that receives an event)

Control Loop

Data Node

56

Example of a Program with Loops

MovingJoints

stopped

Valid Not_Valid

A1:Valid(Arm_ID)

A3:toNotValidState(Arm_ID)

A2: NotValidConfiguration(Arm_ID)

A4:toValidState(Arm_ID)

A5:stop(Arm_ID)

abort_var=1;

Foreach Joint{ Generate J1:Configure(Joint(Joint_ID).Joint_ID);} arm_status=0;

A6:terminate(Arm_ID)

arm_status=0;

arm_status=1;

EndEffector xUML program Arm xUML program

Idle

Checking constraints

Initial positioning

Following Desired

Trajectory

EE2: CheckLimits(EE_ID)

EE3: BacktoIdle(EE_ID)

EE6: MoveEndEffector(EE_ID)

EE5: back(EE_ID)

EE4: CheckConstraints(EE_ID)

ee_reference=0; end_position=0;

ee_reference=1; LOOP MARKER ….. if(*Current_position>=final_point) end_position=1; ……

For (int i=0;i<6;i++){ if (Current_position[i]>Limit[i]{ End_position=1;}} ……

vEEF

||

57

Control Abstraction

Goal: Verify Control Properties (properties defined over states that input events) of programs with loops

Approach: Abstract execution of loops from valuation of data

- Construct an abstract program which contains all of the control flow paths in the concrete program

Atomicity of Program Structure:

Basic Block is a sequence of statements which can

be entered by an event only at the head statement

and which always run to completion.

statements

e2 e3

e1

e2

e4 e1

e3

e4

e3

e5

e1

Event queue of process I

Event queue of process J

Event queue of process K

58

Key Ideas

1. Exploit atomicity of the program structure (not predicates) to identify control flow sequences

2. Compute output ranges of each basic block that has loop predicates

3. Transform control flow predicates of each basic block into a multi-way selector

expression (non-deterministic choice) that controls all possible sequences of events.

Each output of an abstract basic block is controlled by a single value

4. Perform dependency analysis and source-to-source transformation

59

Loop Abstraction for Model Checking of Control Properties

TRUE

OB_xUML Model (graphical designs)

OB_xUML Model (textual file)

SR Model

Refinements Constraints

Fairness Constraints

CONTROL PROPERTY

AUTOMATA

Abstraction Tool (Loop_Abstraction)

Loop Detection

Labeling

Translation GOB2TOB

Transformation

Syntactic Analysis

Formalization and Translation TOB2SR

Re-design

Consistency Check

Refinement

FALSE NEGATIVE

REAL ERROR

Testing Tool (OB Simulator)

Model Checker (COSPAN)

60

Verification Statistics: Robot Controller

1.00E+041.00E+061.00E+081.00E+101.00E+121.00E+141.00E+161.00E+181.00E+201.00E+221.00E+24

ControlTermination

SafetyCoordination

2(C)

2(A)

3(C)

3(A)

4(C)

4(A)

5(C)

5(A)

6(C)

6(A)

Memory/Timeexhaustion

6(C)5(C)

i(C) Concrete program, i – number of joints

i(A) Abstract program

Platform: HP9000 (440MHz) with 6144 MB RAM

Memory

Models

61

Lessons Learned• Integration design and verification environment enables verification of non-

trivial software systems

– Translation of software into a formal language of model checkers is simplified

– Model checker is used as a debugging tool during software design

– Software design management enhances the applicability of the existing state space reduction techniques

– Testing of design-level specifications is helpful for specification of useful abstractions

• Integrated state space reduction supports verifying larger systems

– State space reduction techniques conducted by the syntactic program transformation are applicable without a change to the model checking tools and can be combined with the existing reduction techniques

– Effectiveness of reduction algorithms depends on interaction (combination, order of application, etc.) among reduction algorithms

62

Some Ideas for Future Work

• Integration of abstraction techniques• Combined abstraction approach: syntactic program transformation and reductions of the state

transition graphs• Systematic design and analysis of refinement procedures for design-level specifications

• Predictable component assembly• Abstraction techniques to check consistency of the component interfaces

• Integration of model checking and testing techniques• Model checking to support interactive simulation during analysis of errors

- To track multiple processes- To allow analysis backwards and forward during the execution

• Testing of data dependent parts of the code which can not be handled by finite-state model checking techniques

• Engineering of software systems to enable symbolic verification to be efficient

model checking large-scale software natasha sharygina carnegie mellon university software...

Documents

system slide

formal model

application of model

model limited

state graph

formal verification

generated slide

state space explosion