Model Checking for Probabilistic Timed

SystemsJeremy Sproston

Università di TorinoVOSS Dagstuhl seminar

9th December 2002

The problem

• Model checking probabilistic timed systems– In probabilistic systems:

• Probabilistic choice between alternativesExample: electronic coin flipping in randomized algorithms

– In timed systems:• Timing parameters are critical for the correct

functioning of the systemExample: the system must meet a certain deadline

– In probabilistic timed systems:• Coexistence of probabilistic choice and timing

The focus

• Probabilistic versions/extensions of timed automata (Alur and Dill 1994)

• Timed automaton = finite-state graph + clocks + clock constraints

• Clocks are an appropriate device for modelling time-dependent behaviour– A clock is a real-valued variable which increases at the

same rate as real timeClocks can be reset when system transitions occurTherefore, clocks can measure the exact amount of time elapsed since a particular transition

Timed automata

• Finite-state graph + clocks + clock constraints (examples: x3, x-y>5)

• Example: light switch

off

x2

x3on {x:=0}

{x:=0}

Timed CTL

• CTL: a request will always follow a response ⃞(request -> ( ⃟ response))

• TCTL: timed CTL– Alur, Courcoubetis and Dill (1993)– Henzinger et al. (1994)– A request will always follow a response within 5

milliseconds ⃞(request -> ( ⃟ 5 response))

– Use ⊨T for the satisfaction relation of TCTL

Timed automata: semantics• Problem: underlying semantic model is

– infinite-state: (node space) x R(number of clocks)

– infinitely branching: for example

• Model checking classically assumes a finite state space

Off, x=3.5

Off, x=3.5 Off, x=3.7… …

Model checking for timed automata

• Reduce to a finite state space: clock equivalence

• Partitioning bounded by the maximal constant used in the timed automaton or the TCTL formula

• Clock equivalent states satisfy the same clock constraints now and in the future

x

y

1

1

2

2

Model checking for timed automata

• Region equivalent states have the same– node– clock equivalence class

• Construct finite-state region graph (transition system)– States: region equivalence classes– Transitions:

Timetransitions

DiscretetransitionsE.g. crossing an edgewith {x:=0}

Model checking for timed automata

• Let: – TA be a timed automaton, T be a TCTL formula,– RG(TA, T) be the region graph of TA, T

• TA ⊨T T if and only if RG(TA, T) ⊨ – where ⊨ and are “untimed” versions of ⊨T and T

• Key result of Alur, Courcoubetis and Dill (1993)

Real-time probabilistic processes

• Alur, Courcoubetis and Dill (1991:ICALP, 1991:Real-Time)

• Similar to Generalized Semi-Markov Processes (Whitt (1980), Glynn (1989))

• A fully probabilistic model

Real-time probabilistic processes

• Finite-state graph+ clocks+ clock scheduling function+ probabilistic branching over edges+ probabilistic clock resetting

• Example: light switch

off

{x}

x,yon {y}

{y}

y

y:=Uniform(1,30)x:=3

Timed CTL revisited

• Interpreting “branching-time” logic over fully probabilistic systems

s ⊨ means “the probability that the computations starting in s satisfy is > 0”

s ⊨ means “the probability that the computations starting in s satisfy is =1”

• Alur, Courcoubetis and Dill (1991:ICALP) interpret TCTL (branching-time) over real-time probabilistic processes

Timed CTL revisited

• For example:

⃞(request -> ( ⃟ 5 response))

With probability 1, a request is followed by a response within 5 milliseconds

• Use R-TCTL to denote the logic, and ⊨R for its satisfaction relation

Real-time probabilistic processes: semantics

• Real-time probabilistic processes use clocks, so are infinite-state Markov processes

• Clocks are set to negative values drawn from continuous probability distributions

• When at least one clock reaches 0, a transition is triggered

Model checking for real-time probabilistic processes

• Again, reduce to a finite state space using (a version of) clock equivalence

• The set of clocks to reach 0 first is the same for all clock equivalent states

xy

-1-2-3

-1-2-3

Model checking for real-time probabilistic processes

• Construct finite-state region graph (transition system)– States: region equivalence classes– Transitions:

Timetransitions

DiscretetransitionsE.g. crossing an edgetriggered by y; reset y within(1,2)

Model checking for real-time probabilistic processes

• Let:– RTPP be a real-time probabilistic process R be a R-TCTL formula,– RG(RTPP, R) be the region graph of RTPP, R

• RTPP ⊨R R if and only if RG(RTPP, R) ⊨ – where ⊨ and are “untimed” versions of ⊨R and R

• Key result of Alur, Courcoubetis and Dill (1991:ICALP)

Probabilistic timed automata

• Introduced by Jensen (1995), Kwiatkowska et al. (2002)

• Finite-state graph + clocks + clock constraints+ probabilistic branching over edges

• Example: light switch

off

x2

x3on

{x:=0}{x:=0}0.990.010.99

0.01

Probabilistic timed CTL• PCTL (Probabilistic CTL): Hansson and Jonsson

(1994), Bianco and de Alfaro (1995)– The system will fail with probability < 0.01

P<0.01[⃟ failure]

• PTCTL (timed PCTL): Kwiatkowska et al. (2002) • The system will fail within 5 hours with

probability < 0.01P<0.01[ ⃟ 5 failure]

• Use ⊨P to denote the satisfaction relation of PTCTL

Model checking probabilistic timed automata

• Probabilistic timed automaton semantics:– Infinite-state, infinite-branching Markov

decision process• Again, reduce to a finite state space

using clock equivalence

x

y

1

1

2

2

Model checking probabilistic timed automata

• Construct finite-state region graph (Markov decision process)– States: region equivalence classes– Transitions:

• Time transitions are as standard• Discrete transitions: for example

on{x:=0}0.99 0.01

faily<3 x<7

on0.99 0.01

fail

Model checking probabilistic timed automata

• Construct finite-state region graph (Markov decision process)– States: region equivalence classes– Transitions:

• Time transitions are as standard• Discrete transitions: for example

on{x:=0}0.99 0.01

faily<3 x<7

on

0.99

0.01

fail

{y:=0}

on

Model checking probabilistic timed automata

• Let: – PTA be a probabilistic timed automaton, P be a PTCTL formula,– RG(PTA, P) be the region graph of PTA, P

• PTA ⊨P P if and only if RG(PTA, P) ⊨ – where ⊨ and are “untimed” versions of ⊨P

and p

• Key result of Kwiatkowska et al. (2002)

Continuous probabilistic timed automata

• Introduced by Kwiatkowska et al. (2000)• Finite-state graph + clocks + clock constraints

+ probabilistic branching over edges+ probabilistic clock resetting

• Example: light switch

x20.990.01

0.990.01

off1 on off2y

y30x,y

x3 ∧ y30 y30

y=30y=30

y=30

y:=Uniform(0,29)x:=0

Model checking continuous probabilistic timed automata• Continuous probabilistic timed

automata semantics– Infinite-state, infinitely branching

probabilistic-nondeterministic system with continuous probability distributions

• Again, reduce to a finite state space using clock equivalence

Model checking continuous probabilistic timed automata• Problems with clock equivalence: an example by Alur

• Clock x is reset within (0,1) in node A; clock y is arbitrary• Some time elapses in node A• Then we move to node B; clock y is reset within (0,1)• 3 cases: (1) x<y, (2) x=y, (3) x>y• Probability of (2) is 0, but we do not know the probabilities

of (1) and (3) (clock equivalence abstracts from the duration of the time transition in node A)

xx=1

yx<1 y=1A B

Model checking continuous probabilistic timed automata• A partial solution: change the granularity of the

time scale – For example, from granularity of 1 to granularity of 0.5

– Say we know that x (0,0.5)– Say that y is then set within (0.5,1)– We know that y>x

1

1 1

1

0.5

0.5

Model checking continuous probabilistic timed automata• Given a time granularity, construct a

finite-state region graph (Markov decision process)– States: region equivalence classes– Transitions:

• Time transitions are standard• Handling of probabilistic branching over edges is

straightforward• But how do we deal with resetting clocks

according to continuous probability distributions?

Model checking continuous probabilistic timed automata• Representing continuously distributed clock

resets in the region graph:– Integrating over time-unit intervals gives the

probability of a clock being set within an interval• E.g. with a time granularity of 1, we integrate over

intervals such as (0,1), (1,2), …• E.g. with a time granularity of 0.5, we integrate over

intervals such as (0,0.5), (0.5, 1), …– But the relationship between the ordering on the

fractional parts of the newly set clocks and the clocks which keep their old values is not obtainable

– The probabilistic choice regarding this relationship is replaced with a nondeterministic choice

Model checking continuous probabilistic timed automata• Let:

– CPTA be a probabilistic timed automaton, P be a PTCTL formula,– n1 be the chosen time granularity,– RG(CPTA, P, n) be the region graph of CPTA, P, n

• CPTA ⊨P P if RG(CPTA, P, n) ⊨ – where ⊨ and are “untimed” versions of ⊨P and p

• Key result of Kwiatkowska et al. (2000)

Model checking continuous probabilistic timed automata• Replacing probabilistic choice with nondeterministic

choice introduces the possibility of an error in the computed probabilities

• But we know that the maximum probability that CPTA satisfies a path formula is bounded from above by the maximum probability that the RG(CPTA, P, n) satisfies the path formula (similar with minimum)

• For example: CPTA ⊨P P<0.01[⃟ failure]

ifRG(CPTA, P, n) ⊨ P<0.01[⃟ failure]

Conclusions: model checking timed automata

• Achieved success in the form of the development of tools such as UPPAAL (Uppsala/Aalborg) and KRONOS (Grenoble)

• Use of zone-based algorithms– Manipulate sets of clock equivalence classes

Conclusions: model checking real-time probabilistic

processes• Activity died off after Alur, Courcoubetis

and Dill’s 1991 papers

• Interest renewed by the development of process algebras with generally distributed delays (Bravetti et al., D’Argenio et al)

• Model checking of Semi-Markov Chains: Infante-Lopez et al. (2001)

Conclusions: model checking probabilistic timed automata• Model checking using PRISM (Kwiatkowska,

Norman and Parker (2002)) and:– Region graphs– Discrete-time semantics (given restrictions on

clock constraints to xc and xc)• Based on discrete-time semantics for timed automata

developed by Henzinger et al. (1992), Asarin et al. (1998), Bozga et al. (1999)

• Case studies: FireWire (Kwiatkowska et al. (2002:FAC)), IEEE802.11 (Kwiatkowska et al. (2002:PAPM-PROBMIV))

Conclusions: model checking probabilistic timed automata• Zone-based algorithms for probabilistic

timed automata:– Must carefully distinguish zones which have

different probabilities• Kwiatkowska et al. (2001:CONCUR, 2002:TCS)

– Case study: FireWire • Kwiatkowska et al. (2002:FAC), Daws et al. (2002)

Conclusions: model checking continuous probabilistic

timed automata• Increasing the time granularity blows up

the state space

• Exists a need to concentrate on restricted subclasses