tutorial: probabilistic modelchecking...probabilisticmodels purely probabilistic probabilisticand...
TRANSCRIPT
![Page 1: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/1.jpg)
Tutorial:
Probabilistic Model Checking
Christel BaierTechnische Universitat Dresden
1 / 373
![Page 2: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/2.jpg)
Probability elsewhere
2/373
![Page 3: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/3.jpg)
Probability elsewhere
• randomized algorithms [Rabin 1960]
symmetry breaking, fingerprint techniques,random choice of waiting times or IP addresses, ...
• stochastic control theory [Bellman 1957]
operations research
• performance modeling [Markov, Erlang, Kolm., ∼∼∼ 1900]
• biological systems
• resilient systems.........
3 / 373
![Page 4: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/4.jpg)
Probability elsewhere
• randomized algorithms [Rabin 1960]
symmetry breaking, fingerprint techniques,random choice of waiting times or IP addresses, ...
• stochastic control theory [Bellman 1957]
operations research
• performance modeling [Markov, Erlang, Kolm., ∼∼∼ 1900]
• biological systems
• resilient systems
discrete or continuous-time Markovian models
memoryless property: future system behavior dependsonly on the current state, but not on the past
4 / 373
![Page 5: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/5.jpg)
Probabilistic models
5/373
![Page 6: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/6.jpg)
Probabilistic models
purelyprobabilistic
probabilistic andnondeterministic
discretetime
continuoustime
6 / 373
![Page 7: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/7.jpg)
Probabilistic models
purelyprobabilistic
probabilistic andnondeterministic
discretetime
discrete-timeMarkov chain(DTMC)
Markov decisionprocess (MDP)
continuoustime
7 / 373
![Page 8: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/8.jpg)
Probabilistic models
purelyprobabilistic
probabilistic andnondeterministic
discretetime
discrete-timeMarkov chain(DTMC)
Markov decisionprocess (MDP)
continuoustime
continuous-timeMarkov chain(CTMC)
continuous-time MDP
interactive Markovchains
probabilistic timedautomata
stochastic automata.........
8 / 373
![Page 9: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/9.jpg)
Probabilistic models
purelyprobabilistic
probabilistic andnondeterministic
discretetime
discrete-timeMarkov chain(DTMC)
Markov decisionprocess (MDP)
continuoustime
continuous-timeMarkov chain(CTMC)
continuous-time MDP
interactive Markovchains
probabilistic timedautomata
stochastic automata.........
9 / 373
![Page 10: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/10.jpg)
Model checking
functionalrequirements
specification ΦΦΦe.g., temporal formula
reactivesystem
operationalmodelMMM
model checker:
doesM |= ΦM |= ΦM |= Φ hold ?
no +++ counterexample yes +++ witness10 / 373
![Page 11: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/11.jpg)
Probabilistic model checking
quantitativerequirements
specification ΦΦΦe.g., temporal formula
reactivesystem
probabilisticmodelMMM
probabilistic model checker:
doesM |= ΦM |= ΦM |= Φ hold ?
no +++ counterexample yes +++ witness11 / 373
![Page 12: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/12.jpg)
Probabilistic model checking
quantitativerequirements
specification ΦΦΦe.g., temporal formula
reactivesystem
probabilisticmodelMMM
probabilistic model checker:
quantitative analysis ofMMM against ΦΦΦ
probability for “bad behaviors” is < 10−6< 10−6< 10−6
probability for “good behaviors” is 111expected costs for ....
12 / 373
![Page 13: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/13.jpg)
Probabilistic model checking
• termination of probabilistic programs[Hart/Sharir/Pnueli’83]
• qualitative linear time properties [Vardi/Wolper’86]
for discrete-time Markov models [Courcoubetis/Yannak.’88]
13 / 373
![Page 14: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/14.jpg)
Probabilistic model checking
• termination of probabilistic programs[Hart/Sharir/Pnueli’83]
• qualitative linear time properties [Vardi/Wolper’86]
for discrete-time Markov models [Courcoubetis/Yannak.’88]
• probabilistic computation tree logic [Hansson/Jonsson’94]
for discrete-time Markov models [Bianco/de Alfaro’95]
• continuous stochastic logic [Aziz et al’96]
for continuous-time Markov chains [Baier et al’99]
• probabilistic timed automata [Jensen’96]
[Kwiatkowska et al’00].........
tools: PRISM, MRMC, STORM, IscasMC, PASS,ProbDiVinE, MARCIE, YMER, . . .. . .. . .
14 / 373
![Page 15: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/15.jpg)
Tutorial: Probabilistic Model Checking
Discrete-time Markov chains (DTMC)
∗ basic definitions
∗ probabilistic computation tree logic PCTL/PCTL*
∗ rewards, cost-utility ratios, weights
∗ conditional probabilities
Markov decision processes (MDP)
∗ basic definitions
∗ PCTL/PCTL* model checking
∗ fairness
∗ conditional probabilities
∗ rewards, quantiles
∗ mean-payoff
∗ expected accumulated weights15 / 373
![Page 16: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/16.jpg)
Tutorial: Probabilistic Model Checking
Discrete-time Markov chains (DTMC)
∗ basic definitions
∗ probabilistic computation tree logic PCTL/PCTL*
∗ rewards, cost-utility ratios, weights
∗ conditional probabilities
Markov decision processes (MDP)
∗ basic definitions
∗ PCTL/PCTL* model checking
∗ fairness
∗ conditional probabilities
∗ rewards, quantiles
∗ mean-payoff
∗ expected accumulated weights16 / 373
![Page 17: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/17.jpg)
Markov chains
... transition systems with probabilistic distributionsfor the successor states
17 / 373
![Page 18: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/18.jpg)
Markov chains
... transition systems with probabilistic distributionsfor the successor states
ααα βββ γγγ
transition systemnondeterministic branching
choice betweenaction-labeled transitions
131313 1
61616
121212
Markov chainprobabilistic branching
discrete-time
18 / 373
![Page 19: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/19.jpg)
Discrete-time Markov chain (DTMC)
MMM === (S ,P, . . .)(S ,P, . . .)(S ,P, . . .)
19 / 373
![Page 20: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/20.jpg)
Discrete-time Markov chain (DTMC)
MMM === (S ,P, . . .)(S ,P, . . .)(S ,P, . . .)
• countable state space SSS
20 / 373
![Page 21: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/21.jpg)
Discrete-time Markov chain (DTMC)
MMM === (S ,P, . . .)(S ,P, . . .)(S ,P, . . .)
• countable state space SSS ←−←−←− here: finite
21 / 373
![Page 22: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/22.jpg)
Discrete-time Markov chain (DTMC)
MMM === (S ,P, . . .)(S ,P, . . .)(S ,P, . . .)
• countable state space SSS ←−←−←− here: finite
• transition probability function P : S × S → [0, 1]P : S × S → [0, 1]P : S × S → [0, 1]
s.t.∑s ′∈S
P(s, s ′) = 1∑s ′∈S
P(s , s ′) = 1∑s ′∈S
P(s, s ′) = 1
22 / 373
![Page 23: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/23.jpg)
Discrete-time Markov chain (DTMC)
MMM === (S ,P, . . .)(S ,P, . . .)(S ,P, . . .)
• countable state space SSS ←−←−←− here: finite
• transition probability function P : S × S → [0, 1]P : S × S → [0, 1]P : S × S → [0, 1]
s.t.∑s ′∈S
P(s, s ′) = 1∑s ′∈S
P(s , s ′) = 1∑s ′∈S
P(s, s ′) = 1�discrete-time or time-abstract:
probability for the step s −→ s ′s −→ s ′s −→ s ′
23 / 373
![Page 24: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/24.jpg)
Discrete-time Markov chain (DTMC)
MMM === (S ,P,AP, L, . . .)(S ,P,AP, L, . . .)(S ,P,AP, L, . . .)
• countable state space SSS ←−←−←− here: finite
• transition probability function P : S × S → [0, 1]P : S × S → [0, 1]P : S × S → [0, 1]
s.t.∑s ′∈S
P(s, s ′) = 1∑s ′∈S
P(s , s ′) = 1∑s ′∈S
P(s, s ′) = 1
• APAPAP set of atomic propositions
• labeling function L : S → 2APL : S → 2APL : S → 2AP
24 / 373
![Page 25: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/25.jpg)
Discrete-time Markov chain (DTMC)
MMM === (S ,P,AP, L, . . .)(S ,P,AP, L, . . .)(S ,P,AP, L, . . .)
• countable state space SSS ←−←−←− here: finite
• transition probability function P : S × S → [0, 1]P : S × S → [0, 1]P : S × S → [0, 1]
s.t.∑s ′∈S
P(s, s ′) = 1∑s ′∈S
P(s , s ′) = 1∑s ′∈S
P(s, s ′) = 1
• APAPAP set of atomic propositions
• labeling function L : S → 2APL : S → 2APL : S → 2AP
• µ : S → [0, 1]µ : S → [0, 1]µ : S → [0, 1] initial distribution
• wgt : S → Zwgt : S → Zwgt : S → Z where wgt(s)wgt(s)wgt(s) is the reward (or weight)earned per visit of state sss
25 / 373
![Page 26: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/26.jpg)
Example: DTMC for communication protocol
MMM === (S ,P,AP, L, . . .)(S ,P,AP, L, . . .)(S ,P,AP, L, . . .)
• countable state space SSS ←−←−←− here: finite
• transition probability function P : S × S → [0, 1]P : S × S → [0, 1]P : S × S → [0, 1]
s.t.∑s ′∈S
P(s, s ′) = 1∑s ′∈S
P(s , s ′) = 1∑s ′∈S
P(s, s ′) = 1
startstartstarttrytrytry tototosendsendsend
delivereddelivereddelivered
messagemessagemessagelostlostlost
0.980.980.98
0.020.020.02
26 / 373
![Page 27: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/27.jpg)
Example: DTMC for communication protocol
MMM === (S ,P,AP, L, . . .)(S ,P,AP, L, . . .)(S ,P,AP, L, . . .)
• countable state space SSS ←−←−←− here: finite
• transition probability function P : S × S → [0, 1]P : S × S → [0, 1]P : S × S → [0, 1]
s.t.∑s ′∈S
P(s, s ′) = 1∑s ′∈S
P(s , s ′) = 1∑s ′∈S
P(s, s ′) = 1
startstartstart111 trytrytry tototo
sendsendsend
delivereddelivereddelivered
messagemessagemessagelostlostlost
1110.980.980.98
0.020.020.02
111
111
27 / 373
![Page 28: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/28.jpg)
Example: DTMC for communication protocol
MMM === (S ,P,AP, L, . . .)(S ,P,AP, L, . . .)(S ,P,AP, L, . . .)
• countable state space SSS ←−←−←− here: finite
• transition probability function P : S × S → [0, 1]P : S × S → [0, 1]P : S × S → [0, 1]
s.t.∑s ′∈S
P(s, s ′) = 1∑s ′∈S
P(s , s ′) = 1∑s ′∈S
P(s, s ′) = 1
startstartstart111 trytrytry tototo
sendsendsend
delivereddelivereddelivered
messagemessagemessagelostlostlost
1110.980.980.98
0.020.020.02
111
111
e.g., AP = {try , del}AP = {try , del}AP = {try , del}
∅∅∅
∅∅∅
{del}{del}{del}
{try}{try}{try}startstartstart
trytrytry tototosendsendsend
delivereddelivereddelivered
messagemessagemessagelostlostlost
28 / 373
![Page 29: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/29.jpg)
Example: DTMC for communication protocol
MMM === (S ,P,AP, L, . . .)(S ,P,AP, L, . . .)(S ,P,AP, L, . . .)
• countable state space SSS
• transition probability function P : S × S → [0, 1]P : S × S → [0, 1]P : S × S → [0, 1]
s.t.∑s ′∈S
P(s, s ′) = 1∑s ′∈S
P(s , s ′) = 1∑s ′∈S
P(s, s ′) = 1
startstartstarttrytrytry tototosendsendsend
delivereddelivereddelivered
messagemessagemessagelostlostlost
0.980.980.98
0.020.020.02
rewards for counting the number of trials
000
000
000
111
29 / 373
![Page 30: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/30.jpg)
Probability measure of a Markov chain
30/373
![Page 31: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/31.jpg)
Probability measure of a Markov chain
MMM === (S ,P,AP, L, µ)(S ,P,AP , L, µ)(S ,P,AP, L, µ) where µ : S → [0, 1]µ : S → [0, 1]µ : S → [0, 1]���initial distribution
31 / 373
![Page 32: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/32.jpg)
Probability measure of a Markov chain
MMM === (S ,P,AP, L, µ)(S ,P,AP , L, µ)(S ,P,AP, L, µ) where µ : S → [0, 1]µ : S → [0, 1]µ : S → [0, 1]���initial distribution
probability measure for measurable sets of paths:
32 / 373
![Page 33: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/33.jpg)
Probability measure of a Markov chain
MMM === (S ,P,AP, L, µ)(S ,P,AP , L, µ)(S ,P,AP, L, µ) where µ : S → [0, 1]µ : S → [0, 1]µ : S → [0, 1]���initial distribution
probability measure for measurable sets of paths:
consider the σσσ-algebra generated by cylinder sets
∆(s0 s1 . . . sn)∆(s0 s1 . . . sn)∆(s0 s1 . . . sn) === set of infinite pathss0 s1 . . . sn sn+1 sn+2 sn+3 . . .s0 s1 . . . sn sn+1 sn+2 sn+3 . . .s0 s1 . . . sn sn+1 sn+2 sn+3 . . .���
finite path
33 / 373
![Page 34: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/34.jpg)
Probability measure of a Markov chain
MMM === (S ,P,AP, L, µ)(S ,P,AP , L, µ)(S ,P,AP, L, µ) where µ : S → [0, 1]µ : S → [0, 1]µ : S → [0, 1]���initial distribution
probability measure for measurable sets of paths:
consider the σσσ-algebra generated by cylinder sets
∆(s0 s1 . . . sn)∆(s0 s1 . . . sn)∆(s0 s1 . . . sn) === set of infinite paths . . .. . .. . .
σσσ-algebra on universe UUU : set V ⊆ 2UV ⊆ 2UV ⊆ 2U s.t.
1. U ∈ VU ∈ VU ∈ V2. if T ∈ VT ∈ VT ∈ V then U \ T ∈ VU \ T ∈ VU \ T ∈ V3. if Ti ∈ VTi ∈ VTi ∈ V for i ∈ Ni ∈ Ni ∈ N then
⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ V34 / 373
![Page 35: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/35.jpg)
Probability measure of a Markov chain
MMM === (S ,P,AP, L, µ)(S ,P,AP , L, µ)(S ,P,AP, L, µ) where µ : S → [0, 1]µ : S → [0, 1]µ : S → [0, 1]���initial distribution
probability measure for measurable sets of paths:
consider the σσσ-algebra generated by cylinder sets
∆(s0 s1 . . . sn)∆(s0 s1 . . . sn)∆(s0 s1 . . . sn) === set of infinite paths . . .. . .. . .
here: UUU === set of infinite paths ⊆⊆⊆ SωSωSω
VVV === smallest subset of 2U2U2U that containsall cylinder sets and is closed undercomplement and countable unions
35 / 373
![Page 36: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/36.jpg)
Probability measure of a Markov chain
MMM === (S ,P,AP, L, µ)(S ,P,AP , L, µ)(S ,P,AP, L, µ) where µ : S → [0, 1]µ : S → [0, 1]µ : S → [0, 1]���initial distribution
probability measure for measurable sets of paths:
consider the σσσ-algebra generated by cylinder sets
∆(s0 s1 . . . sn)∆(s0 s1 . . . sn)∆(s0 s1 . . . sn) === set of infinite pathss0 s1 . . . sn sn+1 sn+2 sn+3 . . .s0 s1 . . . sn sn+1 sn+2 sn+3 . . .s0 s1 . . . sn sn+1 sn+2 sn+3 . . .
probability measure is given by:
PrM(∆(s0 s1 . . . sn)
)= µ(s0) ·
∏1�i�n
P(si−1, si)PrM(∆(s0 s1 . . . sn)
)= µ(s0) ·
∏1�i�n
P(si−1, si)PrM(∆(s0 s1 . . . sn)
)= µ(s0) ·
∏1�i�n
P(si−1, si)
36 / 373
![Page 37: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/37.jpg)
Example: Markov chain
startstartstart trytrytry
deldeldel
lostlostlost
0.980.980.98
0.020.020.02
probability for delivering the message within 555 steps:
37 / 373
![Page 38: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/38.jpg)
Example: Markov chain
startstartstart trytrytry
deldeldel
lostlostlost
0.980.980.98
0.020.020.02
probability for delivering the message within 555 steps:
=== PrM(start try del
)PrM
(start try del
)PrM
(start try del
)+++ PrM
(start try lost try del
)PrM
(start try lost try del
)PrM
(start try lost try del
)
notation: PrM( s0 s1 . . . sn )PrM( s0 s1 . . . sn )PrM( s0 s1 . . . sn ) === PrM(∆(s0 s1 . . . sn)
)PrM
(∆(s0 s1 . . . sn)
)PrM
(∆(s0 s1 . . . sn)
)38 / 373
![Page 39: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/39.jpg)
Example: Markov chain
startstartstart trytrytry
deldeldel
lostlostlost
0.980.980.98
0.020.020.02
probability for delivering the message within 555 steps:
=== PrM(start try del
)PrM
(start try del
)PrM
(start try del
)+++ PrM
(start try lost try del
)PrM
(start try lost try del
)PrM
(start try lost try del
)=== 0.98 + 0.02 · 0.98 = 0.99960.98 + 0.02 · 0.98 = 0.99960.98 + 0.02 · 0.98 = 0.9996
notation: PrM( s0 s1 . . . sn )PrM( s0 s1 . . . sn )PrM( s0 s1 . . . sn ) === PrM(∆(s0 s1 . . . sn)
)PrM
(∆(s0 s1 . . . sn)
)PrM
(∆(s0 s1 . . . sn)
)39 / 373
![Page 40: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/40.jpg)
Example: Markov chain
startstartstart trytrytry
deldeldel
lostlostlost
0.980.980.98
0.020.020.02
probability for eventually delivering the message:
40 / 373
![Page 41: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/41.jpg)
Example: Markov chain
startstartstart trytrytry
deldeldel
lostlostlost
0.980.980.98
0.020.020.02
probability for eventually delivering the message:
===∞∑n=0
PrM(start try (lost try )n del
)∞∑n=0
PrM(start try (lost try )n del
)∞∑n=0
PrM(start try (lost try)n del
)
41 / 373
![Page 42: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/42.jpg)
Example: Markov chain
startstartstart trytrytry
deldeldel
lostlostlost
0.980.980.98
0.020.020.02
probability for eventually delivering the message:
===∞∑n=0
PrM(start try (lost try )n del
)∞∑n=0
PrM(start try (lost try )n del
)∞∑n=0
PrM(start try (lost try)n del
)
===∞∑n=0
0.02n · 0.98 = 1∞∑n=0
0.02n · 0.98 = 1∞∑n=0
0.02n · 0.98 = 1
42 / 373
![Page 43: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/43.jpg)
Measurability of classical properties
43/373
![Page 44: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/44.jpg)
Measurability of classical properties
A σσσ-algebra is a pair (U ,V)(U ,V)(U ,V) where UUU is a set and V ⊆ 2UV ⊆ 2UV ⊆ 2U
such that: 1. U ∈ VU ∈ VU ∈ V2. if T ∈ VT ∈ VT ∈ V then U \ T ∈ VU \ T ∈ VU \ T ∈ V3. if Ti ∈ VTi ∈ VTi ∈ V for i ∈ Ni ∈ Ni ∈ N then
⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ VThe elements of VVV are called events.
44 / 373
![Page 45: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/45.jpg)
Measurability of classical properties
A σσσ-algebra is a pair (U ,V)(U ,V)(U ,V) where UUU is a set and V ⊆ 2UV ⊆ 2UV ⊆ 2U
such that: 1. U ∈ VU ∈ VU ∈ V2. if T ∈ VT ∈ VT ∈ V then U \ T ∈ VU \ T ∈ VU \ T ∈ V3. if Ti ∈ VTi ∈ VTi ∈ V for i ∈ Ni ∈ Ni ∈ N then
⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ VThe elements of VVV are called events.
DTMCs: UUU === set of infinite paths
VVV ===
{σσσ-algebra generated by thecylinder sets
∆(s0 s1 . . . sn)∆(s0 s1 . . . sn)∆(s0 s1 . . . sn) ===
{set of infinite paths πππ of the forms0 s1 . . . sn sn+1 sn+2 sn+3 . . .s0 s1 . . . sn sn+1 sn+2 sn+3 . . .s0 s1 . . . sn sn+1 sn+2 sn+3 . . .
45 / 373
![Page 46: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/46.jpg)
Measurability of classical properties
A σσσ-algebra is a pair (U ,V)(U ,V)(U ,V) where UUU is a set and V ⊆ 2UV ⊆ 2UV ⊆ 2U
such that: 1. U ∈ VU ∈ VU ∈ V2. if T ∈ VT ∈ VT ∈ V then U \ T ∈ VU \ T ∈ VU \ T ∈ V3. if Ti ∈ VTi ∈ VTi ∈ V for i ∈ Ni ∈ Ni ∈ N then
⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ VThe elements of VVV are called events.
step-bounded reachability: “visit GGG within nnn steps”
♦�nG♦�nG♦�nG ===⋃
0�i�n
⋃0�i�n
⋃0�i�n
⋃s0,...,si
⋃s0,...,si
⋃s0,...,si
∆(s0 s1 . . . si−1 si
)∆(s0 s1 . . . si−1 si
)∆(s0 s1 . . . si−1 si
)where si ∈ Gsi ∈ Gsi ∈ G
46 / 373
![Page 47: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/47.jpg)
Measurability of classical properties
A σσσ-algebra is a pair (U ,V)(U ,V)(U ,V) where UUU is a set and V ⊆ 2UV ⊆ 2UV ⊆ 2U
such that: 1. U ∈ VU ∈ VU ∈ V2. if T ∈ VT ∈ VT ∈ V then U \ T ∈ VU \ T ∈ VU \ T ∈ V3. if Ti ∈ VTi ∈ VTi ∈ V for i ∈ Ni ∈ Ni ∈ N then
⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ VThe elements of VVV are called events.
step-bounded reachability: “visit GGG within nnn steps”
♦�nG♦�nG♦�nG ===⋃
0�i�n
⋃0�i�n
⋃0�i�n
⋃s0,...,si
⋃s0,...,si
⋃s0,...,si
∆(s0 s1 . . . si−1 si
)∆(s0 s1 . . . si−1 si
)∆(s0 s1 . . . si−1 si
)where si ∈ Gsi ∈ Gsi ∈ G and s0, . . . , si−1 /∈ Gs0, . . . , si−1 /∈ Gs0, . . . , si−1 /∈ G
47 / 373
![Page 48: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/48.jpg)
Measurability of classical properties
A σσσ-algebra is a pair (U ,V)(U ,V)(U ,V) where UUU is a set and V ⊆ 2UV ⊆ 2UV ⊆ 2U
such that: 1. U ∈ VU ∈ VU ∈ V2. if T ∈ VT ∈ VT ∈ V then U \ T ∈ VU \ T ∈ VU \ T ∈ V3. if Ti ∈ VTi ∈ VTi ∈ V for i ∈ Ni ∈ Ni ∈ N then
⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ VThe elements of VVV are called events.
step-bounded reachability: “visit GGG within nnn steps”
♦�nG♦�nG♦�nG ===⋃
0�i�n
⋃0�i�n
⋃0�i�n
⋃s0,...,si
⋃s0,...,si
⋃s0,...,si
∆(s0 s1 . . . si−1 si
)∆(s0 s1 . . . si−1 si
)∆(s0 s1 . . . si−1 si
)PrM
(♦�nG
)PrM
(♦�nG
)PrM
(♦�nG
)===
∑0�i�n
∑0�i�n
∑0�i�n
∑s0,...,si
∑s0,...,si
∑s0,...,si
PrM(s0 s1 . . . si−1 si
)PrM
(s0 s1 . . . si−1 si
)PrM
(s0 s1 . . . si−1 si
)48 / 373
![Page 49: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/49.jpg)
Measurability of classical properties
A σσσ-algebra is a pair (U ,V)(U ,V)(U ,V) where UUU is a set and V ⊆ 2UV ⊆ 2UV ⊆ 2U
such that: 1. U ∈ VU ∈ VU ∈ V2. if T ∈ VT ∈ VT ∈ V then U \ T ∈ VU \ T ∈ VU \ T ∈ V3. if Ti ∈ VTi ∈ VTi ∈ V for i ∈ Ni ∈ Ni ∈ N then
⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ VThe elements of VVV are called events.
unbounded reachability: “visit GGG eventually”
♦G♦G♦G ===⋃i∈N
⋃i∈N
⋃i∈N
⋃s0,...,si
⋃s0,...,si
⋃s0,...,si
∆(s0 s1 . . . si−1 si
)∆(s0 s1 . . . si−1 si
)∆(s0 s1 . . . si−1 si
)where si ∈ Gsi ∈ Gsi ∈ G and s0, . . . , si−1 /∈ Gs0, . . . , si−1 /∈ Gs0, . . . , si−1 /∈ G
49 / 373
![Page 50: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/50.jpg)
Measurability of classical properties
A σσσ-algebra is a pair (U ,V)(U ,V)(U ,V) where UUU is a set and V ⊆ 2UV ⊆ 2UV ⊆ 2U
such that: 1. U ∈ VU ∈ VU ∈ V2. if T ∈ VT ∈ VT ∈ V then U \ T ∈ VU \ T ∈ VU \ T ∈ V3. if Ti ∈ VTi ∈ VTi ∈ V for i ∈ Ni ∈ Ni ∈ N then
⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ VThe elements of VVV are called events.
unbounded reachability: “visit GGG eventually”
♦G♦G♦G ===⋃i∈N
⋃i∈N
⋃i∈N
⋃s0,...,si
⋃s0,...,si
⋃s0,...,si
∆(s0 s1 . . . si−1 si
)∆(s0 s1 . . . si−1 si
)∆(s0 s1 . . . si−1 si
)PrM
(♦G
)PrM
(♦G
)PrM
(♦G
)===
∑i∈N
∑i∈N
∑i∈N
∑s0,...,si
∑s0,...,si
∑s0,...,si
PrM(s0 s1 . . . si−1 si
)PrM
(s0 s1 . . . si−1 si
)PrM
(s0 s1 . . . si−1 si
)50 / 373
![Page 51: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/51.jpg)
Measurability of classical properties
A σσσ-algebra is a pair (U ,V)(U ,V)(U ,V) where UUU is a set and V ⊆ 2UV ⊆ 2UV ⊆ 2U
such that: 1. U ∈ VU ∈ VU ∈ V2. if T ∈ VT ∈ VT ∈ V then U \ T ∈ VU \ T ∈ VU \ T ∈ V3. if Ti ∈ VTi ∈ VTi ∈ V for i ∈ Ni ∈ Ni ∈ N then
⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ VThe elements of VVV are called events.
repeated reachability: “visit GGG infinitely often”
�♦G�♦G�♦G ===⋂n∈N
⋂n∈N
⋂n∈N
⋃i�n
⋃i�n
⋃i�n
⋃s0,...,si
⋃s0,...,si
⋃s0,...,si
∆(s0 s1 . . . si−1 si
)∆(s0 s1 . . . si−1 si
)∆(s0 s1 . . . si−1 si
)where si ∈ Gsi ∈ Gsi ∈ G
51 / 373
![Page 52: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/52.jpg)
Measurability of classical properties
A σσσ-algebra is a pair (U ,V)(U ,V)(U ,V) where UUU is a set and V ⊆ 2UV ⊆ 2UV ⊆ 2U
such that: 1. U ∈ VU ∈ VU ∈ V2. if T ∈ VT ∈ VT ∈ V then U \ T ∈ VU \ T ∈ VU \ T ∈ V3. if Ti ∈ VTi ∈ VTi ∈ V for i ∈ Ni ∈ Ni ∈ N then
⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ VThe elements of VVV are called events.
repeated reachability: “visit GGG infinitely often”
�♦G�♦G�♦G ===⋂n∈N
⋂n∈N
⋂n∈N
⋃i�n
⋃i�n
⋃i�n
⋃s0,...,si
⋃s0,...,si
⋃s0,...,si
∆(s0 s1 . . . si−1 si
)∆(s0 s1 . . . si−1 si
)∆(s0 s1 . . . si−1 si
)where si ∈ Gsi ∈ Gsi ∈ G , but possibly sj ∈ Gsj ∈ Gsj ∈ G for some j < ij < ij < i
52 / 373
![Page 53: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/53.jpg)
Measurability of classical properties
A σσσ-algebra is a pair (U ,V)(U ,V)(U ,V) where UUU is a set and V ⊆ 2UV ⊆ 2UV ⊆ 2U
such that: 1. U ∈ VU ∈ VU ∈ V2. if T ∈ VT ∈ VT ∈ V then U \ T ∈ VU \ T ∈ VU \ T ∈ V3. if Ti ∈ VTi ∈ VTi ∈ V for i ∈ Ni ∈ Ni ∈ N then
⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ VThe elements of VVV are called events.
persistence: “from some moment on always GGG”
♦�G♦�G♦�G === PathsM \ �♦¬GPathsM \ �♦¬GPathsM \ �♦¬G
53 / 373
![Page 54: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/54.jpg)
Measurability of classical properties
A σσσ-algebra is a pair (U ,V)(U ,V)(U ,V) where UUU is a set and V ⊆ 2UV ⊆ 2UV ⊆ 2U
such that: 1. U ∈ VU ∈ VU ∈ V2. if T ∈ VT ∈ VT ∈ V then U \ T ∈ VU \ T ∈ VU \ T ∈ V3. if Ti ∈ VTi ∈ VTi ∈ V for i ∈ Ni ∈ Ni ∈ N then
⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ V⋃i∈N
Ti ∈ VThe elements of VVV are called events.
persistence: “from some moment on always GGG”
♦�G♦�G♦�G === PathsM \ �♦¬GPathsM \ �♦¬GPathsM \ �♦¬G
PrM(♦�G
)PrM
(♦�G
)PrM
(♦�G
)=== 1− PrM
(�♦¬G )
1− PrM(�♦¬G )
1− PrM(�♦¬G )
54 / 373
![Page 55: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/55.jpg)
Stochastic process of a Markov chain
55/373
![Page 56: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/56.jpg)
Stochastic process of a Markov chain
general definition of a stochastic process:
family(Xt
)t∈Time
(Xt
)t∈Time
(Xt
)t∈Time
of random variables Xt : U → SXt : U → SXt : U → S
56 / 373
![Page 57: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/57.jpg)
Stochastic process of a Markov chain
general definition of a stochastic process:
family(Xt
)t∈Time
(Xt
)t∈Time
(Xt
)t∈Time
of random variables Xt : U → SXt : U → SXt : U → S
• TimeTimeTime is a time domain, e.g., NNN or R�0R�0R�0
• SSS is a set with fixed σσσ-algebra
• UUU is a sample space with fixed σσσ-algebra
57 / 373
![Page 58: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/58.jpg)
Stochastic process of a Markov chain
DTMCM = (S ,P, . . .)M = (S ,P, . . .)M = (S ,P, . . .)
family(Xt
)t∈Time
(Xt
)t∈Time
(Xt
)t∈Time
of random variables Xt : U → SXt : U → SXt : U → S
• TimeTimeTime is a time domain ←−←−←− Time = NTime = NTime = N
• SSS is a set ←−←−←− state space
• UUU is a sample space ←−←−←− set of infinite paths
58 / 373
![Page 59: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/59.jpg)
Stochastic process of a Markov chain
DTMCM = (S ,P, . . .)M = (S ,P, . . .)M = (S ,P, . . .)
family(Xt
)t∈Time
(Xt
)t∈Time
(Xt
)t∈Time
of random variables Xt : U → SXt : U → SXt : U → S
• TimeTimeTime is a time domain ←−←−←− Time = NTime = NTime = N
• SSS is a set ←−←−←− state space
• UUU is a sample space ←−←−←− set of infinite paths
If t ∈ Nt ∈ Nt ∈ N and π = s0 s1 s2 s3 . . . st . . .π = s0 s1 s2 s3 . . . st . . .π = s0 s1 s2 s3 . . . st . . . then Xt(π) = stXt(π) = stXt(π) = st .
59 / 373
![Page 60: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/60.jpg)
Stochastic process of a Markov chain
DTMCM = (S ,P, . . .)M = (S ,P, . . .)M = (S ,P, . . .)
family(Xt
)t∈Time
(Xt
)t∈Time
(Xt
)t∈Time
of random variables Xt : U → SXt : U → SXt : U → S
• TimeTimeTime is a time domain ←−←−←− Time = NTime = NTime = N
• SSS is a set ←−←−←− state space
• UUU is a sample space ←−←−←− set of infinite paths
If t ∈ Nt ∈ Nt ∈ N and π = s0 s1 . . . st−2 u st . . .π = s0 s1 . . . st−2 u st . . .π = s0 s1 . . . st−2 u st . . . then Xt(π) = stXt(π) = stXt(π) = st .
Markov property:
PrM(Xt = s
∣∣Xt−1 = u)
PrM(Xt = s
∣∣Xt−1 = u)
PrM(Xt = s
∣∣Xt−1 = u)
===
PrM(Xt = s
∣∣Xt−1 = u, Xt−2 = st−2, . . . , X0 = s0)
PrM(Xt = s
∣∣Xt−1 = u, Xt−2 = st−2, . . . , X0 = s0)
PrM(Xt = s
∣∣Xt−1 = u, Xt−2 = st−2, . . . , X0 = s0)
60 / 373
![Page 61: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/61.jpg)
Stochastic process of a Markov chain
DTMCM = (S ,P, . . .)M = (S ,P, . . .)M = (S ,P, . . .)
family(Xt
)t∈Time
(Xt
)t∈Time
(Xt
)t∈Time
of random variables Xt : U → SXt : U → SXt : U → S
• TimeTimeTime is a time domain ←−←−←− Time = NTime = NTime = N
• SSS is a set ←−←−←− state space
• UUU is a sample space ←−←−←− set of infinite paths
If t ∈ Nt ∈ Nt ∈ N and π = s0 s1 . . . st−2 u st . . .π = s0 s1 . . . st−2 u st . . .π = s0 s1 . . . st−2 u st . . . then Xt(π) = stXt(π) = stXt(π) = st .
Markov property:
PrM(Xt = s
∣∣Xt−1 = u)
PrM(Xt = s
∣∣Xt−1 = u)
PrM(Xt = s
∣∣Xt−1 = u)
=== P(u, s)P(u, s)P(u, s) ===
PrM(Xt = s
∣∣Xt−1 = u, Xt−2 = st−2, . . . , X0 = s0)
PrM(Xt = s
∣∣Xt−1 = u, Xt−2 = st−2, . . . , X0 = s0)
PrM(Xt = s
∣∣Xt−1 = u, Xt−2 = st−2, . . . , X0 = s0)
61 / 373
![Page 62: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/62.jpg)
Stochastic process of a Markov chain
DTMCM = (S ,P, . . .)M = (S ,P, . . .)M = (S ,P, . . .)
family(Xt
)t∈Time
(Xt
)t∈Time
(Xt
)t∈Time
of random variables Xt : U → SXt : U → SXt : U → S
• TimeTimeTime is a time domain ←−←−←− Time = NTime = NTime = N
• SSS is a set ←−←−←− state space
• UUU is a sample space ←−←−←− set of infinite paths
If t ∈ Nt ∈ Nt ∈ N and π = s0 s1 . . . st−2 u st . . .π = s0 s1 . . . st−2 u st . . .π = s0 s1 . . . st−2 u st . . . then Xt(π) = stXt(π) = stXt(π) = st .
Markov property:
PrM(Xt = s
∣∣Xt−1 = u)
PrM(Xt = s
∣∣Xt−1 = u)
PrM(Xt = s
∣∣Xt−1 = u)
=== P(u, s)P(u, s)P(u, s) ===
PrM(X1 = s
∣∣ X0 = u)
PrM(X1 = s
∣∣ X0 = u)
PrM(X1 = s
∣∣ X0 = u)
time-homogeneous62 / 373
![Page 63: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/63.jpg)
Transient and long-run distribution
63/373
![Page 64: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/64.jpg)
Transient and long-run distribution
transient: ... refers to a fixed time point ttt
long-run: ... when time tends to infinity
64 / 373
![Page 65: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/65.jpg)
Transient distribution
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC, t ∈ Nt ∈ Nt ∈ N and s ∈ Ss ∈ Ss ∈ S .
transient state probability:
µt(s)µt(s)µt(s) === PrM{s0 s1 s2 . . . ∈ PathsM : st = s
}PrM
{s0 s1 s2 . . . ∈ PathsM : st = s
}PrM
{s0 s1 s2 . . . ∈ PathsM : st = s
}=== PrM
(Xt = s
)PrM
(Xt = s
)PrM
(Xt = s
)
65 / 373
![Page 66: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/66.jpg)
Transient distribution
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC, t ∈ Nt ∈ Nt ∈ N and s ∈ Ss ∈ Ss ∈ S .
transient state probability:
µt(s)µt(s)µt(s) === PrM{s0 s1 s2 . . . ∈ PathsM : st = s
}PrM
{s0 s1 s2 . . . ∈ PathsM : st = s
}PrM
{s0 s1 s2 . . . ∈ PathsM : st = s
}=== µ · P t · idsµ · P t · idsµ · P t · ids���
initial distribution(row vector)
66 / 373
![Page 67: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/67.jpg)
Transient distribution
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC, t ∈ Nt ∈ Nt ∈ N and s ∈ Ss ∈ Ss ∈ S .
transient state probability:
µt(s)µt(s)µt(s) === PrM{s0 s1 s2 . . . ∈ PathsM : st = s
}PrM
{s0 s1 s2 . . . ∈ PathsM : st = s
}PrM
{s0 s1 s2 . . . ∈ PathsM : st = s
}=== µ · P t · idsµ · P t · idsµ · P t · ids���ttt-th power of
transition probability matrix
P t = P t−1 · PP t = P t−1 · PP t = P t−1 · P
67 / 373
![Page 68: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/68.jpg)
Transient distribution
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC, t ∈ Nt ∈ Nt ∈ N and s ∈ Ss ∈ Ss ∈ S .
transient state probability:
µt(s)µt(s)µt(s) === PrM{s0 s1 s2 . . . ∈ PathsM : st = s
}PrM
{s0 s1 s2 . . . ∈ PathsM : st = s
}PrM
{s0 s1 s2 . . . ∈ PathsM : st = s
}=== µ · P t · idsµ · P t · idsµ · P t · ids���
column vector (0 . . . 0, 1, 0, . . .0)(0 . . . 0, 1, 0, . . .0)(0 . . . 0, 1, 0, . . .0)representing Dirac distribution
for state sss
68 / 373
![Page 69: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/69.jpg)
Transient distribution
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC, t ∈ Nt ∈ Nt ∈ N and s ∈ Ss ∈ Ss ∈ S .
transient state probability:
µt(s)µt(s)µt(s) === PrM{s0 s1 s2 . . . ∈ PathsM : st = s
}PrM
{s0 s1 s2 . . . ∈ PathsM : st = s
}PrM
{s0 s1 s2 . . . ∈ PathsM : st = s
}=== µ · P t · idsµ · P t · idsµ · P t · ids === µt−1 · P · idsµt−1 · P · idsµt−1 · P · ids���
column vector (0 . . . 0, 1, 0, . . .0)(0 . . . 0, 1, 0, . . .0)(0 . . . 0, 1, 0, . . .0)representing Dirac distribution
for state sss
69 / 373
![Page 70: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/70.jpg)
Transient distribution
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC, t ∈ Nt ∈ Nt ∈ N and s ∈ Ss ∈ Ss ∈ S .
transient state probability:
µt(s)µt(s)µt(s) === PrM{s0 s1 s2 . . . ∈ PathsM : st = s
}PrM
{s0 s1 s2 . . . ∈ PathsM : st = s
}PrM
{s0 s1 s2 . . . ∈ PathsM : st = s
}=== µ · P t · idsµ · P t · idsµ · P t · ids === µt−1 · P · idsµt−1 · P · idsµt−1 · P · ids���
transient state distributionfor time point t−1t−1t−1
Thus: µ0µ0µ0 === µµµ initial distribution
µtµtµt === µt−1 · Pµt−1 · Pµt−1 · P for t � 1t � 1t � 170 / 373
![Page 71: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/71.jpg)
Long-run distributions
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC.
steady-state probability: µ(s)µ(s)µ(s) === limt→∞µt(s)limt→∞µt(s)limt→∞µt(s)
µt(s)µt(s)µt(s) probability for being in state sss after ttt steps71 / 373
![Page 72: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/72.jpg)
Long-run distributions
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC.
steady-state probability: µ(s)µ(s)µ(s) === limt→∞µt(s)limt→∞µt(s)limt→∞µt(s)
• limit may not exist
µt(s)µt(s)µt(s) probability for being in state sss after ttt steps72 / 373
![Page 73: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/73.jpg)
Long-run distributions
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC.
steady-state probability: µ(s)µ(s)µ(s) === limt→∞µt(s)limt→∞µt(s)limt→∞µt(s)
• limit may not exist
sss
uuu
111111
µ2t(s) = 1µ2t(s) = 1µ2t(s) = 1 µ2t+1(s) = 0µ2t+1(s) = 0µ2t+1(s) = 0���even
time points
���odd
time points
73 / 373
![Page 74: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/74.jpg)
Long-run distributions
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC.
steady-state probability: µ(s)µ(s)µ(s) === limt→∞µt(s)limt→∞µt(s)limt→∞µt(s)
• limit may not exist or depend on the initialdistribution µµµ
sss
uuu
111111
µ2t(s) = 1µ2t(s) = 1µ2t(s) = 1 µ2t+1(s) = 0µ2t+1(s) = 0µ2t+1(s) = 0���even
time points
���odd
time points
74 / 373
![Page 75: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/75.jpg)
Long-run distributions
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC.
steady-state probability: µ(s)µ(s)µ(s) === limt→∞µt(s)limt→∞µt(s)limt→∞µt(s)
• limit may not exist or depend on the initialdistribution µµµ
sss
uuu
111
111
111
If µ(s) = 1µ(s) = 1µ(s) = 1 then: µ(s) = 1µ(s) = 1µ(s) = 1
75 / 373
![Page 76: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/76.jpg)
Long-run distributions
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC.
steady-state probability: µ(s)µ(s)µ(s) === limt→∞µt(s)limt→∞µt(s)limt→∞µt(s)
• limit may not exist or depend on the initialdistribution µµµ
sss
uuu
111
111
111
If µ(s) = 1µ(s) = 1µ(s) = 1 then: µ(s) = 1µ(s) = 1µ(s) = 1
If µ(u) = 1µ(u) = 1µ(u) = 1 then: µ(s) = 0µ(s) = 0µ(s) = 0
76 / 373
![Page 77: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/77.jpg)
Long-run distributions
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC.
steady-state probability: µ(s)µ(s)µ(s) === limt→∞µt(s)limt→∞µt(s)limt→∞µt(s)
• limit may not exist or depend on the initialdistribution µµµ
sss
uuu
121212
121212
111
111
If µ(s) = 1µ(s) = 1µ(s) = 1 then: µ(s) = 1µ(s) = 1µ(s) = 1
If µ(u) = 1µ(u) = 1µ(u) = 1 then: µ(s) = 0µ(s) = 0µ(s) = 0
If µ(s) = µ(u) = 12
µ(s) = µ(u) = 12µ(s) = µ(u) = 12 then:
µ(s) = 12
µ(s) = 12µ(s) = 12
77 / 373
![Page 78: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/78.jpg)
Long-run distributions
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC.
steady-state probability: µ(s)µ(s)µ(s) === limt→∞µt(s)limt→∞µt(s)limt→∞µt(s)
• limit may not exist or depend on the initialdistribution µµµ
• if existing for all states sss then µ = µ · Pµ = µ · Pµ = µ · P���balanceequation
78 / 373
![Page 79: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/79.jpg)
Long-run distributions
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC.
steady-state probability: µ(s)µ(s)µ(s) === limt→∞µt(s)limt→∞µt(s)limt→∞µt(s)
long-run fraction of being in state sss (Cesaro limit):
µt(s)µt(s)µt(s) probability for being in state sss after ttt steps79 / 373
![Page 80: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/80.jpg)
Long-run distributions
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC.
steady-state probability: µ(s)µ(s)µ(s) === limt→∞µt(s)limt→∞µt(s)limt→∞µt(s)
long-run fraction of being in state sss (Cesaro limit):
θ(s)θ(s)θ(s) === limT→∞lim
T→∞lim
T→∞1
T+1 ·T∑t=0
µt(s)1
T+1 ·T∑t=0
µt(s)1
T+1 ·T∑t=0
µt(s)
µt(s)µt(s)µt(s) probability for being in state sss after ttt steps80 / 373
![Page 81: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/81.jpg)
Long-run distributions
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC.
steady-state probability: µ(s)µ(s)µ(s) === limt→∞µt(s)limt→∞µt(s)limt→∞µt(s)
long-run fraction of being in state sss (Cesaro limit):
θ(s)θ(s)θ(s) === limT→∞lim
T→∞lim
T→∞1
T+1 ·T∑t=0
µt(s)1
T+1 ·T∑t=0
µt(s)1
T+1 ·T∑t=0
µt(s)
• Cesaro limit always exists
µt(s)µt(s)µt(s) probability for being in state sss after ttt steps81 / 373
![Page 82: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/82.jpg)
Long-run distributions
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC.
steady-state probability: µ(s)µ(s)µ(s) === limt→∞µt(s)limt→∞µt(s)limt→∞µt(s)
long-run fraction of being in state sss (Cesaro limit):
θ(s)θ(s)θ(s) === limT→∞lim
T→∞lim
T→∞1
T+1 ·T∑t=0
µt(s)1
T+1 ·T∑t=0
µt(s)1
T+1 ·T∑t=0
µt(s)
• Cesaro limit always exists
• if the steady-state probabilities exists: µ(s) = θ(s)µ(s) = θ(s)µ(s) = θ(s)
µt(s)µt(s)µt(s) probability for being in state sss after ttt steps82 / 373
![Page 83: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/83.jpg)
Long-run distributions
LetM = (S ,P, µ, . . .)M = (S ,P, µ, . . .)M = (S ,P, µ, . . .) be a DTMC.
steady-state probability: µ(s)µ(s)µ(s) === limt→∞µt(s)limt→∞µt(s)limt→∞µt(s)
long-run fraction of being in state sss (Cesaro limit):
θ(s)θ(s)θ(s) === limT→∞lim
T→∞lim
T→∞1
T+1 ·T∑t=0
µt(s)1
T+1 ·T∑t=0
µt(s)1
T+1 ·T∑t=0
µt(s)
• Cesaro limit always exists
• if the steady-state probabilities exists: µ(s) = θ(s)µ(s) = θ(s)µ(s) = θ(s)
• ifMMM is strongly connected: θθθ is computable viathe balance equation θ = θ · Pθ = θ · Pθ = θ · P where
∑s∈S
θ(s) = 1∑s∈S
θ(s) = 1∑s∈S
θ(s) = 1
83 / 373
![Page 84: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/84.jpg)
Fundamental property of finite Markov chains
84/373
![Page 85: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/85.jpg)
Fundamental property of finite Markov chains
Almost surely, i.e., with probability 111:
A bottom strongly connected component will bereached and all its states visited infinitely often.
85 / 373
![Page 86: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/86.jpg)
Fundamental property of finite Markov chains
Almost surely, i.e., with probability 111:
A bottom strongly connected component will bereached and all its states visited infinitely often.
PrM{
PrM{
PrM{s0 s1 s2 . . . ∈ PathsMs0 s1 s2 . . . ∈ PathsMs0 s1 s2 . . . ∈ PathsM :::
there exists i � 0i � 0i � 0 and a BSCC CCC s.t.
∀j � i . sj ∈ C ∧ ∀s ∈ C∞∃ j . sj = s
}∀j � i . sj ∈ C ∧ ∀s ∈ C∞∃ j . sj = s
}∀j � i . sj ∈ C ∧ ∀s ∈ C∞∃ j . sj = s
}=== 111
︸ ︷︷ ︸eventuallyforever CCC
︸ ︷︷ ︸visit each state in CCC
infinitely often86 / 373
![Page 87: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/87.jpg)
Fundamental property of finite Markov chains
Almost surely, i.e., with probability 111:
A bottom strongly connected component will bereached and all its states visited infinitely often.
787878
181818
252525
353535
131313
121212
161616
141414
343434
87 / 373
![Page 88: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/88.jpg)
Fundamental property of finite Markov chains
Almost surely, i.e., with probability 111:
A bottom strongly connected component will bereached and all its states visited infinitely often.
222 BSCCs
787878
181818
252525
353535
131313
121212
161616
141414
343434
88 / 373
![Page 89: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/89.jpg)
Fundamental property of finite Markov chains
Almost surely, i.e., with probability 111:
A bottom strongly connected component will bereached and all its states visited infinitely often.
long-run distribution:
θ(s) > 0θ(s) > 0θ(s) > 0 iff sss belongs to some BSCC
89 / 373
![Page 90: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/90.jpg)
Fundamental property of finite Markov chains
Almost surely, i.e., with probability 111:
A bottom strongly connected component will bereached and all its states visited infinitely often.
long-run distribution:
• θ(s) > 0θ(s) > 0θ(s) > 0 iff sss belongs to some BSCC
• if sss is a state of BSCC BBB then:
θ(s)θ(s)θ(s) === PrM(♦B) · θB(s)PrM(♦B) · θB(s)PrM(♦B) · θB(s)︸ ︷︷ ︸probability forreaching B
↖↖↖long-run probabilityfor state s inside B
90 / 373
![Page 91: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/91.jpg)
Tutorial: Probabilistic Model Checking
Discrete-time Markov chains (DTMC)
∗ basic definitions
∗ probabilistic computation tree logic PCTL/PCTL*
∗ rewards, cost-utility ratios, weights
∗ conditional probabilities
Markov decision processes (MDP)
∗ basic definitions
∗ PCTL/PCTL* model checking
∗ fairness
∗ conditional probabilities
∗ rewards, quantiles
∗ mean-payoff
∗ expected accumulated weights91 / 373
![Page 92: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/92.jpg)
Probabilistic computation tree logic
92/373
![Page 93: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/93.jpg)
Probabilistic computation tree logic
PCTL/PCTL* [Hansson/Jonsson 1994]
• probabilistic variants of CTL/CTL*
• contains a probabilistic operator PPPto specify lower/upper probability bounds
93 / 373
![Page 94: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/94.jpg)
Probabilistic computation tree logic
PCTL/PCTL* [Hansson/Jonsson 1994]
• probabilistic variants of CTL/CTL*
• contains a probabilistic operator PPPto specify lower/upper probability bounds
• operators for expected costs, long-run averages, ...will be considered later
94 / 373
![Page 95: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/95.jpg)
Syntax of PCTL*
state formulas:
ΦΦΦ ::=::=::= true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ . . .. . .. . .
path formulas:
ϕϕϕ ::=::=::= . . .. . .. . .
95 / 373
![Page 96: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/96.jpg)
Syntax of PCTL*
state formulas:
ΦΦΦ ::=::=::= true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ PI(ϕ)PI(ϕ)PI(ϕ)
path formulas:
ϕϕϕ ::=::=::= . . .. . .. . .
where a ∈ APa ∈ APa ∈ AP is an atomic proposition
I ⊆ [0, 1]I ⊆ [0, 1]I ⊆ [0, 1] is a probability interval
96 / 373
![Page 97: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/97.jpg)
Syntax of PCTL*
state formulas:
ΦΦΦ ::=::=::= true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ PI(ϕ)PI(ϕ)PI(ϕ)
path formulas:
ϕϕϕ ::=::=::= . . .. . .. . .
where a ∈ APa ∈ APa ∈ AP is an atomic proposition
I ⊆ [0, 1]I ⊆ [0, 1]I ⊆ [0, 1] is a probability interval
qualitative properties: P>0(ϕ)P>0(ϕ)P>0(ϕ) or P=1(ϕ)P=1(ϕ)P=1(ϕ)
quantitative properties: e.g., P>0.5(ϕ)P>0.5(ϕ)P>0.5(ϕ) or P�0.01(ϕ)P�0.01(ϕ)P�0.01(ϕ)
97 / 373
![Page 98: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/98.jpg)
Syntax of PCTL* path formulas
state formulas:
ΦΦΦ ::=::=::= true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ PI(ϕ)PI(ϕ)PI(ϕ)
path formulas:
ϕϕϕ ::=::=::= ΦΦΦ∣∣∣∣∣∣ ϕ1 ∧ ϕ2ϕ1 ∧ ϕ2ϕ1 ∧ ϕ2
∣∣∣∣∣∣ ¬ϕ¬ϕ¬ϕ ∣∣∣∣∣∣ . . .. . .. . .���state formula
98 / 373
![Page 99: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/99.jpg)
Syntax of PCTL* path formulas
state formulas:
ΦΦΦ ::=::=::= true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ PI(ϕ)PI(ϕ)PI(ϕ)
path formulas:
ϕϕϕ ::=::=::= ΦΦΦ∣∣∣∣∣∣ ϕ1 ∧ ϕ2ϕ1 ∧ ϕ2ϕ1 ∧ ϕ2
∣∣∣∣∣∣ ¬ϕ¬ϕ¬ϕ ∣∣∣∣∣∣©ϕ©ϕ©ϕ ∣∣∣∣∣∣ . . .. . .. . .© =© =© = next
99 / 373
![Page 100: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/100.jpg)
Syntax of PCTL* path formulas
state formulas:
ΦΦΦ ::=::=::= true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ PI(ϕ)PI(ϕ)PI(ϕ)
path formulas:
ϕϕϕ ::=::=::= ΦΦΦ∣∣∣∣∣∣ ϕ1 ∧ ϕ2ϕ1 ∧ ϕ2ϕ1 ∧ ϕ2
∣∣∣∣∣∣ ¬ϕ¬ϕ¬ϕ ∣∣∣∣∣∣©ϕ©ϕ©ϕ ∣∣∣∣∣∣ ϕ1Uϕ2ϕ1 Uϕ2ϕ1Uϕ2
© =© =© = next U =U =U = until
100 / 373
![Page 101: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/101.jpg)
Syntax of PCTL* path formulas
state formulas:
ΦΦΦ ::=::=::= true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ PI(ϕ)PI(ϕ)PI(ϕ)
path formulas:
ϕϕϕ ::=::=::= ΦΦΦ∣∣∣∣∣∣ ϕ1 ∧ ϕ2ϕ1 ∧ ϕ2ϕ1 ∧ ϕ2
∣∣∣∣∣∣ ¬ϕ¬ϕ¬ϕ ∣∣∣∣∣∣©ϕ©ϕ©ϕ ∣∣∣∣∣∣ ϕ1Uϕ2ϕ1 Uϕ2ϕ1Uϕ2
state formulaΦΦΦ
. . .. . .. . .
101 / 373
![Page 102: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/102.jpg)
Syntax of PCTL* path formulas
state formulas:
ΦΦΦ ::=::=::= true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ PI(ϕ)PI(ϕ)PI(ϕ)
path formulas:
ϕϕϕ ::=::=::= ΦΦΦ∣∣∣∣∣∣ ϕ1 ∧ ϕ2ϕ1 ∧ ϕ2ϕ1 ∧ ϕ2
∣∣∣∣∣∣ ¬ϕ¬ϕ¬ϕ ∣∣∣∣∣∣©ϕ©ϕ©ϕ ∣∣∣∣∣∣ ϕ1Uϕ2ϕ1 Uϕ2ϕ1Uϕ2
state formulaΦΦΦ
. . .. . .. . .
next©a©a©aaaa
. . .. . .. . .
102 / 373
![Page 103: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/103.jpg)
Syntax of PCTL* path formulas
state formulas:
ΦΦΦ ::=::=::= true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ PI(ϕ)PI(ϕ)PI(ϕ)
path formulas:
ϕϕϕ ::=::=::= ΦΦΦ∣∣∣∣∣∣ ϕ1 ∧ ϕ2ϕ1 ∧ ϕ2ϕ1 ∧ ϕ2
∣∣∣∣∣∣ ¬ϕ¬ϕ¬ϕ ∣∣∣∣∣∣©ϕ©ϕ©ϕ ∣∣∣∣∣∣ ϕ1Uϕ2ϕ1 Uϕ2ϕ1Uϕ2
state formulaΦΦΦ
. . .. . .. . .
next©a©a©aaaa
. . .. . .. . .
until aU baU baU baaa aaa aaa bbb
. . .. . .. . .103 / 373
![Page 104: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/104.jpg)
Derived path operators: eventually, always
104/ 373
![Page 105: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/105.jpg)
Derived path operators: eventually, always
syntax of path formulas:
ϕϕϕ ::=::=::= Φ∣∣ ϕ1 ∧ ϕ2
∣∣ ¬ϕ ∣∣ ©ϕ ∣∣ ϕ1Uϕ2Φ∣∣ ϕ1 ∧ ϕ2
∣∣ ¬ϕ ∣∣ ©ϕ ∣∣ ϕ1 Uϕ2Φ∣∣ ϕ1 ∧ ϕ2
∣∣ ¬ϕ ∣∣ ©ϕ ∣∣ ϕ1Uϕ2
until aU baU baU baaa aaa aaa bbb
. . .. . .. . .
eventually
♦b def= true U b♦b def= true U b♦b def= true U b
bbb. . .. . .. . .
105 / 373
![Page 106: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/106.jpg)
Derived path operators: eventually, always
syntax of path formulas:
ϕϕϕ ::=::=::= Φ∣∣ ϕ1 ∧ ϕ2
∣∣ ¬ϕ ∣∣ ©ϕ ∣∣ ϕ1Uϕ2Φ∣∣ ϕ1 ∧ ϕ2
∣∣ ¬ϕ ∣∣ ©ϕ ∣∣ ϕ1 Uϕ2Φ∣∣ ϕ1 ∧ ϕ2
∣∣ ¬ϕ ∣∣ ©ϕ ∣∣ ϕ1Uϕ2
until aU baU baU baaa aaa aaa bbb
. . .. . .. . .
eventually
♦b def= true U b♦b def= true U b♦b def= true U b
bbb. . .. . .. . .
always
�a def= ¬♦¬a�a def= ¬♦¬a�a def= ¬♦¬a
aaa aaa aaa aaa aaa aaa. . .. . .. . .
106 / 373
![Page 107: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/107.jpg)
Semantics of PCTL*
107/373
![Page 108: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/108.jpg)
Semantics of PCTL*
LetMMM === (S ,P,AP, L)(S ,P,AP, L)(S ,P,AP , L) be a Markov chain.
Define by structural induction:
• a satisfaction relation |=|=|= forstates s ∈ Ss ∈ Ss ∈ S and PCTL* state formulas
• a satisfaction relation |=|=|= for infinitepaths πππ inMMM and PCTL* path formulas
108 / 373
![Page 109: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/109.jpg)
Semantics of PCTL*
s |= trues |= trues |= true
s |= as |= as |= a iff a ∈ L(s)a ∈ L(s)a ∈ L(s)
s |= ¬Φs |= ¬Φs |= ¬Φ iff s �|= Φs �|= Φs �|= Φ
s |= Φ1 ∧ Φ2s |= Φ1 ∧ Φ2s |= Φ1 ∧ Φ2 iff s |= Φ1s |= Φ1s |= Φ1 and s |= Φ2s |= Φ2s |= Φ2
s |= PI(ϕ)s |= PI(ϕ)s |= PI(ϕ) iff PrMs (ϕ)PrMs (ϕ)PrMs (ϕ) ∈ I∈ I∈ I
109 / 373
![Page 110: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/110.jpg)
Semantics of PCTL*
s |= trues |= trues |= true
s |= as |= as |= a iff a ∈ L(s)a ∈ L(s)a ∈ L(s)
s |= ¬Φs |= ¬Φs |= ¬Φ iff s �|= Φs �|= Φs �|= Φ
s |= Φ1 ∧ Φ2s |= Φ1 ∧ Φ2s |= Φ1 ∧ Φ2 iff s |= Φ1s |= Φ1s |= Φ1 and s |= Φ2s |= Φ2s |= Φ2
s |= PI(ϕ)s |= PI(ϕ)s |= PI(ϕ) iff PrMs (ϕ)PrMs (ϕ)PrMs (ϕ) ∈ I∈ I∈ I���probability measure of the set of
paths πππ with π |= ϕπ |= ϕπ |= ϕ
when sss is viewed as the unique starting state110 / 373
![Page 111: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/111.jpg)
Semantics of PCTL* path formulas
let π = s0 s1 s2 s3 . . .π = s0 s1 s2 s3 . . .π = s0 s1 s2 s3 . . . be an infinite path inMMM
111 / 373
![Page 112: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/112.jpg)
Semantics of PCTL* path formulas
let π = s0 s1 s2 s3 . . .π = s0 s1 s2 s3 . . .π = s0 s1 s2 s3 . . . be an infinite path inMMM
π |= Φπ |= Φπ |= Φ iff s0 |= Φs0 |= Φs0 |= Φ
π |= ¬ϕπ |= ¬ϕπ |= ¬ϕ iff π �|= ϕπ �|= ϕπ �|= ϕ
π |= ϕ1 ∧ ϕ2π |= ϕ1 ∧ ϕ2π |= ϕ1 ∧ ϕ2 iff π |= ϕ1π |= ϕ1π |= ϕ1 and π |= ϕ2π |= ϕ2π |= ϕ2
π |=©ϕπ |=©ϕπ |=©ϕ iff s1 s2 s3 . . . |= ϕs1 s2 s3 . . . |= ϕs1 s2 s3 . . . |= ϕ
π |= ϕ1Uϕ2π |= ϕ1 Uϕ2π |= ϕ1Uϕ2 iff there exists � � 0� � 0� � 0 such that
s� s�+1 s�+2 . . .s� s�+1 s�+2 . . .s� s�+1 s�+2 . . . |=|=|= ϕ2ϕ2ϕ2
si si+1 si+2 . . .si si+1 si+2 . . .si si+1 si+2 . . . |=|=|= ϕ1ϕ1ϕ1 for 0 � i < �0 � i < �0 � i < �
112 / 373
![Page 113: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/113.jpg)
Examples for PCTL*-specifications
113/ 373
![Page 114: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/114.jpg)
Examples for PCTL*-specifications
communication protocol:
P=1
(�( try to send −→ P�0.9(©delivered) )
)P=1
(�( try to send −→ P�0.9(©delivered) )
)P=1
(�( try to send −→ P�0.9(©delivered) )
)P=1
(�( try to send −→ ¬start U delivered )
)P=1
(�( try to send −→ ¬start U delivered )
)P=1
(�( try to send −→ ¬start U delivered )
)
startstartstart try to sendtry to sendtry to send
delivereddelivereddelivered
lostlostlost
0.980.980.98
0.020.020.02
114 / 373
![Page 115: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/115.jpg)
Examples for PCTL*-specifications
communication protocol:
P=1
(�( try to send −→ P�0.9(©delivered) )
)P=1
(�( try to send −→ P�0.9(©delivered) )
)P=1
(�( try to send −→ P�0.9(©delivered) )
)P=1
(�( try to send −→ ¬start U delivered )
)P=1
(�( try to send −→ ¬start U delivered )
)P=1
(�( try to send −→ ¬start U delivered )
)
leader election protocol for nnn processes:
P=1
(♦ leader elected
)P=1
(♦ leader elected
)P=1
(♦ leader elected
)P�0.9
( ∨i�n©i leader elected
)P�0.9
( ∨i�n©i leader elected
)P�0.9
( ∨i�n©i leader elected
)
115 / 373
![Page 116: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/116.jpg)
PCTL* model checking for DTMC
116/373
![Page 117: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/117.jpg)
PCTL* model checking for DTMC
given: Markov chainM = (S ,P,AP, L, s0)M = (S ,P,AP, L, s0)M = (S ,P,AP, L, s0)
PCTL* state formula ΦΦΦ
task: check whether s0 |= Φs0 |= Φs0 |= Φ
117 / 373
![Page 118: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/118.jpg)
PCTL* model checking for DTMC
given: Markov chainM = (S ,P,AP, L, s0)M = (S ,P,AP, L, s0)M = (S ,P,AP, L, s0)
PCTL* state formula ΦΦΦ
task: check whether s0 |= Φs0 |= Φs0 |= Φ
main procedure as for CTL*:
recursively compute the satisfaction sets
Sat(Ψ)Sat(Ψ)Sat(Ψ) ==={s ∈ S : s |= Ψ
}{s ∈ S : s |= Ψ
}{s ∈ S : s |= Ψ
}for all state subformulas ΨΨΨ of ΦΦΦ
118 / 373
![Page 119: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/119.jpg)
Recursive computation of the satisfaction sets
119/ 373
![Page 120: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/120.jpg)
Recursive computation of the satisfaction sets
Sat(true)Sat(true)Sat(true) === SSS state space ofMMMSat(a)Sat(a)Sat(a) ===
{s ∈ S : a ∈ L(s)
}{s ∈ S : a ∈ L(s)
}{s ∈ S : a ∈ L(s)
}Sat(Φ1 ∧ Φ2)Sat(Φ1 ∧ Φ2)Sat(Φ1 ∧ Φ2) === Sat(Φ1) ∩ Sat(Φ2)Sat(Φ1) ∩ Sat(Φ2)Sat(Φ1) ∩ Sat(Φ2)
Sat(¬Φ)Sat(¬Φ)Sat(¬Φ) === S \ Sat(Φ)S \ Sat(Φ)S \ Sat(Φ)
120 / 373
![Page 121: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/121.jpg)
Recursive computation of the satisfaction sets
Sat(true)Sat(true)Sat(true) === SSS state space ofMMMSat(a)Sat(a)Sat(a) ===
{s ∈ S : a ∈ L(s)
}{s ∈ S : a ∈ L(s)
}{s ∈ S : a ∈ L(s)
}Sat(Φ1 ∧ Φ2)Sat(Φ1 ∧ Φ2)Sat(Φ1 ∧ Φ2) === Sat(Φ1) ∩ Sat(Φ2)Sat(Φ1) ∩ Sat(Φ2)Sat(Φ1) ∩ Sat(Φ2)
Sat(¬Φ)Sat(¬Φ)Sat(¬Φ) === S \ Sat(Φ)S \ Sat(Φ)S \ Sat(Φ)Sat(PI(ϕ) )Sat(PI(ϕ) )Sat(PI(ϕ) ) ===
{s ∈ S : PrMs (ϕ) ∈ I
}{s ∈ S : PrMs (ϕ) ∈ I
}{s ∈ S : PrMs (ϕ) ∈ I
}
121 / 373
![Page 122: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/122.jpg)
Recursive computation of the satisfaction sets
Sat(true)Sat(true)Sat(true) === SSS state space ofMMMSat(a)Sat(a)Sat(a) ===
{s ∈ S : a ∈ L(s)
}{s ∈ S : a ∈ L(s)
}{s ∈ S : a ∈ L(s)
}Sat(Φ1 ∧ Φ2)Sat(Φ1 ∧ Φ2)Sat(Φ1 ∧ Φ2) === Sat(Φ1) ∩ Sat(Φ2)Sat(Φ1) ∩ Sat(Φ2)Sat(Φ1) ∩ Sat(Φ2)
Sat(¬Φ)Sat(¬Φ)Sat(¬Φ) === S \ Sat(Φ)S \ Sat(Φ)S \ Sat(Φ)Sat(PI(ϕ) )Sat(PI(ϕ) )Sat(PI(ϕ) ) ===
{s ∈ S : PrMs (ϕ) ∈ I
}{s ∈ S : PrMs (ϕ) ∈ I
}{s ∈ S : PrMs (ϕ) ∈ I
}���special case: ϕ = ♦Φϕ = ♦Φϕ = ♦Φ
122 / 373
![Page 123: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/123.jpg)
Recursive computation of the satisfaction sets
Sat(true)Sat(true)Sat(true) === SSS state space ofMMMSat(a)Sat(a)Sat(a) ===
{s ∈ S : a ∈ L(s)
}{s ∈ S : a ∈ L(s)
}{s ∈ S : a ∈ L(s)
}Sat(Φ1 ∧ Φ2)Sat(Φ1 ∧ Φ2)Sat(Φ1 ∧ Φ2) === Sat(Φ1) ∩ Sat(Φ2)Sat(Φ1) ∩ Sat(Φ2)Sat(Φ1) ∩ Sat(Φ2)
Sat(¬Φ)Sat(¬Φ)Sat(¬Φ) === S \ Sat(Φ)S \ Sat(Φ)S \ Sat(Φ)Sat(PI(ϕ) )Sat(PI(ϕ) )Sat(PI(ϕ) ) ===
{s ∈ S : PrMs (ϕ) ∈ I
}{s ∈ S : PrMs (ϕ) ∈ I
}{s ∈ S : PrMs (ϕ) ∈ I
}���special case: ϕ = ♦Φϕ = ♦Φϕ = ♦Φ1. compute recursively Sat(Φ)Sat(Φ)Sat(Φ)
2. compute xs = PrMs (♦Φ)xs = PrMs (♦Φ)xs = PrMs (♦Φ) by solving alinear equation system
123 / 373
![Page 124: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/124.jpg)
Simulating a dice by a coin [Knuth]
• • •••• • •
• •• ••• •
• •• •• •
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
124 / 373
![Page 125: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/125.jpg)
Simulating a dice by a coin [Knuth]
• • •••• • •
• •• ••• •
• •• •• •
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
• •• •• •
probability for the outcome sixsixsix
PrM(♦ six ) =PrM(♦ six ) =PrM(♦ six ) = ?125/373
![Page 126: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/126.jpg)
Simulating a dice by a coin [Knuth]
outcome sixsixsix unreachable
• • •••• • •
• •• ••• •
• •• •• •
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
• •• •• •
126 / 373
![Page 127: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/127.jpg)
Simulating a dice by a coin [Knuth]
outcome sixsixsix unreachable,i.e., xs = 0xs = 0xs = 0
• • •••• • •
• •• ••• •
• •• •• •
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
• •• •• •
127 / 373
![Page 128: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/128.jpg)
Simulating a dice by a coin [Knuth]
outcome sixsixsix unreachable,i.e., xs = 0xs = 0xs = 0
• • •••• • •
• •• ••• •
• •• •• •
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
• •• •• •xsix = 1xsix = 1xsix = 1
128 / 373
![Page 129: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/129.jpg)
Simulating a dice by a coin [Knuth]
outcome sixsixsix unreachable,i.e., xs = 0xs = 0xs = 0
• • •••• • •
• •• ••• •
• •• •• •
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
• •• •• •xsix = 1xsix = 1xsix = 1
x1 =x1 =x1 = ?
x2 =x2 =x2 = ?
x3 =x3 =x3 = ?
129/373
![Page 130: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/130.jpg)
Simulating a dice by a coin [Knuth]
outcome sixsixsix unreachable,i.e., xs = 0xs = 0xs = 0
• • •••• • •
• •• ••• •
• •• •• •
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
• •• •• •xsix = 1xsix = 1xsix = 1
x1 =x1 =x1 =12x212x212x2
x2 =x2 =x2 = ?
x3 =x3 =x3 = ?
130/373
![Page 131: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/131.jpg)
Simulating a dice by a coin [Knuth]
outcome sixsixsix unreachable,i.e., xs = 0xs = 0xs = 0
• • •••• • •
• •• ••• •
• •• •• •
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
• •• •• •xsix = 1xsix = 1xsix = 1
x1 =x1 =x1 =12x212x212x2
x2 =x2 =x2 =12x312x312x3
x3 =x3 =x3 = ?
131/373
![Page 132: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/132.jpg)
Simulating a dice by a coin [Knuth]
outcome sixsixsix unreachable,i.e., xs = 0xs = 0xs = 0
• • •••• • •
• •• ••• •
• •• •• •
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
• •• •• •xsix = 1xsix = 1xsix = 1
x1 =x1 =x1 =12x212x212x2
x2 =x2 =x2 =12x312x312x3
x3x3x3 ===12x2 +
12
12x2 +
12
12x2 +
12
132 / 373
![Page 133: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/133.jpg)
Simulating a dice by a coin [Knuth]
• • •••• • •
• •• ••• •
• •• •• •
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
• •• •• •
1 −1
2 00 1 −1
2
0 −12 1
·
x1x2x3
=
0012
1 −1
2 00 1 −1
2
0 −12 1
·
x1x2x3
=
0012
1 −1
2 00 1 −1
2
0 −12 1
·
x1x2x3
=
0012
xsix = 1xsix = 1xsix = 1
x1 =x1 =x1 =12x212x212x2
x2 =x2 =x2 =12x312x312x3
x3x3x3 ===12x2 +
12
12x2 +
12
12x2 +
12
133 / 373
![Page 134: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/134.jpg)
Simulating a dice by a coin [Knuth]
• • •••• • •
• •• ••• •
• •• •• •
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
121212
• •• •• •
1 −1
2 00 1 −1
2
0 −12 1
·
x1x2x3
=
0012
1 −1
2 00 1 −1
2
0 −12 1
·
x1x2x3
=
0012
1 −1
2 00 1 −1
2
0 −12 1
·
x1x2x3
=
0012
PrM(♦ six )PrM(♦ six )PrM(♦ six )
= x1 =16
= x1 =16= x1 =16
xsix = 1xsix = 1xsix = 1
x1 =x1 =x1 =12x212x212x2 =
16
= 16= 16
x2 =x2 =x2 =12x312x312x3 =
13
= 13= 13
x3x3x3 ===12x2 +
12
12x2 +
12
12x2 +
12=== 2
32323
134 / 373
![Page 135: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/135.jpg)
Computing reachability probabilities
135/ 373
![Page 136: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/136.jpg)
Computing reachability probabilities
given: DTMCM = (S ,P, . . .)M = (S ,P, . . .)M = (S ,P, . . .) and T ⊆ ST ⊆ ST ⊆ S
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ Ss ∈ Ss ∈ S
♦T♦T♦T “eventually reaching TTT”
136 / 373
![Page 137: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/137.jpg)
Computing reachability probabilities
given: DTMCM = (S ,P, . . .)M = (S ,P, . . .)M = (S ,P, . . .) and T ⊆ ST ⊆ ST ⊆ S
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ Ss ∈ Ss ∈ S
1. compute S0S0S0 and S1S1S1
S0S0S0 ==={s ∈ S : xs = 0
}{s ∈ S : xs = 0
}{s ∈ S : xs = 0
}S1S1S1 ===
{s ∈ S : xs = 1
}{s ∈ S : xs = 1
}{s ∈ S : xs = 1
}2. . . .. . .. . .
♦T♦T♦T “eventually reaching TTT”
137 / 373
![Page 138: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/138.jpg)
Computing reachability probabilities
given: DTMCM = (S ,P, . . .)M = (S ,P, . . .)M = (S ,P, . . .) and T ⊆ ST ⊆ ST ⊆ S
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ Ss ∈ Ss ∈ S
1. compute S0S0S0 and S1S1S1
S0S0S0 ==={s ∈ S : xs = 0
}{s ∈ S : xs = 0
}{s ∈ S : xs = 0
}S1S1S1 ===
{s ∈ S : xs = 1
}{s ∈ S : xs = 1
}{s ∈ S : xs = 1
}2. . . .. . .. . .
TTTstate space SSS
138 / 373
![Page 139: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/139.jpg)
Computing reachability probabilities
given: DTMCM = (S ,P, . . .)M = (S ,P, . . .)M = (S ,P, . . .) and T ⊆ ST ⊆ ST ⊆ S
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ Ss ∈ Ss ∈ S
1. compute S0S0S0 and S1S1S1
S0S0S0 ==={s ∈ S : xs = 0
}{s ∈ S : xs = 0
}{s ∈ S : xs = 0
}S1S1S1 ===
{s ∈ S : xs = 1
}{s ∈ S : xs = 1
}{s ∈ S : xs = 1
}2. . . .. . .. . .
TTTstate space SSS
xs = 0xs = 0xs = 0 xs = 1xs = 1xs = 1139 / 373
![Page 140: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/140.jpg)
Computing reachability probabilities
given: DTMCM = (S ,P, . . .)M = (S ,P, . . .)M = (S ,P, . . .) and T ⊆ ST ⊆ ST ⊆ S
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ Ss ∈ Ss ∈ S
1. compute S0S0S0 and S1S1S1
S0S0S0 ==={s ∈ S : xs = 0
}{s ∈ S : xs = 0
}{s ∈ S : xs = 0
}===
{s : s �|= ∃♦T}{s : s �|= ∃♦T}{s : s �|= ∃♦T}
S1S1S1 ==={s ∈ S : xs = 1
}{s ∈ S : xs = 1
}{s ∈ S : xs = 1
}2. . . .. . .. . .
TTTstate space SSS
xs = 0xs = 0xs = 0 xs = 1xs = 1xs = 1140 / 373
![Page 141: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/141.jpg)
Computing reachability probabilities
given: DTMCM = (S ,P, . . .)M = (S ,P, . . .)M = (S ,P, . . .) and T ⊆ ST ⊆ ST ⊆ S
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ Ss ∈ Ss ∈ S
1. compute S0S0S0 and S1S1S1
S0S0S0 ==={s ∈ S : xs = 0
}{s ∈ S : xs = 0
}{s ∈ S : xs = 0
}===
{s : s �|= ∃♦T}{s : s �|= ∃♦T}{s : s �|= ∃♦T}
S1S1S1 ==={s ∈ S : xs = 1
}{s ∈ S : xs = 1
}{s ∈ S : xs = 1
}===
{s : s �|= ∃(¬T ) US0
}{s : s �|= ∃(¬T ) US0
}{s : s �|= ∃(¬T ) US0
}2. . . .. . .. . .
TTTstate space SSS
xs = 0xs = 0xs = 0 xs = 1xs = 1xs = 1141 / 373
![Page 142: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/142.jpg)
Computing reachability probabilities
given: DTMCM = (S ,P, . . .)M = (S ,P, . . .)M = (S ,P, . . .) and T ⊆ ST ⊆ ST ⊆ S
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ Ss ∈ Ss ∈ S
1. compute S0S0S0 and S1S1S1 ←−←−←− graph algorithms
S0S0S0 ==={s ∈ S : xs = 0
}{s ∈ S : xs = 0
}{s ∈ S : xs = 0
}===
{s : s �|= ∃♦T}{s : s �|= ∃♦T}{s : s �|= ∃♦T}
S1S1S1 ==={s ∈ S : xs = 1
}{s ∈ S : xs = 1
}{s ∈ S : xs = 1
}===
{s : s �|= ∃(¬T ) US0
}{s : s �|= ∃(¬T ) US0
}{s : s �|= ∃(¬T ) US0
}2. . . .. . .. . .
TTTstate space SSS
xs = 0xs = 0xs = 0 xs = 1xs = 1xs = 1142 / 373
![Page 143: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/143.jpg)
Computing reachability probabilities
given: DTMCM = (S ,P, . . .)M = (S ,P, . . .)M = (S ,P, . . .) and T ⊆ ST ⊆ ST ⊆ S
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ Ss ∈ Ss ∈ S
1. compute S0S0S0 and S1S1S1 ←−←−←− graph algorithms
S0S0S0 ==={s ∈ S : xs = 0
}{s ∈ S : xs = 0
}{s ∈ S : xs = 0
}===
{s : s �|= ∃♦T}{s : s �|= ∃♦T}{s : s �|= ∃♦T}
S1S1S1 ==={s ∈ S : xs = 1
}{s ∈ S : xs = 1
}{s ∈ S : xs = 1
}===
{s : s �|= ∃(¬T ) US0
}{s : s �|= ∃(¬T ) US0
}{s : s �|= ∃(¬T ) US0
}2. compute xsxsxs for s ∈s ∈s ∈ S? = S \ (S0 ∪ S1)S? = S \ (S0 ∪ S1)S? = S \ (S0 ∪ S1)
TTTstate space SSS
xs = 0xs = 0xs = 0 xs = 1xs = 1xs = 1
S?S?S?
143 / 373
![Page 144: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/144.jpg)
Computing reachability probabilities
given: DTMCM = (S ,P, . . .)M = (S ,P, . . .)M = (S ,P, . . .) and T ⊆ ST ⊆ ST ⊆ S
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ Ss ∈ Ss ∈ S
1. compute S0S0S0 and S1S1S1 ←−←−←− graph algorithms
S0S0S0 ==={s ∈ S : xs = 0
}{s ∈ S : xs = 0
}{s ∈ S : xs = 0
}===
{s : s �|= ∃♦T}{s : s �|= ∃♦T}{s : s �|= ∃♦T}
S1S1S1 ==={s ∈ S : xs = 1
}{s ∈ S : xs = 1
}{s ∈ S : xs = 1
}===
{s : s �|= ∃(¬T ) US0
}{s : s �|= ∃(¬T ) US0
}{s : s �|= ∃(¬T ) US0
}2. compute xsxsxs for s ∈s ∈s ∈ S? =
{s : 0 < xs < 1
}S? =
{s : 0 < xs < 1
}S? =
{s : 0 < xs < 1
}TTT
state space SSS
xs = 0xs = 0xs = 0 xs = 1xs = 1xs = 1
S?S?S?
144 / 373
![Page 145: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/145.jpg)
Computing reachability probabilities
given: DTMCM = (S ,P, . . .)M = (S ,P, . . .)M = (S ,P, . . .) and T ⊆ ST ⊆ ST ⊆ S
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ Ss ∈ Ss ∈ S
1. compute S0S0S0 and S1S1S1
S0S0S0 ==={s ∈ S : xs = 0
}{s ∈ S : xs = 0
}{s ∈ S : xs = 0
}===
{s : s �|= ∃♦T}{s : s �|= ∃♦T}{s : s �|= ∃♦T}
S1S1S1 ==={s ∈ S : xs = 1
}{s ∈ S : xs = 1
}{s ∈ S : xs = 1
}===
{s : s �|= ∃(¬T ) US0
}{s : s �|= ∃(¬T ) US0
}{s : s �|= ∃(¬T ) US0
}2. compute xsxsxs for s ∈s ∈s ∈ S? =
{s : 0 < xs < 1
}S? =
{s : 0 < xs < 1
}S? =
{s : 0 < xs < 1
}���by solving a linear equation system
145 / 373
![Page 146: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/146.jpg)
Computing reachability probabilities
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ S?s ∈ S?s ∈ S?
by solving the equation system:
xsxsxs ===∑s ′∈S?
P(s, s ′) · xs ′∑s ′∈S?
P(s, s ′) · xs ′∑s ′∈S?
P(s , s ′) · xs ′ +++ P(s , S1)P(s, S1)P(s, S1)
P(s, S1)P(s, S1)P(s, S1) ===∑u∈S1
P(s, u)∑u∈S1
P(s, u)∑u∈S1
P(s, u)
146 / 373
![Page 147: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/147.jpg)
Computing reachability probabilities
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ S?s ∈ S?s ∈ S?
by solving the equation system:
xsxsxs ===∑s ′∈S?
P(s, s ′) · xs ′∑s ′∈S?
P(s, s ′) · xs ′∑s ′∈S?
P(s , s ′) · xs ′ +++ P(s , S1)P(s, S1)P(s, S1)︸ ︷︷ ︸���probability for paths of the form
sss u1 u2 . . . uku1 u2 . . . uku1 u2 . . . uk ttt with t ∈ Tt ∈ Tt ∈ T︸ ︷︷ ︸uj ∈ S1uj ∈ S1uj ∈ S1
147 / 373
![Page 148: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/148.jpg)
Computing reachability probabilities
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ S?s ∈ S?s ∈ S?
by solving the equation system:
xsxsxs ===∑s ′∈S?
P(s, s ′) · xs ′∑s ′∈S?
P(s, s ′) · xs ′∑s ′∈S?
P(s , s ′) · xs ′ +++ P(s , S1)P(s, S1)P(s, S1)
︸ ︷︷ ︸���probability for paths of the form
sss s1 s2 . . . sms1 s2 . . . sms1 s2 . . . sm u1 u2 . . . uku1 u2 . . . uku1 u2 . . . uk ttt with t ∈ Tt ∈ Tt ∈ T︸ ︷︷ ︸si ∈ S?si ∈ S?si ∈ S?
︸ ︷︷ ︸uj ∈ S1uj ∈ S1uj ∈ S1 m � 1m � 1m � 1
148 / 373
![Page 149: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/149.jpg)
Computing reachability probabilities
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ S?s ∈ S?s ∈ S?
by solving the equation system:
xsxsxs ===∑s ′∈S?
P(s, s ′) · xs ′∑s ′∈S?
P(s, s ′) · xs ′∑s ′∈S?
P(s , s ′) · xs ′ +++ P(s , S1)P(s, S1)P(s, S1)
xxx === A · x + bA · x + bA · x + b
149 / 373
![Page 150: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/150.jpg)
Computing reachability probabilities
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ S?s ∈ S?s ∈ S?
by solving the equation system:
xsxsxs ===∑s ′∈S?
P(s, s ′) · xs ′∑s ′∈S?
P(s, s ′) · xs ′∑s ′∈S?
P(s , s ′) · xs ′ +++ P(s , S1)P(s, S1)P(s, S1)
xxx === A · x + bA · x + bA · x + bmatrix AAA ===
(P(s, s ′)
)s,s ′∈S?
(P(s, s ′)
)s ,s ′∈S?
(P(s, s ′)
)s,s ′∈S?
vectors xxx ===(xs)s∈S?
(xs)s∈S?
(xs)s∈S?
bbb ===(P(s, S1)
)s∈S?
(P(s, S1)
)s∈S?
(P(s, S1)
)s∈S?
150 / 373
![Page 151: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/151.jpg)
Computing reachability probabilities
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ S?s ∈ S?s ∈ S?
by solving the equation system:
xsxsxs ===∑s ′∈S?
P(s, s ′) · xs ′∑s ′∈S?
P(s, s ′) · xs ′∑s ′∈S?
P(s , s ′) · xs ′ +++ P(s , S1)P(s, S1)P(s, S1)
xxx === A · x + bA · x + bA · x + b
iff
(I− A) · x(I− A) · x(I− A) · x === bbb
matrix AAA ===(P(s, s ′)
)s,s ′∈S?
(P(s, s ′)
)s ,s ′∈S?
(P(s, s ′)
)s,s ′∈S?
vectors xxx ===(xs)s∈S?
(xs)s∈S?
(xs)s∈S?
bbb ===(P(s, S1)
)s∈S?
(P(s, S1)
)s∈S?
(P(s, S1)
)s∈S?
identity matrix III151 / 373
![Page 152: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/152.jpg)
Computing reachability probabilities
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ S?s ∈ S?s ∈ S?
by solving the equation system:
xsxsxs ===∑s ′∈S?
P(s, s ′) · xs ′∑s ′∈S?
P(s, s ′) · xs ′∑s ′∈S?
P(s , s ′) · xs ′ +++ P(s , S1)P(s, S1)P(s, S1)
xxx === A · x + bA · x + bA · x + b
iff
(I− A) · x(I− A) · x(I− A) · x === bbb
linear equation system withnon-singular matrix I− AI− AI− A
152 / 373
![Page 153: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/153.jpg)
Computing reachability probabilities
task: compute xsxsxs === PrMs (♦T )PrMs (♦T )PrMs (♦T ) for all s ∈ S?s ∈ S?s ∈ S?
by solving the equation system:
xsxsxs ===∑s ′∈S?
P(s, s ′) · xs ′∑s ′∈S?
P(s, s ′) · xs ′∑s ′∈S?
P(s , s ′) · xs ′ +++ P(s , S1)P(s, S1)P(s, S1)
xxx === A · x + bA · x + bA · x + b
iff
(I− A) · x(I− A) · x(I− A) · x === bbb
linear equation system withnon-singular matrix I− AI− AI− A���������
unique solution
153 / 373
![Page 154: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/154.jpg)
PCTL
sublogic of PCTL* where only path formulas of theform©Φ©Φ©Φ and Φ1UΦ2Φ1UΦ2Φ1UΦ2 are allowed
154 / 373
![Page 155: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/155.jpg)
PCTL
sublogic of PCTL* where only path formulas of theform©Φ©Φ©Φ and Φ1UΦ2Φ1UΦ2Φ1UΦ2 are allowed
state formulas:
ΦΦΦ ::=::=::= true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ PI(ϕ)true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ PI(ϕ)true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ PI(ϕ)
path formulas:ϕϕϕ ::=::=::= ©Φ
∣∣ Φ1UΦ2©Φ∣∣ Φ1UΦ2©Φ∣∣ Φ1UΦ2
155 / 373
![Page 156: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/156.jpg)
PCTL
sublogic of PCTL* where only path formulas of theform©Φ©Φ©Φ and Φ1UΦ2Φ1UΦ2Φ1UΦ2 are allowed
state formulas:
ΦΦΦ ::=::=::= true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ PI(ϕ)true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ PI(ϕ)true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ PI(ϕ)
path formulas:ϕϕϕ ::=::=::= ©Φ
∣∣ Φ1UΦ2©Φ∣∣ Φ1UΦ2©Φ∣∣ Φ1UΦ2
∣∣ ♦Φ ∣∣ �Φ∣∣ ♦Φ ∣∣ �Φ∣∣ ♦Φ ∣∣ �ΦPI(♦Φ)PI(♦Φ)PI(♦Φ) def
=def=def= PI(true UΦ)PI(true UΦ)PI(true UΦ)
156 / 373
![Page 157: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/157.jpg)
PCTL
sublogic of PCTL* where only path formulas of theform©Φ©Φ©Φ and Φ1UΦ2Φ1UΦ2Φ1UΦ2 are allowed
state formulas:
ΦΦΦ ::=::=::= true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ PI(ϕ)true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ PI(ϕ)true∣∣ a ∣∣ Φ1 ∧ Φ2
∣∣ ¬Φ ∣∣ PI(ϕ)
path formulas:ϕϕϕ ::=::=::= ©Φ
∣∣ Φ1UΦ2©Φ∣∣ Φ1UΦ2©Φ∣∣ Φ1UΦ2
∣∣ ♦Φ ∣∣ �Φ∣∣ ♦Φ ∣∣ �Φ∣∣ ♦Φ ∣∣ �ΦPI(♦Φ)PI(♦Φ)PI(♦Φ) def
=def=def= PI(true UΦ)PI(true UΦ)PI(true UΦ)
e.g., P<0.4(�Φ)P<0.4(�Φ)P<0.4(�Φ) def=def=def= P>0.6(♦¬Φ)P>0.6(♦¬Φ)P>0.6(♦¬Φ)
note: PrM(s,�Φ)PrM(s,�Φ)PrM(s,�Φ) === 1− PrM(s,♦¬Φ)1− PrM(s,♦¬Φ)1− PrM(s,♦¬Φ)157 / 373
![Page 158: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/158.jpg)
PCTL model checking
given: Markov chainM = (S ,P,AP, L, s0)M = (S ,P,AP , L, s0)M = (S ,P,AP, L, s0)PCTL state formula ΦΦΦ
task: check whether s0 |= Φs0 |= Φs0 |= Φ
158 / 373
![Page 159: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/159.jpg)
PCTL model checking
given: Markov chainM = (S ,P,AP, L, s0)M = (S ,P,AP , L, s0)M = (S ,P,AP, L, s0)PCTL state formula ΦΦΦ
task: check whether s0 |= Φs0 |= Φs0 |= Φ
recursive computation of Sat(Ψ) ={s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}for all state subformulas ΨΨΨ of ΦΦΦ
in bottom-up manner, i.e.,inner subformulas first
159 / 373
![Page 160: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/160.jpg)
PCTL model checking
given: Markov chainM = (S ,P,AP, L, s0)M = (S ,P,AP , L, s0)M = (S ,P,AP, L, s0)PCTL state formula ΦΦΦ
task: check whether s0 |= Φs0 |= Φs0 |= Φ
recursive computation of Sat(Ψ) ={s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}for all state subformulas ΨΨΨ of ΦΦΦ
• treatment of propositional logic fragment:
160 / 373
![Page 161: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/161.jpg)
PCTL model checking
given: Markov chainM = (S ,P,AP, L, s0)M = (S ,P,AP , L, s0)M = (S ,P,AP, L, s0)PCTL state formula ΦΦΦ
task: check whether s0 |= Φs0 |= Φs0 |= Φ
recursive computation of Sat(Ψ) ={s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}for all state subformulas ΨΨΨ of ΦΦΦ
• treatment of propositional logic fragment:√√√
Sat(true)Sat(true)Sat(true) === SSS
Sat(a)Sat(a)Sat(a) ==={s ∈ S : a ∈ L(s)
}{s ∈ S : a ∈ L(s)
}{s ∈ S : a ∈ L(s)
}Sat(¬Ψ)Sat(¬Ψ)Sat(¬Ψ) === S \ Sat(Ψ)S \ Sat(Ψ)S \ Sat(Ψ)
Sat(Ψ1∧Ψ2)Sat(Ψ1∧Ψ2)Sat(Ψ1 ∧Ψ2) === Sat(Ψ1) ∩ Sat(Ψ2)Sat(Ψ1) ∩ Sat(Ψ2)Sat(Ψ1) ∩ Sat(Ψ2)
161 / 373
![Page 162: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/162.jpg)
PCTL model checking
given: Markov chainM = (S ,P,AP, L, s0)M = (S ,P,AP , L, s0)M = (S ,P,AP, L, s0)PCTL state formula ΦΦΦ
task: check whether s0 |= Φs0 |= Φs0 |= Φ
recursive computation of Sat(Ψ) ={s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}for all state subformulas ΨΨΨ of ΦΦΦ
• treatment of propositional logic fragment:√√√
• treatment of the probability operator PI(ϕ)PI(ϕ)PI(ϕ)
162 / 373
![Page 163: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/163.jpg)
PCTL model checking
given: Markov chainM = (S ,P,AP, L, s0)M = (S ,P,AP , L, s0)M = (S ,P,AP, L, s0)PCTL state formula ΦΦΦ
task: check whether s0 |= Φs0 |= Φs0 |= Φ
recursive computation of Sat(Ψ) ={s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}for all state subformulas ΨΨΨ of ΦΦΦ
• treatment of propositional logic fragment:√√√
• treatment of the probability operator PI(ϕ)PI(ϕ)PI(ϕ)
compute PrMs (ϕ)PrMs (ϕ)PrMs (ϕ) for all states sss and return
Sat(PI(ϕ)
)=
{s ∈ S : PrMs (ϕ) ∈ I
}Sat
(PI(ϕ)
)=
{s ∈ S : PrMs (ϕ) ∈ I
}Sat
(PI(ϕ)
)=
{s ∈ S : PrMs (ϕ) ∈ I
}163 / 373
![Page 164: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/164.jpg)
PCTL model checking
given: Markov chainM = (S ,P,AP, L, s0)M = (S ,P,AP , L, s0)M = (S ,P,AP, L, s0)PCTL state formula ΦΦΦ
task: check whether s0 |= Φs0 |= Φs0 |= Φ
recursive computation of Sat(Ψ) ={s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}for all state subformulas ΨΨΨ of ΦΦΦ
• treatment of propositional logic fragment:√√√
• treatment of the probability operator PI(ϕ)PI(ϕ)PI(ϕ)
graph algorithms +++ matrix/vector operations���next: matrix/vector multiplication
until: linear equation system164 / 373
![Page 165: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/165.jpg)
PCTL model checking
given: Markov chainM = (S ,P,AP, L, s0)M = (S ,P,AP , L, s0)M = (S ,P,AP, L, s0)PCTL state formula ΦΦΦ
task: check whether s0 |= Φs0 |= Φs0 |= Φ
recursive computation of Sat(Ψ) ={s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}for all state subformulas ΨΨΨ of ΦΦΦ
• treatment of propositional logic fragment:√√√
• treatment of the probability operator PI(ϕ)PI(ϕ)PI(ϕ)
graph algorithms +++ matrix/vector operations
time complexity: O( poly(M) · |Φ| )O( poly(M) · |Φ| )O( poly(M) · |Φ| )165 / 373
![Page 166: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/166.jpg)
PCTL* model checking
given: Markov chainM = (S ,P,AP, L, s0)M = (S ,P,AP , L, s0)M = (S ,P,AP, L, s0)PCTL* state formula ΦΦΦ
task: check whether s0 |= Φs0 |= Φs0 |= Φ
recursive computation of Sat(Ψ) ={s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}for all state subformulas ΨΨΨ of ΦΦΦ
• treatment of propositional logic fragment:√√√
• treatment of the probability operator PI(ϕ)PI(ϕ)PI(ϕ)
PCTL* path formula ϕϕϕ ��� LTL formula ϕ′ϕ′ϕ′���path formula withoutprobability operator
166 / 373
![Page 167: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/167.jpg)
PCTL* model checking
given: Markov chainM = (S ,P,AP, L, s0)M = (S ,P,AP , L, s0)M = (S ,P,AP, L, s0)PCTL* state formula ΦΦΦ
task: check whether s0 |= Φs0 |= Φs0 |= Φ
recursive computation of Sat(Ψ) ={s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}Sat(Ψ) =
{s ∈ S : s |= Ψ
}for all state subformulas ΨΨΨ of ΦΦΦ
• treatment of propositional logic fragment:√√√
• treatment of the probability operator PI(ϕ)PI(ϕ)PI(ϕ)
PCTL* path formula ϕϕϕ ��� LTL formula ϕ′ϕ′ϕ′
... automata-based approach for ϕ′ϕ′ϕ′ ...
167 / 373
![Page 168: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/168.jpg)
PCTL* model checking
given: Markov chainM = (S ,P,AP, L, s0)M = (S ,P,AP , L, s0)M = (S ,P,AP, L, s0)PCTL* state formula ΦΦΦ
task: check whether s0 |= Φs0 |= Φs0 |= Φ
treatment of the probability operator PI(ϕ)PI(ϕ)PI(ϕ)
PCTL* path formula ϕϕϕ��� LTL formula ϕ′ϕ′ϕ′
by replacing each maximal state subformulawith a fresh atomic proposition
168 / 373
![Page 169: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/169.jpg)
PCTL* model checking
given: Markov chainM = (S ,P,AP, L, s0)M = (S ,P,AP , L, s0)M = (S ,P,AP, L, s0)PCTL* state formula ΦΦΦ
task: check whether s0 |= Φs0 |= Φs0 |= Φ
treatment of the probability operator PI(ϕ)PI(ϕ)PI(ϕ)
PCTL* path formula ϕϕϕ��� LTL formula ϕ′ϕ′ϕ′
by replacing each maximal state subformulawith a fresh atomic proposition
♦(aU♦(aU♦(aU P�0.7(�♦b)P�0.7(�♦b)P�0.7(�♦b) ∧ �∧ �∧ � P<0.3(©�c)P<0.3(©�c)P<0.3(©�c)
)))
169 / 373
![Page 170: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/170.jpg)
PCTL* model checking
given: Markov chainM = (S ,P,AP, L, s0)M = (S ,P,AP , L, s0)M = (S ,P,AP, L, s0)PCTL* state formula ΦΦΦ
task: check whether s0 |= Φs0 |= Φs0 |= Φ
treatment of the probability operator PI(ϕ)PI(ϕ)PI(ϕ)
PCTL* path formula ϕϕϕ��� LTL formula ϕ′ϕ′ϕ′
by replacing each maximal state subformulawith a fresh atomic proposition
♦(aU♦(aU♦(aU P�0.7(�♦b)P�0.7(�♦b)P�0.7(�♦b) ∧ �∧ �∧ � P<0.3(©�c)P<0.3(©�c)P<0.3(©�c)
)))
170 / 373
![Page 171: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/171.jpg)
PCTL* model checking
given: Markov chainM = (S ,P,AP, L, s0)M = (S ,P,AP , L, s0)M = (S ,P,AP, L, s0)PCTL* state formula ΦΦΦ
task: check whether s0 |= Φs0 |= Φs0 |= Φ
treatment of the probability operator PI(ϕ)PI(ϕ)PI(ϕ)
PCTL* path formula ϕϕϕ��� LTL formula ϕ′ϕ′ϕ′
by replacing each maximal state subformulawith a fresh atomic proposition
♦(aU♦(aU♦(aU P�0.7(�♦b)P�0.7(�♦b)P�0.7(�♦b) ∧ �∧ �∧ � P<0.3(©�c)P<0.3(©�c)P<0.3(©�c)
)))���������♦(aU d ∧ � e
)♦(aU d ∧ � e
)♦(aU d ∧ � e
)171 / 373
![Page 172: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/172.jpg)
pctl-315
PCTL* formula PI(ϕ)PI(ϕ)PI(ϕ)
MarkovchainMMM
probabilistic model checker
probability that ϕϕϕ holds forMMM172 / 373
![Page 173: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/173.jpg)
pctl-315
PCTL* formula PI(ϕ)PI(ϕ)PI(ϕ)
LTL formula ϕ′ϕ′ϕ′MarkovchainMMM
probabilistic model checker
probability that ϕϕϕ holds forMMM173 / 373
![Page 174: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/174.jpg)
pctl-315
PCTL* formula PI(ϕ)PI(ϕ)PI(ϕ)
LTL formula ϕ′ϕ′ϕ′
automaton AAA for ϕ′ϕ′ϕ′
MarkovchainMMM
probabilistic model checker
probability that ϕϕϕ holds forMMM174 / 373
![Page 175: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/175.jpg)
pctl-315
PCTL* formula PI(ϕ)PI(ϕ)PI(ϕ)
LTL formula ϕ′ϕ′ϕ′
deterministicautomaton AAA for ϕ′ϕ′ϕ′
MarkovchainMMM
probabilistic model checker
probability that ϕϕϕ holds forMMM175 / 373
![Page 176: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/176.jpg)
pctl-315
PCTL* formula PI(ϕ)PI(ϕ)PI(ϕ)
LTL formula ϕ′ϕ′ϕ′
deterministic Rabinautomaton AAA for ϕ′ϕ′ϕ′
MarkovchainMMM
probabilistic model checker
probability that ϕϕϕ holds forMMM176 / 373
![Page 177: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/177.jpg)
Deterministic Rabin automata (DRA)
177/373
![Page 178: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/178.jpg)
Deterministic Rabin automata (DRA)
A DRA is a tuple AAA === (Q,Σ, δ, q0,Acc)(Q,Σ, δ, q0,Acc)(Q,Σ, δ, q0,Acc) where
• QQQ finite state space
• q0 ∈ Qq0 ∈ Qq0 ∈ Q initial state
• ΣΣΣ alphabet
• δ : Q × Σ −→ Qδ : Q × Σ −→ Qδ : Q × Σ −→ Q deterministic transition function
178 / 373
![Page 179: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/179.jpg)
Deterministic Rabin automata (DRA)
A DRA is a tuple AAA === (Q,Σ, δ, q0,Acc)(Q,Σ, δ, q0,Acc)(Q,Σ, δ, q0,Acc) where
• QQQ finite state space
• q0 ∈ Qq0 ∈ Qq0 ∈ Q initial state
• ΣΣΣ alphabet
• δ : Q × Σ −→ Qδ : Q × Σ −→ Qδ : Q × Σ −→ Q deterministic transition function
• acceptance condition AccAccAcc is a set of pairs (L,U)(L,U)(L,U)with L,U ⊆ QL,U ⊆ QL,U ⊆ Q
179 / 373
![Page 180: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/180.jpg)
Deterministic Rabin automata (DRA)
A DRA is a tuple AAA === (Q,Σ, δ, q0,Acc)(Q,Σ, δ, q0,Acc)(Q,Σ, δ, q0,Acc) where
• QQQ finite state space
• q0 ∈ Qq0 ∈ Qq0 ∈ Q initial state
• ΣΣΣ alphabet
• δ : Q × Σ −→ Qδ : Q × Σ −→ Qδ : Q × Σ −→ Q deterministic transition function
• acceptance condition AccAccAcc is a set of pairs (L,U)(L,U)(L,U)with L,U ⊆ QL,U ⊆ QL,U ⊆ Q, say Acc =
{(L1,U1), ..., (Lk,Uk)
}Acc =
{(L1,U1), ..., (Lk,Uk)
}Acc =
{(L1,U1), ..., (Lk,Uk)
}semantics of the acceptance condition:∨
1�i�k
(♦�¬Li ∧ �♦Ui
)∨1�i�k
(♦�¬Li ∧ �♦Ui
)∨1�i�k
(♦�¬Li ∧ �♦Ui
)180 / 373
![Page 181: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/181.jpg)
Accepted language of a DRA
A DRA is a tuple AAA === (Q,Σ, δ, q0,Acc)(Q,Σ, δ, q0,Acc)(Q,Σ, δ, q0,Acc) where
AccAccAcc ==={(L1,U1), . . . , (Lk ,Uk)
}{(L1,U1), . . . , (Lk ,Uk)
}{(L1,U1), . . . , (Lk,Uk)
}LiLiLi , Ui ⊆ QUi ⊆ QUi ⊆ Q
accepted language:
Lω(A)Lω(A)Lω(A) ==={{{σ ∈ Σωσ ∈ Σωσ ∈ Σω: the run for σσσ in AAA fulfills AccAccAcc
}}}
181 / 373
![Page 182: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/182.jpg)
Accepted language of a DRA
A DRA is a tuple AAA === (Q,Σ, δ, q0,Acc)(Q,Σ, δ, q0,Acc)(Q,Σ, δ, q0,Acc) where
AccAccAcc ==={(L1,U1), . . . , (Lk ,Uk)
}{(L1,U1), . . . , (Lk ,Uk)
}{(L1,U1), . . . , (Lk,Uk)
}LiLiLi , Ui ⊆ QUi ⊆ QUi ⊆ Q
accepted language:
Lω(A)Lω(A)Lω(A) ==={{{σ ∈ Σωσ ∈ Σωσ ∈ Σω: the run for σσσ in AAA fulfills AccAccAcc
}}}Let ρ = q0 q1 q2 . . .ρ = q0 q1 q2 . . .ρ = q0 q1 q2 . . . be the run for some infinite word σσσ.
ρρρ fulfills AccAccAcc iff
∃i ∈ {1, . . . , k}. inf(ρ) ∩ Li = ∅ ∧ inf(ρ) ∩ Ui �= ∅∃i ∈ {1, . . . , k}. inf(ρ) ∩ Li = ∅ ∧ inf(ρ) ∩ Ui �= ∅∃i ∈ {1, . . . , k}. inf(ρ) ∩ Li = ∅ ∧ inf(ρ) ∩ Ui �= ∅
where inf(ρ)inf(ρ)inf(ρ) ==={q ∈ Q :
∞∃ � ∈ N. q = q�
}{q ∈ Q :
∞∃ � ∈ N. q = q�
}{q ∈ Q :
∞∃ � ∈ N. q = q�
}182 / 373
![Page 183: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/183.jpg)
Example: DRA
q0q0q0 q1q1q1
BBB AAA
AAA
BBB
Acc ={({q0}, {q1})
}Acc =
{({q0}, {q1})
}Acc =
{({q0}, {q1})
}
183 / 373
![Page 184: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/184.jpg)
Example: DRA
q0q0q0 q1q1q1
BBB AAA
AAA
BBB
Acc ={({q0}, {q1})
}Acc =
{({q0}, {q1})
}Acc =
{({q0}, {q1})
}= ♦�¬q0 ∧ �♦q1= ♦�¬q0 ∧ �♦q1= ♦�¬q0 ∧ �♦q1
♦�♦�♦� “eventually forever”�♦�♦�♦ “infinitely often”
184 / 373
![Page 185: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/185.jpg)
Example: DRA
q0q0q0 q1q1q1
BBB AAA
AAA
BBB
Acc ={({q0}, {q1})
}Acc =
{({q0}, {q1})
}Acc =
{({q0}, {q1})
}= ♦�¬q0 ∧ �♦q1= ♦�¬q0 ∧ �♦q1= ♦�¬q0 ∧ �♦q1
accepted language: (A+ B)∗Aω(A+ B)∗Aω(A+ B)∗Aω
♦�♦�♦� “eventually forever”�♦�♦�♦ “infinitely often”
185 / 373
![Page 186: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/186.jpg)
Example: DRA
q0q0q0 q1q1q1
BBB AAA
AAA
BBB
Acc ={({q0}, {q1})
}Acc =
{({q0}, {q1})
}Acc =
{({q0}, {q1})
}= ♦�¬q0 ∧ �♦q1= ♦�¬q0 ∧ �♦q1= ♦�¬q0 ∧ �♦q1
accepted language: (A+ B)∗Aω(A+ B)∗Aω(A+ B)∗Aω
q0q0q0 q1q1q1
BBB AAA
AAA
BBB
Acc ={(∅, {q1})
}Acc =
{(∅, {q1})
}Acc =
{(∅, {q1})
}= �♦q1= �♦q1= �♦q1
�♦�♦�♦ “infinitely often”186 / 373
![Page 187: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/187.jpg)
Example: DRA
q0q0q0 q1q1q1
BBB AAA
AAA
BBB
Acc ={({q0}, {q1})
}Acc =
{({q0}, {q1})
}Acc =
{({q0}, {q1})
}= ♦�¬q0 ∧ �♦q1= ♦�¬q0 ∧ �♦q1= ♦�¬q0 ∧ �♦q1
accepted language: (A+ B)∗Aω(A+ B)∗Aω(A+ B)∗Aω
q0q0q0 q1q1q1
BBB AAA
AAA
BBB
Acc ={(∅, {q1})
}Acc =
{(∅, {q1})
}Acc =
{(∅, {q1})
}= �♦q1= �♦q1= �♦q1
accepted language: (B∗A)ω(B∗A)ω(B∗A)ω187 / 373
![Page 188: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/188.jpg)
Fundamental result: LTL-2-DRA
188/373
![Page 189: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/189.jpg)
Fundamental result: LTL-2-DRA
For each LTL formula ϕϕϕ over APAPAP there exists aDRA AAA with the alphabet Σ = 2APΣ = 2APΣ = 2AP s.t.
Lω(A)Lω(A)Lω(A) ==={σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}
189 / 373
![Page 190: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/190.jpg)
Fundamental result: LTL-2-DRA
For each LTL formula ϕϕϕ over APAPAP there exists aDRA AAA with the alphabet Σ = 2APΣ = 2APΣ = 2AP s.t.
Lω(A)Lω(A)Lω(A) ==={σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}LTL formula
NBA
DRA
determinization[Safra’88]
190 / 373
![Page 191: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/191.jpg)
Fundamental result: LTL-2-DRA
For each LTL formula ϕϕϕ over APAPAP there exists aDRA AAA with the alphabet Σ = 2APΣ = 2APΣ = 2AP s.t.
Lω(A)Lω(A)Lω(A) ==={σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}LTL formula
NBA
DRA
determinization[Safra’88]
LTL formula
DRA
compositional[Esparza/Kretinsky’14]
191 / 373
![Page 192: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/192.jpg)
Fundamental result: LTL-2-DRA
For each LTL formula ϕϕϕ over APAPAP there exists aDRA AAA with the alphabet Σ = 2APΣ = 2APΣ = 2AP s.t.
Lω(A)Lω(A)Lω(A) ==={σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}
Example: AP = {a, b}AP = {a, b}AP = {a, b}
192 / 373
![Page 193: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/193.jpg)
Fundamental result: LTL-2-DRA
For each LTL formula ϕϕϕ over APAPAP there exists aDRA AAA with the alphabet Σ = 2APΣ = 2APΣ = 2AP s.t.
Lω(A)Lω(A)Lω(A) ==={σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}
Example: AP = {a, b}AP = {a, b}AP = {a, b} ��� Σ ={∅, {a}, {b}, {a, b}
}Σ =
{∅, {a}, {b}, {a, b}
}Σ =
{∅, {a}, {b}, {a, b}
}
193 / 373
![Page 194: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/194.jpg)
Fundamental result: LTL-2-DRA
For each LTL formula ϕϕϕ over APAPAP there exists aDRA AAA with the alphabet Σ = 2APΣ = 2APΣ = 2AP s.t.
Lω(A)Lω(A)Lω(A) ==={σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}
Example: AP = {a, b}AP = {a, b}AP = {a, b} ��� Σ ={∅, {a}, {b}, {a, b}
}Σ =
{∅, {a}, {b}, {a, b}
}Σ =
{∅, {a}, {b}, {a, b}
}
q0q0q0 q1q1q1
¬a ∨ b¬a ∨ b¬a ∨ b a ∧ ¬ba ∧ ¬ba ∧ ¬b
a ∧ ¬ba ∧ ¬ba ∧ ¬b
¬a ∨ b¬a ∨ b¬a ∨ bacceptance condition:♦�¬q0 ∧ �♦q1♦�¬q0 ∧ �♦q1♦�¬q0 ∧ �♦q1
194 / 373
![Page 195: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/195.jpg)
Fundamental result: LTL-2-DRA
For each LTL formula ϕϕϕ over APAPAP there exists aDRA AAA with the alphabet Σ = 2APΣ = 2APΣ = 2AP s.t.
Lω(A)Lω(A)Lω(A) ==={σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}
Example: AP = {a, b}AP = {a, b}AP = {a, b} ��� Σ ={∅, {a}, {b}, {a, b}
}Σ =
{∅, {a}, {b}, {a, b}
}Σ =
{∅, {a}, {b}, {a, b}
}
q0q0q0 q1q1q1
¬a ∨ b¬a ∨ b¬a ∨ b a ∧ ¬ba ∧ ¬ba ∧ ¬b
a ∧ ¬ba ∧ ¬ba ∧ ¬b
¬a ∨ b¬a ∨ b¬a ∨ bacceptance condition:♦�¬q0 ∧ �♦q1♦�¬q0 ∧ �♦q1♦�¬q0 ∧ �♦q1LTL formula ♦�(a ∧ ¬b)♦�(a ∧ ¬b)♦�(a ∧ ¬b)
195 / 373
![Page 196: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/196.jpg)
Fundamental result: LTL-2-DRA
For each LTL formula ϕϕϕ over APAPAP there exists aDRA AAA with the alphabet Σ = 2APΣ = 2APΣ = 2AP s.t.
Lω(A)Lω(A)Lω(A) ==={σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}
Example: AP = {a, b}AP = {a, b}AP = {a, b} ��� Σ ={∅, {a}, {b}, {a, b}
}Σ =
{∅, {a}, {b}, {a, b}
}Σ =
{∅, {a}, {b}, {a, b}
}
q0q0q0 q1q1q1
¬a¬a¬a ¬b ∨ a¬b ∨ a¬b ∨ a
aaa
b ∧ ¬ab ∧ ¬ab ∧ ¬aacceptance condition:♦�¬q1 ∧ �♦q0♦�¬q1 ∧ �♦q0♦�¬q1 ∧ �♦q0
196 / 373
![Page 197: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/197.jpg)
Fundamental result: LTL-2-DRA
For each LTL formula ϕϕϕ over APAPAP there exists aDRA AAA with the alphabet Σ = 2APΣ = 2APΣ = 2AP s.t.
Lω(A)Lω(A)Lω(A) ==={σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}{σ ∈ Σω : σ |= ϕ
}
Example: AP = {a, b}AP = {a, b}AP = {a, b} ��� Σ ={∅, {a}, {b}, {a, b}
}Σ =
{∅, {a}, {b}, {a, b}
}Σ =
{∅, {a}, {b}, {a, b}
}
q0q0q0 q1q1q1
¬a¬a¬a ¬b ∨ a¬b ∨ a¬b ∨ a
aaa
b ∧ ¬ab ∧ ¬ab ∧ ¬aacceptance condition:♦�¬q1 ∧ �♦q0♦�¬q1 ∧ �♦q0♦�¬q1 ∧ �♦q0LTL formula�(a→ ♦(b ∧ ¬a)) ∧ ♦�¬a�(a→ ♦(b ∧ ¬a)) ∧ ♦�¬a�(a→ ♦(b ∧ ¬a)) ∧ ♦�¬a
197 / 373
![Page 198: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/198.jpg)
PCTL* model checking
198/373
![Page 199: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/199.jpg)
PCTL* model checking
PCTL* formula PI(ϕ)PI(ϕ)PI(ϕ)
LTL formula ϕ′ϕ′ϕ′
deterministic Rabinautomaton AAA for ϕ′ϕ′ϕ′
MarkovchainMMM
probabilistic model checker
probability that ϕϕϕ holds forMMM199 / 373
![Page 200: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/200.jpg)
PCTL* model checking
PCTL* formula PI(ϕ)PI(ϕ)PI(ϕ)
LTL formula ϕ′ϕ′ϕ′
deterministic Rabinautomaton AAA for ϕ′ϕ′ϕ′
MarkovchainMMM
probabilistic model checker
quantitative analysis inM×AM×AM×A
probability that ϕϕϕ holds forMMM200 / 373
![Page 201: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/201.jpg)
Product of a Markov chain and a DRA
201/373
![Page 202: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/202.jpg)
Product of a Markov chain and a DRA
given: Markov chainM = (S ,P,AP, L)M = (S ,P,AP, L)M = (S ,P,AP, L)
DRA A = (Q, 2AP , δ, q0,Acc)A = (Q, 2AP , δ, q0,Acc)A = (Q, 2AP , δ, q0,Acc)
goal: define a Markov chainM×AM×AM×A
202 / 373
![Page 203: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/203.jpg)
Product of a Markov chain and a DRA
given: Markov chainM = (S ,P,AP, L)M = (S ,P,AP, L)M = (S ,P,AP, L)
DRA A = (Q, 2AP , δ, q0,Acc)A = (Q, 2AP , δ, q0,Acc)A = (Q, 2AP , δ, q0,Acc)
goal: define a Markov chainM×AM×AM×A s.t.
PrMs (A)PrMs (A)PrMs (A) === PrM{π ∈ Paths(s) : trace(π) ∈ Lω(A)
}PrM
{π ∈ Paths(s) : trace(π) ∈ Lω(A)
}PrM
{π ∈ Paths(s) : trace(π) ∈ Lω(A)
}can be derived by a probabilistic reachability analysisin the product-chainM×AM×AM×A
trace(s0 s1 s2 . . .)trace(s0 s1 s2 . . .)trace(s0 s1 s2 . . .) === L(s0) L(s1) L(s2) . . . ∈(2AP
)ωL(s0) L(s1) L(s2) . . . ∈
(2AP
)ωL(s0) L(s1) L(s2) . . . ∈
(2AP
)ω203 / 373
![Page 204: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/204.jpg)
Product of a Markov chain and a DRA
given: Markov chainM = (S ,P,AP, L)M = (S ,P,AP, L)M = (S ,P,AP, L)
DRA A = (Q, 2AP , δ, q0,Acc)A = (Q, 2AP , δ, q0,Acc)A = (Q, 2AP , δ, q0,Acc)
idea: define a Markov chainM×AM×AM×A s.t. . . .. . .. . .
path πππinMMMs0s0s0
s1s1s1
s2s2s2
...
...
...204 / 373
![Page 205: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/205.jpg)
Product of a Markov chain and a DRA
given: Markov chainM = (S ,P,AP, L)M = (S ,P,AP, L)M = (S ,P,AP, L)
DRA A = (Q, 2AP , δ, q0,Acc)A = (Q, 2AP , δ, q0,Acc)A = (Q, 2AP , δ, q0,Acc)
idea: define a Markov chainM×AM×AM×A s.t. . . .. . .. . .
path πππinMMMs0s0s0
s1s1s1
s2s2s2
...
...
...
run for trace(π)trace(π)trace(π)in AAAq0q0q0
q1q1q1
q2q2q2
...
...
...
L(s0)L(s0)L(s0)
L(s1)L(s1)L(s1)
L(s2)L(s2)L(s2)
205 / 373
![Page 206: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/206.jpg)
Product of a Markov chain and a DRA
given: Markov chainM = (S ,P,AP, L)M = (S ,P,AP, L)M = (S ,P,AP, L)
DRA A = (Q, 2AP , δ, q0,Acc)A = (Q, 2AP , δ, q0,Acc)A = (Q, 2AP , δ, q0,Acc)
idea: define a Markov chainM×AM×AM×A s.t. . . .. . .. . .
path πππinMMMs0s0s0
s1s1s1
s2s2s2
...
...
...
run for trace(π)trace(π)trace(π)in AAAq0q0q0
q1q1q1
q2q2q2
...
...
...
L(s0)L(s0)L(s0)
L(s1)L(s1)L(s1)
L(s2)L(s2)L(s2)
path inM×AM×AM×A
〈s0, q1〉〈s0, q1〉〈s0, q1〉〈s1, q2〉〈s1, q2〉〈s1, q2〉〈s2, q3〉〈s2, q3〉〈s2, q3〉
206 / 373
![Page 207: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/207.jpg)
Fundamental property of the product
207/ 373
![Page 208: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/208.jpg)
Fundamental property of the product
given: Markov chainMMM and DRA AAA where
AccAccAcc ==={(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}{(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}{(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}
208 / 373
![Page 209: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/209.jpg)
Fundamental property of the product
given: Markov chainMMM and DRA AAA where
AccAccAcc ==={(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}{(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}{(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}For each state sss inMMM, let qsqsqs === δ
(q0, L(s)
)δ(q0, L(s)
)δ(q0, L(s)
).�
successor state in AAA of theinitial DRA-state q0q0q0 for theinput symbol L(s) ∈ 2APL(s) ∈ 2APL(s) ∈ 2AP
209 / 373
![Page 210: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/210.jpg)
Fundamental property of the product
given: Markov chainMMM and DRA AAA where
AccAccAcc ==={(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}{(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}{(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}For each state sss inMMM, let qsqsqs === δ
(q0, L(s)
)δ(q0, L(s)
)δ(q0, L(s)
).
PrMs (A)PrMs (A)PrMs (A)
probability measure of all paths π ∈ PathsM(s)π ∈ PathsM(s)π ∈ PathsM(s)such that trace(π) ∈ Lω(A)trace(π) ∈ Lω(A)trace(π) ∈ Lω(A)
210 / 373
![Page 211: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/211.jpg)
Fundamental property of the product
given: Markov chainMMM and DRA AAA where
AccAccAcc ==={(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}{(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}{(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}For each state sss inMMM, let qsqsqs === δ
(q0, L(s)
)δ(q0, L(s)
)δ(q0, L(s)
).
PrMs (A)PrMs (A)PrMs (A)
=== PrM×A〈s ,qs〉( ∨
1�i�k(♦�¬Li ∧ �♦Ui )
)PrM×A〈s ,qs〉
( ∨1�i�k
(♦�¬Li ∧ �♦Ui ))
PrM×A〈s,qs〉( ∨
1�i�k(♦�¬Li ∧ �♦Ui )
)
probability measure of all paths πππ in the products.t. π
∣∣Aπ∣∣Aπ∣∣A satisfies the acceptance condition of AAA
211 / 373
![Page 212: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/212.jpg)
Fundamental property of the product
given: Markov chainMMM and DRA AAA where
AccAccAcc ==={(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}{(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}{(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}For each state sss inMMM, let qsqsqs === δ
(q0, L(s)
)δ(q0, L(s)
)δ(q0, L(s)
).
PrMs (A)PrMs (A)PrMs (A)
=== PrM×A〈s ,qs〉( ∨
1�i�k(♦�¬Li ∧ �♦Ui )
)PrM×A〈s ,qs〉
( ∨1�i�k
(♦�¬Li ∧ �♦Ui ))
PrM×A〈s,qs〉( ∨
1�i�k(♦�¬Li ∧ �♦Ui )
)
=== PrM×A〈s ,qs〉(♦ accBSCC
)PrM×A〈s ,qs〉
(♦ accBSCC
)PrM×A〈s,qs〉
(♦ accBSCC
)212 / 373
![Page 213: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/213.jpg)
Fundamental property of the product
given: Markov chainMMM and DRA AAA where
AccAccAcc ==={(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}{(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}{(L1,U1), (L2,U2), . . . , (Lk ,Uk)
}For each state sss inMMM, let qsqsqs === δ
(q0, L(s)
)δ(q0, L(s)
)δ(q0, L(s)
).
PrMs (A)PrMs (A)PrMs (A)
=== PrM×A〈s ,qs〉(♦ accBSCC
)PrM×A〈s ,qs〉
(♦ accBSCC
)PrM×A〈s,qs〉
(♦ accBSCC
)���
union of accepting BSCCs inM×AM×AM×A i.e., BSCC CCC s.t.
∃i ∈ {1, . . . , k}.∃i ∈ {1, . . . , k}.∃i ∈ {1, . . . , k}. C ∩ Li = ∅C ∩ Li = ∅C ∩ Li = ∅ ∧∧∧ C ∩ Ui �= ∅C ∩ Ui �= ∅C ∩ Ui �= ∅213 / 373
![Page 214: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/214.jpg)
Summary: PCTL* model checking
given: Markov chainM = (S ,P,AP, L, s0)M = (S ,P,AP , L, s0)M = (S ,P,AP, L, s0)PCTL* state formula ΦΦΦ
task: check whetherM |= ΦM |= ΦM |= Φ
method: bottom-up treatment of state subformulas ΨΨΨto compute
Sat(Ψ)Sat(Ψ)Sat(Ψ) ==={s ∈ S : s |= Ψ
}{s ∈ S : s |= Ψ
}{s ∈ S : s |= Ψ
}• propositional logic fragment: obvious
• probability operator PI(ϕ)PI(ϕ)PI(ϕ) via
∗ construction of a DRA AAA for ϕϕϕ
∗ probabilistic reachability analysis inM×AM×AM×A214 / 373
![Page 215: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/215.jpg)
DTMCMMMPCTL* formula PI(ϕ)PI(ϕ)PI(ϕ)
LTL formula ϕ′ϕ′ϕ′
DRA AAA
probabilistic reachability analysisin the productM×AM×AM×A
1. graph analysis to compute theaccepting BSCCs of the product
2. linear equation system for the probabilitiesto reach an accepting BSCC
215 / 373
![Page 216: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/216.jpg)
DTMCMMMPCTL* formula PI(ϕ)PI(ϕ)PI(ϕ)
LTL formula ϕ′ϕ′ϕ′
DRA AAA
probabilistic reachability analysisin the productM×AM×AM×A
1. graph analysis to compute theaccepting BSCCs of the product
2. linear equation system
time complexity:
polynomial in thesizes ofMMM and AAA
216 / 373
![Page 217: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/217.jpg)
2 exp2 exp2 exp in |ϕ||ϕ||ϕ|
DTMCMMMPCTL* formula PI(ϕ)PI(ϕ)PI(ϕ)
LTL formula ϕ′ϕ′ϕ′
DRA AAA
probabilistic reachability analysisin the productM×AM×AM×A
1. graph analysis to compute theaccepting BSCCs of the product
2. linear equation system
time complexity:
polynomial in thesizes ofMMM and AAA
217 / 373
![Page 218: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/218.jpg)
Exponential-time algorithms for DTMC and LTL
218/373
![Page 219: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/219.jpg)
Exponential-time algorithms for DTMC and LTL
given: Markov chainMMM, LTL formula ϕϕϕ
task: compute PrM(ϕ)PrM(ϕ)PrM(ϕ)
single exponential-time algorithms:
• iterative, automata-less approach[Courcoubetis/Yannakakis’88]
• using weak alternating automata[Bustan/Rubin/Vardi’04]
• using separated Buchi automata[Couvreur/Saheb/Sutre’03]
• using unambiguous Buchi automata[Baier/Kiefer/Klein/Kluppelholz/Muller/Worrell’16]
219 / 373
![Page 220: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/220.jpg)
Tutorial: Probabilistic Model Checking
Discrete-time Markov chains (DTMC)
∗ basic definitions
∗ probabilistic computation tree logic PCTL/PCTL*
∗ rewards, cost-utility ratios, weights
∗ conditional probabilities
Markov decision processes (MDP)
∗ basic definitions
∗ PCTL/PCTL* model checking
∗ fairness
∗ conditional probabilities
∗ rewards, quantiles
∗ mean-payoff
∗ expected accumulated weights220 / 373
![Page 221: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/221.jpg)
Markov reward model (MRM)
221/373
![Page 222: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/222.jpg)
Markov reward model (MRM)
Markov chainM = (S ,P,AP, L, rew)M = (S ,P,AP, L, rew)M = (S ,P,AP , L, rew) with areward function for the states:
rew : S → Nrew : S → Nrew : S → N
idea: reward rew(s)rew(s)rew(s) will be earned when leaving sss
analogously: rewards for edges rew : S × S → Nrew : S × S → Nrew : S × S → N222 / 373
![Page 223: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/223.jpg)
Markov reward model (MRM)
Markov chainM = (S ,P,AP, L, rew)M = (S ,P,AP, L, rew)M = (S ,P,AP , L, rew) with areward function for the states:
rew : S → Nrew : S → Nrew : S → N
idea: reward rew(s)rew(s)rew(s) will be earned when leaving sss
formalization by accumulated rewards of finite paths
rew(s0 s1 . . . sn) =∑
0�i<nrew(si)rew(s0 s1 . . . sn) =
∑0�i<n
rew(si)rew(s0 s1 . . . sn) =∑
0�i<nrew(si)
analogously: rewards for edges rew : S × S → Nrew : S × S → Nrew : S × S → N223 / 373
![Page 224: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/224.jpg)
Example: Markov reward model
communication protocol with reward function thatcounts the number of trials
startstartstart
000
try to sendtry to sendtry to send
111
deldeldel 000
lostlostlost 000
0.980.980.98
0.020.020.02
accumulated reward of finite paths, e.g.,
rew(start try lost try del)rew(start try lost try del)rew(start try lost try del) === 222
224 / 373
![Page 225: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/225.jpg)
Example: Markov reward model
communication protocol with reward function thatcounts the number of trials
startstartstart
000
try to sendtry to sendtry to send
111
deldeldel 000
lostlostlost 000
0.980.980.98
0.020.020.02
measures of interest, e.g.,
PrM(♦�3del)PrM(♦�3del)PrM(♦�3del) probability to deliver a messagewithin at most three trials�
reachability with reward bound � 3� 3� 3225 / 373
![Page 226: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/226.jpg)
Example: Markov reward model
communication protocol with reward function thatcounts the number of trials
startstartstart
000
try to sendtry to sendtry to send
111
deldeldel 000
lostlostlost 000
0.980.980.98
0.020.020.02
measures of interest, e.g.,
PrM(♦�3del)PrM(♦�3del)PrM(♦�3del) probability to deliver a messagewithin at most three trials
E( del)E( del)E( del) expected number of trials until delivered
226 / 373
![Page 227: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/227.jpg)
Reward-based extension of PCTL
227/373
![Page 228: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/228.jpg)
Reward-based extension of PCTL
probability operator for reward-bounded path formulas:
PI( Φ1U�r Φ2 )PI( Φ1U�r Φ2 )PI( Φ1U�r Φ2 ) until with upper reward bound
228 / 373
![Page 229: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/229.jpg)
Reward-based extension of PCTL
probability operator for reward-bounded path formulas:
PI( Φ1U�r Φ2 )PI( Φ1U�r Φ2 )PI( Φ1U�r Φ2 ) until with upper reward bound
expected accumulated reward operator: E�r( Φ)E�r( Φ)E�r( Φ)
s |= E�r( Φ)s |= E�r( Φ)s |= E�r( Φ) iff
{expected accumulated reward onpaths from sss to a ΦΦΦ-state is � r� r� r
229 / 373
![Page 230: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/230.jpg)
Reward-based extension of PCTL
probability operator for reward-bounded path formulas:
PI( Φ1U�r Φ2 )PI( Φ1U�r Φ2 )PI( Φ1U�r Φ2 ) until with upper reward bound
expected accumulated reward operator: E�r( Φ)E�r( Φ)E�r( Φ)
s |= E�r( Φ)s |= E�r( Φ)s |= E�r( Φ) iff
{expected accumulated reward onpaths from sss to a ΦΦΦ-state is � r� r� r
example: communication protocol
P�0.9(♦�3del)P�0.9(♦�3del)P�0.9(♦�3del) probability for delivering the message withinat most three trials is at least 0.9
E�5( del)E�5( del)E�5( del) average number of trials is less or equal 5230 / 373
![Page 231: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/231.jpg)
Model checking reward-based properties
231/ 373
![Page 232: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/232.jpg)
Model checking reward-based properties
treatment of PI(Φ1U�r Φ2)PI(Φ1U�r Φ2)PI(Φ1U�r Φ2) where r ∈ Nr ∈ Nr ∈ N
compute PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2) iteratively
for increasing reward bound i = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , r
232 / 373
![Page 233: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/233.jpg)
Model checking reward-based properties
treatment of PI(Φ1U�r Φ2)PI(Φ1U�r Φ2)PI(Φ1U�r Φ2) where r ∈ Nr ∈ Nr ∈ N
compute PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2) iteratively
for increasing reward bound i = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , r
Let xs,ixs,ixs,i === PrMs(Φ1U
�i Φ2
)PrMs
(Φ1 U
�i Φ2
)PrMs
(Φ1U
�i Φ2
). Then:
233 / 373
![Page 234: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/234.jpg)
Model checking reward-based properties
treatment of PI(Φ1U�r Φ2)PI(Φ1U�r Φ2)PI(Φ1U�r Φ2) where r ∈ Nr ∈ Nr ∈ N
compute PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2) iteratively
for increasing reward bound i = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , r
Let xs,ixs,ixs,i === PrMs(Φ1U
�i Φ2
)PrMs
(Φ1 U
�i Φ2
)PrMs
(Φ1U
�i Φ2
). Then:
if s |= ∃(Φ1UΦ2) ∧ ¬Φ2s |= ∃(Φ1UΦ2) ∧ ¬Φ2s |= ∃(Φ1UΦ2) ∧ ¬Φ2 and i � rew(s)i � rew(s)i � rew(s) then
xs,i =∑s ′∈S
P(s, s ′) · xs ′,i−rew(s)xs,i =∑s ′∈S
P(s, s ′) · xs ′,i−rew(s)xs,i =∑s ′∈S
P(s, s ′) · xs ′,i−rew(s)
234 / 373
![Page 235: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/235.jpg)
Model checking reward-based properties
treatment of PI(Φ1U�r Φ2)PI(Φ1U�r Φ2)PI(Φ1U�r Φ2) where r ∈ Nr ∈ Nr ∈ N
compute PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2) iteratively
for increasing reward bound i = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , r
Let xs,ixs,ixs,i === PrMs(Φ1U
�i Φ2
)PrMs
(Φ1 U
�i Φ2
)PrMs
(Φ1U
�i Φ2
). Then:
if s |= ∃(Φ1UΦ2) ∧ ¬Φ2s |= ∃(Φ1UΦ2) ∧ ¬Φ2s |= ∃(Φ1UΦ2) ∧ ¬Φ2 and i � rew(s)i � rew(s)i � rew(s) then
xs,i =∑s ′∈S
P(s, s ′) · xs ′,i−rew(s)xs,i =∑s ′∈S
P(s, s ′) · xs ′,i−rew(s)xs,i =∑s ′∈S
P(s, s ′) · xs ′,i−rew(s)
if s |= Φ2s |= Φ2s |= Φ2 then: xs,i = 1xs,i = 1xs,i = 1
in all other cases: xs,i = 0xs,i = 0xs,i = 0235 / 373
![Page 236: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/236.jpg)
Model checking reward-based properties
treatment of PI(Φ1U�r Φ2)PI(Φ1U�r Φ2)PI(Φ1U�r Φ2) where r ∈ Nr ∈ Nr ∈ N
compute PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2) iteratively
for increasing reward bound i = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , r
treatment of the E�r( Φ)E�r( Φ)E�r( Φ)
compute the expected accumulated rewardsby solving the linear equation system
236 / 373
![Page 237: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/237.jpg)
Model checking reward-based properties
treatment of PI(Φ1U�r Φ2)PI(Φ1U�r Φ2)PI(Φ1U�r Φ2) where r ∈ Nr ∈ Nr ∈ N
compute PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2) iteratively
for increasing reward bound i = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , r
treatment of the E�r( Φ)E�r( Φ)E�r( Φ), assuming PrM(♦Φ) = 1PrM(♦Φ) = 1PrM(♦Φ) = 1
compute the expected accumulated rewardsby solving the linear equation system
xsxsxs === rew(s) +∑s ′∈S
P(s, s ′) · xs ′rew(s) +∑s ′∈S
P(s , s ′) · xs ′rew(s) +∑s ′∈S
P(s, s ′) · xs ′ if s �|= Φs �|= Φs �|= Φ
xsxsxs === 000 if s |= Φs |= Φs |= Φ
237 / 373
![Page 238: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/238.jpg)
Model checking reward-based properties
treatment of PI(Φ1U�r Φ2)PI(Φ1U�r Φ2)PI(Φ1U�r Φ2) where r ∈ Nr ∈ Nr ∈ N
compute PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2) iteratively
for increasing reward bound i = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , r
treatment of the E�r( Φ)E�r( Φ)E�r( Φ), assuming PrM(♦Φ) = 1PrM(♦Φ) = 1PrM(♦Φ) = 1
compute the expected accumulated rewardsby solving the linear equation system
xsxsxs === rew(s) +∑s ′∈S
P(s, s ′) · xs ′rew(s) +∑s ′∈S
P(s , s ′) · xs ′rew(s) +∑s ′∈S
P(s, s ′) · xs ′ if s �|= Φs �|= Φs �|= Φ
also applicable for rational-valued weight fct.
238 / 373
![Page 239: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/239.jpg)
Model checking reward-based properties
treatment of PI(Φ1U�r Φ2)PI(Φ1U�r Φ2)PI(Φ1U�r Φ2) where r ∈ Nr ∈ Nr ∈ N
compute PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2) iteratively
for increasing reward bound i = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , r
treatment of the E�r( Φ)E�r( Φ)E�r( Φ), assuming PrM(♦Φ) = 1PrM(♦Φ) = 1PrM(♦Φ) = 1
compute the expected accumulated rewardsby solving the linear equation system
time complexity:
expected rewards: polynomial in size(M)size(M)size(M)
reward-bounded until: polynomial in size(M)size(M)size(M) and rrr239 / 373
![Page 240: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/240.jpg)
Model checking reward-based properties
treatment of PI(Φ1U�r Φ2)PI(Φ1U�r Φ2)PI(Φ1U�r Φ2) where r ∈ Nr ∈ Nr ∈ N
compute PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2) iteratively
for increasing reward bound i = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , r
treatment of the E�r( Φ)E�r( Φ)E�r( Φ), assuming PrM(♦Φ) = 1PrM(♦Φ) = 1PrM(♦Φ) = 1
compute the expected accumulated rewardsby solving the linear equation system
time complexity:
reward-bounded until: polynomial in size(M)size(M)size(M) and rrr
pseudo-polynomial︷ ︸︸ ︷240 / 373
![Page 241: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/241.jpg)
Complexity: reward-bounded until
treatment of PI(Φ1U�r Φ2)PI(Φ1U�r Φ2)PI(Φ1U�r Φ2) where r ∈ Nr ∈ Nr ∈ N
compute PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2) iteratively
for increasing reward bound i = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , r
unit rewards: polynomial in size(M)size(M)size(M) and log rlog rlog rrepeated squaring
general case: polynomial in size(M)size(M)size(M) and rrr“pseudo-polynomial”
241 / 373
![Page 242: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/242.jpg)
Complexity: reward-bounded until
treatment of PI(Φ1U�r Φ2)PI(Φ1U�r Φ2)PI(Φ1U�r Φ2) where r ∈ Nr ∈ Nr ∈ N
compute PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2) iteratively
for increasing reward bound i = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , r
unit rewards: polynomial in size(M)size(M)size(M) and log rlog rlog rrepeated squaring
general case: polynomial in size(M)size(M)size(M) and rrr
decision problem “does PrMs (Φ1U�r Φ2) > qPrMs (Φ1U�r Φ2) > qPrMs (Φ1U�r Φ2) > q hold ?”
NP-hard [Laroussinie/Sproston’05]
PosSLP-hard, in PSPACE [Haase/Kiefer’15]
242 / 373
![Page 243: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/243.jpg)
Complexity: reward-bounded until
treatment of PI(Φ1U�r Φ2)PI(Φ1U�r Φ2)PI(Φ1U�r Φ2) where r ∈ Nr ∈ Nr ∈ N
compute PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2)PrMs (Φ1U�i Φ2) iteratively
for increasing reward bound i = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , ri = 0, 1, 2, . . . , r
unit rewards: polynomial in size(M)size(M)size(M) and log rlog rlog rrepeated squaring
general case: polynomial in size(M)size(M)size(M) and rrr
decision problem “does PrMs (Φ1U�r Φ2) > qPrMs (Φ1U�r Φ2) > qPrMs (Φ1U�r Φ2) > q hold ?”
NP-hard [Laroussinie/Sproston’05]
PosSLP-hard, in PSPACE [Haase/Kiefer’15]
243 / 373
![Page 244: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/244.jpg)
NP-hardness [Laroussinie/Sproston’05]
244 / 373
![Page 245: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/245.jpg)
NP-hardness [Laroussinie/Sproston’05]
The threshold problem for Markov chains is NP-hard:
given: Markov chainMMM === (S ,P, sinit, rew)(S ,P, sinit , rew)(S ,P, sinit, rew),
G ⊆ SG ⊆ SG ⊆ S , r ∈ Nr ∈ Nr ∈ N and q ∈ ]0, 1[∩Qq ∈ ]0, 1[∩Qq ∈ ]0, 1[∩Qtask: check whether Prsinit
(♦�rG
)� qPrsinit
(♦�rG
)� qPrsinit
(♦�rG
)� q
245 / 373
![Page 246: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/246.jpg)
NP-hardness [Laroussinie/Sproston’05]
The threshold problem for Markov chains is NP-hard:
given: Markov chainMMM === (S ,P, sinit, rew)(S ,P, sinit , rew)(S ,P, sinit, rew),
G ⊆ SG ⊆ SG ⊆ S , r ∈ Nr ∈ Nr ∈ N and q ∈ ]0, 1[∩Qq ∈ ]0, 1[∩Qq ∈ ]0, 1[∩Qtask: check whether Prsinit
(♦�rG
)� qPrsinit
(♦�rG
)� qPrsinit
(♦�rG
)� q
Polynomial reduction from counting variant of SUBSUM:
given: x1, . . . , xnx1, . . . , xnx1, . . . , xn,yyy ,k ∈ Nk ∈ Nk ∈ N
task: check whether there are at least kkk subsets NNN
of {1, . . . , n}{1, . . . , n}{1, . . . , n} s.t. ∑i∈N
xi � y∑i∈N
xi � y∑i∈N
xi � y
246 / 373
![Page 247: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/247.jpg)
Polynomial reduction [Laroussinie/Sproston’05]
247 / 373
![Page 248: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/248.jpg)
Polynomial reduction [Laroussinie/Sproston’05]
counting variant of SUBSUM:
given: x1, . . . , xnx1, . . . , xnx1, . . . , xn,y , k ∈ Ny , k ∈ Ny , k ∈ N
task: check whether there are at least kkk subsets NNN
of {1, . . . , n}{1, . . . , n}{1, . . . , n} s.t. ∑i∈N
xi � y∑i∈N
xi � y∑i∈N
xi � y
Markov chain: 2n+12n+12n+1 states
. . .. . .. . .
121212
121212
121212
121212
121212
121212
121212
121212
121212
248 / 373
![Page 249: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/249.jpg)
Polynomial reduction [Laroussinie/Sproston’05]
counting variant of SUBSUM:
given: x1, . . . , xnx1, . . . , xnx1, . . . , xn,y , k ∈ Ny , k ∈ Ny , k ∈ N
task: check whether there are at least kkk subsets NNN
of {1, . . . , n}{1, . . . , n}{1, . . . , n} s.t. ∑i∈N
xi � y∑i∈N
xi � y∑i∈N
xi � y
Markov chain: 2n+12n+12n+1 states and rewards for the states
. . .. . .. . .
x1x1x1 x2x2x2 x3x3x3 xnxnxn
000 000 000 000 000
121212
121212
121212
121212
121212
121212
121212
121212
121212
249 / 373
![Page 250: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/250.jpg)
Polynomial reduction [Laroussinie/Sproston’05]
counting variant of SUBSUM:
given: x1, . . . , xnx1, . . . , xnx1, . . . , xn,y , k ∈ Ny , k ∈ Ny , k ∈ N
task: check whether there are at least kkk subsets NNN
of {1, . . . , n}{1, . . . , n}{1, . . . , n} s.t. ∑i∈N
xi � y∑i∈N
xi � y∑i∈N
xi � y
Markov chain: 2n+12n+12n+1 states and rewards for the states
. . .. . .. . .sinitsinitsinit GGG
x1x1x1 x2x2x2 x3x3x3 xnxnxn
000 000 000 000 000
121212
121212
121212
121212
121212
121212
121212
121212
121212
250 / 373
![Page 251: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/251.jpg)
Polynomial reduction [Laroussinie/Sproston’05]
counting variant of SUBSUM:
given: x1, . . . , xnx1, . . . , xnx1, . . . , xn,y , k ∈ Ny , k ∈ Ny , k ∈ N
task: check whether there are at least kkk subsets NNN
of {1, . . . , n}{1, . . . , n}{1, . . . , n} s.t. ∑i∈N
xi � y∑i∈N
xi � y∑i∈N
xi � y
. . .. . .. . .sinitsinitsinit GGG
x1x1x1 x2x2x2 x3x3x3 xnxnxn
000 000 000 000 000
121212
121212
121212
121212
121212
121212
121212
121212
121212
Prsinit(♦�yG ) � k2n
Prsinit(♦�yG ) � k2nPrsinit(♦�yG ) � k2n iff there are at least kkk subsets ....
251 / 373
![Page 252: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/252.jpg)
Mean-payoff (a.k.a. long-rung average)
252/ 373
![Page 253: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/253.jpg)
Mean-payoff (a.k.a. long-rung average)
given: a weighted graph without trap states
mean-payoff functions MPMPMP, MPMPMP ::: InfPaths → RInfPaths → RInfPaths → R:
MP(s0 s1 s2 . . .)MP(s0 s1 s2 . . .)MP(s0 s1 s2 . . .) === lim supn→∞lim supn→∞lim supn→∞
1n+11
n+11
n+1···
n∑i=0
wgt(si)n∑
i=0
wgt(si)n∑
i=0
wgt(si)
MP(s0 s1 s2 . . .)MP(s0 s1 s2 . . .)MP(s0 s1 s2 . . .) === lim infn→∞lim infn→∞lim infn→∞
1n+11
n+11
n+1 ···n∑
i=0
wgt(si)n∑
i=0
wgt(si)n∑
i=0
wgt(si)
253 / 373
![Page 254: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/254.jpg)
Mean-payoff (a.k.a. long-rung average)
given: a weighted graph without trap states
mean-payoff functions MPMPMP, MPMPMP ::: InfPaths → RInfPaths → RInfPaths → R:
MP(s0 s1 s2 . . .)MP(s0 s1 s2 . . .)MP(s0 s1 s2 . . .) === lim supn→∞lim supn→∞lim supn→∞
1n+11
n+11
n+1···
n∑i=0
wgt(si)n∑
i=0
wgt(si)n∑
i=0
wgt(si)
MP(s0 s1 s2 . . .)MP(s0 s1 s2 . . .)MP(s0 s1 s2 . . .) === lim infn→∞lim infn→∞lim infn→∞
1n+11
n+11
n+1 ···n∑
i=0
wgt(si)n∑
i=0
wgt(si)n∑
i=0
wgt(si)
if wgt(s) = +1wgt(s) = +1wgt(s) = +1, wgt(t) = −1wgt(t) = −1wgt(t) = −1 then there exists n1, n2, . . .n1, n2, . . .n1, n2, . . .and k1, k2, . . . ∈ Nk1, k2, . . . ∈ Nk1, k2, . . . ∈ N s.t. for π = sn1 tk1 sn2 tk2 . . .π = sn1 tk1 sn2 tk2 . . .π = sn1 tk1 sn2 tk2 . . .:
MP(π)MP(π)MP(π) <<< 000 <<< MP(π)MP(π)MP(π)254 / 373
![Page 255: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/255.jpg)
Expected mean-payoff in finite MC
fundamental results:
in finite MC: Es(MP)Es(MP)Es(MP) === Es(MP)Es(MP)Es(MP)
notation: Es(MP)Es(MP)Es(MP) rather than Es(MP)Es(MP)Es(MP) resp. Es(MP)Es(MP)Es(MP)
255 / 373
![Page 256: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/256.jpg)
Expected mean-payoff in finite MC
fundamental results:
in finite MC: Es(MP)Es(MP)Es(MP) === Es(MP)Es(MP)Es(MP)
notation: Es(MP)Es(MP)Es(MP) rather than Es(MP)Es(MP)Es(MP) resp. Es(MP)Es(MP)Es(MP)
Almost all paths eventually enter a BSCC andvisit all its states infinitely often.
BSCC: bottom strongly connected component256 / 373
![Page 257: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/257.jpg)
Expected mean-payoff in finite MC
fundamental results:
in finite MC: Es(MP)Es(MP)Es(MP) === Es(MP)Es(MP)Es(MP)
notation: Es(MP)Es(MP)Es(MP) rather than Es(MP)Es(MP)Es(MP) resp. Es(MP)Es(MP)Es(MP)
Almost all paths eventually enter a BSCC andvisit all its states infinitely often ...
... with the same long-run frequencies ...
BSCC: bottom strongly connected component257 / 373
![Page 258: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/258.jpg)
Long-run frequencies in finite MC
steady-state probabilities in BSCC BBB of a finite MC:
θB(s)θB(s)θB(s) === limn→∞limn→∞limn→∞
1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)for each t ∈ Bt ∈ Bt ∈ B
©is©is©is === “after iii steps in state sss”258 / 373
![Page 259: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/259.jpg)
Long-run frequencies in finite MC
steady-state probabilities in BSCC BBB of a finite MC:
θB(s)θB(s)θB(s) === limn→∞limn→∞limn→∞
1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)for each t ∈ Bt ∈ Bt ∈ B
computable by a linear equation system:
θB(s)θB(s)θB(s) ===∑t∈B
θB(t) · P(t, s)∑t∈B
θB(t) · P(t, s)∑t∈B
θB(t) · P(t, s)
“balance equations”
©is©is©is === “after iii steps in state sss”259 / 373
![Page 260: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/260.jpg)
Long-run frequencies in finite MC
steady-state probabilities in BSCC BBB of a finite MC:
θB(s)θB(s)θB(s) === limn→∞limn→∞limn→∞
1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)for each t ∈ Bt ∈ Bt ∈ B
computable by a linear equation system:
θB(s)θB(s)θB(s) ===∑t∈B
θB(t) · P(t, s)∑t∈B
θB(t) · P(t, s)∑t∈B
θB(t) · P(t, s)∑s∈B
θB(s)∑s∈B
θB(s)∑s∈B
θB(s) === 111
©is©is©is === “after iii steps in state sss”260 / 373
![Page 261: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/261.jpg)
Long-run frequencies in finite MC
steady-state probabilities in BSCC BBB of a finite MC:
θB(s)θB(s)θB(s) === limn→∞limn→∞limn→∞
1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)for each t ∈ Bt ∈ Bt ∈ B
computable by a linear equation system:
θB(s)θB(s)θB(s) ===∑t∈B
θB(t) · P(t, s)∑t∈B
θB(t) · P(t, s)∑t∈B
θB(t) · P(t, s)∑s∈B
θB(s)∑s∈B
θB(s)∑s∈B
θB(s) === 111
unique solution of thelinear equation system
x = x · P |Bx = x · P|Bx = x · P |B∑s∈B
xs = 1∑s∈B
xs = 1∑s∈B
xs = 1
©is©is©is === “after iii steps in state sss”261 / 373
![Page 262: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/262.jpg)
Long-run frequencies in finite MC
steady-state probabilities in BSCC BBB of a finite MC:
θB(s)θB(s)θB(s) === limn→∞limn→∞limn→∞
1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)for each t ∈ Bt ∈ Bt ∈ B
for almost all paths π = s0 s1 s2 . . .π = s0 s1 s2 . . .π = s0 s1 s2 . . . with π |= ♦Bπ |= ♦Bπ |= ♦B:θB(s)θB(s)θB(s) === lim
n→∞limn→∞limn→∞
1n+11
n+11
n+1 ··· freq(s, s0 s1 . . . sn)freq(s, s0 s1 . . . sn)freq(s, s0 s1 . . . sn)︸ ︷︷ ︸long-run frequency ofstate sss in path πππ
... limit exists for almost all paths ...
262 / 373
![Page 263: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/263.jpg)
Long-run frequencies in finite MC
steady-state probabilities in BSCC BBB of a finite MC:
θB(s)θB(s)θB(s) === limn→∞limn→∞limn→∞
1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)for each t ∈ Bt ∈ Bt ∈ B
for almost all paths π = s0 s1 s2 . . .π = s0 s1 s2 . . .π = s0 s1 s2 . . . with π |= ♦Bπ |= ♦Bπ |= ♦B:θB(s)θB(s)θB(s) === lim
n→∞limn→∞limn→∞
1n+11
n+11
n+1 ··· freq(s, s0 s1 . . . sn)freq(s, s0 s1 . . . sn)freq(s, s0 s1 . . . sn)︸ ︷︷ ︸long-run frequency ofstate sss in path πππ
freq(s, s0 s1 . . . sn)freq(s, s0 s1 . . . sn)freq(s, s0 s1 . . . sn) ===
{number of occurrences of sssin the sequence s0 s1 . . . sns0 s1 . . . sns0 s1 . . . sn
263 / 373
![Page 264: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/264.jpg)
Mean-payoff in finite weighted MC
steady-state probabilities in BSCC BBB of a finite MC:
θB(s)θB(s)θB(s) === limn→∞limn→∞limn→∞
1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)for each t ∈ Bt ∈ Bt ∈ B
for almost all paths π = s0 s1 s2 . . .π = s0 s1 s2 . . .π = s0 s1 s2 . . . with π |= ♦Bπ |= ♦Bπ |= ♦B:θB(s)θB(s)θB(s) === lim
n→∞limn→∞limn→∞
1n+11
n+11
n+1 ··· freq(s, s0 s1 . . . sn)freq(s, s0 s1 . . . sn)freq(s, s0 s1 . . . sn)
if π |= ♦Bπ |= ♦Bπ |= ♦B where BBB is a BSCC then almost surely
MP(π)MP(π)MP(π) ===∑s∈B
θB(s) · wgt(s)∑s∈B
θB(s) · wgt(s)∑s∈B
θB(s) · wgt(s)
264 / 373
![Page 265: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/265.jpg)
Mean-payoff in finite weighted MC
steady-state probabilities in BSCC BBB of a finite MC:
θB(s)θB(s)θB(s) === limn→∞limn→∞limn→∞
1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)for each t ∈ Bt ∈ Bt ∈ B
for almost all paths π = s0 s1 s2 . . .π = s0 s1 s2 . . .π = s0 s1 s2 . . . with π |= ♦Bπ |= ♦Bπ |= ♦B:θB(s)θB(s)θB(s) === lim
n→∞limn→∞limn→∞
1n+11
n+11
n+1 ··· freq(s, s0 s1 . . . sn)freq(s, s0 s1 . . . sn)freq(s, s0 s1 . . . sn)
if π |= ♦Bπ |= ♦Bπ |= ♦B where BBB is a BSCC then almost surely
MP(π)MP(π)MP(π) ===∑s∈B
θB(s) · wgt(s)∑s∈B
θB(s) · wgt(s)∑s∈B
θB(s) · wgt(s)︸ ︷︷ ︸only depends on BBB
265 / 373
![Page 266: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/266.jpg)
Mean-payoff in finite weighted MC
steady-state probabilities in BSCC BBB of a finite MC:
θB(s)θB(s)θB(s) === limn→∞limn→∞limn→∞
1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)for each t ∈ Bt ∈ Bt ∈ B
for almost all paths π = s0 s1 s2 . . .π = s0 s1 s2 . . .π = s0 s1 s2 . . . with π |= ♦Bπ |= ♦Bπ |= ♦B:θB(s)θB(s)θB(s) === lim
n→∞limn→∞limn→∞
1n+11
n+11
n+1 ··· freq(s, s0 s1 . . . sn)freq(s, s0 s1 . . . sn)freq(s, s0 s1 . . . sn)
if π |= ♦Bπ |= ♦Bπ |= ♦B where BBB is a BSCC then almost surely
MP(π)MP(π)MP(π) ===∑s∈B
θB(s) · wgt(s)∑s∈B
θB(s) · wgt(s)∑s∈B
θB(s) · wgt(s) def
=def
=def
= MP(B)MP(B)MP(B)
︸ ︷︷ ︸only depends on BBB
266 / 373
![Page 267: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/267.jpg)
Mean-payoff in finite weighted MC
steady-state probabilities in BSCC BBB of a finite MC:
θB(s)θB(s)θB(s) === limn→∞limn→∞limn→∞
1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)1n ·
n∑i=1
Prt(©is
)for each t ∈ Bt ∈ Bt ∈ B
for almost all paths π = s0 s1 s2 . . .π = s0 s1 s2 . . .π = s0 s1 s2 . . . with π |= ♦Bπ |= ♦Bπ |= ♦B:θB(s)θB(s)θB(s) === lim
n→∞limn→∞limn→∞
1n+11
n+11
n+1 ··· freq(s, s0 s1 . . . sn)freq(s, s0 s1 . . . sn)freq(s, s0 s1 . . . sn)
if π |= ♦Bπ |= ♦Bπ |= ♦B where BBB is a BSCC then almost surely
MP(π)MP(π)MP(π) ===∑s∈B
θB(s) · wgt(s)∑s∈B
θB(s) · wgt(s)∑s∈B
θB(s) · wgt(s) def
=def
=def
= MP(B)MP(B)MP(B)
expected mean-payoff:∑B
Prs0(♦B) ·MP(B)∑B
Prs0(♦B) ·MP(B)∑B
Prs0(♦B) ·MP(B)267 / 373
![Page 268: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/268.jpg)
Long-run ratios in finite MC
MC with two reward functions costcostcost,util : S → Nutil : S → Nutil : S → N
Examples:
• energy-utility ratio
• number of SLA violations per day
• recovery time per failure268 / 373
![Page 269: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/269.jpg)
Long-run ratios in finite MC
MC with two reward functions costcostcost,util : S → Nutil : S → Nutil : S → N
long-run cost-utility ratio lrrat : InfPaths → Rlrrat : InfPaths → Rlrrat : InfPaths → R
lrrat(s0 s1 s2 . . .)lrrat(s0 s1 s2 . . .)lrrat(s0 s1 s2 . . .) === limn→∞limn→∞limn→∞
cost(s0 s1 . . . sn)util(s0 s1 . . . sn)cost(s0 s1 . . . sn)util(s0 s1 . . . sn)cost(s0 s1 . . . sn)util(s0 s1 . . . sn)
Examples:
• energy-utility ratio
• number of SLA violations per day
• recovery time per failure269 / 373
![Page 270: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/270.jpg)
Long-run ratios in finite MC
MC with two reward functions costcostcost,util : S → Nutil : S → Nutil : S → N
long-run cost-utility ratio lrrat : InfPaths → Rlrrat : InfPaths → Rlrrat : InfPaths → R
lrrat(s0 s1 s2 . . .)lrrat(s0 s1 s2 . . .)lrrat(s0 s1 s2 . . .) === limn→∞limn→∞limn→∞
cost(s0 s1 . . . sn)util(s0 s1 . . . sn)cost(s0 s1 . . . sn)util(s0 s1 . . . sn)cost(s0 s1 . . . sn)util(s0 s1 . . . sn)
does the limit exist for almost all paths ?
• energy-utility ratio
• number of SLA violations per day
• recovery time per failure270 / 373
![Page 271: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/271.jpg)
Long-run ratios in finite MC
MC with two reward functions costcostcost,util : S → Nutil : S → Nutil : S → N
long-run cost-utility ratio lrrat : InfPaths → Rlrrat : InfPaths → Rlrrat : InfPaths → R
lrrat(s0 s1 s2 . . .)lrrat(s0 s1 s2 . . .)lrrat(s0 s1 s2 . . .) === limn→∞limn→∞limn→∞
cost(s0 s1 . . . sn)util(s0 s1 . . . sn)cost(s0 s1 . . . sn)util(s0 s1 . . . sn)cost(s0 s1 . . . sn)util(s0 s1 . . . sn)
=== limn→∞limn→∞limn→∞
1n+1 ·
n∑i=0
cost(si)
1n+1 ·
n∑i=0
util(si)
1n+1 ·
n∑i=0
cost(si)
1n+1 ·
n∑i=0
util(si)
1n+1 ·
n∑i=0
cost(si)
1n+1 ·
n∑i=0
util(si)
271 / 373
![Page 272: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/272.jpg)
Long-run ratios in finite MC
MC with two reward functions costcostcost,util : S → Nutil : S → Nutil : S → N
long-run cost-utility ratio lrrat : InfPaths → Rlrrat : InfPaths → Rlrrat : InfPaths → R
lrrat(s0 s1 s2 . . .)lrrat(s0 s1 s2 . . .)lrrat(s0 s1 s2 . . .) === limn→∞limn→∞limn→∞
cost(s0 s1 . . . sn)util(s0 s1 . . . sn)cost(s0 s1 . . . sn)util(s0 s1 . . . sn)cost(s0 s1 . . . sn)util(s0 s1 . . . sn)
=== limn→∞limn→∞limn→∞
1n+1 ·
n∑i=0
cost(si)
1n+1 ·
n∑i=0
util(si)
1n+1 ·
n∑i=0
cost(si)
1n+1 ·
n∑i=0
util(si)
1n+1 ·
n∑i=0
cost(si)
1n+1 ·
n∑i=0
util(si)
===MP[cost](s0 s1 s2 . . .)MP[util ](s0 s1 s2 . . .)MP[cost](s0 s1 s2 . . .)MP[util ](s0 s1 s2 . . .)MP[cost](s0 s1 s2 . . .)MP[util ](s0 s1 s2 . . .)
272 / 373
![Page 273: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/273.jpg)
Long-run ratios in finite MC
MC with two reward functions costcostcost,util : S → Nutil : S → Nutil : S → N
long-run cost-utility ratio lrrat : InfPaths → Rlrrat : InfPaths → Rlrrat : InfPaths → R
lrrat(s0 s1 s2 . . .)lrrat(s0 s1 s2 . . .)lrrat(s0 s1 s2 . . .) === limn→∞limn→∞limn→∞
cost(s0 s1 . . . sn)util(s0 s1 . . . sn)cost(s0 s1 . . . sn)util(s0 s1 . . . sn)cost(s0 s1 . . . sn)util(s0 s1 . . . sn)
=== limn→∞limn→∞limn→∞
1n+1 ·
n∑i=0
cost(si)
1n+1 ·
n∑i=0
util(si)
1n+1 ·
n∑i=0
cost(si)
1n+1 ·
n∑i=0
util(si)
1n+1 ·
n∑i=0
cost(si)
1n+1 ·
n∑i=0
util(si)
===MP[cost](s0 s1 s2 . . .)MP[util ](s0 s1 s2 . . .)MP[cost](s0 s1 s2 . . .)MP[util ](s0 s1 s2 . . .)MP[cost](s0 s1 s2 . . .)MP[util ](s0 s1 s2 . . .)
in particular:limit exists foralmost all paths
273 / 373
![Page 274: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/274.jpg)
Long-run ratios in finite MC
MC with two reward functions costcostcost,util : S → Nutil : S → Nutil : S → N
long-run cost-utility ratio lrrat : InfPaths → Rlrrat : InfPaths → Rlrrat : InfPaths → R
lrrat(s0 s1 s2 . . .)lrrat(s0 s1 s2 . . .)lrrat(s0 s1 s2 . . .) === limn→∞limn→∞limn→∞
cost(s0 s1 . . . sn)util(s0 s1 . . . sn)cost(s0 s1 . . . sn)util(s0 s1 . . . sn)cost(s0 s1 . . . sn)util(s0 s1 . . . sn)
if π |= ♦Bπ |= ♦Bπ |= ♦B where BBB is a BSCC then almost surely
lrrat(π)lrrat(π)lrrat(π) ===MP[cost](B)MP[util ](B)MP[cost](B)MP[util ](B)MP[cost](B)MP[util ](B)
MP[wgt](B) =∑s∈B
θB(s) · wgt(s)MP[wgt](B) =∑s∈B
θB(s) · wgt(s)MP[wgt](B) =∑s∈B
θB(s) · wgt(s) mean-payoff forweight function
274 / 373
![Page 275: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/275.jpg)
Long-run ratios in finite MC
MC with two reward functions costcostcost,util : S → Nutil : S → Nutil : S → N
long-run cost-utility ratio lrrat : InfPaths → Rlrrat : InfPaths → Rlrrat : InfPaths → R
lrrat(s0 s1 s2 . . .)lrrat(s0 s1 s2 . . .)lrrat(s0 s1 s2 . . .) === limn→∞limn→∞limn→∞
cost(s0 s1 . . . sn)util(s0 s1 . . . sn)cost(s0 s1 . . . sn)util(s0 s1 . . . sn)cost(s0 s1 . . . sn)util(s0 s1 . . . sn)
if π |= ♦Bπ |= ♦Bπ |= ♦B where BBB is a BSCC then almost surely
lrrat(π)lrrat(π)lrrat(π) ===MP[cost](B)MP[util ](B)MP[cost](B)MP[util ](B)MP[cost](B)MP[util ](B)
def
=def
=def
= lrrat(B)lrrat(B)lrrat(B)
︸ ︷︷ ︸only depends on BBB
275 / 373
![Page 276: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/276.jpg)
Long-run ratios in finite MC
MC with two reward functions costcostcost,util : S → Nutil : S → Nutil : S → N
long-run cost-utility ratio lrrat : InfPaths → Rlrrat : InfPaths → Rlrrat : InfPaths → R
lrrat(s0 s1 s2 . . .)lrrat(s0 s1 s2 . . .)lrrat(s0 s1 s2 . . .) === limn→∞limn→∞limn→∞
cost(s0 s1 . . . sn)util(s0 s1 . . . sn)cost(s0 s1 . . . sn)util(s0 s1 . . . sn)cost(s0 s1 . . . sn)util(s0 s1 . . . sn)
if π |= ♦Bπ |= ♦Bπ |= ♦B where BBB is a BSCC then almost surely
lrrat(π)lrrat(π)lrrat(π) ===MP[cost](B)MP[util ](B)MP[cost](B)MP[util ](B)MP[cost](B)MP[util ](B)
def
=def
=def
= lrrat(B)lrrat(B)lrrat(B)
expected long-run ratio:∑B
PrM(♦B) · lrrat(B)∑B
PrM(♦B) · lrrat(B)∑B
PrM(♦B) · lrrat(B)276 / 373
![Page 277: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/277.jpg)
Best threshold for long-run ratios
given: MC with reward functions costcostcost,util : S → Nutil : S → Nutil : S → N
rational probability bound ppp
compute roptroptropt === inf{r ∈ R : PrM( lrrat � r ) > p
}inf
{r ∈ R : PrM( lrrat � r ) > p
}inf
{r ∈ R : PrM( lrrat � r ) > p
}���random variable for thelong-run cost-utility ratio
(as before)
277 / 373
![Page 278: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/278.jpg)
Best threshold for long-run ratios
given: MC with reward functions costcostcost,util : S → Nutil : S → Nutil : S → N
rational probability bound ppp
compute roptroptropt === inf{r ∈ R : PrM( lrrat � r ) > p
}inf
{r ∈ R : PrM( lrrat � r ) > p
}inf
{r ∈ R : PrM( lrrat � r ) > p
}roptroptropt === inf
{r ∈ R : PrM(�♦(costutil � r) ) > p
}inf
{r ∈ R : PrM(�♦(costutil � r) ) > p
}inf
{r ∈ R : PrM(�♦(costutil � r) ) > p
}
if π = s0 s1 s2 . . .π = s0 s1 s2 . . .π = s0 s1 s2 . . . is an infinite path then
π |= �♦(costutil � r)π |= �♦(costutil � r)π |= �♦(costutil � r) iff∞∃ n∞∃ n∞∃ n s.t. cost(s0 s1...sn)
util(s0 s1...sn)� r
cost(s0 s1...sn)util(s0 s1...sn)
� rcost(s0 s1...sn)util(s0 s1...sn)
� r
278 / 373
![Page 279: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/279.jpg)
Best threshold for long-run ratios
given: MC with reward functions costcostcost,util : S → Nutil : S → Nutil : S → N
rational probability bound ppp
compute roptroptropt === inf{r ∈ R : PrM( lrrat � r ) > p
}inf
{r ∈ R : PrM( lrrat � r ) > p
}inf
{r ∈ R : PrM( lrrat � r ) > p
}roptroptropt === inf
{r ∈ R : PrM(�♦(costutil � r) ) > p
}inf
{r ∈ R : PrM(�♦(costutil � r) ) > p
}inf
{r ∈ R : PrM(�♦(costutil � r) ) > p
}=== inf
{r ∈ R : PrM(♦�(costutil � r) ) > p
}inf
{r ∈ R : PrM(♦�(costutil � r) ) > p
}inf
{r ∈ R : PrM(♦�(costutil � r) ) > p
}
π |= �♦(costutil � r)π |= �♦(costutil � r)π |= �♦(costutil � r) iff∞∃ n∞∃ n∞∃ n s.t. cost(s0 s1...sn)
util(s0 s1...sn)� r
cost(s0 s1...sn)util(s0 s1...sn)
� rcost(s0 s1...sn)util(s0 s1...sn)
� r
279 / 373
![Page 280: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/280.jpg)
Best threshold for long-run ratios
given: MC with reward functions costcostcost,util : S → Nutil : S → Nutil : S → N
rational probability bound ppp
compute roptroptropt === inf{r ∈ R : PrM( lrrat � r ) > p
}inf
{r ∈ R : PrM( lrrat � r ) > p
}inf
{r ∈ R : PrM( lrrat � r ) > p
}roptroptropt === inf
{r ∈ R : PrM(�♦(costutil � r) ) > p
}inf
{r ∈ R : PrM(�♦(costutil � r) ) > p
}inf
{r ∈ R : PrM(�♦(costutil � r) ) > p
}=== inf
{r ∈ R : PrM(♦�(costutil � r) ) > p
}inf
{r ∈ R : PrM(♦�(costutil � r) ) > p
}inf
{r ∈ R : PrM(♦�(costutil � r) ) > p
}=== min
{r ∈ Q : PrM(♦Cr) > p
}min
{r ∈ Q : PrM(♦Cr) > p
}min
{r ∈ Q : PrM(♦Cr) > p
}
where CrCrCr === union of all BSCCs BBB with lrrat(B) � rlrrat(B) � rlrrat(B) � r280 / 373
![Page 281: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/281.jpg)
Best threshold for long-run ratios
given: MC with reward functions costcostcost,util : S → Nutil : S → Nutil : S → N
rational probability bound ppp
compute roptroptropt === inf{r ∈ R : PrM( lrrat � r ) > p
}inf
{r ∈ R : PrM( lrrat � r ) > p
}inf
{r ∈ R : PrM( lrrat � r ) > p
}=== min
{r ∈ Q : PrM(♦Cr) > p
}min
{r ∈ Q : PrM(♦Cr) > p
}min
{r ∈ Q : PrM(♦Cr) > p
}where CrCrCr === union of all BSCCs BBB with lrrat(B) � rlrrat(B) � rlrrat(B) � r���
expected long-runratio of BBB
281 / 373
![Page 282: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/282.jpg)
Best threshold for long-run ratios
given: MC with reward functions costcostcost,util : S → Nutil : S → Nutil : S → N
rational probability bound ppp
compute roptroptropt === inf{r ∈ R : PrM( lrrat � r ) > p
}inf
{r ∈ R : PrM( lrrat � r ) > p
}inf
{r ∈ R : PrM( lrrat � r ) > p
}=== min
{r ∈ Q : PrM(♦Cr) > p
}min
{r ∈ Q : PrM(♦Cr) > p
}min
{r ∈ Q : PrM(♦Cr) > p
}where CrCrCr === union of all BSCCs BBB with lrrat(B) � rlrrat(B) � rlrrat(B) � r
1. compute the BSCCs B1, . . . ,BkB1, . . . ,BkB1, . . . ,Bk and ri = lrrat(Bi)ri = lrrat(Bi)ri = lrrat(Bi)
282 / 373
![Page 283: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/283.jpg)
Best threshold for long-run ratios
given: MC with reward functions costcostcost,util : S → Nutil : S → Nutil : S → N
rational probability bound ppp
compute roptroptropt === inf{r ∈ R : PrM( lrrat � r ) > p
}inf
{r ∈ R : PrM( lrrat � r ) > p
}inf
{r ∈ R : PrM( lrrat � r ) > p
}=== min
{r ∈ Q : PrM(♦Cr) > p
}min
{r ∈ Q : PrM(♦Cr) > p
}min
{r ∈ Q : PrM(♦Cr) > p
}where CrCrCr === union of all BSCCs BBB with lrrat(B) � rlrrat(B) � rlrrat(B) � r
1. compute the BSCCs B1, . . . ,BkB1, . . . ,BkB1, . . . ,Bk and ri = lrrat(Bi)ri = lrrat(Bi)ri = lrrat(Bi)
w.l.o.g. r1 < r2 < . . . < rkr1 < r2 < . . . < rkr1 < r2 < . . . < rk
283 / 373
![Page 284: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/284.jpg)
Best threshold for long-run ratios
given: MC with reward functions costcostcost,util : S → Nutil : S → Nutil : S → N
rational probability bound ppp
compute roptroptropt === inf{r ∈ R : PrM( lrrat � r ) > p
}inf
{r ∈ R : PrM( lrrat � r ) > p
}inf
{r ∈ R : PrM( lrrat � r ) > p
}=== min
{r ∈ Q : PrM(♦Cr) > p
}min
{r ∈ Q : PrM(♦Cr) > p
}min
{r ∈ Q : PrM(♦Cr) > p
}where CrCrCr === union of all BSCCs BBB with lrrat(B) � rlrrat(B) � rlrrat(B) � r
1. compute the BSCCs B1, . . . ,BkB1, . . . ,BkB1, . . . ,Bk and ri = lrrat(Bi)ri = lrrat(Bi)ri = lrrat(Bi)
w.l.o.g. r1 < r2 < . . . < rkr1 < r2 < . . . < rkr1 < r2 < . . . < rk
2. determine the minimal i ∈ {1, . . . , k}i ∈ {1, . . . , k}i ∈ {1, . . . , k} such that
PrM(♦B1) + . . .+ PrM(♦Bi) > pPrM(♦B1) + . . .+ PrM(♦Bi) > pPrM(♦B1) + . . .+ PrM(♦Bi) > p and return ririri284 / 373
![Page 285: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/285.jpg)
Cost-utility ratios: invariances
Given an MC with two positive reward functionscostcostcost,utilutilutil ::: S → NS → NS → N, consider their ratio:
ratio = costutil
ratio = costutilratio = costutil ::: FinPaths → QFinPaths → QFinPaths → Q
ratio(π)ratio(π)ratio(π) ===cost(π)util(π)cost(π)util(π)cost(π)util(π)
for all finite paths πππ
decision problems: given an ωωω-regular property ϕϕϕ andprobability bound q ∈ [0, 1[q ∈ [0, 1[q ∈ [0, 1[, ratio threshold r ∈ Qr ∈ Qr ∈ Q:
• does PrM(�(ratio � r) ∧ ϕ) > qPrM(�(ratio � r) ∧ ϕ) > qPrM(�(ratio � r) ∧ ϕ) > q hold ?
• does PrM(�(ratio � r) ∧ ϕ) = 1PrM(�(ratio � r) ∧ ϕ) = 1PrM(�(ratio � r) ∧ ϕ) = 1 hold ?285/373
![Page 286: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/286.jpg)
Cost-utility ratio via weight functions
Given an MC with two positive reward functionscostcostcost,utilutilutil ::: S → NS → NS → N, consider their ratio:
ratio = costutil
ratio = costutilratio = costutil ::: FinPaths → QFinPaths → QFinPaths → Q
ratio(π)ratio(π)ratio(π) ===cost(π)util(π)cost(π)util(π)cost(π)util(π)
for all finite paths πππ
�(ratio � r
)�(ratio � r
)�(ratio � r
) ≡≡≡ �(wgt � 0
)�(wgt � 0
)�(wgt � 0
)replace ratio by weight constraints:
286 / 373
![Page 287: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/287.jpg)
Cost-utility ratio via weight functions
Given an MC with two positive reward functionscostcostcost,utilutilutil ::: S → NS → NS → N, consider their ratio:
ratio = costutil
ratio = costutilratio = costutil ::: FinPaths → QFinPaths → QFinPaths → Q
ratio � rratio � rratio � r iff wgt � 0wgt � 0wgt � 0
where wgtwgtwgt === cost − r · utilcost − r · utilcost − r · util
�(ratio � r
)�(ratio � r
)�(ratio � r
) ≡≡≡ �(wgt � 0
)�(wgt � 0
)�(wgt � 0
)287 / 373
![Page 288: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/288.jpg)
Cost-utility ratio via weight functions
Given an MC with two positive reward functionscostcostcost,utilutilutil ::: S → NS → NS → N, consider their ratio:
ratio = costutil
ratio = costutilratio = costutil ::: FinPaths → QFinPaths → QFinPaths → Q
ratio � rratio � rratio � r iff wgt � 0wgt � 0wgt � 0
where wgtwgtwgt === cost − r · utilcost − r · utilcost − r · util ∈∈∈ QQQ
�(ratio � r
)�(ratio � r
)�(ratio � r
) ≡≡≡ �(wgt � 0
)�(wgt � 0
)�(wgt � 0
)288 / 373
![Page 289: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/289.jpg)
Cost-utility ratio via weight functions
Given an MC with two positive reward functionscostcostcost,utilutilutil ::: S → NS → NS → N, consider their ratio:
ratio = costutil
ratio = costutilratio = costutil ::: FinPaths → QFinPaths → QFinPaths → Q
ratio � rratio � rratio � r iff wgt > 0wgt > 0wgt > 0
where wgtwgtwgt === (cost − r · util) · const ∈ Z(cost − r · util) · const ∈ Z(cost − r · util) · const ∈ Z�integer-valuedweight function
�(ratio � r
)�(ratio � r
)�(ratio � r
) ≡≡≡ �(wgt > 0
)�(wgt > 0
)�(wgt > 0
)289 / 373
![Page 290: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/290.jpg)
Weight invariances for MC
Given an MC with a weight function wgtwgtwgt ::: S → ZS → ZS → Z.
290 / 373
![Page 291: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/291.jpg)
Weight invariances for MC
Given an MC with a weight function wgtwgtwgt ::: S → ZS → ZS → Z.
almost-sure problem:
does PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
=== 111 hold ?
positive problem:
does PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
>>> 000 hold ?
quantitative problems, e.g.:
does PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
>>> 121212 hold ?
291/373
![Page 292: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/292.jpg)
Weight invariances for MC
Given an MC with a weight function wgtwgtwgt ::: S → ZS → ZS → Z.
almost-sure problem:
does PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
=== 111 hold ?
positive problem:
does PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
>>> 000 hold ?
quantitative problems, e.g.:
does PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
>>> 121212 hold ?
simple
difficult
292/373
![Page 293: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/293.jpg)
Almost-sure weight invariances
PrMs(�(wgt > 0) ∧ ϕ )
PrMs(�(wgt > 0) ∧ ϕ )
PrMs(�(wgt > 0) ∧ ϕ )
=== 111
293 / 373
![Page 294: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/294.jpg)
Almost-sure weight invariances
PrMs(�(wgt > 0) ∧ ϕ )
PrMs(�(wgt > 0) ∧ ϕ )
PrMs(�(wgt > 0) ∧ ϕ )
=== 111
iff PrMs(�(wgt > 0)
)PrMs
(�(wgt > 0)
)PrMs
(�(wgt > 0)
)=== 111 and PrMs
(ϕ)
PrMs(ϕ)
PrMs(ϕ)=== 111
294 / 373
![Page 295: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/295.jpg)
Almost-sure weight invariances
PrMs(�(wgt > 0) ∧ ϕ )
PrMs(�(wgt > 0) ∧ ϕ )
PrMs(�(wgt > 0) ∧ ϕ )
=== 111
iff PrMs(�(wgt > 0)
)PrMs
(�(wgt > 0)
)PrMs
(�(wgt > 0)
)=== 111 and PrMs
(ϕ)
PrMs(ϕ)
PrMs(ϕ)=== 111
iff s �|= ∃♦(wgt � 0)s �|= ∃♦(wgt � 0)s �|= ∃♦(wgt � 0) and PrMs(ϕ)
PrMs(ϕ)
PrMs(ϕ)=== 111
295 / 373
![Page 296: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/296.jpg)
Almost-sure weight invariances
PrMs(�(wgt > 0) ∧ ϕ )
PrMs(�(wgt > 0) ∧ ϕ )
PrMs(�(wgt > 0) ∧ ϕ )
=== 111
iff PrMs(�(wgt > 0)
)PrMs
(�(wgt > 0)
)PrMs
(�(wgt > 0)
)=== 111 and PrMs
(ϕ)
PrMs(ϕ)
PrMs(ϕ)=== 111
iff s �|= ∃♦(wgt � 0)s �|= ∃♦(wgt � 0)s �|= ∃♦(wgt � 0) and PrMs(ϕ)
PrMs(ϕ)
PrMs(ϕ)=== 111���
solvable by standardshortest-path algorithms
296 / 373
![Page 297: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/297.jpg)
Almost-sure weight invariances
PrMs(�(wgt > 0) ∧ ϕ )
PrMs(�(wgt > 0) ∧ ϕ )
PrMs(�(wgt > 0) ∧ ϕ )
=== 111
iff PrMs(�(wgt > 0)
)PrMs
(�(wgt > 0)
)PrMs
(�(wgt > 0)
)=== 111 and PrMs
(ϕ)
PrMs(ϕ)
PrMs(ϕ)=== 111
iff s �|= ∃♦(wgt � 0)s �|= ∃♦(wgt � 0)s �|= ∃♦(wgt � 0) and PrMs(ϕ)
PrMs(ϕ)
PrMs(ϕ)=== 111���
solvable by standardshortest-path algorithms
���standard methods for
ωωω-regular path properties
polynomially time-bounded forreachability or Buchi properties
297 / 373
![Page 298: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/298.jpg)
Almost-sure weight invariances
PrMs(�(wgt > 0) ∧ ϕ )
PrMs(�(wgt > 0) ∧ ϕ )
PrMs(�(wgt > 0) ∧ ϕ )
=== 111
iff PrMs(�(wgt > 0)
)PrMs
(�(wgt > 0)
)PrMs
(�(wgt > 0)
)=== 111 and PrMs
(ϕ)
PrMs(ϕ)
PrMs(ϕ)=== 111
iff s �|= ∃♦(wgt � 0)s �|= ∃♦(wgt � 0)s �|= ∃♦(wgt � 0) and PrMs(ϕ)
PrMs(ϕ)
PrMs(ϕ)=== 111
Best threshold computable by shortest-path algorithms:
sup{r ∈ Z : PrMs ( �(wgt > r) ∧ ϕ ) = 1
}sup
{r ∈ Z : PrMs ( �(wgt > r) ∧ ϕ ) = 1
}sup
{r ∈ Z : PrMs ( �(wgt > r) ∧ ϕ ) = 1
}1 + length of a shortest path starting in state sss ,
provided that ϕ holds almost surely and there are no negative cycles298 / 373
![Page 299: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/299.jpg)
Weight invariances for MC
Given an MC with a weight function wgtwgtwgt ::: S → ZS → ZS → Z.
almost-sure problem:
does PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
=== 111 hold ?
positive problem:
does PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
>>> 000 hold ?
quantitative problems, e.g.:
does PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
PrMs0(�(wgt > 0) ∧ ϕ )
>>> 121212 hold ?
simple
difficult
299/373
![Page 300: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/300.jpg)
Markov chain with weight function
sss
s+s+s+s−s−s− ppp1−p1−p1−p
wgt(s)wgt(s)wgt(s) === +1+1+1
wgt(s−)wgt(s−)wgt(s−) === −2−2−2wgt(s+)wgt(s+)wgt(s+) === 000
probability parameter0 < p < 10 < p < 10 < p < 1
300 / 373
![Page 301: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/301.jpg)
Markov chain with weight function
sss
s+s+s+s−s−s− ppp1−p1−p1−p
wgt(s)wgt(s)wgt(s) === +1+1+1
wgt(s−)wgt(s−)wgt(s−) === −2−2−2wgt(s+)wgt(s+)wgt(s+) === 000
random walk:
. . .. . .. . . . . .. . .. . .
〈s , -2〉〈s , -2〉〈s, -2〉 〈s , -1〉〈s, -1〉〈s , -1〉 〈s , 0〉〈s, 0〉〈s , 0〉 〈s, 1〉〈s , 1〉〈s , 1〉 〈s , 2〉〈s , 2〉〈s, 2〉301 / 373
![Page 302: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/302.jpg)
Markov chain with weight function
sss
s+s+s+s−s−s− ppp1−p1−p1−p
wgt(s)wgt(s)wgt(s) === +1+1+1
wgt(s−)wgt(s−)wgt(s−) === −2−2−2wgt(s+)wgt(s+)wgt(s+) === 000
weight −1−1−1 for thecycle s s− ss s− ss s− srandom walk:
. . .. . .. . . . . .. . .. . .
〈s, -2〉〈s, -2〉〈s, -2〉 〈s, -1〉〈s, -1〉〈s, -1〉 〈s, 0〉〈s, 0〉〈s, 0〉 〈s, 1〉〈s, 1〉〈s, 1〉 〈s, 2〉〈s, 2〉〈s, 2〉
1−p1−p1−p 1−p1−p1−p 1−p1−p1−p 1−p1−p1−p 1−p1−p1−p 1−p1−p1−p
302 / 373
![Page 303: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/303.jpg)
Markov chain with weight function
sss
s+s+s+s−s−s− ppp1−p1−p1−p
wgt(s)wgt(s)wgt(s) === +1+1+1
wgt(s−)wgt(s−)wgt(s−) === −2−2−2wgt(s+)wgt(s+)wgt(s+) === 000
weight +1+1+1 for thecycle s s+ ss s+ ss s+ srandom walk:
. . .. . .. . . . . .. . .. . .
〈s, -2〉〈s, -2〉〈s, -2〉 〈s, -1〉〈s, -1〉〈s, -1〉 〈s, 0〉〈s, 0〉〈s, 0〉 〈s, 1〉〈s, 1〉〈s, 1〉 〈s, 2〉〈s, 2〉〈s, 2〉
1−p1−p1−p 1−p1−p1−p 1−p1−p1−p 1−p1−p1−p 1−p1−p1−p 1−p1−p1−p
ppp ppp ppp ppp ppp ppp
303 / 373
![Page 304: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/304.jpg)
Markov chain with weight function
sss
s+s+s+s−s−s− ppp1−p1−p1−p
wgt(s)wgt(s)wgt(s) === +1+1+1
wgt(s−)wgt(s−)wgt(s−) === −2−2−2wgt(s+)wgt(s+)wgt(s+) === 000
random walk:
. . .. . .. . . . . .. . .. . .
〈s, -2〉〈s, -2〉〈s, -2〉 〈s, -1〉〈s, -1〉〈s, -1〉 〈s, 0〉〈s, 0〉〈s, 0〉 〈s, 1〉〈s, 1〉〈s, 1〉 〈s, 2〉〈s, 2〉〈s, 2〉
1−p1−p1−p 1−p1−p1−p 1−p1−p1−p 1−p1−p1−p 1−p1−p1−p 1−p1−p1−p
ppp ppp ppp ppp ppp ppp
Prs(�(wgt > 0) ) > 0Prs(�(wgt > 0) ) > 0Prs(�(wgt > 0) ) > 0 iff p > 12p > 12p > 12
304 / 373
![Page 305: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/305.jpg)
Weight invariance problem: positive case
The problem “does Prs(�(wgt > r) ∧ ϕ ) > 0Prs(�(wgt > r) ∧ ϕ ) > 0Prs(�(wgt > r) ∧ ϕ ) > 0 hold ?”
• depends on the concrete transition probabilities
where ϕϕϕ is a ωωω-regular property and 0 � q < 10 � q < 10 � q < 1
305 / 373
![Page 306: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/306.jpg)
Weight invariance problem: positive case
The problem “does Prs(�(wgt > r) ∧ ϕ ) > 0Prs(�(wgt > r) ∧ ϕ ) > 0Prs(�(wgt > r) ∧ ϕ ) > 0 hold ?”
• depends on the concrete transition probabilities
• is solvable in polynomial time
BSCC-analysis and variants of shortest-paths algorithms,assuming ϕϕϕ is a Rabin or Streett or reachability condition
[Brazdil/Kiefer/Kucera/Novotny/Katoen’14]
[Krahmann/Schubert/Baier/Dubslaff’15]
where ϕϕϕ is a ωωω-regular property and 0 � q < 10 � q < 10 � q < 1
306 / 373
![Page 307: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/307.jpg)
Weight invariance problem: positive case
The problem “does Prs(�(wgt > r) ∧ ϕ ) > 0Prs(�(wgt > r) ∧ ϕ ) > 0Prs(�(wgt > r) ∧ ϕ ) > 0 hold ?”
• depends on the concrete transition probabilities
• is solvable in polynomial time
BSCC-analysis and variants of shortest-paths algorithms,assuming ϕϕϕ is a Rabin or Streett condition
check whether there exists a good BSCC BBB s.t.
1. MP(B) > 0MP(B) > 0MP(B) > 0 or MP(B) = 0MP(B) = 0MP(B) = 0 & no negative cycle in BBB
2. there is a path πππ from sss to BBB s.t. πππ and itsprefixes have sufficiently high weight
307 / 373
![Page 308: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/308.jpg)
Weight invariance problem: quantitative case
The problem “does Prs(�(wgt > r) ∧ ϕ ) > 0Prs(�(wgt > r) ∧ ϕ ) > 0Prs(�(wgt > r) ∧ ϕ ) > 0 hold ?”
• depends on the concrete transition probabilities
• is solvable in polynomial time
BSCC-analysis and variants of shortest-paths algorithms,assuming ϕϕϕ is a Rabin or Streett condition
The problem “does Prs(�(wgt > 0) ∧ ϕ ) > qPrs(�(wgt > 0) ∧ ϕ ) > qPrs(�(wgt > 0) ∧ ϕ ) > q hold ?”
• is reducible to the threshold problem forprobabilistic pushdown automata (exponential blowup)
308 / 373
![Page 309: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/309.jpg)
Weight invariance problem: quantitative case
The problem “does Prs(�(wgt > r) ∧ ϕ ) > 0Prs(�(wgt > r) ∧ ϕ ) > 0Prs(�(wgt > r) ∧ ϕ ) > 0 hold ?”
• depends on the concrete transition probabilities
• is solvable in polynomial time
BSCC-analysis and variants of shortest-paths algorithms,assuming ϕϕϕ is a Rabin or Streett condition
The problem “does Prs(�(wgt > 0) ∧ ϕ ) > qPrs(�(wgt > 0) ∧ ϕ ) > qPrs(�(wgt > 0) ∧ ϕ ) > q hold ?”
• is reducible to the threshold problem forprobabilistic pushdown automata (exponential blowup)
• is PosSLP-hard, even for unit weights and ϕ = trueϕ = trueϕ = true[Etessami/Yannak.’09], [Brazdil/Brozek/Etes./Kucera/Wojt.’10]
309 / 373
![Page 310: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/310.jpg)
Weight invariance problem: almost-sure case
The problem “does Prs(�(wgt > r) ∧ ϕ ) = 1Prs(�(wgt > r) ∧ ϕ ) = 1Prs(�(wgt > r) ∧ ϕ ) = 1 hold ?”
• independent from the concrete transition probabilities
• is solvable in polynomial time
310 / 373
![Page 311: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/311.jpg)
Weight invariance problem: almost-sure case
The problem “does Prs(�(wgt > r) ∧ ϕ ) = 1Prs(�(wgt > r) ∧ ϕ ) = 1Prs(�(wgt > r) ∧ ϕ ) = 1 hold ?”
• independent from the concrete transition probabilities
• is solvable in polynomial time
Prs(�(wgt > r) ∧ ϕ ) = 1Prs(�(wgt > r) ∧ ϕ ) = 1Prs(�(wgt > r) ∧ ϕ ) = 1
iff Prs(�(wgt > r) ) = 1Prs(�(wgt > r) ) = 1Prs(�(wgt > r) ) = 1 and Prs(ϕ) = 1Prs(ϕ) = 1Prs(ϕ) = 1
311 / 373
![Page 312: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/312.jpg)
Weight invariance problem: almost-sure case
The problem “does Prs(�(wgt > r) ∧ ϕ ) = 1Prs(�(wgt > r) ∧ ϕ ) = 1Prs(�(wgt > r) ∧ ϕ ) = 1 hold ?”
• independent from the concrete transition probabilities
• is solvable in polynomial time
Prs(�(wgt > r) ∧ ϕ ) = 1Prs(�(wgt > r) ∧ ϕ ) = 1Prs(�(wgt > r) ∧ ϕ ) = 1
iff Prs(�(wgt > r) ) = 1Prs(�(wgt > r) ) = 1Prs(�(wgt > r) ) = 1 and Prs(ϕ) = 1Prs(ϕ) = 1Prs(ϕ) = 1�standard algorithmpolynomial-time for
reachability, Rabin or Streett312 / 373
![Page 313: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/313.jpg)
Weight invariance problem: almost-sure case
The problem “does Prs(�(wgt > r) ∧ ϕ ) = 1Prs(�(wgt > r) ∧ ϕ ) = 1Prs(�(wgt > r) ∧ ϕ ) = 1 hold ?”
• independent from the concrete transition probabilities
• is solvable in polynomial time
Prs(�(wgt > r) ∧ ϕ ) = 1Prs(�(wgt > r) ∧ ϕ ) = 1Prs(�(wgt > r) ∧ ϕ ) = 1
iff Prs(�(wgt > r) ) = 1Prs(�(wgt > r) ) = 1Prs(�(wgt > r) ) = 1 and Prs(ϕ) = 1Prs(ϕ) = 1Prs(ϕ) = 1�shortest-path algorithm
check chether the weight of a shortestpath from s is at least r+1
313 / 373
![Page 314: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/314.jpg)
Weight-bounded reachability in MC
314/373
![Page 315: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/315.jpg)
Weight-bounded reachability in MC
given: weighted MCMMM, weight bound r ∈ Zr ∈ Zr ∈ Z
and a distinguished states sss, goalgoalgoal
decision problems:
positive prob: does Prs(♦�rgoal) > 0Prs(♦�rgoal) > 0Prs(♦�rgoal) > 0 hold ?
almost-sure: does Prs(♦�rgoal) = 1Prs(♦�rgoal) = 1Prs(♦�rgoal) = 1 hold ?
quantitative: does Prs(♦�rgoal) > 12
Prs(♦�rgoal) > 12Prs(♦�rgoal) > 12 hold ?
315/373
![Page 316: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/316.jpg)
Weight-bounded reachability in MC
given: weighted MCMMM, weight bound r ∈ Zr ∈ Zr ∈ Z
and a distinguished states sss, goalgoalgoal
decision problems:
positive prob: does Prs(♦�rgoal) > 0Prs(♦�rgoal) > 0Prs(♦�rgoal) > 0 hold ?solvable in poly-time using shortest-path algorithms
almost-sure: does Prs(♦�rgoal) = 1Prs(♦�rgoal) = 1Prs(♦�rgoal) = 1 hold ?solvable in poly-time using shortest-path algorithms;a bit tricky if goal is not a trap
quantitative: does Prs(♦�rgoal) > 12
Prs(♦�rgoal) > 12Prs(♦�rgoal) > 12 hold ?
solvable in poly-space using algorithms for prob PDA
316 / 373
![Page 317: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/317.jpg)
Weight-bounded reachability in MC
given: weighted MCMMM, weight bound r ∈ Zr ∈ Zr ∈ Z
and a distinguished states sss, goalgoalgoal
decision problems:
positive prob: does Prs(♦�rgoal) > 0Prs(♦�rgoal) > 0Prs(♦�rgoal) > 0 hold ?solvable in poly-time using shortest-path algorithms
almost-sure: does Prs(♦�rgoal) = 1Prs(♦�rgoal) = 1Prs(♦�rgoal) = 1 hold ?solvable in poly-time using shortest-path algorithms;a bit tricky if goal is not a trap
quantitative: does Prs(♦�rgoal) > 12
Prs(♦�rgoal) > 12Prs(♦�rgoal) > 12 hold ?
solvable in poly-space using algorithms for prob PDA
Is there an algorithm to compute Prs(♦�rgoal)Prs(♦�rgoal)Prs(♦�rgoal) ?317/373
![Page 318: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/318.jpg)
Weight-bounded reachability in MC [Kiefer’17]
sss
s+s+s+s−s−s−
goalgoalgoal
pppqqq
1−p−q1−p−q1−p−q
wgt(s)wgt(s)wgt(s) === 000
wgt(s−)wgt(s−)wgt(s−) === −1−1−1wgt(s+)wgt(s+)wgt(s+) === +1+1+1
probability parametersppp and qqq with 0 < p, q < 10 < p, q < 10 < p, q < 1
and p + q < 1p + q < 1p + q < 1
318 / 373
![Page 319: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/319.jpg)
Weight-bounded reachability in MC [Kiefer’17]
sss
s+s+s+s−s−s−
goalgoalgoal
pppqqq
1−p−q1−p−q1−p−q
wgt(s)wgt(s)wgt(s) === 000
wgt(s−)wgt(s−)wgt(s−) === −1−1−1wgt(s+)wgt(s+)wgt(s+) === +1+1+1
Prs(♦=0goal
)Prs
(♦=0goal
)Prs
(♦=0goal
)=== (1−p−q) ·
∞∑n=0
(2nn
)· pn · qn(1−p−q) ·
∞∑n=0
(2nn
)· pn · qn(1−p−q) ·
∞∑n=0
(2nn
)· pn · qn
319 / 373
![Page 320: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/320.jpg)
Weight-bounded reachability in MC [Kiefer’17]
sss
s+s+s+s−s−s−
goalgoalgoal
pppqqq
1−p−q1−p−q1−p−q
wgt(s)wgt(s)wgt(s) === 000
wgt(s−)wgt(s−)wgt(s−) === −1−1−1wgt(s+)wgt(s+)wgt(s+) === +1+1+1
Prs(♦=0goal
)Prs
(♦=0goal
)Prs
(♦=0goal
)=== (1−p−q) ·
∞∑n=0
(2nn
)· pn · qn(1−p−q) ·
∞∑n=0
(2nn
)· pn · qn(1−p−q) ·
∞∑n=0
(2nn
)· pn · qn
=== 1−p−q√1− 4 · p · q1−p−q√1− 4 · p · q1−p−q√1− 4 · p · q ... irrational
320 / 373
![Page 321: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/321.jpg)
Best threshold for ratio invariances
Given a Markov chainMMM with two reward functionsrew1, rew2 : S → Nrew1, rew2 : S → Nrew1, rew2 : S → N with rew2 > 0rew2 > 0rew2 > 0, consider their ratio
ratio : FinPaths → Qratio : FinPaths → Qratio : FinPaths → Q, ratio(π) =rew1(π)rew2(π)
ratio(π) =rew1(π)rew2(π)
ratio(π) =rew1(π)rew2(π)
examples:
• energy-utility ratio
• cost of repair mechanisms per failure
• SLA violations per day321 / 373
![Page 322: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/322.jpg)
Best threshold for ratio invariances
Given a Markov chainMMM with two reward functionsrew1, rew2 : S → Nrew1, rew2 : S → Nrew1, rew2 : S → N with rew2 > 0rew2 > 0rew2 > 0, consider their ratio:
ratio : FinPaths → Qratio : FinPaths → Qratio : FinPaths → Q, ratio(π) =rew1(π)rew2(π)
ratio(π) =rew1(π)rew2(π)
ratio(π) =rew1(π)rew2(π)
Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
examples:
• energy-utility ratio
• cost of repair mechanisms per failure
• SLA violations per day322 / 373
![Page 323: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/323.jpg)
Best threshold for ratio invariances
Given a Markov chainMMM with two reward functionsrew1, rew2 : S → Nrew1, rew2 : S → Nrew1, rew2 : S → N with rew2 > 0rew2 > 0rew2 > 0, consider their ratio:
ratio : FinPaths → Qratio : FinPaths → Qratio : FinPaths → Q, ratio(π) =rew1(π)rew2(π)
ratio(π) =rew1(π)rew2(π)
ratio(π) =rew1(π)rew2(π)
best threshold for qualitative ratio invariances:
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}supsupsup
{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) = 1Prs(�(ratio > r) ) = 1Prs(�(ratio > r) ) = 1
}}}examples:
• energy-utility ratio
• cost of repair mechanisms per failure
• SLA violations per day323 / 373
![Page 324: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/324.jpg)
Best threshold for ratio invariances
Given a Markov chainMMM with two reward functionsrew1, rew2 : S → Nrew1, rew2 : S → Nrew1, rew2 : S → N with rew2 > 0rew2 > 0rew2 > 0, consider their ratio:
ratio : FinPaths → Qratio : FinPaths → Qratio : FinPaths → Q, ratio(π) =rew1(π)rew2(π)
ratio(π) =rew1(π)rew2(π)
ratio(π) =rew1(π)rew2(π)
best threshold for qualitative ratio invariances:
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}supsupsup
{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) = 1Prs(�(ratio > r) ) = 1Prs(�(ratio > r) ) = 1
}}}... are computable in polynomial time ...
[Krahmann/Schubert/Baier/Dubslaff’15]
324 / 373
![Page 325: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/325.jpg)
Best threshold for ratio invariances
Given a Markov chainMMM with two reward functionsrew1, rew2 : S → Nrew1, rew2 : S → Nrew1, rew2 : S → N with rew2 > 0rew2 > 0rew2 > 0, consider their ratio:
ratio : FinPaths → Qratio : FinPaths → Qratio : FinPaths → Q, ratio(π) =rew1(π)rew2(π)
ratio(π) =rew1(π)rew2(π)
ratio(π) =rew1(π)rew2(π)
best threshold for qualitative ratio invariances:
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}supsupsup
{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) = 1Prs(�(ratio > r) ) = 1Prs(�(ratio > r) ) = 1
}}}... are computable in polynomial time ...
[Krahmann/Schubert/Baier/Dubslaff’15]
325 / 373
![Page 326: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/326.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}
ratio = rew1rew2
: FinPaths → Qratio = rew1rew2
: FinPaths → Qratio = rew1rew2
: FinPaths → Q where rew2 > 0rew2 > 0rew2 > 0
326 / 373
![Page 327: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/327.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}• inner decision problem for fixed rrr is solvable in
polynomial time
ratio = rew1rew2
: FinPaths → Qratio = rew1rew2
: FinPaths → Qratio = rew1rew2
: FinPaths → Q where rew2 > 0rew2 > 0rew2 > 0
327 / 373
![Page 328: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/328.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}• inner decision problem for fixed rrr is solvable in
polynomial time
reduction to positive weight invariances:
ratio > rratio > rratio > r iff rew1 − r ·rew2 > 0rew1 − r ·rew2 > 0rew1 − r ·rew2 > 0
ratio = rew1rew2
: FinPaths → Qratio = rew1rew2
: FinPaths → Qratio = rew1rew2
: FinPaths → Q where rew2 > 0rew2 > 0rew2 > 0
328 / 373
![Page 329: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/329.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}• inner decision problem for fixed rrr is solvable in
polynomial time
reduction to positive weight invariances:
ratio > rratio > rratio > r iff rew1 − r ·rew2 > 0rew1 − r ·rew2 > 0rew1 − r ·rew2 > 0︸ ︷︷ ︸weight function
ratio = rew1rew2
: FinPaths → Qratio = rew1rew2
: FinPaths → Qratio = rew1rew2
: FinPaths → Q where rew2 > 0rew2 > 0rew2 > 0
329 / 373
![Page 330: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/330.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}• inner decision problem for fixed rrr is solvable in
polynomial time
reduction to positive weight invariances:
ratio > rratio > rratio > r iff rew1 − r ·rew2 > 0rew1 − r ·rew2 > 0rew1 − r ·rew2 > 0︸ ︷︷ ︸weight function
If r ∈ Qr ∈ Qr ∈ Q then pick some c ∈ Nc ∈ Nc ∈ N such that(rew1 − r · rew2) · c(rew1 − r · rew2) · c(rew1 − r · rew2) · c is an integer weight function.
330 / 373
![Page 331: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/331.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}• inner decision problem for fixed rrr is solvable in
polynomial time
• quantile can be approximated using a binary search
ratio = rew1rew2
: FinPaths → Qratio = rew1rew2
: FinPaths → Qratio = rew1rew2
: FinPaths → Q where rew2 > 0rew2 > 0rew2 > 0331 / 373
![Page 332: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/332.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}• inner decision problem for fixed rrr is solvable in
polynomial time
• quantile can be approximated using a binary search
for all finite paths πππ:
0 � ratio(π) � max rew1min rew2
0 � ratio(π) � max rew1min rew2
0 � ratio(π) � max rew1min rew2
ratio = rew1rew2
: FinPaths → Qratio = rew1rew2
: FinPaths → Qratio = rew1rew2
: FinPaths → Q where rew2 > 0rew2 > 0rew2 > 0332 / 373
![Page 333: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/333.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}• inner decision problem for fixed rrr is solvable in
polynomial time
• quantile can be approximated using a binary searchand is one of the values
∗ expected long-run ratio of a BSCC
If BBB is a BSCC then the expected long-run ratio is:
MPB [rew1]
MPB [rew2]
MPB [rew1]
MPB [rew2]
MPB [rew1]
MPB [rew2]where MPB [rew ]MPB [rew ]MPB [rew ] ===
{mean-payoffof rewrewrew in BBB
333 / 373
![Page 334: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/334.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}• inner decision problem for fixed rrr is solvable in
polynomial time
• quantile can be approximated using a binary searchand is one of the values
∗ expected long-run ratio of a BSCC
∗ ratio(π)ratio(π)ratio(π) for a simple path πππ from sss
∗ ratio(π)ratio(π)ratio(π) for a simple cycle πππ reachable from sss
334 / 373
![Page 335: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/335.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}• inner decision problem for fixed rrr is solvable in
polynomial time
• quantile can be approximated using a binary searchand is one of the values
∗ expected long-run ratio of a BSCC
∗ ratio(π)ratio(π)ratio(π) for a simple path πππ from sss
∗ ratio(π)ratio(π)ratio(π) for a simple cycle πππ reachable from sssfinitely
many
values
335/ 373
![Page 336: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/336.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}• inner decision problem for fixed rrr is solvable in
polynomial time
• quantile can be approximated using a binary searchand is one of the values . . .. . .. . . and therefore rational
∗ expected long-run ratio of a BSCC
∗ ratio(π)ratio(π)ratio(π) for a simple path πππ from sss
∗ ratio(π)ratio(π)ratio(π) for a simple cycle πππ reachable from sssfinitely
many
values
336/ 373
![Page 337: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/337.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}• inner decision problem for fixed rrr is solvable in
polynomial time
• quantile can be approximated using a binary searchand is one of the values . . .. . .. . . and therefore rational
∗ expected long-run ratio of a BSCC
∗ ratio(π)ratio(π)ratio(π) for a simple path πππ from sss
∗ ratio(π)ratio(π)ratio(π) for a simple cycle πππ reachable from sssfinitely
many
values
• computation using the continued-fraction method337 / 373
![Page 338: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/338.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}=== c
dcdcd
where ccc ,d ∈ Nd ∈ Nd ∈ N with d > 0d > 0d > 0
• quantile can be approximated using a binary searchand is one of the values . . .. . .. . . and therefore rational
∗ expected long-run ratio of a BSCC
∗ ratio(π)ratio(π)ratio(π) for a simple path πππ from sss
∗ ratio(π)ratio(π)ratio(π) for a simple cycle πππ reachable from sssfinitely
many
values
• computation using the continued-fraction method338 / 373
![Page 339: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/339.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}=== c
dcdcd
where d � Dd � Dd � D === max{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}• quantile can be approximated using a binary search
and is one of the values . . .. . .. . . and therefore rational
∗ expected long-run ratio cB/dBcB/dBcB/dB of BSCC BBB
∗ ratio(π)ratio(π)ratio(π) for a simple path πππ from sss
∗ ratio(π)ratio(π)ratio(π) for a simple cycle πππ reachable from sss
• computation using the continued-fraction method339 / 373
![Page 340: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/340.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}=== c
dcdcd
where d � Dd � Dd � D === max{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}1. compute an approximation ppp of the quantile
up to precision ε = 1/2D2ε = 1/2D2ε = 1/2D2
∣∣cd − p
∣∣∣∣cd − p
∣∣∣∣cd − p
∣∣ <<< εεε340 / 373
![Page 341: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/341.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}=== c
dcdcd
where d � Dd � Dd � D === max{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}1. compute an approximation ppp of the quantile
up to precision ε = 1/2D2ε = 1/2D2ε = 1/2D2
The quantile is the best rational approximation of ppp withdenominator at most DDD∣∣c
d − p∣∣∣∣c
d − p∣∣∣∣c
d − p∣∣ <<< εεε
341 / 373
![Page 342: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/342.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}=== c
dcdcd
where d � Dd � Dd � D === max{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}1. compute an approximation ppp of the quantile
up to precision ε = 1/2D2ε = 1/2D2ε = 1/2D2
The quantile is the best rational approximation of ppp withdenominator at most DDD, i.e., if aaa,b ∈ Nb ∈ Nb ∈ N with 0 < b � D0 < b � D0 < b � Dthen: ∣∣a
b − p∣∣∣∣a
b − p∣∣∣∣a
b − p∣∣ <<< εεε iff a
babab === c
dcdcd
342 / 373
![Page 343: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/343.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}=== c
dcdcd
where d � Dd � Dd � D === max{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}1. compute an approximation ppp of the quantile
up to precision ε = 1/2D2ε = 1/2D2ε = 1/2D2
2. apply the continued-fraction method to ppp
The quantile is the best rational approximation of ppp withdenominator at most DDD, i.e., if aaa,b ∈ Nb ∈ Nb ∈ N with 0 < b � D0 < b � D0 < b � Dthen: ∣∣a
b − p∣∣∣∣a
b − p∣∣∣∣a
b − p∣∣ <<< εεε iff a
babab === c
dcdcd
343 / 373
![Page 344: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/344.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}=== c
dcdcd
where d � Dd � Dd � D === max{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}1. compute an approximation ppp of the quantile
up to precision ε = 1/2D2ε = 1/2D2ε = 1/2D2
2. apply the continued-fraction method to ppp
ppp === p1 +1
p2 +1
p3 +1
p4 +1
. . .
p1 +1
p2 +1
p3 +1
p4 +1
. . .
p1 +1
p2 +1
p3 +1
p4 +1
. . . 344 / 373
![Page 345: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/345.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}=== c
dcdcd
where d � Dd � Dd � D === max{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}1. compute an approximation ppp of the quantile
up to precision ε = 1/2D2ε = 1/2D2ε = 1/2D2
2. apply the continued-fraction method to ppp[Grotschel/Lovasz/Schrijver’87]
ppp
345 / 373
![Page 346: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/346.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}=== c
dcdcd
where d � Dd � Dd � D === max{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}1. compute an approximation ppp of the quantile
up to precision ε = 1/2D2ε = 1/2D2ε = 1/2D2
2. apply the continued-fraction method to ppp[Grotschel/Lovasz/Schrijver’87]
pppq1q1q1
346 / 373
![Page 347: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/347.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}=== c
dcdcd
where d � Dd � Dd � D === max{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}1. compute an approximation ppp of the quantile
up to precision ε = 1/2D2ε = 1/2D2ε = 1/2D2
2. apply the continued-fraction method to ppp[Grotschel/Lovasz/Schrijver’87]
pppq1q1q1 q2q2q2
347 / 373
![Page 348: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/348.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}=== c
dcdcd
where d � Dd � Dd � D === max{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}1. compute an approximation ppp of the quantile
up to precision ε = 1/2D2ε = 1/2D2ε = 1/2D2
2. apply the continued-fraction method to ppp[Grotschel/Lovasz/Schrijver’87]
pppq1q1q1 q2q2q2q3q3q3
348 / 373
![Page 349: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/349.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}=== c
dcdcd
where d � Dd � Dd � D === max{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}1. compute an approximation ppp of the quantile
up to precision ε = 1/2D2ε = 1/2D2ε = 1/2D2
2. apply the continued-fraction method to ppp[Grotschel/Lovasz/Schrijver’87]
pppq1q1q1 q2q2q2q3q3q3 q4q4q4
349 / 373
![Page 350: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/350.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}=== c
dcdcd
where d � Dd � Dd � D === max{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}1. compute an approximation ppp of the quantile
up to precision ε = 1/2D2ε = 1/2D2ε = 1/2D2
2. apply the continued-fraction method to ppp[Grotschel/Lovasz/Schrijver’87]
pppq1q1q1 q2q2q2q3q3q3 q4q4q4q5q5q5
350 / 373
![Page 351: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/351.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}=== c
dcdcd
where d � Dd � Dd � D === max{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}1. compute an approximation ppp of the quantile
up to precision ε = 1/2D2ε = 1/2D2ε = 1/2D2
2. apply the continued-fraction method to ppp[Grotschel/Lovasz/Schrijver’87]
︸ ︷︷ ︸εεε
︸ ︷︷ ︸εεε
pppq1q1q1 q2q2q2q3q3q3 q4q4q4q5q5q5
351 / 373
![Page 352: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/352.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}=== c
dcdcd
where d � Dd � Dd � D === max{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}1. compute an approximation ppp of the quantile
up to precision ε = 1/2D2ε = 1/2D2ε = 1/2D2
2. apply the continued-fraction method to ppp
︸ ︷︷ ︸εεε
︸ ︷︷ ︸εεε
pppq1q1q1 q2q2q2q3q3q3 q4q4q4q5q5q5
denominator > D> D> D
352 / 373
![Page 353: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/353.jpg)
Positive ratio quantiles
supsupsup{{{r ∈ Qr ∈ Qr ∈ Q ::: Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0Prs(�(ratio > r) ) > 0
}}}=== c
dcdcd
where d � Dd � Dd � D === max{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}max
{maxB
dB , |S |·max rew2
}1. compute an approximation ppp of the quantile
up to precision ε = 1/2D2ε = 1/2D2ε = 1/2D2
2. apply the continued-fraction method to ppp
︸ ︷︷ ︸εεε
︸ ︷︷ ︸εεε
ppp cdcdcd
q1q1q1 q2q2q2q3q3q3 q5q5q5
denominator > D> D> D
353 / 373
![Page 354: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/354.jpg)
Polynomially computable ratio quantiles in MC
qualitative quantiles for ratio invariances:
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
> 0}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
> 0}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
> 0}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
= 1}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
= 1}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
= 1}
where ϕϕϕ is a reachability, Rabin or Streett condition
354 / 373
![Page 355: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/355.jpg)
Polynomially computable ratio quantiles in MC
qualitative quantiles for ratio invariances:
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
> 0}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
> 0}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
> 0}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
= 1}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
= 1}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
= 1}
where ϕϕϕ is a reachability, Rabin or Streett condition
︸ ︷︷ ︸Prs(ϕ) = 1Prs(ϕ) = 1Prs(ϕ) = 1 and s �|= ∃♦(wgtr � 0)s �|= ∃♦(wgtr � 0)s �|= ∃♦(wgtr � 0)
where wgtr = cost − r ·utilwgtr = cost − r ·utilwgtr = cost − r ·util... binary search for maximal rrr and shortest-path algorithms ...
355 / 373
![Page 356: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/356.jpg)
Polynomially computable ratio quantiles in MC
qualitative quantiles for ratio invariances:
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
> 0}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
> 0}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
> 0}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
= 1}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
= 1}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
= 1}
qualitative and quantitative quantiles for long-run ratios:
sup{r ∈ Q : Prs
(�♦(ratio > r) ∧ ϕ )
= 1}
sup{r ∈ Q : Prs
(�♦(ratio > r) ∧ ϕ )
= 1}
sup{r ∈ Q : Prs
(�♦(ratio > r) ∧ ϕ )
= 1}
sup{r ∈ Q : Prs
(�♦(ratio > r) ∧ ϕ )
> q}
sup{r ∈ Q : Prs
(�♦(ratio > r) ∧ ϕ )
> q}
sup{r ∈ Q : Prs
(�♦(ratio > r) ∧ ϕ )
> q}
where ϕϕϕ is a reachability, Rabin or Streett condition
356 / 373
![Page 357: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/357.jpg)
Polynomially computable ratio quantiles in MC
qualitative quantiles for ratio invariances:
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
> 0}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
> 0}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
> 0}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
= 1}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
= 1}
sup{r ∈ Q : Prs
(�(ratio > r) ∧ ϕ )
= 1}
qualitative and quantitative quantiles for long-run ratios:
sup{r ∈ Q : Prs
(�♦(ratio > r) ∧ ϕ )
= 1}
sup{r ∈ Q : Prs
(�♦(ratio > r) ∧ ϕ )
= 1}
sup{r ∈ Q : Prs
(�♦(ratio > r) ∧ ϕ )
= 1}
sup{r ∈ Q : Prs
(�♦(ratio > r) ∧ ϕ )
> q}
sup{r ∈ Q : Prs
(�♦(ratio > r) ∧ ϕ )
> q}
sup{r ∈ Q : Prs
(�♦(ratio > r) ∧ ϕ )
> q}
=== minminmin{r ∈ Q : Prs
(♦Cr
)> q
}{r ∈ Q : Prs
(♦Cr
)> q
}{r ∈ Q : Prs
(♦Cr
)> q
}where CrCrCr === union of “good” BSCCs BBB with lrrat(B) � rlrrat(B) � rlrrat(B) � r
357 / 373
![Page 358: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/358.jpg)
Tutorial: Probabilistic Model Checking
Discrete-time Markov chains (DTMC)
∗ basic definitions
∗ probabilistic computation tree logic PCTL/PCTL*
∗ rewards, cost-utility ratios, weights
∗ conditional probabilities
Markov decision processes (MDP)
∗ basic definitions
∗ PCTL/PCTL* model checking
∗ fairness
∗ conditional probabilities
∗ rewards, quantiles
∗ mean-payoff
∗ expected accumulated weights358 / 373
![Page 359: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/359.jpg)
Conditional probabilities
359/ 373
![Page 360: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/360.jpg)
Conditional probabilities
• useful for various multi-objective properties
e.g. analyze the gained utility for a given energy budget
Prs(♦�u goal | ♦�e goal
)Prs
(♦�u goal | ♦�e goal
)Prs
(♦�u goal | ♦�e goal
)or
ExpUtils(
goal | ♦�e goal )ExpUtils(
goal | ♦�e goal )ExpUtils(
goal | ♦�e goal )
♦�u goal♦�u goal♦�u goal “gained utility for reaching the goal is at least u”
♦�e goal♦�e goal♦�e goal “consumed energy until reaching the goal is at most e”
360 / 373
![Page 361: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/361.jpg)
Conditional probabilities
• useful for various multi-objective properties
e.g. analyze the gained utility for a given energy budget
Prs(♦�u goal | ♦�e goal
)Prs
(♦�u goal | ♦�e goal
)Prs
(♦�u goal | ♦�e goal
)or
ExpUtils(
goal | ♦�e goal )ExpUtils(
goal | ♦�e goal )ExpUtils(
goal | ♦�e goal )• useful for failure diagnosis
e.g. study the impact of failures and cost of repair mechanisms
in resilient systems
Prs(♦goal | ♦failure )Prs
(♦goal | ♦failure )Prs
(♦goal | ♦failure ) or
ExpCosts(
goal | ♦failure )ExpCosts(
goal | ♦failure )ExpCosts(
goal | ♦failure )361 / 373
![Page 362: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/362.jpg)
Conditional probabilities
for Markov chains:
PrMs (ϕ |ψ )PrMs (ϕ |ψ )PrMs (ϕ |ψ ) ===PrMs (ϕ ∧ ψ )PrMs (ϕ ∧ ψ )PrMs (ϕ ∧ ψ )
PrMs (ψ )PrMs (ψ )PrMs (ψ )
provided PrMs (ψ) > 0PrMs (ψ) > 0PrMs (ψ) > 0
362 / 373
![Page 363: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/363.jpg)
Conditional probabilities
for Markov chains:
PrMs (ϕ |ψ )PrMs (ϕ |ψ )PrMs (ϕ |ψ ) ===PrMs (ϕ ∧ ψ )PrMs (ϕ ∧ ψ )PrMs (ϕ ∧ ψ )
PrMs (ψ )PrMs (ψ )PrMs (ψ )
• discrete MCs and PCTL [Andres/Rossum’08]
[Ji/Wu/Chen’13]
• continuous-time MCs and CSL [Gao/Xu/Zhan/Zhang’13]
PCTL: probabilistic computation tree logic
CSL: continuous stochastic logic363 / 373
![Page 364: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/364.jpg)
Conditional probabilities
for Markov chains:
PrMs (ϕ |ψ )PrMs (ϕ |ψ )PrMs (ϕ |ψ ) ===PrMs (ϕ ∧ ψ )PrMs (ϕ ∧ ψ )PrMs (ϕ ∧ ψ )
PrMs (ψ )PrMs (ψ )PrMs (ψ )
• discrete MCs and PCTL [Andres/Rossum’08]
[Ji/Wu/Chen’13]
• continuous-time MCs and CSL [Gao/Xu/Zhan/Zhang’13]
transformation-based approach for LTL conditions
MCMMM��� MCMψMψMψ: [Baier/Klein/Kluppelholz/Marcker’14]
PrMs (ϕ |ψ )PrMs (ϕ |ψ )PrMs (ϕ |ψ ) === PrMψs (ϕ )PrMψs (ϕ )PrMψs (ϕ )
364 / 373
![Page 365: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/365.jpg)
Transformation-based approach for MC
given: Markov chainM = (S ,P)M = (S ,P)M = (S ,P) and ψ = ♦Gψ = ♦Gψ = ♦G
define Markov chainMψMψMψ s.t. for all LTL formulas ϕϕϕ
PrMs (ϕ |♦G )PrMs (ϕ |♦G )PrMs (ϕ |♦G ) === PrMψs (ϕ )PrMψs (ϕ )PrMψs (ϕ )
LTL: linear temporal logic365 / 373
![Page 366: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/366.jpg)
Transformation-based approach for MC
given: Markov chainM = (S ,P)M = (S ,P)M = (S ,P) and ψ = ♦Gψ = ♦Gψ = ♦G
define Markov chainMψMψMψ s.t. for all LTL formulas ϕϕϕ
PrMs (ϕ |♦G )PrMs (ϕ |♦G )PrMs (ϕ |♦G ) === PrMψs (ϕ )PrMψs (ϕ )PrMψs (ϕ )
MCMMM
GGG
LTL: linear temporal logic366 / 373
![Page 367: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/367.jpg)
Transformation-based approach for MC
given: Markov chainM = (S ,P)M = (S ,P)M = (S ,P) and ψ = ♦Gψ = ♦Gψ = ♦G
define Markov chainMψMψMψ s.t. for all LTL formulas ϕϕϕ
PrMs (ϕ |♦G )PrMs (ϕ |♦G )PrMs (ϕ |♦G ) === PrMψs (ϕ )PrMψs (ϕ )PrMψs (ϕ )
MCMMM
GGG
¬∃♦G¬∃♦G¬∃♦G ∃♦G∃♦G∃♦G LTL: linear temporal logic367 / 373
![Page 368: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/368.jpg)
Transformation-based approach for MC
given: Markov chainM = (S ,P)M = (S ,P)M = (S ,P) and ψ = ♦Gψ = ♦Gψ = ♦G
define Markov chainMψMψMψ s.t. for all LTL formulas ϕϕϕ
PrMs (ϕ |♦G )PrMs (ϕ |♦G )PrMs (ϕ |♦G ) === PrMψs (ϕ )PrMψs (ϕ )PrMψs (ϕ )
MCMMM
GGG
¬∃♦G¬∃♦G¬∃♦G ∃♦G∃♦G∃♦G
MCMψMψMψ
GGG
“before GGG”copy ofM∣∣
∃♦GM∣∣∃♦GM∣∣∃♦G
“after GGG”copy ofMMM
368 / 373
![Page 369: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/369.jpg)
Transformation-based approach for MC
given: Markov chainM = (S ,P)M = (S ,P)M = (S ,P) and ψ = ♦Gψ = ♦Gψ = ♦G
define Markov chainMψMψMψ s.t. for all LTL formulas ϕϕϕ
PrMs (ϕ |♦G )PrMs (ϕ |♦G )PrMs (ϕ |♦G ) === PrMψs (ϕ )PrMψs (ϕ )PrMψs (ϕ )
MCMMM
GGG
¬∃♦G¬∃♦G¬∃♦G ∃♦G∃♦G∃♦G
MCMψMψMψ
GGG
“before GGG”copy ofM∣∣
∃♦GM∣∣∃♦GM∣∣∃♦G
Pψ(s, t)Pψ(s, t)Pψ(s, t) === P(s, t)P(s, t)P(s, t)
369 / 373
![Page 370: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/370.jpg)
Transformation-based approach for MC
given: Markov chainM = (S ,P)M = (S ,P)M = (S ,P) and ψ = ♦Gψ = ♦Gψ = ♦G
define Markov chainMψMψMψ s.t. for all LTL formulas ϕϕϕ
PrMs (ϕ |♦G )PrMs (ϕ |♦G )PrMs (ϕ |♦G ) === PrMψs (ϕ )PrMψs (ϕ )PrMψs (ϕ )
MCMMM
GGG
¬∃♦G¬∃♦G¬∃♦G ∃♦G∃♦G∃♦G
MCMψMψMψ
GGG
Pψ(s, t)Pψ(s, t)Pψ(s, t) === P(s, t)P(s, t)P(s, t) ··· PrMt (♦G )PrMt (♦G )PrMt (♦G)
PrMs (♦G )PrMs (♦G )PrMs (♦G)370 / 373
![Page 371: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/371.jpg)
Transformation-based approach for MC
given: Markov chainM = (S ,P)M = (S ,P)M = (S ,P) and ψ = ♦Gψ = ♦Gψ = ♦G
define Markov chainMψMψMψ s.t. for all LTL formulas ϕϕϕ
PrMs (ϕ |♦G )PrMs (ϕ |♦G )PrMs (ϕ |♦G ) === PrMψs (ϕ )PrMψs (ϕ )PrMψs (ϕ )
... can be generalized for other temporal conditions ψψψ
either by adapting the definition ofMψMψMψ orby using an ωωω-automaton for LTL conditions
371 / 373
![Page 372: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/372.jpg)
Transformation-based approach for MC
given: Markov chainM = (S ,P)M = (S ,P)M = (S ,P) and ψ = ♦Gψ = ♦Gψ = ♦G
define Markov chainMψMψMψ s.t. for all LTL formulas ϕϕϕ
PrMs (ϕ |♦G )PrMs (ϕ |♦G )PrMs (ϕ |♦G ) === PrMψs (ϕ )PrMψs (ϕ )PrMψs (ϕ )
... can be generalized for other temporal conditions ψψψ
same method applicable for conditional expectations
EMs ( f |ψ )EMs ( f |ψ )EMs ( f |ψ ) === EMψs ( f ′ )EMψs ( f ′ )EMψs ( f ′ )
e.g.: EMs (EMs (EMs ( “energy until reaching the goal” ||| ♦goal )♦goal )♦goal )372 / 373
![Page 373: Tutorial: Probabilistic ModelChecking...Probabilisticmodels purely probabilistic probabilisticand nondeterministic discrete time discrete-time Markovchain (DTMC) Markovdecision process(MDP)](https://reader035.vdocuments.net/reader035/viewer/2022062609/5fb7f856e219c828c62f028a/html5/thumbnails/373.jpg)
Tutorial: Probabilistic Model Checking
Discrete-time Markov chains (DTMC)
∗ basic definitions
∗ probabilistic computation tree logic PCTL/PCTL*
∗ rewards, cost-utility ratios, weights
∗ conditional probabilities
Markov decision processes (MDP)
∗ basic definitions
∗ PCTL/PCTL* model checking
∗ fairness
∗ conditional probabilities
∗ rewards, quantiles
∗ mean-payoff
∗ expected accumulated weights373 / 373