model checking epistemic–probabilistic logic using probabilistic interpreted systems

Knowledge-Based Systems 50 (2013) 279–295

Contents lists available at SciVerse ScienceDirect

Knowledge-Based Systems

journal homepage: www.elsevier .com/ locate /knosys

Model checking epistemic–probabilistic logic using probabilisticinterpreted systems q

0950-7051/$ - see front matter � 2013 Elsevier B.V. All rights reserved.http://dx.doi.org/10.1016/j.knosys.2013.06.017

q This manuscript is an extension of a previous version accepted for publication atSoMeT_2012 [50]. The new version extends the previous one by: (1) extending thediscussion of related work (Section 2); (2) refining and extending the theoreticalresults (Section 4); (3) extending the model checking technique (Section 5); (4)adding a complete case study along with its implementation, the performanceanalysis, and comparison with two existing model-checkers’ performances (Section6); and (5) adding new examples and illustrations in different sections.⇑ Corresponding author.

E-mail addresses: [email protected] (W. Wan), [email protected] (J. Bentahar), [email protected] (A. Ben Hamza).

1 This work has been performed while Jamal Bentahar is visiting Khalifa Universityof Science, Technology, and Research, UAE.

Wei Wan, Jamal Bentahar ⇑,1, Abdessamad Ben HamzaConcordia Institute for Information Systems Engineering, Concordia University, Montreal, Canada

a r t i c l e i n f o a b s t r a c t

Article history:Received 24 June 2012Received in revised form 22 June 2013Accepted 24 June 2013Available online 5 July 2013

Keywords:Model CheckingVerificationInterpreted systemsMulti-agent systemsMarkov chainsMarkov decision processesProbabilistic and epistemic logic

Model checking is a formal technique widely used to verify security and communication protocols in epi-stemic multi-agent systems against given properties. Qualitative properties such as safety and livelinesshave been widely analyzed in the literature. However, systems have also quantitative and uncertain (i.e.,probabilistic) properties such as degree of reliability and reachability, which still need further attentionfrom the model checking perspective. In this paper, we analyze such properties and present a newmethod for probabilistic model checking of epistemic multi-agent systems specified by a new probabilis-tic–epistemic logic PCTLK. We model multi-agent systems as distributed knowledge bases using proba-bilistic interpreted systems and define transformations from those interpreted systems into discrete-timeMarkov chains and from PCTLK formulae to PCTL formulae, an existing extension of CTL with probabili-ties. By so doing, we are able to convert the PCTLK model checking problem into the PCTL one. Thus, wemake use of PRISM, the model checker of PCTL without adding new computation cost. A concrete casestudy has been implemented to show the applicability of the proposed technique along with performanceanalysis and comparison with MCK, an epistemic–probabilistic model checker, and MCMAS, a modelchecker for multi-agent systems, in terms of execution time and state space scalability.

� 2013 Elsevier B.V. All rights reserved.

1. Introduction

Model checking is a formal, fully automatic, well-designed tech-nique to verify whether or not system design models satisfy givenrequirements [30]. In recent years, this technique has been appliedto a wide range of systems and applications including process-based systems [40], multi-agent applications [35,39], agent com-munication [5], and service composition [6,36]. In conventionalmodel checking, such as the technique used in [6], verification onlyfocuses on the absolute accuracy of properties in the model beingconstructed, which means whether the checked properties are trueor false. However, actual scenarios are rarely absolutely reliablebut most often probabilistic and systems are subject to stochastic

phenomena. For instance, in distributed systems, situations suchas ‘‘the message will be delivered successfully with probability of95%’’ and ‘‘the channel is 75% error free’’ are common. In multi-agent settings, it is also desirable to express properties such as‘‘an agent knows that items could be lost with a chance of 30%’’.Considering quantitative aspects when modeling the systemallows the assessment of the likelihood of different events. In fact,an appropriate reaction to an event depends on the confidence onewould have about the occurrence of that event. For instance, if theagent knows that the message will be successfully delivered with aprobability 0.8, then she should consider other ways such as send-ing duplicate copies. Accounting for stochastic phenomena in epi-stemic systems, which are the main focus of this paper, andverifying their correctness are important aspects in concrete appli-cations [3,10,24,22,49].

There are two main frameworks for representing and reasoningabout epistemic systems: Partially Observable Markov DecisionProcesses (POMDPs) and interpreted systems. On the one hand,POMDPs, which are a generalization of Markov Decision Processes(MDPs), have been used to model the uncertainty of knowledgeand behavior for stochastic agents since the 1990s [8,18,26,27].Recently, POMDPs have been used extensively in machine learning[1,42,12], agent decision making [38], and robotic applications[28,43]. In the POMDPs-based framework, agents only observethe underlying states partially and maintain a probability

http://crossmark.dyndns.org/dialog/?doi=10.1016/j.knosys.2013.06.017&domain=pdf

http://dx.doi.org/10.1016/j.knosys.2013.06.017

mailto:[email protected]




http://dx.doi.org/10.1016/j.knosys.2013.06.017

http://www.sciencedirect.com/science/journal/09507051

http://www.elsevier.com/locate/knosys

280 W. Wan et al. / Knowledge-Based Systems 50 (2013) 279–295

distribution over the set of possible states, called belief states,which are computed based on a set of observations. On the otherhand, the interpreted systems formalism [16] that formalizes agentmodels has proven its value in representing, modeling and verify-ing epistemic systems [46]. Using interpreted systems, epistemiclogics to reason about knowledge and time are thoroughly investi-gated and extensively used in specifying and verifying multi-agentsystems [34,35,37,39,46]. Epistemic modalities have been devel-oped to represent not only an individual agent’s knowledge, butalso a group’s knowledge such as common knowledge in differentmodels of time including linear and branching [45]. They have alsobeen investigated within a first order logic where quantified inter-preted systems are introduced to define a semantics to reasonabout knowledge and time in a first order setting [3]. Soundnessand completeness issues are discussed in this paper. However,the quantifications in this work are first order quantifications,and not uncertainty quantifications as we propose in our work.In fact, using interpreted systems to specify agents’ uncertaintyof knowledge is still in the early stages and verifying agents’ prob-abilistic (i.e., uncertain) knowledge using interpreted systems isstill a fertile research topic. In this paper, we aim to use knowledgerepresentation techniques through the definition of a new logicand interpreted systems to express not only qualitative, but alsoquantitative and uncertain knowledge and investigate the modelchecking of the defined logic.

To summarize, there are two ways of representing and reason-ing about stochastic epistemic systems: POMDPs and extension ofinterpreted systems with probabilistic and uncertain behavior. Thefirst option has been widely studied. However, the second option isyet to be investigated. The purpose of this paper is to examine thisoption, not only from the knowledge representation perspective,but also from the verification and model checking points of view.This choice is motivated by the fact that interpreted systems pro-vide a natural and elegant way of capturing the philosophical foun-dations of knowledge using possible and accessible worlds, agentlocal states, and system global states. Simply put, in this formalism,an agent’s knowledge is captured by the information stored in alllocal equivalent states of the current state, which means states thatthe agent cannot distinguish. Thus, an agent knows u in a givenstate iff u is true in all the equivalent local states of that state(those states are said to be possible or accessible). Such a rich inter-pretation is not captured by POMDPs.

There are two questions that must be answered in order tocheck uncertain, quantitative-epistemic properties: how to specifymeasurable epistemic properties and how to represent modelscapturing measurable epistemic features. Uncertain knowledgecan be represented using probabilities and the multi-agent systemcan be modeled as a probabilistic Kripke-like model. In fact, themulti-agent system is a distributed probabilistic knowledge-basedsystem where components are autonomous and selfish. In this pa-per, we integrate Markov chains structure into interpreted systemsto express probabilistic multi-agent systems. To specify the quan-titative properties of these systems, we build on and extend ourprevious work [49,50], in which a probabilisticepistemic logicwas proposed via the combination of temporal and epistemic log-ics at the probabilistic level, by adding the degree of epistemicproperties.

The contributions of this paper are twofold. First, we defineprobabilistic–epistemic logic PCTLK. PCTLK not only allows proba-bilities of paths (i.e. runs), but also represents quantified anduncertain knowledge. Discrete-Time Markov Chains (DTMCs) inte-grated into interpreted systems are used to model multi-agent sys-tems. DTMCs are widely used to model systems with probabilityinformation and are formal models of PCTL [2], the probabilisticextension of computation tree logic CTL. The second contributionis the reduction of PCTLK model checking to PCTL model checking.

This reduction is achieved by transforming the models of PCTLKinto MDPs, which are then transformed to DTMCs using the notionof scheduler [32]. We show that a PCTLK formula is satisfied in amodel of PCTLK iff a corresponding PCTL formula is satisfied in aDTMC model of PCTL. By doing so, formulae of PCTLK can be simplychecked using PRISM [31], the model checker of PCTL.

This paper is organized as follows. Section 2 discusses and com-pares relevant related work on modeling and specifying knowledgeand probability. In Section 3, we present the models and introduceprobabilistic interpreted systems. We define a new logic PCTLK inSection 4 and state its syntax and semantics. In Section 5, we ex-plain how model checking PCTLK can be reduced to model check-ing PCTL. We implement our approach with PRISM [31] and applyit to a case study in Section 6. In the same section, we experimen-tally compare our work with other related approaches and showthat our approach outperforms the others in terms of both execu-tion time and space. Finally, we summarize the paper and suggestfurther work in Section 7.

2. Related work

As an automatic verification technique at design time, modelchecking has been used to verify different desirable properties,such as deadlock freedom, safety, and reachability. Recently, thistechnique has been used to verify, in a static way, if compositeWeb services design models satisfy such properties, which allowsus to check the soundness and completeness of the models [6]. Un-like our work that aims at proposing a new probabilistic–epistemiclogic and a new model checking technique for the underlying mod-els, the authors in [6] simply expressed the desired properties inthe existing CTL and LTL logics with no knowledge operator, andused the classic (non-probabilistic) symbolic model checking tech-nique to perform the verification.

Model checking epistemic logic from agent programming per-spective has been investigated by Dennis et al. in [11]. The authorsproposed a framework for verifying agent-based solutions. Thereare two components in their framework: the agent infrastructurelayer, which is a set of Java classes designed to interpret belief, de-sire, and intension agent programming languages, and the agentJava pathfinder, which is an extended Java Pathfinder (JPF) modelchecker for agent programs. This framework emphasizes the veri-fication of agents’ beliefs, plans, and goals. However, uncertaintyhas not been considered in this work.

Dealing with uncertainty within distributed knowledge baseshas been recently addressed by some researchers. Lawry and Tangin [33] use valuation pairs, which represent absolutely true and notabsolutely false as a model of truth-gaps for propositional logicsentences. Instead of two proposition values of either absolutelytrue or absolutely false, valuation pairs set three-value proposi-tions: true, borderline, and false. A sentence having a value, whichis neither absolutely true nor absolutely false is borderline. This al-lows agents to consider uncertain and vague propositions. How-ever, this logic is limited as it cannot express probability valuesover propositions as we propose in our logic. Moreover, practicalmodel checking of this three-value logic is a complex procedurebecause the borderline value cannot be mapped to true so thatthe corresponding model states can be returned by the modelchecking algorithm. Such an algorithm is yet to be proposed. A re-lated work has been investigated by Khan and Banerjee in [29] byproposing a logic for multiple-source approximation systemswhere the agent knowledge base is distributed. The authors usedthe theory of rough sets to define an approximation space, in whicha domain of discourse and an equivalence relation on this domainare paired. Based on this approximation space, the lower and upperapproximation can be computed. To express the properties related

W. Wan et al. / Knowledge-Based Systems 50 (2013) 279–295 281

with rough set theory in the multiple-source situation, quantifiedfirst and second order modal logics are introduced. The formal lan-guage is characterized by (i) a non-empty countable set Var of vari-ables; (ii) a (possibly empty) countable set Con of constants; (iii) anon-empty countable set PV of propositional variables; and (iv) thepropositional constants > and \. The set T of terms is given byVar [ Con. Using the standard Boolean logical connectives : (nega-tion) and ^ (conjunction), a unary modal connective hti (possibil-ity) for each term t 2 T, the universal quantifier ", well-formedformulae are defined recursively as follows:

>j ? jpj:uju ^ujhtiuj8xu

where p 2 PV, x 2 Var, and u a well-formed formula. The uncer-tainty is considered under the form of lack of incomplete informa-tion. However, these logics do not support explicit probabilityvalues to express stochastic scenarios and properties as our logicdoes. In addition, unlike our work that focuses on the practical ver-ification through model checking, this work explores insteadexpressibility, axiomatization and decidability issues. Model check-ing of this logic has not been developed yet.

Deceit and indefeasible knowledge can be seen as another kindof uncertain knowledge. Uckelman [44] put forward DeceitfulAnnouncement Logic (DAL) to model this kind of knowledge. DALcan be considered as a fragment of Dynamic Epistemic Logic(DEL) [48], which combines epistemic operators with programmes(specifically, test programmes for formulas) in analogy to PublicAnnouncement Logic (PAL). Considering a set H of propositionalletters and a set A of agents, the set of well-formed formulas ofDAL is defined as follows:

u ::¼ p 2 Hj:uju ^ujKau : a 2 Aj½u?�w: u does not contain ½a?� for any a

Unlike our logic that expresses uncertain knowledge explicitlythrough probabilities, this logic models uncertainty only implicitlythrough a kind of model change and revision. In fact, instead ofexpressing regular incomplete knowledge, this approach focuseson allowing an agent to hide some knowledge. Consequently, thislogic cannot express the scenarios described in the introduction.Furthermore, because of the implicit modeling of uncertain knowl-edge and the model change procedure, classic model checking DALis not possible.

In this paper, we deal with uncertainty by using a probabilisticlogic. Logics of probability [20] and knowledge [16] and modelchecking techniques of probabilistic and epistemic properties havebeen largely studied during the last decade [35,39,31,21,19,47,7].Halpern and Vardi in [19] defined a semantic model of knowledgefor model checking epistemic properties instead of applying theo-rem proving to see if a given formula follows from an agent’sknowledge base. Hansson and Jonsson in [21] introduced the prob-abilistic computation tree logic (PCTL), which concentrates on non-epistemic probabilistic properties and proposed model checkingPCTL by combining reachability-based computation and resolutionof systems of linear equations to compute probabilities associatedwith path formulae. CTLK, computation tree logic of knowledgeand its model checking have been studied by Lomuscio and Pen-czek [34,39]. CTLK combines epistemic logic and temporal logicCTL, but does not consider probabilistic behaviors. ProbabilisticAlternating-Time Temporal Logic (PATL⁄) is proposed by Huang,Su, and Zhang in [23] to account for incomplete information inmulti-player synchronous games. The semantics of the logic is de-fined using probabilistic interpreted systems and partially ob-served probabilistic concurrent game structures where playershave perfect recall memory over observations. The paper provesthat the model checking problem of PATL⁄ is in general undecid-

able, which makes the use of this logic to verify scalable concreteapplications impossible.

Delgado and Benevides in [10] modeled each individual agent inthe multi-agent system by a homogeneous DTMC with synchroni-zation actions. In this DTMC model, a state either has a synchro-nized action with probability 1 or regular probabilistictransitions. Agents collaborate using synchronization actions. Theproblem of this method is that there are two kinds of actions in-volved in one DTMC model. Therefore, this DTMC is converted intoan MDP model that composes two parallel DTMC models. Delgadoand Benevides also defined K-PCTL logic to specify the probabilisticand epistemic properties. The syntax of this logic is as follows:

/ ::¼ truejaj:/j/ ^ /jPrelp½w�jKi/jCG/jEG/

w ::¼ /j/U6k/j/U/

where a is an atomic proposition, rel 2 {6, <, P, >} p 2 [0, 1], i anagent and k 2 N. As shown in the syntax, the limitation of K-PCTLis that only probabilities over path formulae can be expressed.The reason is because the knowledge operators cannot be precededby the probability operator. Our PCTLK logic overcomes this limita-tion by allowing probabilities over knowledge operators to be partof the syntax. Thus, our logic is more expressive than K-PCTL. Infact, PCTLK logic not only expresses probabilities on path formulae,but also includes probabilities of knowledge. Consequently, fromthe semantics perspective, K-PCTL uses MDP models augmentedwith the accessibility relation, so that probabilities over paths canbe defined and classic knowledge formula can de captured. How-ever, probabilities of knowing cannot be captured because accessi-bility transitions are not probabilistic. Our approach addresses thislimitation by defining probabilistic interpreted systems that arecombined with MDP. Moreover, unlike our proposal that uses a fullyimplemented reduction-based model checking technique, the mod-el checking approach proposed in this work only consists in a the-oretical extension of PRISM with no performance evaluation. Infact, our approach avoids such an extension, which can increasethe problem complexity.

Cao in [7] proposed PETL, a probabilistic epistemic temporal lo-gic. Like our PCTLK logic, PETL is a combination of temporal logicand probabilistic knowledge logic that supports probabilistic com-mon knowledge. Knowledge modalities such as ‘‘everyone knowswith probability’’ and ‘‘probabilistic common knowledge’’ can beexpressed in PETL. However, this logic has limitations that our lo-gic aims to overcome. We discuss these limitations from the syn-tax, semantics, and model checking perspectives. The syntax ofPETL is defined as follows:

u ::¼ qj:uju _u�uj j½�ujuUujKpau Ep

Cu�� Cp

Cu

where s means next, [] means always, U means until, K meansknowledge, E means everyone knows, and C means common knowl-edge. Intuitively, Kp

au means that agent a knows the probability ofu is greater or equal than p. Ep

C means that every agent in C knowsthe probability of u is greater or equal than p. Cp

C means that ‘‘theprobability of u is greater or equal than p’’ is a common knowledgeby every agent in C. As shown in the syntax, the main limitation ofPETL is that probabilities come only over knowledge operatorsKp

au; EpCu, and Cp

Cu. Consequently, properties such as ‘‘the probabil-ity that in future u holds’’ and ‘‘the probability that an agent knowsu’’ cannot be expressed in PETL. In our PCTLK logic, probabilitiescan be expressed over the whole formula, which allows us to ex-press those properties which are about the probability of paths(probability over the next, until, and global operators) and the prob-ability of knowing, on top of properties about knowing the probabil-ity of formulae that PETL can express. Furthermore, PETL is in factbased on LTL, while our logic is an extension of CTL, and LTL andCTL are incomparable in terms of expressiveness, which means no


one can be a subset of the other. To define the semantics of PETL,Cao uses a probabilistic epistemic temporal model M = (Q, T, �1,. . ., �n, P1, . . ., Pn, V), where Q is the set of the global states forthe system; T is a total binary (successor) relation on Q; �a is anepistemic accessibility relation for each agent a defined by s�as0

iff la(s) = la(s0), where the function la returns the local state of agenta from a global state s; Pa is a probability function defined asfollows: Pa: Q � 2Q ? [0, 1], such that for every agent a,Pa(s, s0jla(s) = la(s0)) = 1; and V is a valuation function that assignsto each state a set of propositional variables that are assumed tobe true at that state. The semantics of probabilistic knowledge isthen defined using the probability function Pa that associates toeach accessibility transition a probability value. The main problemwith hat definition is the computation of those probabilities overaccessibility transitions, which are not part of the system beingchecked, but rather part of the agent’s accessibility relation. In fact,it is not known how an agent can define those probabilities. If theprobabilities of the system being checked can be computed byobserving the system’s behavior, observing an agent’s behavior can-not reveal those probabilities. To solve this issue, in our semanticswe assume that all the accessible states from a given state areequally accessible. Thus, accessibility transitions are not probabilis-tic. However, probabilistic knowledge is naturally defined by com-puting the number of accessible states that satisfy the knowledgeover the total number of accessible states. On the other hand, inPETL, the system being checked is not assumed to be probabilisticas only probabilistic interpreted systems are used. In our frame-work, even the system is considered probabilistic, which we captureby integrating probabilistic interpreted systems into DTMC.

In the same paper [7], Cao extends PETL to lPETL (PETL# lPETL), which incorporates both CTL and LTL. The syntax oflPETL is as follows:

u ::¼ qjXj:uju _uj �u Kpau

�� lX:uðXÞ

where X is an atomic proposition variable and u(X) is any formulasyntactically monotone in the propositional variable X, i.e., X occurspositively in u(X) in the sense that all free occurrences of X fall un-der an even number of negations. To define the semantics of lPETL,the same model as the one used for PETL is utilized. l calculus mod-el checking algorithm is then extended to lPETL. Although lPETL ismore expressive than PETL as it can express both LTL and CTL prop-erties, its syntax shows that it has the same problem as PETL as itcannot express probabilities aver paths and the probability thatan agent knows u. In fact, lPETL only allows one to express thatan agent knows the probability of a fact, but it does not enable usto express the probability that an agent knows a fact, while ourPCTLK logic supports the expression of both properties. From thesemantics perspective, lPETL mainly focuses on probabilisticknowledge through the probability function Pa defined for eachagent, which, as in PETL, assigns each state a probability distribu-tion over the accessible states. This function is only used to accountfor the probability on the epistemic domain, not the system domain.In our logic, both domains are considered through the combinationof two models DTMC (for the system domain) and probabilisticinterpreted systems (for the epistemic domain). Thus, as for PETL,the system being checked with lPETL is not probabilistic. Finally,from the model checking perspective, only two algorithms havebeen proposed in [7], one for PETL and one for lPETL with no imple-mentation. In our paper, the model checking technique is fullyimplemented and tested. Moreover, a careful analysis of thosetwo algorithms reveal their similarity with the one implementedin the MCK model checker [17,22]. In Section 6.4, we will show thatour model checking approach outperforms MCK in terms of bothexecution time and space.

In [3,4], Belardineli and Lomuscio defined ’’dynamic’’ and ‘‘sta-tic’’ quantified interpreted systems, which enable the use of quan-tifiers on epistemic models within a first-order temporal epistemiclogic. However, in these proposals, the stochastic behavior ofagents and probabilistic properties are not considered. In [24], Jam-roga proposed Markov temporal logic MTLx. Two frames are stud-ied: MTL0 for Markov chains and MTL1 for MDPs. MTLx (x 2 {0, 1})allows agents to perform flexible reasoning about their outcomesin stochastic environments. Multi-agent systems are then modeledusing Markov Chains and MDPs and unlike our proposal, Jamrogauses utility fluent instead of probabilistic transition functions topresent the truth values for DTMCs and MDPs. Another differencewith respect to our proposal, is that this work only focuses onquantitative properties not the epistemic ones. Furthermore, mod-el checking MTLx has not been considered.

Our work comes close to Huang et al. [22], where interpretedpartially observed discrete-time Markov chain (PO-DTMC) isproposed. Unlike our interpreted discrete-time Markov chain,PO-DTMC is based on partial observations with assumption on syn-chronous with perfect recall. In [22], the set of states is defined asstates of a special agent, called the environment, while the remain-ing agents observe the environment and perform actions based ontheir observations. Probabilistic knowledge is expressed by a ra-tional linear combination of every agents’ probabilities in the sys-tem: every agent has its own probability for each accessible state,which is supposed to be known. The main problem is that theprobabilities associated to accessible (i.e., equivalent) states arenot part of the system, but part of the agents’ programs and it isnot clear how an agent can get those probabilities. Practically, anagent can see which state is equivalent to or indistinguishable fromthe current state, but it is hard to see how much the state is indis-tinguishable. However, in our approach probabilistic knowledge iscomputed in a simpler and practical way as a direct and naturalextension of non-probabilistic knowledge where the probabilityonly depends on the number of equivalent states, as all the statesare equally accessible. Technically speaking, in our approach thereis no need to solve any linear equation, which makes the proceduresimpler. Furthermore, thanks to this simple extension, our modelchecking technique is implemented through a transformation ofan existing technique without adding any new computation costcaused by a new algorithm, which is the case in Huang et al.’s pro-posal. Finally, we will show in Section 6.4 that our approach out-performs Huang et al.’s one in terms of execution time andscalability in terms of state space.

MCK [17] and MCMAS [37] are designed for verifying epistemicproperties of multi-agent systems. MCK supports several differentways of defining agents’ knowledge in the multi-agent system:either using observation alone or observation and clock with eithersynchronous or asynchronous perfect recall of all the observations.Both linear and branching time temporal logics are supported inMCK. In MCMAS, multi-agent systems are described by the Inter-preted Systems Programming Language (ISPL). ISPL can also beused to define atomic propositions, action formulae and the spec-ification of properties to be checked. MCMAS supports CTLK andbranching temporal logic. There are a few model checking toolsthat are designed for the verification of quantitative specifications.PRISM is a probabilistic symbolic model checker that can checkPCTL, CSL, LTL and PCTL⁄ formulae, as well as extensions for quan-titative specifications and costs/rewards formulae. PRISM supportsseveral types of probabilistic models, such as discrete-time Markovchains (DTMCs), continuous-time Markov chains (CTMCs), Markovdecision processes (MDPs), and probabilistic automata (PAs).Huang and colleagues in [22] extended MCK by a new symbolic-based algorithm and replacing the nondeterministic if constructin the MCK programming language with a weighted if constructso that MCK is able to verify probabilistic knowledge. In this paper,


we introduce a reduction technique to convert probabilistic–epi-stemic properties into quantitative properties so that our extendedversion of PRISM with this reduction can be used. We will comparethe efficiency of our method with extended MCK and MCMAS andshow that our extended PRISM outperforms both extended MCKand MCMAS.

3. Models description

3.1. Discrete-time Markov chains

Markov chains are probabilistic finite automata that are used tomodel transitions in probabilistic systems. Discrete-Time Markovchain (DTMC) is a type of Markov chain in which a system is in agiven state at each ‘‘step’’, with the state changes randomly be-tween steps. Thus, the next state in a DTMC is chosen probabilisti-cally. In [2], DTMC is defined as follows.

Definition 1. DTMCOver a set of atomic propositions AP, DTMC is atuple (S,P, Iinit,L,AP), where:

� S is a non-empty and finite set of states.� P: S � S ? [0, 1] is the transition probability function (or

matrix), such that for every state s 2 S, we haveP

s02SPðs; s0Þ ¼ 1.� Iinit: S ? [0,1] is the initial distribution such that for all states

s 2 S,P

s2SIinitðsÞ ¼ 1.� L: S ? 2AP is a state labeling function.

The initial distribution Iinit can be viewed as a column vector ofjSj rows where jSj is the cardinality of S. The value of every row rep-resents the probability that the corresponding state is an initialstate. The transition probability function P: S � S ? [0,1] is repre-sented by the matrix (P(s, t))s, t2S. The probabilities of moving fromstate s to its successors are shown in the row P(s, �) of the matrix,while the probabilities of entering state s from other states areshown in the column P(�,s) of the matrix.

Fig. 1 is an example of a DTMC model. S = {s0,s1,s2,s3} is the setof states. AP = {r, f,s}. L(s0) = £, L(s1) = {r}, L(s2) = {s} and L(s3) = {f}.The transition probability function P viewed as a 4 � 4 matrixand the initial distribution Iinit are as follows:

P ¼

0 1 0 00 0 0:9 0:10 0 1 01 0 0 0

26664

37775 Iinit ¼

1000

26664

37775

3.2. Probabilistic interpreted systems

Let A = {1, . . . , n} be a set of n agents in the system. Every agenti 2 A is associated with its local state set Li, and possible actions setActi. A set of global states C # L1 � � � � � Ln is the set of all possibletuples (l1, . . . , ln), and each tuple represents a computational statefor the whole system. If we assume that all actions have evenchance, we can map actions to the probabilistic transition functionT for the system. T is defined as T: C � Act � C ? [0,1], where

Fig. 1. An example of Discrete-Time Markov Chain (DTMC).

Act # Act1 � � � � � Actn is the set of actions that are executed byagents in the system for collaboration, such that for every globalstate c 2 C;

Pc02CTðc;acc0 ; c0Þ ¼ 1, where acc0 2 Act is the action

labeling the transition from c to c0. Each agent is associated witha local probabilistic transition function Ti: Li � Acti � Li ? [0,1],such that for every local state li 2 Li;

Pl0i2Li

Ti li;ali l0i ; l0i

� �¼ 1 for

i 2 A, where ali l0i 2 Acti is the agent i’s action labeling the transition

from li to l0i. For li 2 c; l0i 2 c0, the probabilistic transition function Tfor the system can be calculated by Eq. (3.1) as follows:

Tðc;acc0 ; c0Þ ¼ gYi2A^li2c^l0i2c0

Ti li;ali l0i ; l0i

� �ð3:1Þ

where g is a normalizing factor that forces transitions fit for proba-bility distribution

Pc02Ctðc;acc0 ; c0Þ ¼ 1 for every global state c. A

global initial distribution Iinit expresses how the system starts andsatisfies

Pc2CIinitðcÞ ¼ 1. Definition 2 defines the formal models of

PCTLK MPIS.

Definition 2 (Models MPIS of PCTLK). Over a set of atomic propo-sitions AP, a model MPIS is a tuple: MPIS = (W, Pt, Iinit, �1, . . . , �n, V)where:

� W # C is the set of reachable states. A state w is reachable ifand only if there exists a sequence of transitions from an initialstate to w such that all of the transitions have probabilitygreater than 0.� Iinit: W ? [0,1] is the initial distribution of the model, such that:P

w2W IinitðwÞ ¼ 1.� Pt: W �W ? [0,1] is the transition probability function defined

by Pt(w,w0) = p(p 2 [0,1]) if and only if there exists a collabora-tion action (a1, . . . , an) 2 Act such that

Pi2ATiðw; ai;w0Þ > 0 and

the value of Pt is equal to tðw;aww0 ;wÞ in probabilistic inter-preted systems. For all w 2W we have:

Pw02W Ptðw;w0Þ ¼ 1.

� �i # W �W is the epistemic accessibility relation for the agenti, such that for two global states (l1, . . . , ln) and l01; . . . ; l0n

� �, we

have: ðl1; . . . ; lnÞ�i l01; . . . ; l0n� �

iff li ¼ l0i.� V is a global state labeling function V: W ? 2AP.

The initial distribution Iinit can be viewed as a column vector ofjSj rows where jSj is the cardinality of S(Iinit(s))s2S, in which the va-lue of every row represents the probability that the correspondingstate is an initial state.

The transition probability function Pt: W �W ? [0,1] can berepresented by the matrix (Pt(s, t))s, t2W. The probabilities of mov-ing from state s to its successors are shown on the rows Pt(s, �) ofthe matrix, while the probabilities of entering state s from otherstates are shown on the columns Pt(�,s) of the matrix.

Let us consider the example illustrated in Fig. 2 showing an MPIS

model where two agents are included. W = {s0,s1,s2,s3,s4} is the set ofreachable states. AP = {p,q}. The labeling function V is: V(s0) = {q},V(s1) = £, V(s2) = {p,q}, V(s3) = {q}, and V(s4) = {p}. For the epistemicaccessibility relations, we have: {(s0,s0), (s0,s1), (s0,s2), (s1,s0), (s2,s0),(s1,s1), (s2,s2), (s3,s3), (s4,s4)} #�1 and

fðs0;s0Þ;ðs0;s2Þ;ðs0;s3Þ;ðs2;s0Þ;ðs3;s0Þ;ðs1;s1Þ;ðs2;s2Þ;ðs3;s3Þ;ðs4;s4Þg#�2

The initial distribution Iinit and the transition probability function Pviewed as a 5 � 5 matrix are as follows:

Iinit ¼

10000

26666664

37777775

P ¼

0:4 0:3 0:3 0 00 1 0 0 00 0 0:4 0 0:60 1 0 0 00 0 0 0:5 0:5

26666664

37777775

Fig. 2. Model MPIS.


4. Epistemic–probabilistic logic

Specifications for interpreted DTMC models MPIS can be ex-pressed in PCTLK (Probabilistic Computation Tree Logic of Knowl-edge), which combines CTL logic, epistemic logic [16], andprobabilistic logic [2,21]. PCTLK can be used to reason aboutprobabilistic knowledge and specify properties of probabilistic–epistemic multi-agent systems.

A PCTLK formula is capable of formulating conditions on a stateof an epistemic Markov chain. Besides the standard propositionallogic operators, PCTLK also includes the probabilistic operator Pr.Unlike standard CTL, path quantifiers $ and " are not valid inPCTLK formulae, all path formulae are immediately preceded bythe probabilistic operator Pr.

4.1. Syntax of PCTLK

PCTLK is comprised of three types of formulae: state formulae /,path formulae w, and epistemic formulae j. State and path formu-lae of CTL are state and path formulae of PCTLK. Epistemic formu-lae are expressed using knowledge and group knowledgeoperators. The syntax of PCTLK is defined as follows:

Definition 3 (Syntax of PCTLK). Let p, p1, p2, . . . range over the setof atomic propositions Up. Let A = {1, . . . , n} be a set of agents andG # A be a group of agents, the PCTLK formulae are defined by thefollowing BNF grammar:

/ ::¼ truejpj/ ^ /j:/jjjPrObðwÞjPrObðjÞw ::¼ �/j/U6n/j/U/

j ::¼ Ki/jEG/jCG/jDG/

where 0 6 b 6 1 is a real number giving the rational boundary,O 2 {<,6,P,>} presents relationship boundary of the probability,and n 2 N is the maximum steps to achieve a specific state.

Formulae j, called epistemic formulae, are special state formu-lae in PCTLK that can describe epistemic properties. There are fourepistemic modalities: Ki, EG, CG, and DG that represent respectively‘‘agent i knows’’, ‘‘every agent in the group G knows’’, ‘‘commonknowledge in the group G’’, and ‘‘distributed knowledge in thegroup G’’. PrOb(Ki/) represents the probability that agent i knows/, where / is a state formula. This probability is Ob. KiPrObw statesthat agent i knows the probability that the path formula w holds,which is Ob. Ob indicates a lower or upper bound on the probabil-ity (e.g. <b, 6b, Pb, or >b). The difference between these two kindsof formulae is that PrOb(Ki/) expresses the degree of agent i know-ing something, while KiPrObw indicates that agent i knows someuncertain things. To illustrate, PrP0.9(K1/) indicates that agent 1knows / with at least 0.9 probability. K1(PrP0.9/) means agent 1

knows that with at least 0.9 probability, / holds. }/ and h/ arethe usual abbreviations for eventually and globally:}/ � true U / and �/ � :}:/.

There are no universal (") and existential ($) path quantifiers inPCTLK. Instead, the linear temporal operators s (next), U (until),and U6n (bounded until) are required to follow the probabilisticoperator PrOb immediately. The propositional temporal fragmentof PCTLK has the same meaning as in CTL. For example, the formulas/ has the meaning of ‘‘in the next state / holds’’. /1U/2 means‘‘/1 holds until /2’’. A new step-bounded variant of until(/1U6n/2) is added, meaning that ‘‘/2 will hold within at most nsteps while /1 holds in all states before a /2-state has beenreached’’. The step-bounded until is necessary in probabilistic logicbecause the probability of reaching a /2-state after at most n stepsis different from reaching this state after at most (n + 1) steps. InCTL, temporal operators s and U are required to be immediatelypreceded by a path quantifier, while in PCTLK, they must followthe operator PrOb immediately.

The probabilistic operator on path formulae PrOb (w) expressesthat ‘‘w holds with a probability Ob’’. For instance, PrP0.75(smes-sage_receive) asserts that ‘‘with at least 0.75 probability, in the nextstate the message will be received’’. The probabilistic operator onepistemic formulae PrOb(j) states the degree of the knowledge:how much the agent is confident about his knowledge. For exam-ple, the following formula: Pr60.8(K1(agent_2_has_resource_A)) ex-presses that agent 1 knows with a maximum probability of 0.8that agent 2 has resource A. More specifically, this means thatout of X accessible states for agent 1, Y states satisfy the fact thatagent 2 has resource A, where Y/X 6 0.8.

4.2. PCTLK semantics for DTMC

Before we give the formal semantics of PCTLK, we briefly reviewthe notion of probability space [13] and then define group episte-mic relations. A probability space is a mathematical constructor inprobability theory. It is expressed as a triple ðX; E;PÞ that models aprocess consisting of events that occur randomly.

� X is a sample space. To fit our case, we assume that X is a set ofstates.� E is a set of events. E is a subset of X. If E# X, thenE ¼ X E# X;� P : E ! ½0;1� is a function, also called a probability measure that

assigns probabilities to events. The measure of the whole sam-ple space PðXÞ ¼ 1. If E1 # X; E2 # X, and E1 \ E2 ¼ ;, thenPðE1 [ E2Þ ¼ PðE1Þ þ PðE2Þ.

The probabilistic operator PrOb can be placed either in front of apath formula w or in front of an epistemic formula j. Formulae areevaluated on states or along paths where a path is an infinite


sequence of states. For the formula PrOb(w), s�PrOb(w) means that‘‘the probability from state s that w holds for all outgoing pathsis Ob’’. For example, s�Pr<0.25(true [ 65/) asserts that ‘‘the probabil-ity that the system model satisfies / within 5 steps of all outgoingpaths from the state s is less than 0.25’’. We use the symbol Prob tostand for the probability measure. In order to compute the proba-bility measure of a path, we need to associate a probability spacewith probabilities in the model MPIS. Let p̂ ¼ s0 . . . sm be a finitefragment of the path p, the cylinder set Cylðp̂Þ [2] is the set of allinfinite reachable paths emanating from p̂ in MPIS. The probabilitymeasure of this cylinder set Cylðp̂Þ can be calculated by:

ProbðCylðp̂ÞÞ ¼ ProbðCylðs0 . . . smÞÞ ¼ Iinitðs0Þ � Ptðs0 . . . smÞ ð4:1Þ

where

Ptðs0 . . . smÞ ¼Y

06i<m

Ptðsi; siþ1Þ ð4:2Þ

For model checking purposes, Eq. (4.1) will be used when weare interested in determining if the model satisfies a given formula.However, when the question is about determining if a given state ssatisfies a given formula, we assume that s is the unique initialstate, so that we use Is instead of Iinit to compute ProbðCylðp̂ÞÞ.The Is value is defined as follows:

IsðtÞ ¼1; if s ¼ t

0; otherwise

�ð4:3Þ

Let G # A be a group of agents. To define the semantics of the epi-stemic operators EG, CG, and DG, we define the group epistemicaccessibility relations from the accessibility relation �i as follows:

Definition 4 (Group Epistemic Accessibility Relations).

� �EG is the union of group G0s accessibility relations: �E

G ¼S

i2G�i.� �C

G is the transitive closure of �EG.

� �DG is the intersection of G0s accessibility relations: �D

G ¼T

i2G�i.

For the state formula PrOb(j),s�PrOb(j) means that ‘‘on state s,the probability that the epistemic formula j holds is Ob’’. Wedenote the number of states s0 such that for a given state s we haves�is0 for agent i (i 2 A) by js�is0j. The sample space of agent i at state sis the set of possible worlds or equivalent states of i at s and is equalto js�is0j. Similarly, we denote the number of states s0 that are acces-sible from a given state s through �E

G by s�EGs0

�� , through �CG by

s�CGs0

�� , and through �DG by s�D

Gs0�� . We also define js�/j as follows:

js /j ¼1; if s /

0; otherwise

�ð4:4Þ

Let s 2W be a state, p = s0, s1, s2, . . . a path, i.e. an infinite se-quence of states related by transitions, a 2 AP an atomic proposi-tion, / a PCTLK state formula (i.e. evaluated over states), and w aPCTLK path formula (i.e. evaluated through paths). The (i + 1)thstate in p is denoted by p(i) (i.e., p(i) = si). r(s) is the set of all pathsemanating from s. Given the model MPIS = (W, Pt, Iinit, �1, . . . , �n, V),(MPIS,s)�/ stands for ‘‘state s satisfies / in the system model MPIS’’or ‘‘/ is true at state s in the system model MPIS’’. (MPIS,p) �w is readas ‘‘the path p satisfies w in the system model MPIS’’ or ‘‘w is truethrough the path p in the system model MPIS’’. If MPIS is clear fromthe context, we simply write s�/ and p�w. In the following, we de-fine the semantics of PCTLK.

� For a state s:

s a iff a 2 VðsÞs /1 ^ /2 iff s /1 and s /2

s :/ iff s2/

The semantics of the state formulae PrOb(w) and PrOb(j) will be gi-ven later.� For a path p:

p�/ iff pð1Þ/

p/1U6n/2 iff 906 k6n; pðkÞ/2 and 806 i< k pðiÞ/1

p/1U/2 iff 9k P 0; pðkÞ/2 and 806 i< k pðiÞ/1

� The semantics of epistemic formulae j over a state s is based onthe epistemic accessibility relation and group epistemic accessi-bility relations as given in Definition 4:

s Ki/ iff 8s0 2W if s�is0 then s0 /

s EG/ iff 8s0 2W if s�EGs0 then s0 /

s CG/ iff 8s0 2W if s�CGs0 then s0 /

s DG/ iff 8s0 2W if s�DGs0 then s0 /

� In terms of probability space, the set of all reachable states fromthe initial states is our sample space X and for each formula, theset of states satisfying it is the set of events E. Based on thisobservation, we define the semantics of the probabilistic opera-tor Pr that works on path and epistemic formulae in thefollowing.– For a probabilistic operator working on a path formula,

where p̂ ¼ s0 . . . sm:

s PrObð�/Þ iff Probðs;rðsÞ;�/ÞOb; where :

Probðs;rðsÞ;�/Þ¼X

p2rðsÞs:t:pð1Þ/

Ptðs;pð1ÞÞ

s PrObð/1U6n/2Þ iff Probðs;rðsÞ;/1U6n/2ÞOb; where :

ð4:5Þ

Probðs;rðsÞ;/1U6n/2Þ¼

1; if s/2X8p2rðsÞs:t:p/1U6n/2

ProbðCylðp̂ÞÞ; ifsm /2;0<m6n;

and 806 i<m;si /1

0; otherwise

8>>>>>><>>>>>>:

s PrObð/1U/2Þ iff Probðs;rðsÞ;/1U/2ÞOb; where :

ð4:6Þ

Probðs;rðsÞ;/1U/2Þ¼

1; if s/2X8p2rðsÞs:t:p/1U/2

ProbðCylðp̂ÞÞ; ifsm /2 and806 i<m;si /1

0; otherwise

8>>>>>>><>>>>>>>:

ð4:7Þ

– For a probabilistic operator working on an epistemic formula:

s PrObðKi/Þ iff Probðs Ki/ÞOb; where :

Probðs Ki/Þ ¼P

s�is0js0 /jjs�is0j

ð4:8Þ

s PrObðEG/Þ iff Probðs EG/ÞOb; where :

Probðs EG/Þ ¼P

s�EG

s0 js0 /jjs�E

Gs0j ð4:9Þ

s PrObðCG/Þ iff Probðs CG/ÞOb; where :

Probðs CG/Þ ¼P

s�CG

s0 js0 /js�C

Gs0�� ð4:10Þ

s PrObðDG/Þ iff Probðs Dj/ÞOb; where :

Probðs DG/Þ ¼P

s�DG

s0 js0 /js�D

Gs0�� ð4:11Þ

Fig. 4. Another example of MPIS model.


Example 1. In this example, we will illustrate how to check if astate satisfies a probabilistic formula using Fig. 3 that shows a sim-ple MPIS model. The solid lines with numbers are probabilistic tran-sitions. The dash lines are epistemic accessibility relations �1 foragent 1. To keep the example simple, only �1 from s0 are shownin the figure. Pt and Iinit are as follows:

Pt ¼1=2 1=2 0

0 1=4 3=40 0 1

264

375; Iinit ¼

100

264

375

We want to check if state s0 satisfies the formula Pr>0.9(}p). To doso, we have to compute the probability of all paths from s0 that sat-isfy }p � true U p, and then check if the summation of these proba-bilities is greater than 0.9 (see Eqs. (4.1), (4.2) and (4.7)). We have:

Probðs0;rðs0Þ; true U pÞ ¼ Ptðs0; s1Þ þ Ptðs0; s0Þ � Ptðs0; s1Þþ Ptðs0; s0Þ � Ptðs0; s0Þ � Ptðs0; s1Þ þ � � �

¼X/n¼1

ð1=2Þn ¼ 1 > 0:9

Consequently, state s0 satisfies this formula.Let us now check, for example, if s0 satisfies Pr>0.7(K1q). We use

Eq. (4.8) to compute the probability Prob(s0�K1q). Since thenumber of accessible states from s0 using �1 is 3, out of which 2satisfy q, we obtain:

Probðs0 K1qÞ ¼ 2=3 < 0:7

Thus, state s0 does not satisfy the formula.

Example 2. Let us consider another example mainly for probabilis-tic epistemic operators using Fig. 4 that illustrates another MPIS

model where two agents are considered. For the sake of simplicity,only the accessibility relations from state s0 are shown in the figureand one from s2 needed to show�C

G. Furthermore, the ones that canbe deduced from the properties of the accessibility relation (forinstance transitivity and Euclideanity) are omitted. We also omitthe probabilities of transitions. Let G be the group of agents 1 and2. So, for the union of �1 and �2 we have: fðs0; s0Þ; ðs0; s1Þ;ðs0; s2Þ; ðs0; s3Þg#�E

G; for the transitive closure of �EG, we have:

fðs0; s0Þ; ðs0; s1Þ; ðs0; s2Þ; ðs0; s3Þ; ðs0; s4Þg#�CG; and for the intersection

of �1 and �2 we have: fðs0; s0Þ; ðs0; s2Þg#�DG .

Now, we use semantics equations to calculate the probability ofepistemic operators.

1. Probðs0 K2qÞ ¼ js0qjþjs2qjþjs3qj3 ¼ 1þ1þ0

3 ¼ 23

2. Probðs0 K1qÞ ¼ js0qjþjs1qjþjs2qj3 ¼ 1þ0þ1

3 ¼ 23

3. Probðs0 EGqÞ ¼ js0qjþjs1qjþjs2qjþjs3qj4 ¼ 1þ0þ1þ0

4 ¼ 24 ¼ 1

24. Probðs0 DGqÞ ¼ js0qjþjs2qj

2 ¼ 1þ12 ¼ 1

5. Probðs0 CGqÞ ¼ js0qjþjs1qjþjs2qjþjs3qjþjs4qj5 ¼ 1þ0þ1þ0þ1

5 ¼ 35

Fig. 3. An example of MPIS model.

Thus, for instance, we have s0 satisfies the formula PrP0.6(CGq),but does not satisfy PrP0.7(CGq).

4.3. Some properties of probabilistic knowledge

PCTLK inherits epistemic properties from CTLK [35] and proba-bilistic properties of path formulae from PCTL [2]. We list a numberof new properties that are not included in conventional epistemicand probabilistic logics. For the conventional ones, please refer to[2,16].

There are a number of equivalences between a probabilisticepistemic formula and the conventional knowledge formula. Ifthe probability that an agent knows / is greater than or equalto 1 holds at state s, then the agent knows / holds at the states. We can expand this validity to everyone’s knowledge EG/,common knowledge CG/, and distributed knowledge DG/.

Theorem 1 (Probabilistic and Epistemic Equivalence).

s PrP1ðKi/Þ iff s Ki/ s Pr<1ðKi/Þ iff s :Ki/

s PrP1ðEG/Þ iff s EG/ s Pr<1ðEGð/ÞÞ iff s :EG/

s PrP1ðCG/Þ iff s CG/ s Pr<1ðCG/Þ iff s :CG/

s PrP1ðDG/Þ iff s DG/ s Pr<1ðDG/Þ iff s :DG/

s Pr60ðKi/Þ iff s Ki:/ s Pr60ðEG/Þ iff s EG:/

s Pr60ðCG/Þ iff s CG:/ s Pr60ðDG/Þ iff s DG:/

Proof. We prove the first equivalence; the same method can beused to prove the others.

We first prove ).We have: s�PrP1(Ki(/)). According to the semantics, we get:

Probðs Ki/Þ ¼P

s�is0js0 /jjs�is0j

¼ 1:

Therefore,P

s�i s0js0 /j ¼ js�is0j.

This means "s0 such that s�is0, s0�/. Thus, s�Ki/.Next we prove �.s�Ki/ iff "s0 such that s�i s0, we have s0�/.Consequently,

Ps�is0js0 /j¼ js�is0j Therefore, s�PrP1(Ki(/)) h

Theorem 2 (Probabilistic and Non-Probabilistic Knowledge). Let b1

and b2 be two boundaries in [0,1]. The following validity holds:

Pr>b1ðKi/Þ ^ Pr<b2

ðKi/Þ ) :Ki/ ^ :Ki:/


Proof. From the left side of the validity, we conclude that0 6 b1 < b2 6 1. Let s be a state such that: s Pr>b1 ðKi/Þ

^Pr<b2 ðKi/Þ. So we have: 0 < Probðs Ki/Þ ¼P

s�i s0js0/j

js�is0 j< 1. Thus,

0 <P

s�is0js0 /j < js�is0j. This means some accessible states from

s satisfy /, but not all of them, and so some others satisfy :/ butnot all of them, so the result flows from the semantics of Ki/. h

Theorem 3 (Probabilities of Subgroup).

1. If s�EG/ and G0 # G, then s EG0/. The same result holdsfor CG but not for DG.

2. If s�PrOb(EG/) and G0 � G, then it is not the case thats PrObðEG0/Þ. The same result holds for CG and DG.

Proof.

� For 1, we prove the theorem for EG, the same idea can beused for CG. Let s be a state such that s�EG/. By semantics:"s0 s.t. s�E

Gs0, we have s0�/. Because �EG ¼

Si2G�i, we obtain:

"i 2 G"s0 s.t. s�is0 we have s0�/, which also holds for anysubset G0 of G, so we are done. However, for DG as the inter-section of �i is considered, it is easy to imagine a scenariowhere a subgroup G0 outside the whole intersection so thatDG0 does not satisfy /.

� For 2, we prove the theorem for EG; CG and DG can be provedsimilarly. The proof is done by providing two examples withdifferent conclusions. Assume that (s�PrOb(EG/)) and G0 � G.Suppose there are four agents in a group: G = {Ag1,Ag2,Ag3,Ag4}and each agent has only two epistemic accessible states, which,except the state itself, are all different. Three agents Ag1, Ag2,and Ag3 at state s know /: s � K1/, s � K2/, and s � K3/. Buts =�K4/. Therefore, s�E

Gs0�� ¼ 5 and s�PrP0.8

(EG/) because Probðs EG/Þ ¼ 45 ¼ 0:8. There is a subgroup

G1 = {Ag1,Ag2,Ag3}� G where s�PrP0.8(EG1/) becauses�E

G1s0�� ¼ 4, so Probðs EG10/Þ ¼ 4

4 ¼ 1 P 0:8, while foranother subgroup G2 = {Ag1,Ag2,Ag4}� G, s =�PrP0.8(EG2/)because s�E

G2s0�� ¼ 3 and Probðs EG2/Þ ¼ 3

4¼ 0:75 < 0:8. h

Theorem 4 (Extended Properties). For all formulas / and w, and allagents i = 1, . . . , n, the following extended properties for probability hold:

1. Ki(PrObKi/)) Ki(Ki(PrObKi/))2. PrObKi/ ^ Ki(/) u)) PrObKiu3. Ki(PrObKi/)) PrObKi/

Proof

� 1 follows directly from the transitivity of �i.� For 2, if s�PrObKi/ ^ Ki(/) u), then for all states s0 such that

(s�is0) we have s0�(/) u) and

Ps�i s0js0/j

js�is0 jOb. It follows, using

modus ponens, thatP

s�is0js0 uj ¼

Ps�is0js0 /j. ConsequentlyP

s�i s0js0uj

js�is0 jOb, so the results.

� 3 follows directly from the reflexivity of �i. h

5. Model checking technique

We propose a reduction-based approach to transform the prob-lem of model checking PCTLK into the problem of model checkingPCTL so that the PCTL’s model checker, PRISM, can be used to verifyPCTLK formulae. Given a PCTLK model MPIS and a PCTLK formula uPIS,we can define a PCTL model M ¼ FðMPISÞ and a PCTL formula

u ¼ FðuPISÞ such that (MPIS, s0) �uPIS iff ðFðMPISÞ; s0Þ FðuPISÞ.Thus, the reduction is implemented in two parts: (1) transform-ing the probabilistic epistemic model; and (2) reducing PCTLKformulae.

The workflow for model checking PCTLK is summarized inFig. 5. The transformation F translates the inputs: probabilisticinterpreted system MPIS and PCTLK formula uPIS into a regularDTMC model FðMPISÞ and PCTL formula FðuPISÞ. This transforma-tion is done automatically by a tool we have implemented, thenwe use the PRISM model checker to verify if the obtained modelsatisfies the obtained formula. In the following, we split theexplanation into two parts: models transformation and formulaereduction to introduce the details of our model checkingapproach.

5.1. Translation of MPIS models

The translation from an MPIS model, which is an epistemicDTMC model, to an FðMPISÞ model is done in two steps. In thefirst step we translate the probabilistic epistemic model MPIS intoan equivalent interpreted MDP model MP

IS that will be formallydefined later (see Fig. 5). In this step, the key operation is themapping of the epistemic relations �i;�E

G;�CG;�D

G

� �used in MPIS

to specific actions ACCi;ACCEG;ACCC

G;ACCDG

� �needed for MP

IS. Then,

in the second step we transform this equivalent MDP model MPIS

into a regular DTMC model M ¼ FðMPISÞ by selecting the specificaction for the formula. These two steps will be explained in thissection.

The motivation behind the first step is that an MPIS is a DTMCmodel, which therefore cannot model the nondeterministic behav-ior of the concurrent processes in an adequate manner because itonly allows deterministic choices. However, Markov Decision Pro-cesses (MDP) that can be viewed as a variant of Markov chainsextending DTMC by allowing nondeterministic choices. Thus, bothnondeterministic and probabilistic choices coexist in MDP. Thedefinition of MDP as given in [2] is as follows.

Definition 5 (MDP). Over a set of atomic propositions AP, an MDPcan be expressed as a tuple (S,Act,P, Iinit,L,AP), where:

� S is a non-empty and finite set of states.� P:S � Act � S ? [0,1] is the transition probability function,

such that for every state s 2 S and action a 2 Act, we havePs02SPðs;a; s0Þ 2 f0;1g.

� Act is a set of actions. At the state s 2 S, the action a isenabled iff

Ps02SPðs;a; s0Þ ¼ 1.

� Iinit:S ? [0, 1] is the initial distribution such that for allstates s 2 S;

Ps2SIinitðsÞ ¼ 1.

� L:S ? 2AP is a state labeling function.

For any state s 2 S, there is at least one enabled action a. The
operational behavior of an MDP starts with a state s0 such thatIinit(s0) > 0. At every state s, the system first chooses an enabledaction nondeterministically from the set Act(s) of enabledactions at s. Then, it performs this action probabilisticallyaccording to the transition probability function. Thus, a DTMCis a special MDP in which for any state s, there is only one ac-tion in Act(s). Therefore, by adding actions into a DTMC, we cancreate an MDP. Now we can define the MP
IS MDP model obtainedfrom our MPIS model (step 1).

Definition 6 (MPIS Model). Given an epistemic DTMC model

MPIS ¼ ðW;Pt; Iinit; �1; . . . ;�n;VÞ;MPIS ¼ S;ActP;PP

t ; IPinit;V

P� �

is anMDP model defined from MPIS as follows:

Fig. 5. Verification workflow for PCTLK.


� S = W.� ActP ¼ Run;Acc1; . . . ;Accn;AccE

G;AccCG;AccD

G

n ois a set of

actions. The action Run labels the transitions obtained from

the MPIS transitions, while each action Acci labels the tran-

sitions obtained from the epistemic accessibility relation

�i. The actions AccEG;AccC

G, and AccDG label the transitions

obtained from the accessibility relations �EG;�C

G, and �DG

respectively.� PP

t : S� ActP � S! ½0;1� is the transition probability func-tion defined as follows: for all s 2 S,

PPt ðs;a; s0Þ ¼

Ptðs; s0Þ; if a ¼ Run1

js�i s0 j; if a ¼ Acci

1js�E

Gs0 j ; if a ¼ AccE

G

1js�C

Gs0 j ; if a ¼ AccC

G

1js�D

Gs0 j ; if a ¼ AccD

G

8>>>>>>>>><>>>>>>>>>:

� IPinit ¼ Iinit .

� VP = V.

According to this translation, the transitions of MPIS are obtained (1)

from the transitions of MPIS with the same probabilities; and (2) fromthe epistemic relations where the probabilities are equally distributeddepending on the number of accessible states from each given state.For example, if 3 states are accessible from a state s, then each of thethe three obtained transitions will have 1/3 as probability.

Because MPIS, as an MDP, is not deterministic, a policy or strategy

should be used to resolve all the nondeterministic choices by pickingan enabled transition for a state, which induces a Markov chain. Inthe literature related to MDP, this policy is called scheduler (or alsoadversary). A scheduler is a function from the state set S to the actionset ActP such that it chooses in any state s one of the enabled actions.We define five particular schedulers k; ki; k

EG; k

CG, and kD

G that are usedto obtain a DTMC from the obtained MDP model MP

IS (step 2).From each state in MP

IS, the following actions are enabled:Run;Acci;AccE

G;AccCG, and AccD

G . We set the scheduler k that always se-lects the transitions labeled by Run (so the transitions obtained by theaccessibility relations are ignored). The scheduler ki always selects theaction Acci from a state s and then selects the action Run from all thefollowing states (so first the accessibility relations �i are consideredand then the normal transitions). Similarly, the schedulers kE

G; kCG,

and kDG always select the actions AccE

G;AccCG, and AccD

G respectively froma state s and then select the action Run from all the following states. Itis easy to see that the obtained models are DTMC for PCTL, whichmeans, the models M ¼ FðMPISÞ as explained above.

The following example illustrates this transformation procedure.Fig. 6(a) is the MDP model for a system with two agents (i 2 {1,2}).The two agents have the same two local states: Li = {s1,s2}. Thus fourcombinations of those local states are possible, making the number

of possible global states equal to four: {g1,g2,g3,g4}, whereg1 = (s1,s1); g2 = (s1,s2); g3 = (s2,s1); and g4 = (s2,s2). For simplicity,the two agents have the same probabilistic transition function Ti

specified as follows: Ti(s1,Run,s1) = 0.4; Ti(s1,Run,s2) = 0.6; Ti(s2, -Run,s1) = 0.9; and Ti(s2,Run,s2) = 0.1. p and q are atomic propositionsand the probabilistic transitions are shown in the figure as labeledtransitions, where the labels have the form (action name:probabil-ity value), for instance (Run:0.24) and (Acc1:0.5). The probabilitiesassociated with the action Run are calculated based on the Ti func-tion. For instance, the probability of the transition (g1,Run,g2) iscomputed as follows: Ti(s1,Run,s1) � Ti(s1,Run,s2) = 0.4 � 0.6 = 0.24.The probabilities associated with the action Acci are given and areused to compute those associated with the actions AccE

G;AccCG, and

AccDG . Fig. 6(b) is the same system with scheduler k1 at the global state

g1. This means, at g1 the action Acc1 is chosen and then the action Runis chosen from all the other three global states g2, g3, and g4.

5.2. Reducing PCTLK to PCTL

Having transformed the models, now we need to reduce PCTLKformulae into PCTL formulae. PCTL has the same syntax as PCTLK,except that the epistemic formulae j are not included. This reduc-tion process works in stages inductively. Each PCTLK formula is di-vided into ‘‘maximal state subformulae’’ such that each maximalstate subformula /: (1) includes the probabilistic or epistemicoperators; (2) differs from /; and (3) is not contained in any otherstate subformula of /. Each maximal state subformula is a PCTLKformula, which can be divided into other maximal state subformu-lae until no new maximal state subformula can be identified. Thus,in stage k, formulae of level k are decomposed into maximal statesubformulae of level k 1 or lower. The lowest level, level 0, con-tains only atomic propositions and the highest level is the wholeformula. In stage k all maximal state subformulae of / of levelsmaller than k are processed and replaced by new atomic proposi-tions, which are labels of the subformulae. Before giving an exam-ple, let us introduce the reduction rules. Let /, /1, /2 be PCTLKformulae and a be an atomic proposition, the transformation rulesare defined recursively as follows:

FðaÞ ¼ a;

Fð:/Þ ¼ :Fð/Þ;Fð/1 ^ /2Þ ¼ Fð/1Þ ^ Fð/2Þ;FðPrOb� /Þ ¼ PrOb�Fð/Þ;FðPrObð/1U/2ÞÞ ¼ PrObðFð/1ÞUFð/2ÞÞ;FðPrObð/1U6n/2ÞÞ ¼ PrObðFð/1ÞU6nFð/2ÞÞ;FðKi/Þ ¼ PrP1 �Fð/Þð Þ;FðEG/Þ ¼ PrP1 �Fð/Þð Þ;FðCG/Þ ¼ PrP1 �Fð/Þð Þ;FðDG/Þ ¼ PrP1 �Fð/Þð Þ;FðPrObKi/Þ ¼ PrOb �Fð/Þð Þ;FðPrObEG/Þ ¼ PrOb �Fð/Þð Þ;FðPrObCG/Þ ¼ PrOb �Fð/Þð Þ;FðPrObDG/Þ ¼ PrOb �Fð/Þð Þ:

Fig. 6. (a): MDP MPIS model. (b): Scheduler k1 at state g1.


To complete the reduction process, a DTMC modelM = (S,P, Iinit,L,AP) associated with each PCTL formula should be de-fined. This is done by specifying which scheduler is associated withwhich formula. In the following, (M, s)�Sch/ means the PCTL for-mula / is satisfied in the model M obtained by applying the sched-uler Sch at state s. The following theorem is a direct consequence ofthe definition of F and can be easily proved by induction on thestructure of the formula.

Theorem 5 (Satisfaction Equivalence).

ðMPIS; sÞ a iff ðFðMPISÞ; sÞka

ðMPIS; sÞ :/ iff ðFðMPISÞ; sÞk:Fð/ÞðMPIS; sÞ /1 ^ /2 iff ðFðMPISÞ; sÞkFð/1Þ ^ Fð/2ÞðMPIS; sÞ PrOb� / iff ðFðMPISÞ; sÞkPrOb�Fð/ÞðMPIS; sÞ PrObð/1U/2Þ iff ðFðMPISÞ; sÞkPrObðFð/1ÞUFð/2ÞÞðMPIS; sÞ PrObð/1U6n/2Þ iff ðFðMPISÞ; sÞkPrObðFð/1ÞU6nFð/2ÞÞðMPIS; sÞ Ki/ iff ðFðMPISÞ; sÞki

PrP1 �Fð/Þð ÞðMPIS; sÞ EG/ iff ðFðMPISÞ; sÞkE

GPrP1 �Fð/Þð Þ

ðMPIS; sÞ CG/ iff ðFðMPISÞ; sÞkCGPrP1 �Fð/Þð Þ

ðMPIS; sÞ DG/ iff ðFðMPISÞ; sÞkDGPrP1 �Fð/Þð Þ

ðMPIS; sÞ PrObðKi/Þ iff ðFðMPISÞ; sÞkiPrOb �Fð/Þð Þ

ðMPIS; sÞ PrObðEG/Þ iff ðFðMPISÞ; sÞkEGPrOb �Fð/Þð Þ

ðMPIS; sÞ PrObðCG/Þ iff ðFðMPISÞ; sÞkCGPrOb �Fð/Þð Þ

ðMPIS; sÞ PrObðDG/Þ iff ðFðMPISÞ; sÞkDGPrOb �Fð/Þð Þ

Simply put, this theorem states that if the PCTLK formula doesnot include epistemic operators, then the corresponding PCTL for-mula is satisfied in the DTMC obtained by only considering the nor-mal transitions. However, if the formula has the form of Ki/, thenthe corresponding PCTL formula is satisfied in the DTMC obtainedby considering first the epistemic accessibility relation �i and thenthe normal transitions, which shows why the K operator is trans-lated to the next operator. The same intuition holds for the theother epistemic formulae.

We use the previous example shown in Fig. 6 to illustrate howto convert PCTLK formulae into PCTL by levels and identify thestates that satisfy each maximal state subformula. The PCTLK for-mula K1(PrP0.50((Pr>0.70sp) Uq)) asserts that agent 1 knows that

at least in 50% of the cases, there is more than 70% of chance thatin the next state p holds until q. The levels of the subformulae ofthis formula and the new atomic propositions labeling each maxi-mal state subformula are shown in Table 1:

6. Case study and performance comparison

We implement our reduction-based model checking techniqueon top of PRISM as a tool that takes as input PCTLK formulae andMPIS model and produces as output PCTL formulae and DTMC mod-el FðMPISÞ, which are the inputs of PRISM. PRISM [31], a probabilis-tic symbolic model checker, is a tool for formal modeling andanalysis of systems which exhibit random or probabilistic behav-ior. It supports three types of probabilistic models, discrete-timeMarkov chains (DTMCs), continuous-time Markov chains (CTMCs)and Markov decision processes (MDPs). These models can be spec-ified in the PRISM modeling language with a simple state-basedlanguage. The property specification language incorporates thetemporal logics PCTL, CSL, LTL, and PCTL⁄.

We apply the approach using the modified protocol of Chaum’sdining cryptographers [9] as case study. The Dining Cryptographersprotocol (DC) aims to get information from anonymous broadcast-ing messages and has already been modeled using agents by vari-ous authors [22,25,34,47]. We add an uncertainty situation to theoriginal protocol and extend it into the Cheating Dining Cryptogra-phers protocol (CDC). The epistemic and probabilistic properties ofthe CDC protocol are automatically transformed into PCTL and ver-ified by the PRISM probabilistic model checker.

6.1. Protocol description

Anonymity is an important issue in the field of modern cryptog-raphy. The original Dining Cryptographers (DC) Protocol is intro-duced by the following scenario [9]:

Three cryptographers are sitting down to dinner at their favor-ite three-star restaurant. Their waiter informs them thatarrangements have been made with the maitre d’hotel for thebill to be paid anonymously. One of the cryptographers mightbe paying for dinner, or it might have been NSA (U.S. NationalSecurity Agency). The three cryptographers respect each othersright to make an anonymous payment, but they wonder if NSAis paying.

Table 1Example of reduction of the formula K1(PrP0.50((Pr>0.70sp) Uq)).

Level Subformulae Transformedsubformulae

Labeledstates set

Level 0 p /01 ¼ p {g1,g2,g3}

q /02 ¼ q {g2,g3,g4}

Level 1 Pr>0.7sp /11 ¼ Pr>0:7� /0

1{g1,g2,g3,g4}

Level 2 PrP0.5((Pr>0.7sp) Uq) /21 ¼ PrP0:5 /1

1U/02

� �{g1,g2,g3,g4}

Level 3 K1(PrP0.5((Pr>0.7sp) Uq)) /31 ¼ PrP1 � /2

1{g1,g2,g3,g4}

2 The code for the implementation results and comparison with extended MCK andMCMAS is available at: https://sourceforge.net/projects/mcepistemicprob/files/?source=navbar.


Based on the assumption that at most one cryptographer is pay-ing, the following rules can solve the dining cryptographers’ quan-dary [9]:

1. Each cryptographer flips an unbiased coin and only showsthe outcome to the cryptographer on his right.

2. Each cryptographer states whether the two coins he can seeare on the same side or on different sides.

3. The cryptographer who pays the dinner states the oppositeof what he sees.

After running this protocol, all the cryptographers can deter-mine whether it was the NSA or one of the cryptographers whopaid for dinner: an odd number of differences indicates that a cryp-tographer is paying; an even number indicates that NSA is paying.Also, if a cryptographer is paying, neither of the other two can fig-ure out who is paying. This protocol can also be applied to morethan three cryptographers.

We assume that cryptographers may make mistakes or deliber-ately break the protocol by certain probability (we use cheating in-dex p to indicate it). We aim to investigate how cheating index paffects the accuracy of cryptographers’ inference. Kacprzak et al.in [25] also mentioned Cheating Dining Cryptographers (CDC) pro-tocol but without indicating the degree of cheating. The DC proto-col can be seen as a special CDC protocol with cheating index p = 0for all the cryptographers. We discuss the encoding and verifica-tion of the CDC protocol in the following section.

6.2. Protocol encoding

Our modeling techniques are completely compatible withPRISM implementation. We translate every agent into a modulein PRISM and the entire multi-agent system is defined as a systemwith agent modules which are all synchronized. This PRISM systemcan be viewed as DTMC model scheduled using k. To implementthe epistemic schedulers ki; k

EG; k

CG, and kC

G, we use labels to definethe epistemic relations. To illustrate, label A1s1 includes a set ofstates that are equivalent for agent A1 at s1 (k1), while labelA1A2s0E or A1A2s0D contains the union or joint of group (agent1 and agent 2) accessibility relations at global state (s0)respectively.

label ‘‘A1s1"=(s1 = 0|s1 = 1|s1 = 2);

label ‘‘A1A2s0E"=(s1 = 0|s1 = 1|s1 = 2|s1 = 4|s1 = 5);

label ‘‘A1A2s0D"=(s1 = 0);

Therefore, A1s1 is considered as the set of states that comprisesall the states related to s1 by �1. Similarly, A1A2s0E and A1A2s0Dare the state set related to s0 by �E

G and �DG .

We model the CDC protocol differently from that presented in[25]. We add cheating index p to indicate the probability of a cryp-tographer being dishonest (or making mistakes) since probabilisticinterpreted systems allow probabilistic transitions. For simplifica-tion but without loss of generality, we assume that all the cryptog-

raphers have the same cheating index p and generalization to nindices is straightforward. When we set all the cryptographers’cheating indices to 0, the CDC protocol becomes DC protocol.

To formalize the protocol, we naturally assume that NSA andthe three cryptographers have an equal chance of paying the bill.We encode the scenario by using a probabilistic interpreted sys-tem. Each cryptographer Ci is comprised of two variables: flippingthe coin variable and stating results variable. The variable flippingthe coin uses 3 states: flipping coin, head, tail. The variable statingresults has 9 states with the meaning is intuitively explained bytheir labels: NotDecided, pay, notPay, seeEqual/pay, seeDiff/pay, see-Equal/notPay, seeDiff/notPay, saidEqual, and saidDiff (see Fig. 7).

To model the CDC protocol in PRISM, we add a synchronizingflag for every cryptographer agent to avoid deadlock. When a cryp-tographer has announced the outcome, the synchronizing value isset to 1 and stays. After all the cryptographers’ synchronizing flagshave been set, the system will count all saidDiff and to see if thenumber of saidDiff is even or odd we use the integer modulo oper-ation mod.

label ‘‘even"=fun (mod, (diff1 + diff2 + diff3), 2)=0;

label ‘‘odd"=fun (mod, (diff1 + diff2 + diff3),2)=1;

6.3. Experimental results

We analyze the verification results for the DC and CDCprotocols.2 The presented experimental results were performed ona DELL desktop computer with 1.86 GHz Intel Core Duo T6300 pro-cessor and 3.25 GB memory under 32-bit Windows WinXP profes-sional version 2002 Service Pack 3 Operating System.

Table 2 shows the size and run time for the models that we builtfor 3–20 cryptographers for the DC and CDC protocols. The numberof states and transitions are our model size. The construction timeis the time for converting the PRISM model into Multi-TerminalBinary Decision Diagram (MTBDD) symbolic model. We can seeas the number of cryptographers increases, the model size in-creases dramatically, but PRISM can still handle the large numberof states (2.31E + 13 and 2.00E + 15) and transitions (2.31E + 14and 2.40E + 16) thanks to the symbolic approach and some internaloptimization techniques that PRISM uses to reduce the model size.For the same number of cryptographers, the CDC protocol modelrequires more states and transitions than the DC protocol modelbecause of the increasing uncertainty. Generally speaking, thebuilding model time for CDC is also slightly more than the buildingmodel time for DC (see part a of Fig. 8). The execution time is thetotal time of constructing the model and computing iterativelythe set of states which are reachable from the initial states andthe transition matrix. As the number of cryptographersincreases, the gap between construction time and execution timeincreases as well. This means the more cryptographers, the longertime to compute reachability (see part b. of Fig. 8).

We use Cipaid to stand for cryptographer i paid the bill andNpaid for NSA paid the bill. p is the cheating index for cryptogra-phers. We set 3 cryptographers in the systems and assume thatall the cryptographers have the same cheating index for simplifica-tion reasons. p = 0 means that cryptographers do not cheat and fol-low the rules completely. Some desired properties of the protocolare expressed in Table 3 and are self-explanatory.

The properties expressed in the first four formulae are checkedfor the classical DC protocol since CDC and DC are equivalent whenp = 0. The property encoded in formula 1 expresses that if NSA payfor the dinner and all the cryptographers are honest, an even num-

http://https://sourceforge.net/projects/mcepistemicprob/files/?source=navbar

http://https://sourceforge.net/projects/mcepistemicprob/files/?source=navbar

Fig. 7. Cheating cryptographers protocol: (a) flipping coins and (b) stating outcomes.

Table 2Experimental results with our extended PRISM.

Number ofcrypt. (n)

Dining cryptographer Cheating dining cryptographer

Number ofstates

Number oftransitions

Constructiontime (s)

Executiontime (s)

Number ofstates

Number oftransitions

Constructiontime (s)

Executiontime (s)

3 261 452 <0.01 <0.01 503 1036 0.016 0.0174 1286 2725 0.015 0.015 3146 8063 0.016 0.0215 6151 15,750 0.031 0.046 18,753 57,882 0.032 0.0406 28,680 86,919 0.047 0.062 109,378 399,273 0.034 0.0497 131,081 460,808 0.063 0.078 625,003 2,643,480 0.078 0.1098 589,834 2,363,913 0.110 0.157 3,515,628 16,936,299 0.094 0.1269 2,621,451 11,806,730 0.125 0.172 19,531,253 105,670,630 0.141 0.17710 11,534,348 57,694,219 0.157 0.220 107,421,878 645,191,965 0.188 0.26611 5.03E+7 2.77E+8 0.128 0.327 5.86E+8 3.87E+9 0.265 0.37412 2.18E+8 1.31E+9 0.203 0.281 3.26E+9 2.36E+10 0.204 0.29713 9.40E+8 6.11E+9 0.328 0.515 1.71E+10 1.33E+11 0.453 0.62514 4.03E+9 2.82E+10 0.469 0.734 9.16E+10 7.69E+11 0.562 0.84315 1.72E+10 1.29E+11 0.515 0.812 4.88E+11 4.39E+12 0.563 0.86016 7.30E+10 5.84E+11 0.703 1.047 2.59E+12 2.49E+13 0.765 1.15617 3.09E+11 2.63E+12 0.813 1.251 1.37E+13 1.40E+14 0.86 1.31318 1.31E+12 1.18E+13 1.063 1.782 7.25E+13 7.83E+14 1.172 1.79719 5.50E+12 5.22E+13 1.250 2.047 3.81E+14 4.35E+15 1.515 2.45220 2.31E+13 2.31E+14 1.546 2.577 2.00E+15 2.40E+16 1.703 2.719


ber of differences is announced so that all the cryptographers knowthat the bill is paid by NSA. The properties expressed in the formu-lae 2–4 express that if cryptographer i does not pay the bill and anodd number of differences is stated, then i knows that the dinner ispaid by one of his partners rather than NSA, but he does not knowexactly who paid.

In PRISM, if / is a formula, operator P = ?/ is used to calculatethe probability of /. Formulae 5–8 show that the probability trendsof certain properties vary as the cheating index changes from 0 (0%cheating) to 1 (100% cheating). The results of formulae 5a and 5bshow respectively the probability that NSA and one of the cryptog-raphers is paying the bill. The value does not change with cheatingindex because the probability of paying the bill is independentfrom this index. However, the number of differences announcedchanges with the cheating index (formulae 6a and 6b). When thecheating index increases from 0 to 1 the results becomeunpredictable because of the high uncertainty (Formulae 7a, 7b,8a, and 8b).

6.4. Comparison with extended MCK and MCMAS

Although the DC protocol has already been modeled by manyresearchers [22,25,34,47] in different ways to represent and reasonabout knowledge, most of them [25,34,47] only focus on qualita-tive properties of knowledge using MCMAS and MCK, so quantita-

tive properties cannot be expressed in these approaches. Huanget al. in [22] present a symbolic model checking algorithm forthe verification of probabilistic knowledge. Huang et al.’s approachworks on a particular class of interpreted systems called partiallyobserved discrete-time Markov chains (PO-DTMC), which are syn-chronous with perfect recall. The model checking algorithm hasbeen implemented on top of the MCK model checker. In this sec-tion, we compare the performance of our reduction-based modelchecking technique implemented on top of PRISM against the ex-tended MCK and MCMAS as benchmarks. The comparison focuseson both epistemic and probabilistic properties.

We verify two properties of the DC protocol, one epistemic andone probabilistic, using Huang et al.’s method with extended MCKand our approach with extended PRISM. Extended MCK is a modelchecker developed for the logic of knowledge and probability. Itsupports both linear and branching temporal logics with severaldifferent ways of defining knowledge based on the observationsmade by the agents: observation alone, observation and clock,and perfect recall of all observations. We also include MCMAS asbenchmark to verify the epistemic (non-probabilistic) property.MCMAS is a symbolic model checker that is designed for multi-agent systems. It supports temporal, knowledge, and commitmentlogics [14,15]. The comparison is under Fedora 16 with 3.6 Linuxand the following characteristics: 1.86 GHz Intel Core Duo T6300processor and 3.25 GB memory.

Fig. 8. Construction and execution time for the CD and CDC protocols.

Table 3Examples of verified properties for the CDC protocol with 3 cryptographers.

No. Formulae Results Time for model checking (s)

1 (even ^ p = 0))V

i(Ki(!C1paid ^ !C2paid ^ !C3paid)) True 0.0022 (odd ^!Cipaid ^ p = 0)) (Ki(

Wj–iCjpaid)) True 0.002

3 (odd ^ !Cipaid ^ p = 0)) Ki(P60.5(Cjpaid))j – i True (8/16 = 0.5) 0.0054 (odd ^!Cipaid ^ p = 0)) Ki(Cjpaid) j – i False 0.0025a. (P = ?Npaid) (for 0 < p < 1step 0.1) Fig. 9: NSA pays 0.0045b. (P = ?!Npaid) (for 0 < p < 1step 0.1) Fig. 9: Crypt pays 0.0036a. (P = ?even) (for 0 < p < 1 step 0.1) Fig. 9: even difference 0.0026b. (P = ?odd) (for 0 < p < 1 step 0.1) Fig. 9: odd difference 0.0027a. (P = ?(odd ^!Npaid)) (for 0 < p < 1 step 0.1) Fig. 9: odd and Crypt paid 0.0037b. (P = ?(even ^!Npaid)) (for 0 < p < 1 step 0.1) Fig. 9: even and Crypt paid 0.0018a. (P = ?(even ^Npaid)) (for 0 < p < 1 step 0.1) Fig. 9: even and NSA paid 0.0018b. (P = ?(odd ^Npaid)) (for 0 < p < 1 step 0.1) Fig. 9: odd and NSA paid 0.003

Table 4Experimental results with extended MCK, our extended PRISM, and MCMAS.

Numberof crypt.(n)

Time for extended MCK Time for our extendedPRISM

Time forMCMAS

Epistemicproperty(s)

Probabilisticproperty (s)


Probabilisticproperty (s)


3 0.071 1.405 0.003 0.006 0.0114 0.382 136.6 0.004 0.018 0.0145 6.181 – 0.005 0.053 0.0556 – – 0.011 0.063 0.1167 – – 0.019 0.138 0.218 – – 0.022 0.254 0.5349 – – 0.033 0.48 0.943

10 – – 0.044 0.863 1.84411 – – 0.059 1.445 2.21912 – – 0.079 2.366 9.88313 – – 0.109 3.741 –14 – – 0.124 5.756 –15 – – 0.216 8.476 –16 – – 0.224 12.301 –17 – – 0.287 17.695 –18 – – 0.352 24.567 –19 – – 0.403 33.546 –20 – – 0.607 45.686 –


The first property is that if the first cryptographer did not payfor the dinner, he knew that either NSA paid or one of the cryptog-raphers paid, but he did not know which particular cryptographerpaid. This epistemic property for three cryptographers can be ex-pressed in MCK as follows:

spec spr xn ¼ Xnðneg paid½1�Þ) ððKnows C1 ððneg paid½2�Þ ^ ðneg paid½3�ÞÞÞ_ ððKnows C1 ðpaid½2� _ paid½3�Þ^ neg Knows C1 paid½2�Þ^ neg Knows C1 paid½3�ÞÞ ð6:1Þ

where spec_spr_xn is a specification identifier indicating thatagents’s knowledge is based on synchronous perfect recall and theverification uses binary decision diagram symbolic model checkingalgorithm. neg is a negation operator and Knows is a knowledge oper-ator. paid[i] is a Boolean variable that indicates if cryptographer i paidthe bill. n is an integer stating that in n steps the specification holdsand X is the next operator. The same property is expressed in PRISM(Formula (6.2)) and MCMAS (Formula (6.3)) respectively as follows:

filterðforall; !\C1paid") ðP >¼ 1 ½X!\C2paid"�& P >

¼ 1½X!\C3paid"�ÞjððP >¼ 1½X!\C2paid"�j P >

¼ 1½X!\C3paid"�Þ &!P >

¼ 1½X\C2paid�&!P >

¼ 1½X\C3paid"�Þ; \done"&!\C1paid"Þ ð6:2Þ

AGðð!C1paidÞ ! ðKðC1; !C2paidand!C3paidÞÞorðKðC1;C2paid or C3paidÞand !KðC1; C2paidÞ and !KðC1;C3paidÞÞÞ;

ð6:3Þ

Fig. 9. Verification results of some properties in the CDC protocol for 3 cryptographers with regard to the cheating index.

Fig. 10. Runtime for verifying the DC protocol.


In Formula (6.2), filter returns values for the identified set of stateswhich satisfy the filter command. In this case, the system will com-pute values for all states where cryptographer C1 did not pay thebill. In Formula (6.3), C1paid, C2paid, C3paid are defined to respec-tively represent the fact that Cryptographer 1, 2, and 3 paid the bill.

The second property is probabilistic. It indicates that one cryp-tographer knows that the other cryptographers have the equal

probability to pay the bill. The probability is either 0 or 1/(n 1)for n cryptographers. In MCK, the probabilistic property is ex-pressed as follows:

spec spr xn ¼ X nðnegpaid½0�Þ ) ððProb C1 paid½1� ¼¼ Prob C2 paid½2�Þ ^ ððProb C1 paid½1� ¼¼ 0Þ _ ðProb C1 paid½1� ¼¼ 0:5ÞÞÞ ð6:4Þ


Although it is easy to calculate path probability in PRISM, it can-not compute the probability of knowledge directly. However, whenwe verify a filter specification, PRISM gives the number of statessatisfying the filter (n1) and the total number of filtered states(n2). Based on our definition of probabilistic knowledge, we areable to calculate the knowledge probability by computing n1/n2.So, in PRISM, we use the calculated result of the following expres-sion to get the probability of knowledge:

filterðforall; P >¼ 1½X\C2paid"�; \done"&!\C1paid"&\odd"Þ ð6:5Þ

Table 4 reports the runtime for verifying the epistemic andprobabilistic properties with extended MCK, our extended PRISM,and MCMAS. With extended MCK, we can only get the runtimeof verifying (1) the epistemic property for up to 5 cryptographersand (2) the probabilistic property for up to 4 cryptographers. WithMCMAS, the case can be scaled up to 12 agents. However, with ourextended PRISM, we can get the verification results for up to 20cryptographers. The runtime for verifying the DC protocol includ-ing 20 cryptographers with this extended version of PRISM is stillrelatively low, which shows that our approach is time efficientand scalable. In fact, for all the numbers of cryptographers rangingfrom 3 to 20, our extended PRISM shows better performance forboth epistemic and probabilistic properties. The results indicatethat for both probabilistic model checkers, namely extended MCKand extended PRISM, probabilistic properties need longer time tobe verified, but our reduction-based model checking techniqueoutperforms Huang et al.’s approach (See Fig. 10). Also, as the sizeof the system increases, the execution time of all the three modelcheckers increases: with MCMAS and extended PRISM, the increaseis polynomial with a lower rate for extended PRISM, whereas withextended MCK, the increase is exponential.

The experimental results show that in verifying the DC protocol,our extended PRISM has the best performance. This is because inthe DC protocol, properties are checked after all the cryptographershave finished their actions. The verification only happens on thestatic environment. In fact, PRISM is designed for quantitative ver-ification, while MCK and MCMAS are originally introduced forknowledge verification. Our method reduces knowledge verifica-tion into probabilistic computation tree logic verification, so it ben-efits from the efficiency of PRISM in verifying such a logic, thanksto the Multi-Terminal Binary Decision Diagram (MTBDD) packageused in this model checker, compared to the limited Binary Deci-sion Diagram (BDD) package used in MCK and MCMAS. However,for reactive dynamic systems that need to recall previous settings,PRISM does not function well. In contrast, MCK is good at verifyingsuch systems using observations with perfect synchronous recall.MCK is designed for supporting several different ways of definingknowledge, including synchronous and asynchronous perfect recallfor observations. MCMAS is tailored to the verification of scalablemulti-agent systems. It can be used to verify group knowledgeand collaboration among agents.

7. Conclusion and future work

In this paper, we introduced epistemic DTMC models MPIS,which extend interpreted systems with probabilistic features.Epistemic DTMC can model systems that are subject to stochasticphenomena. We also defined PCTLK, a logic to reason aboutprobabilistic and uncertain knowledge. The semantics of this logicis defined with respect to the epistemic DTMC models. To modelcheck PCTLK, a reduction technique to the model checking problemof PCTL has been proposed. This technique uses models transfor-mation and formulae reduction to verify probabilistic and episte-mic properties. We implemented our technique to verify the CDand CDC protocols.

As future work, we plan to investigate the completeness prop-erty of our logic and the use of other model checking techniques,namely unbounded model checking. Model checking can also beused to complement the detection of logical inconsistencies inknowledge-based systems [41]. In this context, we are planing toinvestigate the extension of our verification approach to thisapplication.

Another direction for future work is the employment of knowl-edge bases as supportive mechanisms for the verification of episte-mic and probabilistic-based multi-agent systems. The idea is toenrich the verification process by a control process by addressingthe following synthesis problem: given the knowledge bases ofthe agents in the multi-agent system, is there any strategy to re-strict the model in order to satisfy the desirable properties, suchas safety and reachability? Model checking epistemic and probabi-listic logic can also be applied to the knowledge revision area tocheck if by revising and adding new knowledge, the model still sat-isfies the properties the designer wants. Finally, using ontology toprovide knowledge on how agents can reflect on uncertainty is an-other interesting research direction.

Acknowledgments

We would like to thank the four anonymous reviewers for theirvaluable technical comments and suggestions for improvements.We also thank NSERC (Canada), FQRNT, and FQRSC (Quebec) fortheir fincial support. The second author would like to thank KhalifaUniversity of Science, Technology and Research for the financialsupport.

References

[1] I. Anciutti, A learning classifier system for emergent team behavior in real-timePOMDP, in: Proceedings of the IEEE International Conference on IntelligentComputing and Intelligent Systems (ICIS), 2009, pp. 733–738.

[2] C. Baier, J.-P. Katoen, Principles of Model Checking, MIT Press, 2008.[3] F. Belardinelli, A. Lomuscio, A complete first-order logic of knowledge and

time, in: Proceedings of the Eleventh International Conference on Principles ofKnowledge Representation and Reasoning, 2008, pp. 705–714.

[4] F. Belardinelli, A. Lomuscio, Quantified epistemic logics for reasoning aboutknowledge in multi-agent systems, Artificial Intelligence 173 (9-10) (2009)982–1013.

[5] J. Bentahar, J.J. Meyer, W. Wan, Model checking communicative agent-basedsystems, Knowledge-Based Systems 22 (3) (2009) 142–159.

[6] J. Bentahar, H. Yahyaoui, M. Kova, Z. Maamar, Symbolic model checkingcomposite Web services using operational and control behaviors, ExpertSystems with Applications 40 (2) (2013) 508–522.

[7] Z. Cao, Model checking for epistemic and temporal properties of uncertainagents, in: Agent Computing and Multi-Agent Systems, LNAI, vol. 4088,Springer, 2006, pp. 46–58.

[8] A.R. Cassandra, L.P. Kaelbling, M.L. Littman, Acting optimally in partiallyobservable stochastic domains, in: Proceedings of the National Conference onArtificial Intelligence, 1995, pp. 1023–1023.

[9] D. Chaum, The dining cryptographers problem: unconditional sender andrecipient untraceability, Journal of Cryptology 1 (1) (1988) 65–75.

[10] C. Delgado, M. Benevides, Verification of epistemic properties in probabilisticmulti-agent systems, in: Proceeding of the 7th German Conference onMultiagent System Technologies (MATES), LNAI, vol. 5774, Springer-Verlag,2009, pp. 16–28.

[11] L. Dennis, M. Fisher, M. Webster, R. Bordini, Model checking agentprogramming languages, Automated Software Engineering 19 (1) (2012) 5–63.

[12] L.T. Dung, T. Komeda, M. Takagi, Reinforcement learning for POMDP usingstate classification, in: Proceedings of the International Conference onMachine Learning: Models, Technologies & Applications (MLMTA), 2007, pp.45–51.

[13] R. Durrett, Probability: Theory and Examples, Cambridge University Press,2010.

[14] M. El-Menshawy, J. Bentahar, W. El-Kholy, R. Dssouli, Reducing modelchecking commitments for agent communication to model checking arctland gctl⁄, Autonomous Agents and Multi-Agent Systems (2013), http://dx.doi.org/10.1007/s10458-012-9208-7.

[15] M. El-Menshawy, J. Bentahar, W. El-Kholy, R. Dssouli, Verifying conformance ofmulti-agent commitment-based protocols, Expert Systems with Applications40 (1) (2013) 122–138.

[16] R. Fagin, J.Y. Halpern, Y. Moses, Y.M. Vardi, Reasoning About Knowledge, MITPress, Cambridge, 1995.

http://refhub.elsevier.com/S0950-7051(13)00199-8/h0005


























http://dx.doi.org/10.1007/s10458-012-9208-7

http://dx.doi.org/10.1007/s10458-012-9208-7








[17] P. Gammie, R. van der Meyden, MCK: model checking the logic of knowledge,in: Proceedings of the 16th International Conference on Computer AidedVerification (CAV), LNCS, vol. 3114, Springer-Verlag, 2004, pp. 479–483.

[18] H. Geffner, J. Wainer, Modeling action, knowledge and control, in: Proceedingsof the European Conference on Artificial Intelligence (ECAI), 1998, pp. 532–536.

[19] J. Halpern, M. Vardo, Model checking vs. theorem proving: a manifesto, in:Proceddings of the 2nd International Conference on Principles of KnowledgeRepresentation and Reasoning (KR), 1991, pp. 325–334.

[20] J.Y. Halpern, Reasoning About Uncertainty, MIT Press, 2003.[21] H. Hansson, B. Jonsson, A logic for reasoning about time and reliability, Formal

Aspects of Computing 6 (5) (1994) 512–535.[22] X. Huang, C. Luo, D.M. Van, Symbolic model checking of probabilistic

knowledge, in: Proceedings of the13th Conference on Theoretical Aspects ofRationality and Knowledge (TARK), 2011, pp. 177–186..

[23] X. Huang, K. Su, C. Zhang, Probabilistic alternating-time temporal logic ofincomplete information and synchronous perfect recall, in: Proceedings ofTwenty-Sixth AAAI Conference on Artificial Intelligence, 2012, pp. 765–771.

[24] W. Jamroga, A temporal logic for markov chains, in: Proceeding of the 7thInternational Conference on Autonomous Agents and Multi-Agent Stystems(AAMAS), 2008, pp. 697–704.

[25] M. Kacprzak, A. Lomuscio, A. Niewiadomski, W. Penczek, F. Raimondi, M.Szreter, Comparing BDD and SAT based techniques for model checkingchaum’s dining cryptographers protocol, Fundamenta Informaticae 72 (1–3)(2006) 215–234.

[26] L.P. Kaelbling, M.L. Littman, A.R. Cassandra, Partially observable markovdecision processes for artificial intelligence, in: Reasoning with Uncertainty inRobotics, LNCS, vol. 1093, Springer-Verlag, 1996, pp. 146–163.

[27] L.P. Kaelbling, M.L. Littman, A.R. Cassandra, Planning and acting in partiallyobservable stochastic domains, Artificial Intelligence 101 (1–2) (1998) 99–134.

[28] R. Kaplow, A. Atrash, J. Pineau, Variable resolution decomposition for roboticnavigation under a POMDP framework, in: Proceedings of the IEEEInternational Conference on Robotics and Automation, 2010, pp. 369–376.

[29] M. Khan, M. Banerjee, A logic for multiple-source approximation systems withdistributed knowledge base, Journal of Philosophical Logic 40 (5) (2011) 663–692.

[30] O. Kupferman, M. Vardi, P. Wolper, An automata-theoretic approach tobranching-time model checking, Journal of the ACM 47 (2) (2000) 312–360.

[31] M. Kwiatkowska, G. Norman, D. Parker, PRISM: probabilistic symbolic modelchecker, in: Proceedings of the 12th International Conference on ComputerPerformance Evaluation: Modelling Techniques and Tools (TOOLS), LNAI, vol.2324, 2002. pp. 113–140.

[32] M. Kwiatkowska, G. Norman, D. Parker, Probabilistic symbolic model checkingwith PRISM: a hybrid approach, International Journal on Software Tools forTechnology Transfer 6 (2) (2004) 128–142.

[33] J. Lawry, Y. Tang, On truth-gaps, bipolar belief and the assertability of vaguepropositions, Artificial Intelligence (2012) 20–41.

[34] A. Lomuscio, C. Pecheur, F. Raimondi, Automatic verification of knowledge andtime with NuSMV, in: Proceedings of the 20th International Conference onArtificial Intelligence (IJCAI), 2007, pp. 1384–1389.

[35] A. Lomuscio, W. Penczek, Symbolic model checking for temporal-epistemiclogics, SIGACT News 38 (3) (2007) 76–100.

[36] A. Lomuscio, H. Qu, M. Solanki, Towards verifying contract regulated servicecomposition, Journal of Autonomous Agents and Multi-Agent Systems 24 (3)(2011) 345–373.

[37] A. Lomuscio, F. Raimondi, MCMAS: a model checker for multi-agent systems,in: Proceedings of the 12th International Conference on Tools and Algorithmsfor the Construction and Analysis of Systems (TACAS), LNCS, vol. 3920,Springer-Verlag, 2006, pp. 450–454.

[38] S. Paquet, L. Tobin, B. Chaib-draa, Real-time decision making for large POMDPs,in: Proceedings of the 18th Conference of the Canadian Society forComputational Studies of Intelligence: Advances in Artificial Intelligence(Canadian AI), LNAI, vol. 3501, Springer-Verlag, 2005, pp. 450–455.

[39] W. Penczek, A. Lomuscio, Verifying epistemic properties of multi-agentsystems via bounded model checking, Fundamenta Informaticae 55 (2)(2003) 167–185.

[40] E. Pulvermueller, S. Feja, A. Speck, Developer-friendly verification of process-based systems, Knowledge-Based Systems 23 (7) (2010) 667–676.

[41] J. Ramírez, A. de Antonio, Checking the consistency of a hybrid knowledge basesystem, Knowledge-Based Systems 20 (3) (2007) 225–237.

[42] G. Shani, R. Brafman, S. Shimony, Model-based online learning of POMDPs, in:Proceedings of the 16th European Conference on Machine Learning (ECML),LNCS, vol. 3720, Springer-Verlag, 2005, pp. 353–364.

[43] T. Taha, J. Miro, G. Dissanayake, A POMDP framework for modelling humaninteraction with assistive robots, in: Proceedings of the IEEE InternationalConference on Robotics and Automation, 2011, pp. 544–549.

[44] S. Uckelman, Deceit and indefeasible knowledge: the case of dubitatio, Journalof Applied Non-Classical Logics 21 (3-4) (2011) 503–519.

[45] R. van der Meyden, Axioms for knowledge and time in distributed systemswith perfect recall, in: Proceedings of the Symposium on Logic in ComputerScience (LICS), 1994, pp. 448–457.

[46] R. van der Meyden, N. Shilov, Model checking knowledge and time in systemswith perfect recall, in: Proceedings of the 19th Conference on the Foundationsof Software Technology and Theoretical Computer Science, LNCS, vol. 1738,Springer, -Verlag, 1999, pp. 432–445.

[47] R. van der Meyden, K. Su, Symbolic model checking the knowledge of thedining cryptographers, in: Proceedings of the 17th IEEE Computer SecurityFoundations Workshop (CSFW), vol. 17, 2004, pp. 280–291.

[48] H. Van Ditmarsch, W. Van Der Hoek, B. Kooi, Dynamic Epistemic Logic,Springer, 2007.

[49] W. Wan, J. Bentahar, A. Ben-Hamza, Model checking epistemic andprobabilistic properties of multi-agent systems, in: Proceedings of the 24thInternational Conference on Industrial Engineering and Other Applications ofApplied Intelligent Systems (IEA/AIE), LNAI, vol. 6704, Springer-Verlag, 2011,pp. 68–78.

[50] W. Wan, J. Bentahar, A. Ben-Hamza, Quantitative model checking ofknowledge, Proceedings of the 11th International Conference on IntelligentSoftware Methodologies, Tools and Techniques (SoMeT) from Frontiers inArtificial Intelligence and Applications, vol. 246, IOS Press, 2012, pp. 91–107.doi:http://dx.doi.org/10.3233/978-1-61499-125-0-91.








































































http://dx.doi.org/10.3233/978-1-61499-125-0-91

model checking epistemic–probabilistic logic using probabilistic interpreted systems

Documents