Control of Stochastic Discrete Event Systems Modeled

by Probabilistic Languages 1

Ratnesh KumarDept. of Electrical Engineering, Univ. of Kentucky

Lexington, KY 40506

Vijay K. GargDept. of Elec. and Comp. Eng., Univ. of Texas at Austin

Austin, TX 78712

1This research is supported in part by the National Science Foundation under Grants NSF-

ECS-9409712, NSF-ECS-9709796, ECS-9414780, and CCR-9520540, in part by the Office of Naval

Research under the Grant ONR-N00014-96-1-5026, a General Motors Fellowship, a Texas Higher

Education Coordinating Board Grant ARP-320, and an IBM grant.

Abstract

In earlier papers [7, 6, 5] we introduced the formalism of probabilistic languages for mod-eling the stochastic qualitative behavior of discrete event systems (DESs). In this paper westudy their supervisory control where the control is exercised by dynamically disabling cer-tain controllable events thereby nulling the occurrence probabilities of disabled events, andincreasing the occurrence probabilities of enabled events proportionately. This is a specialcase of “probabilistic supervision” introduced in [15]. The control objective is to design asupervisor such that the controlled system never executes any illegal traces (their occurrenceprobability is zero), and legal traces occur with minimum pre-specified occurrence proba-bilities. In other words, the probabilistic language of the controlled system lies within apre-specified range, where the upper bound is a “non-probabilistic language” representing alegality constraint. We provide a condition for the existence of a supervisor. We also presentan algorithm to test this existence condition when the probabilistic languages are regular(so that they admit probabilistic automata representation with finitely many states). Nextwe give a technique to compute a maximally permissive supervisor on-line.

Keywords: Stochastic discrete event systems, supervisory control, probabilistic languages.

1 Introduction

Discrete event systems (DESs) are systems with discrete states that are event-driven,i.e., that change states in response to the occurrence of events at random instants of time.Supervisory control of DESs pioneered by Ramadge-Wonham [22] and subsequently extendedby other researchers (see for example [12]) provides a framework for designing controllers forcontrolling the non-stochastic untimed or qualitative behavior of DESs. The non-stochasticqualitative behavior of DESs is generally modeled by the set of all possible sequences ofevents, a formal language [10], that can occur in the system. Such non-stochastic behaviorof a discrete event system can alternatively be viewed as a binary valued map over the setof all possible sequences of events, called traces. (A trace is mapped to one if and only if itbelongs to the system behavior.) We call such a map non-probabilistic language.

For control purposes the set of events is partitioned into the sets of controllable anduncontrollable events. A supervisor controls the given system, called plant, by dynami-cally disabling certain controllable events based on its observations of the executed traces.The control objective is to design a supervisor so that the non-probabilistic language of thecontrolled plant is bounded within a certain range specified as a pair of non-probabilisticlanguages. The upper bound specifies the set of legal traces, whereas the lower bound givesthe minimally adequate set of traces. A supervisor exists if and only if the infimal control-

lable superlanguage [14] of the lower bound is bounded above by the supremal controllable

sublanguage of the upper bound [23]. (A language is said to be controllable if it is closedwith respect to the executions of feasible uncontrollable events.)

In [7, 6, 5] we introduced a more general map over the set of all traces that take valuesin the closed unit interval (instead of just in the set of binary numbers) to describe thestochastic qualitative behavior of a DES. The interpretation being that the value associatedwith a certain trace under such a map is its occurrence probability. In order for sucha probabilistic map to describe the stochastic behavior of a DES it must satisfy certainconsistency properties obtained in [7]: (i) the probability of the zero length trace is one,and (ii) the probability of any trace is at least as much as the cumulative probability ofall its extensions. The first constraint follows from the fact that the execution of the zerolength trace is always possible, whereas the second constraint follows from the fact that ifthe execution of a trace is possible, then the execution of any of its prefix is also possible. Wecall such maps probabilistic languages and use them for modeling the stochastic qualitativebehavior of DESs.

A probabilistic language can be viewed as a formal power series [24], but satisfying theconstraints mentioned above. As discussed in [7], this probabilistic language model differsin various ways from other existing models of stochastic behavior of DESs such as Markovchains [1], stochastic Petri nets [18], Rabin’s probabilistic automata [21, 20, 4] and theirsupervisory control as studied by Mortzavian [19], fuzzy set theory [17], etc., and it is bettersuited for modeling stochastic qualitative behavior of DESs.

In [7] we defined the set of regular language operators for the probabilistic languages,and also introduced the notion of regularity, i.e., finiteness of automata representation, that

1

is preserved under the operations of regular language operators. We also endowed the set ofprobabilistic languages with a partial order under which it forms a complete partial order[3]: A probabilistic language is bounded above by another one if the occurrence probabilityof each trace in the first is bounded above by that in the second.

Sengupta [25, Chapter 5] studies the problem of optimal control of stochastic behaviorsof DESs. The controller changes the occurrence probabilities of events. A cost is assignedwith each control action, and the control objective is to minimize the cumulative cost over aninfinite horizon. The resulting problem is one of infinite horizon optimal control of Markovprocesses [11].

In contrast, in this paper we study the problem of supervisory control of stochastic be-havior of DESs modeled as probabilistic languages. As above, a supervisor restricts thebehavior of the plant by dynamically disabling certain set of controllable events based onits observations of the executed traces. Thus the occurrence probability of disabled eventsbecomes zero, whereas the occurrence probability of the enabled ones is obtained as con-ditionals given that certain controllable events are disabled, which increases the occurrenceprobabilities of the enabled events proportionately. Hence supervision restricts the support

of the probabilistic language of the plant, but increases the occurrence probabilities of thesurviving traces within this restricted support. A notion of probabilistic supervision, which ismore general than the “non-probabilistic supervision” considered here, was first introducedin [15].

The control objective is specified as a lower and an upper bound constraint. The upperbound constraint imposes a legality constraint specifying that a trace be enabled if and onlyif it is legal. Thus the upper bound constraint can be represented as a non-probabilisticlanguage that maps a trace to the value one if and only if it is legal. The second constraintimposes a level of desirability on the legal traces by specifying a lower bound on theiroccurrence probabilities. This constraint is given as a probabilistic map over the set oftraces, and the control objective is to ensure that each legal trace occurs with probability atleast as much as specified by this lower bound. Summarizing, the upper bound constraintis given as a non-probabilistic language, whereas the lower bound constraint is given as aprobabilistic map, and the control objective is to ensure that the probabilistic language ofthe controlled plant lies within the two bounds. Intuitively, we are interested in designing asupervisor so that “bad” traces never occur, whereas the “good” traces occur with certainminimum probabilities. This generalizes the supervisory control problem studied in the non-stochastic setting where both the upper and lower bounds are non-probabilistic languages.A special case of the control objective, where the upper bound constraint is the same asthe support of the lower bound constraint, was first studied in [15]; however, a more generalprobabilistic supervisor was used to achieve that control objective.

We obtain a necessary and sufficient condition for the existence of the supervisor for theabove control problem as: A supervisor exists if and only if the probabilistic language of thesystem, controlled so that the set of surviving traces is the infimal controllable superlanguageof the support of the lower bound constraint, itself lies within the prescribed bound. Thus totest the existence of a supervisor we need to (i) compute infimal controllable superlanguage

2

of a non-probabilistic language, which is known [14, 13], and (ii) check whether a givenprobabilistic language is bounded above by another one, which we show can be effectivelychecked. The complexity of the later is the same as that of shortest path computation, i.e.,O(n3), where n is the product of the number of states in the automata representations ofthe two probabilistic languages.

Next we study the problem of finding a maximally permissive supervisor for the abovecontrol problem, where a supervisor is said to be more permissive than another one if thefirst one does not disable a trace that is allowed by the second one. We show that unlike inthe non-stochastic case no unique maximally permissive supervisor exists. This is becausethe set of probabilistic languages is not a complete upper semi-lattice [7]. However, sincethe set of controllable non-probabilistic languages is a complete upper semi-lattice [22],and the set of probabilistic languages is a complete partial order [7], non-unique maximallypermissive supervisors do exist. We present an effective algorithm for on-line computation ofa maximally permissive supervisor (refer [9, 2, 8] for other on-line supervisory computationalgorithms). The computational complexity of each step is again O(n3), where n is theproduct of the number of states in the automata representations of the plant and the lowerbound specification.

We also show that a unique minimally permissive supervisor does exist. This is becausethe set of controllable non-probabilistic languages as well as the set of probabilistic languagesis a complete lower semi-lattice. We present an effective algorithm for off-line computation ofthe unique minimally permissive supervisor. The computational complexity of this algorithmis again O(n3).

The rest of the paper is organized as follows. Section 2 gives the notations and prelimi-naries. Section 3 formulates the supervisory control problem, and gives the existence results,whereas Section 4 presents an algorithm for checking existence. Section 5 shows the existenceof a non-unique maximally permissive and a unique minimally permissive supervisor, andgives effective algorithms for their on-line and off-line computation, respectively. Section 6concludes the work presented and identifies some future research directions.

2 Notation and Preliminaries

We use Σ to denote the universe of events over which a given DES evolves. The set Σ∗

is the set of all finite length event sequences, called traces, including the zero length trace,denoted ε. A subset of Σ∗ is called a language. Given traces s and t, we use s ≤ t to denotethat s is a prefix of t, in which case the notation s−1t is used to denote the suffix of t obtainedby removing the prefix s, i.e., t = ss−1t. The notation |t| denotes the length of trace t ∈ Σ∗.Given a language K, we use pr(K), prefix closure of K, to denote the set of all prefixes ofK; K is said to be prefix closed if K = pr(K).

Qualitative behavior of DESs is described by languages. A language L ⊆ Σ∗ can beviewed as a unit interval valued map—a probabilistic map—over Σ∗, L : Σ∗ → [0, 1]. Fora probabilistic map L, its support, denoted supp(L) ⊆ Σ∗, is the set of traces such thatL(s) > 0. L is said to be a non-probabilistic map if L(s) ∈ {0, 1} for each trace s. Clearly,

3

languages can also be represented by non-probabilistic maps. A non-probabilistic map L

models the non-stochastic qualitative behavior of a DES if

L(ε) = 1; ∀s ∈ Σ∗, σ ∈ Σ : L(sσ) = 1 ⇒ L(s) = 1.

This is because a system can always execute the epsilon trace, and if it can execute a trace,then it can also execute all its prefixes. We call such maps non-probabilistic languages ornp-languages.

The notion of probabilistic languages or p-languages was introduced in [7] to model thestochastic qualitative behavior of DESs. A definition of p-languages based on their under-lying probability measure space was presented in [7]. As is discussed in [7] a p-language L

can alternatively be viewed as a probabilistic map satisfying the following constraints:

P1: L(ε) = 1P2: ∀s ∈ Σ∗ :

∑

σ∈Σ L(sσ) ≤ L(s)

Here for each trace s, L(s) gives its probability of occurrence. Condition P1 follows from thefact that a system can always execute the epsilon trace, whereas the condition P2 followsfrom the fact that for any extension of a trace s to be executable, s must itself be executable.Note although we view L a probabilistic map over the set of traces, it actually is a probabilitymeasure over the set of σ-algebra F defined in [7], and we also use it in that mannersometimes.

It follows from the definition of a p-language L that ∆(L) : Σ∗ → [0, 1] defined as:

∀s ∈ Σ∗ : ∆(L)(s) := L(s) −∑

σ∈Σ

L(sσ)

satisfies ∆(L) ≥ 0. ∆(L)(s) gives the probability that the system modeled as p-languageL terminates following the execution of s. It was shown in [7] that

∑

s ∆(L)(s) ≤ 1 withthe equality holding if and only if limk→∞

∑

|t|=k L(t) = 0. In other words, the probabilitythat a system terminates following the execution of arbitrary traces is bounded above byone (a system is not necessarily guaranteed to terminate), and it equals one if and only ifthe probability of traces of arbitrary length converges to zero. We say that a p-language L

is a terminating system if∑

s ∆(L)(s) = 1.Qualitative behavior of DESs can alternatively be represented by automata. An automa-

ton G over the event set Σ is a quadruple, G := (X, Σ, xinit, P ), where X is the set of statesof G, xinit ∈ X is the initial state of G, and P : X × Σ × X → [0, 1] is the state transitionfunction of G. A triple (x, σ, x′) ∈ X × Σ × X is called a transition. G is called a non-

probabilistic automaton or np-automaton if P (x, σ, x′) ∈ {0, 1} for each transition (x, σ′, x);it is said to be a probabilistic automaton or p-automaton [7] if

∀x ∈ X :∑

x′∈X

∑

σ∈Σ

P (x, σ, x′) ≤ 1.

For a p-automaton G, we define

∀x ∈ X : ∆(G)(x) := 1 −∑

x′∈X

∑

σ∈Σ

P (x, σ, x′)

4

to be the probability of termination at state x. An np-automaton (resp., p-automaton) is saidto be a deterministic np-automaton or dnp-automaton (resp., deterministic p-automaton ordp-automaton) if

∀x ∈ X, σ ∈ Σ : |{x′ ∈ X s.t. P (x, σ, x′) > 0}| ≤ 1.

The state transition function of G can be extended to the set of paths X(ΣX)∗, where apath is obtained by concatenating transitions such that the end and start states of consecutivetransitions are the same. Given a path π = x0σ1x1 . . . σnxn ∈ X(ΣX)∗, we use |π| = n todenote its length; for each k ≤ |π|, πk := x0σ1x1 . . . σkxk to denote its initial sub-path oflength k; and tr(π) := σ1 . . . σn to denote its trace. The state transition function is extendedinductively to the set of paths as follows:

∀x ∈ X : P (x) = 1; ∀π ∈ X(ΣX)∗, σ ∈ Σ, x′ ∈ X : P (πσx′) = P (π)P (x|π|, σ, x′).

If G is a np-automaton, then np-language generated by G is given by

[LG(s) := 1] ⇔ [∃π ∈ X(ΣX)∗ : tr(π) = s, P (π) = 1].

If G is a p-automaton, then the p-language generated by G is given by

LG(s) :=∑

π:tr(π)=s,π0=xinit

P (π).

It is easy to see that LG is a np-language when G is a np-automaton, and it was shown in [7]that LG is a p-language when G is a p-automaton. Conversely, given a np-language (resp.,p-language) there exists a deterministic np-automaton (resp., deterministic p-automaton)that generates it [7].

A np-language (resp., p-language) L is said to be regular if there exists a np-automaton(resp., p-automaton) G with finitely many states such that LG = L. A regular np-language(resp., regular p-language) L is called deterministic regular if there exists a dnp-automaton(resp., dp-automaton) with finite states such that LG = L. It is known that the class ofdeterministic regular np-languages is the same as the class of regular np-languages [10],whereas whether or not the class of deterministic regular p-languages is a strict subclass ofthe class of regular p-languages is an open problem.

Given a pair of automata Gi := (X i, Σ, xiinit, P

i), (i = 1, 2), their synchronous composi-tion is another automaton G := (X, Σ, xinit, P ), where X := X1 × X2, xinit := (x1

init, x2init),

and

∀x1, x1 ∈ X1, x2, x2 ∈ X2, σ ∈ Σ : P ((x1, x2), σ, (x1, x2)) := P 1(x1, σ, x1)P 2(x2, σ, x2).

It is easy to see that if Gi’s are deterministic, regular, p-automata, np-automata, respectively,then so is G. Furthermore, supp(LG) = supp(LG1) ∩ supp(LG2).

Given a set X, a partial order on X, denoted �, is a binary relation that is reflexive,antisymmetric, and transitive. The pair (X,�) is called a partially order set or a poset.

5

For a pair of elements x, y ∈ X, their infimum and supremum whenever defined are unique,denoted by x u y and x t y, respectively. A poset (x,�) is said to be a upper (resp., lower)semi-lattice if the supremum (resp., infimum) for any pair of elements in X exists; it is saidto be a complete upper (resp., lower) semi-lattice if supremum (resp., infimum) of any subsetof X exists; it is said to be a (complete) lattice if it is both a (complete) upper and lowersemi-lattice. The infimum (resp., supremum) of X whenever defined is called the bottom(resp., top) element, and is denoted ⊥ (resp., >). A set Y ⊆ X is called a chain if it istotally ordered, in which case Y can be written as a monotonically increasing sequence ofposet elements Y = {xi}i≥0 with xi � xj whenever i ≤ j. A poset (X,�) is called a completepartial order or a cpo if it has the bottom element and every chain has the supremum element.A function f : X → X is called monotone if it preserves ordering under its transformation;it is said to be continuous if it distributes with supremum taken over a chain.

The set of unit interval valued probabilistic maps over Σ∗ forms a poset under the fol-lowing natural ordering relation introduced in [7]:

∀K,L : Σ∗ → [0, 1] : [K � L] ⇔ [∀s ∈ Σ∗ : K(s) ≤ L(s)].

It is easy to see that the set of all non-probabilistic maps is a complete lattice under thisordering. Also, it was shown in [7] that the set of all p-languages forms a cpo as well as acomplete lower semi-lattice under this ordering. The bottom element for the ordering, calledthe nil p-language, denoted I : Σ∗ → [0, 1], is given by

I(ε) = 1; ∀s 6= ε : I(s) = 0.

For supervisory control of the qualitative behavior of a discrete event plant the set ofevents is partitioned into Σu ∪ (Σ − Σu), the sets of uncontrollable and controllable events.For a discrete event plant with behavior modeled by a np-language or a p-language L, asupervisor S with complete observation of traces is a map S : supp(L) → 2Σ−Σu that,following the occurrence of a trace s ∈ supp(L), disables the controllable events in the setS(s) ⊆ Σ − Σu from occurring next. The behavior of the controlled plant is denoted by LS.For a np-language L, the controlled behavior LS is also a np-language defined inductivelyas:

LS(ε) := 1; ∀s ∈ Σ∗, σ ∈ Σ : [LS(sσ) := 1] ⇔ [LS(s) = 1, L(sσ) = 1, σ 6∈ S(s)].

Given a language K ⊆ Σ∗, and a plant with np-language or p-language L, K is saidto be controllable with respect to L if pr(K)Σu ∩ supp(L) ⊆ pr(K). It is known thatthere exists a supervisor S for a plant with np-language L such that supp(LS) = K if andonly if K is nonempty, prefix closed, and controllable [22]. The set of prefix closed andcontrollable languages forms a complete lattice. So the infimal prefix closed and controllable

superlanguage of a language K, denoted infPC(K) [14], and its supremal prefix closed and

controllable sublanguage, denoted supPC(K) [22], exist and are effectively computable.

6

3 Existence of Supervisor for Stochastic DESs

In this section we extend the supervisory control framework to the stochastic setting.Given a DES with stochastic behavior modeled as p-language L, we use (s−1L)(t) to denotethe probability of trace t given that trace s has already occurred:

∀s, t ∈ Σ∗ : (s−1L)(t) := L(st|s) =

{

L(st∧s)L(s)

= L(st)L(s)

if L(s) 6= 0

I(t) otherwise,

where we have used the fact that the outcome that trace st and trace s have occurred isequivalent to the outcome that the trace st has occurred, since occurrence of st impliesoccurrence of the prefix s also. We thus have

L(st) = L(st ∧ s) = L(st|s)L(s) = (s−1L)(t)L(s).

From now on we will assume that L(s) 6= 0 whenever we refer to s−1L. We have thefollowing simple lemma about s−1L.

Lemma 1 Let L be a p-language. Then for each s ∈ Σ∗ we have:

1. ∀t ∈ Σ∗ : ∆(s−1L)(t) = ∆(L)(st)L(s)

.

2. s−1L is a p-language.

Proof: We begin by proving the first part. Pick t ∈ Σ∗. Then

∆(s−1L)(t)

{by definition of ∆}

= (s−1L)(t) −∑

σ

(s−1L)(tσ)

{by definition of s−1L}

=L(st)

L(s)−

∑

σ

L(stσ)

L(s)

{simplifying}

=L(st) −

∑

σ L(stσ)

L(s)

{by definition of ∆}

=∆(L)(st)

L(s),

as desired.To prove the second part we need to establish P1 and P2 for s−1L. By definition,

s−1L(ε) = L(s)L(s)

= 1, i.e., P1 holds. To show P2, it suffices to show that ∆(s−1L) ≥ 0. Thisfollows from the first part since for each t ∈ Σ∗, we have

∆(s−1L)(t) =∆(L)(st)

L(s)≥ 0,

7

where the final inequality follows from the fact that ∆(L), L ≥ 0.

Remark 1 Note that ∆(s−1L)(t) = ∆(L)(st)L(s)

is the conditional probability of terminationfollowing the execution of trace t given that trace s has already occurred.

Also, if L is a deterministic p-language, so that there exists a dp-automaton G :=(X, Σ, xinit, P ) such that LG = L, then for each trace s there exists a unique path πs ∈X(ΣX)∗ such that tr(πs) = s, and L(s) = P (πs). So for a trace s and an event σ we have:

L(sσ) = P (πsσ) = P (πs)P (x|πs|, σ, x|πsσ |) = L(s)P (x|πs|, σ, x|πsσ |).

This implies

P (x|πs|, σ, x|πsσ |) =L(sσ)

L(s)= (s−1L)(σ) (1)

Similarly it can be shown that

∆(G)(x|πs|) = ∆(s−1L)(ε). (2)

As described above, a supervisor S : supp(L) → 2Σ−Σu determines the set of controllableevents S(s) ⊆ Σ−Σu to be disabled following the execution of trace s. In the next lemma weobtain the value of (s−1LS)(σ), the occurrence probability of event σ in the controlled plantgiven that trace s has already occurred. It states that this probability is zero when σ ∈ S(s),and otherwise it equals the corresponding occurrence probability in the uncontrolled plantscaled by an appropriate normalization factor. For notational convenience, given a plantwith p-language L and a supervisor S : supp(L) → 2Σ−Σu , we define a probabilistic mapΦS(L) : Σ∗ → [0, 1]:

∀s ∈ Σ∗ : ΦS(L)(s) := ∆(s−1L)(ε) +∑

σ̂∈Σ−S(s)

(s−1L)(σ̂).

ΦS(L)(s) computes the probability of either termination or execution of an enabled event

given that the trace s has already occurred. The following lemma can be viewed as a specialcase of [15, Equation (1)] when [15, Section IV] is taken into consideration.

Lemma 2 Let L be the p-language of a DES, and S : supp(L) → 2Σ−Σu be a supervisor.Then

∀s ∈ Σ∗, σ ∈ Σ : (s−1LS)(σ) =

{

0 if σ ∈ S(s)(s−1L)(σ)ΦS(L)(s)

if σ ∈ Σ − S(s)

Proof: Pick s ∈ Σ∗ and σ ∈ Σ. If σ ∈ S(s), then it is disabled following the occurrence ofs. So, clearly, (s−1LS)(σ) = 0. Otherwise, when σ ∈ Σ − S(s), we have:

(s−1LS)(σ)

{by definition of supervisor S}

= (s−1L)(σ | σ̂ ∈ S(s) does not occur following s)

8

{rewriting [σ̂ ∈ S(s) does not occur following s]}

= (s−1L)(σ | termination or σ̂ ∈ Σ − S(s) occurs following s)

{definition of conditional: Prob[A|B] =Prob[A ∩ B]

Prob[B]}

=(s−1L)(σ ∧ [termination or σ̂ ∈ Σ − S(s) occurs following s])

(s−1L)(termination or σ̂ ∈ Σ − S(s) occurs following s)

{since σ ∈ Σ − S(s)}

=(s−1L)(σ)

(s−1L)(termination or σ̂ ∈ Σ − S(s) occurs following s)

{rewriting (s−1L)(termination or σ̂ ∈ Σ − S(s) occurs following s)}

=(s−1L)(σ)

ΦS(L)(s)

This completes the proof.Using the result of Lemma 2 we define the p-language of the controlled plant next (that

it is indeed a p-language is established below in Theorem 1):

Definition 1 Let L be a p-language of a DES, and S : supp(L) → 2Σ−Σu be a supervisor.The p-language of the controlled plant, denoted LS, is defined inductively as:

LS(ε) := 1; ∀s ∈ Σ∗, σ ∈ Σ : LS(sσ) := LS(s)(s−1LS)(σ).

Example 1 Consider a plant with Σ = {a, b} and Σu = {b} shown in Figure 1(a). The plant

b(q)

b(q)

a(p)a(p)

a(r)

(a) (b)

b( )

b( )

a(r),b(e)

b(e)

a(r),b(e)

1-re

q1-p

Figure 1: A plant, and the corresponding controlled plant

p-language L is deterministic regular represented by a two state dp-automaton. The tran-sitions are labeled by events along with their occurrence probabilities shown in parenthesis.Consider a supervisor S that disables a after occurrences of a, bb, abb, i.e.,

S(a) = S(bb) = S(abb) = {a}, and S(s) = ∅, otherwise.

The resulting controlled plant is depicted in Figure 1(b). Note from Lemma 2, the transitionprobability of b given that a has already occurred is:

(a−1LS)(b) =(a−1L)(b)

ΦS(L)(a)=

q

(1 − p − q) + q=

q

1 − p.

9

Similarly, the transition probability of b given that bb has already occurred is:

((bb)−1LS)(b) =((bb)−1L)(b)

ΦS(L)(bb)=

e

(1 − r − e) + e=

e

1 − r.

The following corollary states that the effect of the control is to restrict the support, butto increase the occurrence probabilities of the surviving traces within this restricted support.

Corollary 1 Let L be the p-language of a DES, and S : supp(L) → 2Σ−Σu be a supervisor.Then

1. supp(LS) ⊆ supp(L).

2. supp(LS) is nonempty, prefix closed, and controllable.

3. ∀s ∈ supp(LS) : L(s) ≤ LS(s).

Proof: The first part is obvious since by definition S restricts the support of the plant be-havior by dynamically disabling events. The second part follows from the fact that supp(LS)is the language of the controlled plant, which must be nonempty, prefix closed, and control-lable [22]. Finally, the third part follows from induction on the length of traces, and thefact that for each s ∈ supp(LS) and σ ∈ Σ − S(s), (s−1L)(σ) ≤ (s−1LS)(σ), where this finalinequality follows from Lemma 2 and the fact that

ΦS(L)(s) ≤ 1.

The following theorem shows that LS is indeed a p-language.

Theorem 1 Let L be the p-language of a DES, and S : supp(L) → 2Σ−Σu be a supervisor.Then

1. ∀s ∈ Σ∗ : ∆(LS)(s) = LS(s)[

∆(s−1L)(ε)ΦS(L)(s)

]

2. LS in Definition 1 is a p-language.

Proof: We begin with the proof for the first part. Pick s ∈ Σ∗. Then

∆(LS)(s)

{by definition of ∆}

= LS(s) −∑

σ∈Σ

LS(sσ)

{by definition of LS}

= LS(s) −∑

σ∈Σ

LS(s)(s−1LS)(σ)

{rewriting}

10

= LS(s)[1 −∑

σ∈Σ

(s−1LS)(σ)]

{from Lemma 2, (s−1LS)(σ) = 0,∀σ ∈ S(s)}

= LS(s)[1 −∑

σ∈Σ−S(s)

(s−1LS)(σ)]

{applying Lemma 2}

= LS(s)

1 −∑

σ∈Σ−S(s)

(s−1L)(σ)

ΦS(L)(s)

{using definition of ΦS(L)(s) and simplifying}

= LS(s)

[

∆(s−1L)(ε)

ΦS(L)(s)

]

This proves the first assertion.To prove the second part, it suffices to show that LS satisfies P1 and P2. P1 holds from

definition since LS(ε) := 1. To show P2, it suffices to show that ∆(LS) ≥ 0, which followsfrom the first part.

Remark 2 Since from the first part of Lemma 1, ∆(s−1LS)(ε) = ∆(LS)(s)LS(s)

, using the resultof the first part of Theorem 1 it follows

∆(s−1LS)(ε) =∆(s−1L)(ε)

ΦS(L)(s).

This equates the probability of termination given that trace s has already occurred in thecontrolled plant to various parameters of the uncontrolled plant.

For a plant with p-language L we have defined a supervisor S and the resulting controlledplant p-language LS. The control objective is to ensure that the controlled plant p-languagelies within a certain pre-specified range, i.e., given a pair of probabilistic maps K � D, thetask is to design a supervisor such that K � LS � D. Here D is actually a non-probabilisticmap, i.e., D : Σ∗ → {0, 1}, and specifies a legality constraint. D maps the legal traces to one,and the illegal traces to zero. The control objective is to ensure LS � D, i.e., illegal tracesnever occur in the controlled plant implying their occurrence probabilities are zero, whereasno constraint is imposed on the occurrence probabilities of legal traces in the upper boundspecification D. The lower bound K on the other hand specifies the level of desirability oflegal traces by specifying their minimum acceptable occurrence probabilities. The controlobjective is to ensure K � LS, i.e., legal traces in the controlled plant occur with at least asmuch probability as specified by the lower bound constraint K. The existence of S such thatK � LS � D also implies supp(K) ⊆ supp(LS) ⊆ supp(D), which is the supervisory controlproblem studied in the non-stochastic setting [23]. Thus the supervisory control problemformulated here generalizes the one studied in the non-stochastic setting.

In the following theorem we give a necessary and sufficient condition for the existence ofa supervisor for the supervisory control problem described above.

11

Theorem 2 Let L be the p-language of a DES, K be a probabilistic map representing thelower bound constraint, and D be a non-probabilistic map representing the upper boundconstraint. Then there exists a supervisor S : supp(L) → 2Σ−Σu such that K � LS � D ifand only if

K � LS↓

� D,

where S↓ is the supervisor that restricts the support of the plant to infPC(supp(K)). Inthis case S↓ can be used as a supervisor.

Proof: We prove the sufficiency first. Suppose the given condition is satisfied. Select thesupervisor S = S↓. Then from the hypothesis it is a desired supervisor.

Next to see the necessity, suppose there exists a supervisor S such that K � LS � D.We first show that K � LS↓

. From hypothesis that K � LS, we have supp(K) ⊆ supp(LS),i.e., supp(LS) is a superlanguage of supp(K). By the second part of Corollary 1, supp(LS)is prefix closed and controllable. So it must also be a superlanguage of infPC(supp(K)),which is the infimal such language, i.e.,

infPC(supp(K)) = supp(LS↓

) ⊆ supp(LS). (3)

We can view S↓ to be a supervisor for the “plant” with p-language LS. So from the thirdpart of Corollary 1

∀s ∈ supp(LS↓

) : LS(s) ≤ LS↓

(s). (4)

From hypothesis K � LS, so we have from Equation (4)

∀s ∈ supp(LS↓

) : K(s) ≤ LS(s) ≤ LS↓

(s). (5)

From Equation (3) we have supp(K) ⊆ infPC(supp(K)) = supp(LS↓

). This together withEquation (5) implies that

∀s ∈ supp(K) : K(s) ≤ LS↓

(s). (6)

On the other hand, we have

∀s 6∈ supp(K) : 0 = K(s) ≤ LS↓

(s). (7)

Combining Equations (6) and (7) we obtained the first part of necessity condition: K � LS↓

.It remains to show that LS↓

� D. Since D is a non-probabilistic map this is equivalentto showing supp(LS↓

) ⊆ supp(D). From hypothesis we have LS � D, which is equivalentto supp(LS) ⊆ supp(D). This together with Equation (3) yields the desired result thatsupp(LS↓

) ⊆ supp(LS) ⊆ supp(D).In the following remark we compare the existence condition of Theorem 2 with the

corresponding condition in the non-stochastic setting.

Remark 3 Given a plant with p-language L, a probabilistic map lower bound specificationK, and a non-probabilistic map upper bound specification D such that K � D, in thenon-stochastic setting we are interested in designing a supervisor S such that supp(K) ⊆

12

supp(LS) ⊆ supp(D). It can be shown using known results that such a supervisor exists ifand only if

supp(K) ⊆ supp(LS↓

) = infPC(supp(K)) ⊆ supp(D).

Clearly this condition is implied by the condition of Theorem 2. Thus the condition for theexistence of a supervisor in the non-stochastic setting is weaker than that in the stochasticsetting, as expected.

To see that the condition in the non-stochastic setting is strictly weaker consider thefollowing example with Σ = {a, b, c} and Σu = ∅:

L(ε) = 1, L(a) = L(b) = L(c) = 13, and L(s) = 0, otherwise

K(a) = 34, K(b) = 1

4, and K(s) = 0, otherwise

D(c) = 0, and D(s) = 1, otherwise

Thensupp(K) = {a, b} ⊂ supp(D) = {ε, a, b} ⊂ supp(L) = {ε, a, b, c}.

This implies that c must be disabled initially, and a, b must be enabled initially. So there isonly one choice for the supervisor S with S(ε) = {c}. This gives:

LS(ε) = 1, LS(a) = LS(b) =1

2, and LS(s) = 0, otherwise.

So clearly, supp(K) ⊂ supp(LS) = {ε, a, b} = supp(D). However, since K(a) = 34

> 12

=LS(a), K 6� LS � D.

4 Verification of Existence Condition

It follows from Theorem 2 that the existence of a supervisor S for a plant with p-language L and specifications K � D satisfying K � LS � D can be checked by testingwhether K � LS↓

� D, where S↓ is the supervisor that restricts the support of the plantto infPC(supp(K)). We next provide an algorithm for testing this condition. For thiswe assume that all maps L,K, and D are deterministic regular. We first construct a dnp-automaton G such that LG = LS↓

.

Algorithm 1 Given deterministic regular p-languages L,K, let finite state dp-automataGL := (XL, Σ, xL

init, PL), GK := (XK , Σ, xK

init, PK) be such that LGL = L,LGK = K. We

construct a finite state dnp-automaton G := (X, Σ, xinit, P ) such that LG = LS↓

as follows:

1. Obtain dnp-automata GL, GK from GL, GK respectively by replacing each transitionprobability with non-zero value by that of value one, and deleting transitions with zeroprobability value.

13

2. Obtain dnp-automaton G1 such that the support of its generated language equalsinfPC(supp(K)), i.e.,

supp(LG1) = supp(LS↓

) = infPC(supp(K)) = pr(supp(K))Σ∗u ∩ supp(L).

This is done in two steps: first by adding a dump state in GK with self-loops onuncontrollable events and also adding a transition from each state of GK to the dumpstate on each uncontrollable event that is undefined in that state; and next taking thesynchronous composition of the resulting dnp-automaton with GL.

3. Next obtain the dp-automaton G such that LG = LS↓

. This is done by attaching appro-priate probability values to each transition of G1 as follows. Let xL, xK , respectively,denote typical states of GL, and GK augmented with the dump state. Then (xL, xK)denotes a typical state of G1. For each state (xL, xK) of G1, let Σ(xL, xK) ⊆ Σ denotethe set of events that are defined at state (xL, xK) in G1. Finally, since all automataare deterministic we suppress the destination state in any transition by representing itby a “∗”. For each transition ((xL, xK), σ, ∗) of G1 define its probability to be

P ((xL, xK), σ, ∗) :=PL(xL, σ, ∗)

∆(GL)(xL) +∑

σ̂∈Σ(xL,xK) PL(xL, σ̂, ∗), (8)

where P L(·, ·, ·) gives the transition probability of GL.

The following example illustrates the steps of Algorithm 1.

Example 2 Consider the plant of Example 1 with p-language L generated by the dp-automaton depicted in Figure 1(a). Suppose the lower bound specification K is the deter-ministic regular p-language generated by the dp-automaton depicted in Figure 2(a). Then

p

2

2

q

a(p) a(p)a(p)

b(q) b(q)

(a) (b)

b( )

a ( )

b( )1-r

b( )1-rb( )1-r

e

e e

Figure 2: Lower bound specification K, and LS↓

GL and GK are the automata depicted in Figure 1(a) and Figure 2(a), respectively. UsingAlgorithm 1 we obtain the automaton G with LG = LS↓

:

14

1. The dnp-automata GL and GK are the automata GL and GK , respectively, with theprobability labels removed from the transitions.

2. The dnp-automaton G1 that generates infPC(supp(K)) is the automaton shown inFigure 2(b), but without the probability labels on the transitions.

3. The dp-automaton G with p-language LS↓

is shown in Figure 2(b).

The following theorem proves the correctness of Algorithm 1.

Theorem 3 Let L,K,GL, GK , G1, G be as in Algorithm 1. Then LG = LS↓

, where S↓ is asupervisor that restricts the support of the plant with p-language L to infPC(supp(K)).

Proof: First note that from construction

supp(LG1) = supp(LG) = pr(supp(K))Σ∗u ∩ supp(L) = infPC(supp(K)) = supp(LS↓

). (9)

This also implies supp(LG) ⊆ supp(L) = supp(LGL).In light of Equation (9), we only need to show that

∀s ∈ supp(LG) : LG(s) = LS↓

(s).

This we prove by induction on length of s. Since both LG and LS↓

are p-languages, LG(ε) =LS↓

= 1, which establishes the base step. For the induction step, set s = tσ ∈ supp(LG).Then LG(s) = LG(t)(t−1LG)(σ) and LS↓

(s) = LS↓

(t)(t−1LS↓

)(σ). From induction hypothesis,LG(t) = LS↓

(t). So it suffices to show that

(t−1LG)(σ) = (t−1LS↓

)(σ).

Since σ ∈ Σ − S↓(t), from Lemma 2 we have:

(t−1LS↓

)(σ) =(t−1L)(σ)

ΦS↓(L)(t)=

(t−1L)(σ)

∆(t−1L)(ε) +∑

σ̂∈Σ−S↓(t)(t−1L)(σ̂)(10)

Let xL be the state reached by execution of t in GL. Then

Σ − S↓(t) = Σ(xL, xK); ∆(t−1L)(ε) = ∆(GL)(xL); ∀σ ∈ Σ : (t−1L)(σ) = P L(xL, σ, ∗),(11)

where the last two equalities follow from Equations (1) and (2). So combining Equations (10)and (11) we get:

(t−1LS↓

)(σ) =PL(xL, σ, ∗)

∆(GL)(xL) +∑

σ̂∈Σ(xL,xK) PL(xL, σ̂, ∗).

This together with Equation (8) of construction yields

(t−1LS↓

)(σ) = P ((xL, xK), σ, ∗).

15

This gives the desired result since (xL, xK) is the state reached by execution of t in G implyingfrom Equation (1) that P ((xL, xK), σ, ∗) = (t−1LG)(σ).

Next we present an algorithm for testing whether a p-language K is upper bounded byanother p-language L, assuming that both K,L are deterministic regular. This algorithmcombined with Algorithm 1 can be used to test the condition of Theorem 2. We first presentan algorithm to construct a deterministic automaton G such that for each trace s ∈ supp(K),

LG(s) = L(s)K(s)

.

Algorithm 2 Given deterministic regular p-languages L,K, let finite state dp-automataGL := (XL, Σ, xL

init, PL), GK := (XK , Σ, xK

init, PK) be such that LGL = L,LGK = K. We

construct a deterministic automaton G = (X, Σ, xinit, P ) with finite state such that for each

trace s ∈ supp(K), LG(s) = L(s)K(s)

as follows:

1. Define X := XL × XK

2. Define xinit := (xLinit, x

Kinit)

3. For each xL, xL ∈ XL, xK , xK ∈ XK , σ ∈ Σ define

P ((xL, xK), σ, (xL, xK)) :=

P L(xL,σ,xL)

P K(xK ,σ,xK)if P K(xK , σ, xK) 6= 0

∞ otherwise

Then we have the following straightforward theorem. The first part states the correctnessof Algorithm 2, whereas the second part reduces the problem of checking K � L to that ofa shortest path computation.

Theorem 4 Let L,K,GL, GK , G be as in Algorithm 2. Then

1. For each trace s ∈ supp(K), LG(s) = L(s)K(s)

.

2. K � L if and only if mins∈supp(K) LG(s) ≥ 1.

Proof: The first part is straightforward by induction on the length of traces in supp(K)and using the fact that both GL and GK are deterministic.

The second part follows from the fact that K � L if and only if for each s ∈ supp(K),

K(s) ≤ L(s), or equivalently, L(s)K(s)

≥ 1. So the result follows from the first part.

Example 3 Consider the plant of Example 1 with p-language L generated by the dp-automaton of Figure 1(a), and the lower bound specification K of Example 2 generatedby the dp-automaton of Figure 2(a). Suppose the upper bound specification is D = 1, i.e.,all traces are legal. Then from Theorem 2, a supervisor exists if and only if K � LS↓

. Using

Algorithm 2 we obtain the automaton G with LG(s) = LS↓(s)

K(s)for each trace s. G is depicted

in Figure 3. Since all transitions of G are labeled by numbers at least one, it follows thatK � LS↓

.

16

10

10

b( )

b( )

a(1)a(2)

b(2)

b(1)

Figure 3: Automaton G with LG(s) = LS↓(s)

K(s)

The second part of Theorem 4 reduces the problem of verification of K � L to thatof computation of the “least probable path” in G. An algorithm for the shortest pathcomputation can be modified to compute this as follows.

Algorithm 3 Relabel the states of G by numbers 1, . . . , n, where n is the total number ofstates in G. The notation dk[i, j] is used to denote the “least probable path” from state i tostate j, visiting intermediate states with label at most k.

1. Initiation step:

k := 0; ∀i, j ≤ n : d0[i, j] := minσ∈Σ

P (i, σ, j).

2. Iteration step:

k := k+1; ∀i, j ≤ n : dk[i, j] :=

{

min{dk−1[i, j], dk−1[i, k]dk−1[k, j]} if dk−1[k, k] ≥ 10 otherwise

3. Termination step:

If ∀i, j ≤ n : dk−1[i, j] = dk[i, j], then stop; else go to step 2.

Note the iteration step is obtained using the following standard observation: The set ofpaths from i to j visiting states with label at most k is the union of the set of such pathsvisiting states with label at most k − 1, and the set of such paths visiting state k. So theleast probable path may be obtained by taking the minimum over the least probable paths ofthe two sets. Now consider those paths that visit state k. Then a segment of each such pathforms a cycle at state k, and dk−1[k, k] is the least probable cyclic path at k visiting stateswith label at most k−1. Hence if dk−1[k, k] < 1, by executing the cycle an arbitrary numberof times dk[i, j] can be made arbitrarily close to zero. On the other hand if dk−1[k, k] ≥ 1,then the cycle must never be executed in computing dk[i, j].

17

Remark 4 Using a proof similar to the proof of the correctness of shortest path computation[16], it is easily shown that the above iterative computation terminates in at most n iterationsperforming O(n2) computations in each iteration, and upon termination dn[i, j] equals the“least probable” path from the state i to state j. So if the initial state is relabeled as say i,then K � L if and only if minj≤n dn[i, j] ≥ 1. The computational complexity of this test isthus O(n3), where n is the product of the number of states of GL and GK .

5 Maximally Permissive Supervisor

In Theorem 2 we obtained a condition for the existence of a supervisor S for a plantwith p-language L with control specifications K � D, that ensures K � LS � D. We alsomentioned that a supervisor S↓ that restricts the support of the plant to infPC(supp(K))can be used as a supervisor whenever the existence condition is satisfied. However, S↓ neednot be a maximally permissive supervisor. In fact, we show below that it is the minimallypermissive supervisor. In this section we present a technique to construct supervisors thatare maximally permissive.

Definition 2 Given a plant with p-language L, and supervisors S1, S2 : supp(L) → 2Σ−Σu ,S1 is said to be less permissive than S2 (or equivalently, S2 is said to be more permissivethan S1), denoted S1 � S2, if for each s ∈ supp(L), S1(s) ⊇ S2(s), i.e., S1 disables moreevents than S2 following the execution of any trace s. The infimum and supremum of S1

and S2 are defined as follows:

∀s ∈ supp(L) : S1 u S2(s) := S1(s) ∪ S2(s); S1 t S2(s) := S1(s) ∩ S2(s).

It is clear that the set of all supervisors together with the partial order of Definition 2 formsa complete lattice. Also,

supp(LS1uS2) = supp(LS1) ∩ supp(LS2); supp(LS1tS2) = supp(LS1) ∪ supp(LS2).

Given a plant with p-language L and specifications K � D, define the following class ofsupervisors:

S := {S : supp(L) → 2Σ−Σu | K � LS � D}. (12)

S is the class of supervisors which can control the plant to meet the given specifications.The following theorem states a few properties of this class of supervisors.

Theorem 5 Let L be the p-language of a plant with control specifications K � D. Considerthe class of supervisors S as defined in Equation (12). Then

1. (S,�) is a complete lower semi-lattice.

2. If S 6= ∅, then S↓ is the bottom element of (S,�), where S↓ is the supervisor thatrestricts the support of the plant to infPC(supp(K)).

18

3. (S,�) is not a upper semi-lattice.

4. (S,�) is a complete partial order.

Proof: 1. To see the first part, let Λ be an index set such that for each λ ∈ Λ, Sλ ∈ S.Then

∀λ ∈ Λ : K � LSλ � D. (13)

We need to show that K � LuλSλ � D.First note from Equation (13):

∀λ ∈ Λ : supp(K) ⊆ supp(LSλ) ⊆ supp(D),

which impliessupp(K) ⊆ ∩λsupp(LSλ) = supp(LuλSλ) ⊆ supp(D). (14)

Since D is a np-language, this gives us one of the desired inequalities:

LuλSλ � D.

So it suffices to show that

∀s ∈ supp(K) : K(s) ≤ LuλSλ(s). (15)

From Equation (13),

∀λ ∈ Λ, s ∈ supp(K) : K(s) ≤ LSλ(s). (16)

Since for each λ ∈ Λ, uλSλ is more restrictive than Sλ, we can view uλSλ to be a supervisorfor the “plant” with p-language LSλ . So from the third part of Corollary 1,

∀λ ∈ Λ, s ∈ supp(LuλSλ) : LSλ(s) ≤ LuλSλ(s). (17)

Since, from Equation (14), supp(K) ⊆ supp(LuλSλ), Equation (17) gives:

∀λ ∈ Λ, s ∈ supp(K) : LSλ(s) ≤ LuλSλ(s).

This together with Equation (16) gives the desired inequality of Equation (15).

2. For the second part we need to show that whenever S 6= ∅, S↓ = uλ∈ΛSΛ, where theindexing set Λ is as introduced in the proof of the first part. From Theorem 2, S 6= ∅ if andonly if S↓ ∈ S. So from hypothesis S↓ ∈ S. Since from the first part, uλSλ is the bottomelement of (S,�), it follows that uλSλ � S↓.

Thus it suffices to show that S↓ � uλSλ. Since supp(LS↓

) = infPC(supp(K)), this isequivalent to showing that

infPC(supp(K)) ⊆ supp(LuλSλ). (18)

From the first part we have K � LuλSλ � D. This implies supp(LuλSλ) is a prefix closed andcontrollable superlanguage of supp(K), which in turn implies that the desired containmentof Equation (18) holds, and completes the proof.

3. In order to see that (S,�) is not a upper semi-lattice consider the following examplewith Σ = {a, b, c} and Σu = ∅:

19

L(ε) = 1, L(a) = L(b) = L(c) = 13, L(s) = 0, otherwise

K(a) = 12, K(s) = 0, otherwise

D(s) = 1,∀s

Let supervisor S1 be such that S1(ε) = {b}, i.e, it disables b initially. Then

LS1(ε) = 1, LS1(a) = LS1(c) = 12, and LS1(s) = 0, otherwise.

This implies K � LS1 � D. Let supervisor S2 be such that S2(ε) = {c}, i.e, it disables c

initially. Then

LS2(ε) = 1, LS2(a) = LS2(b) = 12, and LS2(s) = 0, otherwise.

This also implies K � LS2 � D. Thus we have S1, S2 ∈ S. Now consider the supervisorS1 t S2. Then

S1 t S2(ε) = S1(ε) ∩ S2(ε) = {b} ∩ {c} = ∅,

i.e., this supervisor does not disable any event. So LS1tS2 = L. Since K(a) = 12

> 13

= L(a),K 6� L = LS1tS2 , which shows that S1 t S2 6∈ S.

4. We proved in the second part that S possesses the bottom element namely, S↓. So weonly need to prove that given a chain {Si | ∀i ≥ 0 : Si ∈ S}, tiSi ∈ S. By hypothesis

∀i ≥ 0 : K � LSi � D,

which implies∀i ≥ 0 : supp(K) ⊆ supp(LSi) ⊆ supp(D),

which in turn implies

supp(K) ⊆ ∪isupp(LSi) = supp(LtiSi) ⊆ supp(D).

Since D is a np-language, this gives us one of the desired inequalities: LtiSi � D.It remains to be shown that K � LtiSi . From hypothesis, for each i ≥ 0, K � LSi � D.

Pick a trace s ∈ Σ∗. We will show that LtiSi(s) := limi→∞ LSi(s) exists and satisfiesK(s) ≤ LtiSi(s). First consider the case when there exists an integer is ≥ 0 such thatLSis (s) > 0 (such an integer exists for only s ∈ supp(D)). Then since {Si}i≥0 is a chainwith monotonically increasing permissiveness, it follows from the part 3 of Corollary 1 that{LSi(s)}i≥is is a monotonically decreasing sequence. By hypothesis, the numbers in thissequence are bounded below by K(s), and hence their limit LtiSi(s) exists and satisfiesK(s) ≤ LtiSi(s). Note that LtiSi(s) is also the limit of the sequence {LSi(s)}i≥0 obtained byadding a finite number of numbers to the sequence {LSi(s)}i≥is . On the other hand, if thereexists no is ≥ 0 such that LSis (s) > 0 (note that this can only hold for s 6∈ supp(K), sincefor s ∈ supp(K) we have is = 0), then LtiSi(s) = 0. Thus LtiSi(s) exists for each s ∈ Σ∗

and satisfies K(s) ≤ LtiSi(s).

20

The first part of Theorem 5 shows that a unique minimally permissive supervisor exists,whereas the second part of the theorem shows that the supervisor given by Theorem 2 is ac-tually the unique minimally permissive supervisor. The third part of Theorem 5 shows thatno unique maximally permissive supervisor exists, but the fourth part of the theorem showsthat non-unique maximally permissive supervisors do exist. So we next present an algorithmfor the on-line computation of such a supervisor assuming that all maps L,K,D are deter-ministic regular. For each trace t ∈ supp(L), we define S

↓t : supp(t−1L) → 2Σ−Σu to be the

supervisor for plant with p-language (t−1L) that restricts its support to infPC(supp(t−1K)).

Algorithm 4 Consider a plant with deterministic regular p-language L, a lower boundspecification K which is also a deterministic regular p-language, and an upper bound spec-ification D which is a deterministic regular np-language. Assume that K � LS↓

� D,i.e., a supervisor ensuring the given specifications exists. Obtain a maximal supervisorSmax : supp(L) → 2Σ−Σu as follows.

For each s ∈ supp(L), define Smax(s) := Σ − Σmax(s), where Σmax(s) ⊆ supp(s−1D) ∩ Σis a maximal set such that

∀t = sσ ∈ sΣmax(s) :K(t)

LSmax(t)(t−1K) � (t−1L)S

↓t � (t−1D), (19)

where

LSmax

(t) := LSmax

(s)(s−1LSmax

)(σ) = LSmax

(s)(s−1L)(σ)

ΦSmax(L)(s).

Algorithm 4 computes Σmax(s), the set of events to be enabled by Smax following theexecution of each trace s as follows. Smax enables events in Σmax(s) if (i) after the occurrenceof any event σ ∈ Σmax(s) it is possible to control the plant in future so that the givenspecifications are satisfied, which is captured by Equation (19), and (ii) Σmax(s) is a maximalset of events satisfying the upper bound constraint (recall Σmax(s) ⊆ supp(s−1(D)) ∩ Σ)having such a property.

It should be noted that Σmax(s) is defined as a “fixed point”, and its definition shouldnot be taken to be a circular one. To determine Σmax(s), one guesses an initial value forthis event set and verifies whether Equation (19) holds for it. If it does hold (resp., does nothold), then initial guess is replaced by a larger (resp., smaller) event set, and the verificationstep is repeated till no larger (resp., smaller) event set can be found.

Note that in Equation (19) the scaling factor K(t)LSmax

(t)has been used to take into account

the effect of the probability of the trace t when determining control beyond t. As an example,given that t has already occurred, an event σ can occur with a probability lower thant−1K(σ), since this single step may be outweighed in the overall probability of tσ due to a

greater probability of t occurring in LSmax

(this is precisely what the scaling factor K(t)LSmax

(t)

determines).The next theorem states the correctness of Algorithm 4. We first prove a lemma.

21

Lemma 3 Given a plant with p-language L and specifications K � D, let S be a supervisorsuch that K � LS � D, i.e., S ∈ S. Then

∀t = sσ ∈ s(Σ − S(s)) :K(t)

LS(t)(t−1K) � (t−1L)S

↓t � (t−1D) (20)

Proof: Pick t = sσ ∈ s(Σ − S(s)) and u ∈ Σ∗. Then from hypothesis that S ∈ S we have:

K(tu) = K(t)(t−1K)(u) � LS(tu) = LS(t)(t−1LS)(u) � D(tu) = D(t)(t−1D)(u).

Since u ∈ Σ∗ is arbitrary, this implies

K(t)

LS(t)(t−1K) � (t−1LS) �

D(t)

LS(t)(t−1D).

Since σ is enabled after s by S, it follows that LS(sσ) = LS(t) ≤ 1. By hypothesis, LS(t) ≤

D(t) ∈ {0, 1}. So we must have D(t) = 1, and hence D(t)LS(t)

≥ 1. So the last inequality isequivalent to

K(t)

LS(t)(t−1K) � (t−1LS) � (t−1D). (21)

Note K(t)LS(t)

(t−1K) is a p-map (by hypothesis K(t) ≤ LS(t)), (t−1D) is a np-language, and

supp( K(t)LS(t)

(t−1K)) = supp(t−1K). Thus Equation (21) implies the existence of a supervisor

for the plant (t−1L) such that the controlled plant satisfies the specifications K(t)LS(t)

(t−1K) �

(t−1D). So it follows from the necessity part of Theorem 2 that

K(t)

LS(t)(t−1K) � (t−1L)S

↓t � (t−1D),

as desired.

Theorem 6 Let L,K,D, Smax be as in Algorithm 4. Then Smax is a maximal supervisorsuch that K � LSmax

� D.

Proof: From the hypothesis that K � LS↓

� D (refer to Algorithm 4) we know that asupervisor exists. So from the fourth part of Theorem 5, a maximal supervisor also exists.We need to show that Smax is such a supervisor.

We first show using induction on the length of traces that Smax ∈ S, i.e., K � LSmax

� D.The base step follows from the hypothesis that K � LS↓

� D, which implies

K(ε) ≤ LS↓

(ε) = 1 = LSmax

(ε) ≤ D(ε).

For the induction step consider a trace s and an event σ such that t := sσ ∈ supp(L). Weneed to show that

K(t) ≤ LSmax

(t) ≤ D(t),

22

or equivalently,K(t)

LSmax(t)≤ 1, and LSmax

(t) ≤ D(t).

The first part follows from construction by evaluating Equation (19) at ε, since

(t−1K)(ε) = (t−1L)S↓t (ε) = 1.

Using t = sσ, the second part can be rewritten as

LSmax

(s)[(s−1LSmax

)(σ)] ≤ D(s)[(s−1D)(σ)].

This follows from the induction hypothesis, which gives LSmax

(s) ≤ D(s), and the construc-tion, which gives supp(s−1LSmax

) ∩ Σ = Σmax(s) ⊆ supp(s−1D) ∩ Σ.Next suppose for contradiction that Smax as obtained in Algorithm 4 is not maxi-

mal. Then there exists another supervisor S ∈ S such that Smax � S, or equivalently,supp(LSmax

) ⊆ supp(LS). This implies that there exists a trace in supp(LS) − supp(LSmax

).Let t be such a minimal length trace. Since ε ∈ supp(LSmax

), t 6= ε. So t = sσ, wheres ∈ supp(LSmax

) and σ ∈ Σ. Since t is a minimal trace in supp(LS) − supp(LSmax

), we have

∀u < s : S(u) = Smax(u); S(s) ⊂ Smax(s).

This implies

∀t = sσ ∈ s(Σ − S(s)),∀u < t : LS(u) = LSmax

(u); LS(t) ≤ LSmax

(t).

By multiplying both sides of the last inequality by K(t)(t−1K) and rearranging we get

∀t = sσ ∈ s(Σ − S(s)) :K(t)

LSmax(t)(t−1K) �

K(t)

LS(t)(t−1K). (22)

Since S ∈ S, from Lemma 3, Equation (20) holds. This together with Equation (22) gives:

∀t = sσ ∈ s(Σ − S(s)) :K(t)

LSmax(t)(t−1K) �

K(t)

LS(t)(t−1K) � (t−1L)S

↓t � (t−1D).

By construction Σmax(s) is a maximal set such that Equation (19) holds. So we musthave Σ − S(s) ⊆ Σmax(s) for at least one such maximal sets, implying Smax(s) ⊆ S(s), acontradiction.

Remark 5 In Algorithm 4 the computation of Smax(s) at each trace s requires testing thecondition of Equation (19) for all possible set of events Σmax(s). Ignoring the dependence ofcomputational complexity on the size of the set Σ, we see that the computational complexityat each step of Algorithm 4 is O(n3), where n is the product of the number of states in theautomata representations of the plant and the lower bound specification.

23

Remark 6 Algorithm 4 can be used to derive some intuition regarding the off-line com-putation of a maximally permissive supervisor. It follows that given two traces s and t, amaximally permissive supervisor Smax exists that takes identical control actions in futurewhenever

1. ∀u : [D(su) = D(tu)],

2. ∀u : [L(su)L(s)

= L(tu)L(t)

],

3. ∀u : [K(su)K(s)

= K(tu)K(t)

], and

4. K(s)LSmax

(s)= K(t)

LSmax(t)

.

In other words, the pair of traces s and t are equivalent in the maximally permissive super-vised behavior whenever they are equivalent according to all the above four relations. Thefirst three relations have finite cardinality whenever D,L,K are all regular. Determining thecardinality of the final relation remains open.

In the following remark we compare the maximally permissive supervisor of Algorithm 4with the maximally permissive supervisor in the non-stochastic setting.

Remark 7 In the non-stochastic setting, we are interested in designing a supervisor S

such that for a plant with p-language L, a probabilistic map lower bound specification K,and a non-probabilistic map upper bound specification D, satisfying K � D, we obtainsupp(K) ⊆ supp(LS) ⊆ supp(D).

It can be shown that whenever such a supervisor exists, there exists a unique maximallypermissive supervisor Ssup, called supremely permissive supervisor, which can be computedas follows. For each s ∈ supp(L), define Ssup(s) := Σ − Σsup(s), where Σsup(s) ⊆ Σ is thesupremal set such that

∀t = sσ ∈ sΣsup(s) : supp(t−1K) ⊆ supp((t−1L)S↓t ) ⊆ supp(t−1D). (23)

In other words, Ssup enables an event following the occurrence of s if it is possible to controlthe plant in the future so that the specifications of the non-stochastic setting are satisfied.Since supp[ K(t)

LSmax(s) (t−1K)] = supp(t−1K), it follows from Equation (19) that

∀t = sσ ∈ sΣmax(s) : supp(t−1K) ⊆ supp((t−1L)S↓t ) ⊆ supp(t−1D).

This together with the definition of Ssup given in Equation (23) implies that Σmax(s) ⊆Σsup(s), i.e., any maximally permissive supervisor in the stochastic setting is less permissivethan the maximally permissive supervisor in the non-stochastic setting, as expected.

The following example with Σ = {a, b, c} and Σu = ∅ illustrates that the converse doesnot hold in general.

24

L(ε) = 1, L(a) = L(b) = L(c) = 13, and L(s) = 0, otherwise

K(a) = K(b) = 12, and K(s) = 0, otherwise

D(s) = 1,∀s

Then it is clear that the supremely permissive supervisor in the non-stochastic setting dis-ables no events initially, whereas the maximally permissive supervisor in the stochastic set-ting disables the event c initially.

Example 4 Let L,K,D be as in Example 3. Then since supp(K) ⊂ supp(L), and sinceall traces are legal under the specification D, in the non-stochastic setting the supremelypermissive supervisor Ssup disables no events, i.e., LSsup

= L.However, in the stochastic setting, it may not hold that

Ssup ∈ S = {S : supp(L) → 2Σ−Σu | K � LS � D}.

To see this suppose r > 12, which implies 2(1 − r) < 1. Then

K(bb) =qe

2(1 − r)> qe = L(bb) = LSsup

(bb),

which implies Ssup 6∈ S. From Example 3 we know that S 6= ∅. Any supervisor S ∈ S mustbe such that K(bb) ≤ LS(bb), which is possible only if S disables a following the occurrenceof b, i.e., S(b) = {a}. (Note that if S(b) = {a}, then

LS(bb) = LS(b)(b−1L)(b)

ΦS(L)(b)= q

e

(1 − r − e) + e=

qe

1 − r>

qe

2(1 − r)= K(bb).)

From Example 2 we know that the minimally permissive supervisor S↓ disables a followingthe occurrence of b, implying that S↓ is the only supervisor in S, i.e., it is also the maximallypermissive supervisor.

6 Conclusion

In this paper we have formalized the supervisory control of stochastic qualitative behaviorof DESs. It generalizes the supervisory control formalism of the non-stochastic setting ina natural way. The control objective in the stochastic setting is to design a supervisor sothat the controlled plant only generates legal traces (specified as a non-probabilistic map),and that the traces it generates occur with certain minimum probabilities (specified as aprobabilistic map). We have shown that the computational complexity of the test for theexistence of a supervisor, and also that for the on-line computation of a maximally permissivesupervisor (in the case when the languages involved are deterministic regular) are both O(n3),where n is the product of the number of states in the automata representations of the plantand the lower bound specification.

25

The supervisory control formalization presented here can be extended in many ways.Firstly, the upper bound constraint may be more general, specified as a probabilistic map.Secondly, the control objective may be different, such as: in the controlled plant only legalstates can be visited, and that the probability of visiting each legal state exceeds a certainminimum probability. Finally, an off-line computation of a maximally permissive supervisorremains an interesting open problem.

References

[1] C. G. Cassandras and S. Lafortune. Introduction to Discrete Event Systems. KluwerAcademic Publishers, Norwell, MA, 1995.

[2] S. L. Chung, S. Lafortune, and F. Lin. Supervisory control using variable lookaheadpolicies. Discrete Event Dynamic Systems: Theory and Applications, 4(3):237–268, July1994.

[3] B. A. Davey and H. A. Priestley. Introduction to Lattices and Order. CambridgeUniversity Press, Cambridge, UK, 1990.

[4] E. Doberkat. Stochastic Automata: Stability, Nondeterminism and Prediction, volume113 of Lecture Notes in Computer Science. Springer-Verlag, New York, 1981.

[5] V. K. Garg. An algebraic approach to modeling probabilistic discrete event systems.In Proceedings of 1992 IEEE Conference on Decision and Control, pages 2348–2353,Tucson, AZ, December 1992.

[6] V. K. Garg. Probabilistic languages for modeling of deds. In Proceedings of Conference

on Information Sciences and Systems, pages 198–203, Princeton, NJ, March 1992.

[7] V. K. Garg, R. Kumar, and S. I. Marcus. A probabilistic language formalism forstochastic discrete event systems. IEEE Transactions on Automatic Control, 44(2):280–293, 1999.

[8] N. B. Hadj-Alouane, S. Lafortune, and F. Lin. Centralized and distributed algorithm foron-line synthesis of maximal control policies under partial observation. Discrete Event

Dynamical Systems: Theory and Applications, 6(41):379–427, 1996.

[9] M. Heymann and F. Lin. On-line control of partially observed discrete event systems.Discrete Event Dynamical Systems: Theory and Applications, 4(3):221–236, July 1994.

[10] J. E. Hopcroft and J. D. Ullman. Introduction to Automata Theory, Languages and

Computation. Addison-Wesley, Reading, MA, 1979.

[11] P. R. Kumar and P. Varaiya. Stochastic Systems: Estimation, identification and adaptive

control. Prentice Hall, 1986.

26

[12] R. Kumar and V. K. Garg. Modeling and Control of Logical Discrete Event Systems.Kluwer Academic Publishers, Boston, MA, 1995.

[13] R. Kumar and M. A. Shayman. Formulae relating controllability, observability, andco-observability. Automatica, 34(2):211–215, 1998.

[14] S. Lafortune and E. Chen. On the infimal closed and controllable superlanguage of agiven language. IEEE Transactions on Automatic Control, 35(4):398–404, 1990.

[15] M. Lawford and W. M. Wonham. Supervisory control of probabilistic discrete eventsystems. In Proceedings of 36th Midwest Symposium on Circuits and Systems, pages327–331, 1993.

[16] E. Lawler. Combinatorial Optimization - Networks and Matroids. Holt Rinehart andWinston, 1976.

[17] E. T. Lee and L. A. Zadeh. Note on fuzzy languages. Information Sciences, pages421–434, 1969.

[18] M. K. Molloy. Performance analysis using stochastic Petri nets. IEEE Transactions on

Computers, C-31(9):913–917, September 1982.

[19] H. Mortzavian. Controlled stochastic languages. In Proceedings of 1993 Allerton Con-

ference, pages 938–947, Urbana, IL, 1993.

[20] A. Paz. Introduction to Probabilistic Automata. Academic Press, New York, 1971.

[21] M. O. Rabin. Probabilistic automata. Information and Control, 6:230–245, 1963.

[22] P. J. Ramadge and W. M. Wonham. Supervisory control of a class of discrete eventprocesses. SIAM Journal of Control and Optimization, 25(1):206–230, 1987.

[23] P. J. Ramadge and W. M. Wonham. The control of discrete event systems. Proceedings

of IEEE: Special Issue on Discrete Event Systems, 77:81–98, 1989.

[24] A. Salomaa. Formal languages and power series. In J. v. Leeuwen, editor, Handbook of

theoretical computer science. MIT Press, Cambridge, MA, 1994.

[25] R. Sengupta. Optimal control of discrete event systems. PhD thesis, Department ofElectrical Engineering and Computer Science, University of Michigan at Ann Arbor,1995.

27