on ergodicity, infinite flow, and consensus in random models

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 7, JULY 2011 1593

On Ergodicity, Infinite Flow, and Consensusin Random Models

Behrouz Touri, Student Member, IEEE, and Angelia Nedic, Member, IEEE

Abstract—We consider the ergodicity and consensus problem fora discrete-time linear dynamic model driven by random stochasticmatrices, which is equivalent to studying these concepts for theproduct of such matrices. Our focus is on the model where therandom matrices have independent but time-variant distribution.We introduce a new phenomenon, the infinite flow, and we studyits fundamental properties and relations with the ergodicity andconsensus. The central result is the infinite flow theorem estab-lishing the equivalence between the infinite flow and the ergodicityfor a class of independent random models, where the matrices inthe model have a common steady state in expectation and a feed-back property. For such models, this result demonstrates that theexpected infinite flow is both necessary and sufficient for the er-godicity. The result is providing a deterministic characterizationof the ergodicity, which can be used for studying the consensus andaverage consensus over random graphs.

Index Terms—Ergodicity, infinite flow, linear random model,product of random matrices, random consensus.

I. INTRODUCTION

T HERE is evidence of a growing number of applicationsin decentralized control of networked agents, as well as

social and other networks where the consensus is used as amechanism for decentralized coordination of agent actions. Thefocus of this paper is on a canonical consensus problem for alinear discrete-time dynamic system driven by a general modelof random matrices, where the matrices are row-stochastic. In-vestigating whether the model reaches a consensus or not isoften done by exploring the conditions that ensure the ergod-icity, which in turn always guarantees the consensus.

In this paper, we propose an alternative approach by intro-ducing a concept of the model with infinite flow property, whichcan be interpreted as infinite information flow over time betweena group of agents and the other agents in the network. We showthat the infinite flow property is closely related to the ergodicityand, hence, to the consensus. In particular, we show the equiva-lence between the infinite flow and the ergodicity for a class ofindependent random models.

We start by a comprehensive study of the fundamental rela-tions and properties of the ergodicity, consensus, and infinite

Manuscript received January 06, 2010; revised May 29, 2010; accepted Oc-tober 29, 2010. Date of publication November 09, 2010; date of current versionJuly 07, 2011. This work was supported by the National Science Foundationunder CAREER Grant CMMI 07-42538. Recommended by Associate EditorC.-H. Chen.

The authors are with the Department of Industrial and Enterprise SystemsEngineering, University of Illinois at Urbana–Champaign, Urbana, IL 61801USA (e-mail: [email protected]; [email protected]).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TAC.2010.2091174

flow for general random models, independent models, and inde-pendent identically distributed (i.i.d.) random models. We theninvestigate the random models with a feedback property andthe models with a common steady state for the expected ma-trices. Both of these properties have been used in the analysis ofconsensus models, but a deeper understanding of their roles hasnot been observed. We classify feedback property in three basictypes from weak to strong and show some relations for them.Then, we study the models with a common steady state in ex-pectation. By putting all the pieces together, we show that theergodicity of the model is equivalent to the infinite flow prop-erty for a class of independent random models with feedbackproperty and a common steady state in expectation, as given ininfinite flow theorem (Theorem 7). The infinite flow theoremalso establishes the equivalence between the infinite flow prop-erties of the model and the expected model. Furthermore, thetheorem also shows the equivalence between the ergodicity ofthe model and the ergodicity of the expected model. As such,the theorem provides a novel deterministic characterization ofthe ergodicity, thus rendering another tool for studying the con-sensus over random networks and convergence of random con-sensus algorithms.

The main contributions of this paper include: 1) the equiva-lence of the ergodicity of the model and the expected model fora class of independent random models with a feedback propertyand a common steady state in expectation; 2) the new insightsand understanding of the ergodicity and consensus events overrandom networks brought to light through a new phenomenaof infinite flow event, which to the best of our knowledge hasnot been known prior to this work; 3) novel comprehensivestudy of the fundamental properties of the consensus and er-godicity events for general class of independent random models;4) new insights into the role of feedback property and the roleof a common steady state in expectation for the ergodicity andconsensus.

The study of the random product of stochastic matrices datesback to the early work in [1], where the convergence of theproduct of i.i.d. random stochastic matrices was studied usingthe algebraic and topological structures of the set of stochasticmatrices. This work was further extended in [2]–[4] by usingresults from ergodic theory of stationary processes and their al-gebraic properties. In [5], the ergodicity and consensus of theproduct of i.i.d. random stochastic matrices was studied usingtools from linear algebra and probability theory, and a neces-sary and sufficient condition for the ergodicity was establishedfor a class of i.i.d. models. Independently, the same problem wastackled in [6], where an exponential convergence bound was es-tablished. Recently, the work in [5] was extended to ergodic sta-tionary processes in [7].

0018-9286/$26.00 © 2010 IEEE

1594 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 7, JULY 2011

In all of the works [1]–[7], the underlying random modelsare assumed to be either i.i.d. or stationary processes, both ofwhich imply time-invariant distribution on the random model.Unlike these works, our work in this paper is focused on theindependent random models with time-variant distributions.Furthermore, we study ergodicity and consensus for suchmodels using martingale and supermartingale convergenceresults combined with the basic tools from probability theory.Our work is also related to the consensus over random networks[8], optimization over random networks [9], and the consensusover a network with random link failures [10]. Also related aregossip and broadcast-gossip schemes giving rise to a randomconsensus over a given connected bidirectional communicationnetwork [11]–[14]. On a broader basis, the paper is related tothe literature on the consensus over networks with noisy links[15]–[18] and the deterministic consensus in decentralizedsystems models [19]–[27] including the effects of quantizationand delay [14], [28]–[32].

The paper is organized as follows. In Section II, we describe adiscrete-time random linear dynamic system of our interest andintroduce the ergodicity, consensus, and infinite flow events. InSection III, we explore the relations among these events and es-tablish their 0–1 law and other properties by considering gen-eral, independent, and i.i.d. random models. In Section IV, wediscuss models with feedback properties and provide classifi-cation of such properties with insights into their relations. Wealso consider an independent random model with a commonsteady state in expectation. In Section V, we focus on indepen-dent random models with infinite flow property. We establishnecessary and sufficient conditions for ergodicity and brieflydiscuss some implications of these conditions. We conclude inSection VI

Notation and Basic Terminology: We view all vectors ascolumns. For a vector , we write to denote its th entry,and we write to denote that all its entries arenonnegative (positive). We use to denote the transpose of avector . We write to denote the standard Euclidean vectornorm i.e., . We use to denote the vectorwith the th entry equal to 1 and all other entries equal to 0,and use to denote the vector with all entries equal to 1. Avector is stochastic when and . We write

or to denote a sequence ofsome elements, and we write to denote the truncatedsequence for . For a set and a subsetof , we write to denote that is a proper subset of

. A set such that is referred to as a nontrivialsubset of . We write to denote the integer set .For a set , we let denote the complement set ofwith respect to , i.e., .

We denote the identity matrix by . For a vector , we useto denote the diagonal matrix with diagonal entries

being the components of the vector . For a matrix , wewrite to denote its th entry, to denote its th columnvector, and to denote its transpose. For an matrix ,we use to denote the summation of the entriesover all , with . A matrix is row-stochasticwhen its entries are nonnegative and . Since we dealexclusively with row-stochastic matrices, we will refer to suchmatrices simply as stochastic. We let denote the set of

stochastic matrices. A matrix is doubly stochastic when bothand are stochastic.

We write to denote the expected value of a randomvariable . We use and to denote the probability

and the characteristic function of an event , respectively. If, we say that happens almost surely. We often

abbreviate “almost surely” by

II. PROBLEM FORMULATION AND TERMINOLOGY

Throughout this paper, we deal exclusively with the matricesin the set of stochastic matrices. We consider thetopology induced by the open sets in with respect to theEuclidean norm and the Borel sigma-algebra of thistopology. We assume that we are given a probability space

and a measurable function .To every , the function is assigning a discrete timeprocess in the countable product measurable space

, where is the random matrix

of the process at time . We refer to the processinterchangeably as a random chain or a random model and,when suitable, we suppress the explicit dependence on thevariable . We say that the chain is independent if thesigma algebras generated by the ’s for different areindependent. If in addition ’s are identically distributed,then the model is i.i.d.

With a given random chain , we associate a lineardiscrete-time dynamic system of the following form:

for (1)

where is a state vector at time and is theinitial state vector. We will often refer to the system in (1) as thedynamic system driven by the chain .

We are interested in providing conditions guaranteeing thatthe dynamic system reaches a consensus almost surely. Sincereaching the consensus is closely related to the ergodicity ofthe chain, we are also interested in studying the ergodicity onthe fundamental level. In our study of the random consensusand ergodicity, we use another property of the chain, an infiniteflow property. We start by providing these basic notions for adeterministic chain.

Definition 1: Given a deterministic chain , wesay the following.

— The system reaches a consensus iffor any initial state , there exists a scalarsuch that .

— The chain is ergodic if for any and ,there is a scalar such that

for all

where for and.

— The chain has infinite flow property iffor any

nontrivial subset .In the definition of the infinite flow property, the quantity

can be interpreted as a flow be-tween the subset and its complement in a weighted graph.

TOURI AND NEDIC: ON ERGODICITY, INFINITE FLOW, AND CONSENSUS IN RANDOM MODELS 1595

In particular, consider the undirected weighted graph withthe node set , the edge set induced by the positive entries in

, and the weight matrix . Then,the quantity represents the flow ingraph across the cut for a nontrivial node set

and its complement . For the graphs induced by thematrices , the infinite flow property requires that the totalflow in time across any nontrivial cut is infinite, whichcould be viewed as infinite information exchange between thenodes in and .

The ergodicity is equivalent to the following condition [33]:For any and , there is a scalar such that

. Since the matrices have fi-nite dimension, the ergodicity is also equivalent to the followingcondition: For any and any , there is a scalarsuch that . Also, due to the linearityand finite dimension of the system , theconsensus can be studied by considering only the initial states

, , rather than all .Clearly, the ergodicity of the chain implies reaching a con-

sensus. However, a consensus may be reached even if thechain is not ergodic, as seen in the following example.

Example 1: Let for a stochastic vector , andlet for all . Then, we havefor all , implying that the systemreaches a consensus. However, the chain is not ergodicsince for any .

Using Definition 1, we now introduce the correspondingevents of consensus, ergodicity, and infinite flow. Given arandom chain , let denote the event that the systemin (1) reaches a consensus for any initial state . Letdenote the event that the chain is ergodic, and letdenote the event that the chain has the infinite flow property. Werefer to , , and as the consensus event, the ergodicityevent, and the infinite flow event, respectively. We say that themodel is ergodic if the ergodicity event occurs almost surely.The model admits consensus if the consensus event occursalmost surely. The model has infinite flow if the infinite flowevent occurs almost surely. The model has expected infiniteflow if its expected chain has infinite flow.

III. INFINITE FLOW, ERGODICITY, AND CONSENSUS

In this section, we further study the ergodicity event ,the consensus event , and the infinite flow event underdifferent assumptions on the nature of the randomness in themodel. In particular, in Section III-A we establish some fun-damental relations among , , and . In Section III-B,we investigate the 0–1 law properties of these events, while inSection III-C we provide some relations for a random modeland its expected model.

Throughout the rest of the paper, we use the following nota-tion. For a stochastic matrix and a nontrivial subset ,we define as follows:

(2)

Note that for a given a random model , the infinite flowevent is given by

(3)

Note also that the infinite flow event requires that, across anynontrivial cut , the total flow in the random graphs inducedby the matrices , , is infinite.

A. Basic Relations

As discussed in Section II, we have for any randommodel. We show here that the ergodicity event is also alwayscontained in the infinite flow event, i.e., . We establishthis by using the following result for a deterministic model.

Lemma 1: Let be a deterministic sequence,and let be generated by for all

with an initial state . Then, for any nontrivialsubset and , we have

where for .Proof: Let be an arbitrary nontrivial set, and let

be arbitrary. Let and. Since , by the

stochasticity of we have forall and all . Then, we obtain for

where the inequality follows by . By the stochas-ticity of , we also obtain

By the definition of in (2), we have. Since , it follows

where the last inequality holds since. By taking the maximum over

all in the preceding relation and by recursively usingthe resulting inequality, we obtain


Fig. 1. Relations among the ergodicity, consensus, and infinite flow events fora general random model.

and, recursively, we get.

The relation for follows from the precedingrelation by considering generated with the starting point

.Using Lemma 1, we now show that the ergodicity event is

contained in the infinite flow event.Theorem 1: Let be an ergodic deterministic

chain. Then, for any nontrivial . In

particular, we have for any random model.Proof: To arrive at a contradiction, assume that there is

a nontrivial set such that . Sincethe matrices are stochastic, we have forall . Therefore, there exists large enough such that

.Now, define the vector , where

for and for . Consider the dynamicsystem for , which is started attime in state . Note that Lemma 1 applies to thecase where the time is taken as initial time, in whichcase corresponds to .Also, note that by the definition of the startingstate . Thus, by applying Lemma 1, we have for all

,and .Since and , itfollows that and

. Using these relations and, we have

for any and , thusshowing that the chain is not ergodic—a contradiction.Therefore, we must have for any nontrivial

.From the preceding argument and the definitions of and, we conclude that implies for any random

model . Hence, for any random model.Theorem 1 shows that an ergodic model must have an infinite

flow property. In other words, the infinite flow property of anyrandom model is necessary for the ergodicity of the model. Laterin Theorem 6, for a certain class of random models, we willshow that the infinite flow is also sufficient for the ergodicity.

Fig. 1 illustrates the inclusions and for ageneral random model. The inclusion in Fig. 1can be strict, as seen in the following example.

Example 2: Consider the 2 2 chain defined by

for

For any stochastic matrix , we have , and hencethe model admits consensus. Furthermore, the model has infiniteflow property. However, the chain does not admit

consensus. Therefore, in this case, (the entirespace of realizations), while .

Next, we provide a sufficient condition for the ergodicity andconsensus events to coincide.

Lemma 2: Let be a (not necessarily independent)random chain such that each matrix is invertible almostsurely. Then, we have almost surely.

Proof: The inclusion follows from the definition,so it suffices to show almost surely. In turn, to show

almost surely, it suffices to prove thatfor a set such that . For each , let be theset of instances such that the matrix is invertible.Define . We have for eachby our assumption that each matrix is invertible almostsurely. In view of this and the fact that the collection iscountable, it follows that .

We next show that . Let so thathas full rank for all . To simplify notation, let

. Consider an arbitrary starting time .We show that the consensus is reached for the dynamic

with , i.e., for any , we havefor some . For a given ,

define and considerthe dynamic started at timewith the initial vector . By the definition of , we have

. Therefore, forall

By the definition of , we have for some(since ). Therefore, it follows that

, thus showing that the dynamic system, , reaches a consensus. Since this is true for arbitrary

and , the chain is ergodic, whichimplies .

In general, there may be no further refinements of inclusionrelations among the events , , and even when the modelis independent, as indicated by the following example.

Example 3: Consider an independent random model where,for , we have with probability ,

with probability , and with prob-ability 1 for all . In this case, the consensus eventhappens with probability . However, the infinite flow andthe ergodicity events are empty sets.

Example 3 shows that we can have and, while . Thus, even for an independent model,

the consensus event need not be contained in either or .However, if we further restrict our attention to i.i.d. models, wecan show that almost surely. To establish this, we makeuse of the following lemma.

Lemma 3: Let and . Also, let be such thatfor any ,


where denotes the th component of a vector . Then, wehave for any .

Proof: Let with for any . Then,we have for any ,

By the assumption on , we obtain. Hence,

, implying .

We now provide our main result for i.i.d. models, whichstates that the ergodicity and the consensus events are almostsurely equal. We establish this result by using Lemma 3 andthe Borel–Cantelli lemma (see [34, p. 50]).

Theorem 2: We have almost surely for any i.i.d.random model.

Proof: Since , the assertion is true when consensusoccurs with probability 0. Therefore, it suffices to show that ifthe consensus occurs with a probability other than 0, the twoevents are almost surely equal. Let with .Then, for all

where

and is the sequence generated by the dynamicsystem (1) with any .

For every , let be the sequence generatedby the dynamic system in (1) with . Then, for any

, there is the smallest integer such that

for all

Note that is a nonincreasing sequence (of ) foreach . Hence, by letting , weobtain for all and .Thus, by applying Lemma 3, we have for almost all

(4)

for all and . By the definition of con-sensus, we have . Thus,by the continuity of the measure, there exists an integer suchthat .

Now, let time be arbitrary, and let denote the-tuple of the matrices driving the system (1) for

and , i.e.,

for all

Let denote the collection of all -tuplesof matrices , such that for

with , we have. By the definitions of and , re-

lation (4) and relation state that

Fig. 2. Ergodicity and consensus coincide ��, and all three events assume0–1 law for an i.i.d model.

. By the i.i.d. property of themodel, the events , , are i.i.d., and theprobability of their occurrence is equal to ,implying that for all .Consequently, . Sincethe events are i.i.d., by the Borel–Cantellilemma, , where

stands for infinitely often. Observing that the eventis contained in the consensus

event for the chain , we see that the consensusevent for the chain occurs almost surely.Since this is true for arbitrary , it follows that the chain

is ergodic , implying This and theinclusion yield

Theorem 2 extends the equivalence result betweenthe consensus and ergodicity for i.i.d. models given in[5, Theorems 3.a and 3.b] (and hence [5, Corollary 4]),which are established there assuming that the matrices havepositive diagonal entries almost surely. The relations among

, , and for the i.i.d. case are illustrated in Fig. 2.

B. 0–1 Laws

In this section, we discuss 0–1 laws for the events , ,and for independent random models. The 0–1 laws specifythe trivial (or 0–1) events, which are the events occurring witheither probability 0 or 1. The ergodicity event is a 0–1 event, asshown1 in [5, Lemma 1]. Since the ergodicity event is alwayscontained in the consensus event, the ergodicity event occurswith probability 0 whenever the consensus event occurs witha probability . In other words, we may have

only if .We next show that the infinite flow is also a 0–1 event.Lemma 4: For an independent random model, the infinite

flow event is a 0–1 event.Proof: For a nontrivial , the sequence

of undirected flows across the cut [see (2)] is a se-quence of independent (finitely valued) random variables.The event is a tale event and, byKolmogorov’s 0–1 law [34, p. 61], this event is a 0–1 event.Since there are finitely many nontrivial sets , the event

is also a 0–1 event.

While both events and are trivial for an independentmodel, the situation is not the same for the consensus event .In particular, by Example 3 where , we see that the

1Even though the result there was stated assuming a more restrictive randommodel, the proof itself relies only on the independence property of the model.


consensus event need not assume 0–1 law since it can occur witha probability .

However, the situation is very different for i.i.d. models. Inparticular, in this case the consensus event is also a trivial event,as seen in the following lemma.

Lemma 5: For an i.i.d. random model, the consensus eventis a 0–1 event.

Proof: The result follows from the fact that is a trivialevent and Theorem 2, which states that almost surelyfor i.i.d. models.

Fig. 2 illustrates the 0–1 laws of , , and for an i.i.d.model. Our next example demonstrates that the inclusion rela-tion in Fig. 2 can be strict.

Example 4: Consider the independent identical randommodel where each is equally likely to be any of the

permutation matrices. Then, in view of the uniformdistribution, we have for all . Hence,by Theorem 3, it follows that the infinite flow event ishappening almost surely. However, since the chain isa sequence of permutation matrices, the consensus eventnever happens.

C. Random Model and Its Expected Model

Here, we investigate the properties of an independent randommodel and its corresponding expected model. We establish tworesults in forthcoming Theorems 3 and 4 that later on play animportant role in the establishment the Infinite Flow Theorem inSection V. The first result shows the equivalence of the infiniteflow property for a random chain and its expected chain

, as given in the following theorem.Theorem 3: Let be an independent random model.

Then, the model has infinite flow property if and only if theexpected model has infinite flow property.

Proof: Let be nontrivial. Since the model is in-dependent, the random variables are independent. Bythe definition of and the stochasticity of , we have

for all . Thus,by monotone convergence theorem [34, p. 225], the infinite flowproperty of the model implies the infinite flow property of theexpected model. On the other hand, since the model is inde-pendent and , by Kolmogorov’s three-seriestheorem [34, p. 64], it follows that if ,then . Since is atrivial event, we have . Thus, since

is arbitrary, the model has infinite flow property.There is no analog of Theorem 3 for the ergodicity or con-

sensus event, unless additional assumptions are imposed. How-ever, a weaker result holds as seen in the following.

Lemma 6: Let be an independent model, and assumethat the model admits consensus (is ergodic). Then, the expectedmodel reaches a consensus (is ergodic). The proof2

of Lemma 6 can be found in [5] and [6].The following theorem states another important result for

later use. As a consequence of Lemma 6 and Theorems 1 and 3,the result provides an equivalent deterministic characterizationof the ergodicity for a class of independent models.

2Assuming more restrictive assumptions on the model, the result of the lemmawas stated in [5] (Theorem 3, �� ) and [6] (Remark 3.3). However, theproofs there rely only on the independence of the model.

Theorem 4: Let be an independent random modelsuch that almost surely. Then, the model is ergodic ifand only if the expected model is ergodic.

Proof: If the ergodicity event is almost sure, then byLemma 6, the expected model is ergodic. For the conversestatement, let the chain be ergodic. Then, byTheorem 1, the chain has infinite flow. Therefore,by Theorem 3, the infinite flow event is almost sure, andsince a.s., the ergodicity event is almost sure.

IV. MODEL WITH FEEDBACK PROPERTY AND STEADY STATE

IN EXPECTATION

In this section, we discuss two properties of a random modelthat are important in the development of our main results inSection V. In particular, we introduce and study a model withfeedback properties and a model with a common steady statein expectation. Stronger forms of these properties have alwaysbeen used when establishing consensus both for deterministicand random models. Here, we provide some new fundamentalinsights into these properties.

A. Feedback Properties

We define several types of feedback property. Recall thatdenotes the th column vector of a matrix .

Definition 2: A random model has strong feedbackproperty if there exists such that

a.s. for all and all

The model has feedback property if there exists such that

for all , and all , with . The model has weakfeedback property if there exists such that

for all , and all , with . The scalar isreferred to as a feedback constant.

While the difference between feedback and strong feedbackproperty is apparent, the difference between weak feedback andfeedback property may not be so obvious. The following ex-ample illustrates the difference between these concepts.

Example 5: Consider the static deterministic chain

for

Since and for all , the model does nothave feedback property. At the same time, sincefor , it follows .Thus, has weak feedback property with .

It can be seen that strong feedback property implies feedbackproperty, which in turn implies weak feedback property. The de-terministic consensus and averaging models in [19], [21], [26],and [29] require that the matrices have nonzero diagonal entriesand uniformly bounded nonzero entries, which is more restric-tive than the strong feedback property.

We next show that the feedback property of a random modelimplies the strong feedback property of its expected model.


Lemma 7: Let a random model have feedback prop-erty with constant . Then, its expected model hasstrong feedback property with .

Proof: Let the model have feedback property with a con-stant . Then, by the definition of the feedback property, we have

for any and ,with . Since , it follows that

for all , , . The matrices are stochastic,so we have . Hence, for every and

, there exists an index (the dependence on andis suppressed) such that . If , thenwe are done. Otherwise, we have

for all . Hence, the expected chain has the strongfeedback property with constant .

We now focus on an independent model. We have the fol-lowing result.

Lemma 8: Consider an independent model . Supposethat the model is such that there is an with the followingproperty: For all and , with

or the following property: For all and , with

Then, respectively, the model has feedback property with con-stant or weak feedback property with constant .

Proof: We prove only the case of feedback propertysince the other case uses the same line of argument. If

for some and , with , then therelation is satisfied trivially(with both sides equal to zero). If , then by theassumption of the lemma, we have .Furthermore, since for all , , and , it followsthat , thus showing thatthe model has feedback property with constant .

The i.i.d. models with almost surely positive diag-onal entries have been studied in [5]–[7]. Such modelshave feedback property as seen in the following corollary.

Corollary 1: If is an i.i.d. model with almost surelypositive diagonal entries, then the model has feedback propertywith constant

Proof: Let for some , . Sinceand , we have

. Define . Sincethe model is i.i.d., the constant is independent of time. Hence,by Lemma 8, it follows that the model has feedback propertywith constant .

B. Steady State in Expectation

Here, we consider a model with another special property.Specifically, we discuss a random model such that itsexpected chain has a common steady state.

Definition 3: A random model has a common steady statein expectation if there is a stochastic vector such that

for all .For example, the matrices that are doubly stochastic in

expectation satisfy the preceding definition with ,such as the matrices arising in a randomized broadcast or gossipover a connected (static) network [11], [13].

Consider the function given by

for (5)

The function measures the weighted spread of the vectorentries with respect to the weighted average value .

We at first study the behavior of the weighted averagesalong the sequence . The main observation

is that the random scalar sequence is a boundedmartingale, which leads us to the following result.

Lemma 9: Let be an independent random modelwith a common steady state in expectation. Then, the se-quence converges almost surely for any .

Proof: By the model independency and ,it follows that the process is a martingale with respectto the natural filtration of the process for any initial .Since the matrices are stochastic, the sequenceis bounded. Thus, is a bounded martingale. By themartingale convergence theorem (see [35, Theorem 35.5]), thesequence converges a.s.

We next characterize the limit of the martingale .Let be the limit of the martingale for theinitial state , and let be the vector defined by

with for (6)

In the following lemma, we provide some properties of therandom vector .

Lemma 10: Let be an independent model witha common steady state in expectation. Then, the randomvector has the following properties.

(a) a.s. for any .(b) is a stochastic vector.(c) .(d) For every , the limit points of the

sequence lie in the random hyperplanealmost surely.

Proof: By Lemma 9 and the definition of , we havealmost surely for initial state . Using

the linearity of the system in (1), we obtain for any

thus showing part (a).Since , by part (a) it fol-

lows . The matricesand the vector have nonnegative entries implying that thevector also has nonnegative entries. By letting


and using the stochasticity of , we have forall , implying for all . Thus, we have

, where the second equalityholds by part (a). Hence, is a stochastic vector.

To show part (c), we note that by the martingale property ofthe process , we have for all

and . By the boundedness of the martingale,we have for any

. The preceding two relations imply .For part (d), we note that the sequence is bounded

for every by the stochasticity of ; thus, it hasaccumulation points. By part (a), each accumulation point ofthe sequence satisfies a.s.

We now focus on the sequence . We show that itis a convergent supermartingale, which indicates that is astochastic Lyapunov function for the random system in (1).

Theorem 5: Let the random model be independentwith a common steady state in expectation. Then, we almostsurely have for all

(7)

where . Furthermore,converges almost surely.

Proof: By using , from the definition of thefunction in (5), we have

where the second equality is obtained by using ,, and . In view of , it

follows that for all

(8)

Since the model is independent, by taking the expectationconditioned on , we obtain

almost surely for all. Since , we further have

where the inequality follows by Jensen’s inequality (see[34, p. 225]) and the convexity of the function . Theexpected matrices have the same steady state ,implying that almost surely for all

By combining the preceding two relations, we see that almostsurely for all

By adding and subtracting to the right-hand sideof the preceding relation and using (8), we obtain almost surely

for all (9)

Now, we show that. By the definition of , we

have , so that for

Since is stochastic, we have, implying that

where the last equality follows from . Since, the preceding relation yields

(10)

Therefore, for any , we have, which can be further written as

where the last equality follows from relation (10). The matrixis symmetric, so that

implying thatTherefore, we have


By combining the preceding relation with (9), we conclude thatrelation (7) holds a.s. for all .

Since each is a stochastic matrix and is stochasticvector, the matrix has nonnegative entries for all . Hence,from the preceding relation it follows that is a super-martingale. The convergence of follows straightfor-wardly from the nonnegative supermartingale convergence (see[34, (2.11) Corollary, p. 236]).

We conclude this section with another result for the weighteddistance function . This result plays crucial role in estab-lishing our result in Section V-B.

Lemma 11: Let be a stochastic vector, and letbe such that . Then, we have

Proof: We establish two relations

(11)

(12)

Observe that the desired result follows from (11) and (12).We now show relation (11). We have for all . Sinceis stochastic, we also have . Thus,

. By writing, we obtain

where the last inequality holds by the convexity of the function. Using and the preceding relation,

we obtain relation (11).To prove relation (12), we write

as , which is equalto with given by

To estimate , we consider scalars and and letand . Then, by Hölder’s inequality with

, , we have , where isthe -norm. Hence

(13)

By using (13) with and fordifferent indices and , , we obtain

which completes the proof.

V. MODEL WITH INFINITE FLOW PROPERTY

We consider an independent random model with infinite flow.We show that this property, together with weak feedback and acommon steady state in expectation, is necessary and sufficientfor almost sure ergodicity. Moreover, we establish that the er-godicity of the model is equivalent to the ergodicity of the ex-pected model.

A. Preliminary Result

We now provide an important relation that we use later inSection V-B.

Lemma 12: Let andfor all and some . Let be a permutation ofthe index set corresponding to the nondecreasing orderingof the entries , i.e., is a permutation on such that

. Also, let be such that

for every (14)

where is arbitrary. Then, we have

Proof: Relation (14) holds for any nontrivial set .Hence, without loss of generality we may assume that the per-mutation is identity (otherwise we will relabel the indices ofthe entries in and update the matrices accordingly). Thus,we have . For each , let

and define time as follows:

Since the entries of are nondecreasing, we havefor all

. Thus, by relation (14), the time ex-ists, and for each .


We next estimate for all and any time. For this, we introduce for

and the index sets as follows:

Let for some . Since , we haveand . Thus, by Lemma 1, we have for any

Furthermore, andsince and . Thus,

it follows

By the definition of time , we havefor .

Hence, by using this and the definition of , for any, we have

(15)

(16)

Now, suppose that for someand . By choosing in (15) and in(16), and by letting , we obtain

Since for all , we have, which

combined with the preceding relation yields. Using

and , we further have

(17)

By (17), it follows that

where the last inequality holds by . In the last term inthe preceding relation, the coefficient of is equal to

. Furthermore, by the definition of , we haveonly when . Therefore

Summing these relations over , we obtain

where the last inequality follows by exchanging theorder of summation. By the definition of and using

, we have, implying

B. Sufficient Conditions for Ergodicity

We now establish one of our main results for indepen-dent random models with infinite flow property and having acommon vector in expectation and weak feedback property.

Let , and for any , let

(18)where , are arbitrary. Define

for

(19)Since the model has infinite flow property, the infinite flowevent occurs a.s. Therefore, the time is finite for all .

We next show that either the infinite flow or expected infiniteflow property is sufficient for the ergodicity of the model.

Theorem 6: (Sufficient Ergodicity Condition): Letbe an independent random model with a common steady state

in expectation and weak feedback property. Also, let themodel have either infinite flow or expected infinite flow prop-erty. Then, the model is ergodic. In particular,


almost surely for all , where is the randomvector of (6).

Proof: Assume that the model has infinite flow property.Let be arbitrary initial state. Let us denote the(random) ordering of the entries of the vector by forall . Thus, at time , we have .

Now, let be arbitrary and fixed, and considerthe set in (19). By the definition of , we have

for any and . Thus,

by Lemma 12, we obtain for any

where and the last inequality followsby Lemma 11. We can compactly write the inequality as

(20)

where is the indicator function of the event .Observe that and are independent since the model

is independent. Therefore, by Theorem 5, we have

with and . Let, and note that by . Thus

Therefore

Furthermore, by using relation (20), we obtain

where the last inequality follows by and the factthat and are independent (since depends on

information prior to time and the set relies on informa-tion at time and later). Hence, it follows

Therefore, for arbitrary , we have

implying that . In view of the non-negativity of , by the monotone convergence theorem

[35, Theorem 16.6], it follows ,

implying According to Theorem 5,the sequence is convergent, which together with thepreceding relation implies that

To show the ergodicity of the model, we note that by the con-vergence result for the martingale in Lemma 10(a),we have a.s., where the randomvector is given by (6). Now, usingand the fact that all norms in are equivalent, we obtain

for all , thusshowing the ergodicity of the model.

Assume now that has expected infinite flow, i.e.,for any nontrivial . By

Theorem 3, has infinite flow property if and onlyif it has expected infinite flow, and the result follows by thepreceding case.

C. Necessary and Sufficient Conditions for Ergodicity

Here, we provide the central result of this paper. The resultestablishes necessary and sufficient conditions for ergodicity ofrandom models with weak feedback property and a commonsteady state in expectation. The conditions are relianton infinite flow and guarantee that the ergodicity of the modelis equivalent to the ergodicity of the expected model. The resultemerges as an outcome of several important results that we havedeveloped so far. In particular, we combine the resultstating that the ergodicity event is always contained in the infi-nite flow event (Theorem 1), the deterministic characterizationof the infinite flow of Theorem 3, and the sufficient conditionsof Theorem 6. We also make use of Theorem 4 providing con-ditions for equivalence of the ergodicity of the chain and theexpected chain.

Theorem 7: (Infinite Flow Theorem): Let the randommodel be independent and have a common steadystate in expectation and weak feedback property. Then,the following conditions are equivalent.

(a) The model is ergodic.(b) The model has infinite flow property.(c) The expected model has infinite flow property.(d) The expected model is ergodic.

Proof: First, we establish that parts (a)–(c) are equivalentby showing that (a) (b) (c) (a). In particular, byTheorem 1 we have , showing that (a) (b). ByTheorem 3, parts (b) and (c) are equivalent. By Theorem 6,part (c) implies part (a). Now, we prove (a) (d). Since


(a) (b), we have a.s. Hence, by Theorem 4, theparts (a) and (d) are equivalent.

The infinite flow theorem combined with the deterministiccharacterization of the infinite flow model of Theorem 3 leadsus to the following result.

Corollary 2: Let be a deterministic modelthat has a common steady-state vector and weak feed-back property. Then, the chain is ergodic if and only if

for every nontrivial .Under the conditions of Theorem 7, the model admits con-

sensus, which follows directly from relation .Corollary 3: Let the assumptions of Theorem 7 hold. Then,

the model admits consensus.The infinite flow theorem establishes the equivalence between

the ergodicity of a chain and the expected chain for a class ofindependent random models. The central role in this result isplayed by the infinite flow and its equivalent deterministic char-acterization. Another crucial result is the interplay between theergodicity and infinite flow of Theorem 4 yielding the equiva-lence between ergodicity of the chain and the expected chain.The following two examples are provided to illustrate somestraightforward applications of the infinite flow theorem.

1) i.i.d. Models: Consider an i.i.d. model . Then,the expected matrix is independent of . Since

is stochastic, we have for a stochastic vector. Therefore, an i.i.d. model is an independent model with

a common steady state in expectation.In [5], it is shown that for the class of i.i.d. models that have

a.s. positive diagonal entries, the ergodicity of the expectedmodel and the ergodicity of the original model are equivalent.The application of this result is reliant on the condition of thea.s. positive diagonal entries, which implies that the modelhas feedback property, as shown in Corollary 1. This property,however, is stronger than weak feedback property. At the sametime, no requirement on the steady-state vector is needed.

The application of the infinite flow theorem to the i.i.d. casewould require weak feedback property and the existence of asteady-state vector . Thus, there is a tradeoff in the con-ditions for the ergodicity provided by the infinite flow theoremand those given in [5]. To further illustrate the difference inthe conditions, we consider the homogeneous deterministicmodel of Example 5. The model has weakfeedback property and the steady-state vector , sothe ergodicity of the model can be deduced from the infiniteflow theorem. At the same time, as seen in Example 5, themodel does not have positive diagonal entries and, therefore,the ergodicity of the model cannot be deduced from the resultsin [5] and [7]. In light of this, the infinite flow theorem providesconditions for ergodicity that complement the conditions of [5]and [7].

2) Gossip Algorithms on Time-Varying Networks: As an-other application of the infinite flow theorem, we consider anextension of the standard gossip algorithm to time-varying net-works. In particular, the gossip algorithm originally proposedin [11] and [36] is for static networks. Here, we give a suffi-cient condition for the convergence of a gossip algorithm fornetworks with time-changing topology. Consider a network of

agents viewed as nodes of a graph with the node set .Suppose that each agent has a private scalar value at time

. Now, let the interactions of the agents be random at non-negative integer-valued time instances as follows: At any time

, two different agents , wake up with probability, where and . Then,

they set their values to the average of their current values, i.e.,. The choices of

the pairs of interacting agents at different time instancesare independent.

Based on the agent interaction model, define the independentrandom model by

with prob. (21)

Then, the dynamic system (1) driven by the randomchain describes the evolution of the vectorthat has its th component value equal to agent value, .As seen from (21), any realization of the model isdoubly stochastic. Hence, the model has a common steadystate in expectation. Also, the model has strongfeedback property (with ).

Lemma 13: For extended gossip algorithm (21), the con-sensus is almost sure if for any nontrivialset .

Proof: The extended gossip algorithm satisfies the assump-tion of the infinite flow Theorem 7. Since

for all and , it follows that. Thus, by the infinite flow theorem, the model

admits consensus if for any .When for all as in [11] and [36], we can consider

the graph , where the edge if and onlyif . In this case, it can be seen that the conditionof Lemma 13 is equivalent to the requirement that the graph isconnected. One can further modify the algorithm in (21) to allowfor time-varying weights, i.e.,

andwith . Such a scheme is a natural generalization ofthe symmetric gossip model proposed in [6]. In this case, it canbe verified that if for , then theresult of Lemma 13 still holds.

VI. CONCLUSION

We have studied the ergodicity and consensus problem for alinear discrete-time dynamic model driven by random stochasticmatrices. We have introduced a concept of the infinite flow eventand studied the relations among this event, ergodicity event, andconsensus event. The central result is the infinite flow theoremproviding necessary and sufficient conditions for ergodicity ofindependent random models. The theorem captures the condi-tions ensuring the convergence of the random consensus algo-rithms, such as gossip and broadcast schemes [11]–[13]. More-over, the infinite flow theorem simultaneously captures the con-ditions on the connectivity of the system and the sufficient in-formation flow over time that have been important in studyingthe consensus and average consensus in deterministic settings[19], [20], [24], [26], [29], [37], [38]. As illustrated briefly intwo examples, the infinite flow theorem provides a convenienttool for studying the ergodicity of a model as well as consensusalgorithms. Finally, we note that the work in this paper is readily


extendible to the case when the initial state in (1) is itselfrandom and independent of the chain .

VII. ACKNOWLEDGEMENT

The authors are grateful to the anonymous referees for theirvaluable comments and suggestions that improved the paper.

REFERENCES

[1] M. Rosenblatt, “Products of independent identically distributed sto-chastic matrices,” J. Math. Anal. Appl., vol. 11, no. 1, pp. 1–10, 1965.

[2] K. Nawrotzki, “Discrete open systems on Markov chains in a randomenvironment. I,” Elektron. Inf. Kybernetik, vol. 17, pp. 569–599, 1981.

[3] K. Nawrotzki, “Discrete open systems on markov chains in a randomenvironment. II,” Elektron. Inf. Kybernetik, vol. 18, pp. 83–98, 1982.

[4] R. Cogburn, “On products of random stochastic matrices,” in RandomMatrices and Their Applications. Providence, RI: Amer. Math. Soc.,1986, pp. 199–213.

[5] A. Tahbaz-Salehi and A. Jadbabaie, “A necessary and sufficient condi-tion for consensus over random networks,” IEEE Trans. Autom. Con-trol, vol. 53, no. 3, pp. 791–795, Apr. 2008.

[6] F. Fagnani and S. Zampieri, “Randomized consensus algorithms overlarge scale networks,” IEEE J. Sel. Areas Commun., vol. 26, no. 4, pp.634–649, May 2008.

[7] A. Tahbaz-Salehi and A. Jadbabaie, “Consensus over ergodic sta-tionary graph processes,” IEEE Trans. Autom. Control, vol. 55, no. 1,pp. 225–230, Jan. 2010.

[8] Y. Hatano, A. Das, and M. Mesbahi, “Agreement in presence of noise:Pseudogradients on random geometric networks,” in Proc. 44th IEEEConf. Decision Control & Eur. Control Conf., 2005, pp. 6382–6387.

[9] I. Lobel and A. Ozdaglar, “Distributed subgradient methods overrandom networks,” MIT, Cambridge, MA, Report 2800, 2008.

[10] S. Patterson, B. Bamieh, and A. E. Abbadi, “Distributed average con-sensus with stochastic communication failures,” in Proc. 46th IEEEConf. Decision Control, 2007, pp. 4215–4220.

[11] S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah, “Randomized gossipalgorithms,” IEEE Trans. Inf. Theory, vol. 52, no. 6, pp. 2508–2530,Jun. 2006.

[12] A. Dimakis, A. Sarwate, and M. Wainwright, “Geographic gossip: Ef-ficient averaging for sensor networks,” IEEE Trans. Signal Process.,vol. 56, no. 3, pp. 1205–1216, Mar. 2008.

[13] T. Aysal, M. Yildriz, A. Sarwate, and A. Scaglione, “Broadcast gossipalgorithms for consensus,” IEEE Trans. Signal Process., vol. 57, no. 7,pp. 2748–2761, Jul. 2009.

[14] P. F. R. Carli, F. Fagnani, and S. Zampieri, “Gossip consensus algo-rithms via quantized communication,” Automatica, vol. 46, no. 1, pp.70–80, 2010.

[15] M. Huang and J. Manton, “Stochastic approximation for consensusseeking: Mean square and almost sure convergence,” in Proc. 46thIEEE Conf. Decision Control, 2007, pp. 306–311.

[16] M. Huang and J. Manton, “Coordination and consensus of networkedagents with noisy measurements: Stochastic algorithms and asymptoticbehavior,” SIAM J. Control Optimiz., vol. 48, no. 1, pp. 134–161, 2009.

[17] S. Kar and J. Moura, “Distributed consensus algorithms in sensor net-works: Link failures and channel noise,” IEEE Trans. Signal Process.,vol. 57, no. 1, pp. 355–369, Jan. 2009.

[18] B. Touri and A. Nedic, “Distributed consensus over network with noisylinks,” in Proc. 12th Int. Conf. Inf. Fusion, 2009, pp. 146–154.

[19] J. Tsitsiklis, “Problems in decentralized decision making and compu-tation,” Ph.D. dissertation, Dept. Elect. Eng. Comput. Sci., MIT, Cam-bridge, MA, 1984.

[20] J. Tsitsiklis and M. Athans, “Convergence and asymptotic agreementin distributed decision problems,” IEEE Trans. Autom. Control, vol.AC-29, no. 1, pp. 42–50, Jan. 1984.

[21] J. Tsitsiklis, D. Bertsekas, and M. Athans, “Distributed asynchronousdeterministic and stochastic gradient optimization algorithms,” IEEETrans. Autom. Control, vol. AC-31, no. 9, pp. 803–812, Sep. 1986.

[22] D. Bertsekas and J. Tsitsiklis, Parallel and Distributed Computation:Numerical Methods. Upper Saddle River, NJ: Prentice-Hall, 1989.

[23] S. Li and T. Basar, “Asymptotic agreement and convergence of asyn-chronous stochastic algorithms,” IEEE Trans. Autom. Control, vol.AC-32, no. 7, pp. 612–618, Jul. 1987.

[24] A. Jadbabaie, J. Lin, and S. Morse, “Coordination of groups of mobileautonomous agents using nearest neighbor rules,” IEEE Trans. Autom.Control, vol. 48, no. 6, pp. 988–1001, Jun. 2003.

[25] R. Olfati-Saber and R. Murray, “Consensus problems in networks ofagents with switching topology and time-delays,” IEEE Trans. Autom.Control, vol. 49, no. 9, pp. 1520–1533, Sep. 2004.

[26] A. Olshevsky and J. Tsitsiklis, “Convergence speed in distributed con-sensus and averaging,” SIAM J. Control Optimiz., vol. 48, no. 1, pp.33–55, 2008.

[27] W. Ren and R. Beard, “Consensus seeking in multiagent systemsunder dynamically changing interaction topologies,” IEEE Trans.Autom. Control, vol. 50, no. 5, pp. 655–661, May 2005.

[28] A. Kashyap, T. Basar, and R. Srikant, “Quantized consensus,” Auto-matica, vol. 43, no. 7, pp. 1192–1203, 2007.

[29] A. Nedic, A. Olshevsky, A. Ozdaglar, and J. Tsitsiklis, “On distributedaveraging algorithms and quantization effects,” IEEE Trans. Autom.Control, vol. 54, no. 11, pp. 2506–2517, Nov. 2009.

[30] R. Carli, F. Fagnani, A. Speranzon, and S. Zampieri, “Communicationconstraints in the average consensus problem,” Automatica, vol. 44, no.3, pp. 671–684, 2008.

[31] R. Carli, F. Fagnani, P. Frasca, T. Taylor, and S. Zampieri, “Averageconsensus on networks with transmission noise or quantization,” inProc. IEEE Amer. Control Conf., 2007, pp. 4189–4194.

[32] P. Bliman, A. Nedic, and A. Ozdaglar, “Rate of convergence for con-sensus with delays,” in Proc. 47th IEEE Conf. Decision Control, 2008,pp. 4849–4854.

[33] S. Chatterjee and E. Seneta, “Towards consensus: Some convergencetheorems on repeated averaging,” J. Appl. Probab., vol. 14, no. 1, pp.89–97, Mar. 1977.

[34] R. Durrett, Probability: Theory and Examples, 3rd ed. Belmont, CA:Curt Hinrichs, 2005.

[35] P. Billingsley, Probability and Measure. New York: Wiley, 1995.[36] S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah, “Gossip algorithms:

Design, analysis, and applications,” in Proc. IEEE INFOCOM, 2005,pp. 1653–1664.

[37] A. Nedic and A. Ozdaglar, “On the rate of convergence of distributedsubgradient methods for multi-agent optimization,” in Proc. IEEECDC, 2007, pp. 4711–4716.

[38] A. Nedic and A. Ozdaglar, “Distributed subgradient methods for multi-agent optimization,” IEEE Trans. Autom. Control, vol. 54, no. 1, pp.48–61, Jan. 2009.

Behrouz Touri (S’10) received the B.S. degree inelectrical engineering from Isfahan University ofTechnology, Isfahan, Iran, in 2006, and the M.S.degree in communications, systems, and electronicsfrom Jacobs University, Bremen, Germany, in 2008,and is currently pursuing the Ph.D. degree in indus-trial and enterprise systems engineering, Universityof Illinois at Urbana–Champaign.

His research interests include random dynamics,decentralized optimization, control and estimation,coding theory, and optimization theory.

Mr. Touri is a member of the Phi Kappa Phi Honor Society.

Angelia Nedic (M’06) received the B.S. degree inmathematics from the University of Montenegro,Podgorica, Montenegro, in 1987, the M.S. degreein mathematics from the University of Belgrade,Belgrade, Serbia, in 1990, the Ph.D. degree in math-ematics and mathematical physics from MoscowState University, Moscow, Russia, in 1994, and thePh.D. degree in electrical engineering and computerscience from the Massachusetts Institute of Tech-nology, Cambridge, in 2002.

She was with BAE Systems Advanced InformationTechnology from 2002 to 2006. Since 2006, she has been an Assistant Pro-fessor with the Department of Industrial and Enterprise Systems Engineering,University of Illinois at Urbana–Champaign. Her general interest is in opti-mization including fundamental theory, models, algorithms, and applications.Her current research interest is focused on large-scale convex optimization, dis-tributed multiagent optimization, and duality theory with applications in dis-tributed optimization.

Dr. Nedic received a National Science Foundation (NSF) Faculty Early Ca-reer Development Award in Operations Research in 2008.

on ergodicity, infinite flow, and consensus in random models

Documents