multicasting multiple correlated sources to multiple sinks over a noisy channel network

4 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 1, JANUARY 2011

Multicasting Multiple Correlated Sources to MultipleSinks Over a Noisy Channel Network

Te Sun Han, Fellow, IEEE

Abstract—The problem of network coding for multicasting asingle source to multiple sinks has first been studied by Ahlswede,Cai, Li, and Yeung in 2000, in which they have established thecelebrated max-flow min-cut theorem on nonphysical informationflow over a network of independent channels. On the other hand,in 1980, Han has studied the case with multiple correlated sourcesand a single sink from the viewpoint of polymatroidal functions inwhich a necessary and sufficient condition has been demonstratedfor reliable transmission over the network. This paper presents anattempt to unify both cases, which leads to establish a necessaryand sufficient condition for reliable transmission over a networkfor multicasting multiple correlated sources to multiple sinks.Here, the problem of separation of source coding and networkcoding is also discussed.

Index Terms—Capacity function, copolymatroid, correlatedsources, entropy rate, min-cut, multiple sinks, multiple sources,network coding, polymatroid, transmissibility.

I. INTRODUCTION

T HE problem of network coding for multicasting asingle source to multiple sinks has first been studied by

Ahlswede et al. [1] in 2000, in which they have established thecelebrated max-flow min-cut theorem on nonphysical infor-mation flow over a network of independent channels. On theother hand, in 1980, Han [3] had studied the case with multiplecorrelated sources and a single sink from the viewpoint ofpolymatroidal functions in which a necessary and sufficientcondition has been demonstrated for reliable transmission overa network.

This paper presents an attempt to unify both cases and to gen-eralize it to quite a general case with stationary ergodic corre-lated sources and noisy channels (with arbitrary nonnegativereal values of capacity that are not necessarily integers) satis-fying the strong converse property (cf. Verdú and Han [6], Han[4]), which leads to establish a necessary and sufficient condi-tion for reliable transmission over a noisy network for multi-casting multiple correlated sources altogether to every multiplesinks.

Manuscript received January 06, 2009; revised April 11, 2010; accepted July02, 2010. Date of current version December 27, 2010. The material in this paperwas presented (in part) at the International Symposium on Information Theoryand its Applications, Auckland, New Zealand, December 7–10, 2008.

The author is with the Quantum-ICT Group, National Institute of Informa-tion and Communications Technology (NICT), Tokyo 184-8795, Japan (e-mail:[email protected], [email protected]).

Communicated by M. Gastpar, Associate Editor for Shannon Theory.Color versions of one or more of the figures in this paper are available online

at http://ieeexplore.ieee.org.Digital Object Identifier 10.1109/TIT.2010.2090223

It should be noted here that in such a situation with correlatedmultiple sources, the central issue turns out to be how to con-struct the matching condition between source and channel (i.e.,joint source-channel coding), instead of of the traditional con-cept of capacity region (i.e., channel coding), although in thespecial case with noncorrelated independent multiple sourcesthe problem reduces again to how to describe the capacity re-gion.

Several network models with multiple correlated sourceshave been studied in the literature, e.g., by Barros and Servetto[9], Ho et al. [13], Ho et al. [14], Ramamoorthy et al. [15].Among others, [13], [14] and [15] consider (without attentionto the converse part) error-free network coding for two (or pos-sibly multiple) stationary memoryless correlated sources with asingle (or possibly multiple) sink(s) to study the error exponentproblem, where we notice that all the arguments in [13], [14]and [15] can be validated only within the class of stationarymemoryless sources of integer bit rates and error-free channels(i.e., the identity mappings) all with one bit capacity (or integerbits capacity, which is allowed by introducing multiple edges);these restrictions are needed solely to invoke “Menger’s the-orem” in graph theory. The main result in the present paper isquite free from such seemingly severe restrictions, because wecan dispense with the use of Menger’s theorem.

On the other hand, [9] revisits the same model as in Han [3],while [15] mainly focuses on the network with two correlatedsources and two sinks to discuss the separation problem of dis-tributed source coding and network coding, where, in addition,cases of two-sources three-sinks, and three-sources two-sinksare also studied. It should be noted that, in the case of networkswith multiple correlated sources, such a separation problem isanother central issue. In this paper, we describe a necessary andsufficient condition for separability in the case with multiplesources and multiple sinks. (cf. Remark 5.2).

On the other hand, we may consider another network modelwith independent multiple sources but with multiple sinks eachof which is required to reliably reproduce a prescribed subsetof the multiple sources that depends on each sink. However, theproblem with this general model looks quite hard, although, e.g.,Yan et al. [11] and Song et al.[12] have demonstrated the en-tropy characterizations of the capacity region, which still con-tain limiting operations and are not computable. Incidentally,Yan et al. [22] have considered, as a computable special case,degree-2 three-layer networks with -pairs transmission re-quirements to derive the explicit capacity region. In this paper,for the same reason, we focus on the case in which all the mul-tiple correlated sources is to be multicast to all the multiple sinksand derive a simple necessary and sufficient matching condi-

0018-9448/$26.00 © 2010 IEEE

HAN: MULTICASTING MULTIPLE CORRELATED SOURCES TO MULTIPLE SINKS OVER A NOISY CHANNEL NETWORK 5

tion in terms of conditional entropy rates and capacity func-tions. This case can be regarded as the network counterpart ofthe non-network compound Slepian–Wolf system [21].

We notice here the following; although throughout in thepaper we are encountered with the subtleties coming from thegeneral channel and source characteristics assumed, the mainlogical stream remains essentially unchanged if we considersimpler models, e.g., such as stationary correlated Markovsources together with stationary memoryless noisy channels.This means that considering only simple cases does not helpso much at both of the conceptual and notational levels of thearguments. For this reason, we preferred here the compactgeneral settings.

The present paper consists of five sections. In Section II nota-tions and preliminaries are described, and in Section III we statethe main result as well as its proof. In Section IV two examplesare shown. Section V provides another type of necessary andsufficient condition for transmissibility. Finally, some detailedcomments on the previous papers are given.

II. PRELIMINARIES AND NOTATIONS

A. Communication Networks

Let us consider an acyclic directed graph where, , but

for all . Here, elements of are called nodes, and ele-ments of are called edges or channels from to . Eachedge is assigned the capacity , which specifiesthe maximum amount of information flow passing through thechannel . If we want to emphasize the graph thus capaci-tated, we write it as where . Agraph is sometimes called a (communication)network, and indicated also by . We considertwo fixed subsets , of such that (the emptyset) with

where elements of are called source nodes, while elements ofare called sink nodes. Here, to avoid subtle irregularities, we

assume that there are no edges such thatInformally, our problem is how to simultaneously transmit the

information generated at the source nodes in altogether to allthe sink nodes in . More formally, this problem is describedas in the following subsection.

Remark 2.1: In the above we have assumed that .However, we can reduce the case of to the case of

by equivalently modifying the given network. In fact,suppose and let for some . Then, weadd a new source node to , and generate a new edgewith capacity , and remove the node from . Repeat thisprocedure until we have . The assumption that thereare no edges such that also can be dispensed withby repeating a similar procedure.

B. Sources and Channels

Each source node generates a stationary and ergodicsource process

(2.1)

where ( ) takes values in finite source alphabet. Throughout in this paper we consider the case in which the

whole joint process is stationary and ergodic.It is then evident that the joint process is alsostationary and ergodic for any such that . Thecomponent processes may be correlated. We write

as

(2.2)

and put

(2.3)

where ( ) takes values in .On the other hand, it is assumed that all the channelsspecified by the transition probabilities

with finite input alphabet and finite output alphabet , arestatistically independent and satisfy the strong converse prop-erty (see Verdú and Han [6]). It should be noted here that sta-tionaty and memoryless (noisy or noiseless) channels with finiteinput/output alphabets satisfy, as very special cases, this prop-erty (cf. Gallager [7], Han [4]). Barros and Servetto [9] haveconsidered the case of stationary and memoryless sources/chan-nels with finite alphabets. The following lemma plays a crucialrole in establishing the relevant converse of the main result.

Lemma 2.1: (Verdú and Han [6]) The channel capacity ofa channel satisfying the strong converse property with finiteinput/output alphabets is given by

where are the input and the output of the channel ,respectively, and is the mutual information (cf.Cover and Thomas [8]).

C. Encoding and Decoding

In this section let us state the necessary operation ofencoding and decoding for network coding with multiplecorrelated sources to be multicast to multiple sinks.

With arbitrarily small and , we introducean code as the one as specified by(2.4) (2.9) below, where we use the notationto indicate . How to construct a “good”

code will be shown in Direct partof the proof of Theorem 3.1.

1) For all , the encoding function is

(2.4)


where the output of is carried over to the encoderof channel , while the decoder of outputs anestimate of the output of , which is specified by thestochastic composite function:

(2.5)

2) For all , the encoding function is

(2.6)

where the output of is carried over to the encoderof channel , while the decoder of outputs anestimate of the output of , which is specified by the sto-chastic composite function:

(2.7)Here, if is empty, we use the conventionthat is an arbitrary constant function taking a value in

;3) For all , the decoding function is

(2.8)

4) Error probabilityAll sink nodes are required to reproduce a “good” esti-mate ( ) of , throughthe network , so that the error probability

be as small as possible. Formally, for all, the probability of decoding error committed at sink

is required to satisfy

(2.9)

for all sufficiently large . Clearly, are the random vari-ables induced by that were generated at all source nodes

.

Remark 2.2: In the above coding process, is applied be-fore is if , and is applied before if

Such an indexing is possible because we are dealing withacyclic directed graphs. This defines the order in which the en-coding functions are applied. Since if , a nodedoes not encode until all the necessary informations are receivedon the input channels (see, Ahlswede, Cai, Li and Yeung [1],Yeung [2]). In this sense, the coding procedure with the codes

defined above is in accordance with thenatural ordering on an acyclic graph. This observation will befully used in the proof of Converse part of Theorem 3.1 in orderto establish a Markov chain property.

We now need the following definitions.

Definition 2.1 (Rate Achievability): If there exists ancode for any arbitrarily small as

well as any sufficiently small , and for all sufficientlylarge , then we say that the rate is achievable forthe network .

Definition 2.2 (Transmissibility): If, for any small , theaugmented capacity rate is achievable,then we say that the source is transmissible over the network

where is called the -capacity of channel.

The proof of Theorem 3.1 (both of the converse part and thedirect part) are based on these definitions.

D. -Typical Sequences

Let denote the sequence of length such as

Similarly, we denote by the sequence such as

We set

and let be the entropy rate of the process . With anysmall , we say that is a -typical sequence if

(2.10)

where is the projection of on the -direction, i.e.,( is the complement of in ). We shall denote

by the set of all -typical sequences. For any subset, let denote the projection of on

; that is

(2.11)Furthermore, set for any ,

(2.12)

We say that is jointly typical with if .Now we have (e.g., cf. Cover and Thomas [8]).

Lemma 2.2:1) For any small and for all sufficiently large :

(2.13)

2) for any :

(2.14)

where is the conditionalentropy rate (cf. Cover [5]). Specifically,

This lemma will be used in the process of proving the transmis-sibility of the source over the network .


E. Capacity Functions

Let be a network. For any subset wesay that (or simply, ) is a cut and

the cutset of (or simply, of ). Also, we call

(2.15)

the value of the cut . Moreover, for any subsetsuch that (the source node set) and for any(the sink node sets), define

(2.16)

(2.17)

We call this the capacity function of for thenetwork .

Remark 2.3: A set function on is called a copolyma-troid1 (function) if it holds that

It is not difficult to check that is a copoly-matroid (see Han [3]). On the other hand, a set function on

is called a polymatroid if it holds that

It is also not difficult to check that for each the functionin (2.16) is a polymatroid (cf. Han [3], Meggido [23]),

but in (2.17)) is not necessarily a polymatroid. Theseproperties have been fully invoked in establishing the matchingcondition between source and channel for the special case of

(cf. Han [3]). In this paper too, they play a relevantrole in order to argue about the separation problem betweendistributed source coding and network coding. This problem ismentioned later in Section V (cf. Remark 5.2).

With these preparations we will demonstrate the main resultin the next section.

III. MAIN RESULT

The problem that we deal with here is not that of establishingthe “capacity region” as usual, because the concept of “capacityregion” does not make sense for the general network withcorrelated sources. Instead, we are interested in the matchingproblem between the correlated source and the network

(transmissibility: cf. Definition 2.2). Under

1In Zhang et al. [18], the co-polymatroid here is called the contra-polyma-troid.

what condition is such a matching possible? This is the keyproblem here. An answer to this question is just our main resultto be stated here.

Theorem 3.1: The source is transmissible over the net-work if and only if

(3.1)

holds.

Remark 3.1: The case of was investigated by Han[3], and subsequently revisited by Barros and Servetto [9], whilethe case of was investigated by Ahlswede et al.[1].

Remark 3.2: If the sources are mutually independent, (3.1)reduces to

Then, setting the rates as we have another equiv-alent form:

(3.2)

This specifies the capacity region of independent message ratesin the traditional sense. In other words, in case the sources areindependent, the concept of capacity region makes sense. In thiscase too, channel coding looks like for nonphysical flows (as forthe case of , see Ahlswede, Cai, Li and Yeung [1]; andas for the case of see, e.g., Koetter and Medárd [16],Li and Yeung [17]). It should be noted that formula (3.2) is notderivable by a naive extension of the arguments as used in thecase of single-source ( ), irrespective of the comment in[1].

Proof of Theorem 3.1:1) Converse Part: Suppose that the source is trans-

missible over the network with error proba-bility ( ) under encoding func-tions and decoding functions It is also supposed that

( ) with the -capacity.Here, the input to and the output from channel

may be regarded as random variables that were induced by therandom variable . In the following, wefix an element , where is the complement of in .Set

(3.3)

then

(3.4)For and , let be a minimum cut, i.e., a cutsuch that

(3.5)


and list all the channels such that , as

(3.6)

Furthermore, let the input and the output of channel bedenoted by respectively ( . Set

(3.7)

Since we are considering those codes asdefined by (2.4) (2.9) in Section II on an acyclic directedgraph (cf. Remark 2.2) and hence there is no feedback, it iseasy to see that (conditioned on

) forms a Markov chain in this order. Therefore, byvirtue of the data processing lemma (cf. Cover and Thomas [8]),we have

(3.8)

On the other hand, noticing that takes values inand applying Fano’s lemma (cf. Cover and Thomas [8]),

we have

(3.9)Hence,

(3.10)

From (3.8) and (3.10):

(3.11)

On the other hand, since all the the channels on the network aremutually independent and satisfy the strong converse property,it follows by virtue of Lemma 2.1 that

(3.12)

for all sufficiently large , where the first inequality of (3.12)follows from the property that all the channels are assumed tobe mutually independent.2

2Specifically, let � � � � � � � �� be random variables such that�� (� � �� ) (channel independence), then�� (cf. Cover and Thomas[8]).

It should be noted here that we are now considering the -ca-pacity (cf. Definition 2.2). Thus, averaging both side of (3.11)and (3.12) with respect to , we have

(3.13)

where

Noting that is stationary and ergodic and taking the limiton both sides of (3.13), it follows that

(3.14)

where is the conditional entropy rate and we havenoticed that as . Since is arbitrarilysmall, we have

(3.15)

Since is arbitrary, we conclude that

2) Direct Part: Suppose that inequality (3.1) holds. It suf-fices to show that for is achievable for any small

(see Definitions 2.1, 2.2). To do so, we will use belowthe random coding argument. Before that, we need some prepa-ration. First, with sufficiently small in Definition 2.1 wehave

(3.16)

The second inequality guarantees that, for each channel ,-capacity is enough, with appropriate choice

of an encoder and a decoder , to attain reliable repro-duction of the input of the encoder (i.e., the output ofwith domain size ) at the decoder with maximumdecoding error probability such that as

(cf. e.g., Gallager [7], Csiszár and Körner [21]). On theother hand, the first inequality of (3.16) will be used later.

In order to first evaluate the error probability

let us define the error event:

or more formally:

(3.17)

where ’s and ’s ( ) have been specified in (2.4)(2.9). Then

(3.18)


where indicates the complement of , i.e.

(3.19)

Now define

(3.20)

then it is not difficult to check that becauseis the maximum decoding error probability. Moreover, we

see that

Therefore,

(3.21)

with , where the first equality comesfrom the fact that all component channels are independent. Itis obvious that as .

Thus, in order to demonstrate , it suffices to showthat

(3.22)

which means that we may assume throughout in the sequel thatall the channels in the network are regarded as noiseless (i.e.,the identity mappings). Accordingly, then,

reduces to with domain size , andconsequently , where denotes the value of as afunction of similarly for . Thus, we can separate channelcoding from network coding. Hereafter, for this reason, we useonly the notation , instead of , .

Let us now return to show, in view of Definition 2.2, thatis achievable for any snall . To do so,

we invoke the random coding argument: for each

make take values uniformly and independently in(cf. (2.6)). First, define the associated random

variables, as functions of , such that

It is evident that ’s thus defined carry on all the informa-tion received at node during the coding process.

In the sequel we use the following notation: fix anand decompose it as where ( ).We indecate by an such that ,

, where means componentwise unequality,i.e., for all . It should be remarked here that two

distinct sequences are indistinguishable at the de-coder if and only if . The proof tobe stated below is basically along in the same spirit as that ofAhlswede et al.[1], although we need here to invoke the jointtypicality argument as well as subtle arguments on the classi-fication of error patterns. Let us now evaluate the probabilityof decoding error under the encoding scheme as specified inSection II-C. We first fix a typical sequence , andfor and , define

if there exists some suchthat is jointly typical with and

otherwise.(3.23)

Furthermore, set

(3.24)

where we notice that if and only if cannot beuniquely recovered by at least one sink node .

Here, for any node let denote the set of all thestarting nodes of the longest directed paths ending at node ,and set

Furthermore, we consider any and define

(3.25)

(3.26)

where is the set of nodes at which two sources andare distinguishable, and is the set of nodes at whichand are indistinguishable. It is obvious that ,

and .Now let us fix any and suppose that ,

which implies that . Then, we see that for somesuch that and , that is, is a fixed cut

between and . Then, for and :

(3.27)

where we have used the first inequality in (3.16). Notice herethat , are random sets under the random coding for ’s.Therefore, in view of the assumed acyclicity of the network:

(3.28)


where3 . Furthermore:

(3.29)

where was specified in Section II.In conclusion, it follows from (3.28) and (3.29) that, for any

fixed cut separating and ,

(3.30)

so that

(3.31)

On the other hand, as is seen from the definition of in(3.23), condition is equivalent to the statement“ for some such that isjointly typical with ” As a consequence, by virtue of Lemma2.2 and (3.31), we obtain

(3.32)

where we have chosen , since can be arbitrarilysmall. Then, in view of (3.24), it follows that

(3.33)

which together with condition (3.1) yields

(3.34)for all sufficiently large where and denotesthe expectation due to random coding.

Finally, in order to show the existence of a deterministic codeto attain the transmissibility over network , set

and set for , then, again by Lemma2.2:

3One of the reviewers suggests that another formal derivation of (3.28) isfound in Avestimehr [28], though essentially the same.

(3.35)

On the other hand, the left-hand side of (3.35) is rewritten as

Thus, we have shown that there exists at least one deterministiccode with probability of decoding error at most .

IV. EXAMPLES

In this section, we show two examples of Theorem 3.1 withand .

Example 1: Consider the network as in Fig. 1(called the but-terfly) where all the solid edges have capacity 1 and the indepen-dent sources , are binary and uniformly distributed (citedfrom Yan, Yang and Zhang [22]). The capacity function of thisnetwork is computed as follows:

On the other hand:

Therefore, condition (3.1) in Theorem 3.1 is satisfied withequality, so that the sourse is transmissible over the network.Then, how to attain this transmissibility? That is depicted inFig. 2 where denotes the exclusive OR. Fig. 3 depicts thecorresponding capacity region, which is within the frameworkof the previous work (e.g., see Ahlswede et al. [1]).

Example 2: Consider the network with noisy channels as inFig. 4.1.2 where the solid edges have capacity 1 and the brokenedges have capacity . Here, ( ) is thebinary entropy defined by


Fig. 1. Example 1.

Fig. 2. Coding for Example 1.

Fig. 3. Capacity region for Example 1.

The source generated at the nodes is thebinary symmetric source with crossover probability , i.e.

Notice that , are not independent. The capacity functionof this network is computed as follows:

On the other hand:

Fig. 4. Coding for Example 2.

Therefore, condition (3.1) in Theorem 3.1 is satisfied with strictinequality, so that the source is transmissible over the network.Then, how to attain this transmissibility? That is depicted inFig. 2 where , are independent copies of , , respec-tively, and is an matrix ( ). Notice thatthe entropy of (componentwise exclusive OR) isbits and hence it is possible to recover from(of length ) with asymtoticaly negligible probabilityof decoding error, provided that is appropriately chosen (seeKörner and Marton [20]). It should be remarked that this ex-ample cannot be justified by the previous works such as Ho etal. [13], Ho et al. [14], and Ramamoorthy et al. [15], becauseall of them assume noiseless channels with capacity of one bit,i.e., this example is outside the previous framework.

V. ALTERNATIVE TRANSMISSIBILITY CONDITION

In this section we demonstrate an alternative transmissibilitycondition equivalent to the necessary and sufficient condition(3.1) given in Theorem 3.1.

To do so, for each we define the polyhedron as theset of all nonnegative rates such that

(5.1)

where is the capacity function as defined in (2.16) ofSection II. Moreover, define the polyhedron as the set ofall nonnegative rates such that

(5.2)

where is the conditional entropy rate as defined inSection II. Then, we have the following theorem on the trans-missibility over the network .

Theorem 5.1: The following two statements are equivalent:

(5.3)

(5.4)

In order to prove Theorem 5.1 we need the following lemma.

Lemma 5.1 (Han [3]): Let , be a copolymatroidand a polymatroid, respectively, as defined in Remark 2.3. Then,


the necessary and sufficient condition for the existence of somenonnegative rates such that

(5.5)

is that

(5.6)

Proof of Theorem 5.1: Suppose that (5.3) holds, then, inview of (2.17), this implies

(5.7)

Since, as was pointed out in Remark 2.3,and are a copolymatroid and a polymatroid, re-spectively, application of Lemma 5.1 ensures the existence ofsome nonnegative rates ( ) such that

(5.8)which is nothing but (5.4).

Next, suppose that (5.4) holds. This implies (5.8), which inturn implies (5.7), i.e., (5.3) holds.

Remark 5.1: The necessary and sufficient condition of theform (5.4) appears (without the proof) in Ramamoorthy, Jain,Chou and Effros [15], which they call the feasibility. Theyattribute the sufficiency part simply to Ho et al.[13] with

, (also, cf. Ho et al. [14] with ,), while attributing the necessity part to Han [3],

Barros and Servetto [9]. However, notice that all the argumentsin [13], [14] ([13] is included in [14]) can be validated onlywithin the class of stationary memoryless sources of integerbit rates and error-free channels (i.e., the identity mappings)all with one bit capacity (this restriction is needed to invoke“Menger’s theorem” in graph theory); while the present paper,without such “seemingly” severe restrictions, treats “general”acyclic networks, allowing for general correlated stationaryergodic sources as well as general statistically independentchannels with each satisfying the strong converse property(cf. Lemma 2.1). Moreover, as long as we are concerned alsowith noisy channels, the way of approaching the problem asin [13], [14] does not work as well, because in this noisy casewe have to cope with two kinds of error probabilities, one dueto error probabilities for source coding and the other due toerror probabilities for network coding (i.e., channel coding);thus in the noisy channel case or in the noiseless channel casewith noninteger capacities and/or i.i.d. sources of nonintegerbit rates, [15] cannot attribute the sufficiency part of (5.4) to[13], [14].

It should be noted here also that [13] and [14], though demon-strating relevant error exponents (the direct part), do not have theconverse part.

Remark 5.2 (Separation): Here, the term of separation isused to mean “two step” separation of distributed source coding

(Step 1: based on Slepian–Wolf theorem) and network codingwith independent sources (Step 2: based on Theorem 3.1 withRemark 3.2). Now suppose that there exist some nonnegativerates ( ) such that

(5.9)

Then, the first inequality ensures reliable distributed sourcecoding by virtue of the theorem of Slepian and Wolf (Step 1),while the second inequality ensures reliable network codingby virtue of Theorem 3.1 with Remark 3.2 for independentsources of rates , (Step 2). Thus, (5.9) is sufficient forseparability. Condition (5.9) is equivalently written as

(5.10)

for any general network . Moreover, in view of Remark 3.2,it is not difficult to check that (5.10) is also necessary. Thus,our conclusion is that, in general, condition (5.10) is not onlysufficient but also necessary for separability in the sense abovestated.

Remark 5.3: In this connection, [15] claims that, in the casewith and with rational capacities as well assources of integer bit rates, “ separation” always holds, whilein the case of or no conclusive claim is still notmade. Although this seems to contradict Remark 5.2, it shouldbe merely due to the different definitions for separability.

Remark 5.4: It is possible also to consider network codingwith cost. In this regard the reader may refer to, e.g., Han [3],Ramamoorthy [26], Lee et al. [27].

Remark 5.5: So far we have focused on the case where thechannels of a network are quite general but are statisticallyindependent. On the other hand, we may think of the case wherethe channels are not necessarily statistically independent. Thisproblem is quite hard in general. A typical tractable exampleof such networks would be a class of acyclic deterministicrelay networks with no interference (called the Aref network)in which the concept of “channel capacity” is irrelevant. Inthis connection, Ratnakar and Kramer [24] have studied Arefnetworks with a single source and multiple sinks, while Koradaand Vasudevan [25] have studied Aref networks with multiplecorrelated sources and multiple sinks. The network capacityformula as well as the network matching formula obtained bythem are in nice correspondence with the formula obtained byAhlswede et al. [1] as well as Theorem 3.1 established in thispaper, respectively.

ACKNOWLEDGMENT

The author is very grateful to Prof. S. Oishi for providing himwith pleasant research facilities during this work. Also, thanksgo to all the colleagues with the Oishi laboratory for their kindhelps. In addition, thanks are due to D. Vasudevan for bringingthe author’s attention to reference [25]. The author would like toexpress sincere gratitude to the reviewers for their thorough andconscientious reports, from which the author largely benefittedto improve the draft.


REFERENCES

[1] R. Ahlswede, N. Cai, S. Y. R. Li, and R. W. Yeung, “Network informa-tion flow,” IEEE Trans. Inf. Theory, vol. IT-46, pp. 1204–1216, 2000.

[2] R. W. Yeung, A First Course in Information Theory. Norwell, MA:Kluwer, 2002.

[3] T. S. Han, “Slepian-Wolf-Cover theorem for a network of channels,”Inform. Control, vol. 47, no. 1, pp. 67–83, 1980.

[4] T. S. Han, Information-Spectrum Methods in Information Theory.Berlin, Germany: Springer-verlag, 2003.

[5] T. M. Cover, “A simple proof of the data compression theorem ofSlepian and Wolf for ergodic sources,” IEEE Trans. Inf. Theory, vol.IT-21, pp. 226–228, 1975.

[6] S. Verdú and T. S. Han, “A general formula for channel capacity,” IEEETrans. Inf. Theory, vol. IT-40, pp. 1147–1157, 1994.

[7] R. G. Gallager, Information Theory and Reliable Communication.New York: Wiley, 1968.

[8] T. M. Cover and J. A. Thomas, Elements of Information Theory. NewYork: Wiley, 1991.

[9] J. Barros and S. D. Servetto, “Network information flow with correlatedsources,” IEEE Trans. Inf. Theory, vol. IT-52, pp. 155–170, 2006.

[10] L. Song, R. W. Yeung, and N. Cai, “A separation theorem forsingle-source network coding,” IEEE Trans. Inf. Theory, vol. IT-52,pp. 1861–1871, 2006.

[11] X. Yan, R. W. Yeung, and Z. Zhang, “The capacity region for mul-tiple-source multiple-sink network coding,” in Proc. IEEE Int. Symp.Information Theory, Jun. 2007.

[12] L. Song, R. W. Yeung, and N. Cai, “Zero-error network coding foracyclic networks,” IEEE Trans. Inf. Theory, vol. IT-49, pp. 3129–3139,2003.

[13] T. Ho, M. Médard, M. Effros, and R. Koetter, “Network coding forcorrelated sources,” in Proc. Conf. Information Science and Systems,2004.

[14] T. Ho, M. Médard, R. Koetter, D. R. Karger, M. Effros, J. Shi, and B.Leong, “A random linear network coding approach to multicast,” IEEETrans. Inf. Theory, vol. IT-52, pp. 4413–4430, 2006.

[15] A. Ramamoorthy, K. Jain, A. Chou, and M. Effros, “Separating dis-tributed source coding from network coding,” IEEE Trans. Inf. Theory,vol. IT-52, pp. 2785–2795, 2006.

[16] R. Koetter and M. Medárd, “An algebraic approach to network coding,”IEEE Trans. Inf. Theory, vol. IT-49, pp. 782–795, 2003.

[17] A. Li and R. W. Yeung, “Network information flow – multiple source,”in IEEE Int. Symp. Information Theory, 2001.

[18] X. Zhang, J. Chen, S. B. Wicker, and T. Berger, “Successive coding inmultiuser information theory,” IEEE Trans. Inf. Theory, vol. IT-53, pp.2246–2254, 2007.

[19] I. Csiszár, “Linear codes for sources and source networks: Error ex-ponents, universal coding,” IEEE Trans. Inf. Theory, vol. IT-28, pp.585–592, 1982.

[20] J. Körner and Marton, “How to encode the modulo-two sum of binarysources,” IEEE Trans. Inf. Theory, vol. IT-25, pp. 219–221, 1979.

[21] I. Csiszár and J. Körner, Information Theory: Coding Theorems forDiscrete Memoryless Systems. New York: Academic, 1981.

[22] X. Yan, J. Yang, and Z. Zhang, “An outer bound for multiple sourcemultiple sink coding with minimum cost consideration,” IEEE Trans.Inf. Theory, vol. IT-52, pp. 2373–2385, 2006.

[23] N. Meggido, “Optimal flows in networks with multiple sources andmultiple sinks,” in Math. Program. . : , 1974, vol. 7, pp. 97–107.

[24] N. Ratnakar and G. Kramer, “The multicast capacity of deterministicrelay networks with no interference,” IEEE Trans. Inf. Theory, vol. 52,pp. 2425–2432, Jun. 2006.

[25] S. B. Korada and D. Vasudevan, “Broadcast and Slepian-Wolf multicastover Aref networks,” in Proc. IEEE Int. Symp. Information Theory,Toronto, ON, Canada, Jul. 6–11, 2008, pp. 1656–1660.

[26] A. Ramamoorthy, “Minimum cost distributed source coding over a net-work,” in Proc. IEEE Int. Symp. Information Theory, Nice, France,2007, IEEE Trans. Inform. Theory.

[27] A. Lee, M. Medárd, K. Z. Haigh, S. Gowan, and P. Rupel, “Minimum-cost subgraphs for joint distributed source and network coding,” inThirs Workshop on Network Coding, Theory, and Applications, Jan.2007.

[28] S. Avestimehr, ““Minimum Cost Distributed Source Coding Over aNetwork,” Wireless Network Information Flow: A Deterministic Ap-proach,” Ph.D. dissertation, Univ. California, Berkeley, 2008.

Te Sun Han (M’79–S’88–F’90) received B.Eng., M.Eng., and D.Eng. degreesin mathematical engineering from University of Tokyo, Japan, in 1964, 1966,and 1971, respectively.

From 1972 to 1975, he has been was a Research Associate at the Universityof Tokyo. From 1975 to 1983 he was an Associate Professor in the Departmentof Information Sciences, Sagami Institute of Technology, Fujisawa, Japan. Hewas a Visiting Professor with the Faculty of Mathematics , University of Biele-feld, Germany, in 1980 (spring) and a Visiting Fellow with the Laboratory ofInformation Systems, Stanford University, Stanford, CA, in 1981 (summer).From 1983 to 1985, he was a Professor of mathematics, Toho University, Chiba,Japan. From 1985 to 1993, he was a Professor with the Department of Infor-mation Systems, Senshu University, Kawasaki, Japan. From 1993 to 2007, hewas a Professor with the Graduate School of Information Systems, Univer-sity of Electro-Communications, Tokyo. From 1990 to 1991, he was a VisitingFellow, Department of Electrical Engineering, Princeton University, Princeton,NJ, and at the School of Electrical Engineering, Cornell University, Ithaca, NY.In 1994 (Summer) he was a Visiting Fellow, Faculty of Mathematics, Univer-sity of Bielefeld. Since 2007, he is a Visiting Professor at Waseda University,Japan and since April 2010, a Senior Visiting Researcher with the National In-stitute of Information and Communications Technology (NICT), Japan. His re-search interests include basic problems in Shannon Theory, multi-user source/channel coding systems, multiterminal hypothesis-testing and parameter esti-mation under data compression, large-deviation approach to information-theo-retic problems, and especially, information-spectrum theory.

Dr. Han was engaged in organizing activities in the Board of Governers forSociety of Information Theory and Its Applications, Japan, from 1978 to 1990.He was a member of the program committee for 1988 IEEE ISIT, Kobe, Japan;1995 IEEE ISIT Whistler, BC, Canada; 1997 IEEE ISIT, Ulm, Germany;1998 IEEEE ISIT, Boston, MA: 2004 IEEE ISIT, San Antonio, TX; 2007IEEE ISIT, Nice, France; and 2009 IEEE ISIT, Toronto, ON, Canada. He wasalso co-chairmen for the 3rd, 5th, 6th, and 7th Benelux-Japan Workshops onInformation Theory and Communication, Eindhoven, The Netherlands, June,1994; Hakone, Japan, 1996; Essen, Germany, 1996; and Eltville, Germany,1997, respectively. Moreover, he was a co-chairman of the program committeefor 2003 IEEE ISIT, Yokohama, Japan. From 1994 to 1996, he was a chairmanof Tokyo Chapter for IEEE Information Theory Society. From 1994 to 1997,he was an associate editor for Shannon Theory for IEEE TRANSACTIONS ON

INFORMARTION THEORY, and a member of the Board of Governers for the IEEEInformation Theory Society. He received the Shannon Award in 2010.

multicasting multiple correlated sources to multiple sinks over a noisy channel network

Documents