on multi-source multi-sink hyperedge networks: enumeration ... · on multi-source multi-sink...

On Multi-source Multi-sink Hyperedge Networks:Enumeration, Rate Region Computation, & Hierarchy

—Ph. D. Dissertation Defense

Congduan Li

ASPITRG & MANLDrexel University

[email protected]

May 29, 2015

C. Li (ASPITRG & MANL) Ph. D. Dissertation Defense May 29, 2015 1 / 73

Motivation: information flow in networks

Coding (mix the input) outperforms routing (receive-forward) in manycircumstances

Efficient information flow in networks: admissible source rate v.schannel capacities

?Data Sources

Relay Nodes/ Routers

Network

Users

?


Motivation: information storage in data centers

Coding (mix sources) saves resources, compared with simplereplication

Efficient information storage in data centers: admissible source filesize v.s hard disk capacities

Source Files

Hard Drives

Users

?

?


Generalized model

A multisource multisink hyperedge network (hyperedge MSNC)

Paths

In(i) i

e 2 EU

1

s

S T

K |T |

t

1

Ys �(t)

.

.

.

.

.

.

.

.

.

.

.

.

Paths

In(i) i

ReUe

Out(i)

1

s

K |T |

t

1

�(t)

.

.

.

.

.

.

.

.

.

.

.

.

Sources Network

...

Sinks

A[R1, R2, . . .]T � B[H(Y1), H(Y2), . . .]

T

Rate/capacity region: compute

Notation: (K , |EU |) means Ksources, |EU | intermediatehyperedges

We made three major contributions


Contribution 1: Revolution, use computer for rate regioncalculation

Conventional: 1 (special) network, info. ineq., manually, 1 paper

Computational: 103, 106, 1012 (arbitrary) networks, computer, # ofpapers?

In(i) i

eReUe

Out(i)

1

s

S T

|S| |T |

t

1

i

1

s

S T

|S| |T |

t

1

In(i) i

eReUe

Out(i)

1

s

S T

|S| |T |

t

1

i

1

s

S T

|S| |T |

t

1

In(i) i

eReUe

Out(i)

1

s

S T

|S| |T |

t

1

i

1

s

S T

|S| |T |

t

1

In(i) i

eReUe

Out(i)

1

s

S T

|S| |T |

t

1

i

1

s

S T

|S| |T |

t

1

In(i) i

eReUe

Out(i)

1

s

S T

|S| |T |

t

1

i

1

s

S T

|S| |T |

t

1

. . .. . .

Conventional way Computational way


Contribution 2: enumeration of all networks

Conventional: almost impossible to list all networks due to the largenumber of instances

Computational: enumerate ≥ 109, 1012 general networks

In(i) i

eReUe

Out(i)

1

s

S T

|S| |T |

t

1

i

1

s

S T

|S| |T |

t

1

In(i) i

eReUe

Out(i)

1

s

S T

|S| |T |

t

1

i

1

s

S T

|S| |T |

t

1

In(i) i

eReUe

Out(i)

1

s

S T

|S| |T |

t

1

i

1

s

S T

|S| |T |

t

1

In(i) i

eReUe

Out(i)

1

s

S T

|S| |T |

t

1

i

1

s

S T

|S| |T |

t

1

. . .. . .


?

Trillions of Networks

Enumeration Algorithm


Contribution 3: build hierarchy between networks

Conventional: what to do with so many rate regions?

Computational: define embedding and combination operators, build ahierarchy, analyze the rate regions and use them to solve even morenetworks in scale


?

Trillions of Rate Regions

Hierarchy Operators

. . .. . .

... ...

...

...

... ...

Build hierarchy between themand solve more networks in scale

... ...


Outline

1 Enumeration

Network Model

Minimality

Network Equivalence Class

Enumeration algorithm

2 Rate Region Computation

3 Network Operations

Embedding Operations

Combination Operations

Play with both

Enumerate networks

New networks

Software to calculaterate regions or bounds

Combinationoperators

Embeddingoperators

ForbiddenMinors


Outline

1 Enumeration

Network Model

Minimality







Play with both

Enumerate networks

New networks



Embeddingoperators

ForbiddenMinors


Network Coding: Sources

We assume UOut(s) = Ys

Paths

In(i) i

e

1

s

S T

|S| |T |

t

1

Ys �(t)

.

.

.

.

.

.

.

.

.

.

.

.

Paths

In(i) i

ReUe

Out(i)

1

s

|T |

t

1

�(t)

.

.

.

.

.

.

.

.

.

.

.

.

Sources Network

...

Sinks

K

H(Ys) ≥ ωs

H(YS) =�

s∈SH(Ys)

H(UOut(s)|Ys) = 0

Source Rate

Source Independence

Source Encoding


Network Coding: Edges

Paths

In(i) i

e

1

s

S T

|S| |T |

t

1

Ys �(t)

.

.

.

.

.

.

.

.

.

.

.

.

Paths

In(i) i

ReUe

Out(i)

1

s

|T |

t

1

�(t)

.

.

.

.

.

.

.

.

.

.

.

.

Sources Network

...

Sinks

K

Coding Rate Re ≥ H(Ue), e ∈ E


Network Coding: Nodes

Paths

In(i) i

e

1

s

S T

|S| |T |

t

1

Ys �(t)

.

.

.

.

.

.

.

.

.

.

.

.

Paths

In(i) i

ReUe

Out(i)

1

s

|T |

t

1

�(t)

.

.

.

.

.

.

.

.

.

.

.

.

Sources Network

...

Sinks

K

Coding Constraints H(UOut(i)|UIn(i)) = 0, i ∈ V\(S ∪ T )


Network Coding: Sinks

Paths

In(i) i

e

1

s

S T

|S| |T |

t

1

Ys �(t)

.

.

.

.

.

.

.

.

.

.

.

.

Paths

In(i) i

ReUe

Out(i)

1

s

|T |

t

1

�(t)

.

.

.

.

.

.

.

.

.

.

.

.

Sources Network

...

Sinks

K

Decoding Constraints H(Yβ(t)|UIn(t)) = 0, t ∈ T


Special Case: Independent Distributed Source Coding(IDSC)

s1

s2

g1

t1

t2U2

U1

U3

Y1

Y2 t3

Y1

Y2

Y1Y2

E1

E2

D1

D2

D3E3

Y1

Y2

Y1Y2

U2

U1

U3

Y1

Y2

Sources available to all encoders

Decoders demand various subsets of of sources

When sources are prioritized, it becomes MDCS


Special Case: Index Coding

Only one intermediate edge that transmits all information of sources

Sources may be directly available at sinks as side information

(K , 1) hyperedge MSNC


Outline

1 Enumeration

Network Model

Minimality







Play with both

Enumerate networks

New networks



Embeddingoperators

ForbiddenMinors


Source minimality example

Source minimality example: Source s3 redundant due to not demanded byany sink.

s1

s2

g1

g2

U1

U2

U3

Y1

Y2t3

Y2

t2

t1

Y2

s3Y3

s1

s2

g1

g2

U1

U2

U3

Y1

Y2t3

Y2

t2

t1

Y2

Y1 Y1


Node minimality example

Node minimality example: g1, g2 can be merged due to same input

s1

s2

g1

g2

U1

U2

U3

Y1

Y2t3

Y2

t2

t1

Y2

s1

s2

g2

U1

U2

U3

Y1

Y2t3

Y2

t2

t1

Y2

Y1 Y1


Edge minimality example

Edge minimality example: U2,U3 are parallel and can be merged

s1

s2

g1

g2

U1

U2

U3

Y1

Y2 t2

t1

Y2

s1

s2

g1

g2

U1

U2

Y1

Y2 t2

t1

Y2

Y1 Y1


Sink minimality example

Sink minimality example: decoding ability of Y2 at t2 implied by t1,equivalent to let t2 demand Y1 only

s1

s2

g1

g2

U1

U2

U3

Y1

Y2t3

Y2

t2

t1

Y2

Y1Y2

s1

s2

g1

g2

U1

U2

U3

Y1

Y2t3

Y2

t2

t1

Y2

Y1


Outline

1 Enumeration

Network Model

Minimality







Play with both

Enumerate networks

New networks



Embeddingoperators

ForbiddenMinors


Equivalent Networks

s1

s2

g1

g2

U1

U2

Y1

Y2

Y2

t2

t1

Y1

Y2

s1

s2

g1

g2

U1

U2

t2

t1

s1

s2 g1

g2

U1

U2Y1

Y2 Y2

t2

t1

Y1

Y1

Y1

Y2

Y1 $ Y2 & U1 $ U2

⇥ Y1 6$ Y2

I.)

II.)

III.)

I , II are equivalent: permutesources and edges

I , III are not equivalent: sourcesare only permuted at source side

II , III are not equivalent:sources are only permuted atsink side


Representing a Network

Ordered pair (Q,W), sources 1 . . .K , edges K + 1, . . . ,K + |EU |;Edge definitions Q ⊆{(i ,A)|i ∈ {K + 1, . . . ,K + |EU |}, A ⊆ {1, . . . ,K + |EU |} \ {i}};Sink definitionsW ⊆ {(i ,A)|i ∈ {1, . . . ,K}, A ⊆ {1, . . . ,K + |EU |} \ {i}};Same i allowed to appear in W but not in Q

12

34

12 2

(3, {1, 2}) (4, {1, 3})

(1, {3, 4})(2, {3}) (2, {4})

Q

W


Equivalence Under Group Action

Symmetry group G := S{1,2,...,K} × S{K+1,...,K+|EU |};

π ∈ G, then π(Q) 7→ {π((i ,A))|(i ,A) ∈ Q};π((Q,W)) = (π(Q), π(W));

Isomorphic or Equivalent: ∃π ∈ G such thatπ((Q1,W1)) = (Q2,W2).

12

34

12 2

(3, {1, 2}) (4, {1, 3})

(1, {3, 4})(2, {3}) (2, {4})

Q

W

12

34

21 1

(3, {1, 2}) (4, {2, 3})

(2, {3, 4})(1, {3}) (1, {4})

Q0

W 0

⇡ : permute 1, 2


Canonical Network: minimal representative in each orbit

Orbit: O(Q,W) := {(π(Q), π(W))|π ∈ G}Networks in an orbit are isomorphic or equivalent to each other

Transversal: one representative for each orbit, the canonical one

Lexicographically order the pairs (i ,A) according to (i ,A) > (j ,A′) ifj < i or i = j ,A′ < A under the lexicographic ordering

Canonical: apply order to (Q,W) and get the minimal one

# 1 2 3 4` 2 1 2 1

# 1 2 3 4` 3 3 4 4

# 1 2 3 4` 2 1 2 1

# 1 2 3 4` 2 1 2 1

# 1 2 3 4` 1 2 1 2

# 1 2 3 4` 4 4 3 3

# 1 2 3 4` 1 2 1 2

# Q W1 {(3, {1, 2}), (4, {1, 3})} {(1, {3, 4}), (2, {3}), (2, {4})}2 {(3, {1, 2}), (4, {2, 3})} {(1, {3}), (1, {4}), (2, {3, 4})}3 {(3, {1, 4}), (4, {1, 2})} {(1, {3, 4}), (2, {3}), (2, {4})}4 {(3, {2, 4}), (4, {1, 2})} {(1, {3}), (1, {4}), (2, {3, 4})}

# 1 is the canonical representative


Why this representation

Some minimality constraints easily to take care, e.g., repeated edges,redundant nodes, etc

Smaller orbits, compared with node representation

Orbit-stabilizer theorem: |O(Q,W)| = |G ||Stab((Q,W))|

# 1 2` 2 1

# 1 2` 2 1

# 1 2` 2 1

# 1 2` 1 2

# 1 2` 4 4

# 1 2` 1 2

# Q W1 {(3, {1, 2}), (4, {1, 2})} {(1, {3, 4}), (2, {3}), (2, {4})}2 {(3, {1, 2}), (4, {1, 2})} {(1, {3}), (1, {4}), (2, {3, 4})}

c

d e f

a b

a ab

Subgroup of S{1,2} ⇥ S{3,4} stabilizing (G, T ) = S{3,4}

Subgroup of S{a,b} ⇥ S{c,d,e,f}stabilizing (V, E) = S{d,f}

# a b c d e f1 2 1 3 4 5 62 1 2 3 4 5 63 2 1 4 5 6 3......

24 1 2 3 4 6 5

Isomorphs in Edge and Sink Definition Representation (2)

Isomorphs in Node Representation (24)

# 1 2` 3 3


Outline

1 Enumeration

Network Model

Minimality







Play with both

Enumerate networks

New networks



Embeddingoperators

ForbiddenMinors


Leiterspiel Algorithm [BettenBraunFripertinger, 2006]

Computes transversal of orbits on size j subsets Pj(X ) of set X ,incrementally in j ; also gives symmetry group (stabilizer)Lists directly the canonical representatives satisfying some testfunction f , as long as this test has the inherited property, i.e., if asuperset satisfies, its subsets also satisfyInput: set X , group G , inherited test function fOutput: transversal of subsets of different sizes until stop, either fixedj or other stop conditions, symmetry group (stabilizer)

Orbits on Pj�1(X )

Orbits on Pj(X )

Extensionobeying f

Transversal: canonical representatives


Enumeration based on Leiterspiel Algorithm

Target: (K , |EU |) non-isomorphic networks

Recall representation of networks: (Q,W), list pool for Q first

Leiterspiel defines edges incrementally from 1 to |EU |

X := {(i, A) |i 2 {K + 1, . . . , K + |EU |}, A ✓ {1, . . . , K + |EU |} \ {i}}

Acting group G := S{1,...,K} ⇥ S{K+1,...,K+|EU |}

Call Leiterspiel T|EU | = Leiterspiel(G, Pf|EU |(X ))

With some minimality constraints inherited

Stop condition: reach number of edges


Enumeration based on Leiterspiel Algorithm

Now for each Q ∈ T|EU |, list possible WLeiterspiel incrementally adds sinks from 1 to no possible new sink

Acting group G := S{1,...,K} ⇥ S{K+1,...,K+|EU |}

With some minimality constraints inherited

Y := {(i, A) |9 a directed path in Q from i to at least one edge in A}

Call Leiterspiel T|T | = Leiterspiel(G, Pf|T |(Y))

Stop condition: cannot increase j obeying f

For each W ∈ {T1, . . . ,T|T |}, test all the other minimality conditions on(Q,W), if it passes, we obtain a non-isomorphic network instance


Enumeration Results

|M|: number of non-isomorphic networks, listed;|M|: number of networks with edge isomorphism, counted;|Mn|: number of networks with node isomorphism, counted;

(K , |EU |) (1,2) (1,3) (2,1) (2,2) (2,3) (3,1) (3,2)

|M| 4 132 1 333 485 890 9 239 187

|M| 7 749 1 1 270 5 787 074 31 2 829 932

|Mn| 39 18,401 6 ≥ 105 ≥ 1012 582 ≥ ×1011


All (2, 2) networks: no direct access btw sources & sinks

s1

s2

g1

g2

U1

U2

Y1

Y2

Y1Y2t1s1

s2

g1

g2

U1

U2

Y1

Y2

t1

Y1Y2t2

Y1 s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2

Y1

Y2

s1

s2

g1

g2

U1

U2

Y1

Y2 Y1Y2

t1 Y1s1

s2

g1

g2

U1

U2

Y1

Y2

Y1Y2t1

t2 t2 Y1

s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2

Y1

Y2

s1

s2

g1

g2

U1

U2

Y1

Y2

t2

Y1

Y2

t3

t1

Y1

s1

s2

g1

g2

U1

U2

Y1

Y2

Y1Y2t1s1

s2

g1

g2

U1

U2

Y1

Y2

Y1Y2t1

s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2

Y1

Y1Y2

s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2

Y1

Y2

s1

s2

g1

g2

U1

U2

Y1

Y2

t1 Y1

t2 Y2

s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2 Y1

Y2 s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2

Y1

Y2

s1

s2

g1

g2

U1

U2

Y1

Y2

t2

Y1

Y2

t3

t1

Y1

s1

s2

g1

g2

U1

U2

Y1

Y2

t1

Y1Y2t2

Y2 s1

s2

g1

g2

U1

U2

Y1

Y2

t1 Y1Y2

t2 Y2

s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2 Y2

Y1s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2 Y2

Y1s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2

Y2

Y1

s1

s2

g1

g2

U1

U2

Y1

Y2

t2

Y1

Y2

t3

t1

Y1

s1

s2

g1

g2

U1

U2

Y1

Y2

Y1Y2t1

s1

s2

g1

g2

U1

U2

Y1

Y2

Y1Y2t1s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2

Y1Y2

Y1Y2

s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2 Y1Y2

Y2 s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2

Y1Y2

Y2

s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2 Y1Y2

Y1s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2

Y1

Y2

s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2

Y1

Y2

s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2 Y1

Y1Y2 s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2 Y1

Y2 s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2 Y1

Y2s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2

Y2

Y1

s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2

Y1

Y2

s1

s2

g1

g2

U1

U2

Y1

Y2

t2 Y1

Y2

t3

t1

Y2

s1

s2

g1

g2

U1

U2

Y1

Y2

t2

Y1

Y2

t3

t1

Y1

s1

s2

Y1

Y2

g1

t1

t2

Y2U1

U2 Y1Y2

s1

s2

Y1

Y2

g1

t1

t2 Y2

U1

U2

Y1s1

s2

Y1

Y2

g1

U1

U2

t1

t2 Y2

Y1 s1

s2

Y1

Y2

g1

U1

U2

t2 Y1

Y2

t3

t1

Y2

s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2

Y2

Y1Y2

s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2 Y2

Y1Y2

s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2 Y2

Y1s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2 Y2

Y1s1

s2

g1

g2

U1

U2

Y1

Y2

t1

t2

Y2

Y1

g1

g2

U1

U2

t2 Y1

Y2

t3

t1s1

s2

Y1

Y2

Y2

Instance #1 Instance #2 Instance #3 Instance #4 Instance #6 Instance #7

Instance #8 Instance #9 Instance #10 Instance #11 Instance #13 Instance #14

Instance #15 Instance #16 Instance #17 Instance #18 Instance #19 Instance #20 Instance #21



Instance #36 Instance #38 Instance #39 Instance #40Instance #41

Instance #42

Instance #43 Instance #44 Instance #45 Instance #46

Instance #5

Instance #12

Instance #37


Outline

1 Enumeration

Network Model

Minimality







Play with both

Enumerate networks

New networks



Embeddingoperators

ForbiddenMinors


Rate region

Rate region: all possible rate and source entropy vectors satisfying allnetwork constraints.

Collect the N network random variables and their joint entropies.

Define Γ∗N : 2N − 1-dim. cone, region of valid entropy vectors. (revisitlater)

Constraints from network A:L1 = {h ∈ Γ∗N : hYS = Σs∈ShYs} (1)

L2 = {h ∈ Γ∗N : hXOut(k)|Ys= 0} (2)

L3 = {h ∈ Γ∗N : hXOut(i)|XIn(i)= 0} (3)

L4 = {(hT ,RT )T ∈ R2N−1+|E|+ : Re ≥ hUe , e ∈ E} (4)

L5 = {h ∈ Γ∗N : hY β(t)|UIn(t)= 0}. (5)

Rate region (cone) in terms of edge rates and source entropies(derived from [Yan, Yeung, Zhang TranIT 2012]):

R∗(A) = projRE ,H(YS)(con(Γ∗N ∩ L123) ∩ L45) (6)


Rate Region Example

A (3, 3) network and its rate region R∗(A)

Rate region: a cone with dimensions of all variables in the network

s1

s2g1

g2

U1

U2

U3Y1

Y2

t3

t2

t1

s3Y3 Y3

Y1Y2g3 R1 � H(Y3)

R2 � H(Y1)

R1 + R2 � H(Y1) + H(Y2) + H(Y3)

R2 + R3 � H(Y1) + H(Y3)

R1 + R2 + 2R3 � H(Y1) + H(Y2) + 2H(Y3)

Y3


Rate region

Rate region: a cone in terms of edge rates and source entropies:

R∗(A) = projRE ,H(YS)(con(Γ∗N ∩ L123) ∩ L45) (7)

Involves Γ∗N : 2N − 1-dim. cone, region of valid entropy vectors.


Region of Entropic Vectors

Γ∗N :

Open set in general

Γ∗N not fully characterized for N ≥ 4: convex but containsnon-polyhedral part

�⇤N

Inner bounds

Outer bounds


Sandwich Bounds

Γ∗N → ΓOutN : Rout(A) = projRE ,H(YS)(ΓOut

N ∩ L12345)

Γ∗N → ΓInN : Rin(A) = projRE ,H(YS)(ΓIn

N ∩ L12345)

R∗(A) = Rout(A) = Rin(A), if Rout(A) = Rin(A)

It becomes: Initial polyhedra → ∩ constraints → projections

Our work following this idea: Li, et. al, Allerton 2012, NetCod 2013,submission TransIT 2014.

Constraints Projection

�N

�InN

�⇤N

Constraints �N

�InN

�⇤N

Constraints


Notion of sufficiency

Outer bound typically used is Shannon outer bound→ Ro(A); innerbounds from representable matroids: scalar and vector bounds.

Scalar sufficiency: R∗(A) = Rs,q(A)

Vector sufficiency: R∗(A) = RN′q (A)

Sufficient

Insufficient

Scalar Rs,q(A) Vector RN 0q (A)


Rate Region Computation Results

|M|: number of all network instances;The other numbers represent the numbers of instances we can close thegap using various bounds, and hence exact rate region can be obtained

(K , |E|) |M| Rs,2(A) RN+12 (A) RN+2

2 (A) RN+42 (A)

(1, 2) 4 4 4 4 4

(1, 3) 132 122 132 132 132

(2, 1) 1 1 1 1 1

(2, 2) 333 301 319 323 333

(2, 3) 485890 341406 403883 432872 –

(3, 1) 9 4 4 9 9

(3, 2) 239187 118133 168761 202130 –


Outline

1 Enumeration

Network Model

Minimality







Play with both

Enumerate networks

New networks



Embeddingoperators

ForbiddenMinors


Motivated from graph and matroid theory

Inheritance property regarding sufficiency of class of codes

Minor-closed graphs: finite number of forbidden minors[RobertsonSeymour1983-2004]

Rota’s conjecture in matroid theory: finite number of forbiddenminors for Fq representability [Oxley2011]

N � 2

N � 1

N

. . .

. . .

. . . . . .

. . .

. . .

ForbiddenNot Fobidden Size

New Minimal Forbidden Minors


Similar characterization for networks?

If networks have similar characterization? Possible list of forbiddenembedded networks for sufficiency of linear codes over a field.

Network operations to obtain such embedded networks preservinginsufficiency, & region relationships

Three operations

Sources Network Destinations


Source deletion

A′ = A \ kSource Yk deleted, source kstops sending information to thenetwork, H(Yk) = 0

Sinks requiring Yk will no longerdemand it.

kYk

k

...

...

H(Yk0) = 0

k

...

...

t �(t) \ k


Edge deletion

A′ = A \ eEdge e deleted, nothing on Ue ,Re = H(Ue) = 0. Re0 = 0

ee


Edge contraction

A′ = A/e

Edge e contracted, input to tailof e available for head of e,Re =∞, H(Ue) free.

ee

Re0 = 1


Source deletion: A′ = A \ k

For each i ∈ {∗, q, (s, q), o},

Ri (A′) = ProjY\k ,RE′ ({R ∈ Ri (A) |H(Yk) = 0}) (8)

Sufficiency preserved from large to small network as the equation shows.Equivalently, insufficiency preserved from small to large network.

R(A)

R(A0)

H(Yk) = 0Projection


Example: Forbidden embedded networks

Goal: minimal forbidden networks for sufficiency

Scalar binary codes considered

k = 1, 2, 3; |E| = 2, 3, 4, 7360 non-isomorphic MDCS

1922 sufficient, 5438 insufficient

12 minimal forbidden minors (Li, et. al submission TransIT 2014)

5438 / 7360 insu�cient

1922 / 7360su�cient

12 forbiddenembedded networks


Summary for embedding operators

Three embedding operators: source deletion, edge deletion, edgecontraction

Rate region of smaller network derivable from the associated largernetwork

Sufficiency of linear codes preserved from larger to smaller networksunder embedding operations

Equivalently, insufficiency of linear codes preserved from smallernetworks to larger one


Outline

1 Enumeration

Network Model

Minimality







Play with both

Enumerate networks

New networks



Embeddingoperators

ForbiddenMinors


Source merge

After merge: Mergedsources serve as commonsources to the twonetworks

...

...

......


Sink merge

After merge: Union theinput and requests ofsinks being merged,respectively

...

...

...

...


Intermediate node merge

After merge: union inputand output of the twonodes

...

...

...

......

...


Edge merge

Edge merge: create one extranode and four associated edgesto replace the two original edges

First two edges are ordinaryedges connecting with the extranode, the other two edgesconnect the extra node with allthe head nodes of the originaltwo edges, respectively.

Equivalent: create a relay nodeon the two edges, respectively,and then merge the two relaynodes.

...

...

......


Source merge

A is obtained by merging A1.S = A2.π(S), then for eachi ∈ {∗, q, (s, q), o}Ri (A) = Proj\π(S)((Ri (A1)×Ri (A2)) ∩ L0),

L0 ={H(Ys) = H(Yπ(s)),∀s ∈ S

}

Remark: essentially replace variables Yπ(s) with the Ys for each s ∈ S.

R(A1) R(A2)8s 2 S, H(Ys) = H(Y⇡(s))

+


Obtain Rate Region for Larger Networks

1 2 3 4 5 6

1

2

43

67

8

910

12

05

11

1

{2,3}

2

{1,2,4}

3

{4}

4

{5}

5

{6}

6

{6}

7

{1,3}

8

{2,3}

9

{5} 11

{6}

10

{4,5}

X1 X2 X3 X4 X5 X6

R1

R2

R3R4

R5R6

R7 R9

R10R11

R12

R40

R70

R100

A :

1 2

1

{2}

2

{1,2}

12

R1 � H(X2)R2 � H(X2)

R1 + R2 � H(X1) + H(X2)

X1X2

R1 R2

A1 :5 6

8 76

6

{6}

5

{6}

4

{5}R6 � H(X6)

R7 + R8 � H(X6)R6 + R7 � H(X5) + H(X6)

R6 + R7 + 2R8 � H(X5) + 2H(X6)

X5 X6

R8 R6 R7

A3 :

9 10

1' 2'3'

7

{1',3'}

8

{2',3'}

R9 � H(X30)R10 � H(X10) + H(X20) + H(X30)

X10 X20X30

R9 R10

A4 :

Ri � H(X2), i = 1, 2R1 + R2 � H(X1) + H(X2)

Ri � H(X4), i = 3, 4, 40

R3 + Ri � H(X3) + H(X4), i = 4, 40

R3 + Ri + R5 � H(X3) + 2H(X4), i = 4, 40

R6 � H(X6)Ri + R8 � H(X6), i = 7, 70

R6 + Ri � H(X5) + H(X6), i = 7, 70

R6 + Ri + 2R8 � H(X5) + 2H(X6), i = 7, 70

R9 � H(X3)

R10 � P3i=1 H(Xi)

R11 � H(X5)R12 � H(X4) + H(X5)

R100 + R11 + R12 � H(X4) + 2H(X5) + H(X6)

5 4 3

3

{4}

2'

{4}

1'

{3}

3 4X3 X4

R3R4R5

A2 :4'5'6'

10' 11 12

9

{5'}

11

{6'}

10

{4'5'}

X40X50X60

R11R12R100

A5 :

Ri � H(X4), i = 3, 4R3 + R4 � H(X3) + H(X4)

R3 + R4 + 2R5 � H(X3) + 2H(X4)

R11 � H(X50)R12 � H(X40) + H(X50)

R100 + R11 + R12 � H(X40) + 2H(X50) + H(X60)


Summary for combination operators

Four combination operators: source, sink, node, and edge merge

Rate region of combined network derivable from regions of smallernetworks in the combination


Outline

1 Enumeration

Network Model

Minimality







Play with both

Enumerate networks

New networks



Embeddingoperators

ForbiddenMinors


Combination operations suffice?

Answer is NO.

Need cap to limit the network size in the combinations.

NetworkCap: nobeyond

Combinations reach a portion of

large networks

Embeddings enlarge thereachable

portionSmall size

Tiny size

Moderate size

Large size


Partial closure of networks

Worst case partial closure ofnetworks: cap the predicted sizeof networks involved in theprocess

Let the pool produce newnetworks until no new networkcan be generated

Seed list of networks

Combinations of pairs

Minors from embedding

New network?

Add to seed list

If within network cap

Yes

StopNO


New networks found from a tiny seed list

Start with the single (1, 1), single (2, 1), and the four (1, 2) networks;

!"#"$ !"#%$ !%#%$ !"#"$ !"#%$ !%#%$!&#"$ % % % % % %!&#%$ ' &' &' ' &' &'!(#($ " " " ) &* &+!(#"$ &" &+ &+ "' &"& &**!(#%$ ' ,- &'& ' *&+ +%)!"#($ ( " ( % &' &&!"#"$ (% (% (% %( "*" )""!"#%$ ' &"* &"* ' ("+& *%)&!%#($ ' ' " ' ' "!%#"$ ' ' &- ' ' %%!%#%$ ' ' (*" ' ' %%"'.// %+ (,( *+) )) "%'' &&+"*

012345.64157189:.61:;715/< 9239==45>7.5=7012345.6415;;[email protected]



6 tiny networks can generate new 11635 networks with small cap!

!"#"$ !"#%$ !%#%$ !"#"$ !"#%$ !%#%$!&#"$ % % % % % %!&#%$ ' &' &' ' &' &'!(#($ " " " ) &* &+!(#"$ &" &+ &+ "' &"& &**!(#%$ ' ,- &'& ' *&+ +%)!"#($ ( " ( % &' &&!"#"$ (% (% (% %( "*" )""!"#%$ ' &"* &"* ' ("+& *%)&!%#($ ' ' " ' ' "!%#"$ ' ' &- ' ' %%!%#%$ ' ' (*" ' ' %%"'.// %+ (,( *+) )) "%'' &&+"*

012345.64157189:.61:;715/< 9239==45>7.5=7012345.6415;;[email protected]



With the increase of cap size, number of new networks increases!

!"#"$ !"#%$ !%#%$ !"#"$ !"#%$ !%#%$!&#"$ % % % % % %!&#%$ ' &' &' ' &' &'!(#($ " " " ) &* &+!(#"$ &" &+ &+ "' &"& &**!(#%$ ' ,- &'& ' *&+ +%)!"#($ ( " ( % &' &&!"#"$ (% (% (% %( "*" )""!"#%$ ' &"* &"* ' ("+& *%)&!%#($ ' ' " ' ' "!%#"$ ' ' &- ' ' %%!%#%$ ' ' (*" ' ' %%"'.// %+ (,( *+) )) "%'' &&+"*

012345.64157189:.61:;715/< 9239==45>7.5=7012345.6415;;[email protected]



Embedding operations are important in the process!

!"#"$ !"#%$ !%#%$ !"#"$ !"#%$ !%#%$!&#"$ % % % % % %!&#%$ ' &' &' ' &' &'!(#($ " " " ) &* &+!(#"$ &" &+ &+ "' &"& &**!(#%$ ' ,- &'& ' *&+ +%)!"#($ ( " ( % &' &&!"#"$ (% (% (% %( "*" )""!"#%$ ' &"* &"* ' ("+& *%)&!%#($ ' ' " ' ' "!%#"$ ' ' &- ' ' %%!%#%$ ' ' (*" ' ' %%"'.// %+ (,( *+) )) "%'' &&+"*

012345.64157189:.61:;715/< 9239==45>7.5=7012345.6415;;[email protected]


Example to see why integrating embedding is important

Start with the single (1, 1), single (2, 1), and the four (1, 2) networksOnly combination with cap (3, 4), get only 3 networks with size (2, 2)

1

2

3 4

1 12

1

1 1

3 4

1

2

3

1

1

2

3

11

2

3

11

1 2

2 1

3

(2,1)

(1,2)

1

1

3

2

2

4

1

2

1

1

2

1

nodemerge

=

2 2

2

1 1

1

3 4

=

1 2

2 1

3

1 2

2 1

3

sources merge

+

1

2

1

1 2

2 1

3

sources merge

+

=

1

2

1

(1,1)

ALL (1,1), (2,1), (1,2)MINIMAL NETWORKCODING PROBLEMS

(there are 6)

ALLPOSSIBLECOMBINATIONOPERATORS

All (2,2) Minimal Network Coding Problems That are direct Combinations of (1,1),(2,1),and (1,2) Problems.

(There are only 3 out of 333 total (2,2) problems)

(2,2)


Example to see why integrating embedding is important

Start with the single (1, 1), single (2, 1), and the four (1, 2) networks

Consider both combination and embedding with same cap, found(2, 2) networks unreachable by combination only

2

2

1

1

34

2

2

1

1

3 4

Contracting edge 3

5

12

2

1

1

34

5

1

Contracting edge 46

2

2

2

1

1

34

2

2

1

1

34

Source merge [1,2] with [2,1]

2

1,2

1

34

5

1

2

1,2

1

34

5

1,2

6

Contracting edge 3

Contracting edge 4

1

2

1

1

3

1

2

1

1

3

2

21

34

5

1

Contracting edge 3

2

1

1

34 5

2

6

Deleting edge 3

Merge two sinks

1

1Edge merge

2 & 22+ +

+

1

2

1

1

3


Summary of work thus far

Enumeration, Rate Region Computation, Forbidden Minors, New Networks

Enumerate > 7 ⇥ 105 non-isomorphic networks

Combination operations Embedding operations

Forbidden Minors for su�ciencyNew solvable networks in scale

Calculate their exact rate regions or bounds


Dream

Though we have onlinerepository

Want a user-friendly interface toeasily get answer

Draw or input a network

In Database?

Minimality & canonical conversion

Directly Computable?

Reachable by construction?

Get the answer

Not solvable right now

Yes

No

Yes

No

Yes

No


Future work

When Shannon outer bound is tight? Any common structure?

Is Shannon outer bound tight for all MDCS, or IDSC?

Is the number of forbidden minors regarding the sufficiency of a classof linear codes finite?

Coverage of the operators in all problems

More operations: node & edge merge, source & sink merge

A notion of forbidden minor which can harness both combination andembedding operators


Publication List

1 Steven Weber, Congduan Li, and John Walsh, Rate Region for a class of delay mitigatting codes and P2P networks,46th Annual Conference on Information Sciences and Systems, Princeton University, March, 2012, Invited Paper.

2 Congduan Li, John Walsh, and Steven Weber, ”A Computational Approach for Determining Rate Regions and Codesusing Entropic Vector Bounds”, 50th Allerton Conference, UIUC, Oct 2012.

3 Congduan Li, Jayant Apte, John M. Walsh, and Steven Weber, ”A new computational approach for determining rateregions and optimal codes for coded networks”, The 2013 IEEE International Symposium on Network Coding (NetCod2013), Calgary, Canada, Jun 7-9, 2013.

4 Congduan Li, John M. Walsh, and Steven Weber, ”Matroid bounds on region of entropic vectors”, 51th Allertonconference on Communication, Control and Computation, UIUC, IL, Oct 2013.

5 Jayant Apte, Congduan Li, John MacLaren Walsh, and Steven Weber, Exact pepair problems with multiple sources, inThe 48th annual IEEE Conference on Information Sciences and Systems(CISS), Mar. 2014.

6 Jayant Apte, Congduan Li, and John MacLaren Walsh, Algorithms for Computing Network Coding Rate Regions viaSingle Element Extensions of Matroids, IEEE International Symposium on Information Theory (ISIT) 2014, Honolulu,Hawaii.

7 Congduan Li, Steven Weber, and John M. Walsh, ”Network Embedding Operations Preserving the Insufficiency ofLinear Network Codes”, 52th Allerton conference on Communication, Control and Computation, UIUC, IL, Oct 2014.

8 Congduan Li, Steven Weber, and John M. Walsh, ”Multilevel Diversity Coding Systems: Rate Regions, Codes,Computation, & Forbidden Minors”, IEEE Transactions on Information Theory, 2014, submitted.

9 Congduan Li, Steven Weber, and John M. Walsh, ”Computer Aided Proofs for Independent Distributed Source CodingProblems”, The 2015 IEEE International Symposium on Network Coding (NetCod 2015), Jun, 2015.

10 Congduan Li, Steven Weber, and John M. Walsh, ”Network Combination Operations Preserving the Sufficiency ofLinear Network Codes”, The 2015 IEEE Information Theory Workshop (ITW 2015), submitted.

11 Congduan Li, Steven Weber, and John M. Walsh, ”On Multi-source Networks: Enumeration, Rate Region Computation,and Hierarchy”, IEEE Transactions on Information Theory, 2015, ready to submit.


Selected References

X. Yan, R.W. Yeung, and Z. Zhang (2012)

An Implicit Characterization of the Achievable Rate Region for Acyclic MultisourceMultisink Network Coding

IEEE Transactions on Information Theory 58(9), 5625-5639.

A. Betten, M. Braun, H. Fripertinger (2006)

Error-Correcting Linear Codes: Classification by Isometry and Applications

Springer Berlin Heidelberg.

N. Robertson, P. Seymour (2004)

Graph minors. xx. wagner’s conjecture

Journal of Combinatorial Theory, Series B, 92(2), 325-357.

J.G. Oxley (2011)

Matroid Theory

Oxford University, 2011.


Q & A

Thank you!


on multi-source multi-sink hyperedge networks: enumeration ... · on multi-source multi-sink...

Documents