Distributed and Secure Computation of Convex Programs over a Network of

Connected Processors

Michael J. Neely

University of Southern California

http://www-rcf.usc.edu/~mjneely

f2(x, p2) b2

f1(x, p1) b1

fk(x, pk) bk

1

2

3

k

Context:

Parallel Processing and Distributed Sub-Gradient Algorithms: -Tsitsiklis, Bertsekas, Athens [1986] -Ferris, Mangasarian [1991] -Bertsekas, Tseng [1995] -Miller, Stout [1996]

Sorting and Averaging over Graphs: -Nassimi, Sahni [1979] -Bordim, Nakano, Shen [2002] -Kempe, Dobra, Gehrke [2003] -Singh, Prasanna, Rolim [2003]

Distributed Computation of Eigenvectors over Graphs: -Kempe, McSherry [2004]

Distributed Computation of Linear Programs for Networks: -Bartal, Byers, Raz [2004]

Problem A: A General Convex Program

1

2

3

K

Assign each set of constraints and utility term to a different processor…

1

2

3

K

Assign each set of constraints and utility term to a different processor…

1

2

3

K

Assign each set of constraints and utility term to a different processor…

1

2

3

K

Assign each set of constraints and utility term to a different processor…

1

2

3

K

Assign each set of constraints and utility term to a different processor…

1

2

3

K

How to ensure all public variable constraints?

1

2

3

K

How to ensure all public variable constraints?

Idea: Define different variables at each node k.

Idea: Define different variables at each node k.

1

2

3

K

Idea: Define different variables at each node k.

1

2

3

K

Idea: Define different variables at each node k.

1

2

3

K

Problem B:

1

2

3

Kshortestpath tree

Interior Point Assumption:

Assume there is a point and a positivevalue such that:

Interior Point Assumption:

Assume there is a point and a positivevalue such that:

max

max

Iterative Algorithm:

We develop a distributed procedure where each node performs“update” computations every timeslot = {0, 1, 2, …, t}.

Algorithm motivated by Queueing Theory:

Update Equation:

Likewise, for constraints:

(the [t] vector is needed because there is no interior point associated with the above constraints)

A measure of the parent-child inequalities -- Define:

The Distributed Algorithm: fix a parameter V > 0

Initialize all queue backlogs to zero for t=0

On iteration t (where t=0, 1, 2, …) do:

Each node k transmits to its parent node.

Each node k computes as solutions to:

Each node k passes to its children

Each node k updates according to the queueing eqs.

Algorithm Security:

Analysis via Lyapunov Drift. Define Lyapunov function:

Conclusions:

Computation of General Convex Programs over Graphs

Analysis via Lyapunov Drift / Queueing Theory

Solution is given by an average, improved every slot(differs from classical subgradient methods, which often require solutions for each slot to be evaluated and compared). No initial seed point is necessary.

Enables Distributed Computation and Maintains Privacy/Security

f2(x, p2) b2

f1(x, p1) b1

fk(x, pk) bk

1

2

3

k