Decomposable Optimisation Methods

LCA Reading Group, 12/04/2011Dan-Cristian Tomozei

• Convex function

• Unique minimum over convex domain

Convexity

2

Roadmap• (Sub)Gradient Method• Convex Optimisation crash course• NUM• Basic Decomposition Methods• Implicit Signalling

3

Roadmap• (Sub)Gradient Method• Convex Optimisation crash course• NUM• Basic Decomposition Methods• Implicit Signalling

4

• Unconstrained convex optimisation problem

• If objective is differentiable,

• Else,

• Gain sequence – Constant– Diminishing

(Sub)gradient method

5

Roadmap• (Sub)Gradient Method• Convex Optimisation crash course• NUM• Basic Decomposition Methods• Implicit Signalling

6

• “Primal” formulation

• Convex constraints unique solution• Lagrangian

• “Dual” function – For all “feasible” points – lower bound

– Slater’s condition zero duality gap

Constrained Convex Optimisation

7

• “Primal” and “dual” formulations

• Karush-Kuhn-Tucker (KKT)

Optimality conditions

Primal variables Dual variables (i.e., Lagrange multipliers)

8

Optimum

Roadmap• (Sub)Gradient Method• Convex Optimisation crash course• NUM• Basic Decomposition Methods• Implicit Signalling

9

• Population of users • Concave utility functions (e.g., rates)• Typical formulation (e.g., [Kelly97]):– Network flows of rates– Physical links of max capacity– Routing matrix

– Dual variables = congestion shadow prices

Network Utility Maximisation

10

Roadmap• (Sub)Gradient Method• Convex Optimisation crash course• NUM• Basic Decomposition Methods• Implicit Signalling

11

• Coupling constraint

• To decouple – simply write the dual objective

• Iterative dual algorithm:– Each user computes – Use a gradient method to update dual variables, e.g.,

Dual Decomposition

12

• Coupling variable

• To decouple – consider fixed coupling variable• Iterative primal algorithm:– Solve individual problems and get partial optima

– Update primal coupling variable using gradient method

Primal Decomposition

13

Implementation issues• Certain problems can be decoupled• Dual decomposition dual algorithm– Primal vars (rates) depend directly on dual vars (prices) – Price adaptation relies on current rates– Always closed form?

• Primal decomposition– The other way around…

• Do we really need to keep track of both primal and dual variables? Can duals be “measured” instead?

14

Roadmap• (Sub)Gradient Method• Convex Optimisation crash course• NUM• Basic Decomposition Methods• Implicit Signalling

15

• Graph• Supported rate region • Network cost function – Unsupported rate allocation – Marginal cost positive and strictly increasing

• Source s wants to send data to receiver r at rate at minimum cost– Supported min-cut is at least

Multipath unicast min-cost live streaming

16

Optimisation formulation

• Write Lagrangian

• Primal-dual provably converges to optimum

17

Is it that hard?• Recall

• Dual variables have queue-like evolution!• We already queue packets!

18

Implicit Primal-Dual• Rate control via

• Rate on link (i,j)– Increase prop to backlog difference– Decrease prop to marginal cost (measurable – RTT, …)

• Influence of parameter s– Small closer optimal allocation, huge queue sizes– Large manageable queue sizes, optimality trade-off

19

Conclusion• Finding a fit-all recipe is hard• We can handle some cases• Specific formulations may lead to nice protocols

• See also– R. Srikant’s “Mathematics of Internet Congestion Control”– Kelly, Mauloo, Tan - ***– Palomar, Chiang - ***

20

Questions

21