![Page 1: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/1.jpg)
Decomposition Methods
• separable problems, complicating variables
• primal decomposition
• dual decomposition
• complicating constraints
• general decomposition structures
Prof. S. Boyd, EE364b, Stanford University
![Page 2: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/2.jpg)
Separable problem
minimize f1(x1) + f2(x2)subject to x1 ∈ C1, x2 ∈ C2
• we can solve for x1 and x2 separately (in parallel)
• even if they are solved sequentially, this gives advantage ifcomputational effort is superlinear in problem size
• called separable or trivially parallelizable
• generalizes to any objective of form Ψ(f1, f2) with Ψ nondecreasing(e.g., max)
Prof. S. Boyd, EE364b, Stanford University 1
![Page 3: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/3.jpg)
Complicating variable
consider unconstrained problem,
minimize f(x) = f1(x1, y) + f2(x2, y)
x = (x1, x2, y)
• y is the complicating variable or coupling variable; when it is fixedthe problem is separable in x1 and x2
• x1, x2 are private or local variables; y is a public or interface orboundary variable between the two subproblems
Prof. S. Boyd, EE364b, Stanford University 2
![Page 4: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/4.jpg)
Primal decomposition
fix y and define
subproblem 1: minimizex1 f1(x1, y)subproblem 2: minimizex2 f2(x2, y)
with optimal values φ1(y) and φ2(y)
original problem is equivalent to master problem
minimizey φ1(y) + φ2(y)
with variable y
called primal decomposition since master problem manipulates primal(complicating) variables
Prof. S. Boyd, EE364b, Stanford University 3
![Page 5: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/5.jpg)
• if original problem is convex, so is master problem
• can solve master problem using
– bisection (if y is scalar)– gradient or Newton method (if φi differentiable)– subgradient, cutting-plane, or ellipsoid method
• each iteration of master problem requires solving the two subproblems
• if master algorithm converges fast enough and subproblems aresufficiently easier to solve than original problem, we get savings
Prof. S. Boyd, EE364b, Stanford University 4
![Page 6: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/6.jpg)
Primal decomposition algorithm
(using subgradient algorithm for master)
repeat1. Solve the subproblems.
Find x1 that minimizes f1(x1, y), and a subgradient g1 ∈ ∂φ1(y).Find x2 that minimizes f2(x2, y), and a subgradient g2 ∈ ∂φ2(y).
2. Update complicating variable.y := y − αk(g1 + g2).
step length αk can be chosen in any of the standard ways
Prof. S. Boyd, EE364b, Stanford University 5
![Page 7: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/7.jpg)
Example
• x1, x2 ∈ R20, y ∈ R
• fi are PWL (max of 100 affine functions each); f⋆ ≈ 1.71
−1 −0.5 0 0.5 10.5
1
1.5
2
2.5
3
3.5
y
φ1(y)
φ2(y)
φ1(y) + φ2(y)
Prof. S. Boyd, EE364b, Stanford University 6
![Page 8: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/8.jpg)
primal decomposition, using bisection on y
1 2 3 4 5 6 7 810
−4
10−3
10−2
10−1
100
k
f(k
)−
f⋆
Prof. S. Boyd, EE364b, Stanford University 7
![Page 9: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/9.jpg)
Dual decomposition
Step 1: introduce new variables y1, y2
minimize f(x) = f1(x1, y1) + f2(x2, y2)subject to y1 = y2
• y1, y2 are local versions of complicating variable y
• y1 = y2 is consensus constraint
Prof. S. Boyd, EE364b, Stanford University 8
![Page 10: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/10.jpg)
Step 2: form dual problem
L(x1, y1, x2, y2) = f1(x1, y1) + f2(x2, y2) + νT (y1 − y2)
separable; can minimize over (x1, y1) and (x2, y2) separately
g1(ν) = infx1,y1
(
f1(x1, y1) + νTy1)
= −f∗1 (0,−ν)
g2(ν) = infx2,y2
(
f2(x2, y2)− νTy2)
= −f∗2 (0, ν)
dual problem is: maximize g(ν) = g1(ν) + g2(ν)
• computing gi(ν) are the dual subproblems
• can be done in parallel
• a subgradient of −g is y2 − y1 (from solutions of subproblems)
Prof. S. Boyd, EE364b, Stanford University 9
![Page 11: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/11.jpg)
Dual decomposition algorithm
(using subgradient algorithm for master)
repeat1. Solve the dual subproblems.
Find x1, y1 that minimize f1(x1, y1) + νTy1.Find x2, y2 that minimize f2(x2, y2)− νTy2.
2. Update dual variables (prices).ν := ν − αk(y2 − y1).
• step length αk can be chosen in standard ways
• at each step we have a lower bound g(ν) on p⋆
• iterates are generally infeasible, i.e., y1 6= y2
Prof. S. Boyd, EE364b, Stanford University 10
![Page 12: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/12.jpg)
Finding feasible iterates
• reasonable guess of feasible point from (x1, y1), (x2, y2):
(x1, y), (x2, y), y = (y1 + y2)/2
– projection onto feasible set y1 = y2– gives upper bound p⋆ ≤ f1(x1, y) + f2(x2, y)
• a better feasible point: replace y1, y2 with y and solve primal
subproblems minimizex1f1(x1, y), minimizex2f2(x2, y)
– gives (better) upper bound p⋆ ≤ φ1(y) + φ2(y)
Prof. S. Boyd, EE364b, Stanford University 11
![Page 13: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/13.jpg)
(Same) example
−1 −0.5 0 0.5 10
0.5
1
1.5
2
2.5
ν
g1(ν)
g2(ν)
g1(ν) + g2(ν)
Prof. S. Boyd, EE364b, Stanford University 12
![Page 14: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/14.jpg)
dual decomposition convergence (using bisection on ν)
0 5 10 151.6
1.7
1.8
1.9
2
2.1
2.2
2.3
2.4
2.5
2.6
k
better boundworse boundg(ν)
Prof. S. Boyd, EE364b, Stanford University 13
![Page 15: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/15.jpg)
Interpretation
• y1 is resources consumed by first unit, y2 is resources generated bysecond unit
• y1 = y2 is consistency condition: supply equals demand
• ν is a set of resource prices
• master algorithm adjusts prices at each step, rather than allocatingresources directly (primal decomposition)
Prof. S. Boyd, EE364b, Stanford University 14
![Page 16: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/16.jpg)
Recovering the primal solution from the dual
• iterates in dual decomposition:
ν(k), (x(k)1 , y
(k)1 ), (x
(k)2 , y
(k)2 )
– x(k)1 , y
(k)1 is minimizer of f1(x1, y1) + ν(k)Ty1 found in subproblem 1
– x(k)2 , y
(k)2 is minimizer of f2(x2, y2)− ν(k)Ty2 found in subproblem 2
• ν(k) → ν⋆ (i.e., we have price convergence)
• subtlety: we need not have y(k)1 − y
(k)2 → 0
• the hammer: if fi strictly convex, we have y(k)1 − y
(k)2 → 0
• can fix allocation (i.e., compute φi), or add regularization terms ǫ‖xi‖2
Prof. S. Boyd, EE364b, Stanford University 15
![Page 17: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/17.jpg)
Decomposition with constraints
can also have complicating constraints, as in
minimize f1(x1) + f2(x2)subject to x1 ∈ C1, x2 ∈ C2
h1(x1) + h2(x2) � 0
• fi, hi, Ci convex
• h1(x1) + h2(x2) � 0 is a set of p complicating or coupling constraints,involving both x1 and x2
• can interpret coupling constraints as limits on resources shared betweentwo subproblems
Prof. S. Boyd, EE364b, Stanford University 16
![Page 18: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/18.jpg)
Primal decomposition
fix t ∈ Rp and define
subproblem 1:minimize f1(x1)subject to x1 ∈ C1, h1(x1) � t
subproblem 2:minimize f2(x2)subject to x2 ∈ C2, h2(x2) � −t
• t is the quantity of resources allocated to first subproblem(−t is allocated to second subproblem)
• master problem: minimize φ1(t) + φ2(t) (optimal values ofsubproblems) over t
• subproblems can be solved separately when t is fixed
Prof. S. Boyd, EE364b, Stanford University 17
![Page 19: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/19.jpg)
Primal decomposition algorithm
repeat1. Solve the subproblems.
Solve subproblem 1, finding x1 and λ1.Solve subproblem 2, finding x2 and λ2.
2. Update resource allocation.t := t− αk(λ2 − λ1).
• λi is an optimal Lagrange multiplier associated with resource constraintin subproblem i
• λ2 − λ1 ∈ ∂(φ1 + φ2)(t)
• αk is an appropriate step size
• all iterates are feasible (when subproblems are feasible)
Prof. S. Boyd, EE364b, Stanford University 18
![Page 20: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/20.jpg)
Example
• x1, x2 ∈ R20, t ∈ R2; fi are quadratic, hi are affine, Ci are polyhedradefined by 100 inequalities; p⋆ ≈ −1.33; αk = 0.5/k
0 20 40 60 80 10010
−4
10−3
10−2
10−1
100
k
f(k
)−
p⋆
Prof. S. Boyd, EE364b, Stanford University 19
![Page 21: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/21.jpg)
resource allocation t to first subsystem (second subsystem gets −t)
0 20 40 60 80 100−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
k
Prof. S. Boyd, EE364b, Stanford University 20
![Page 22: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/22.jpg)
Dual decomposition
form (separable) partial Lagrangian
L(x1, x2, λ) = f1(x1) + f2(x2) + λT (h1(x1) + h2(x2))
=(
f1(x1) + λTh1(x1))
+(
f2(x2) + λTh2(x2))
fix dual variable λ and define
subproblem 1:minimize f1(x1) + λTh1(x1)subject to x1 ∈ C1
subproblem 2:minimize f2(x2) + λTh2(x2)subject to x2 ∈ C2
with optimal values g1(λ), g2(λ)
Prof. S. Boyd, EE364b, Stanford University 21
![Page 23: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/23.jpg)
• hi(xi) ∈ ∂(−gi)(λ), where xi is any solution to subproblem i
• h1(x1) + h2(x2) ∈ ∂(−g)(λ)
• the master algorithm updates λ using this subgradient
Prof. S. Boyd, EE364b, Stanford University 22
![Page 24: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/24.jpg)
Dual decomposition algorithm
(using projected subgradient method)
repeat1. Solve the subproblems.
Solve subproblem 1, finding an optimal x1.Solve subproblem 2, finding an optimal x2.
2. Update dual variables (prices).λ := (λ+ αk(h1(x1) + h2(x2)))+.
• αk is an appropriate step size
• iterates need not be feasible
• can again construct feasible primal variables using projection
Prof. S. Boyd, EE364b, Stanford University 23
![Page 25: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/25.jpg)
Interpretation
• λ gives prices of resources
• subproblems are solved separately, taking income/expense from resourceusage into account
• master algorithm adjusts prices
• prices on over-subscribed resources are increased; prices onundersubscribed resources are reduced, but never made negative
Prof. S. Boyd, EE364b, Stanford University 24
![Page 26: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/26.jpg)
(Same) example
subgradient method for master; resource prices λ
0 5 10 15 20 25 300
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
k
Prof. S. Boyd, EE364b, Stanford University 25
![Page 27: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/27.jpg)
dual decomposition convergence; f is objective of projected feasibleallocation
0 5 10 15 20 25 30−1.8
−1.6
−1.4
−1.2
−1
−0.8
−0.6
−0.4
k
g
f
Prof. S. Boyd, EE364b, Stanford University 26
![Page 28: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/28.jpg)
0 5 10 15 20 25 30
10−4
10−3
10−2
10−1
100
101
k
p⋆ − g(λ)
f − g(λ)
Prof. S. Boyd, EE364b, Stanford University 27
![Page 29: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/29.jpg)
General decomposition structures
• multiple subsystems
• (variable and/or constraint) coupling constraints between subsets ofsubsystems
• represent as hypergraph with subsystems as vertices, coupling ashyperedges or nets
• without loss of generality, can assume all coupling is via consistencyconstraints
Prof. S. Boyd, EE364b, Stanford University 28
![Page 30: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/30.jpg)
Simple example
1 2 3
• 3 subsystems, with private variables x1, x2, x3, and public variables y1,(y2, y3), and y4
• 2 (simple) edges
minimize f1(x1, y1) + f2(x2, y2, y3) + f3(x3, y4)subject to (x1, y1) ∈ C1, (x2, y2, y3) ∈ C2, (x3, y4) ∈ C3
y1 = y2, y3 = y4
Prof. S. Boyd, EE364b, Stanford University 29
![Page 31: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/31.jpg)
A more complex example
1 2
3 4 5
c1
c2
c3 c4
Prof. S. Boyd, EE364b, Stanford University 30
![Page 32: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/32.jpg)
General form
minimize∑K
i=1 fi(xi, yi)subject to (xi, yi) ∈ Ci, i = 1, . . . , K
yi = Eiz, i = 1, . . . ,K
• private variables xi, public variables yi
• net (hyperedge) variables z ∈ RN ; zi is common value of publicvariables in net i
• matrices Ei give netlist or hypergraphrow k is ep, where kth entry of yi is in net p
Prof. S. Boyd, EE364b, Stanford University 31
![Page 33: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/33.jpg)
Primal decomposition
φi(yi) is optimal value of subproblem
minimize fi(xi, yi)subject to (xi, yi) ∈ Ci
repeat
1. Distribute net variables to subsystems.yi := Eiz, i = 1, . . . , K.
2. Optimize subsystems (separately).Solve subproblems to find optimal xi, gi ∈ ∂φi(yi), i = 1, . . . , K.
3. Collect and sum subgradients for each net.
g :=∑K
i=1ETi gi.
4. Update net variables.z := z − αkg.
Prof. S. Boyd, EE364b, Stanford University 32
![Page 34: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/34.jpg)
Dual decomposition
gi(νi) is optimal value of subproblem
minimize fi(xi, yi) + νTi yisubject to (xi, yi) ∈ Ci
given initial price vector ν that satisfies ETν = 0 (e.g., ν = 0).
repeat
1. Optimize subsystems (separately).Solve subproblems to obtain xi, yi.
2. Compute average value of public variables over each net.z := (ETE)−1ETy.
3. Update prices on public variables.ν := ν + αk(y − Ez).
Prof. S. Boyd, EE364b, Stanford University 33
![Page 35: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/35.jpg)
A more complex example
subsystems: quadratic plus PWL objective with 10 private variables;9 public variables and 4 nets; p⋆ ≈ 11.1; α = 0.5
0 2 4 6 8 100
1
2
3
4
5
6
7
8
k
g(ν)
f(x, y)f(x, y)
Prof. S. Boyd, EE364b, Stanford University 34
![Page 36: DecompositionMethods - University of Houstonzhan2/ECE6397/primal_dual_decomposition_slides.pdfk Prof. S. Boyd, EE364b, Stanford University 25. dual decomposition convergence; fˆis](https://reader034.vdocuments.net/reader034/viewer/2022050416/5f8c1d138ec6610bd30b3f6d/html5/thumbnails/36.jpg)
consistency constraint residual ‖y − Ez‖ versus iteration number
0 2 4 6 8 1010
−4
10−3
10−2
10−1
100
101
k
‖y−
Ez‖
Prof. S. Boyd, EE364b, Stanford University 35