facial reduction for symmetry reduced semidefinite...
TRANSCRIPT
Facial reduction for symmetry reduced semidefiniteprograms
Hao Hua , Renata Sotirova and Henry Wolkowiczb
Updated: 2019/08/07
a Tilburg Universityb University of Waterloo
The graph partitioning problem
The graph partitioning problem (GP)
• GP: partition the vertices of a graph into k subsets of given sizes
so that the number of edges between different subsets is minimized
• Partition the graph below into 2 sets of equal size
1 2
34
1 2
34
1 2
34
Input graph objective value 4 objective value 2
2
Symmetry in the graph partitioning problem
• The input graph is ”invariant” under certain permutation of its
vertices
1 2
34
2 3
41
2 1
43
Input graph rotate clock-wise flip horizontally
• How can we exploit the symmetry to attack the problem?
3
Matrix ∗-algebra
Matrix ∗-algebra
• A set M⊆ Cn×n is a matrix ∗-algebra over C if it is closed
under addition, scalar and matrix multiplication, and taking
conjugate transpose, i.e.,
αX + βY ∈M ∀α, β ∈ CX ∗ ∈MXY ∈M,
for all X ,Y ∈M
5
Block diagonalization of Matrix ∗-algebra
• Theorem (Wedderburn 1907) Matrix ∗-algebras containing the
identity matrix have a canonical block-diagonal structure after
some unitary transformation, i.e., there exists a unitary matrix Q
and some integer t such that
Q∗MQ =
M1 0 · · · 0
0 M2...
.... . . 0
0 · · · 0 Mt
,
where each Mi ⊆ Cni×ni is basic
6
An example of block-diagonalization
• Consider the matrix ∗-algebra spanned by
B0 =
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
,B1 =
0 1 0 1
1 0 1 0
0 1 0 1
1 0 1 0
,B2 =
0 0 1 0
0 0 0 1
1 0 0 0
0 1 0 0
,where B1 is the adjacency matrix in the graph partitioning example
• The unitary matrix Q below diagonalizes B0,B1,B2
Q =1
2
1 1 1 1
1 −i −1 i
1 −1 1 −1
1 i −1 −i
7
An example of block-diagonalization
• The matrix ∗-algebra spanned by
B0 =
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
,B1 =
0 1 0 1
1 0 1 0
0 1 0 1
1 0 1 0
,B2 =
0 0 1 0
0 0 0 1
1 0 0 0
0 1 0 0
• The (block)-diagonalized matrices Bi = QTBiQ are
B0 =
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
, B1 =
8 0 0 0
0 0 0 0
0 0 −8 0
0 0 0 0
, B2 =
4 0 0 0
0 −4 0 0
0 0 4 0
0 0 0 −4
8
Symmetry reduction
Semidefinite program (SDP)
• Consider an SDP in standard form
infX{〈A0,X 〉 | 〈Ai ,X 〉 = bi for i = 1, . . . ,m,X ∈ Sn+}, (1)
where Sn+ is the cone of positive semidefinite matrices
Assume the data matrices A0, . . . ,Am and the identity matrix
are contained in a matrix ∗-algebra M.
If SDP (1) has an optimal solution, then it has an optimal
solution in M.
• References:
a) Kanno et al., 2001, de Klerk 2009, etc
b) Gatermann, Parrilo 2004, Vallentin 2009, etc
c) Schrijver 2005, Laurent 2007, etc10
Symmetry reduction for SDP
• Assume B1, . . . ,Bd is a basis of M. There exists an optimal
solution
X =d∑
k=1
xkBk ∈M
• The p.s.d. constraint X ∈ Sn+ can be simplifies as
∑dk=1 xk
block-diagonal︷ ︸︸ ︷(QTBkQ) ∈ Sn+ ⇐⇒
B∗1(x) 0 · · · 0
0 B∗2(x)...
.... . . 0
0 · · · 0 B∗t (x)
∈ Sn+
where B∗j (x) is the j-th block
• We have X ∈ Sn+ if and only if B∗j (x) ∈ Snj+ for every j = 1 . . . , t11
An example of symmetry reduction
• An SDP relaxation for the cut minimization problem (Pong et al.
’14)
minX 〈C ,X 〉s.t. A(X ) = b,X ≥ 0
X ∈ Snk+ ,
where n is the number of vertices and k is the number of subsets
in the partition
• Instance can161 with n = 161 vertices and k = 3 partitions
• The size of X ∈ Snk+ is nk = 483, and very difficult to solve /
12
An example of symmetry reduction
• The feasible solutions X under certain unitary transformation,
i.e., QTXQ, has the following block-diagonal structure
0 100 200 300 400
nz = 27189
0
50
100
150
200
250
300
350
400
450
• The sizes of these 9 blocks are 60, 60, 60, 60, 60, 60, 60, 33, 30
13
An example of symmetry reduction
• An SDP relaxation for cut minimization problems (Pong et al.
’14) minX 〈C ,X 〉s.t. A(X ) = b,X ≥ 0
X ∈ S483+///////////B∗1 (x) ∈ S60+
...
B∗9 (x) ∈ S30+
• After symmetry reduction,
the sizes of p.s.d. constraints
Original SDP 483
Symmetry reduced 60, 60, 60, 60, 60, 60, 60, 33, 30
Instance can16114
Facial reduction
Facial reduction
• Slater’s condition (strict feasibility) is a constraint qualification
in convex optimization problems
• Without strict feasibility:
- the KKT conditions may not be necessary for the optimality
- strong duality may not hold
- small perturbations may render the problem infeasible
- many solvers might run into numerical errors
• Facial reduction is a regularization technique that can be used
for semidefinite programs that fail strict feasibility (Borwein,
Wolkowicz, ’81)
16
Facial reduction
• Given the SDP in standard form
infX{〈C ,X 〉 | A(X ) = b,X ∈ Sn+} (2)
Then exactly one of the following alternatives holds
1. The SDP (2) is strictly feasible:
A(X ) = b, X ∈ Sn++
2. The auxiliary system is consistent:
0 6= A∗(y) ∈ Sn+ and 〈b, y〉 = 0
• We call A∗(y) an exposing vector
• The feasible region of (2) is contained in A∗(y)⊥ ∩ Sn+17
Facial reduction for the cut minimization problem
• An SDP relaxation for the cut minimization problem (Pong et al.
’14)
minX 〈C ,X 〉s.t. A(X ) = b,X ≥ 0
X ∈ Snk+where n is the number of vertices and k is the number of subsets
in the partition
18
Facial reduction for the cut minimization problem
• An SDP relaxation for the cut minimization problem (Pong et al.
’14)minX 〈C ,X 〉s.t. A(X ) = b,X ≥ 0
X ∈ Snk+////////// =⇒ X = VRV T ,R ∈ S(n−1)(k−1)+
where the columns of V span A∗(y)⊥
the sizes of p.s.d. constraints
Original SDP 483
Facially reduced 321
Symmetry reduced 60, 60, 60, 60, 60, 60, 60, 33, 30
Instance can161
19
Facial reduction for symmetry re-duced SDP
Facial reduction for symmetry reduced SDP
Theorem (H., Sotirov, Wolkowicz) Let W be an exposing
vector of the minimal face of a given SDP instance. Then
1. There exists an exposing vector WG ∈M of the
minimal face of the input SDP instance
2. QTWGQ is an exposing vector of the minimal face of
the symmetry reduced SDP
• In plain words, we know how to do facial reduction for the
symmetry reduced SDP now
21
Facial reduction for the symmetry reduced program
• An SDP relaxation for the cut minimization problem (Pong et al.
’14)
minX 〈C ,X 〉s.t. A(X ) = b,X ≥ 0
X ∈ Snk+where n is the number of vertices and k is the number of subsets
in the partition
22
Facial reduction for the symmetry reduced program
• A symmetry reduced SDP relaxation for cut minimization
problems
minX 〈C ,X 〉s.t. A(X ) = b,X ≥ 0
X ∈ S483+///////////B∗1 (x) ∈ S60+
...
B∗9 (x) ∈ S30+
23
Facial reduction for the symmetry reduced program
• The facially + symmetry reduced SDP relaxation for cut
minimization problems
minX 〈C ,X 〉s.t. A(X ) = b,X ≥ 0
X ∈ S483+///////////B∗1 (x) = V1R1V
T1 and R1 ∈ S40+ ,
...
B∗9 (x) = V9R9VT9 and R9 ∈ S20+ ,
24
Facial reduction for symmetry reduced SDP
• In the cut minimization problem, we obtain
the sizes of p.s.d. constraints
Original SDP 483
Facially reduced 321
Symmetry reduced 60, 60, 60, 60, 60, 60, 60, 33, 30
Facially + Symmetry 40, 40, 40, 40, 38, 40, 40, 21, 20
Instance can161
• Now lets check if our theory works?
25
Numerical results on the cut minimization problem
• We solve the SDP relaxation from Pong et al. ’14 using interior
point method, and the number of partition k = 3
•
Instance Symmetry Facial+Symmetry
can144
bound 0.3838 0.6233
iteration 35 18
time 32.27s 5.8s
can161
bound 0.4828 0.5485
iteration 24 20
time 375.63s 108.05s
• Without facial reduction, it takes longer time and iteration to get
a weaker bound.26
Summary
Input 1 Input 2 Output
SDP matrix ∗-algebra
symmetry reduced SDP
+ reduced problem size
– numerical issues
SDP exposing vector
facially reduced SDP
+ numerically stable
– symmetry not exploited
symmetryreduced
SDP
exposing vectorin the algebra
facially & symmetry reduced SDP
+ numerically stable
+ reduced problem size
27