quadratic programs with complementarity constraints=1joint...

Quadratic Programswith Complementarity Constraints1

John E. Mitchell

Department of Mathematical SciencesRPI, Troy, NY 12180 USA

ISMP, BerlinAugust 23, 2012

1Joint work with Jong-Shi Pang and Lijie Bai. Supported by AFOSR.Mitchell (RPI) QPCCs ISMP, August 23, 2012 1 / 26

Outline

1 Introduction and Applications

2 Completely Positive Relaxation

3 Preprocessing using SDP

4 Conclusions

Mitchell (RPI) QPCCs ISMP, August 23, 2012 2 / 26

Introduction and Applications

Outline




4 Conclusions


Introduction and Applications

Quadratic Programs with Complementarity Constraints

Find x := (x0, x1, x2) ∈ IRn+ × IRm

+ × IRm+ to globally solve the quadratic

program with complementarity constraints (QPCC):

minimizex,(x0,x1,x2)≥0

cT x + 12 xT Qx

subject to Ax = b and ( x1 )T x2 = 0

We assume the quadratic form is convex.

Generalizes the linear complementarity problem:find y ∈ IRm with 0 ≤ y ⊥ q + My ≥ 0.

Can formulate as integer program if know bounds on x1 and x2.


Introduction and Applications Applications

Eg: Bilevel programs

Bilevel program:

minx ,y f (x , y)

s.t. g(x , y) ≤ 0

y ∈ argminy{h(y) :

r(y) ≤ v(x)}

MPCC:

min f (x , y)

s.t. g(x , y) ≤ 0

∇h(y) + ∇r(y)λ = 0

0 ≤ v(x)− r(y) ⊥ λ ≥ 0



Eg: Bilevel programs

Bilevel program:

minx ,y f (x , y)

s.t. g(x , y) ≤ 0

y ∈ argminy{h(y) :

r(y) ≤ v(x)}

MPCC:

min f (x , y)

s.t. g(x , y) ≤ 0

∇h(y) + ∇r(y)λ = 0

0 ≤ v(x)− r(y) ⊥ λ ≥ 0

Equivalent if h(y) and r(y) are convex, and a constraint qualificationholds for the subproblems.

Get a Quadratic Program with Complementarity Constraints (QPCC) iff is convex quadratic and g, r , v are all linear and h linear or quadratic.



Bilevel and Inverse Optimization Applications

• Selecting parameters to force a desired response.Eg: traffic flow.

• Parameter identification to construct better models.Eg: cross validation in data mining.

• Portfolio selection.Eg: VaR.

• System identification.Eg: terrain estimation from behavior of seismic waves.


Completely Positive Relaxation

Outline




4 Conclusions



Expressing the QPCC as a QCQP

The QPCC is a quadratically constrained quadratic program (QCQP).

Complementarity constraint: (x1)T x2 ≤ 0.Generalize to quadratic constraint q(x) ≤ 0 where

Ax = b, x ≥ 0 implies q(x) ≥ 0.

Generalization subsumes a binary restriction:qi(x) = xi(1− xi) satisfies our condition,provided Ax = b, x ≥ 0 implies 0 ≤ xi ≤ 1 (Burer’s key assumption).

Multiple quadratic constraints satisfying the condition can be summed.So we work with just a single quadratic constraint.

We have a QCQP with no Slater point.



Lifting the QCQPWe have a QCQP with a single quadratic constraint:

q(x) := h + gT x + 12xT Px ≤ 0,

such that Ax = b, x ≥ 0 implies q(x) ≥ 0.

Can be lifted to a completely positive program in a well-known manner:

minimizex ,X

cT x + 12 〈Q, X 〉

subject to Ai x = bi and Ai X ATi = b2

i , i = 1, · · · , k ,

h + gT x + 12 〈P,X 〉 = 0

and

(1 xT

x X

)∈ C∗1+n

(cone of completelypositive matrices

).

In general, the copositive program is a relaxation of the QCQP.



Burer’s result

Theorem (Burer, Math Progg, 2009)If q(x) =

∑i∈B xi(1− xi) and if Ax = b, x ≥ 0 implies 0 ≤ xi ≤ 1 ∀i ∈ B

then the QCQP and its completely positive relaxation are equivalent.

Note that Burer imposes no convexity assumption on the objectivefunction.



Our assumptions∗ Ax = b, x ≥ 0 implies q(x) ≥ 0.

∗ P is copositive on the recession cone of the linear constraints,

K , {d > 0 | A d = 0} ,

so dT Pd ≥ 0 for all d ∈ K .Note: for a QPCC, every entry in P is nonnegative.

∗ Let L ,{

d > 0 | A d = 0 and d T Pd = 0}⊆ K .

Assume objective function matrix Q is copositive on L.

∗ Note: No boundedness assumption on any of the variables.

∗ Note: Assumptions all hold if q(x) =∑

i∈B xi(1− xi) and0 ≤ xi ≤ 1∀i ∈ B. (Burer: third assumption holds providedoptimal value of BQP is finite.)



Completely positive relaxation is tight

TheoremUnder our three assumptions, the QCQP and its completely positiverelaxation are equivalent in the sense that

1. The QCQP is feasible if and only if the completely positiveprogram is feasible.

2. Either the optimal values of the QCQP and the completelypositive program are finite and equal, or both of them areunbounded below.

3. Assume both the QCQP and the completely positiveprogram are bounded below, and (x̄ , X̄ ) is optimal for thecompletely positive program, then x̄ is in the convex hullof the optimal solutions of the QCQP.

4. The optimal value of the QCQP is attained if and only ifthe same holds for the completely positive program.



Notes regarding the proof

The structure of our proof is similar to that of Burer, and we examine

(1 xT

x X

)=∑j∈J+

v2j

1ξj

vj

1ξj

vj

T

+∑j∈J0

(0ξj

) (0ξj

)T

for points feasible in the completely positive program.

We are able to show that

* q(ξjvj

)= 0 ∀j ∈ J+, so these points are feasible in the QCQP

* ξTj Pξj = 0 ∀j ∈ J0, so these rays are in our set L.

The existence of such rays makes it necessary to define Land require Q to be copositive on L.



Completely positive representations of QPCCs

CorollaryThe convex QPCC


cT x + 12 xT Qx


(with psd Q) is equivalent to a completely positive program.

Note that we impose no boundedness assumption on thecomplementary variables.



Extensions to convex constraints

Burer extended his results to problems defined over convex cones(Handbook of Semidefinite, Cone, and Polyhedral Programming,2011). (See also Eichfelder and Povh.)

This requires the extension of the definition of a completely positivematrix to a matrix that is completely positive over a cone K:

C∗n(K) , conv{

M ∈ Sn | M = xxT , x ∈ K}.

Our results can be similarly extended.

For example, a convex QCQP with additional convex quadraticconstraints is equivalent to a completely positive program.


Preprocessing using SDP

Outline




4 Conclusions



An example

0

w

y

minw ,y y2 + 4w2

subject to y + w ≥ 50 ≤ y ⊥ w ≥ 0



An example

0

w

y

minw ,y y2 + 4w2

subject to y + w ≥ 50 ≤ y ⊥ w ≥ 0

Solution to relaxation is not feasible in QPCC.



Modify objective function

yw = 0 in any feasible solution, so add multiple of yw to objective.

0

w

y

minw ,y y2 + 4w2 + 4ywsubject to y + w ≥ 5

0 ≤ y ⊥ w ≥ 0



Modify objective function

yw = 0 in any feasible solution, so add multiple of yw to objective.

0

w

y

minw ,y y2 + 4w2 + 4ywsubject to y + w ≥ 5

0 ≤ y ⊥ w ≥ 0

Solution to relaxation is optimal for QPCC.



Generalize the example


cT x + 12 xT Qx




Generalize the example


cT x + 12 xT Qx + ( x1 )T Dx2 + xT M(Ax − b)


D is a positive semidefinite diagonal matrix:so penalize violations of complementarity.

New formulation is equivalent to old formulation.

Choose D and M to ensure objective function remains convex.



Using SDP to find tight relaxationWant to get a good bound when relax complementarity.Solve maxmin problem:

maxD,M


cT x + 12 xT Qx + ( x1 )T Dx2 + xT M(Ax − b)

subject to Ax = b and ( x1 )T x2 = 0//////////////////

D and M are constrained so that D is diagonal and psd,and the resulting quadratic term is psd.

Related work:Billionnet et al have proposed a quadratic convex reformulationmethod for a binary quadratic program, with the aim of obtaining aconvex relaxation that is as tight as possible.With binary variables, can always add x2

i − xi to objective, so easy toensure convexity.



Using Lagrangian relaxation

The previous problem is the Lagrangian dual of

minx,(x0,x1,x2)≥0

cT x + 12xT Qx

subject to Ax = b

(x1)i(x2)i = 0, i = 1, . . . ,m

xi(Ax − b)j = 0, i = 1, . . . ,n, j = 1, . . . , k

Write as:

minx≥0

cT x + 12xT Qx

subject to Ax = b

cTj x + xT Qjx = 0, j = 1, . . . , J



Set up an SDP

Lift the problem and take its SDP relaxation:

minx≥0,X

cT x + 12 < Q,X >

subject to Ax = b

cTj x+ < Qj ,X > = 0, j = 1,2,3, · · ·, J.

and[

1 xT

x X

]� 0.



Get best relaxation by solving SDP

TheoremIf Q is positive definite then the SDP relaxation is equivalent to themax-min problem. Thus, the equivalent convex quadratic formulation ofthe QPCC with the best lower bound can be found by solving an SDP.

Proof exploits Lagrangian duality,and a result of Lemaréchal and Oustry.



Computational experiments

number of # constraints # variables % gap Timecomplementarities closed (secs)

50 eq: 55 110 96.74 3.15100 eq: 102 210 82.62 9.79300 eq: 305 610 67.19 163.63

90 ineq: 10, eq: 90 185 93.68 8.13

Average over ten problems of each size.

Problems generated using QPECgen.

Upper bound comes from known feasible solution.Optimal solution not known.


Conclusions

Outline




4 Conclusions


Conclusions

Conclusions

Convex quadratic programs with complementarity constraints areequivalent to convex conic programs, even if the variables areunbounded.

Relaxations of convex QPCCs can be tightened through anSDP-based preprocessing approach.


quadratic programs with complementarity constraints=1joint...

Documents