recent developments in picos - tu berlinpage.math.tu-berlin.de/~sagnol/papers/siopt_picos.pdf · an...

Recent Developments in PICOS

Guillaume SAGNOL

Zuse Institut Berlin (ZIB)

SIAM conference on Optimization 2014

Guillaume SAGNOL SIAM conference on Optimization 2014 1/ 24

Outline

1 Intro and Motivation

2 Recent advances for SDPInterface to MOSEK (primal / dual form)User-friendly handles for power functionsComplex Semidefinite Programming

3 Perspectives for Robust optimizationRobust CounterpartsRobust Optimization in PICOS


What is PICOS ?

PICOS isa python packagean interface to several optimization solvers: (currently:CVXOPT, SCIP, CPLEX, MOSEK, GUROBI, SMCP)a user-friendly modelling languageparticularly suited for SDP and SOCP

PICOS is nota solvera stand-alone interface


A “Hello world” example

Find the largest integer smaller than 5.2

Import the package and instanciate a problem>>> import picos as pic>>> prob = pic.Problem()

Add a scalar, integer variable>>> x = prob.add_variable(’x’,1, vtype=’integer’)

Add a constraint, and set the objective value>>> prob.add_constraint(x>> prob.set_objective(’max’,x)


A “Hello world” example (continued)Display the problem

>>> print prob---------------------optimization problem (MIP):1 variables, 1 affine constraintsx : (1, 1), integer

maximize xsuch thatx < 5.2

---------------------

Solve the problem (with the ZIB optimization suite)>>> sol = prob.solve(solver=’zibopt’,verbose=0)

Display the solution>>> print x #optimal value of x5.0


Motivation

Personal work on Optimal Experimental Design: manySOCP and SDP.No equivalent for YALMIP or CVX under Python.Transforming optimization problems to a canonical form ispainful and time-consuming

Goal of PICOSQuick implementation of new modelsBenchmark between solversEducation


Features

Interfaced solvers and types of problems PICOS can handle:cvxopt (LP, SOCP, SDP, GP)smcp (LP, SOCP, SDP)mosek (LP, MIP, (MI)SOCP, convex QCQP, MIQP)cplex (LP, MIP, (MI)SOCP, convex QCQP, MIQP)gurobi (LP, MIP, (MI)SOCP, convex QCQP, MIQP)zibopt (soplex + scip : LP, MIP, MIQP, general QCQP).


http://abel.ee.ucla.edu/cvxopt/http://abel.ee.ucla.edu/smcp/http://www.mosek.comhttp://www.ibm.com/software/integration/optimization/cplex-optimizer/http://www.gurobi.com/http://zibopt.zib.de/http://soplex.zib.de/http://scip.zib.de/

Example 2: MAXCUT SDP

maximizeX∈S|V |

14〈L,X 〉

subject to diag(X ) = 1X � 0

G = load_some_graph()N = G.number_of_nodes()

maxcut = pic.Problem()X=maxcut.add_variable(’X’,(N,N),’symmetric’)L=pic.new_param(’L’,1/4.*nx.laplacian(G)) #Laplacian

maxcut.add_constraint(pic.tools.diag_vect(X)==1)maxcut.add_constraint(X>>0) #X positive semidefinitemaxcut.set_objective(’max’,L|X)

maxcut.solve()


Example 2: MAXCUT SDP

maximizeX∈S|V |

14〈L,X 〉

subject to diag(X ) = 1X � 0

G = load_some_graph()N = G.number_of_nodes()

maxcut = pic.Problem()X=maxcut.add_variable(’X’,(N,N),’symmetric’)L=pic.new_param(’L’,1/4.*nx.laplacian(G)) #Laplacian

maxcut.add_constraint(pic.tools.diag_vect(X)==1)maxcut.add_constraint(X>>0) #X positive semidefinitemaxcut.set_objective(’max’,L|X)

maxcut.solve()Guillaume SAGNOL SIAM conference on Optimization 2014 8/ 24

Example 3: A-optimal design (MISOCP)

Given s candidateobservation points withobservation matricesA1, . . . ,As, (yi = Aiθ + �)allocate N trials to the sdesign points.

mint∈Rsn∈Ns

Z1,...,Zs∈Rli×m

∑i

ti

s.t.s∑

i=1

AiZi = I

∀i ∈ [s], ‖Zi‖2F ≤ ni ti ,s∑

i=1

ni = N.

#Problem and optimization Variablesexdesign=pic.Problem()A,s,m,N= load_some_data()I =pic.new_param(’I’,cvx.spdiag([1.]*m))Z=[exdesign.add_variable(’Z[%d]’%i,A[i].T.size) for i in range(s)]n=exdesign.add_variable(’n’,s, vtype=’integer’)t=exdesign.add_variable(’t’,s)

#define the constraints and objective functionexdesign.add_list_of_constraints(

[abs(Z[i])**2

Outline





Semidefinite Programming

Primal Form

maxX

〈C,X 〉

s.t . 〈Ai ,X 〉 = bi (i = 1, . . . ,m)X � 0

Dual Form

miny

m∑i=1

yibi

s.t .m∑

i=1

yiAi � C

Mosek input for SDPs = primal formBut in PICOS, every SDP is stored in dual form


Example (MAXCUT)The original SDP (on the left) is stored as follows in PICOS(modulo the svec operation):

maxX∈S|V|

〈L,X 〉

s.t. Xii = 1 (∀i) (1)X � 0

maxx

∑i,j

li,jxi,j

s.t. xii = 1 (∀i) (2)∑i,j

xi,jEi,j � 0

So a straightforward approach would be to pass to mosek thefollowing SDP:

maxx ,S

∑i,j

li,jxi,j

s.t. xii = 1 (∀i)〈S,Ei,j〉 = xijS � 0

twice as manyvariables


Practical implementationFor a more efficient implementation, PICOS now:

handles the variables X such that X � 0 in a special way→ Problem is passed to MOSEK directly in primal form(option handleBarVars)There is a very similar issue with variables constrained in asecond-order cone.→ PICOS creates auxiliary variables to match the canonicalform of MOSEK only when recquired(option handleConeVars)Conversely, when the most natural formulation of a SDP isthe dual form, PICOS automatically dualize it before passingit to MOSEK (option solve_via_dual)(A similar technique is used in YALMIP)


Example

minw≥0∑i wi=1

trace f (s∑

k=1

wkMk ),

f (x) solve_via_dual = False solve_via_dual = True solve_via_dual = TruehandleBarVars = False handleBarVars = True

−x 13 5.58 4.40 0.58−x 25 12.64 9.94 3.17x−87 14.98 10.48 2.56

x74 11.84 9.16 1.11

1x(1−x) 4.38 3.72 0.49g(x) 103.01 85.51 4.87

CPU times (s) for MOSEK, matrices Mk ∈ Sm generated randomly witheigenvalues in dom(f ), m = 30, s = 25.

Here, g(x) = convex-envelope(x6

6 − 3x42 + 4x

2 + x)Guillaume SAGNOL SIAM conference on Optimization 2014 14/ 24

User-friendly handles for power functionsPower of affine expressionsGeometric means and p-norms of vectors

For example, for all x , t ≥ 0, x5 ≤ t ⇐⇒ ∃u, v ≥ 0 :

u2 ≤ xtv2 ≤ uxx2 ≤ v

And more generally for positive semidefinite matrices:nth root of determinantTrace of matrix powers (e.g. trace(x0M + x1M1)p)

For example, for all X � 0, trace(X 5) ≤ t ⇐⇒

∃U,V ,T � 0 :

(

X UU T

)� 0,

(U VV X

)� 0,

(V XX I

)� 0.

trace T ≤ t


User-friendly handles for power functions

u ≤ t 14 x 12 ⇐⇒ u ≤ 4√

tx2 :>>> u < pic.geomean(t //x //x //1)# geometric mean ineq : u >> M = sdp.add_variable(’M’,(5,5),’symmetric’)>>> t < pic.detrootn(M)# nth root of det ineq : det( M)**1/5>t#

t ≤ trace(X 0.6)>>> X = prob.add_variable(’X’,(3,3),’symmetric’)>>> pic.tracepow(X,0.6) > t# trace of pth power ineq : trace( X)**3/5>t#


Complex Semidefinite ProgrammingNatural extension of SDP to the complex domain(for hermitian matrices A and X , 〈A,X 〉 = trace(X ∗A) ∈ R).Applications in Quantum information theory, Signalprocessing, relaxation of combinatorial optimizationproblems, ...

>>> P = pic.Problem()>>> Z = P.add_variable(’Z’,(3,2),’complex’)>>> Z.real# variable Z_RE:(3 x 2),continuous #>>> Z.imag# variable Z_IM:(3 x 2),continuous #

>>> X = P.add_variable(’X’,(3,3),’hermitian’)>>> X >> 0# (3x3)-LMI constraint X � |0| #


Fidelity between 2 Hermitian operators

In quantum information theory, the fidelity between 2 Hermitianoperators P,Q ∈ H+n is:

F (P,Q) = ‖P1/2Q1/2‖tr = maxU∈Un

∣∣trace (P1/2UQ1/2)∣∣

= maxZ∈Cn×n

12

trace(Z + Z ∗)

s.t.(

P ZZ ∗ Q

)� 0





∣∣trace (P1/2UQ1/2)∣∣= max

Z∈Cn×n

12

trace(Z + Z ∗)

s.t.(

P ZZ ∗ Q

)� 0





∣∣trace (P1/2UQ1/2)∣∣= max

Z∈Cn×n

12

trace(Z + Z ∗)

s.t.(

P ZZ ∗ Q

)� 0

>>> F = pic.Problem()>>> Z = F.add_variable(’Z’,(n,n),’complex’)>>> F.set_objective(’max’,’I’|0.5*(Z+Z.H))>>> F.add_constraint(((P & Z) // (Z.H & Q))>>0 )>>> F.solve(verbose = 0)


Outline





Robust OptimizationParadigm to handle uncertain parameter u:

maxx∈X

f (x ,u) Robust Counterpart (RC)−−−−−−−−−−−−−→ maxx∈X

minu∈U

f (x ,u)

For some uncertainty sets U , the RC of a tractable problem remains tractable

[Ben-Tal & Nemirovski] If u is Gaussian, U is an ellipsoid→ RC of an LP is an SOCP, RC of an SOCP is an SDP.

[Bertsimas & Sim] If for each constraint, no more than Γ parametersdeviate from a reference scenario→ RC of a LP is an LP.[Büsing & D’andreagiovanni] More generally, if historical data aboutrealizations of a row u is known (histograms), we can use a by amultiband uncertainty set (→ RC of a LP is a LP):

U ={

u ∈ Rn : ∀k ∈ {−K , . . . ,K}, #{i : (ui − ūi ) ∈ Ik} ∈ [`k , νk ]}

[Fischetti & Monaci] The framework of light robustness allows for moreflexibility (find the most robust solution that is not too far from theoptimality of the nominal problem).



maxx∈X


minu∈U

f (x ,u)

For some uncertainty sets U , the RC of a tractable problem remains tractable[Ben-Tal & Nemirovski] If u is Gaussian, U is an ellipsoid

→ RC of an LP is an SOCP, RC of an SOCP is an SDP.

[Bertsimas & Sim] If for each constraint, no more than Γ parametersdeviate from a reference scenario→ RC of a LP is an LP.[Büsing & D’andreagiovanni] More generally, if historical data aboutrealizations of a row u is known (histograms), we can use a by amultiband uncertainty set (→ RC of a LP is a LP):

U ={





maxx∈X


minu∈U

f (x ,u)


→ RC of an LP is an SOCP, RC of an SOCP is an SDP.[Bertsimas & Sim] If for each constraint, no more than Γ parametersdeviate from a reference scenario→ RC of a LP is an LP.

[Büsing & D’andreagiovanni] More generally, if historical data aboutrealizations of a row u is known (histograms), we can use a by amultiband uncertainty set (→ RC of a LP is a LP):

U ={





maxx∈X


minu∈U

f (x ,u)


→ RC of an LP is an SOCP, RC of an SOCP is an SDP.[Bertsimas & Sim] If for each constraint, no more than Γ parametersdeviate from a reference scenario→ RC of a LP is an LP.[Büsing & D’andreagiovanni] More generally, if historical data aboutrealizations of a row u is known (histograms), we can use a by amultiband uncertainty set (→ RC of a LP is a LP):

U ={





maxx∈X


minu∈U

f (x ,u)


→ RC of an LP is an SOCP, RC of an SOCP is an SDP.[Bertsimas & Sim] If for each constraint, no more than Γ parametersdeviate from a reference scenario→ RC of a LP is an LP.[Büsing & D’andreagiovanni] More generally, if historical data aboutrealizations of a row u is known (histograms), we can use a by amultiband uncertainty set (→ RC of a LP is a LP):U =

{u ∈ Rn : ∀k ∈ {−K , . . . ,K}, #{i : (ui − ūi ) ∈ Ik} ∈ [`k , νk ]

}[Fischetti & Monaci] The framework of light robustness allows for moreflexibility (find the most robust solution that is not too far from theoptimality of the nominal problem).


Robust counterparts

For example, a constraint of the form: [ellispoidal uncertainty]

∀u : ‖Qu‖ ≤ 1, (a + u)T x ≤ b

can be rewritten as (using strong duality for SOCP):

∃µ :{

QTµ = x‖µ‖ ≤ b − aT x

Similarly, a constraint of the form: [Gamma - uncertainty]

∀u : |ui | ≤ âi , #{i : ui 6= 0} ≤ Γ, (a + u)T x ≤ b

becomes:

∃y ≥ 0, z ≥ 0 :{ ∑

i yi + Γz ≤ b − aT x∀i , âi |xi | ≤ z + yi


Perspectives for Robust optimization

User will be able to specify uncertainty sets for picosparameters (that are handled as constant affineexpressions)>>> u=pic.new_param(’u’,[1,2,3])>>> u.set_gamma_uncertainty(1,[0,0.5,1])

Function robustify() to create the robust counterpart ofa problemOr solve directly by using robustness cuts


The End

Thank you for your attention


Intro and MotivationRecent advances for SDPInterface to MOSEK (primal / dual form)User-friendly handles for power functionsComplex Semidefinite Programming

Perspectives for Robust optimizationRobust CounterpartsRobust Optimization in PICOS

recent developments in picos - tu berlinpage.math.tu-berlin.de/~sagnol/papers/siopt_picos.pdf · an...

Documents