modeling power of mixed integer convex optimization problems...

Modeling Power of Mixed Integer Convex Optimization Problems And Their Effective Solution with Julia and JuMP

Juan Pablo Vielma

MassachuseCs InsDtute of Technology

Michigan InsDtute for ComputaDonal Discovery and Engineering (MICDE) and Department of Industrial and OperaDons Engineering (IOE),

University of Michigan at Ann Arbor,

Ann Arbor, Michigan, October, 2018.

Jennifer ChallisE62-571, 100 Main Street,Cambridge, MA 02142(617) 324-4378jchallis at mit dot edu

CollaboratorsShabbir Ahmed, Daniel Bienstock, Daniel Dadush, Sanjeeb Dash, Santanu S. Dey, Iain Dunning, RodolfoCarvajal, Luis A. Cisternas, Miguel Constantino, Daniel Espinoza, Alexandre S. Freire, Marcos Goycoolea,Oktay Günlük, Joey Huchette, Nathalie E. Jamett, Ahmet B. Keha, Mustafa R. Kılınç, Guido Lagos, MilesLubin, Sajad Modaresi, Sina Modaresi, Eduardo Moreno, Diego Morán, Alan T. Murray, George L.Nemhauser, Luis Rademacher, David M. Ryan, Denis Saure, Alejandro Toriello, Andres Weintraub, SercanYıldız, Tauhid Zaman

Affiliations

Links

Back to top© 2013 Juan Pablo Vielma | Last updated: 02/29/2016 00:47:25 | Based on a template design by AndreasViklund

!"#

$%%&'())$*+,-+.)/012)10)3$45$678%86'94::'67-6;-<685%<8::;6=>>76%$4'6/?/6@8%+4?634::6%>::6;-<

!

!"#$%&'()$*$+&,-(.$#&/012"341-(&5!',/678

9 :&;<

min f(x)

s.t.

x 2 C

xi 2 Z i 2 I

Mostly convex f and C.

=>0?::@@@A*B+-C"AD-2:D-204(E:$#420F$GDBH)-2$+H

!"#$"%#&

! '()*+,)-+&#+./0#1+&%*(+234!56! 7/&+,)-+&#+8/1"#+234!568" 7/&+,)-+&#+),,#88+8/1"#$8+

" 7/&+8/1"#$8+&/$9+



Affiliations

Links




Affiliations

Links


What can we model with MICONV?

Joint work with Miles Lubin and Ilias Zadik

What Can MICONV Model?

• Op3mal discrete experimental design

• Obstacle avoidance and trajectory planning in op3mal control

• Por@olio op3miza3on with nonlinear risk measures and combinatorial constraints

• …

!"#$%&'(()*$+,'-$.'/$01.2!3$0"4&(5

! 1/-&6&7$8"9/-:$9/$8'7';"('$! "# "$ % " & '(5! <,&$:&-$"=$) * )>'-79?&:$@9-,$+,-. / 05! A&-$"=$879>&$/B>;&7:5

�x 2 R2 : 1 kxk 2

�x 2 R2 : 1 + x

21 x

22

A8,&79?'($:,&((5<@"$:,&&-$,)8&7;"('5

MICONV Can Model Any Finite Union of (Closed) Convex Sets

• Let !", … , !% be closed convex set. A MICONV formulaBon of & ∈ ⋃)*"% !):

&), +) ∈ cone !)×{1} ∀5 ∈ {1, … , 6}&) 7

7≤ +)9). ∀5 ∈ {1, … , 6}

;)*"

%&) = &,

;)*"

%+) = 1,= ∈ {0,1}%,9 ∈ ℝ@% ,&) ∈ ℝA ∀5 ∈ {1, … , 6}

!"#$%&'(()*$+,'-$.'/$01.2!3$0"4&(5

! 1/-&6&7$8"9/-:$9/$8'7';"('$! "# "$ % " & '(5! <,&$:&-$"=$) * )>'-79?&:$@9-,$+,-. / 05! A&-$"=$879>&$/B>;&7:5

�x 2 R2 : 1 kxk 2

�x 2 R2 : 1 + x

21 x

22

A8,&79?'($:,&((5<@"$:,&&-$,)8&7;"('5

!

A Simple Obstruction for MICONV Formulations

• cannot have a MICONV formula<on if there exists:§ There exist infinite

✗ Spherical shell�x 2 R2 : 1 kxk 2

R

infinite R ✓ S s.t.u+ v

2/2 S 8u, v 2 R, u 6= v


2/2 S 8u, v 2 R, u 6= v


2/2 S 8u, v 2 R, u 6= v

! !"#$%$&'()*"#+'*"'(,&,-).,'! "# "$ % " & '(/! 01$'+$#')2') * )3,#&*4$+'5*#1'+,-. / 0/! 6$#')2'(&*3$'"73-$&+/" !"#$%&'(#%)")*+")(#,%-"./)"01'.%234%5"607.'8")

8)9':$,..;<'=1,#'>,"'?!>@8A'?)B$./

�x 2 R2 : 1 kxk 2

�x 2 R2 : 1 + x

21 x

22

6(1$&*4,.'+1$../05)'+1$$#'1;($&-).,/

!!!

!!

!"#$%&'('!"#$%"$#&' !"#$%&'()*+$,$-%.*)*+,*-',.'#,*/01'203-'

! #4*'50'6-374*809':*+,*-;'0<8<'=" "*.+*+30'*:>507',.'-?4@0-'

� � �

≥ = ; ≥ ; ∈ Z

� � �

j j ( − ; )j j ≤ ( ); ∈ N

= proj!

∩!R + × Z

""=

#

∈ ∩Zproj

$∩ (R + × f g )

%

q(x1 � 2z)2 + x

22 1� 1/z,

z � 1, z 2 Z

!P1 P2 P3 P4

! A?070'01+-3'4*'+*B704-+*8'.:*B3+,*'h -:B?'3?43="" ?4-'!"#$%&'.,7>:C43+,*

! DE:4C'/,C:>0'! F+*+30'G',.'2?4@0-

Pz ✓ R2regular h(z)-gon centered at (z, 0)

S =

[1

z=1Pz is R-MICPR and periodic

!"#$%&'#(%)*+,-.%/012345607$%857%9"%:;<==>%?!#157@"A

! B"7$"%C'$81"#"%$"#

! -07DE"1'0C'8%!"#%0F%75#3154$%n

x 2 N :p2x�

jp2x

k

/2⇣

", 1�p2"⌘o

np2x�

jp2x

k

: x 2 No

✓ [0, 1]

k(z1, z1)k2 z2 + 1, k(z2, z2)k2 2z1, x1 = y1 � z2,

k(z1, z1)k2 y1, k(y1, y1)k2 2z1, z 2 Z2

k(x1, x1)k2 x2 + ",

k(x2, x2)k2 2x1 + 2", x 2 Z2+

GH0C%25C"%#("%'7#"@"1$I%544%"4$"%'$%#("%&01J%0F%257A%D ="0K04C%L107"8J"1

Why General Integer Formulations?

Answer: Clever techniques for ideal formulations allow us tosolve integer optimization problems in practice.

2 / 23

!"#$$$$$$$$$$$$$$$$$$$$$$$



Affiliations

Links


!"#$%&'()$*+$,-.$/$01231+14'35$0*67&($08&&9:8)$

;< =$>?

! @62*(15AB14$-B8(*7&B&35)$C!"#$%&'()&*'+'&*'&,DE"! 7;FG$C;HH;D$I 7;;$CG""?DE$-./000(1()8&&9:8

"! 7;$CG""HD$I 7JF!$CG";!DE$2345(1()8&&9:8$

! @6)*$4*37&K$3*3613&'(E"! 7JF"$CG";<D$I 7JF!$CG";!D$L:'9('514E$2426(1CMFN$O:33132N$P:4A&55&N$Q:R13N$G";!D

ILOG CPLEX 11.2

CPLEXCPLEX

!"#$%&'()*'+,#-$#./01,'2,013./#45

!"#$%&'()*'+,#-$#./01,'2,013./#45

7 84.(1(9(:'";

!"#"$%&'%()*%!&+,$-.

/0 1%23

! (#"4-$5%678$#-%#89%:4#9-#;<%=>&87<%:4#9-#;<1!?>*@">&AA$-<7#+5

" B?C$8%!&4-<$D

! EA$-F78F5%>&8,$G%H&8+78$#-%=$IFI%!J*@"?C$8K!&4-<$%L%>&AA$-<7#+%+78$#-%()*%!&+,$-%M%>&AA$-<7#+

CPLEX

!"#$%&'()*'+,#-$#./01,'2,013./#45

SCIP >N> !"#$

!""#$$%&'()*+(,-./#0$(1()-2#..%&'(34&'54'#$

67 8(9:

! ;$#0<=0%#&2.>(4.'#?04%"(@-2#..%&'(.4&'54'#$(A!)3BC

! D4$E(4&2(F#0$4E%.#G(?5E("-@H.%"4E#2(A4&2(H-$$%?.>(H0-H0%#E40>B" 3-I<.#/#.(J8JKK($-./#0(-0(J-%&<LM(%&E#0HN4$#$(O(=04@#I-0P$(

! Q6$E(J#&E50>(!)3$C

R4$#2(-&(S#&#04.(34&'54'#(4&2(F#0$4T.#,E4&24.-&#(4&2(D4$E



Affiliations

Links


!"#$%&'($)*+%,*-.*/001(.23-4'551(.%6/(.)/.'#

! 78'(9#-)*:'%/(4%;*''<

! ='>'5-8'4%/$%3?@

! AB5-/$#%51C'%8+$D-(20/$5/EF%#$1(.#%51C'%

&2B-*$*/(G

! H/#+%$-%)#'%/(4%I14'%51E*/*+%

':-#+#$'0%J#8':1/51K'4%/(4%;*-($'(4L

! 7(5+%5/(.)/.'%E'#14'#%&2&MM2B-*$*/(%

$-%#:/5'%$-%"%,'$/;5-8<



Affiliations

Links

Back to top© 2013 Juan Pablo Vielma | Last updated: 02/29/2016 00:47:25 | Based on a template design by AndreasViklund ! 78'(9#-)*:'%/(4%;*''<

! 3-4'551(.%5/(.)/.'F%1($'*;/:'%

/(4%#-;$I/*'%':-#+#$'0%;-*%

-8$101K/$1-(

! H/#+%$-%)#'%/(4%/4>/(:'4%

! ?($'.*/$'4%1($-%N)51/

! &*'/$'4%/$%3?@%/(4%E'+-(4O

!"#$%&'()*+"#%&'*",-&"./&012#".*&3(445.1*6



Affiliations

Links


!"#$%&'()*"#%&'+",-&"./&012#".+&3(445.1+6

June 27-29, 2018.

Martin Biel, KTH • Oscar Dowson, U. of Auckland • Joaquim Dias Garcia, PSR & PUC-Rio • Hassan Hijazi, LANL • Jean-Hubert Hours, Artelys • Oliver Huber, UW-Madison • Joey Huchette, MIT • Ole Kroger, Uni Heidelberg • Benoît Legat, UCLouvain • Miles Lubin, Google • Guillaume Marques, U. de Bordeaux • Harsha Nagarajan, LANL • François Pacaud, CERMICS, ENPC • Abel Soares Siqueira, Federal University of Paraná • Julie Sliwak, RTE • Mohamed Tarek, UNSW Canberra • Matthew Wilhelm, U. of Connecticut • Ulf Worsøe, Mosek

Speakers

Sponsored by:

WORKSHOP

THE SECOND ANNUAL-dev

A Primer on Computing

for OPERATIONS RESEARCH

www.juliaopt.org/meetings/bordeaux2018

I n s t i t u t d e Mathémathiques de B o r d e a u x ____

789:

;"1.&<5..1.$=&>1?%@&!521."./&A(%6&B5,C%D%

!"#$%&'#$($)"*+,-./$0(./&(/+$1$2.#+345('+

! %&)6 *"7(-.$'4+8-9-8$,(./&(/+$:;<0=$! <",>+$(?'#3(8#-".$,(@+3'A

" )(#563"/B('+ 1$)(#5C4#2.#+39(8+

! <",>+3$-.#+39(8+'! <",>+3'A$6(D(3-#"ED,F$6(>-#"ED,! GH#+.'-".'A$<&7C9<I&(3+'ED,F$6",@%&)6ED,! !"J$($!&7KCLM< <4".'"3+*$43"D+8#N

OK2<LP00Q$<6C!<CRG;$6RC%GLST

!"#$%&'$()"$*&&&&&&&&&&&&&&+%,-$."/&



Affiliations

Links


#""0/122-3(4)()567,%62.,88354"92 #:0/122-3(4),;/$%<$%7.,8

!"#$%&$'&%()$$$$$$$$$$$$$$$*$$$$$$$$$$$$$$$$$$$$$$$$+



Affiliations

Links


,$#'-&%./$#0$

!"#$%&'&()#*'+,&'-.&%/+0&1+2'"#31)#*'+4*15"3/

3

PSRProvider of analytical solutions and consulting services in electricity and natural gas since 1987

Our team has more than 60 experts (17 PhDs, 31 MSc) in engineering, optimization, energy systems, statistics, finance, regulation, IT and environment analysis

!"#$%&'()&#*(+#,-&#

!6"1-7)*%+2%"1(8+

4)%)/318+

!"#$%&'"()'*+,-.(/&0'%&'(1%0*0$*0#'

!"#$%&'()!'*#+(

! 23(-'4,50*,-0'5(678459::;0*7<=>&,?:@$%@A$%50B" C,+'-D,"'@5>E@" F-$470&$@D,"'@G'$-%0%;>E@" H<%04'->E@ 6D1)GC(5,@#'-B" F$5F-0"D,"'@5>E@" I( Operated by Los Alamos National Security, LLC for the U.S. Department of Energy's NNSA

UNCLASSIFIED

LA-UR-17-24522

5

A Bit About LANL

6/11/2017 juliaopt.svg

file:///Users/carleton/Downloads/juliaopt.svg 1/1

ANSI LOVES

JuliaOpt

!"#$%&'()'(%&)*+",-./%0.%!&&

!"#$%&'()"(*

! 1.0+%234.0-#-5(."5%)06-.%(0%47".3%87.+9:" .7";/%)."5-%08%<+"#$=-#-5(."5"(3>%7;4%9'?9("('(-9%<507#=50.;>

!"#$%&'()*+,)&'-./*0'12$3)43152%,6.'!"#$/7%#)*

! 8*+,)"/9()*6:;&! 1</+9=!*1%46+>:;&! !"#$%!&"'()*+,-! ?)&>=6@,%:;&! A2&+/B%,/%+6?)&>*)$/%&.:;&

Sum-of-Squares Programming in Julia with JuMP

Benoît Legatú, Chris Coey†, Robin Deits†, Joey Huchette† and Amelia Perry†

ú UCLouvain, † MIT

Sum-of-Squares (SOS)Programming

Nonnegative quadratic forms intosum of squares

(x1, x2, x3)unique

p(x) =x

€Qx

x

21 + 2x1x2 + 5x

22 + 4x2x3 + x

23 =

x

€

Q

ccccccccccccccccca

1 1 01 5 20 2 1

R

dddddddddddddddddb

x

p(x) Ø 0 ’x Q ≤ 0≈∆ cholesky

(x1 + x2)2 + (2x2 + x3)2x

€Q

ccccccccca

1 1 00 2 1

R

dddddddddb

€ Q

ccccccccca

1 1 00 2 1

R

dddddddddb

x

Nonnegative polynomial into sumof squares

(x1, x2, x3)(x1, x1x2, x2) not unique

p(x) =X

€QX

x

21 + 2x

21x2 + 5x

21x

22

+4x1x22 + x

22

=X

€

Q

ccccccccccccccccca

1 1 01 5 20 2 1

R

dddddddddddddddddb

X

p(x) Ø 0 ’x Q ≤ 0≈= cholesky

(x1 + x1x2)2+(2x1x2 + x2)2 X

€Q

ccccccccca

1 1 00 2 1

R

dddddddddb

€ Q

ccccccccca

1 1 00 2 1

R

dddddddddb

X

When is nonnegativity equivalent to sum ofsquares ?Determining whether a polynomial is nonnegative isNP-hard.

Hilbert 1888 Nonnegativity of p(x) of n variablesand degree 2d is equivalent to sum of squares inthe following three cases:•

n = 1 : Univariate polynomials• 2d = 2 : Quadratic polynomials•

n = 2, 2d = 4 : Bivariate quarticsMotzkin 1967 First explicit example:

x

41x

22 + x

21x

42 + 1 ≠ 3x

21x

22 Ø 0 ’x

but is not a sum of squares.

Manipulating Polynomials

Two implementations: TypedPolynomials.jl andDynamicPolynomials.jl.One common independent interface:MultivariatePolynomials.jl.@polyvar y # one variable

@polyvar x[1:2] # tuple/vector

Build a vector of monomials:• (x2

1, x1x2, x

22):

X = monomials(x, 2)• (x2

1, x1x2, x

22, x1, x2, 1):

X = monomials(x, 0:2)

Polynomial variables

By hand, with an integer decision variable a and realdecision variable b:@variable(model, a, Int)@variable(model, b)p = a*x^2 + (a+b)*y^2*x + b*y^3

From a polynomial basis, e.g. the scaled monomial

basis, with integer decision variables as coe�cients:@variable(model,

Poly(ScaledMonomialBasis(X)),Int)

Polynomial constraints

Constrain p(x, y) Ø q(x, y) ’x, y such that x Ø0, y Ø 0, x + y Ø 1 using the scaled monomial basis:S = @set x >= 0 && y >= 0 && x + y >= 1@constraint(model, p >= q, domain = S,

basis = ScaledMonomialBasis)

Interpreted as:@constraint(model, p - q in SOSCone(),

domain = S,basis = ScaledMonomialBasis)

To use DSOS or SDSOS (Ahmadi, Majumdar 2017):@constraint(model, p - q in DSOSCone())@constraint(model, p - q in SDSOSCone())

SOS on algebraic domainThe domain S is defined by equalities forming analgebraic variety V and inequalities q

i

. We searchfor Sum-of-Squares polynomials s

i

such thatp(x)≠ q(x) © s0(x)+s1(x)q1(x)+ · · · (mod V )The Gröbner basis of V is computed the equationis reduced modulo V .

Dual value

The dual of the constraint is a positive semidefinite(PSD) matrix of moments µ. The extractatomsfunction attempts to find an atomic measure withthese moments by solving an algebraic system.

Sum-of-Squares extension

MathOptInterface.jl (MOI)MOI is an abstraction layer for mathematical op-timization solvers. A constraint is defined by a“function” œ “set” pair.

MOI extension: AbstractVectorFunction œSOS(X) (resp. WSOS(X)): SOS constraint without(resp. with) domain equipped with a bridge toAbstractVectorFunction œ PSD (resp. SOS(X)).

JuMPJuMP is a domain-specific modeling language formathematical optimization. It stores the problemdirectly (a cache can optionally be used) in thesolver using MOI.

JuMP extension: p(x) Ø q(x) and p(x) œ SOS()are rewritten into MOI SOS or WSOS constraints,e.g. x

2 + y

2 Ø 2xy is rewritten into [1, ≠2, 1] œSOS(x2

, xy, y

2). p(x) œ DSOS() (resp. SDSOS()) isrewritten into linear (resp. second-order cone) con-straints.

SumOfSquares.jl

Math

OptI

nter

face

.jl

MO

I

Bridging

Caching

Solvers: Mosek, CSDP, SCS...

Bridging Automatic reformulation of a constraintinto an equivalent form supported by the solver,e.g. quadratic constraint into second-order coneconstraint. In particular, reformulatesSOS/WSOS constraints into PSD constraints.An interior-point solver that natively supportsSOS and WSOS without reformulation to SDPusing the approach of (Papp, Yıldız 2017) isunder development.

Caching Cache of the problem data in case thesolver do not support a modification (can bedisabled). For instance, Mosek provides manymodification capabilities in the API but CSDPonly support pre-allocating and then loading thewhole problem at once.

.)/01234)5'2

!"#$%&'(")*+,#--.-/*0.'1*2(,,.3.(-*45(.6#-&%*.-*#*7%%89

:; <*=;

! >(?5#?-/@*A'%%".-/*#*BC#6&(D'%"*'1"(C/1*(E3'#&,%3*FG%.'3<!%6"#8%@:HIJK! !:*0%%8*(L*0("8*E)*M(%)*NC&1%O% L("*APQ+!*RI;

! +(3.?(-*6%3&".E%6*E)*D(,)-(S.#,3@

! A(,C?(-*#DD"(#&1@" 3D,.'*6(S#.-*.-'(*T3#L%U*D(,)1%6"(-3*V*6.3&"%?W%*?S%*.-'(*.-'%"5#,3*

(px(t), py(t))t2[0,1]

!"#$%&'()* !"#$%"&'(# +,(-"./(0&1203-%4/(0&

56 7189

:&"(/47;*3-"&/41<0&="(0&#

:&>*3#>">"/41?-00>@"&A1<0&=">"0&#

B)0"=1<044"#"0&1C1D*-/"&1"&1?/E*1D*A"0&#

F/3"/G4*#1C1H04I&0-"/4#1J {pi : [Ti, Ti+1] ! R2}Ni=1

min

p

XN

i=1||p000i (t)||2

s.t. p1(0) = X0, p0(0) = X 0

0, p00(0) = X 00

0

pN (1) = Xf , p0N (1) = X 0

f , p00N (1) = X 00

f

pi(Ti+1) = pi+1(Ti+1) 8i 2 {1, . . . , N � 1}p0i(Ti+1) = p0i+1(Ti+1) 8i 2 {1, . . . , N � 1}p00i (Ti+1) = p00i+1(Ti+1) 8i 2 {1, . . . , N � 1}_R

r=1[Arpi(t) br] for t 2 [Ti, Ti+1] 8i 2 {1, . . . , N � 1}

K:HL

!"#$%&#'()*'+'!",-).#'/.0'1'23"'#&"4#

56 7'58

9-*#&'9"/#-:%"';)%$2).<=1'#">).0#

?4&-3/%';)%$&-).<@=A'#">).0#

!"#$%&'(")*+,-"*.*/#,''0*1$)2

34 .*35

! 67*8&)$9&:(,#*;"<-":(;=*&>;(,%#"*"?")0*@*A*B7*;"%C*(&*&'(C

How can we solve MICONV?

Joint work with Russell Bent, Chris Coey, Iain Dunning,

Joey Huche@e, Lea Kapelevich, Miles Lubin, Emre

Yamangil, …

Polyhedral Outer-Approxima4on for MICONVA simple polyhedral outer approximation algorithm

mixed-integer linear optimization (MILP) solvers are powerful andstable, enabling practical cutting plane algorithms

polyhedral outer approximation allows leveraging this power for MICP

Build MILP OA model P by replacing S with a polyhedral relaxation

1: solve P, let ? be optimal solution2: if ? is ‘close’ to S then3: return ?

4: else5: find separating hyperplane ( , z)6: update P with cut h , i � z

7: end if

Chris Coey (MIT ORC) Conic OA for MICP SIAM Opt. 2017 4 / 19

min f(x)

s.t.

x 2 C

xi 2 Z i 2 I

P

Con$nuous Convex

Op$miza$on

min f(x)

s.t.

x 2 C

xi 2 Z i 2 I

Linear MIP

MICONV

min f(x)

s.t.

x 2 C

xi 2 Z i 2 I

min f(x)

s.t.

x 2 C

xi 2 Z i 2 I

• Linear MIP and Con$nuous Convex Op$miza$on Solvers > MICONV Solvers

X =!

∈ R : ( ) ≤ , ∀ ∈ ! "".

X =!

∈ R : ( ′) +∇ ( ′) ( − ′) ≤ , ∀ ′ ∈ R , ∈ ! "".

min f(x)

s.t.

x 2 C

xi 2 Z i 2 I

X =!

∈ R : ( ) ≤ , ∀ ∈ ! "".

X =!

∈ R : ( ′) +∇ ( ′) ( − ′) ≤ , ∀ ′ ∈ R , ∈ ! "".P

X =!

∈ R : ( ) ≤ , ∀ ∈ ! "".

X =!

∈ R : ( ′) +∇ ( ′) ( − ′) ≤ , ∀ ′ ∈ R , ∈ ! "".

Xk

Improving OA Algorithms for MICONV

1. May need large # of linear inequalities

2. MIP formulations can break gradient based continuous solvers

3. How to pick “good” linear inequalities

B =

!∈ { , } :

"

=

#−

$≤ −

%

1. Use extended formulaKons

2. Use Conic Solver

3. Use Conic Duality

https://rjlipton.wordpress.com

Problem Solu*on

Mixed Integer Conic Programming (MICP)

• closed convex cones– Linear, SOCP, rotated SOCP, SDP– Exponen>al cone, power cone, …– Spectral norm, rela>ve entropy, sum-of-squares, …

Mixed-integer conic form

min2RN

h , i : (M)

k � k 2 Ck 8k 2 [M]

xi 2 Z 8i 2 [I ]

CK+1, . . . , CM are polyhedral cones, e.g. R+, R�, {0}C1, . . . , CK are closed convex nonpolyhedral cones, e.g.

L second-order cone (epi k·k2)E exponential cone (epi cl per exp)

P positive semidefinite cone (S+ on S)

Assume that if M is feasible then its optimal value is attained



min2RN

h , i : (M)

k � k 2 Ck 8k 2 [M]

xi 2 Z 8i 2 [I ]







min2RN

h , i : (M)

k � k 2 Ck 8k 2 [M]

xi 2 Z 8i 2 [I ]






• Fast and stable interior point algorithms for con>nuous relaxa>on• Geometrically intui>ve conic duality guides linear inequality selec>on• Conic formula>on techniques usually lead to extended formula>ons – MINLPLIB2 instances unsolved since 2001 solved by re-write to MISOCP– SOCP disaggrega>on technique now standard (V., Dunning, HucheRe, Lubin, 2015):

kyk2 y0

y2i zi · y0 8i 2 [n]Xn

i=1zi y0 y2i zi · y0 8i 2 [n]

Xn

i=1zi y0

Pajarito: A Julia-based MICP Solver

MI-convex model:

CBF, Convex.jl, CVXPY, JuMP

MI-conic solver:

Pajarito

Continuous solver:

CSDP, ECOS,MOSEK, SCS, SDPA

MILP solver:

CBC, CPLEX, GLPK,Gurobi, MOSEK, SCIP

conic interface

conic interfacelinear/quadratic interface

(through JuMP)

Figure 1: Pajarito’s integration with MathProgBase.

coefficient vectors, variable and constraint cones expressed as lists of standardprimitive cones (1-dimensional vector sets) with corresponding ordered row in-dices, and a vector of variable types (each continuous, binary, or general integer).In addition to the basic linear cones (nonnegative, nonpositive, zero, and freecones), Pajarito recognizes three standard primitive nonpolyhedral cones intro-duced in section 1.2: exponential cones (see appendix A.1), second-order cones(see appendix A.2), and positive semidefinite cones (see appendix A.3).19

Friberg [2016] designed the Conic Benchmark Format (CBF) as a file for-mat originally to support mixed-integer second-order cone (SOCP) and positivesemidefinite cone (SDP) instances. In collaboration with Henrik Friberg, weextended the format to support exponential cones in Version 2, and developeda Julia interface ConicBenchmarkUtilities.jl to provide utilities for translatingbetween CBF and MathProgBase conic format.20 One may use Pajarito tosolve any instance in the Conic Benchmark Library (CBLIB), which containsthousands of benchmark problems from a wide variety of sources.

Lubin et al. [2016] demonstrate that all 333 known MI-convex instances inMINLPLib2 [Vigerske, 2018] are representable with linear, second-order, expo-nential, and power cones. Since a power cone constraint is representable withlinear and exponential cone constraints, Pajarito can be used to solve any ofthe MI-convex instances in MINLPLib2. We translated 115 instances from theMINLPLIB2 library to CBF and contributed them to CBLIB.21 Many of the

19As we note in appendix A.2, Pajarito also recognizes rotated second-order cones, but forsimplicity converts them to second-order cones during preprocessing.

20Pajarito’s extensive unit tests rely on small example instances loaded from CBF files.21Lubin et al. [2016] first translated these instances from the MINLPLIB2 library into Con-

vex.jl models. We used ConicBenchmarkUtilities.jl to translate these to CBF. The instances,available at github.com/mlubin/MICPExperiments, are 48 ‘rsyn’ instances, 48 ‘syn’ instances,6 ‘tls’ instances, 12 ‘clay’ instances, and the challenging ‘gams01’ instance.

21

• Early version solved gams01, tls5 and tls6 (MINLPLIB2)

<latexit sha1_base64="o3zegCXig6oKMdKvT/9PyPOk8S4=">AAAT/3ichVjNchu5EeZa+dllfnYdHXNBre0teYtiOKR+VgdWydK6Eq9kWbFkW2UNywXMgCRMDDCLwYikUXPILdfkJXJL5ZpHyTPkEXJJ9wxJaWYkLW1hMN1fN7ob3Q2QLJYisZ3Ofz57sPazn//il59/0fzVr3/z2y+/evi7t4lOTcDfBFpqc8FowqVQ/I0VVvKL2HAaMcnfsckh8t9dcZMIrc7tPOaDiI6UGIqAWiB9ePjgfz7jI6GcpSyV1GROEkm+dVuZO7sEmuSbQ20iavvdQQb0bp3e9oBTIfYGWdO3Ojap5M1vyDfEj1JpBdibRgqVB5lLLLVpwpMM2E3fb/pBJEIU2JDmqett7mQERBMtwX6AOD1BpJMiEjafcWO0yWdTo9UIZ8RZEXGykTwlGUGdC5XNYn2jp247c99mzjcaludMzy61ERCAfjBwe53M6ZgrUoQ3y2D9A60ioTYPDtDK3haMWzh4Hgw9HMjWTg+XWkFfPUNodxvG7R4M3T3E44zsdncq0M3vc70d1PtdGbzjdQrwKf1IjbB684/Hp0ebzw9fnaHM9g6MOyiI6NyS3u5uReLw4HAlsIv6e0sBXId4hek+DUNMoCSmwc1Q9e4PleGJNSKwPMwjdXb44pRsbLU77c7TfD2MUw/D8N3SyJ2FR4enx88vyIbXbe8uwHudnI8xWxq3XXU/l9p8+ers+RHZ2MN12t29TSrjMS107GIAUVF3EZHtPNw+09bqqMgDrsJVrn/46hEoyT+kPvEWk0eNxef0w8Mv/+uHOkgjrmwgaZJcep3YDhw1kNuSQ85DQkMQJ3TEL2GqaMSTgcvLNCNPgBISqBD4U5bk1KYfh0Od2ji1fa9ZkqeBFVcQ+L6zJuUtonSgoxjqlgkp7DxrkaFQVLaINSAh1Khf4CbcqOs33NPrtyEo1abveZ0OsKzhNhj3PZyPjVATmA5cJALYcegVGZbPYh6AxXxm3VKd0mpouArGWdnoc2/g0DtglTmOpnbM5KRKjRLoF+MWPFEswUkyj1gLqVZrmVPsOCrF1vEoHvMfy7TZEOJQJiU2omZuwjKVBgHsX1Imsqhi2cjQeCyCWUWjSJWwFSLTegI5lbSWldNadrsyLqAxdt2KxpTdRr9MeIDEgYslVKVQScU+KyafCgmcScEMuOogbEbMcD9pzJMWodAjp0k74pa2SKwTgTphAzF5bIswUDYyOlVhJRzxaBhLnQdpOYWFXJGBfcWnUPtgUNkkRS0TrOyGwtqDhgED8IQKKwmT9x2VRowbHrbIFb7DE4s1hMKSI+g5sP1dXnE/NeXgXublBNKTpD+kMuEDN4a8NYYPs+YT4oPJAfhpuXEBxAaeUJDEaojDjNgxJywdEQGt/+ojNP9exQfKmOFXUMmSw2mEKv1cO8QkjR2UcAzlY5OASiyOE63Q3CeVDzgAeMnVyI6dDwkDJsMimeu1ezFE04eCQiujiEKT8mHr6HxRo5nz2t2srIAyfcUXqZNMBOjZzbXcgDAu9bQE6SAk31MssBU0olDnzo/mdCaSqQgtrNhpb+36WPTF+08IjbkYjS1K9apCq+xpYvDmBOF/aNMYTtoQmtBcQouD/pJYgkhsdATKFtoY5G8z19O/aRkQi8X6paVbTa6gqY94fkFYJAEUAuwJJ1rJYt1Wcx7Rjxr79TJRWs0ZnMV9r0Xm+Ozgk87gvTlDlKQQxaWZFDoe9E+4H9gx4RRqoNWE/9d+5S5UnUNvsGOCOZjepPCp2yKhoVOisaXaeb/T3kZtsDs3syBlcFBdQlf1oUi5KXoX7Cxe1R55cPCW0VbIkN+CL+iFBNy0IIXxbHL8xzS/CGYO/gHjjNuj6SFoQ6HFM3OPfWgqVI0kNxjpxz5BNIIXkDM7XxCOpiCmrFAprBYsZ0vWAVxNJ5lj+QOWi+iEU8gmC+WYO8LVFfRPlS8f6ZDL3JUmgY8PjXJpblJQQj7094MUalzZfJvQv2a2wMMGlESy5Xoayt3UVh/twy3I6v08q/39odTQzfQMLjnc+hhEcF6NqjpA2tcQ2ii2cwQuJwVnRb6iBhrhGBU0/e853B0Mfwm19Ap6CCgzzg8kxAtb2q1cFkLUwru4hiqIaj7epV2rq3w/ru5BFDvG70KEOsocDnfweSwyh8MdfGiYOnP5eAcCJpnD4a4g4V0BvjjgeBcCWiIgcKwUEmOvs0u43vjYwxhzr6u1w9j7EuB9HXBWApxl1TXESQGQkuGdjFtyQnxjFi8VdeJFFfziHvBxFXx8D/ioCj66B/xDFfzDfeBJDf1hcg/+JKniVQldhisNKeRrDcfoSEHGiJDnvZI8Fo+v36B9+Xg/uGxv89nAeXyWufYWHGy1/QgurjcMzgF3Ud3S4KQMOKkB3pcB72trHJUBR1UNRwsNruPXWMdl2eMa4HUZ8LYGOC0DzmqA52XA85r9jBcIFjley3i74tka73zFO6/L0XTF9fGlCpiu2NMaD1rMSlZHfERriHcrwLsab7bizWq8+Yo3r/E+rXifajy54ska79rRmpdw6BbM4liuC1uRlgC3aGBlFQsb6rC0BktrMAbNeRVZmFfZwYob1HvfisdqvGcr3rMq7xz2vsg+c1ueRDfSJKpnibrBVmktdeHbaXyd3vD1b+Je1goYvu4EFdBhFRRrOa9gTjP8ecCr/hhQn7zttr1O2/vz1qP9g8UPBZ83ft/4urHR8Bq7jf3GnxqnjTeNYI2t/XXtb2t/X//L+j/W/7n+rwL64LOFzHqj9Fn/9/8BqfVBPA==</latexit>

Performance for MISOCP Instances (120 from CBLIB)

Exponential Cone + LP / SOCP / SDP

• Discrete experimental design

• Por:olio Op;miza;on with entropic risk constraints• All 333 MICONVs from MINLPLIB2• Pajarito with SCS or Mosek (version 7.5.2)

Fig. 5.1: The boundary of the exponential cone Kexp

. The red isolines are graphs of x2

!x2

log(x1

/x2

) for fixed x1

, see (5.3).

5.2 Modeling with the exponential cone

Extending the conic optimization toolbox with the exponential cone leads to new types ofconstraint building blocks and new types of representable sets. In this section we list thebasic operations available using the exponential cone.

5.2.1 ExponentialThe epigraph t � ex is a section of K

exp

:

t � ex () (t, 1, x) 2 Kexp

. (5.4)

5.2.2 LogarithmSimilarly, we can express the hypograph t log x, x � 0:

t log x () (x, 1, t) 2 Kexp

. (5.5)

5.2.3 EntropyThe entropy function H(x) = �x log x can be maximized using the following representationwhich follows directly from (5.3):

t �x log x () t x log(1/x) () (1, x, t) 2 Kexp

. (5.6)

39

! ⟶ log det )*+,

-!*.*.*/

Chapter 5

Exponential cone optimization

So far we discussed optimization problems involving the major “polynomial” families of cones:linear, quadratic and power cones. In this chapter we introduce a single new object, namelythe three-dimensional exponential cone, together with examples and applications. The ex-ponential cone can be used to model a variety of constraints involving exponentials andlogarithms.

5.1 Exponential cone

The exponential cone is a convex subset of R3 defined as

Kexp

=�

(x1

, x2

, x3

) : x1

� x2

ex3/x2 , x2

> 0 [

{(x1

, 0, x3

) : x1

� 0, x3

0} . (5.1)

Thus the exponential cone is the closure in R3 of the set of points which satisfy

x1

� x2

ex3/x2 , x1

, x2

> 0. (5.2)

When working with logarithms, a convenient reformulation of (5.2) is

x3

x2

log(x1

/x2

), x1

, x2

> 0. (5.3)

Alternatively, one can write the same condition as

x1

/x2

� ex3/x2 , x1

, x2

> 0,

which immediately shows that Kexp

is in fact a cone, i.e. ↵x 2 Kexp

for x 2 Kexp

and ↵ � 0.Convexity of K

exp

follows from the fact that the Hessian of f(x, y) = y exp(x/y), namely

D2(f) = ex/y

y�1 �xy�2

�xy�2 x2y�3

�

is positive semidefinite for y > 0.

38

Chapter 5

Exponential cone optimization

So far we discussed optimization problems involving the major “polynomial” families of cones:linear, quadratic and power cones. In this chapter we introduce a single new object, namelythe three-dimensional exponential cone, together with examples and applications. The ex-ponential cone can be used to model a variety of constraints involving exponentials andlogarithms.

5.1 Exponential cone

The exponential cone is a convex subset of R3 defined as

Kexp

=�

(x1

, x2

, x3

) : x1

� x2

ex3/x2 , x2

> 0 [

{(x1

, 0, x3

) : x1

� 0, x3

0} . (5.1)

Thus the exponential cone is the closure in R3 of the set of points which satisfy

x1

� x2

ex3/x2 , x1

, x2

> 0. (5.2)

When working with logarithms, a convenient reformulation of (5.2) is

x3

x2

log(x1

/x2

), x1

, x2

> 0. (5.3)

Alternatively, one can write the same condition as

x1

/x2

� ex3/x2 , x1

, x2

> 0,

which immediately shows that Kexp

is in fact a cone, i.e. ↵x 2 Kexp

for x 2 Kexp

and ↵ � 0.Convexity of K

exp

follows from the fact that the Hessian of f(x, y) = y exp(x/y), namely

D2(f) = ex/y

y�1 �xy�2

�xy�2 x2y�3

�

is positive semidefinite for y > 0.

38

https://themosekblog.blogspot.com/2018/05/new-modeling-cookbook.html

or

Hypatia: Pure Julia-based IPM Beyond “Standard” Cones

• Extension of methods in CVXOPT and Alfonso– A customizable homogeneous interior-point solver

for nonsymmetric convex– Skajaa and Ye ‘15, Papp and Yıldız ‘17, Andersen,

Dahl, and Vandenberghe ‘04-18

• Cones: LP, dual Sum-of-Squares, SOCP, RSOCP, 3-dim exponential cone, PSD, L∞, n-dim power cone (using AD), spectral norm, …

• Potential: – flexible number types and linear algebra– BOB: bring your own barrier (in ∼50 lines of code)– Alternative prediction steps (Runge–Kutta)

Chris Coey

!"#$%&'()*"#+,(-&.+/0&1$2(-,(&2(#&34&"-5&676

189&:;189&<=

!"#$%!&'( )!"#$%&$'()* +#$,-.'

/&*&0'1

/&0'*('.

/&*6$4'%.&0'

*+#$,%&-&#$.%/

!"#$%&$'()*")+',$-.',$(/)01&$%'!"#+0#1. -2,&/-31"*&7'0'8*."$%*$%%3'*&0*932*:*;*<*!&66'0.%

617+&//81

9//&:.11/

;$<1)/

>,3(1-%/

?&)1/%3.1

63%&=&-$%&3./

+!&66'0.'8*&0*932*:*=*

#300$02*945&0%&* !"#$%"& *>$."*8'5-34.*&7.$&0%*-2-$0%.*."'*5&44&>$02*945&0%&?6*@-.4-A*!&8'*>$."*B."'%-6'C*8'5-34.*&7.$&0%D

7#$6-4*-08*83-4*&AE'!.$,'%*5&#*."'*F34$-*!&8'*-08*@-.4-A*!&8'*6-.!"?*."'*$.'#-.$&0*!&30.%*B,'#(*%$6$4-#C-08*.$6$02%*B$0*%'!&08%C*-#'D

%1/% &%1,/ ?$%)$< !"#$%"& #P&.($" .KK$J,,

8'0%'*47 G: :?H I?J K?L< J?K:

'0,'4&7' <L L?LH: L?LI< L?LKL M

A3.!"'# <K)<L L?G< L?IJ L?<:N L?J<G

!-7#-%%' <J)<L J?<H J?HN J?HL L?:<L

4&.O-P,&4. <J)<L L?IN L?<H L?<N L?JLI

6&.QO$0 IJ)IK L?<: L?KI M L?L:I

#'-!P8$55 KR)<L L?<K L?K< L?JR L?LN:

#&A$0%&0 KR L?<I L?K< L?JN L?L<I

'8$.'8*

*-2,&/-31"*-88'8* /#115 * &.=3 *4-A'4%*&0*932*:

*-2,&/-31"*!"-02'8*."'*.$.4'*5#&6*?$%)$<+-351+-3@#$,&/3.*.&*A&.=3B+?$%)$<+-351+-3@#$,&/3.&0*932*:

*-2,&/-31"*#'5'#'0!'8*."$%*$%%3'*&0*932*G

189&>

3+-?"#&7*@)+A"@(-4($%-()+"$&!-B?$(*?

4($%-()+"$&C+-+)+A"/+(-

D+#,/&E%*"@"&F())+/&G&H8$&<>

First Batch of Tests on CBLIB Instances

• Instances:– SDP– SOCP– RSOCP

• Only 2 – 10K @mes slower than Mosek 8!

0.001

0.01

0.1

1

10

100

1000

0 5 10 15 20 25 30 35 40 45

Tim

e s

Size

hypatia_time mosek_time

Summary

• MICONV can model many problems (but not all)• How to solve MICONVs? Don’t, solve MICPs• Easy access to optimization modeling and solvers with JuMP• Advanced solver development with Julia• Disclaimers:– Julia just reached version 1 ( Yay! )–… JuMP is undergoing a major redesign • Try in Julia 1.0 through “] add JuMP#v0.19-alpha”

modeling power of mixed integer convex optimization problems...

Documents