an introduction to cvx - kthamishi/pdf/presentation_nov_2014.pdf · whatiscvx? • cvxis a...

An Introduction to CVX

Amirpasha Shirazinia

Signals and Systems Division

Department of Engineering Sciences

Uppsala University

November 4, 2014

Amirpasha Shirazinia (UU) An Introduction to CVX Internal Seminar 1 / 17

What is CVX ?

• CVX is a MATLAB–based software package for solving convexoptimization problems.

• Feature: CVX is written in a high–level form, i.e., close tomathematical notations.

• 1748 citations (according to google).

What is CVX ?

• Restriction: can be applied to convex programs,

• What if the problem you are dealing with is non-convex:• Try to convexify the non-convex problem,• Try to relax the problem in order to make it convex,• Go for a sub-optimal locally minimizing approach, e.g., fmincon.

What is CVX ?

• Restriction: can be applied to convex programs,

• What if the problem you are dealing with is non-convex:• Try to convexify the non-convex problem,• Try to relax the problem in order to make it convex,• Go for a sub-optimal locally minimizing approach, e.g., fmincon.

• Download & install CVX .

Scalar Example

minimize (x+ y + 3)2

subject to y ≥ 00 ≤ x ≤ 1.

How to translate Problem (1) into CVX ?

Scalar Example

How to translate Problem (1) into CVX ?>> cvx begin

Scalar Example

>> variables x y

Scalar Example

>> variables x y

>> y >= 0

Scalar Example

>> variables x y

>> y >= 0

>> x >= 0

Scalar Example

>> variables x y

>> y >= 0

>> x >= 0

>> x <= 1

Scalar Example

>> variables x y

>> y >= 0

>> x >= 0

>> x <= 1

>> minimize((x+ y+ 3)2)

Scalar Example

>> variables x y

>> y >= 0

>> x >= 0

>> x <= 1

>> minimize((x+ y+ 3)2)>> cvx end

Some Useful Commands & Hints

• cvx status: gives the status of the solution –solved/infeasible/unbounded,

• cvx optval: gives the optimal value,

• cvx quiet(true): displays off the optimization procedure,

• Affine equality constraints are denoted by ==,

• Flexibility: CVX commands can be used in MATLAB loops,

• Lots of other hints available in CVX user’s guide.

Vector Example

• Problem 4.8e (Boyd): Finding optimal probabilities for some weightingcoefficients {αi}Ni=1

minimize∑N

i=1αipi

subject to pi ≥ 0 , ∀i∑N

i=1pi = 1.

Vector Example

minimize∑N

i=1αipi

i=1pi = 1.

>> N = 10; alpha = rand(1, N)

Vector Example

minimize∑N

i=1αipi

i=1pi = 1.

>> N = 10; alpha = rand(1, N)>> cvx begin

Vector Example

minimize∑N

i=1αipi

i=1pi = 1.

>> variable p(N)

Vector Example

minimize∑N

i=1αipi

i=1pi = 1.

>> variable p(N)>> p >= 0

Vector Example

minimize∑N

i=1αipi

i=1pi = 1.

>> sum(p) == 1

Vector Example

minimize∑N

i=1αipi

i=1pi = 1.

>> sum(p) == 1

>> minimize(alpha ∗ p)

Vector Example

minimize∑N

i=1αipi

i=1pi = 1.

>> sum(p) == 1

>> minimize(alpha ∗ p)>> cvx end

Vector Example

• Problem 4.26 (Boyd):

minimize x⊤x+ y + zsubject to x⊤x ≤ yz

y ≥ 0 , z ≥ 0(3)

• Question 1: Is the objective function convex?

Vector Example

y ≥ 0 , z ≥ 0(3)

• Question 1: Is the objective function convex? Yes! ,

• Question 2: Are the constraints convex sets?

Vector Example

y ≥ 0 , z ≥ 0(3)

• Question 2: Are the constraints convex sets? Let’s call CVX first �

>> N = 10;>> cvx begin

>> variables x(N, 1) y z

>> y >= 0

>> z >= 0

>> x′ ∗ x− y ∗ z <= 0

Vector Example

y ≥ 0 , z ≥ 0(3)

• Question 2: Are the constraints convex sets? Let’s call CVX first �

>> N = 10;>> cvx begin

>> y >= 0

>> z >= 0

>> x′ ∗ x− y ∗ z <= 0

Disciplined convex programming error:

Vector Example (Cont’d)

• Why the constraint x⊤x ≤ yz does not form a convex region?

• convex ≤ ???

• convex ≤ ???• ??? should be an affine or concave function.

• convex ≤ ???• ??? should be an affine or concave function.• Now, via the Hessian matrix, check if f(y, z) = yz is concave (Problem

3.16 Boyd).

• How to convexify Problem (2)?

x⊤x ≤ yz ⇐⇒

∥∥∥∥[

2xy − z

]∥∥∥∥2

≤ y + z. (4)

3.16 Boyd).

• How to convexify Problem (2)?

x⊤x ≤ yz ⇐⇒

∥∥∥∥[

2xy − z

]∥∥∥∥2

≤ y + z. (4)

• Now, we have a convex problem ,

minimize x⊤x+ y + z

subject to

∥∥∥∥[

2xy − z

]∥∥∥∥2

≤ y + z

y ≥ 0 , z ≥ 0

• CVX code:>> N = 10;>> cvx begin

>> y >= 0

>> z >= 0

>> norm([2x; y− z], 2) <= y+ z

>> minimize(x′ ∗ x+ y+ z)>> cvx end

Matrix Example: Sensing/Measurement Matrix design in a

Bayesian Framework

• Consider the linear Gaussian model y = Ax+ n,

• where x ∼ N (0,R), n ∼ N (0, σ2I), such that x,y,n ∈ RN and

A ∈ RN×N .

• The MMSE estimator of x given the noisy measurements y becomes

xmmse = arg minx

E[‖x− x‖22] =

(R−1 +

σ2A⊤A

• The corresponding MSE, to be minimized, is

MSE = Tr

{(R−1 +

σ2A⊤A

)−1}

Matrix Example: Sensing/Measurement Matrix design in a

Bayesian Framework

• Consider the linear Gaussian model y = Ax+ n,

• where x ∼ N (0,R), n ∼ N (0, σ2I), such that x,y,n ∈ RN and

A ∈ RN×N .

• The MMSE estimator of x given the noisy measurements y becomes

xmmse = arg minx

E[‖x− x‖22] =

(R−1 +

σ2A⊤A

• The corresponding MSE, to be minimized, is

MSE = Tr

{(R−1 +

σ2A⊤A

)−1}

• Assume the total available power P > 0, we have

E[‖Ax‖22] = . . . = Tr

{ARA⊤

}≤ P (7)

Matrix Example (Cont’d)

• The sensing/measurement matrix design problem is posed as

minimize Tr{(

R−1 + 1

σ2A⊤A

)−1}

subject to Tr{RA⊤A

}≤ P

• The sensing/measurement matrix design problem is posed as

minimize Tr{(

R−1 + 1

σ2A⊤A

)−1}

subject to Tr{RA⊤A

}≤ P

• Problem (8) is not convex in A, but convex in A⊤A.

• Let X , A⊤A, we have an equivalent optimization problem

minimize Tr{(

R−1 + 1

σ2X)−1

subject to Tr {RX} ≤ PX � 0

• Let’s translate the optimization problem (8) into CVX codes>> N = 3; rho = 0.5; P = 10; sigma sq = 0.01;>> R = [1 rho rho 2; rho 1 rho ; rho 2 rho 1];>> cvx begin sdp

>> variable X(N, N) symmetric>> X == semidefinite(N, N)>> trace(R ∗ X) <= P

>> minimize(trace inv(R (−1) + X/sigma sq))

• Let’s translate the optimization problem (8) into CVX codes>> N = 3; rho = 0.5; P = 10; sigma sq = 0.01;>> R = [1 rho rho 2; rho 1 rho ; rho 2 rho 1];>> cvx begin sdp

>> variable X(N, N) symmetric>> X == semidefinite(N, N)>> trace(R ∗ X) <= P

>> minimize(trace inv(R (−1) + X/sigma sq))>> cvx end

• How to get back A from X?• For example, use the cholesky factorization; in MATLAB A = chol(X).

A Little into Compressed Sensing (CS)

• CS answers the following question:• In a linear set of equations y = Ax, would it be possible to uniquely

solve the equations if the number of unknowns, i.e. dim(x), is largerthan the number of equations, i.e. dim(y)?

A Little into Compressed Sensing (CS)

• CS answers the following question:• In a linear set of equations y = Ax, would it be possible to uniquely

solve the equations if the number of unknowns, i.e. dim(x), is largerthan the number of equations, i.e. dim(y)?

• Yes! But under the condition that the unknown variables are knownto be sparse (K ≪ N) plus some mild conditions.

Optimization framework for CS

• We know that the source, x, is sparse.

• We observe the linear measurements y = Ax.

• Goal: Go for the sparsest solution.

minimize ‖x‖0subject to y = Ax

• where the ℓ0–norm ‖x‖0 , card(x) = #non-zero components.

• Check whether or not the optimization problem (10) is convex ...• Hint: Check the geometry of ℓ0 ...

• Goal: Go for the sparsest solution.

• where the ℓ0–norm ‖x‖0 , card(x) = #non-zero components.

• Check whether or not the optimization problem (10) is convex ...• Hint: Check the geometry of ℓ0 ...• NOT CONVEX!

Optimization framework for CS (Cont’d)

• How to tackle the problem?

• Standard approach: Relax the objective function to the nearestconvex norm, i.e., ℓ1–norm, a.k.a. basis pursuit (BP),

• where ‖x‖1 =∑

n=1|xn|.

Optimization framework for CS (Cont’d)

• How to tackle the problem?

• Standard approach: Relax the objective function to the nearestconvex norm, i.e., ℓ1–norm, a.k.a. basis pursuit (BP),

• where ‖x‖1 =∑

n=1|xn|.

• The miracle happens: Under some conditions, the solutions to (11)and (10) are the same. CS theory also determines how manymeasurements need to be acquired in order for this equivalence tohold.

Optimization framework for CS: Example

>> N = 50; M = 25; K = 5;>> supp = randsample(N, K)′; x = zeros(N, 1); x(supp) = randn(K, 1);>> A = randn(M, N); s = sqrt(sum(A.2)); S = diag(1./s);>> A = A ∗ S;>> y = A ∗ x;>> cvx begin

>> variable x hat(N, 1)>> y == A ∗ x hat

>> minimize(norm(x hat, 1))>> cvx end

>> plot(1 : N, x, 1 : N,′ s′, x hat,′ o′)

Conclusion

• CVX is cool!

References

• S. Boyd and L. Vandenberghe, Convex optimization, CambridgeUniversity Press, 2004.

• http://cvxr.com/cvx.

• A. Shirazinia, Source and channel coding for compressed sensing and

control, PhD thesis, KTH, 2014.

an introduction to cvx - kthamishi/pdf/presentation_nov_2014.pdf · whatiscvx? • cvxis a...

Documents

2008.cvx lge 5.08

calculos cvx rig9

matlab programs for convex optimization and euclidean...

fom ndash; a matlab toolbox of first-order methods for...

unlocbox: matlab convex optimization toolbox...

convex hull calculations: matlab - uc · convex hull...

cvx user guide

convex-optimization-supplementinfoshako.sk.tsukuba.ac.jp/.../course/dc/cvx-handout.pdf(参考文献：h....

cvx uruguay

tutorial on antenna current optimization using matlab...

tutorial on antenna current optimization using matlab and...

convex optimization - university of...

sauter - case · 2019. 8. 12. · maxxum 115 - 140 ep / mc...

convex sets - university of colorado...

motivación asamblea cvx 2012

constructive convex analysis [0.5ex] and disciplined...

the cvx users’ guidecs709/notes/code/cvx/doc/cvx.pdfthe...

principi generali - cvx

nuestro carisma julio - 98. nuestro carisma i. la persona...

type cvx and cvx-1 synchro-verifier relays for class 1e...