convex functions (i)vector composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom...
TRANSCRIPT
![Page 2: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/2.jpg)
Outline Basic Properties
Definition First-order Conditions, Second-order Conditions Jensenโs inequality and extensions Epigraph
Operations That Preserve Convexity Nonnegative Weighted Sums Composition with an affine mapping Pointwise maximum and supremum Composition Minimization Perspective of a function
Summary
![Page 3: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/3.jpg)
Outline Basic Properties
Definition First-order Conditions, Second-order Conditions Jensenโs inequality and extensions Epigraph
Operations That Preserve Convexity Nonnegative Weighted Sums Composition with an affine mapping Pointwise maximum and supremum Composition Minimization Perspective of a function
Summary
![Page 4: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/4.jpg)
Convex Function
is convex if is convex
![Page 5: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/5.jpg)
Convex Function
is convex if is convex
is strictly convex if
![Page 6: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/6.jpg)
Convex Function
is convex if is convex
is concave if is convex is convex
Affine functions are both convex and concave, and vice versa.
![Page 7: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/7.jpg)
Extended-value Extensions
The extended-value extension of is
Example
![Page 8: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/8.jpg)
Extended-value Extensions
The extended-value extension of is
Example Indicator Function of a Set
![Page 9: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/9.jpg)
Zeroth-order Condition
Definition High-dimensional space
A function is convex if and only if it is convex when restricted to any line that intersects its domain. , is convex is convex One-dimensional space
![Page 10: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/10.jpg)
First-order Conditions
is differentiable. Then is convex if and only if is convex For all
๐ ๐ฆ ๐ ๐ฅ ๐ป๐ ๐ฅ ๐ฆ ๐ฅ
First-order Taylor approximation
![Page 11: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/11.jpg)
First-order Conditions
is differentiable. Then is convex if and only if is convex For all
Local Information Global Information
is strictly convex if and only if
๐ ๐ฆ ๐ ๐ฅ ๐ป๐ ๐ฅ ๐ฆ ๐ฅ
๐ ๐ฆ ๐ ๐ฅ ๐ป๐ ๐ฅ ๐ฆ ๐ฅ
![Page 12: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/12.jpg)
Proof
is convex
Necessary condition:๐ ๐ฅ ๐ก ๐ฆ ๐ฅ 1 ๐ก ๐ ๐ฅ ๐ก๐ ๐ฆ , 0 ๐ก 1
โ ๐ ๐ฆ ๐ ๐ฅโ
๐ ๐ฆ ๐ ๐ฅ ๐ ๐ฅ ๐ฆ ๐ฅ
Sufficient condition:๐ง ๐๐ฅ 1 ๐ ๐ฆ
๐ ๐ฅ ๐ ๐ง ๐ ๐ง ๐ฅ ๐ง๐ ๐ฆ ๐ ๐ง ๐ ๐ง ๐ฆ ๐ง
โ ๐ ๐ฅ ๐ ๐ง 1 ๐ ๐ ๐ง ๐ฅ ๐ฆ๐ ๐ฆ ๐ ๐ง ๐๐ ๐ง ๐ฅ ๐ฆ
โ ๐๐ ๐ฅ 1 ๐ ๐ ๐ฆ ๐ ๐ง โ ๐ ๐๐ฅ 1 ๐ ๐ฆ ๐๐ ๐ฅ 1 ๐ ๐ ๐ฆ
![Page 13: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/13.jpg)
Proof
is convex
is convex
๐ is convex โ is convex โ ๐ 1 ๐ 0๐ 0 โ ๐ ๐ฆ ๐ ๐ฅ ๐ป๐ ๐ฅ ๐ฆ ๐ฅ
๐ ๐ก ๐ ๐ก๐ฆ 1 ๐ก ๐ฅ , ๐โฒ ๐ก ๐ป๐ ๐ก๐ฆ 1 ๐ก ๐ฅ ๐ฆ ๐ฅ
๐ is convex
๐ is convex
First-order condition of ๐
First-order condition of ๐
![Page 14: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/14.jpg)
Proof
is convex
is convex
โ ๐ ๐ก ๐ ๐ก ๐ ๐ก ๐ก ๐ก โ ๐ ๐ก is convex โ๐ is convex
๐ ๐ก ๐ ๐ก๐ฆ 1 ๐ก ๐ฅ ,
๐ ๐ก๐ฆ 1 ๐ก ๐ฅ ๐ ๐ก๐ฆ 1 ๐ก ๐ฅ ๐ป๐ ๐ก๐ฆ 1 ๐ก ๐ฅ ๐ฆ ๐ฅ ๐ก ๐ก
๐โฒ ๐ก ๐ป๐ ๐ก๐ฆ 1 ๐ก ๐ฅ ๐ฆ ๐ฅ
๐ is convex
๐ is convex
First-order condition of ๐
First-order condition of ๐
![Page 15: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/15.jpg)
Second-order Conditions
is twice differentiable. Then is convex if and only if is convex For all ,
Attention is strictly convex is strict convex
is strict convex but is convex is necessary,
![Page 16: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/16.jpg)
Examples
Functions on is convex on ,
is convex on when or ,
and concave for
, for , is convex on
is concave on
Negative entropy is convex on
![Page 17: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/17.jpg)
Examples
Functions on Every norm on is convex
Quadratic-over-linear: dom ๐ ๐ฅ, ๐ฆ โ ๐ ๐ฆ 0
/ is concave on
is concave on
![Page 18: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/18.jpg)
Examples
Functions on Every norm on is convex
๐ ๐ฅ is a norm on ๐
๐ ๐๐ฅ 1 ๐ ๐ฆ ๐ ๐๐ฅ ๐ 1 ๐ ๐ฆ
๐๐ ๐ฅ 1 ๐ ๐ ๐ฆ
๐ ๐๐ฅ 1 ๐ ๐ฆ max ๐๐ฅ 1 ๐ ๐ฆ
๐max ๐ฅ 1 ๐ max ๐ฆ
![Page 19: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/19.jpg)
Examples
Functions on
๐ป ๐ ๐ฅ, ๐ฆ ๐ฆ ๐ฅ๐ฆ๐ฅ๐ฆ ๐ฅ
๐ฆ๐ฅ ๐ฆ
๐ฅ โฝ 0
![Page 20: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/20.jpg)
Examples
Functions on
![Page 21: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/21.jpg)
Examples
Functions on
๐ป ๐ ๐ฅ๐
๐ ๐ง diag ๐ง ๐ง๐ง
๐ง ๐ , โฆ ๐
๐ฃ ๐ป ๐ ๐ฅ ๐ฃ๐
โ ๐ง โ ๐ฃ ๐ง
โ ๐ฃ ๐ง 0
Cauchy-Schwarz inequality: ๐ ๐ ๐ ๐
๐ ๐
![Page 22: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/22.jpg)
Examples
Functions on is concave on
๐ ๐ก ๐ ๐ ๐ก๐ , ๐ ๐ก๐ โป 0, ๐ โป 0
๐ ๐ก log det ๐ ๐ก๐
log det ๐ ๐ผ ๐ก๐ ๐๐ ๐
โ log 1 ๐ก๐ log det ๐
๐ , โฆ ๐ are the eigenvalues of ๐ ๐๐
๐ ๐ก โ , ๐ ๐ก โ
det ๐ด๐ต det ๐ด det ๐ต https://en.wikipedia.org/wiki/Determinant
![Page 23: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/23.jpg)
Sublevel Sets
-sublevel set
is convex is convex is convex is convex
-superlevel set
is concave convex
is convex
๐ถ ๐ฅ โ dom ๐ ๐ ๐ฅ ๐ผ
๐ถ ๐ฅ โ dom ๐ ๐ ๐ฅ ๐ผ
![Page 24: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/24.jpg)
Epigraph
Graph of function
Epigraph of function
![Page 25: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/25.jpg)
Epigraph
Epigraph of function
Hypograph
Conditions is convex is convex is concave is convex
![Page 26: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/26.jpg)
Example
Matrix Fractional Function
Quadratic-over-linear:
Schur complement condition is convex Recall Example 2.10 in the book
๐ ๐ฅ, ๐ ๐ฅ ๐ ๐ฅ, dom ๐ ๐ ๐
![Page 27: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/27.jpg)
Application of Epigraph
First order Condition ๐ ๐ฆ ๐ ๐ฅ ๐ป๐ ๐ฅ ๐ฆ ๐ฅ
๐ฆ, ๐ก โ epi ๐ โ ๐ก ๐ ๐ฆ ๐ ๐ฅ ๐ป๐ ๐ฅ ๐ฆ ๐ฅ
![Page 28: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/28.jpg)
Application of Epigraph
First order Condition ๐ ๐ฆ ๐ ๐ฅ ๐ป๐ ๐ฅ ๐ฆ ๐ฅ
๐ฆ, ๐ก โ epi ๐ โ ๐ก ๐ ๐ฅ ๐ป๐ ๐ฅ ๐ฆ ๐ฅ
๐ฆ, ๐ก โ epi ๐ โ ๐ป๐ ๐ฅ1
๐ฆ๐ก
๐ฅ๐ ๐ฅ 0
![Page 29: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/29.jpg)
Jensenโs Inequality
Basic inequality
points
![Page 30: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/30.jpg)
Jensenโs Inequality
Infinite points
๐ ๐ฅ ๐๐ ๐ฅ ๐ง , ๐ง is a zero-mean noisy
Hรถlderโs inequality
![Page 31: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/31.jpg)
Outline Basic Properties
Definition First-order Conditions, Second-order Conditions Jensenโs inequality and extensions Epigraph
Operations That Preserve Convexity Nonnegative Weighted Sums Composition with an affine mapping Pointwise maximum and supremum Composition Minimization Perspective of a function
Summary
![Page 32: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/32.jpg)
Nonnegative Weighted Sums Finite sums ๐ค 0, ๐ is convex ๐ ๐ค ๐ โฏ ๐ค ๐ is convex
Infinite sums ๐ ๐ฅ, ๐ฆ is convex in ๐ฅ, โ๐ฆ โ ๐, ๐ค ๐ฆ 0 ๐ ๐ฅ ๐ ๐ฅ, ๐ฆ ๐ค ๐ฆ๐ ๐๐ฆ is convex
Epigraph interpretation ๐๐ฉ๐ข ๐ค๐ ๐ฅ, ๐ก |๐ค๐ ๐ฅ ๐ก
๐ผ 00 ๐ค ๐๐ฉ๐ข ๐ ๐ฅ, ๐ค๐ก |๐ ๐ฅ ๐ก
๐๐ฉ๐ข ๐ค๐ ๐ผ 00 ๐ค ๐๐ฉ๐ข ๐
![Page 33: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/33.jpg)
Composition with an affine mapping
Affine Mapping
If is convex, so is
If is concave, so is
๐ ๐ฅ ๐ ๐ด๐ฅ ๐
![Page 34: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/34.jpg)
Pointwise Maximum
is convex
is convex with
max ๐ ๐๐ฅ 1 ๐ ๐ฆ , ๐ ๐๐ฅ 1 ๐ ๐ฆmax ๐๐ ๐ฅ 1 ๐ ๐ ๐ฆ , ๐๐ ๐ฅ 1
๐ ๐ ๐ฆ๐ max ๐ ๐ฅ , ๐ ๐ฅ 1 ๐ max ๐ ๐ฆ , ๐ ๐ฆ๐๐ ๐ฅ 1 ๐ ๐ ๐ฆ
๐ , โฆ ๐ is convex โ ๐ ๐ฅ max ๐ ๐ฅ , โฆ ๐ ๐ฅ
![Page 35: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/35.jpg)
Examples
Piecewise-linear functions
Sum of largest components
is convex
Pointwise maximum of !! !
linear functions
![Page 36: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/36.jpg)
Pointwise Supremum
is convex in
is convex with
โ๐
Epigraph interpretation โ๐
Intersection of convex sets is convex Pointwise infimum of a set of
concave functions is concave
โ๐
![Page 37: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/37.jpg)
Examples
Support function of a set
โ
Distance to farthest point of a set
โ
![Page 38: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/38.jpg)
Examples
Maximum eigenvalue of a symmetric matrix
Norm of a matrix is maximum singular value
of
![Page 39: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/39.jpg)
Representation
Almost every convex function can be expressed as the pointwise supremum of a family of affine functions.
โน ๐ ๐ฅ sup ๐ ๐ฅ |๐ affine, ๐ z ๐ ๐ง โ๐ง
๐: ๐ โ ๐ is convex and dom ๐ ๐
![Page 40: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/40.jpg)
Compositions
Definition
Chain Rule
๐ ๐ฅ โ ๐ ๐ฅ
๐ป ๐ ๐ฅ โ ๐ ๐ฅ ๐ป ๐ ๐ฅ โ ๐ ๐ฅ ๐ป๐ ๐ฅ ๐ป๐ ๐ฅ
![Page 41: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/41.jpg)
Scalar Composition
and are twice differentiable
is convex, if
โ is convex and nondecreasing ๐ is convex
โ is convex and nonincreasing, ๐ is concave
๐ ๐ฅ โ ๐ ๐ฅ ๐โฒ ๐ฅ โ ๐ ๐ฅ ๐โฒโฒ ๐ฅ
![Page 42: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/42.jpg)
Scalar Composition
and are twice differentiable
is concave, if
โ is concave and nondecreasing ๐ is concave
โ is concave and nonincreasing, ๐ is convex
๐ ๐ฅ โ ๐ ๐ฅ ๐โฒ ๐ฅ โ ๐ ๐ฅ ๐โฒโฒ ๐ฅ
![Page 43: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/43.jpg)
Without differentiability assumption Without domain condition
โ ๐ฅ 0 with dom โ 1,2 , which is convex and non-decreasing
๐ ๐ฅ ๐ฅ with dom ๐ ๐, which is convex
dom ๐ 2, 1 โช 1, 2
๐ ๐ฅ โ ๐ ๐ฅ 0
Scalar Composition
![Page 44: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/44.jpg)
Without differentiability assumption Without domain condition
โ is convex, โ is nondecreasing, and ๐ is convex โ ๐ is convex
โ is convex, โ is nonincreasing, and ๐ is concave โ ๐ is convex
The conditions for concave are similar
Scalar Composition
![Page 45: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/45.jpg)
Extended-value Extensions
![Page 46: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/46.jpg)
Examples is convex is convex is concave and positive is
concave is concave and positive is
convex is convex and nonnegative and
is convex is convex is convex on
![Page 47: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/47.jpg)
Vector Composition
โ and ๐ are twice differentiable dom ๐ ๐, dom โ ๐
๐ โ โ ๐ โ ๐ ๐ฅ , โฆ , ๐ ๐ฅ
๐ ๐ฅ ๐ ๐ฅ ๐ป โ ๐ ๐ฅ ๐โฒ ๐ฅ ๐ปโ ๐ ๐ฅ ๐โฒโฒ ๐ฅ
๐โฒ ๐ฅ ๐ปโ ๐ ๐ฅ ๐โฒ ๐ฅ
![Page 48: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/48.jpg)
Vector Composition
โ and ๐ are twice differentiable dom ๐ ๐, dom โ ๐
๐ is convex, if ๐ ๐ฅ 0 โ is convex, โ is nondecreasing in each
argument, and ๐ are convex โ is convex, โ is nonincreasing in each
argument, and ๐ are concave
๐ โ โ ๐ โ ๐ ๐ฅ , โฆ , ๐ ๐ฅ
๐ ๐ฅ ๐ ๐ฅ ๐ป โ ๐ ๐ฅ ๐โฒ ๐ฅ ๐ปโ ๐ ๐ฅ ๐โฒโฒ ๐ฅ
![Page 49: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/49.jpg)
Vector Composition
โ and ๐ are twice differentiable dom ๐ ๐, dom โ ๐
๐ is concave, if ๐ ๐ฅ 0 โ is concave, โ is nondecreasing in each
argument, and ๐ are concave
The general case is similar
๐ โ โ ๐ โ ๐ ๐ฅ , โฆ , ๐ ๐ฅ
๐ ๐ฅ ๐ ๐ฅ ๐ป โ ๐ ๐ฅ ๐โฒ ๐ฅ ๐ปโ ๐ ๐ฅ ๐โฒโฒ ๐ฅ
![Page 50: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/50.jpg)
Examples โ ๐ง ๐ง โฏ ๐ง , ๐ง โ ๐ , ๐ , โฆ , ๐ is convex
โ โ โ ๐ is convex
โ ๐ง log โ ๐ , ๐ , โฆ , ๐ is convex โ โ โ๐ is convex
โ ๐ง โ ๐ง/ on ๐ is concave for 0 ๐
1, and its extension is nondecreasing. If ๐ is concave and nonnegative โ โ โ ๐ is concave
![Page 51: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/51.jpg)
Minimization is convex in is convex ๐ ๐ฅ inf
โ๐ ๐ฅ, ๐ฆ is convex if ๐ ๐ฅ
โ, โ ๐ฅ โ dom ๐ dom ๐ ๐ฅ ๐ฅ, ๐ฆ โ dom ๐ for some ๐ฆ โ ๐ถ
Proof by Epigraph epi ๐ ๐ฅ, ๐ก | ๐ฅ, ๐ฆ, ๐ก โ epi ๐ for some ๐ฆ โ ๐ถ The projection of a convex set is convex.
![Page 52: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/52.jpg)
Examples Schur complement ๐ ๐ฅ, ๐ฆ ๐ฅ ๐ด๐ฅ 2๐ฅ ๐ต๐ฆ ๐ฆ ๐ถ๐ฆ ๐ด ๐ต
๐ต ๐ถ โฝ 0, ๐ด, ๐ถ is symmetric โ ๐ ๐ฅ, ๐ฆ is convex ๐ ๐ฅ inf ๐ ๐ฅ, ๐ฆ ๐ฅ ๐ด ๐ต๐ถ ๐ต ๐ฅ is convex
โ ๐ด ๐ต๐ถ ๐ต โฝ 0, ๐ถ is the pseudo-inverse of ๐ถ
Distance to a set ๐ is a convex nonempty set,๐ ๐ฅ, ๐ฆ โ๐ฅ ๐ฆโ is
convex in ๐ฅ, ๐ฆ ๐ ๐ฅ dist ๐ฅ, ๐ inf
โโ๐ฅ ๐ฆโ
![Page 53: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/53.jpg)
Examples
Affine domain โ ๐ฆ is convex ๐ ๐ฅ inf โ ๐ฆ |๐ด๐ฆ ๐ฅ is convex
Proof
๐ ๐ฅ, ๐ฆ โ ๐ฆ if ๐ด๐ฆ ๐ฅ โ otherwise
๐ ๐ฅ, ๐ฆ is convex in ๐ฅ, ๐ฆ ๐ is the minimum of ๐ over ๐ฆ
![Page 54: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/54.jpg)
Perspective of a function
defined as
is the perspective of dom ๐ ๐ฅ, ๐ก |๐ฅ/๐ก โ dom ๐, ๐ก 0 ๐ is convex โ ๐ is convex
Proof
Perspective mapping preserve convexity
๐ฅ, ๐ก, ๐ โ epi ๐ โ ๐ก๐๐ฅ๐ก ๐
โ ๐๐ฅ๐ก
๐ ๐ก
โ ๐ฅ/๐ก, ๐ /๐ก โ epi ๐
![Page 55: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/55.jpg)
Example
Euclidean norm squared
Composition with an Affine function is convex
is convex
![Page 56: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/56.jpg)
Outline Basic Properties
Definition First-order Conditions, Second-order Conditions Jensenโs inequality and extensions Epigraph
Operations That Preserve Convexity Nonnegative Weighted Sums Composition with an affine mapping Pointwise maximum and supremum Composition Minimization Perspective of a function
Summary
![Page 57: Convex Functions (I)Vector Composition ร ร โand ๐are twice differentiable dom ๐ ร๐,dom โ๐ ร ๐is convex, if ๐ รฑ รฑ๐ฅ0 โis convex, โis nondecreasing](https://reader033.vdocuments.net/reader033/viewer/2022041912/5e680cdcce30cc5806271fce/html5/thumbnails/57.jpg)
Summary Basic Properties
Definition First-order Conditions, Second-order Conditions Jensenโs inequality and extensions Epigraph
Operations That Preserve Convexity Nonnegative Weighted Sums Composition with an affine mapping Pointwise maximum and supremum Composition Minimization Perspective of a function