optimization problems 虞台文 大同大學資工所 智慧型多媒體研究室. content...
TRANSCRIPT
![Page 1: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/1.jpg)
Optimization Problems
虞台文大同大學資工所智慧型多媒體研究室
![Page 2: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/2.jpg)
ContentIntroductionDefinitionsLocal and Global OptimaConvex Sets and FunctionsConvex Programming
Problems
![Page 3: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/3.jpg)
Optimization Problems
Introduction
大同大學資工所智慧型多媒體研究室
![Page 4: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/4.jpg)
General Nonlinear Programming Problems
( )f xminimize
( ) 0 1, ,ig x i m subject to
( ) 0 1, ,jh x j p nx R
objective function
constraints
![Page 5: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/5.jpg)
Local Minima vs. Global Minima
( )f xminimize
( ) 0 1, ,ig x i m subject to
( ) 0 1, ,jh x j p nx R
objective function
constraints
local minimum
global minimum
![Page 6: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/6.jpg)
Convex Programming Problems
( )f xminimize
( ) 0 1, ,ig x i m subject to
( ) 0 1, ,jh x j p nx R
objective function
constraints
f (x)
gi (x)
hj (x)
convex
concave
linear
Local optimality Global optimality
![Page 7: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/7.jpg)
Linear Programming Problems
( )f xminimize
( ) 0 1, ,ig x i m subject to
( ) 0 1, ,jh x j p nx R
objective function
constraints
f (x)
gi (x)
hj (x)
linear
linear
linear
Local optimality Global optimality
a special case of convex programming problems
![Page 8: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/8.jpg)
Linear Programming Problems
( )f xminimize
( ) 0 1, ,ig x i m subject to
( ) 0 1, ,jh x j p nx R
objective function
constraints
f (x)
gi (x)
hj (x)
linear
linear
linear
Local optimality Global optimality
![Page 9: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/9.jpg)
Integer Programming Problems
( )f xminimize
( ) 0 1, ,ig x i m subject to
( ) 0 1, ,jh x j p nx Z
objective function
constraints
f (x)
gi (x)
hj (x)
linear
linear
linear
![Page 10: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/10.jpg)
The Hierarchy of Optimization Problems
NonlinearPrograms
ConvexPrograms
LinearPrograms
(Polynomial) IntegerPrograms(NP-Hard)
Flowand
Matching
![Page 11: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/11.jpg)
Optimization Problems
General Nonlinear Programming Problems
Convex Programming Problems
Linear Programming Problems
Integer Linear Programming Problems
![Page 12: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/12.jpg)
Optimization Techniques
General Nonlinear Programming Problems
Convex Programming Problems
Linear Programming Problems
Integer Linear Programming Problems
ContinuousVariables
DiscreteVariables
ContinuousOptimization
CombinatorialOptimization
![Page 13: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/13.jpg)
Optimization Problems
Definitions
大同大學資工所智慧型多媒體研究室
![Page 14: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/14.jpg)
Optimization Problems
( )f xminimize
( ) 0 1, ,ig x i m subject to
( ) 0 1, ,jh x j p nx R
![Page 15: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/15.jpg)
( )f xminimize
Optimization Problems
( ) 0 1, ,ig x i m subject to
( ) 0 1, ,jh x j p nx R
Define the set of feasible points
F
Minimize cost c: FR1
![Page 16: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/16.jpg)
Definition:Instance of an Optimization Problem
(F, c) F: the domain of feasible points
c: F R1 cost function
Goal: To find f F such that
c( f ) c(g) for all gF.
A global optimum
![Page 17: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/17.jpg)
Definition:Optimization Problem
A set of instances of an optimization problem, e.g.– Traveling Salesman Problem (TSP)– Minimal Spanning Tree (MST)– Shortest Path (SP)– Linear Programming (LP)
![Page 18: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/18.jpg)
Traveling Salesman Problem (TSP)
![Page 19: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/19.jpg)
Traveling Salesman Problem (TSP)
Instance of the TSP – Given n cities and an n n distance matrix [dij], t
he problem is to find a Hamiltonian cycle with minimal total length.
on F n all cyclic permutations objects
( )1
n
j jj
c d
1 2 3 4 5 6 7 8
2 5 3 6 1 8 4 7
e.g.,
![Page 20: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/20.jpg)
Minimal Spanning Tree (MST)
![Page 21: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/21.jpg)
Minimal Spanning Tree (MST)
Instance of the MST – Given an integer n > 0 and an n n symmetric distance m
atrix [dij], the problem is to find a spanning tree on n vertices that has minimum total length of its edge.
( , ) {1,2, , }VF E V n all spanning trees with
( , )
: ( , ) iji j E
c V E d
![Page 22: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/22.jpg)
Linear Programming (LP)
minimize 1 1 2 2 n nc x c x c x
11 1 12 2 1 1
21 1 22 2 2 2
1 1 2 2
n n
n n
m m mn n m
a x a x a x b
a x a x a x b
a x a x a x b
1 2, , , 0nx x x
Subject to
![Page 23: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/23.jpg)
Linear Programming (LP)
minimize 1 1 2 2 n nc x c x c x
11 1 12 2 1 1
21 1 22 2 2 2
1 1 2 2
n n
n n
m m mn n m
a x a x a x b
a x a x a x b
a x a x a x b
1 2, , , 0nx x x
Subject to
minimize 1 1 2 2 n nc x c x c x
11 1 12 2 1 1
21 1 22 2 2 2
1 1 2 2
n n
n n
m m mn n m
a x a x a x b
a x a x a x b
a x a x a x b
1 2, , , 0nx x x
Subject to
1
2
n
c
cc
c
11 12 1
21 22 2
1 2
n
n
m m mn
a a a
a a aA
a a a
1
2
m
b
bb
b
1
2
n
x
xx
x
minimize
Subject to
c x
Ax b0x
![Page 24: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/24.jpg)
Linear Programming (LP)
, , 0nx x R AF x b x
:c x c x
minimize 1 1 2 2 n nc x c x c x
11 1 12 2 1 1
21 1 22 2 2 2
1 1 2 2
n n
n n
m m mn n m
a x a x a x b
a x a x a x b
a x a x a x b
1 2, , , 0nx x x
Subject to
minimize 1 1 2 2 n nc x c x c x
11 1 12 2 1 1
21 1 22 2 2 2
1 1 2 2
n n
n n
m m mn n m
a x a x a x b
a x a x a x b
a x a x a x b
1 2, , , 0nx x x
Subject to
minimize
Subject to
c x
Ax b0x
![Page 25: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/25.jpg)
Example:Linear Programming (LP)
minimize 1 1 2 2 n nc x c x c x
11 1 12 2 1 1
21 1 22 2 2 2
1 1 2 2
n n
n n
m m mn n m
a x a x a x b
a x a x a x b
a x a x a x b
1 2, , , 0nx x x
Subject to
minimize 1 1 2 2 n nc x c x c x
11 1 12 2 1 1
21 1 22 2 2 2
1 1 2 2
n n
n n
m m mn n m
a x a x a x b
a x a x a x b
a x a x a x b
1 2, , , 0nx x x
Subject to
1 2 34 2 3x x x
1 2 3
1 2 3
2
, , 0
x x x
x x x
4 2 3c
1 1 1A 2b
minimize
Subject to
![Page 26: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/26.jpg)
Example:Linear Programming (LP)
minimize 1 1 2 2 n nc x c x c x
11 1 12 2 1 1
21 1 22 2 2 2
1 1 2 2
n n
n n
m m mn n m
a x a x a x b
a x a x a x b
a x a x a x b
1 2, , , 0nx x x
Subject to
minimize 1 1 2 2 n nc x c x c x
11 1 12 2 1 1
21 1 22 2 2 2
1 1 2 2
n n
n n
m m mn n m
a x a x a x b
a x a x a x b
a x a x a x b
1 2, , , 0nx x x
Subject to
1 2 34 2 3x x x
1 2 3
1 2 3
2
, , 0
x x x
x x x
minimize
Subject to
x1
x2
x3
v1
v2
v3
c(v1) = 8
c(v2) = 4
c(v3) = 6
The optimum
The optimal point is at one of the vertices.
![Page 27: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/27.jpg)
Example:Minimal Spanning Tree (3 Nodes)
1 2 34 2 3x x x
1 2 3 2x x x
minimize
Subject to
c1=4
c3=3
c2=2
1 2 3, , {0,1}x x x
x1{0, 1}
x2{0, 1}
x3{0, 1}
Integer Programming
x1
x2
x3
![Page 28: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/28.jpg)
Example:Minimal Spanning Tree (3 Nodes)
1 2 34 2 3x x x
1 2 3 2x x x
minimize
Subject to
c1=4
c3=3
c2=2
x1{0, 1}
x2{0, 1}
x3{0, 1}
Linear Programming
x1
x2
x3
1 2 3, , 0x x x 1 2 3, , 1x x x
Some integer programs can be transformed into linear programs.
![Page 29: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/29.jpg)
Optimization Problems
Local and Global Optima
大同大學資工所智慧型多媒體研究室
![Page 30: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/30.jpg)
Neighborhoods
Given an optimization problem with instance
(F, c),
a neighborhood is a mapping
defined for each instance.
: 2FN F
For combinatorial optimization, the choice of N is critical.
![Page 31: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/31.jpg)
TSP (2-Change)
f F gN2(f )
2 ( ) N f g g F g and can be obtained as above
![Page 32: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/32.jpg)
TSP (k-Change)
( )
.k
g F gN f g
k f
and can be obtained
by changing edges of
![Page 33: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/33.jpg)
MST
f F gN(f )1. Adding an edge to form a cycle.2. Deleting any edge on the cycle.
( ) N f g g F g and can be obtained as above
![Page 34: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/34.jpg)
LP
minimize
Subject to
c x
Ax b0x
( ) , 0, N x y Ay b y y x and
![Page 35: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/35.jpg)
Local Optima
Given(F, c)
N
an instance of an optimization problem
neighborhood
f F is called locally optimum with respect to N (or simply
locally optimum whenever N is understood by context) if
c(f ) c(g) for all gN(f ).
![Page 36: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/36.jpg)
0 1 F
c
small
Local Optima
F = [0, 1] R1
( ) , 0, N f x x F y x f and
C
B
A Local minimum
Local minimum
Global minimum
![Page 37: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/37.jpg)
Decent Algorithm
f = initial feasible solution
While Improve(f ) do
f = any element in Improve(f )
return f
Improve( ) ( ) ( ) ( )f s s N f c s c f and
Decent algorithm is usually stuck at a
local minimum unless the neighborhood N
is exact.
![Page 38: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/38.jpg)
Exactness of Neighborhood
Neighborhood N is said to be exact if it makes
Local minimum Global Minimum
![Page 39: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/39.jpg)
Exactness of Neighborhood
0 1 F
c
F = [0, 1] R1
( ) , 0, N f x x F y x f and
C
B
A Local minimum
Local minimum
Global minimum
N is exact if 1.
![Page 40: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/40.jpg)
TSP
N2: not exact
Nn: exact
f F gN2(f )
![Page 41: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/41.jpg)
MST N is exact
f F gN(f )1. Adding an edge to form a cycle.2. Deleting any edge on the cycle.
( ) N f g g F g and can be obtained as above
![Page 42: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/42.jpg)
Optimization Problems
Convex Sets and Functions
大同大學資工所智慧型多媒體研究室
![Page 43: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/43.jpg)
Convex Combination
x, y Rn
0 1 z = x +(1)y
A convex combination of x, y.
A strict convex combination of x, y if 0, 1.
![Page 44: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/44.jpg)
Convex Sets
S Rn
z = x +(1)y
is convex if it contains all convex combinations of pairs x, y S.
convex nonconvex
0 1
![Page 45: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/45.jpg)
Convex Sets
S Rn
z = x +(1)y
is convex if it contains all convex combinations of pairs x, y S.
n = 1
S is convex iff S is an interval.
0 1
![Page 46: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/46.jpg)
Convex Sets
Fact: The intersection of any number of convex sets is convex.
![Page 47: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/47.jpg)
c
Convex Functions
x yx +(1)y
c(x)
c(y)c(x) + (1)c(y)
c(x +(1)y)
S Rn a convex set
c:S R a convex function if
c(x +(1)y) c(x) + (1)c(y), 0 1
Every linear function is convex.
![Page 48: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/48.jpg)
LemmaS
c(x)
t
a convex set
a convex function on S
a real number
( ) ,tS c x x Stx
is convex.
Pf) Let x, y St x +(1)y S
c(x +(1)y) c(x) + (1)c(y)
t + (1)t
= t
x +(1)y St
![Page 49: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/49.jpg)
Level Contours
c = 1
c = 2
c = 3
c = 4
c = 5
![Page 50: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/50.jpg)
Concave Functions
S Rn a convex set
c:S R a concave function if
c is a convex
Every linear function is concave as well as convex.
![Page 51: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/51.jpg)
Optimization Problems
Convex Programming Problems
大同大學資工所智慧型多媒體研究室
![Page 52: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/52.jpg)
Theorem
(F, c) an instance of optimization problem
a convex set
a convex function
Define ( )N x y y F x y and
( )N x is exact for every > 0.
![Page 53: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/53.jpg)
• Let x be a local minimum w.r.t. N for any fixed > 0.• Let yF be any other feasible point.
Theorem
(F, c) an instance of optimization probleman instance of optimization problem
a convex set
a convex function
Defi ne ( )N x y y F x y and
( )N x is exact f or every > 0.
(F, c) an instance of optimization probleman instance of optimization problem
a convex set
a convex function
Defi ne ( )N x y y F x y and
( )N x is exact f or every > 0.
Pf)
xF
( )N x
yNext, we now want to show that c(y) c(x).
![Page 54: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/54.jpg)
• Let x be a local minimum w.r.t. N for any fixed > 0.• Let yF be any other feasible point. <<1 such that• Since c is convex, we have
• Therefore,
Theorem
(F, c) an instance of optimization probleman instance of optimization problem
a convex set
a convex function
Defi ne ( )N x y y F x y and
( )N x is exact f or every > 0.
(F, c) an instance of optimization probleman instance of optimization problem
a convex set
a convex function
Defi ne ( )N x y y F x y and
( )N x is exact f or every > 0.
Pf)
xF
( )N x
yz
(1 ) ( ).x y xz z N and
( ) ( (1 ) )c c x yz
( ) (1 ) ( )c x c y
( ) ( )( )
1
zc c xc y
( ) ( )
1
c x c x
( )c x
( ) ( )zc c x
![Page 55: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/55.jpg)
Convex Programming Problems
(F, c)
Defined by ( ) 0, 1, ,ig x i m
: nig R R
Convex function
an instance of optimization problem
Important property:
Local minimum Global Minimum
Concave functions
![Page 56: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/56.jpg)
Convexity of Feasible Set
(F, c)
Defined by ( ) 0, 1, ,ig x i m
: nig R R
Convex function
an instance of optimization problem
Important property:
Local minimum Global Minimum
Concave functions
( ) : ig x convex
( ) : ig x concave
( ) 0 : ig x convex
( ) 0 : ig x convex
: iF convex
1
: m
ii
F F
convex
![Page 57: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/57.jpg)
Convex Programming Problems
(F, c)
Defined by ( ) 0, 1, ,ig x i m
: nig R R
Convex function
an instance of optimization problem
Important property:
Local minimum Global Minimum
Concave functionsConvex
![Page 58: Optimization Problems 虞台文 大同大學資工所 智慧型多媒體研究室. Content Introduction Definitions Local and Global Optima Convex Sets and Functions Convex Programming](https://reader031.vdocuments.net/reader031/viewer/2022020712/56649cf95503460f949c9ef0/html5/thumbnails/58.jpg)
Theorem
In a convex programming problem, every
point locally optimal with respect to the
Euclidean distance neighborhood N is also
global optimal.