or ii gslm 52800
DESCRIPTION
OR II GSLM 52800. Outline. some terminology differences between LP and NLP basic questions in NLP gradient and Hessian quadratic form contour, graph, and tangent plane. feasible region. C. the neighborhood of a point for a given . . D. B. A. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/1.jpg)
1
OR IIOR IIGSLM 52800GSLM 52800
![Page 2: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/2.jpg)
2
OutlineOutline
some terminology
differences between LP and NLP
basic questions in NLP
gradient and Hessian
quadratic form
contour, graph, and tangent plane
![Page 3: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/3.jpg)
3
Feasible Points, Solution Set, Feasible Points, Solution Set, and Neighborhoodand Neighborhood
feasible point: a point that satisfies all the constraints
solution set (feasible set, feasible region): the collection of all feasible points
neighborhood of x0 = {x| |xx0| < } for some pre-specified
feasible region
the neighborhood of a point for a given
A
C
B
D
only the neighborhood of D is completely
feasible for this
![Page 4: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/4.jpg)
4
Weak and Strong; Weak and Strong; Local and Global Local and Global
local minima: x1, any point in [s, t], x3
strict (strong) local minima: x1, x3
weak local minima: any point in [s, t] strict global minimum: x1
weak local maxima: any point in [s, t]
x
f(x )
12x3
x2
x1 ts
![Page 5: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/5.jpg)
5
Differences Between Differences Between
Linear and Non-Linear ProgrammingLinear and Non-Linear Programming linear programming
there exists an optimal extreme point (a corner point) direction of improvement keeps on being so unless
hitting a constraint a local optimum point is also globally optimal
direction of improvement
optimal point
![Page 6: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/6.jpg)
6
Differences Between Differences Between
Linear and Non-Linear ProgrammingLinear and Non-Linear Programming
none of these necessarily holds for a non-linear program
x
f(x )
12x3
x2
x1 ts
min x2 + y2,s.t. -2 x, y 2
![Page 7: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/7.jpg)
7
Basic Questions Basic Questions in Non-Linear Programming in Non-Linear Programming
main question: given an initial location x0, how to get to a local minimum, or, better, a global minimum (a) the direction of improvement? (b) the necessary conditions of an optimal point? (c) the sufficient conditions of an optimal point? (d) any conditions to simplify the processes in (a),
(b), and (c)? (e) any algorithmic procedures to solve a NLP
problem?
![Page 8: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/8.jpg)
8
Basic Questions Basic Questions in Non-Linear Programming in Non-Linear Programming
calculus required for (a) to (e)
direction of improvement of f = gradient of f shaped by constraints
convexity for (d), and also (b) and (c) identification of convexity: definiteness of
matrices, especially for Hessians
![Page 9: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/9.jpg)
9
Gradient and HessianGradient and Hessian
gradient of f: f(x) = in short
Hessian =
1
( ) ( ),...,n
Tf fx x
x x
( )
j
fx
x
2 2 2
21 2 11
2 2
22 1 2
2 2
21
( ) ( ) ( )
( ) ( )
( ) ( )
n
n n
f f fx x x xx
f fx x x
f fx x x
x x x
x x
x x
L
M
M O M
L L
f and gj usually assumed to be
twice differentiable functions Hessian is a
symmetric matrix
![Page 10: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/10.jpg)
10
Gradient and HessianGradient and Hessian
ej: (0, …, 0, 1, 0, …, 0)T, where “1” at the jth position
for small , f(x+ej) f(x) +
in general, x = (x1, …, xn)T from x,
f(x+x) f(x) +
( ) ( )( )
0lim j
j
f ffx
x e xx
( )
j
fx
x
( )
j
fjx
jx
x
![Page 11: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/11.jpg)
11
Example 1.6.1Example 1.6.1
(a). f(x) = x2; f(3.5+) ? for small
(b). f(x, y) = x2 + y2, f((1, 1) + (x, y)) ? for small x, y
gradient f : direction of steepest accent of the objective fucntion
![Page 12: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/12.jpg)
12
Example 1.6.2Example 1.6.2
find the Hessian of (a). f(x, y) = x2 + 7y2
(b). f(x, y) = x2 + 5xy + 7y2
(c). f(x, y) = x3 + 7y2
![Page 13: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/13.jpg)
13
Quadratic FormQuadratic Form
general form: xTQx/2 + cTx + a, where x is an n-dimensional vector; Q an nn square matrix; c and a are matrices of appropriate dimensions how to derive the gradient and Hessian?
gradient f(x) = Qx+c
Hessian H = Q
![Page 14: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/14.jpg)
14
Quadratic FormQuadratic Form
relate the two forms xTQx/2 + cTx + a and f(x, y) = 1x2+2xy+3y2+4x+5y+6
Example 1.6.3
![Page 15: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/15.jpg)
15
Example 1.6.4Example 1.6.4
Find the first two derivatives of the following f(x) f(x) = x2 for x [-2, 2]
f(x) = -x2 for x [-2, 2]
![Page 16: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/16.jpg)
16
Contour and Graph Contour and Graph (i.e., Surface) of Function (i.e., Surface) of Function ff
Example 1.7.1: f(x1, x2) = 2 21 2x x
![Page 17: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/17.jpg)
17
Contour and Graph Contour and Graph (i.e., Surface) of Function (i.e., Surface) of Function ff
an n-dimensional function
a contour of f: a diagram f(x) = c in the n-dimensional space for a given value c
the graph (surface function) of f: the diagram z = f(x) in the (n+1)st dimensional space as x and z vary
![Page 18: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/18.jpg)
18
Contour and Graph Contour and Graph (i.e., Surface) of Function (i.e., Surface) of Function ff
how do the contours of the one-dimensional function f(x) = x2 look like?
![Page 19: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/19.jpg)
19
An Important Property Between the Gradient An Important Property Between the Gradient and the Tangent Plane at a Contourand the Tangent Plane at a Contour
the gradient of f at point x0 is orthogonal to the tangent of the contour f(x) = c at x0
many optimization results are related to the above property
![Page 20: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/20.jpg)
20
Gradient of f at x0 Being Orthogonal to the Tangent of the Contour f(x) = c at x0
Example 1.7.3: f(x1, x2) = x1+2x2
gradient at (4, 2)?
tangent of contour at (4, 2)?
![Page 21: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/21.jpg)
21
Gradient of f at x0 Being Orthogonal to the Tangent of the Contour f(x) = c at x0
Example 1.7.2: f(x1, x2) =
point (x10, x20) on a contour f(x1, x2) = c
2 21 2x x
![Page 22: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/22.jpg)
22
Tangent at a Contour and the Corresponding Tangent at a Contour and the Corresponding Tangent Plane at a SurfaceTangent Plane at a Surface
the above two are related
for contour of f(x, y) = x2+y2, the tangent at (x0, y0)
(x-x0, y- y0)T(2x0, 2y0) = 0
two orthogonal vectors u and v:
uTv = 0
![Page 23: OR II GSLM 52800](https://reader035.vdocuments.net/reader035/viewer/2022062509/568143cc550346895db05916/html5/thumbnails/23.jpg)
23
Tangent at a Contour and the Corresponding Tangent at a Contour and the Corresponding Tangent Plane at a SurfaceTangent Plane at a Surface
the tangent place at (x0, y0) for the surface of f(x, y) = x2+y2
the surface: z = x2+y2
defining a contour at a higher dimension: F(x, y, z) = x2+y2z tangent plane at (x0, y0, ) of the surface: 2 2
0 0x y
what happens when z = 2 2
0 0 ?x y