Slide 1

Optimality Conditions for Unconstrained optimizationOne dimensional optimizationNecessary and sufficient conditionsMultidimensional optimizationClassification of stationary pointsNecssary and sufficient conditions for local optima.Convexity and global optimality

One dimensional optimizationWe are accustomed to think that if f(x) has a minimum then f(x)=0 but.

1D Optimization jargonA point with zero derivative is a stationary point. x=5, Can be a minimum

A maximum

An inflection point

Optimality criteria for smooth functionsCondition at a candidate point x*f(x*)=0 is the condition for stationarity and a necessary condition for a minimum.f(x*)>0 is sufficient for a minimumf(x*)> cs=surfc(X1,X2,Z);>> xlabel('x1')>> ylabel('x2')>> Z=Z+2*X1.*X2;>> cs=surfc(X1,X2,Z);>> xlabel('x1')>> ylabel('x2')8Global optimizationThe function x+sin(2x)

Convex functionA straight line connecting two points will not dip below the function graph.

Sufficient condition: Positive semi-definite Hessian everywhere.What does that mean geometrically?

Reciprocal approximationReciprocal approximation (linear in one over the variables) is desirable in many cases because it captures decreasing returns behavior.Linear approximation Reciprocal approximation

Conservative-convex approximationAt times we benefit from conservative approximations

All second derivatives of gC are non-negativeCalled convex linearization (CONLIN), Claude Fleury