introduction to optimization - cermics – centre d...
TRANSCRIPT
Introduction
Optimalityconditions
Introduction to Optimization
Pedro Gajardo1 and Eladio Ocana2
1Universidad Tecnica Federico Santa Marıa - Valparaıso -Chile
2IMCA-FC. Universidad Nacional de Ingenierıa - Lima - Peru
Mathematics of Bio-Economics (MABIES) - IHP
January 29, 2013 - Paris
Introduction
Optimalityconditions
Contents
Introduction
Optimality conditions
Introduction
Optimalityconditions
Optimization problems
In order to formulate an optimization problem, the followingconcepts must be very clear:
decision variables
restrictions
objective function
Introduction
Optimalityconditions
Optimization problems
In order to formulate an optimization problem, the followingconcepts must be very clear:
decision variables
restrictions
objective function
Introduction
Optimalityconditions
Optimization problems
In order to formulate an optimization problem, the followingconcepts must be very clear:
decision variables
restrictions
objective function
Introduction
Optimalityconditions
Optimization problemsDecision variables
One or morevariableson whichwe can decide(harvestingrate or effort, level of investment, distribution of tasks,parameters)
Objective: to find thebestvalue for the decision variable
We denote byx the decision variable
x can be a number, a vector, a sequence, a function, ..........
In a general framework, we denote byX the set where thedecision variable is (x ∈ X)
Introduction
Optimalityconditions
Optimization problemsDecision variables
One or more variables on which we can decide (harvestingrate or effort, level of investment, distribution of tasks,parameters)
Objective:to find thebestvalue for the decision variable
We denote byx the decision variable
x can be a number, a vector, a sequence, a function, ..........
In a general framework, we denote byX the set where thedecision variable is (x ∈ X)
Introduction
Optimalityconditions
Optimization problemsDecision variables
One or more variables on which we can decide (harvestingrate or effort, level of investment, distribution of tasks,parameters)
Objective: to find thebestvalue for the decision variable
Wedenote byx the decision variable
x can be a number, a vector, a sequence, a function, ..........
In a general framework, we denote byX the set where thedecision variable is (x ∈ X)
Introduction
Optimalityconditions
Optimization problemsDecision variables
One or more variables on which we can decide (harvestingrate or effort, level of investment, distribution of tasks,parameters)
Objective: to find thebestvalue for the decision variable
We denote byx the decision variable
x can be a number, a vector, a sequence, a function, ..........
In a general framework, we denote byX the set where thedecision variable is (x ∈ X)
Introduction
Optimalityconditions
Optimization problemsDecision variables
One or more variables on which we can decide (harvestingrate or effort, level of investment, distribution of tasks,parameters)
Objective: to find thebestvalue for the decision variable
We denote byx the decision variable
x can be a number, a vector, a sequence, a function, ..........
In a general framework,we denote byX the set where thedecision variable is (x ∈ X)
Introduction
Optimalityconditions
Warning
This tutorial is onContinuous Optimization
That means that decision variables are continuous variables
What that means?
Firstly, the setX where the decision variable belongs, is a(linear) vector space
Introduction
Optimalityconditions
Warning
This tutorial is onContinuous Optimization
That means that decision variables are continuous variables
What that means?
Firstly, the setX where the decision variable belongs, is a(linear) vector space
Introduction
Optimalityconditions
Warning
This tutorial is onContinuous Optimization
That means that decision variables are continuous variables
What that means?
Firstly, the setX where the decision variable belongs, is a(linear) vector space
Introduction
Optimalityconditions
Warning
This tutorial is onContinuous Optimization
That means that decision variables are continuous variables
What that means?
Firstly, the setX where the decision variable belongs, is a(linear) vector space
Introduction
Optimalityconditions
Optimization problemsRestrictions
Constraints that the decision variable has to satisfy
If for a certain value of the decision variable therestrictions are satisfied, we say that it is a feasiblesolution
In a general framework, we denote byC ⊆ X the set of allfeasible solutions
If we have two decision variables,x1 andx2 and they haveto satisfy the following constraints:x1 ≥ 0, x2 ≥ 0,2 x1 + 3 x2 ≤ 5, we denote
C = {(x1, x2) ∈ R2 | x1 ≥ 0, x2 ≥ 0, 2 x1 + 3x2 ≤ 5}
Introduction
Optimalityconditions
Optimization problemsRestrictions
Constraints that the decision variable has to satisfy
If for a certain value of thedecision variable therestrictions are satisfied, we say that it is a feasiblesolution
In a general framework, we denote byC ⊆ X the set of allfeasible solutions
If we have two decision variables,x1 andx2 and they haveto satisfy the following constraints:x1 ≥ 0, x2 ≥ 0,2 x1 + 3 x2 ≤ 5, we denote
C = {(x1, x2) ∈ R2 | x1 ≥ 0, x2 ≥ 0, 2 x1 + 3x2 ≤ 5}
Introduction
Optimalityconditions
Optimization problemsRestrictions
Constraints that the decision variable has to satisfy
If for a certain value of the decision variable therestrictions are satisfied, we say that it is a feasiblesolution
In a general framework,we denote byC ⊆ X the set of allfeasible solutions
If we have two decision variables,x1 andx2 and they haveto satisfy the following constraints:x1 ≥ 0, x2 ≥ 0,2 x1 + 3 x2 ≤ 5, we denote
C = {(x1, x2) ∈ R2 | x1 ≥ 0, x2 ≥ 0, 2 x1 + 3x2 ≤ 5}
Introduction
Optimalityconditions
Optimization problemsRestrictions
Constraints that the decision variable has to satisfy
If for a certain value of the decision variable therestrictions are satisfied, we say that it is a feasiblesolution
In a general framework, we denote byC ⊆ X the set of allfeasible solutions
If we have two decision variables,x1 andx2 and they haveto satisfy the following constraints:x1 ≥ 0, x2 ≥ 0,2 x1 + 3 x2 ≤ 5, we denote
C = {(x1, x2) ∈ R2 | x1 ≥ 0, x2 ≥ 0, 2 x1 + 3x2 ≤ 5}
Introduction
Optimalityconditions
Warning
This tutorial is onContinuous Optimization
That means that decision variables are continuous variables
What that means?
Secondly, the setC of feasible solutions is not a discrete set (setof isolated points)
Introduction
Optimalityconditions
Warning
This tutorial is onContinuous Optimization
That means that decision variables are continuous variables
What that means?
Secondly, the setC of feasible solutions is not a discrete set (setof isolated points)
Introduction
Optimalityconditions
Optimization problemsObjective function
It is the mathematical representation formeasuring thegoodnessof values for the decision variable
This representation is done through a function
f : X −→ R
Introduction
Optimalityconditions
Optimization problemsObjective function
It is the mathematical representation for measuring thegoodnessof values for the decision variable
This representation is done through a function
f : X −→ R
Introduction
Optimalityconditions
Optimization problems
To find thebestvalue for the decision variable from all feasiblesolutions
To find x ∈ C ⊆ X such that
f (x) ≤ f (x) (or f (x) ≥ f (x)) for all x ∈ C
minx∈X
f (x)
(
or maxx∈X
f (x)
)
subject to
x ∈ C
Introduction
Optimalityconditions
Optimization problems
To find thebestvalue for the decision variable from all feasiblesolutions
To find x ∈ C ⊆ X such that
f (x) ≤ f (x) (or f (x) ≥ f (x)) for all x ∈ C
minx∈X
f (x)
(
or maxx∈X
f (x)
)
subject to
x ∈ C
Introduction
Optimalityconditions
Optimization problems
To find thebestvalue for the decision variable from all feasiblesolutions
To find x ∈ C ⊆ X such that
f (x) ≤ f (x) (or f (x) ≥ f (x)) for all x ∈ C
minx∈X
f (x)
(
or maxx∈X
f (x)
)
subject to
x ∈ C
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
With a given fence, to enclose the largest rectangular area
Objective: maximize the rectangular surface
Decision variables: lengths of the rectangular figure
Constraints:
positive lengths
length of the fence
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
With a given fence, to enclose the largest rectangular area
Objective: maximize the rectangular surface
Decision variables: lengths of the rectangular figure
Constraints:
positive lengths
length of the fence
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
With a given fence, to enclose the largest rectangular area
Objective: maximize the rectangular surface
Decision variables: lengths of the rectangular figure
Constraints:
positive lengths
length of the fence
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
With a given fence, to enclose the largest rectangular area
Objective: maximize the rectangular surface
Decision variables: lengths of the rectangular figure
Constraints:
positive lengths
length of the fence
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
With a given fence, to enclose the largest rectangular area
Objective: maximize the rectangular surface
Decision variables: lengths of the rectangular figure
Constraints:
positive lengths
length of the fence
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
Decision variables: a (height) andb (width)
Objective: maximizes(a,b) = a b
Constraints:positive lengths:a > 0, b > 0
length of fence (L > 0): 2(a+ b) = L
b = L2 − a
New objective functionf (a) = a(
L2 − a
)
Rewrite restrictions:a > 0, a < L2
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
Decision variables:a (height) andb (width)
Objective: maximizes(a,b) = a b
Constraints:positive lengths:a > 0, b > 0
length of fence (L > 0): 2(a+ b) = L
b = L2 − a
New objective functionf (a) = a(
L2 − a
)
Rewrite restrictions:a > 0, a < L2
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
Decision variables:a (height) andb (width)
Objective: maximizes(a,b) = a b
Constraints:positive lengths:a > 0, b > 0
length of fence (L > 0): 2(a+ b) = L
b = L2 − a
New objective functionf (a) = a(
L2 − a
)
Rewrite restrictions:a > 0, a < L2
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
Decision variables:a (height) andb (width)
Objective: maximizes(a,b) = a b
Constraints:positive lengths:a > 0, b > 0
length of fence (L > 0): 2(a+ b) = L
b = L2 − a
New objective functionf (a) = a(
L2 − a
)
Rewrite restrictions:a > 0, a < L2
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
Decision variables:a (height) andb (width)
Objective: maximizes(a,b) = a b
Constraints:positive lengths:a > 0, b > 0
length of fence (L > 0): 2(a+ b) = L
b = L2 − a
New objective functionf (a) = a(
L2 − a
)
Rewrite restrictions:a > 0, a < L2
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
Decision variables:a (height) andb (width)
Objective: maximizes(a,b) = a b
Constraints:positive lengths:a > 0, b > 0
length of fence (L > 0): 2(a+ b) = L
b = L2 − a
New objective functionf (a) = a(
L2 − a
)
Rewrite restrictions:a > 0, a < L2
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
Decision variables:a (height) andb (width)
Objective: maximizes(a,b) = a b
Constraints:positive lengths:a > 0, b > 0
length of fence (L > 0): 2(a+ b) = L
b = L2 − a
Newobjective functionf (a) = a(
L2 − a
)
Rewrite restrictions:a > 0, a < L2
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
Decision variables:a (height) andb (width)
Objective: maximizes(a,b) = a b
Constraints:positive lengths:a > 0, b > 0
length of fence (L > 0): 2(a+ b) = L
b = L2 − a
New objective functionf (a) = a(
L2 − a
)
Rewriterestrictions:a > 0, a < L2
Introduction
Optimalityconditions
Optimization problemsExample: maximizing a rectangular surface
maxa∈R
f (a) = a(
L2 − a
)
subject to
a > 0
a < L2
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
A firm wishes to maximize the utilities of its portafolio
Objective: maximize profits
Decision variables: amount to invest in each fund
Constraints:
nonnegative investments
budget restrictions
minimal or maximal bounds for investments
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
A firm wishes to maximize the utilities of its portafolio
Objective: maximize profits
Decision variables: amount to invest in each fund
Constraints:
nonnegative investments
budget restrictions
minimal or maximal bounds for investments
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
A firm wishes to maximize the utilities of its portafolio
Objective: maximize profits
Decision variables: amount to invest in each fund
Constraints:
nonnegative investments
budget restrictions
minimal or maximal bounds for investments
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
A firm wishes to maximize the utilities of its portafolio
Objective: maximize profits
Decision variables: amount to invest in each fund
Constraints:
nonnegative investments
budget restrictions
minimal or maximal bounds for investments
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
A firm wishes to maximize the utilities of its portafolio
Objective: maximize profits
Decision variables: amount to invest in each fund
Constraints:
nonnegative investments
budget restrictions
minimal or maximal bounds for investments
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
A firm wishes to maximize the utilities of its portafolio
Objective: maximize profits
Decision variables: amount to invest in each fund
Constraints:
nonnegative investments
budget restrictions
minimal or maximal bounds for investments
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
Decision variables: x1, x2, . . . , xn, wherexj the quantity toinvest in the fundj
Objective: maximize
(1+ r1)x1 + (1+ r2)x2 + . . .+ (1+ rn)xn
wherer j is the rentability of the fundj
Restrictions:
Non-negative investments:xj ≥ 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . .+ xn ≤ B
Minimal/maximal bounds for investments:
aj ≤ xj ≤ bj for j = 1, 2, . . . , n
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
Decision variables:x1, x2, . . . , xn, wherexj the quantity toinvest in the fundj
Objective: maximize
(1+ r1)x1 + (1+ r2)x2 + . . .+ (1+ rn)xn
wherer j is the rentability of the fundj
Restrictions:
Non-negative investments:xj ≥ 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . .+ xn ≤ B
Minimal/maximal bounds for investments:
aj ≤ xj ≤ bj for j = 1, 2, . . . , n
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
Decision variables:x1, x2, . . . , xn, wherexj the quantity toinvest in the fundj
Objective: maximize
(1+ r1)x1 + (1+ r2)x2 + . . .+ (1+ rn)xn
wherer j is the rentability of the fundj
Restrictions:
Non-negative investments:xj ≥ 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . .+ xn ≤ B
Minimal/maximal bounds for investments:
aj ≤ xj ≤ bj for j = 1, 2, . . . , n
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
Decision variables:x1, x2, . . . , xn, wherexj the quantity toinvest in the fundj
Objective: maximize
(1+ r1)x1 + (1+ r2)x2 + . . .+ (1+ rn)xn
wherer j is the rentability of the fundj
Restrictions:
Non-negative investments: xj ≥ 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . .+ xn ≤ B
Minimal/maximal bounds for investments:
aj ≤ xj ≤ bj for j = 1, 2, . . . , n
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
Decision variables:x1, x2, . . . , xn, wherexj the quantity toinvest in the fundj
Objective: maximize
(1+ r1)x1 + (1+ r2)x2 + . . .+ (1+ rn)xn
wherer j is the rentability of the fundj
Restrictions:
Non-negative investments:xj ≥ 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . .+ xn ≤ B
Minimal/maximal bounds for investments:
aj ≤ xj ≤ bj for j = 1, 2, . . . , n
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
Decision variables:x1, x2, . . . , xn, wherexj the quantity toinvest in the fundj
Objective: maximize
(1+ r1)x1 + (1+ r2)x2 + . . .+ (1+ rn)xn
wherer j is the rentability of the fundj
Restrictions:
Non-negative investments:xj ≥ 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . .+ xn ≤ B
Minimal/maximal bounds for investments:
aj ≤ xj ≤ bj for j = 1, 2, . . . , n
Introduction
Optimalityconditions
Optimization problemsExample: Optimizing a portafolio
maxx1,x2,...,xn∈R
(1+ r1)x1 + (1+ r2)x2 + . . .+ (1+ rn)xn
subject to
xj ≥ 0 j = 1,2, . . . ,n
x1 + x2 + . . .+ xn ≤ B
aj ≤ xj ≤ bj j = 1,2, . . . ,n
Introduction
Optimalityconditions
Optimization problemsExample: Least square problem
Given points on the plane, to find theline of best fitthroughthese points
Objective: To minimize the (squared) error between a lineand the points
Decision variables: the parameters of a line
Introduction
Optimalityconditions
Optimization problemsExample: Least square problem
Given points on the plane, to find theline of best fitthroughthese points
Objective: To minimize the (squared) error between a lineand the points
Decision variables: the parameters of a line
Introduction
Optimalityconditions
Optimization problemsExample: Least square problem
Given points
(x1, y1); (x2, y2); . . . ; (xn, yn)
to find theline of best fitthrough these points
Decision variables (the parameters of a line): mandn,where the expression of a line in the plane is
y = mx+ n
Objective: To minimize
n∑
k=1
(mxk + n− yk)2
Introduction
Optimalityconditions
Optimization problemsExample: Least square problem
Given points
(x1, y1); (x2, y2); . . . ; (xn, yn)
to find theline of best fitthrough these points
Decision variables (the parameters of a line):mandn,where the expression of a line in the plane is
y = mx+ n
Objective: To minimize
n∑
k=1
(mxk + n− yk)2
Introduction
Optimalityconditions
Optimization problemsExample: Least square problem
maxm, n ∈ R
n∑
k=1(mxk + n− yk)
2
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
To maximize the benefit from the harvesting of a naturalresource
Objective: maximize present utilities
Decision variables: harvesting levels (or effort) at eachperiod
Constraints:
nonnegative harvesting
biology
relation between harvesting and the amount of the resource
environmental constraints
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
To maximize the benefit from the harvesting of a naturalresource
Objective: maximize present utilities
Decision variables: harvesting levels (or effort) at eachperiod
Constraints:
nonnegative harvesting
biology
relation between harvesting and the amount of the resource
environmental constraints
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
To maximize the benefit from the harvesting of a naturalresource
Objective: maximize present utilities
Decision variables: harvesting levels (or effort) at eachperiod
Constraints:
nonnegative harvesting
biology
relation between harvesting and the amount of the resource
environmental constraints
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
To maximize the benefit from the harvesting of a naturalresource
Objective: maximize present utilities
Decision variables: harvesting levels (or effort) at eachperiod
Constraints:
nonnegative harvesting
biology
relation between harvesting and the amount of the resource
environmental constraints
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
To maximize the benefit from the harvesting of a naturalresource
Objective: maximize present utilities
Decision variables: harvesting levels (or effort) at eachperiod
Constraints:
nonnegative harvesting
biology
relation between harvesting and the amount of the resource
environmental constraints
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
To maximize the benefit from the harvesting of a naturalresource
Objective: maximize present utilities
Decision variables: harvesting levels (or effort) at eachperiod
Constraints:
nonnegative harvesting
biology
relation between harvesting and the amount of the resource
environmental constraints
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
To maximize the benefit from the harvesting of a naturalresource
Objective: maximize present utilities
Decision variables: harvesting levels (or effort) at eachperiod
Constraints:
nonnegative harvesting
biology
relation between harvesting and the amount of the resource
environmental constraints
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
x(t + 1) = F(x(t)) − h(t) t = t0, t0 + 1, . . . ,T − 1
x(t0) = x0 given (e.g. the current state of the resource)
where
x(t) ≥ 0 is the level of the resource at periodt
h(t) ≥ 0 is the harvesting at periodt
F : R −→ R is the biological growth function of theresource
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
x(t + 1) = F(x(t)) − h(t) t = t0, t0 + 1, . . . ,T − 1
x(t0) = x0 given (e.g. the current state of the resource)
where
x(t) ≥ 0 is the level of the resource at periodt
h(t) ≥ 0 is the harvesting at periodt
F : R −→ R is the biological growth function of theresource
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
x(t + 1) = F(x(t)) − h(t) t = t0, t0 + 1, . . . ,T − 1
x(t0) = x0 given (e.g. the current state of the resource)
where
x(t) ≥ 0 is the level of the resource at periodt
h(t) ≥ 0 is the harvesting at periodt
F : R −→ R is the biological growth function of theresource
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
The total benefits can be represented by
B =T−1∑
t=t0
ρ(t−t0)U(x(t),h(t))
where
U(x,h) is the instantaneous profit if we havex and weharvesth
0 ≤ ρ ≤ 1 is a descount factor
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
The total benefits can be represented by
B =T−1∑
t=t0
ρ(t−t0)U(x(t),h(t))
where
U(x,h) is the instantaneous profit if we havex and weharvesth
0 ≤ ρ ≤ 1 is a descount factor
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
The total benefits can be represented by
B =T−1∑
t=t0
ρ(t−t0)U(x(t),h(t))
where
U(x,h) is the instantaneous profit if we havex and weharvesth
0 ≤ ρ ≤ 1 is a descount factor
Introduction
Optimalityconditions
Optimization problemsExample: Harvesting of a renewable resource
maxh(t0),h(t0+1),...,h(T−1)
T−1∑
t=t0ρ(t−t0)U(x(t),h(t))
subject to
x(t + 1) = F(x(t)) − h(t) t = t0, t0 + 1, . . . ,T − 1
x(t0) = x0
x(t) ≥ 0 t = t0 + 1, t0 + 2, . . . ,T
0 ≤ h(t) ≤ F(x(t)) t = t0, t0 + 1, . . . ,T − 1
Introduction
Optimalityconditions
Optimization problemsImportant remark: min is equivalent to max
We can restrict our study only to minimization problems
To find x ∈ C ⊆ X such that
f (x) ≤ f (x) for all x ∈ C
minx∈X
f (x)
subject to
x ∈ C
Introduction
Optimalityconditions
Optimization problemsImportant remark: min is equivalent to max
We can restrict our study only to minimization problems
To find x ∈ C ⊆ X such that
f (x) ≤ f (x) for all x ∈ C
minx∈X
f (x)
subject to
x ∈ C
Introduction
Optimalityconditions
Optimization problemsImportant remark: min is equivalent to max
To find x ∈ C ⊆ X such that
f (x) ≥ f (x) for all x ∈ C
maxx∈X
f (x)
subject to
x ∈ C
Introduction
Optimalityconditions
Optimization problemsImportant remark: min is equivalent to max
To find x ∈ C ⊆ X such that
f (x) ≥ f (x) for all x ∈ C
maxx∈X
f (x)
subject to
x ∈ C
Introduction
Optimalityconditions
Optimization problemsImportant remark: min is equivalent to max
(Pmax)
maxx∈X
f (x)
subject to
x ∈ C
is equivalent to problem
(Pmin)
minx∈X
−f (x)
subject to
x ∈ C
In the following sense:
x is solution of(Pmax) if and only if it is solution of(Pmin)
The optimal value of(Pmax) is the negative of the optimalvalue of(Pmin)
Introduction
Optimalityconditions
Optimization problemsImportant remark: min is equivalent to max
(Pmax)
maxx∈X
f (x)
subject to
x ∈ C
is equivalent to problem
(Pmin)
minx∈X
−f (x)
subject to
x ∈ C
In the following sense:
x is solution of(Pmax) if and only if it is solution of(Pmin)
The optimal value of(Pmax) is the negative of the optimalvalue of(Pmin)
Introduction
Optimalityconditions
Optimization problemsImportant remark: min is equivalent to max
(Pmax)
maxx∈X
f (x)
subject to
x ∈ C
is equivalent to problem
(Pmin)
minx∈X
−f (x)
subject to
x ∈ C
In the following sense:
x is solution of(Pmax) if and only if it is solution of(Pmin)
The optimal value of(Pmax) is the negative of the optimalvalue of(Pmin)
Introduction
Optimalityconditions
Optimization problemsExistence of solutions
(P)
minx∈X
f (x)
subject to
x ∈ C
When(P) has solution?
If [a,b] = C ⊆ X = R
WhenX = Rn, there exists at least one solution ifC is
closed and bounded
Introduction
Optimalityconditions
Optimization problemsExistence of solutions
(P)
minx∈X
f (x)
subject to
x ∈ C
When(P) has solution?
If [a,b] = C ⊆ X = R
WhenX = Rn, there exists at least one solution ifC is
closed and bounded
Introduction
Optimalityconditions
Contents
Introduction
Optimality conditions
Introduction
Optimalityconditions
Optimality conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
Optimality conditions are mathematical expressions: equationsor system of equations, system of inequalities, ordinarydifferential equations, partial differential equations
Solutions of the mathematical problems obtained fromoptimality conditions are related with the solutions of theoptimization problem
Introduction
Optimalityconditions
Optimality conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
Optimality conditions are mathematical expressions: equationsor system of equations, system of inequalities, ordinarydifferential equations, partial differential equations
Solutions of the mathematical problems obtained fromoptimality conditions are related with the solutions of theoptimization problem
Introduction
Optimalityconditions
Optimality conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
Optimality conditions are mathematical expressions: equationsor system of equations, system of inequalities, ordinarydifferential equations, partial differential equations
Solutions of the mathematical problems obtained fromoptimality conditions are related with the solutions of theoptimization problem
Introduction
Optimalityconditions
Optimality conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
The idea is to obtain optimality conditions and then to solvetheassociated mathematical problems (analytically or numerically)
Introduction
Optimalityconditions
Optimality conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
The idea isto obtain optimality conditionsand thento solve theassociated mathematical problems (analytically or numerically)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
Suppose thatoptimality conditionsof this problem arerepresented by a system of equations(S)
We say that(S) is a necessary condition if a solution of(P) is a solution of(S)
The idea is to find solutions of(S) in order to have a list ofcandidates for the solutions of(P)
If (S) is solved only for one point, then it is solution of(P)or (P) does not have solution
Warning: A solution of(S) may be not a solution of(P)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
Suppose that optimality conditions of this problem arerepresented by a system of equations(S)
We say that(S) is a necessary condition if a solution of(P) is a solution of(S)
The idea is to find solutions of(S) in order to have a list ofcandidates for the solutions of(P)
If (S) is solved only for one point, then it is solution of(P)or (P) does not have solution
Warning: A solution of(S) may be not a solution of(P)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
Suppose that optimality conditions of this problem arerepresented by a system of equations(S)
We say that(S) is a necessary condition if a solution of(P) is a solution of(S)
The idea isto find solutions of(S) in orderto have a list ofcandidates for the solutions of(P)
If (S) is solved only for one point, then it is solution of(P)or (P) does not have solution
Warning: A solution of(S) may be not a solution of(P)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
Suppose that optimality conditions of this problem arerepresented by a system of equations(S)
We say that(S) is a necessary condition if a solution of(P) is a solution of(S)
The idea is to find solutions of(S) in order to have a list ofcandidates for the solutions of(P)
If (S) is solved only for one point, then it is solution of(P)or (P) does not have solution
Warning: A solution of(S) may be not a solution of(P)
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions
(P)
minx∈X
f (x)
subject to
x ∈ C
Suppose that optimality conditions of this problem arerepresented by a system of equations(S)
We say that(S) is a necessary condition if a solution of(P) is a solution of(S)
The idea is to find solutions of(S) in order to have a list ofcandidates for the solutions of(P)
If (S) is solved only for one point, then it is solution of(P)or (P) does not have solution
Warning: A solution of(S) may be not a solution of(P)
Introduction
Optimalityconditions
Optimality conditionsNecessary condition: unrestricted case
(P){
minx∈R
f (x)
where f : R −→ R
A necessary condition isf ′(x) = 0 where
f ′(x) = limt→0
f (x+ t)− f (x)t
That means: Ifx is a solution of(P) thenf (x) = 0
If we know the expression off ′ we can solve the equationf ′(x) = 0 in order to have candidates of solutions
Warning!!!
Introduction
Optimalityconditions
Optimality conditionsNecessary condition: unrestricted case
(P){
minx∈R
f (x)
where f : R −→ R
A necessary condition isf ′(x) = 0 where
f ′(x) = limt→0
f (x+ t)− f (x)t
That means:If x is a solution of(P) thenf (x) = 0
If we know the expression off ′ we can solve the equationf ′(x) = 0 in order to have candidates of solutions
Warning!!!
Introduction
Optimalityconditions
Optimality conditionsNecessary condition: unrestricted case
(P){
minx∈R
f (x)
where f : R −→ R
A necessary condition isf ′(x) = 0 where
f ′(x) = limt→0
f (x+ t)− f (x)t
That means: Ifx is a solution of(P) thenf (x) = 0
If we know the expression off ′ we can solve the equationf ′(x) = 0 in order to have candidates of solutions
Warning!!!
Introduction
Optimalityconditions
Optimality conditionsNecessary condition: unrestricted case
(P){
minx∈R
f (x)
where f : R −→ R
A necessary condition isf ′(x) = 0 where
f ′(x) = limt→0
f (x+ t)− f (x)t
That means: Ifx is a solution of(P) thenf (x) = 0
If we know the expression off ′ we can solve the equationf ′(x) = 0 in order to have candidates of solutions
Warning!!!
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions: restrictions
(P)
minx∈R
f (x)
a ≤ x ≤ b
where f : R −→ R
At a pointx observe thatf increase in the sense of (thesign of) f ′(x)
There are three posibilities:
an interior pointa < x < b is a solution. Thenf ′(x) = 0
x = a is a solution. Thenf ′(x) = f ′(a) ≥ 0
x = b is a solution. Thenf ′(x) = f ′(b) ≤ 0
Observe that iff ′(a) > 0 andf ′(b) < 0, then there existsa < x < b such thatf ′(x) = 0
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions: restrictions
(P)
minx∈R
f (x)
a ≤ x ≤ b
where f : R −→ R
At a pointx observe thatf increase in the sense of (thesign of) f ′(x)
There are three posibilities:
an interior pointa < x < b is a solution. Thenf ′(x) = 0
x = a is a solution. Thenf ′(x) = f ′(a) ≥ 0
x = b is a solution. Thenf ′(x) = f ′(b) ≤ 0
Observe that iff ′(a) > 0 andf ′(b) < 0, then there existsa < x < b such thatf ′(x) = 0
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions: restrictions
(P)
minx∈R
f (x)
a ≤ x ≤ b
where f : R −→ R
At a pointx observe thatf increase in the sense of (thesign of) f ′(x)
There are three posibilities:
an interior pointa < x < b is a solution. Thenf ′(x) = 0
x = a is a solution. Thenf ′(x) = f ′(a) ≥ 0
x = b is a solution. Thenf ′(x) = f ′(b) ≤ 0
Observe that iff ′(a) > 0 andf ′(b) < 0, then there existsa < x < b such thatf ′(x) = 0
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions: restrictions
(P)
minx∈R
f (x)
a ≤ x ≤ b
where f : R −→ R
At a pointx observe thatf increase in the sense of (thesign of) f ′(x)
There are three posibilities:
an interior pointa < x < b is a solution. Thenf ′(x) = 0
x = a is a solution. Thenf ′(x) = f ′(a) ≥ 0
x = b is a solution. Thenf ′(x) = f ′(b) ≤ 0
Observe that iff ′(a) > 0 andf ′(b) < 0, then there existsa < x < b such thatf ′(x) = 0
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions: restrictions
(P)
minx∈R
f (x)
a ≤ x ≤ b
where f : R −→ R
At a pointx observe thatf increase in the sense of (thesign of) f ′(x)
There are three posibilities:
an interior pointa < x < b is a solution. Thenf ′(x) = 0
x = a is a solution. Thenf ′(x) = f ′(a) ≥ 0
x = b is a solution. Thenf ′(x) = f ′(b) ≤ 0
Observe that iff ′(a) > 0 andf ′(b) < 0, then there existsa < x < b such thatf ′(x) = 0
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions: restrictions
(P)
minx∈R
f (x)
a ≤ x ≤ b
where f : R −→ R
At a pointx observe thatf increase in the sense of (thesign of) f ′(x)
There are three posibilities:
an interior pointa < x < b is a solution. Thenf ′(x) = 0
x = a is a solution. Thenf ′(x) = f ′(a) ≥ 0
x = b is a solution. Thenf ′(x) = f ′(b) ≤ 0
Observe thatif f ′(a) > 0 andf ′(b) < 0, thenthere existsa < x < b such thatf ′(x) = 0
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions: restrictions
A necessary condition of the problem
(P)
minx∈R
f (x)
a ≤ x ≤ b
is
(S)
Find x, α, β such that
f ′(x) − α+ β = 0
x ∈ [a,b]
α, β ≥ 0
α(x− a) = 0
β(x− b) = 0
Introduction
Optimalityconditions
Optimality conditionsNecessary conditions: restrictions
A necessary condition of the problem
(P)
minx∈R
f (x)
a ≤ x ≤ b
is
(S)
Find x, α, β such that
f ′(x) − α+ β = 0
x ∈ [a,b]
α, β ≥ 0
α(x− a) = 0
β(x− b) = 0
Introduction
Optimalityconditions
Optimality conditionsGlobal vs local minimum
(P)
minx∈X
f (x)
subject to
x ∈ C
We say thatx ∈ C is a global minimum, if it is a solutionof (P)
We say thatx ∈ C is a local minimum, if for everyx ∈ Cclose enough, one hasf (x) ≤ f (x)
A global optimum is a local optimum
Introduction
Optimalityconditions
Optimality conditionsGlobal vs local minimum
(P)
minx∈X
f (x)
subject to
x ∈ C
We say thatx ∈ C is a global minimum, if it is a solutionof (P)
We say thatx ∈ C is a local minimum, if for everyx ∈ Cclose enough, one hasf (x) ≤ f (x)
A global optimum is a local optimum
Introduction
Optimalityconditions
Optimality conditionsGlobal vs local minimum
(P)
minx∈X
f (x)
subject to
x ∈ C
We say thatx ∈ C is a global minimum, if it is a solutionof (P)
We say thatx ∈ C is a local minimum, if for everyx ∈ Cclose enough, one hasf (x) ≤ f (x)
A global optimum is a local optimum
Introduction
Optimalityconditions
Optimality conditionsSufficient condition: unrestricted case
(P){
minx∈R
f (x)
We say that(S) is a sufficient condition if a solution of(S)is a solution of(P)
The idea is to find solutions of(S) in order to havesolutions of(P)
In the general case, this is very difficult
It is much realistic to have that solutions of(S) are localsolutions of(P)
A (local) sufficient condition to(P) is f ′(x) = 0 andf ′′(x) > 0
Introduction
Optimalityconditions
Optimality conditionsSufficient condition: unrestricted case
(P){
minx∈R
f (x)
We say that(S) is a sufficient condition if a solution of(S)is a solution of(P)
The idea isto find solutions of(S) in order to havesolutions of(P)
In the general case, this is very difficult
It is much realistic to have that solutions of(S) are localsolutions of(P)
A (local) sufficient condition to(P) is f ′(x) = 0 andf ′′(x) > 0
Introduction
Optimalityconditions
Optimality conditionsSufficient condition: unrestricted case
(P){
minx∈R
f (x)
We say that(S) is a sufficient condition if a solution of(S)is a solution of(P)
The idea is to find solutions of(S) in order to havesolutions of(P)
In the general case, this isvery difficult
It is much realistic to have that solutions of(S) are localsolutions of(P)
A (local) sufficient condition to(P) is f ′(x) = 0 andf ′′(x) > 0
Introduction
Optimalityconditions
Optimality conditionsSufficient condition: unrestricted case
(P){
minx∈R
f (x)
We say that(S) is a sufficient condition if a solution of(S)is a solution of(P)
The idea is to find solutions of(S) in order to havesolutions of(P)
In the general case, this is very difficult
It is much realistic to have thatsolutions of(S) are localsolutions of(P)
A (local) sufficient condition to(P) is f ′(x) = 0 andf ′′(x) > 0
Introduction
Optimalityconditions
Optimality conditionsSufficient condition: unrestricted case
(P){
minx∈R
f (x)
We say that(S) is a sufficient condition if a solution of(S)is a solution of(P)
The idea is to find solutions of(S) in order to havesolutions of(P)
In the general case, this is very difficult
It is much realistic to have that solutions of(S) are localsolutions of(P)
A (local) sufficient condition to(P) is f ′(x) = 0 andf ′′(x) > 0
Introduction
Optimalityconditions
Optimality conditionsSufficient condition
If we find x ∈ R such that
(S)
{
f ′(x) = 0
f ′′(x) > 0
Thenx is a local minimum of the problem
(P){
minx∈R
f (x)
Introduction
Optimalityconditions
Optimality conditionsSufficient condition
If we find x ∈ R such that
(S)
{
f ′(x) = 0
f ′′(x) > 0
Thenx is a local minimum of the problem
(P){
minx∈R
f (x)
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
To formulate an optimization problem,the followingconcepts must be clear:
Decision variable(s):x ∈ X
Objective function:f : X −→ R
Restrictions:C ⊂ X
A maximization problem is equivalent to a minimizationproblem (deal with−f (x) instead off (x))
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
To formulate an optimization problem,the followingconcepts must be clear:
Decision variable(s): x ∈ X
Objective function:f : X −→ R
Restrictions:C ⊂ X
A maximization problem is equivalent to a minimizationproblem (deal with−f (x) instead off (x))
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
To formulate an optimization problem,the followingconcepts must be clear:
Decision variable(s):x ∈ X
Objective function: f : X −→ R
Restrictions:C ⊂ X
A maximization problem is equivalent to a minimizationproblem (deal with−f (x) instead off (x))
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
To formulate an optimization problem,the followingconcepts must be clear:
Decision variable(s):x ∈ X
Objective function:f : X −→ R
Restrictions: C ⊂ X
A maximization problem is equivalent to a minimizationproblem (deal with−f (x) instead off (x))
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
To formulate an optimization problem, the followingconcepts must be clear:
Decision variable(s):x ∈ X
Objective function:f : X −→ R
Restrictions:C ⊂ X
A maximization problem is equivalent to a minimizationproblem(deal with−f (x) instead off (x))
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
Optimality conditions are mathematical systems
The idea is to solve these systems in order to:
have candidates of solutions (if the system is a necessarycondition)
obtain a local solution (if the system is a sufficientcondition)
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
Optimality conditions are mathematical systems
The idea isto solve these systemsin order to:
have candidates of solutions (if the system is a necessarycondition)
obtain a local solution (if the system is a sufficientcondition)
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
Optimality conditions are mathematical systems
The idea isto solve these systemsin order to:
have candidates of solutions(if the system is anecessarycondition)
obtain a local solution (if the system is a sufficientcondition)
Introduction
Optimalityconditions
Conclusions
(P)
minx∈X
f (x)
subject to
x ∈ C
Optimality conditions are mathematical systems
The idea isto solve these systemsin order to:
have candidates of solutions (if the system is a necessarycondition)
obtain a local solution(if the system is asufficientcondition)
Introduction
Optimalityconditions
Next part
(P)
minx∈X
f (x)
subject to
x ∈ C
Several decision variablesx1, x2, . . . , xn. That is
x =
x1
x2...
xn
∈ Rn = X
Necessary and sufficient conditions
Why the convexity (or concavity) is relevant?
Introduction
Optimalityconditions
Next part
(P)
minx∈X
f (x)
subject to
x ∈ C
Several decision variablesx1, x2, . . . , xn. That is
x =
x1
x2...
xn
∈ Rn = X
Necessary and sufficient conditions
Why the convexity (or concavity) is relevant?
Introduction
Optimalityconditions
Next part
(P)
minx∈X
f (x)
subject to
x ∈ C
Several decision variablesx1, x2, . . . , xn. That is
x =
x1
x2...
xn
∈ Rn = X
Necessary and sufficient conditions
Why the convexity (or concavity) is relevant?