Title of Presentation

Name: Mehrab Khazraei(145061)Title: Penalty or Exterior penalty function method professor Name: Sahand Daneshvar

IntroductionPenalty or Exterior penalty function method is an approach that deal with the non-linear programming problems, having equality and inequality constraints. This method converts inequality constraints to equalities or into problems having simple bound. Exterior penalty function adds a penalty to the objective function to penalize any violation of the constraint. This procedure generates a sequence infeasible points whose it limits is an optimal solution associated with the original problem.

Methods using penalty functions transform a constrained problem into a single unconstrained problem or into a sequence of unconstrained problems. Assume the following problem having the single constraint h(x) = 0:

Minimize f ( x )subject to h(x) = 0.

It could replace by the following unconstrained problem, where μ > 0 is a large number: Minimize f ( x ) + μh2 (x) subject to x Є R n

obviously must h2(x) be close to zero.Now consider the following problem having single inequality constraint g(x) 5 0: Minimize f ( x ) subject to g(x) < O

It is clear that the form f ( x ) + pg2(x) is not appropriate, since a penalty will be incurred whether g(x) < 0 or g(x) > 0. Needless to say, a penalty is desired only if the point x is not feasible, that is, if g(x)> 0. A suitable unconstrained problem is therefore given by: Minimize f ( x ) + μ maximum (0, g(x)} subject to x Є R n

If g(x) < 0, then max{0, g(x)} = 0, and no penalty is incurred. On the other hand, if g(x) > 0, then max{0, g(x)} > 0 and the penalty term μg(x) is realized. However, observe that at points x where g(x) = 0, the foregoing objective function might not be differentiable, even though g is differentiable. If differentiability is desirable in such a case, we could, for example, consider instead a penalty function term of the type μ [max{0, g(x)}]2.

In general, a suitable penalty function must incur a positive penalty for infeasible points and no penalty for feasible points. If the constraints are of the form gi(x) < 0 for i = 1 ,..., m and hi(x) = 0 for i = 1, ..., l, a suitable penalty function α is defined by α(x)= ∑ φ[gi(x)]+∑ψ[hi(x)]

where φ and ψ are continuous functions satisfying the following:φ(y) = 0 if y<0 and φ(y)>0 if y>0φ(y)=0 if y=0 and φ(y)>0 if y≠0Typically, φ and ψ are of the formsφ(y) = [max{O, y }]p

ψ(Y) =׀y׀p

Where μ is a positive integer. Thus, the penalty function α is usually of the form α(x)= ∑ [max{0, gi(x)}]p+ ∑ ׀hi(x)׀p

Exterior Penalty Function Methods:Presents and proves an important result that justifies using exterior penalty functions as a means for solving constrained problems. Consider the following primal and penalty problems.

Primal Problem Minimize f(x) subject to g(x) < 0 h(x) = 0 x Є X

Where g is a vector function with components gl

,..., gm and h is a vector function with components h1 ,..., hl . Here, f, g1, ..., gm, h1 ,..., h1 are continuous functions defined on Rn, and X is a nonempty set in Rn. The set X might typically represent simple constraints that could easily be handled explicitly, such as lower and upper bounds on the variables.

Penalty ProblemLet α be a continuous function of the form that satisfying the properties. The basic penalty function approach attempts to find

SUP Ө(μ) subject to μ > 0 Where Ө(p) = inf{f(x) + μα(x): x E X )} The main theorem of this section states that inf{f(x) : x Є X , g(x) < 0, h(x) = 0} = sup Ө(μ) = lim Ө(μ)

From this result it is clear that we can get arbitrarily close to the optimal objective value of the primal problem by computing Ө(p) for a sufficiently large μ. This result is established in the next theorem, however, the following lemma is needed.

LemmaSuppose that f; gl, ..., g,, h,,..., hl are continuous functions on Rn, and let X be a nonempty set in Rn . Let a be a continuous function on Rn given by α(x)= ∑ φ[gi(x)]+∑ψ[hi(x)] and suppose that for each μ, there is an x μ Є X such that Ө (μ) = f ( x μ) + μ a(x μ).Then, the following statements hold true:

1. Inf{f(x): x Є X, g(x) < 0, h(x) = 0} > sup Ө(μ)

where

Ө(μ),=inf{f(x) + μα (x): x Є X }and where g is the vector functionwhose components are gl, ..., gm, and h is the vector function whose components are h1,..., hl.

2. f(x μ) is a non-decreasing function of μ > 0, Ө(p) is a non-decreasing function of μ, and α (x,) is a non-increasing function of μ .

Initialization StepLet E > 0 be a termination scalar. Choose an initial point x l , a penalty parameter μ 1> 0, and a scalar B > 1. Let k = 1, and go to the Main Step.

Main Step1. Starting with X k , solve the following problem: Minimize f ( x ) + μk

α (x) subject to x Є XLet x k+1 be an optimal solution and go to Step 2

2. If μk α (xk+1) < E, stop;

otherwise, let μk+1 = B μk replace k by k + 1, and go to Step 1.

ExampleConsider the following problem: Minimize (xl - 2)4 +(XI - 2x2 )2

subject to x 12 - x2

2 = 0

Note that at iteration k, for a given penalty parameter μk, the problem to be solved for obtaining x μk is, using the quadratic penalty function:Minimize (XI - 2)4 + (xl - 2 x2)2+ μk (x 1 - x2 )2

Table summarizes the computations

THANK YOU FOR ATTENTION