1

SMU Course EMIS 8381

Nonlinear Programming

January 19, 2008

Hossam Zaki

2

Outline

• Course Scope – Classification of

Optimization Problems

• Course Syllabus– Description– Pre requisites– Text & References– Calendar– Grading

• On Line Resources• Course Topics

• Course Context– DSS Example

– DSS Paradigm

– Career Roles & Questions

3

Scope

4

What is an Optimization Problem?

• Optimization problems involve the selection of values of a number of interrelated variables by focusing attention on one or more selection criteria designed to measure the quality of the selection

5

Optimization Problem Statement

• Select the values of x

• From a set of possible values X – that satisfy a group of algebraic constraints

• In such a way that will optimize the value f(x)

6

Terminology

• Decision Variables = x

• Objective Function(s) = f(x)

• Feasible Region = X– Constraints defining X

• Inequality constraints : g(x)<=0• Equality constraints : h(x)=0

7

Optimization Problems Three Classifiers

Constraints Objective

Decision Variables

OptimizationProblems

8

DV Attributes

Sequential

InfiniteDims

FiniteDims

Discrete

Continuous

DV

9

Objective Attributes

Non Linear

Linear

NonDifferentiable

Differentiable

Closed form

From Simulation

Many

One

Objective

10

Constraints Attributes

Stochastic

Deterministic

Nonlinear

Block Diagonal

Network

GUB

Simple Bounds

Linear

UnConstrained

Constraints

1111

Optimization Problems

Constraints• Unconstrained• Constrained• Linear• Non linear • Simple bounds• GUB• Network• Block diagonal

Decision Variables (DV)• Discrete• Continuous• Finite • Infinite

Objective• One• Many• Differentiable• Non Differentiable• Closed Form• From Simulation• Linear• Non linear• Deterministic• Stochastic

Unconstrained

Continuous DV

Linear Constraints

One ObjDifferentiable Obj

© 2006, 2007 Zilliant, Inc. -- CONFIDENTIAL

Finite DV

Deterministic Obj

12

Course Scope

• Decision Variable– Continuous– Finite dimensions

• Objective– Single– Closed form or From Simulation– Linear or Nonlinear– Deterministic

• Constraints– Deterministic

13

Exercise # 1

1. Linear Programming

2. Goal Programming

3. Network Programming

4. Integer Programming

5. Non differentiable Optimization

6. Global Optimization

7. Dynamic Programming

8. Stochastic Programming

9. Quadratic Programming

10.Fractional Programming

11.Geometric Programming

12.Multi Objective Optimization

• Identify the attributes that define following optimization problems and prove one simple example. Display the results in table format

14

Course Syllabus

15

Course Description

• This course discusses, presents and explains the most important methods and results used to model and solve nonlinear optimization problems.

16

Prerequisites

• Advanced calculus (partial derivatives)

• Linear algebra (vectors and matrices).

• Knowledge on linear programming is helpful but not required (we will cover what we need).

17

Text & References

• Text Book– "Nonlinear Programming: Theory and Algorithms" by, M. Bazaraa,

H. Sherali and C Shetty, 3rd Edition, John Wiley, ISBN

• References– “Handbooks in OR & MS, Volume 1, Optimization”, Nemhauser et al

editors, North Holand, Chapters I and III, 1989

– Dennis & Schnabel, “Numerical Methods for Unconstrained Optimization”, Prentice-Hall, 1983

– Gill, Murray and Wright, “Practical Optimization”, Academic Press, 1981

– D. Luenberger, “Linear and Nonlinear Programming”, 2nd Edition, Addison Wesely, 1984

– D. Bertsekas, “Nonlinear Programming”, 2nd Edition, Athena

Scientific, 1999

18

Class Calendar

• Class (Section001) – Saturday 10 AM-12:50 PM

• 10 Minute break every 50 minutes

– Meets in 203 Junkins

• 14 Class Periods– First Class Period: January 19

– No Class: March 22

– Last Class Period: April 26

19

Grading

• In-class Exams (70%)– Mid Term on 3/8/ 08 = (30%)– Final on 4/26/08 = (40%)

• Term Paper & Presentation (20%)– A nonlinear optimization topic, e.g. algorithm,

application and /or software demo not covered in class

– Due on 3/29/08

• Homework (10%)

20

On Line Resources

21

Mathematical Programming Glossary

http://glossary.computing.society.informs.org/index.php?page=N.html • General Information - A list of dictionaries, suggested methods of

citation, and contribution instructions. • The Nature of Mathematical Programming - See this for basic terms

and a standard form of a mathematical program that is used throughout this glossary.

• Notation - Read this to clarify notation. • Supplements - A list of supplements that are cited by entries. • Myths and Counter Examples - Some common and uncommon

misconceptions. • Tours - Collections of Glossary entries for a particular subject. • Biographies - Some notes on famous mathematicians. • Please remember this is a glossary, not a survey, so no attempt is

made to cite credits.

22

1998 Nonlinear Programming Software Survey

• http://www.lionhrtpub.com/orms/surveys/nlp/nlp.html

• The information in this survey was provided by the vendors in response to a questionnaire developed by Stephen Nash. The survey should not be considered as comprehensive, but rather as a representation of available NLP packages. The listings are limited to products that fit the parameters of the survey as outlined in the accompanying article.

23

Nonlinear ProgrammingFrequently Asked Questions

http://www-unix.mcs.anl.gov/otc/Guide/faq/nonlinear-programming-faq.html

• Q1. "What is Nonlinear Programming?" • Q2. "What software is there for nonlinear

optimization?" • Q4. "What references are there in this

field?" • Q5. "What's available online in this field?"

24

NEOS Guide

http://www-fp.mcs.anl.gov/OTC/Guide/ • The Optimization Tree. Our thumbnail sketch of

optimization (also known as numerical optimization or mathematical programming) and its various sub disciplines.

• The Optimization Software Guide. Information on software packages from the book by Moré and Wright, updated for the NEOS Guide.

• Frequently Asked Questions on Linear and Nonlinear Programming. These are the FAQs initiated by John Gregory, now maintained by Bob Fourer as part of the NEOS Guide.

25

MIT Nonlinear Course

• Prof. Dimitri Bertsekas http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Computer-Science/6-252JNon-linear-ProgrammingSpring2003/CourseHome/

26

UCLA Nonlinear EE Course

• Prof. Lieven Vandenberghe

http://www.ee.ucla.edu/ee236b/

27

Course Topics

28

Course Topics

• Chapter 1– Intro

• Appendix A– Math Review

• Chapter 2– Convex Sets

• Chapter 3– Convex Functions

• Chapter 4– Optimality Conditions

• Chapter 6– Lagragian Duality

• Chapter 8– Unconstrained Opt

• Chapter 9 – Penalty and Barrier

• Chapter 10– Methods of Feasible

Directions

29

Exercise # 2

Plot and solve graphically:

Maximize x1

Subject to

h1(x1,x2) = x12 – x2 + a = 0

h2(x1,x2) = – x1 + x22 + a = 0

For the following values of a:a = 1, 0.25, 0 and -1

30

Course Context

31

DSS Example

Simulation Model

StochasticShow-up rates

StochasticDaily

Demand

Stochastic Success Rates

Daily Resources

FeedbackCapacity

Configuration

OperationsCharacteristics

Optimization ModelBusiness Rules

Alternative Resources

Existing Capacity

Aggregate DemandProfile

Cost of newResources

Timing Constraints

What if

32

Decision Support System Development Methodology

DV = Decision VariableDB = Database

9. Solve Real Problems

2. Develop DSS High Level Design

3. Formulate Mathematical Model

DV, Objective, Constraints, Goals

5. Estimate and/or Forecast needed data, e.g. demand

6. Develop or Select Solver, prepare input

files, connect to DB

8. Refine Formulation: Aggregate, Decompose,

Transform

7. Prepare & Solve Small Sample Problems

10. Prepare & analyze output.Validate results

12. Estimate Lift (Improvement)

ITERATE

11. Perform What-if

1. Understand Business Context and Formulate Problem Statement in English

4. Receive, Clean and Synthesize Business Data.

Create DB

33

How will the course help u with career questions?

P

SVP

VP

Director

Senior Manager

Manager

Senior Scientist

Scientist, OR Specialist, Consultant

Which problems to invest $ in?

Reformulate?Which solver to use?

How to formulate?

Which parameters?

What is the impact of a data change?

Which is the expected lift?

How many person-months?

How to prioritize tasks?

How to gain tech diff over competitors?

How to validate results?

Should we productize?

Develop in house or buy? How many scientists do we need?

Which algorithm?

Service or license?

34

Questions?