Discrete Optimization

Shi-Chung Chang

Discrete Optimization Lecture #1

Today:

Reading Assignments1. Chapter 1 and the Appendix of [Pas82]2. Chapter 1 of [GaJ79]

Outline:1. Course Overview

» A taxonomy of optimization problems» Course introduction» Requirements and schedule

2. Some Basics of Optimization» Local and global optima» Feasibility» Convexity» Convex Programming

3. Algorithms and Complexity» Problems, algorithms, and Complexity» Polynomial time algorithms» Intractability» NP-Complete Problems

§I.1 Course Overview

• Introduction to Optimization Problems Ingredients of an optimization problem

• A set of independents on the values of variables or parameters• Condition or restrictions on the values of variables• Criterion or objection find the best solution

Example 1: 養馬問題

又要馬兒肥，又要馬兒不吃草

P(x)

XGrass

HorsePrice Expense

XGrass

E(X)

)()(max xExpx

Rx0

• A Standard Mathematical Form max F(x)x sSubject to (x)=0 i=1,…,m (x)≤0 j=1,…,r• Classifications(1) By the time factor

static

dynamic

jghi

Subject to X(t)=

a ≤ u(t) ≤ b 0 ≤ x(t) ≤ R

(2) By the nature of variables

S

=> nonlinear programming => discrete(combinatorial) optimization =>mixed integer programming

I R

0t du

In

Rn

C(t)

Grass Price Factor

Example 2: Example 2:

養馬問題 max P(x())- c(t)E(u(t))d x(t)

t f tt f0

t

(3) By the nature of problem functions

Properties of F(x) Properties of g(x)&h(x)

Function of a single variable Linear function

No Constraints

Sum of squares of linear functions

Simple bounds

Quadratic functions Linear functions

Sum of squares of nonlinear functions

Sparse linear functions

Smooth function Smooth nonlinear functions

Sparse nonlinear function Non-smooth nonlinear functions

Non-smooth function

In this course , we will consider problems with discrete variables.

• Example: The Traveling Salesman Problem (TSP)

Cities: , , , Distance between cities d( , )Find the shortest path that goes through every city once and only once and back to the starting city.Q: Why is this a discrete optimization problem?

c1 c2

c2

c1

c3

c3

c4

c4ci c j

Course Introduction

Nonlinear programming

Convex programming

Linear programming Integer

programming

.

• Linear Programming minimize c’ x x subject to A1 x=b1

A2 X≤ b2

Fact : Optimal solution happens at a vertex discrete (combinatorial) nature LP serves as a bridge between the continuous and discrete

optimization

x1

x2 c

Rn

• Optimization on Networks Example: The general minimum-cost flow problem minimize subject to - = bi i=1,2…,n [Flow balance] ≤ ≤ [Flow balance] LP with special constraint structure efficient solution techniques available by exploiting such a structure

Variations: shortest path problem minimum spanning tree maximum flow Minimum-cost flow

i

j ijc ijx

j ijx k

kix

ijl ijx iju

b1 1 3

4

6

7

2 7 8

x13c13

[ , ]l13 u13

Emphases:(1)Exploiting special problem structure(2)Applications in EE&CS(3)Distributed or parallel algorithms(4)Foundations for problems with nonlinear objectives or other

discrete optimization problems

Integer Programming basic techniques for general problems NP-completeness TSP Knapsack Problems min s. t.

Scheduling Problems

Simulated Annealing an approach to global optimization

xp i

n

ii

1

ni

v

x

xw

i

i

n

ii

,...,2,1

,...1,01

§I.2. Basics of Optimization

• Definition of on Optimization Problem

J= i.e. a mapping A: F ->

• An instance of an optimization: (F,A)• Feasibility : x is feasible e.g.

minFx x arg

0x

min)(xg

R1

Fx

mjx

nixxF

gh

j

i

,...,1,0

,...,1,0

Fx

• Neighborhood: Given an optimization problem with instances(F,A), a

neighborhood is a mapping N: F -> example: (a) In LP with FC , we can define a neighborhood

2F

Rn

xyandybyAyxN ,0,)(

Q: what to do with discrete cases? Example:

2-change of ={g: and g can be obtained from f by

removing two edges from the tour and then replacing them with two edges}

c1

c2

c3 c4

Ff)(

2fN fg

• Local and Global Optima Example: continuous case

A and C are local minima B is the global minima

Q: How to define them for discrete optimization?

F(x)

0 a x

A

BC

Definition : Local Optima Given an instance (F,A) of an optimization problem, and a neighborhood N, is locally optimal with respect to N if A(f)≤A(g) example: A best TSP tour in may not be the solution

tour. Definition : Global Optima If and is locally optimal w.r.t a neigborhood N, it

is then also locally optimal to any other neighborhood f is globally optimal N is exact.

Q: How to check with respect to all N? example: In TSP, is not exact but is for an n-city

problem.

Ff fNg

N 2

Ff

N

N 2 N n

Discrete Optimization Lecture #2

Last time: Course overviewSome basics of optimization

Today: Reading Assignments: 1. Chapter 1 of [GAJ79] 2. Sections 2.1~2.3 of [PaS82] 3. Sections 2.4,2.5,3.1~3.8 of [Lue84].

Outline: 1. Some basics of optimization (cont.)

convexity & Convex Programming 2. Algorithms and Complexity

Problems, algorithms, and complexity Polynomial time algorithms Intractability NP-Complete Problems

3. Basic properties of Linear Programs From of LP Basic feasible solutions

Geometry of LP 4. The simplex method Homework#1 Due:

I.3 Convexity and Convex Programming

Why are we so interested in convex functions and convex sets? min

(1) Globally speaking, If J, F are convex minimum points are global minimum(2) Local convexity local minimumDefinition: Convex Set is convex iff and Q: Facts about convex sets

(1) A hyperplane = is a convex set(2) A half space ≤ is a convex set

(3) Intersection of convex sets is convex Union of convex set?

(4) Contraction of a convex set is convex

(5) An empty set is convex

Fx )(xJ

Rn

cS ,Sx Sy

Syx )1( 10

cT x

cT x

z

z

Definition Convex Functionsf:

If

Then f is convex• Facts about convex functions

(1) A quadratic function is convex if Q≥0(2) The linear extrapolation(approximation) at a point underestimates a convex functioni.e. assume

RRn 1

Rn

yx ,

yfxfyxf 11

10

cxbxQxT

x

xxxxcf 2121,,,

xxxxx Tfff12111

x

xf 2

xxxfxT

f1211

x1 x2

(3) f is convex iff is positive semi-definite over

proof: read by yourself(4) Linear combination ( positive coeff.) of convex functions is convex (5) The level set is convex for all c if f is convex

Min

s.t. is a convex programming problem if f. g. and are convex.

cf2 xf2

x

cxfxxc

,

x xf

0xg

Properties of Convex Minimization(a) s= arg min is a convex set Proof: are two optimal solutions by def. of min.

x xf

xxtt

21

fxxttt

ff 21

ffxxxxtttttt

fff 2121

11

Sxx tt21

1

(b) Every local minimum of a convex programming problem is also a global minimum

Proof: are two local minimum and < < <

xx tt

21

xtf1

xxxxxttttt

ffff12122

11

xxx tttfff

111

xtf1

Q. (a)+(b) implies that when the solution is not unique, all are equivalent

f(X)

s