reduction s & np-completeness jeff edmonds york university historyhistoryhistory course vs...

Reduction s & NP-Completeness

Jeff Edmonds

York University

• History • Course vs Schedule• More about Reductions• Circuit vs Airplane• Circuit vs 3-Colouring• Conclusion• Definitions for 2001/3101• More Classes for 4111• More Detailed Steps for 6111

Thanks Intro & COSC 3101/4111/6111

Colour each node.

Nodes with lines between them must have different colours.

Can you do it with 2 colours?

Can you do it with 3?

Graph Colouring

Please feel free to ask questions!

History of Classifying Problems

Known

Unknown

Euclid (300 BC)

GCD


KnownGCD

Computable Halting

Turing 1936


Computable

Known

GCD

Halting

Exp

Poly

Jack Edmonds 1965


Computable

Exp

Poly

Known

GCDMatching

Halting

Jack Edmonds Steve Cook

NP

• exponential time to search• poly time to verify given witness

Non-Deterministic Polynomial Time

Circuit-Sat Problem: Does a circuit have a satisfying assignment.

SAT


Computable

Exp

Poly

Known

GCDMatching

Halting



SAT


NP

• exponential time to search• poly time to verify given witness


Computable

Exp

Poly

Known

GCDMatching

Halting



NP

Is it easier than other exponential problems?

Games

SAT


Computable

Exp

Poly

Known

GCDMatching

Halting



NP

Is it harder than poly-time problems?

Games

SAT


Computable

Exp

Poly

Known

GCDMatching

Halting

Steve Cook 1971


NP

Let’s compare the problems in NP

SAT

Games

None are harder than SAT!

completepoly Prob’

SAT

Free Lunch

Industry would love a free lunch

• Given a description of a good plane, automatically find one.

• Given a circuit, find a satisfying assignment

• Given a graph find a bichromatic coloring.

• Given course description, find a schedule

X

X X X X

Ingredients:• Instances: The possible inputs to the problem

(and desired cost). • Solutions for Instance: Each instance has an

exponentially large set of solutions. •Valid: Each solution has an easy to compute

cost or value. •Goal: The output is one of the valid solutions

for this instance with optimal cost. (minimum or maximum)

Optimization Problems

•A large space of possibilities•Only a tiny fraction of which satisfy the constraints.

•Brute force takes too long, •No feasible algorithm is known.

Rudich www.discretemath.com

• If one has a fast algorithm•Then they all do.• If one has no fast algorithm •Then none do.

All of these problems have a common story:

Course Scheduling

Given courses students want and the time slots availableschedule courses to time slotsto minimize number of conflicts,i.e. student wants two courses that are scheduledat the same time.

Perspiration Search

Try all possible Schedules

Too many to try. A set of 50 courses has more schedules than the number of atoms.

A Fast Algorithm

Is there a fast algorithm?

Most people think not.

We have not been able to prove that there is not.

It is one of the biggest open problems in the field.

A Graph

Represents • Cities & Roads • Computers & Networks • People & Friendships

A Colouring of a Graph

Colour each node.


Perspiration Search

Try all possible colourings

Too many to try. A 50 node graph has more colourings than the number of atoms.

A Fast Algorithm

Is there a fast algorithm?

Most people think not.

We have not been able to prove that there is not.

It is one of the biggest open problems in the field.

Problems are the Same!

Schedule each course.Courses that conflict can’t be at same time.

Colour each node.Nodes with lines between them must have different colours.

»

Two problems that are cosmetically different,

but substantially the same

Problems are the Same!

Schedule each course.Courses that conflict can’t be at same time.

Colour each node.Nodes with lines between them must have different colours.

»

English

MathScience

course » node

can’t be scheduled at same time

line between them»

scheduled time » colour

1pm Mon

1pm Mon

3pm Fri

Act on

Act with

Constraint

Action

Algorithms Inter-Changeable

Colour Oracle

Sched.

Oracle

Given an oracle for 3-colorability, how can you quickly solve scheduling?


≤Colour Oracle

Sched.

Oracle

Assume have an alg.

Proof by Reductions

Design an alg.


Courses & Conflicts

Colour Oracle

Sched.

Oracle

Schedule

Proof by Reductions

<G, k>

Colouring

≤


≥Colour Oracle

Sched.

Oracle

Assume have an alg.

Proof by Reductions

Design an alg.


<G,k>

Colour Oracle

Sched.

Oracle

Colouring

Proof by Reductions

Schedule

Courses& Conflicts


» Colour Oracle

Sched.

OracleProof by Reductions

But there is likely not a fast algorithm for either!!!

Reductions Iharder

Iharder yes

Pharder

Iharder no

Ieasier

Ieasier yes

Peasier

Ieasier no

Two problems/languages Peasier and Pharder

each with their own definitions of• what a legal input is (Ieasier vs Iharder) and • which are yes-inputs and which no-inputs.

Reductions Iharder

Iharder yes

Pharder

Iharder no

Ieasier

Ieasier yes

Peasier

Ieasier no

We want machines that decide/accept them.

Algharder

Pharder(Iharder)IharderPeasier(Ieasier)Ieasier

Algeasier

ReductionsPeasier ≤comp Pharder

Iharder

Iharder yes

Pharder

Iharder no

Ieasier

Ieasier yes

Peasier

Ieasier no

How do their complexities/difficulties

compare?

Computable/Decidable

Exp

Poly

NP Co-NP

Recognizable Co-Recognizable

ReductionsPeasier ≤comp Pharder

Iharder

Iharder yes

Pharder

Iharder no

Ieasier

Ieasier yes

Peasier

Ieasier no

It is hard to prove problem is Pharder hard.

It is easier to prove Peasier is easy.

• by design an algorithm Algeasier for it.

But you only need to prove Peasier is at least as easy as Pharder .

• Pretend you have an algorithm Algharder for Pharder

to use as a subroutine.

ReductionsPalg ≤comp Poracle

Ioracle

Ioracle yes

Poracle

Ioracle no

Ialg

Ialg yes

Palg

Ialg no

We often call the algorithm assumed to exist, an Oracle. Given

Algoracle

Poracle(Ioracle)Ioracle


Ioracle

Ioracle yes

Poracle

Ioracle no

Ialg

Ialg yes

Palg

Ialg no

We use Algoracle as an subroutinein an algorithm Algalg for solving Palg.

GivenAlgoracle

Poracle(Ioracle)Ioracle

BuildAlgAlg

Palg(Ialg)

Ialg


IoracleIalg

Ialg

Algalg is given an input Ialg.

GivenAlgoracle

Poracle(Ioracle)IoracleIoracle = InstanceMap(Ialg)

BuildAlgAlg

Palg(Ialg)

Ialg

InstanceMap

Ioracle

It maps it to input Ioracle.and gives this to Algoracle.


IoracleIalg

Ialg

Algoracle gives the yes/no answer for his input Ioracle.

GivenAlgoracle


BuildAlgAlg

Palg(Ialg)

Ialg

Return same answer yesyes no no

InstanceMap

Ioracle yes

Poracle

Ioracle no

Algalg returns the same answer.


IoracleIalg

To ensure Algalg works, Ioracle = InstanceMap(Ialg) must

GivenAlgoracle


BuildAlgAlg

Palg(Ialg)

Ialg


InstanceMap

Ioracle yes

Poracle

Ioracle no

Ialg yes

Palg

Ialg no

map yes instances to yes instances and no to no.


IoracleIalg

Must prove Algalg works.

GivenAlgoracle


BuildAlgAlg

Palg(Ialg)

Ialg


InstanceMap

Ioracle yes

Poracle

Ioracle no

Ialg yes

Palg

Ialg no

Ialg is a yes input Ioracle

is a yes input Algoracle

says yes Algalg says yes

Reductions

• Reduction: Design an algorithm for one computational problem, using a supposed algorithm for another problem as a subroutine. • Used to create a new algorithm from a known algorithm.• Learn that a new problem is likely hard,

from knowing a known problem is likely hard.• Learn that two problems have a similar “structure.”

Palg ≤comp Poracle

GIVEN:Network

Flow Oracle

BUILD:Matching

Oracle

Matching ≤comp Network Flows

Who loves whoAnnFred

SueJohn

BethBob

MarySam

Max matching

A network

s t

Max Flow

s t

Reductions

• Learn that a new problem is likely hard, from knowing a known problem is likely hard.

Reductions

HaltingComputable

Exp

Poly

If there is an algorithm for Palg

then there is an algorithm for Palg

then there is not fast

algorithm for Poracle

If there is not an algorithm for Palg

If there is an algorithm for Poracle

If there is not an algorithm for Poracle

?

?

We give an algorithm for Palg using a supposed algorithm for Poracle as a subroutine.

?

???

??

Reductions

Notation: Palg ≤poly Poracle

Palg is “at least as easy as” Poracle

(Modulo polynoM'al terms.)

Poracle is “at least as hard as” Palg

(Modulo polynoM'al terms.)

Conclusions:

We give an algorithm for Palg using a supposed algorithm for Poracle as a subroutine.

Reductionspoly

This is a circuit with

AND/OR/NOT gates

with n input variables/wires

and one output wire.

This is an assignment giving true/false, yes/no, 0/1

to each input variable.

These values percolatedown to the output wire.

F T Fx3x2x1

OR

ORANDAND

OR

NOT

F F

F

F F

T

The Circuit Satisfiablity Problem

This is a circuit with

AND/OR/NOT gates

with n input variables/wires

and one output wire.

This is an assignment giving true/false, yes/no, 0/1

to each input variable.

These values percolatedown to the output wire.

x3x2x1

OR

ORANDAND

OR

NOT


The assignment satisfies the circuit if the output is T.

F F F

F F

T

F T

F

x3x2x1

OR

ORANDAND

OR

NOT


F F F

F F

T

F T

F

Given a circuit, what is a fast algorithm to find a satisfying assignment?

Or even to know whethersuch an assignment

Try all possible

assignments

Too many to try. 50 input wires has more assignments than the number of atoms

We know no faster algorithm!

• Instances: A circuit Icircuit.• Solution: An assignment Scircuit •Valid: Scircuit is a valid solution for instance Icircuit

if it satisfies it, i.e. Icircuit(Scircuit) = T •There is a poly-time alg Validcircuit(Icircuit,Scircuit) to test

whether or not Scircuit is a valid solution for Icircuit.•Goal: Given Icircuit, find a valid solution Scircuit.

Or determine if such a solution exists.


Pcircuit = { Icircuit | Circuit is satsisfiable

i.e. Scircuit, validcircuit(Icircuit,Scircuit) }

• Instances: Requirements Iplane of the plane i.e. cost, efficiency, safety, speed, weight. …

• Solution: A description Splane of how to make the wing. •Valid: Given an instance Iplane and a solution Splane,

there is a poly-time alg Validplane(Iplane,Splane) to test whether or not Splane is a valid solution for Iplane.

•Goal: Given Iplane, find a valid solution Splane. Or determine if such a solution exists.

Design an Airplane

Can be computed by a circuit C.

Pplane = { Iplane | Plane is buildable

i.e. Splane, validplane(Iplane,Splane) }

Reductions for Undecidability

Given an algorithm for Pcircuit,I construct a working algorithm

for Pplane.

Proof Pcircuit is likely not in poly-time.

By way of contradiction assume it.

Contradiction because Pplane likely does not have a poly alg.

Contrapositive:I likely do not have a poly alg for Pplane,hence I likely do not have one for Pcircuit.

{ Icircuit | Circuit is satsisfiable

Scircuit, validcircuit(Icircuit,Scircuit) }{ Iplane | Plane is buildable

Splane, validplane(Iplane,Splane) }≤


Consider the form of the instances of each problem

and which are yes-instances.

Icircuit

Isatisfiable

Pcircuit

Iunsatisfiable

Iplane

Ibuildable

Pplane

Iunbuildable





To ensure Algalg works, Ioracle = InstanceMap(Ialg) must

InstanceMap

map yes instances to yes instances and no to no.

Icircuit

Isatisfiable

Pcircuit

Iunsatisfiable

Iplane

Ibuildable

Pplane

Iunbuildable





≤Circuit SatOracle

PlaneDesign

Oracle

Have an algfor subroutine

Proof by Reductions

Design an alg.


Plane Requirements I

CircuitSat Oracle

DesignPlane Oracle

Plane Description S

Proof by Reductions

C computes Valid(I,?)

S

≤

»Two problems that are cosmetically different, but

substantially the same

Circuit ≤poly 3-Colouring

GIVEN: Alg for 3-colourproblem

BUILD: Alg for circuit problem

C Graph composed of gadgets that mimic the gates in C

colourable or not

satisfiable or not


Circuit Problem:• instance = circuit

of gates• solution = true/false

to each variable• true/false

to internal wires

3-Colouring• instance = graph

• solution = green/red/blue to each node.

F T T

OR

OR

NOT

Output

x y z

G Col

?

of gadgets


A Graph Named “Gadget”

Steven Rudich had computer search for graph with some key properties.

Colour each node.


Could you do it with 2 colours?

No

Colour each node.


Could you do it with 3 colours?

Yes.

How do we know?

Because we colored it!

X Y

Output

variables&wires ~ nodesgate ~ gadgetOR

x y

Output

X Y

Output

Consider your colouring

T F

true ~ greenfalse ~ red ? ~ yellow

F T

OR

x y

Output

T

redyellowgreen

X Y

Output

true ~ greenfalse ~ red ? ~ yellow

OR

x y

Output

F T

T

redyellowgreen

Re-map your colouring

T F

T FX Y

Output?

gate function ~ gadget “function”OR

x y

OutputTT

FT

TF

FF

OrYX

F

T

T

T

F T

T

T FX Y

Output

X Y Out

F F

F T

T F

T T

?F

F


gate function ~ gadget “function”OR

x y

Output

F F

F

F F

T FX Y

Output

F T

T

T

X Y Out

F F

F T

T F

T T


gate function ~ gadget “function”

F T

OR

x y

Output

T

T FX Y

Output

FT

X Y Out

F F

F T

T F

T T

T

T



FT

OR

x y

Output

T

T FX Y

Output

T T

T

T

X Y Out

F F

F T

T F

T T



T

OR

x y

Output

T

T

X Y

Output

T F Consider your colouring

X Y Or

F F F

F T T

T F T

T T T

Or - Gadget


F T

OR

x y

Output

T

T FX Y

Output

F T

T

T

X Y Or

F F

F T

T F

T T

Proof

Or - Gadget


F T

OR

x y

Output

T

T F

X

Output

X NOT

F T

T F

NOT - Gadget

OR

OR

NOT

x y z

xy

z

Output

Output

OR

OR

NOT

x y z

xy

z

OR

Output

Output

OR

OR

NOT

x y z

xy

z

Not

Output

Output

OR

OR

NOT

x y z

xy

z

OR

Output

Output

OR

OR

NOT

x y z xy

z F T T

T

FT

T

F

Satisfiabilty of this circuit = 3-colourability of this graphOutput

Output

Reduction

=

Two problems that are cosmetically different, but substantially the same

Satisfiability of this circuit = 3-colourability of this graph

Free Lunch

Industry would love a free lunch

• Given a description of a good plane, automatically find one.

• Given a circuit, find a satisfying assignment

• Given a graph find a bichromatic coloring.

• Given course description, find a schedule

X

X X X X

NP-Complete Problems

•A very large class of problems:

• Industry would love to solve them quickly •We don’t know how. • If we could solve one quickly, Þ we could solve all quickly.

• If one impossible to solve quickly Þ all impossible to solve quickly.

Boss assigns task:

Given courses students want, schedule them to minimize conflicts.

Everyday industry asks these questions.

Your answer:

Sorry. Very likely no fast algorithm exits that always finds best schedule.

Worthy of wealth & Nobel prize

Else have fast algorithm for all NP-problems.

Cryptography

Sometimes it is useful to know that a problem is computationally hard to do.

For example, breaking into your secret documents.

The End(Of quick overview of NP)

Reductions & NP-Completeness

Jeff Edmonds

York University

• Classes of Problems• Non-Deterministic Polynomial Time• NP vs Other Classes• NP-Complete• Reductions

Thanks Intro & COSC 3101/4111/6111

Definitions NeededFor 3101

CoNPnot SAT

Computable

Exp

Poly

NP

GCD

SAT

HaltingAcceptable CoAcceptable

Not Halting

Classes of Problems

Poly

GCD

Poly:Problems P computable in polynomial. A,c I A(I) = P(I) and Time(A,I) |I|c

Polynomial Time

• Each instance I • has an exponential number of possible

solutions. • Each solution S is either valid or not

(sufficiently high value or not)• The output is • Yes, it has a valid solution.• No, it does not• the solution need not be returned

• There is a poly-time alg Valid(I,S) to test given an instance I and a solution S, whether or not S is a valid solution of I.

• Eg: Network Flows Given I = <G,k>, does network G have flow of value k?

NP


Poly

Network Flows

NP

Non-Deterministic Polynomial Time• Example: Satisfiablity:• Instance: A circuit I.• Solution: A satisfying assignment S. • I is a yes instance iff there is such an assignment . • Given an instance I and a solution S,

there is a poly-time alg Valid(I,S) to test whether or not S is a valid solution of I.

0 1 1 0 1 0 1 Assignment S:

1

Sat(I)=Yes iff S Valid(I,S)

Circuit I:

SAT

Poly

NP

Formal definition:Prob NP iff poly time Validsuch that Prob(I) = Yes iff S Valid(I,S)


SAT

Poly




A Non-Deterministic TM/Java/… is • the same as a deterministic one,

except that its moves are not deterministic. • It has a “choice” as to which step to take next.• Deterministic: (qi,c) = <qj,c',right>

• Non-Deterministic: <qi,c; qj,c',right> ϵ • A non-deterministic machine M on input I

is said to accept its input • if there exists an accepting computation.

• Note problems computed need yes/no answers.

Poly

NP:

NP

• If I is a “yes” instance:• A fairy god mother could

provide you a small witness/solution/proof S with which you can prove to your non-believing poly time boss that I is a “yes” instance.

• If I is a “no” instance:• There is likely no such

witness/solution/proof .


SAT

Poly

CoNP:

NP

• If I is a “No” instance:• A fairy god mother could

provide you a small witness/solution/proof S with which you can prove to your non-believing poly time boss that I is a “no” instance.

• If I is a “Yes” instance:• There is likely no such

witness/solution/proof .


CoNPnot SATSAT

Poly

NPSAT

CoNP

Network Flows

Network Flows: Is there a flow in G with value k?A witness/solution/proof S of Yes is a flow A witness/solution/proof S of No is a Cut.

not SATNetwork Flows Network Flows


Poly

GCD

NP

NP vs Other ClassesIf P Polynomial Time, You don’t even need the fairy good mother. P NP

If P NP, then |S| n1000

In time you can try all witnesses S. P Exp

Exp

n1000

2SAT

Games

NP

Exp

Poly

Computable

GCD

SAT

Halting

Turing 1936

Games

NP vs Other Classes

NP

Exp

Poly

Computable

Acceptable

GCD

SAT

• If I=<M,I’> is a “yes” instance M halts on I the computation is a witness

arbitrarily long but finite.• If I is a “no” instance

M runs forever there is no witness

HaltingCoAcceptable

Not Halting

CoNPnot SAT

Games

NP vs Other Classes

Computable

Exp

Poly

Known

GCD

NP


Problem Pnew NP Pnew not too hard.

Pnew

Test in poly-time if a given solution

is valid

Computable

Exp

Poly

Known

GCD

NP


Pnew

Sat

Problem Pnew is NP-Hard

If being able to solve this problem fast means that you can solve every problem in the class fast.

• PNP, P ≤poly Pnew

• Easier: Sat ≤poly Pnew

• Cook: P ≤poly Sat

Steve Cook

P

Computable

Exp

Poly

Known

GCD

NP


Problem Pnew is NP-Complete

• Pnew NP

• Pnew is NP-Hard

Pnew

Computable

Exp

Poly

Known

GCD

NP

complete


Problem Pnew is NP-Complete

• Pnew NP

• Pnew is NP-Hard

PnewSatSteve Cook

Computable

Exp

Poly

Known

NP

complete

GCDMatching

SATCoNP

completenot SAT

Halting

Games

Most “natural” problems in NP are either• in PolyTime• or NP-Completeexcept?


Computable

Exp

Poly

Known

NP

complete

GCDMatching

SATCoNP

completenot SAT

FactoringGraph Isomorphism

Halting

Games

Most “natural” problems in NP are either• in PolyTime• or NP-Completeexcept


Reductions

• Reduction: Design a fast algorithm for one computational problem, using a supposedly fast algorithm for another problem as a subroutine.

Palg ≤poly Poracle

Reductions

• Reduction: Design a fast algorithm for one computational problem, using a supposedly fast algorithm for another problem as a subroutine. • Use to create a new algorithm from a known algorithm.• Learn that a new problem is likely hard,

from knowing a known problem is likely hard.• Learn that two problems have a similar “structure.”

Palg ≤poly Poracle

GIVEN:Network

Flow Oracle

BUILD:Matching

Oracle

Matching ≤poly Network Flows

Who loves whoAnnFred

SueJohn

BethBob

MarySam

Max matching

A network

s t

Max Flow

s t

Reductions

Exp

PolySteve Cook 1971

NPSAT

completepoly Prob’

Matching ≤poly Network Flows

• Learn that a new problem is likely hard, from knowing a known problem is likely hard.

Reductions

Is there a fast algorithm for Poracle?Is there a fast algorithm for Palg?

?

We give a fast algorithm for Palg using a supposed fast algorithm for Poracle as a subroutine.

Reductions

If there is a fast algorithm for Palg

then there is a fast algorithm for Palg

then there is not fast

algorithm for Poracle

If there is not a fast algorithm for Palg

If there is a fast algorithm for Poracle

If there is not a fast algorithm for Poracle

?

?


?

???

??

Reductions

Notation: Palg ≤poly Poracle

Palg is “at least as easy as” Poracle

(Modulo polynomial terms.)

Poracle is “at least as hard as” Palg

(Modulo polynomial terms.)

Conclusions:


Reductions

The End(Of definitions for 3101)

reduction s & np-completeness jeff edmonds york university historyhistoryhistory course vs...

Documents

searchpoly time

time slotsto

nonbelieving poly time

courses students

different colours

time colour1pm mon1pm

small witnesssolutionproof

course description