pearls of algorithms - roeglin.orgroeglin.org/teaching/ws2013/pearls/smoothedanalysisslides.pdf ·...
TRANSCRIPT
![Page 1: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/1.jpg)
Pearls of AlgorithmsPart 3: Randomized Algorithms and Probabilistic Analysis
Prof. Dr. Heiko RoglinInstitut fur Informatik
Winter 2013/14
Heiko Roglin Pearls of Algorithms
![Page 2: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/2.jpg)
Efficient Algorithms
When is an algorithm considered efficient?
EngineerThe algorithm must be efficient in practice,i.e., it must solve practical instances in anappropriate amount of time.
TheoreticianThe algorithm must be efficient in the worst case,i.e., it must solve all instances in polynomial time.
Heiko Roglin Pearls of Algorithms
![Page 3: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/3.jpg)
Efficient Algorithms
When is an algorithm considered efficient?
EngineerThe algorithm must be efficient in practice,i.e., it must solve practical instances in anappropriate amount of time.
TheoreticianThe algorithm must be efficient in the worst case,i.e., it must solve all instances in polynomial time.
Heiko Roglin Pearls of Algorithms
![Page 4: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/4.jpg)
Efficient Algorithms
When is an algorithm considered efficient?
EngineerThe algorithm must be efficient in practice,i.e., it must solve practical instances in anappropriate amount of time.
TheoreticianThe algorithm must be efficient in the worst case,i.e., it must solve all instances in polynomial time.
Heiko Roglin Pearls of Algorithms
![Page 5: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/5.jpg)
The Knapsack Problem
Knapsack problem (KP)Input
set of items {1, . . . , n}profits p1, . . . , pn
weights w1, . . . ,wn
capacity b
GoalFind a subset of items that fits into theknapsack and maximizes the profit.
Formal description
max p1x1 + · · ·+ pnxn
subject to w1x1 + · · ·+ wnxn ≤ b
and xi ∈ {0, 1}
Heiko Roglin Pearls of Algorithms
![Page 6: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/6.jpg)
The Knapsack Problem
Knapsack problem (KP)Input
set of items {1, . . . , n}profits p1, . . . , pn
weights w1, . . . ,wn
capacity b
GoalFind a subset of items that fits into theknapsack and maximizes the profit.
Formal description
max p1x1 + · · ·+ pnxn
subject to w1x1 + · · ·+ wnxn ≤ b
and xi ∈ {0, 1}
Heiko Roglin Pearls of Algorithms
![Page 7: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/7.jpg)
The Knapsack Problem
Knapsack problem (KP)Input
set of items {1, . . . , n}profits p1, . . . , pn
weights w1, . . . ,wn
capacity b
GoalFind a subset of items that fits into theknapsack and maximizes the profit.
Formal description
max p1x1 + · · ·+ pnxn
subject to w1x1 + · · ·+ wnxn ≤ b
and xi ∈ {0, 1}
Heiko Roglin Pearls of Algorithms
![Page 8: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/8.jpg)
Different Opinions
Theorists say. . .
KP is NP-hard.
FPTAS exists.
No efficient algorithm for KP,unless P = NP.
Engineers say. . .
KP is easy to solve!
Does not even requirequadratic time.
There are very good heuristicsfor practical instances of KP.
Heiko Roglin Pearls of Algorithms
![Page 9: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/9.jpg)
Different Opinions
Theorists say. . .
KP is NP-hard.
FPTAS exists.
No efficient algorithm for KP,unless P = NP.
Engineers say. . .
KP is easy to solve!
Does not even requirequadratic time.
There are very good heuristicsfor practical instances of KP.
Heiko Roglin Pearls of Algorithms
![Page 10: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/10.jpg)
Reason for discrepancy
Reason for discrepancy
Worst-case complexity is too pessimistic!
There are (artificial) worst-case instances for KP on which theheuristics are not efficient. These instances, however, do notoccur in practice.
This phenomenon occurs not only for KP, but also for many otherproblems.
How to make theory more consistent with practice?Find a more realistic performance measure.
Average-case analysis
Is it realistic to consider the averagecase behavior instead of the worstcase behavior?
Heiko Roglin Pearls of Algorithms
![Page 11: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/11.jpg)
Reason for discrepancy
Reason for discrepancy
Worst-case complexity is too pessimistic!
There are (artificial) worst-case instances for KP on which theheuristics are not efficient. These instances, however, do notoccur in practice.
This phenomenon occurs not only for KP, but also for many otherproblems.
How to make theory more consistent with practice?Find a more realistic performance measure.
Average-case analysis
Is it realistic to consider the averagecase behavior instead of the worstcase behavior?
Heiko Roglin Pearls of Algorithms
![Page 12: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/12.jpg)
Reason for discrepancy
Reason for discrepancy
Worst-case complexity is too pessimistic!
There are (artificial) worst-case instances for KP on which theheuristics are not efficient. These instances, however, do notoccur in practice.
This phenomenon occurs not only for KP, but also for many otherproblems.
How to make theory more consistent with practice?Find a more realistic performance measure.
Average-case analysis
Is it realistic to consider the averagecase behavior instead of the worstcase behavior?
Heiko Roglin Pearls of Algorithms
![Page 13: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/13.jpg)
Random Inputs are not Typical
Random inputs are not typical!If real-world data was random, watching TV would be very boring...
Heiko Roglin Pearls of Algorithms
![Page 14: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/14.jpg)
What is a typical instance?
What is a typical instance?
Weight
Profit
Weight
Profit
Weight
Profit
Weight
Profit
It depends very much on the concrete application. We cannot say ingeneral what a typical instance for KP looks like.
Heiko Roglin Pearls of Algorithms
![Page 15: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/15.jpg)
Smoothed Analysis
Step 1:Adversary
chooses input I.
Weight
Profit
Weight
Profit
Weight
Profit
Weight
Profit
Step 2: Randomperturbation.I → per(I)
Weight
Profit
Weight
Profit
Weight
Profit
Weight
Profit
Heiko Roglin Pearls of Algorithms
![Page 16: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/16.jpg)
Smoothed Analysis
Step 1:Adversary
chooses input I.
Weight
Profit
Weight
Profit
Weight
Profit
Weight
Profit
Step 2: Randomperturbation.I → per(I)
Weight
Profit
Weight
Profit
Weight
Profit
Weight
Profit
Heiko Roglin Pearls of Algorithms
![Page 17: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/17.jpg)
Smoothed Analysis
Step 1:Adversarychooses input I.
Step 2: Randomperturbation.I → per(I)
Smoothed Complexity = worst expected running time the adversarycan achieve
Why do we consider this model?
First step alone: worst case analysis.
Second step models random influences, e.g., measurementerrors, numerical imprecision, rounding, . . .
So we have a combination of both: instances of any structure withsome amount of random noise.
Heiko Roglin Pearls of Algorithms
![Page 18: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/18.jpg)
Perturbation
Example
Step 1: Adversary chooses all pi ,wi ∈ [0, 1] arbitrarily.
Step 2: Add an independent Gaussian random variable to eachprofit and weight.
Weight
Profit
0
0.5
1
-5 -4 -3 -2 -1 0 1 2 3 4 5
sigma=0.5sigma=1sigma=2
Heiko Roglin Pearls of Algorithms
![Page 19: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/19.jpg)
Perturbation
Example
Step 1: Adversary chooses all pi ,wi ∈ [0, 1] arbitrarily.
Step 2: Add an independent Gaussian random variable to eachprofit and weight.
Weight
Profit
0
0.5
1
-5 -4 -3 -2 -1 0 1 2 3 4 5
sigma=0.5sigma=1sigma=2
Heiko Roglin Pearls of Algorithms
![Page 20: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/20.jpg)
Perturbation
Example
Step 1: Adversary chooses all pi ,wi ∈ [0, 1] arbitrarily.
Step 2: Add an independent Gaussian random variable to eachprofit and weight.
Weight
Profit
σ1 0
0.5
1
-5 -4 -3 -2 -1 0 1 2 3 4 5
sigma=0.5sigma=1sigma=2
Heiko Roglin Pearls of Algorithms
![Page 21: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/21.jpg)
Perturbation
Example
Step 1: Adversary chooses all pi ,wi ∈ [0, 1] arbitrarily.
Step 2: Add an independent Gaussian random variable to eachprofit and weight.
Weight
Profit
σ1 > σ2 0
0.5
1
-5 -4 -3 -2 -1 0 1 2 3 4 5
sigma=0.5sigma=1sigma=2
Heiko Roglin Pearls of Algorithms
![Page 22: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/22.jpg)
Complexity Measures
Instances of length n.
CA(I)
CA(I) = running time of algorithm A on input IXn = set of inputs of length n,
µn = prob. distribution on Xn
CworstA (n) = maxI∈Xn (CA(I))
CaveA (n) = EIµn← Xn
[CA(I)]
CsmoothA (n, σ) = maxI∈Xn E [CA (perσ(I))]
smoothed complexity low⇒ bad instances are isolated peaks
Heiko Roglin Pearls of Algorithms
![Page 23: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/23.jpg)
Complexity Measures
Instances of length n.
CA(I)CworstA (n)
CA(I) = running time of algorithm A on input IXn = set of inputs of length n,
µn = prob. distribution on Xn
CworstA (n) = maxI∈Xn (CA(I))
CaveA (n) = EIµn← Xn
[CA(I)]
CsmoothA (n, σ) = maxI∈Xn E [CA (perσ(I))]
smoothed complexity low⇒ bad instances are isolated peaks
Heiko Roglin Pearls of Algorithms
![Page 24: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/24.jpg)
Complexity Measures
Instances of length n.
CA(I)CworstA (n)
CaveA (n)
CA(I) = running time of algorithm A on input IXn = set of inputs of length n, µn = prob. distribution on Xn
CworstA (n) = maxI∈Xn (CA(I))
CaveA (n) = EIµn← Xn
[CA(I)]
CsmoothA (n, σ) = maxI∈Xn E [CA (perσ(I))]
smoothed complexity low⇒ bad instances are isolated peaks
Heiko Roglin Pearls of Algorithms
![Page 25: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/25.jpg)
Complexity Measures
Instances of length n.
CA(I)CworstA (n)
CaveA (n)
I
CsmoothA (n, σ1)
CsmoothA (n, σ2)
σ2 < σ1
CA(I) = running time of algorithm A on input IXn = set of inputs of length n, µn = prob. distribution on Xn
CworstA (n) = maxI∈Xn (CA(I))
CaveA (n) = EIµn← Xn
[CA(I)]
CsmoothA (n, σ) = maxI∈Xn E [CA (perσ(I))]
smoothed complexity low⇒ bad instances are isolated peaks
Heiko Roglin Pearls of Algorithms
![Page 26: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/26.jpg)
Complexity Measures
Instances of length n.
CA(I)CworstA (n)
CaveA (n)
I
CsmoothA (n, σ1)
CsmoothA (n, σ2)
σ2 < σ1
CA(I) = running time of algorithm A on input IXn = set of inputs of length n, µn = prob. distribution on Xn
CworstA (n) = maxI∈Xn (CA(I))
CaveA (n) = EIµn← Xn
[CA(I)]
CsmoothA (n, σ) = maxI∈Xn E [CA (perσ(I))]
smoothed complexity low⇒ bad instances are isolated peaks
Heiko Roglin Pearls of Algorithms
![Page 27: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/27.jpg)
Linear Programming
Linear Programs (LPs)
variables: x1, . . . , xn ∈ Rlinear objective function:max cT x = c1x1 + . . .+ cnxn
m linear constraints:
a1,1x1 + . . .+ a1,nxn ≤ b1
...
am,1x1 + . . .+ am,nxn ≤ bm
c
x∗
Complexity of LPs
LPs can be solved in polynomial time by the ellipsoid method[Khachiyan 1979] and the interior point method [Karmarkar 1984].
Heiko Roglin Pearls of Algorithms
![Page 28: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/28.jpg)
Linear Programming
Linear Programs (LPs)
variables: x1, . . . , xn ∈ Rlinear objective function:max cT x = c1x1 + . . .+ cnxn
m linear constraints:
a1,1x1 + . . .+ a1,nxn ≤ b1
...
am,1x1 + . . .+ am,nxn ≤ bm
c
x∗
Complexity of LPs
LPs can be solved in polynomial time by the ellipsoid method[Khachiyan 1979] and the interior point method [Karmarkar 1984].
Heiko Roglin Pearls of Algorithms
![Page 29: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/29.jpg)
Simplex Algorithm
c
Simplex Algorithm
Start at some vertex of thepolytope.
Walk along the edges of thepolytope in the direction of theobjective function cT x .
local optimum = global optimum
c Pivot RulesWhich vertex is chosen if there aremultiple options?
Different pivot rules suggested:random, steepest descent, shadowvertex pivot rule, . . .
Heiko Roglin Pearls of Algorithms
![Page 30: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/30.jpg)
Simplex Algorithm
c
Simplex Algorithm
Start at some vertex of thepolytope.
Walk along the edges of thepolytope in the direction of theobjective function cT x .
local optimum = global optimum
c Pivot RulesWhich vertex is chosen if there aremultiple options?
Different pivot rules suggested:random, steepest descent, shadowvertex pivot rule, . . .
Heiko Roglin Pearls of Algorithms
![Page 31: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/31.jpg)
Simplex Algorithm
c
Simplex Algorithm
Start at some vertex of thepolytope.
Walk along the edges of thepolytope in the direction of theobjective function cT x .
local optimum = global optimum
c Pivot RulesWhich vertex is chosen if there aremultiple options?
Different pivot rules suggested:random, steepest descent, shadowvertex pivot rule, . . .
Heiko Roglin Pearls of Algorithms
![Page 32: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/32.jpg)
Simplex Algorithm
c
Simplex Algorithm
Start at some vertex of thepolytope.
Walk along the edges of thepolytope in the direction of theobjective function cT x .
local optimum = global optimum
c Pivot RulesWhich vertex is chosen if there aremultiple options?
Different pivot rules suggested:random, steepest descent, shadowvertex pivot rule, . . .
Heiko Roglin Pearls of Algorithms
![Page 33: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/33.jpg)
Simplex Algorithm
c
Simplex Algorithm
Start at some vertex of thepolytope.
Walk along the edges of thepolytope in the direction of theobjective function cT x .
local optimum = global optimum
c Pivot RulesWhich vertex is chosen if there aremultiple options?
Different pivot rules suggested:random, steepest descent, shadowvertex pivot rule, . . .
Heiko Roglin Pearls of Algorithms
![Page 34: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/34.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow.
cu
x0
x∗
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Pearls of Algorithms
![Page 35: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/35.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow.
cu
x0
x∗
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Pearls of Algorithms
![Page 36: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/36.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow.
cu
x0
x∗
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Pearls of Algorithms
![Page 37: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/37.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow.
cu
x0
x∗
x∗x0
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Pearls of Algorithms
![Page 38: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/38.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow.
cu
x0
x∗
x∗x0
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Pearls of Algorithms
![Page 39: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/39.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow. cu
x0
x∗
x∗x0
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Pearls of Algorithms
![Page 40: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/40.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow. cu
x0
x∗
x∗x0
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Pearls of Algorithms
![Page 41: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/41.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow. cu
x0
x∗
x∗x0
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Pearls of Algorithms
![Page 42: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/42.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow. cu
x0
x∗
x∗x0
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Pearls of Algorithms
![Page 43: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/43.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow. cu
x0
x∗
x∗x0
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Pearls of Algorithms
![Page 44: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/44.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow. cu
x0
x∗
x∗x0
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Pearls of Algorithms
![Page 45: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/45.jpg)
Simplex Algorithm – Running Time
Theoreticians say. . .
shadow vertex pivot rule requiresexponential number of steps
no pivot rule with sub-exponentialnumber of steps known
ellipsoid and interior point methods areefficient
Engineers say. . .
simplex method usually fastest algorithmin practice
requires usually only Θ(m) steps
clearly outperforms ellipsoid method
Heiko Roglin Pearls of Algorithms
![Page 46: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/46.jpg)
Simplex Algorithm – Running Time
Theoreticians say. . .
shadow vertex pivot rule requiresexponential number of steps
no pivot rule with sub-exponentialnumber of steps known
ellipsoid and interior point methods areefficient
Engineers say. . .
simplex method usually fastest algorithmin practice
requires usually only Θ(m) steps
clearly outperforms ellipsoid method
Heiko Roglin Pearls of Algorithms
![Page 47: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/47.jpg)
Smoothed Analysis
Observation: In worst-case analysis, the adversary is too powerful.Idea: Let’s weaken him!
Perturbed LPs
Step 1: Adversary specifies arbitrary LP:max cT x subject to aT
1 x ≤ b1 . . . aTn x ≤ bn.
W. l. o. g. ‖(ai , bi)‖ = 1.
Step 2: Add Gaussian random variable with standard deviation σto each coefficient in the constraints.
Smoothed Running Time= worst expected running time the adversary can achieve
Heiko Roglin Pearls of Algorithms
![Page 48: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/48.jpg)
Smoothed Analysis
Observation: In worst-case analysis, the adversary is too powerful.Idea: Let’s weaken him!
Perturbed LPs
Step 1: Adversary specifies arbitrary LP:max cT x subject to aT
1 x ≤ b1 . . . aTn x ≤ bn.
W. l. o. g. ‖(ai , bi)‖ = 1.
Step 2: Add Gaussian random variable with standard deviation σto each coefficient in the constraints.
Smoothed Running Time= worst expected running time the adversary can achieve
Heiko Roglin Pearls of Algorithms
![Page 49: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/49.jpg)
Smoothed Analysis
Observation: In worst-case analysis, the adversary is too powerful.Idea: Let’s weaken him!
Perturbed LPs
Step 1: Adversary specifies arbitrary LP:max cT x subject to aT
1 x ≤ b1 . . . aTn x ≤ bn.
W. l. o. g. ‖(ai , bi)‖ = 1.
Step 2: Add Gaussian random variable with standard deviation σto each coefficient in the constraints.
Smoothed Running Time= worst expected running time the adversary can achieve
Heiko Roglin Pearls of Algorithms
![Page 50: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/50.jpg)
Smoothed Analysis
Observation: In worst-case analysis, the adversary is too powerful.Idea: Let’s weaken him!
Perturbed LPs
Step 1: Adversary specifies arbitrary LP:max cT x subject to aT
1 x ≤ b1 . . . aTn x ≤ bn.
W. l. o. g. ‖(ai , bi)‖ = 1.
Step 2: Add Gaussian random variable with standard deviation σto each coefficient in the constraints.
Smoothed Running Time= worst expected running time the adversary can achieve
Heiko Roglin Pearls of Algorithms
![Page 51: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/51.jpg)
Smoothed Analysis
Step 1:Adversarychooses input I
Step 2: RandomperturbationI → perσ(I)
Formal Definition:LP(n,m) = set of LPs with n variables and m constraintsT (I) = number of steps of simplex method on LP I
smoothed run time T smooth(n,m, σ) = maxI∈LP(n,m)E [T (perσ(I))]
Why do we consider this model?
First step models unknown structure of the input.
Second step models random influences, e.g., measurementerrors, numerical imprecision, rounding, . . .
smoothed running time low⇒ bad instances are unlikely to occur
σ determines the amount of randomness
Heiko Roglin Pearls of Algorithms
![Page 52: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/52.jpg)
Smoothed Analysis
Step 1:Adversarychooses input I
Step 2: RandomperturbationI → perσ(I)
Formal Definition:LP(n,m) = set of LPs with n variables and m constraintsT (I) = number of steps of simplex method on LP Ismoothed run time T smooth(n,m, σ) = maxI∈LP(n,m)E [T (perσ(I))]
Why do we consider this model?
First step models unknown structure of the input.
Second step models random influences, e.g., measurementerrors, numerical imprecision, rounding, . . .
smoothed running time low⇒ bad instances are unlikely to occur
σ determines the amount of randomness
Heiko Roglin Pearls of Algorithms
![Page 53: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/53.jpg)
Smoothed Analysis
Step 1:Adversarychooses input I
Step 2: RandomperturbationI → perσ(I)
Formal Definition:LP(n,m) = set of LPs with n variables and m constraintsT (I) = number of steps of simplex method on LP Ismoothed run time T smooth(n,m, σ) = maxI∈LP(n,m)E [T (perσ(I))]
Why do we consider this model?
First step models unknown structure of the input.
Second step models random influences, e.g., measurementerrors, numerical imprecision, rounding, . . .
smoothed running time low⇒ bad instances are unlikely to occur
σ determines the amount of randomness
Heiko Roglin Pearls of Algorithms
![Page 54: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/54.jpg)
Smoothed Analysis of the Simplex Algorithm
Lemma [Spielman and Teng (STOC 2001)]
For every fixed plane and every LP the adversarycan choose, after the perturbation, the expectednumber of edges on the shadow is
O(poly
(n,m, σ−1)) . cu
x0
x∗
Theorem [Spielman and Teng (STOC 2001)]
The smoothed running time of the simplex algorithm with shadowvertex pivot rule is O
(poly
(n,m, σ−1)) .
Already for small perturbations exponential running time is unlikely.
Main Difficulties in Proof of Theorem:
x0 is found in phase I→ no Gaussian distribution of coefficients
In phase II, the plane onto which the polytope is projected is notindependent of the perturbations.
Heiko Roglin Pearls of Algorithms
![Page 55: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/55.jpg)
Smoothed Analysis of the Simplex Algorithm
Lemma [Spielman and Teng (STOC 2001)]
For every fixed plane and every LP the adversarycan choose, after the perturbation, the expectednumber of edges on the shadow is
O(poly
(n,m, σ−1)) . cu
x0
x∗
Theorem [Spielman and Teng (STOC 2001)]
The smoothed running time of the simplex algorithm with shadowvertex pivot rule is O
(poly
(n,m, σ−1)) .
Already for small perturbations exponential running time is unlikely.
Main Difficulties in Proof of Theorem:
x0 is found in phase I→ no Gaussian distribution of coefficients
In phase II, the plane onto which the polytope is projected is notindependent of the perturbations.
Heiko Roglin Pearls of Algorithms
![Page 56: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/56.jpg)
Smoothed Analysis of the Simplex Algorithm
Lemma [Spielman and Teng (STOC 2001)]
For every fixed plane and every LP the adversarycan choose, after the perturbation, the expectednumber of edges on the shadow is
O(poly
(n,m, σ−1)) . cu
x0
x∗
Theorem [Spielman and Teng (STOC 2001)]
The smoothed running time of the simplex algorithm with shadowvertex pivot rule is O
(poly
(n,m, σ−1)) .
Already for small perturbations exponential running time is unlikely.
Main Difficulties in Proof of Theorem:
x0 is found in phase I→ no Gaussian distribution of coefficients
In phase II, the plane onto which the polytope is projected is notindependent of the perturbations.
Heiko Roglin Pearls of Algorithms
![Page 57: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/57.jpg)
Improved Analysis
Theorem [Vershynin (FOCS 2006)]
The smoothed number of steps of the simplex algorithm with shadowvertex pivot rule is
O(poly
(n, log m, σ−1)) .
only polylogarithmic in the number of constraints m
Phase I: add vertex x0 in randomdirection. With constant prob. this doesnot change optimal solution.⇒ The plane is not correlated with theperturbed polytope.
With high prob. no angle betweenconsecutive vertices is too small.
Heiko Roglin Pearls of Algorithms
![Page 58: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/58.jpg)
Improved Analysis
Theorem [Vershynin (FOCS 2006)]
The smoothed number of steps of the simplex algorithm with shadowvertex pivot rule is
O(poly
(n, log m, σ−1)) .
only polylogarithmic in the number of constraints m
cu
x0
x∗Phase I: add vertex x0 in randomdirection. With constant prob. this doesnot change optimal solution.⇒ The plane is not correlated with theperturbed polytope.
With high prob. no angle betweenconsecutive vertices is too small.
Heiko Roglin Pearls of Algorithms
![Page 59: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/59.jpg)
Improved Analysis
Theorem [Vershynin (FOCS 2006)]
The smoothed number of steps of the simplex algorithm with shadowvertex pivot rule is
O(poly
(n, log m, σ−1)) .
only polylogarithmic in the number of constraints m
cu
x0
x∗
x∗x0
Phase I: add vertex x0 in randomdirection. With constant prob. this doesnot change optimal solution.⇒ The plane is not correlated with theperturbed polytope.
With high prob. no angle betweenconsecutive vertices is too small.
Heiko Roglin Pearls of Algorithms
![Page 60: Pearls of Algorithms - roeglin.orgroeglin.org/teaching/WS2013/Pearls/SmoothedAnalysisSlides.pdf · Pearls of Algorithms Part 3: Randomized Algorithms and Probabilistic Analysis Prof](https://reader034.vdocuments.net/reader034/viewer/2022052303/5f0e80227e708231d43f8c4e/html5/thumbnails/60.jpg)
Overview of the coming Lectures
Smoothed Analysis
2-Opt heuristic for the traveling salesperson problem
Nemhauser/Ullmann algorithm for the knapsack problem
Heiko Roglin Pearls of Algorithms