Randomized Algorithms
Prabhakar RaghavanIBM Almaden Research Center
San Jose� CA�
� Typeset by FoilTEX �
Deterministic Algorithms
ALGORITHMINPUT
OUTPUT
Goal� To prove that thealgorithm solves the problemcorrectly �always� and quickly
�typically� the number of stepsshould be polynomial in thesize of the input��
� Typeset by FoilTEX � �
Randomized Algorithms
ALGORITHMINPUT
OUTPUT
RANDOM NUMBERS
� In addition to input� algorithmtakes a source of randomnumbers and makes randomchoices during execution�
� Behavior can vary even on a�xed input�
� Typeset by FoilTEX � �
Randomized Algorithms
ALGORITHMINPUT
OUTPUT
RANDOM NUMBERS
� Design algorithm � analysisto show that this behavior islikely to be good� on every
input�
�The likelihood is over therandom numbers only��
� Typeset by FoilTEX � �
Not to be confused with the
Probabilistic Analysis of
Algorithms
ALGORITHMINPUT
OUTPUT
RANDOM
DISTRIBUTION
� Here the input is assumedto be from a probabilitydistribution�
� Show that the algorithmworksfor most inputs�
� Typeset by FoilTEX � �
Monte Carlo and Las Vegas
A Monte Carlo algorithm runsfor a �xed number of steps�and produces an answer that iscorrect with probability � ����
A Las Vegas algorithm alwaysproduces the correct answer�its running time is a randomvariable whose expectation isbounded �say by a polynomial��
� Typeset by FoilTEX � �
Monte Carlo and Las Vegas
These probabilitiesexpectationsare only over the randomchoices made by the algorithm independent of the input�
Thus independent repetitionsof Monte Carlo algorithms drivedown the failure probabilityexponentially�
� Typeset by FoilTEX � �
Advantages of randomized
algorithms
� Simplicity
� Performance
For many problems� a randomizedalgorithm is the simplest� thefastest� or both�
� Typeset by FoilTEX � �
Scope
� Number�theoretic algorithms�Primality testing �Monte Carlo��
� Data structures� Sorting�order statistics� searching�computational geometry�
� Algebraic identities� Polynomialand matrix identity veri�cation�Interactive proof systems�
� Typeset by FoilTEX �
� Mathematical programming�Faster algorithms for linearprogramming� Rounding linearprogram solutions to integerprogram solutions�
� Graph algorithms� Minimumspanning trees� shortest paths�minimum cuts�
� Counting and enumeration�Matrix permanent� Countingcombinatorial structures�
� Typeset by FoilTEX �
� Parallel and distributed computing�Deadlock avoidance� distributedconsensus�
� Probabilistic existence proofs�Show that a combinatorialobject arises with non�zeroprobability among objects drawnfrom a suitable probabilityspace�
� Typeset by FoilTEX � ��
� Derandomization� First devisea randomized algorithm� thenargue that it can be �derandomized to yield a deterministic algorithm�
� Typeset by FoilTEX � ��
Game�tree evaluation
MAX
MIN
MAX MAX
MIN
MAX MAX
1 7 9 5 3 2 17 12
7 9 3 17
9 3
9
� Tree with alternating MAXMINnodes�Each leaf has a real value�
� Goal� To evaluate the root�
� Typeset by FoilTEX � ��
Game�tree evaluation
MAX
MIN
MAX MAX
MIN
MAX MAX
1 7 9 5 3 2 17 12
7 9 3 17
9 3
9
� Cost� The number of leavesinspected�
� Algorithm� Computes leaf�inspection sequence�
� Typeset by FoilTEX � ��
Simple special case
� Uniform binary tree� height h�with n � �h leaves�
� All values are boolean� soMAX�OR and MIN�AND�
Exercise� Every deterministicalgorithm can be forced to readn leaves�
� Typeset by FoilTEX � ��
Randomized algorithm
AND
OR OR
AND AND AND AND
1 0
1 0 1 1 1 0 1 0
0 1 0 0
Evaluate �recursively� a randomchild of the current node�
If this does not determinethe value of the current node�evaluate the other child�
� Typeset by FoilTEX � ��
Analysis of tree evaluation
AND�OR trees of depth �k �klevels of AND and k levels ofOR on any path�� will prove byinduction on k�
Expected cost for an ORsubtree evaluating to ��
�
�� �k�� � �
�� � � �k�� � �
�� �k���
� Typeset by FoilTEX � ��
Analysis of tree evaluation
For an OR subtree evaluatingto �� we must evaluate bothof its subtrees� incurring cost���k����
Now consider the root AND��a� If it is �� both OR subtreesmust evaluate to �� expectedcost paid is twice above� � �k��b� If it is �� at least one ORsubtree evaluates to ��
� Typeset by FoilTEX � ��
Game tree analysis
Expected cost for root ANDevaluating to ��
� � �k�� � �
�
�
��k�� � �k�
Number of leaves in tree � k�
Thus expected cost of randomizedalgorithm � n�����
� Typeset by FoilTEX � �
Lower bounds and the
minimax principle
Basic Idea� For a lowerbound on the performance ofall randomized algorithms for aproblem� derive instead a lowerbound for any deterministicalgorithm for the problem when
the inputs are drawn from a
probability distribution �of yourchoice��
� Typeset by FoilTEX � �
Let I denote the set ofinstances of the problem� andA the set of deterministicalgorithms for the problem�Any randomized algorithm canbe viewed as a probabilitydistribution on the algorithms inA�
� Typeset by FoilTEX � ��
Minimax Principle
For all distributions p over Iand q over A�minA�A
EC�Ip� A�� � maxI�I
EC�I� Aq���
� Typeset by FoilTEX � ��
Lower bound for game tree
evaluation
We will specify a probabilitydistribution on instances ����values for the leaves� and lowerbound the expected runningtime of any deterministic algorithm�
� Typeset by FoilTEX � ��
NOR trees instead
Exercise� Show that a balancedAND�OR tree of even depth isequivalent to a tree of the samedepth� all of whose nodes areNOR nodes�
We will show the lower boundfor a NOR tree with n leaves�
� Typeset by FoilTEX � ��
The input distribution
Let p � ���p�����
Claim� The value of anynode is � with probability p�independent of all other nodesat the same level�
Claim� Any deterministicalgorithmmay as well determinethe value of one sub�tree of anode� before inspecting any leafof its sibling sub�tree�
� Typeset by FoilTEX � ��
The Analysis
Let W �h� be the expectednumber of leaves it inspects indetermining the value of a nodeat distance h from the leaves�
Clearly
W �h� � W �h�������p�W �h����
Letting h � log� n� this gives alower bound of n������� Typeset by FoilTEX � ��
The ��SAT Problem
Given a set of boolean clausesin CNF each containing twoliterals� �nd a satisfying assignmentif one exists�
�x� � x��� x� � x���x� � x�� � � �
� Start with any tentative assignment
� If there is an unsatis�edclause� pick one of its twoliterals at random� and �ip it�
� Typeset by FoilTEX � ��
� If no solution found in �n�
steps� declare �none exists �
Monte Carlo� If a solutionexists� will �nd it with probability� ����If not� will always declare �noneexists �
� Typeset by FoilTEX � �
Random Walk Analysis
What is the probability ofmissing a satisfying assignmentin �n� steps�
� Fix any particular satisfyingassignment A�
� Consider the number of literalson which the algorithm�stentative assignment agreeswith A�
� Typeset by FoilTEX � �
� This is a random walk on theintegers that increases withprobability at least ��� ateach step�
Expected time to �nd anassignment � n��Markov�s inequality� probabilityof missing an assignment in �n�
steps is � ����
� Typeset by FoilTEX � ��
Binary planar partitions
L1
L2 L3s3
1
s2
L1L2
L3
s
1s s3 s3 s2
Given n non�intersecting line�segments in the plane� build asmall linear decision tree thathas �pieces of� at most onesegment in each cell�
� Typeset by FoilTEX � ��
Autopartitions
Choose the input segments inrandom order�Extend the chosen segment �ifnecessary� to cut any cellswhere it is not �alone �
The expected size of theresulting tree is � n� �nHn�
� Typeset by FoilTEX � ��
Analysis of autopartition size
u
v1
v2
De�ne l�u� to be the linecontaining segment u�
u a v denotes l�u� cuts v inthe constructed tree�
index�u� v� is i if l�u� intersectsi � � other segments beforehitting v�
� Typeset by FoilTEX � ��
Autopartitions
Pru a v� � ���index�u� v�����
Thus the expected size of thetree constructed is
n�X
u
X
v �u
Pru a v� �
n�X
u
X
v �u
�
index�u� v� � �
� Typeset by FoilTEX � ��
For any u� i� index�u� v� � i forat most two v� thus the abovesum is � n� �nHn�
It follows that for any n
segments in the plane� there
always exists an autopartition ofsize O�n log n��
� Typeset by FoilTEX � ��
Matrix product veri�cation
Suppose A�B and C are n �n matrices with entries from a�nite �eld F � We wish to verifywhether AB � C�
Multiplying out takes aboutn� steps this can be cut toO�n���� steps using sophisticatedalgorithms�
� Typeset by FoilTEX � ��
Simple randomized
algorithm
� Pick an n�vector x withentries randomly drawn fromF �
� Compute z � A�Bx��If z � Cx output AB � C�else output AB � C�
Takes O�n�� steps�
� Typeset by FoilTEX � ��
Simple randomized
algorithm
If AB � C� will always outputAB � C�
If AB � C� will outputAB � C with probability atmost ��jFj�
� Typeset by FoilTEX � �