BBM 205 Discrete MathematicsHacettepe University
Lecture 13a: Probabilistic Method
Lale Γzkahya
Resources:Kenneth Rosen, οΏ½Discrete Mathematics and App.οΏ½
http://www.math.caltech.edu/ 2015-16/2term/ma006b
2/4/2016
2
Independent Events
Two events π΄, π΅ β Ξ© are independent ifPr π΄ β© π΅ = Pr π β Pr π .
Example. We flip two fair coins.
β¦ Let ππ,π be the elementary event that coin A
landed on π and coin π΅ on π, where π, π β {β, π‘}. Each of the four events has a probability of 0.25.
β¦ The event where coin π΄ lands on heads is
π = πβ,π‘, πβ,β . For π΅ it is π = ππ‘,β, πβ,β .
β¦ The events are independent since
Pr π and π = Pr πβ,β = 0.25 = Pr π β Pr π .
(Discrete) Uniform Distribution
In a uniform distribution we have a set Ξ©of elementary events, each occurring with
probability 1
Ξ©.
β¦ For example, when flipping a fair die, we have a uniform distribution over the six possible results.
2/4/2016
3
Union Bound
For any two events π΄, π΅, we havePr π΄ βͺ π΅ = Pr π΄ + Pr π΅ β Pr π΄ β© π΅ .
This immediately implies that Pr π΄ βͺ π΅ β€ Pr π΄ + Pr π΅ ,
where equality holds iff π΄, π΅ are disjoint.
Union bound. For any finite set of events π΄1, β¦ , π΄π, we have
Pr βͺπ π΄π β€
π
Pr π΄π .
Recall: Ramsey Numbers
π π, π is the smallest number π such that each blue-red edge coloring of πΎπcontains a monochromatic πΎπ.
Theorem. π π, π > 2π/2.
β¦ We proved this in a previous class.
β¦ Now we provide another proof, using probability.
2/4/2016
4
Probabilistic Proof
For some π, we color the edges of πΎπ.
β¦ Each edge is independently and uniformlycolored either red or blue.
β¦ For any fixed set π of π vertices, the probability that it forms a monochromatic πΎπ
is 21βπ2 .
β¦ There are ππ possible sets of π vertices. By
the union bound, the probability that there is a monochromatic πΎπ is at most
π
21βπ2 =ππ 21βπ2 .
Proof (cont.)
For some π, we color the edges of πΎπ.
β¦ Each edge is colored blue with probability of 0.5, and otherwise red.
β¦ The probability for a monochromatic πΎπ is
β€ππ 21βπ2 .
β¦ If π β€ 2π/2, this probability is smaller than 1.
β¦ In this case, the probability that we do not have any monochromatic πΎπ is positive, so
there exists a coloring of πΎπ with no such πΎπ.
2/4/2016
5
Non-Constructive Proofs
We proved that there exists a coloring of πΎπ with no monochromatic πΎπ, but we
have no idea how to find this coloring.
Such a proof is called non-constructive.
The probabilistic method often proves the existence of objects with surprising properties, but we still have no idea how they look like.
A Tournament
We have π people competing in thumb wrestling.
β¦ Every pair of contestants compete once.
β¦ How can we decide who the overall winner is?
We build a directed graph:
β¦ A vertex for every participant.
β¦ An edge between every two vertices, directed from the winner to the loser.
β¦ An orientation of πΎπ is called a tournament.
2/4/2016
6
The King of the Tournament
The winner can be the vertex with the maximum outdegree (the contestant winning the largest number of matches), but it might not be unique.
A king is a contestant π₯ such that for every other contestant π¦ either π₯ β π¦ or there exists π§ such that π₯ β π§ β π¦.
Theorem. Every tournament has a king.
Proof
π·+(π£) β the number of vertices reachable from π£ by a path of length β€ 2.
Let π£ be a vertex that maximizes π·+ π£ .
β¦ Assume for contradiction that π£ is not a king.
β¦ Then there exists π’ such that π’ β π£ and there is no path of length two from π£ to π’.
β¦ That is, for every π€ such that π£ β π€, we also have π’ β π€.
β¦ But this implies that π·+ π’ β₯ π·+ π£ + 1, contradicting the maximality of π£!
2/4/2016
7
The ππ Property
We say that a tournament π has the ππproperty if for every subset π of πparticipants, there exists a participant that won against everyone in π.
β¦ Formally, this is an orientation of πΎπ, such that for every subset π of π vertices there exists a vertex π£ β π β π with an edge from π£to every vertex of π.
Example. A tournament with the π1 property.
Tournaments with the ππ Property
Theorem. If ππ1 β 2βπ
πβπ< 1 then
there is a tournament on π vertices with the ππ property.
Proof.
β¦ For some π satisfying the above, we randomly orient πΎπ = π, πΈ , such that the orientation of every π β πΈ is chosen uniformly.
β¦ Consider a subset π β π of π vertices. The probability that a given vertex π£ β π β π does not beat all of π is 1 β 2βπ.
2/4/2016
8
Proof (cont.)
Consider a subset π β π of π vertices. The probability that a specific vertex π£ β π β π does not beat all of π is 1 β 2βπ.
π΄π β the event of π not being beat by any vertex of π β π.
We have Pr π΄π = 1 β 2βπ πβπ, since we ask
for π β π independent events to hold.
By the union bound, we have
Pr πβππ =π
π΄π β€ πβππ =π
Pr π΄π =ππ 1 β 2
βπ πβπ < 1.
Completing the Proof
π΄π β the event of π not being beat by any vertex of π β π.
we have
Pr πβππ =π
π΄π < 1.
That is, there is a positive probability that every subset π β π of size π is beat by some vertex of π β π. So such a tournament exists.
2/4/2016
9
Which NBA Player is Related to Mathematics?
Michael Jordan
LeBron James
Shaquille O'Neal
Random Variables
A random variable is a function from the set of possible events to β.
Example. Say that we flip five coins.
β¦ We can define the random variable π to be the number of coins that landed on heads.
β¦ We can define the random variable π to be the percentage of heads in the tosses.
β¦ Notice that π = 20π.
2/4/2016
10
Indicator Random Variables
An indicator random variable is a random variable π that is either 0 or 1, according to whether some event happens or not.
Example. We toss a fair die.
β¦ We can define the six indicator variable π1, β¦ , π6 such that ππ = 1 iff the result of the roll is π.
Expectation
The expectation of a random variable π is
πΈ π =
πβΞ©
π π Pr π .
β¦ Intuitively, πΈ π is the expected value of π in the long-run average value when repeating the experiment π represents.
2/4/2016
11
Expectation Example
We roll a fair six-sided die.
β¦ Let π be a random variable that represents the outcome of the roll.
πΈ π =
πβΞ©
π π Pr π =
πβ{1,β¦,6}
π β 1
6= 3.5
2/4/2016
12
Linearity of Expectation
If π is a random variable, then 5π is a random variable with a value five times that of π
Lemma. Let π1, π2, β¦ , ππ be a collection set of random variables over the same discrete probability. Let π1, β¦ , ππ be constants. Then
πΈ π1π1 + π2π2 +β―+ ππππ =
π=1
π
πππΈ ππ .
Fixed Elements in Permutations
Let π be a uniformly chosen permutation of 1,2, β¦ , π .
β¦ For 1 β€ π β€ π, let ππ be an indicator variable that is 1 if π is fixed by π.
β¦ πΈ ππ = Pr π π = π =πβ1 !
π!=1
π.
β¦ Let π be the number of fixed elements in π.
β¦ We have π = π1 +β―+ ππ.
β¦ By linearity of expectation
πΈ π =
π
πΈ ππ = π β 1
π= 1.
2/4/2016
13
Hamiltonian Paths
Given a directed graph πΊ = π, πΈ , a Hamiltonian path is a path that visits every vertex of π exactly once.
β¦ Major problem in theoretical computer science: Does there exist a polynomial-time algorithm for finding whether a Hamiltonian path exists in a given graph.
Undirected Hamiltonian paths
Hamiltonian Paths in Tournaments
Theorem. There exists a tournament πwith π players that contains at least
π! 2β(πβ1) Hamiltonian paths.
2/4/2016
14
Proof
We uniformly choose an orientation of the edges of πΎπ to obtain a tournament π.
β¦ There is a bijection between the possible Hamiltonian paths and the permutations of 1,2,β¦ , π . Every possible path defines a
unique permutation, according to the order in which it visits the vertices.
β¦ For a permutation π, let ππ be an indicator variable that is 1 if the path corresponding to π exists in π.
β¦ We have πΈ ππ = Pr ππ = 1 = 2β πβ1 .
We uniformly choose an orientation of the edges of πΎπ to obtain a tournament π.
β¦ For a permutation π, let ππ be an indicator variable that is 1 if the path corresponding to π exists π.
β¦ We have πΈ ππ = Pr ππ = 1 = 2βπ+1 .
β¦ Let π be a random variable of the number of Hamiltonian paths in π. Then π = πππ.
πΈ π = πΈ
π
ππ =
π
πΈ ππ = π! 2βπ+1.
β¦ Since this is the expected number of paths in a uniformly chosen tournament, there must is a orientation with at least as many paths.
2/9/2016
2
Reminder: Indicator Random Variables An indicator random variable is a random
variable π that is either 0 or 1, according to whether some event happens or not.
Example. We toss a die.
β¦ We can define the six indicator variable π1, β¦ , π6 such that ππ = 1 iff the result of the roll is π.
Reminder: Expectation
The expectation of a random variable π is
πΈ π =
πβΞ©
π π Pr π .
β¦ Intuitively, πΈ π is the expected value of π in the long-run average value of repetitions of the experiment it represents.
2/9/2016
3
Reminder: Expectation Example
We roll a fair six-sided die.
β¦ Let π be a random variable that represents the outcome of the roll.
πΈ π =
πβΞ©
π π Pr π =
πβ{1,β¦,6}
π β 1
6= 3.5
Reminder: Linearity of Expectation
If π is a random variable, then 5π is a random variable with a value five times that of π
Lemma. Let π1, π2, β¦ , ππ be a collection set of random variables over the same discrete probability. Let π1, β¦ , ππ be constants. Then
πΈ π1π1 + π2π2 +β―+ ππππ =
π=1
π
πππΈ ππ .
2/9/2016
4
Independent Sets
Consider a graph πΊ = π, πΈ . An independent set in πΊ is a subset πβ² β πsuch that there is no edge between any two vertices of πβ².
Finding a maximum independent set in a graph is a major problem in theoretical computer science.
β¦ No polynomial-time algorithm is known.
Warm Up
What are the sizes of the maximum independent sets in:
2 4
2/9/2016
5
Large Independent Sets
Theorem. A graph πΊ = (π, πΈ) has an independent set of size at least
π£βπ
1
1 + deg π£.
Proof. We uniformly choose an ordering for the vertices of π = π£1, β¦ , π£π .
β¦ The set of vertices that appear before all of their neighbors is an independent set.
We uniformly choose an ordering for the vertices of π = π£1, β¦ , π£π .
β¦ The set π of vertices that appear before all of their neighbors is an independent set.
β¦ ππ - indicator that is 1 if π£π β π.
πΈ ππ = Pr ππ = 1 =1
1 + degπ£π.
β¦ π β the random variable of the size of π.
πΈ π = πΈ
π=1
π
ππ =
π=1
π
πΈ ππ =
π=1
π1
1 + deg π£π.
β¦ There must exist an ordering for which |π| has at least this value.
2/9/2016
6
Sum-free Sets
Consider a set π΄ of positive integers. We say that π΄ is sum-free if for every π₯, π¦ β π΄, we have that π₯ + π¦ β π΄(including the case where π₯ = π¦).
What are large sum-free subsets of π = 1,2, β¦ ,π ?
β¦ We can take all of the odd numbers in π.
β¦ We can take all of the numbers in π of size larger than π/2.
Large Sum-Free Sets Always Exist
Theorem. For any set of positive integers π΄, there is a sum-free subset π΅ β π΄ of
size |π΅| β₯1
3|π΄|.
Proof. Consider a prime π such that π > πfor every π β π΄.
β¦ From now on, calculations are πππ π.
β¦ Notice that if π΅ is sum-free πππ π, it is also sum-free under standard addition.
β¦ Thus, it suffices to find a large set that is sum-free πππ π.
2/9/2016
7
Proof (cont.)
The calculations are πππ π.
The set π = π/3 ,β¦ , 2π/3 is sum-free and π β₯ π β 1 /3.
We uniformly choose π₯ β 1,2,β¦ , π β 1 and set π΄π₯ = π β π΄ | ππ₯ β π .
Consider π, π β π΄π₯. Since ππ₯, ππ₯ β π, we have that π + π π₯ = ππ₯ + ππ₯ β π. Thus, π + π β π΄π₯, and π΄π₯ is sum-free.
ππ β indicator that is 1 if π β π΄π₯.
πΈ π΄π₯ = πΈ
πβπ΄
ππ =
πβπ΄
πΈ ππ =
πβπ΄
Pr ππ = 1 .
The set π = π/3 ,β¦ , 2π/3 is sum-free and π β₯ π β 1 /3.
We uniformly choose π₯ β 1,2,β¦ , π β 1 and set π΄π₯ = π β π΄ | ππ₯ β π . π΄π₯ is sum-free.
ππ β indicator that is 1 if π β π΄π₯.
πΈ π΄π₯ = πβπ΄ Pr ππ = 1 .
Recall: If π₯β’π₯β² πππ π then ππ₯β’ππ₯β² πππ π.
We thus have Pr ππ = 1 = |π|/(π β 1).
πΈ π΄π₯ =
πβπ΄
Pr ππ = 1 =π΄ π
π β 1β₯ π΄1
3.
Thus, there exists an π₯ for which π΄π₯ β₯ π΄1
3.
2/9/2016
8
Which Super Villain is a Mathematician?
Austin PowersβDr. Evil
Sherlock HolmesβProfessor Moriarty
SpidermanβsDr. Octopus
More Ramsey Numbers
In the previous class, we used a basic probabilistic argument to prove
π π, π > 2π/2.
Theorem. For any integer π > 0, we have
π π, π > π βππ 21βπ2 .
2/9/2016
9
Proof
Consider a random red-blue coloring of πΎπ = (π, πΈ). The color of each edge is chosen uniformly and independently.
For every subset π β π of size π, we denote by ππ the indicator that π induces a monochromatic πΎπ. We set π = π =πππ .
πΈ ππ = Pr ππ = 1 = 21βπ2
By linearity of expectation, we have
πΈ π =
π =π
πΈ ππ =ππ 21βπ2 .
Completing the Proof
We proved that the in a random red-bluecoloring of πΎπ the expected number of monochromatic copies of πΎπ is
π =ππ 21βπ2 .
There exist a coloring with at mostπmonochromatic πΎπβs.
By removing a vertex from each of these copies, we obtain a coloring of πΎπβπ with no monochromatic πΎπ.
2/9/2016
10
Recap
How we used the probabilistic method:
β¦ Our first applications were simply about making random choices and showing that we obtain some property with non-zero probability.
β¦ We moved to more involved proofs, where we use linearity of expectation to talk about the βexpectedβ result.
β¦ In the previous proof, we used a two step method β first we randomly choose an object, and then we alter it. This method is called the alternation method.
Transmission Towers
Problem. A company wants to establish transmission towers in its large compound.
β¦ Each tower must be on top of a building and each building must be covered by at least one tower.
β¦ We are given the pairs of buildings such that a tower on one covers the other.
β¦ We wish to minimize the number of towers.
2/9/2016
11
Building a graph
We build a graph πΊ = π, πΈ .
β¦ A vertex for every building.
β¦ An edge between every pair of buildings that can cover each other.
β¦ We need to find the minimum subset of vertices πβ² β π such that every vertex of πhas at least one vertex of πβ² as a neighbor.
Dominating Sets
Consider a graph πΊ = π, πΈ . A dominating set of πΊ is a subset πβ² β πsuch that every vertex of π has at least one neighbor in πβ².
It is not known whether there exists a polynomial-time algorithm for fining a minimum dominating set in a graph.
2/9/2016
12
Warm Up
Problem. Let πΊ = π, πΈ be a graph with maximum degree π. Give a lower bound for the size of any dominating set of πΊ.
Answer.
β¦ Every vertex covers itself and at most π other vertices, so any dominating set is of size at least
π
π + 1.
The Case of a Minimum Degree
Theorem. Let πΊ = π, πΈ be a graph with minimum degree π. Then there exists a
dominating set of size at most π β 1+lg π
π+1.
Proof. We consider a random subset π β π by independently taking each
vertex of π with probability π =lg π
π+1.
Let π β π β π be the vertices that have no neighbors in π.
β¦ π βͺ π is a dominating set.
2/9/2016
13
π β π β a random subset formed by independently taking each vertex of π with
probability π =lg π+1
π+1.
π β π β π β the vertices with no neighbors in π.
β¦ π βͺ π is a dominating set.
πΈ π = π£βπ Pr[π£ β π] = π£ π = π π .
A vertex is in π if it is not in π and none of its neighbors are in π. The probability for this is at most 1 β π π+1.
πΈ π = π£βπ Pr[π£ β π] β€ π£ 1 β ππ+1
= 1 β π π+1 π .
Proof (cont.)
Completing the Proof
π =lg π+1
π+1.
Famous inequality. 1 β π β€ πβπ for any positive π.
Thus, 1 β π π+1 β€ πβπ π+1 =1
π+1.
We provedπΈ π| + |π = πΈ π + πΈ |π|
β€ π + 1 β π π+1 π =lg π + 1 + 1
π + 1|π|.
There must exist a dominating set of size.
2/9/2016
14
How to Choose the Probability?
In the previous problem, we knew to
choose π =lg(π+1)
π+1. But how?
β¦ When you solve a question and the choice is not uniform, first mark the probability as π.
β¦ At the end of the analysis you will obtain some expression with π in it. Choose the value of π that optimizes the expression.
The End: Professor Moriarty
Professor Moriarty is a mathematician.
β¦ βAt the age of twenty-one he wrote a treatise upon the binomial theoremβ.
β¦ So a combinatorist?!
β¦ Dr. Octopus is a nuclear physicist.
β¦ Dr. Evil is a medical doctor.