1

Algorithms

CSCI 235, Fall 2012Lecture 9Probability

2

Permutations

How many ways can you take k items from a set of n items?Order matters (ab is counted separately from ba)No duplicates allowed (don't count aa)

Example: S = {a, b, c, d}, how many ways can we take 2 items?

abacad

babcbd

cacbcd

dadbdc

Total pairs = 124 ways to pick first3 ways to pick second

In general: The number of permutations of k items from a set of n is:

€

n(n −1)(n − 2)...(n − k +1) =n!

(n − k)!

3

Combinations

Combinations are like permutations, except order doesn't matter.ab and ba are considered the same.

abacad

bcbd cd

Combination of 4 things taken 2 at a time:

Each set of k items has k! possible permutations.number of combinations = number of permutations divided by k!

€

n

k

⎛

⎝ ⎜

⎞

⎠ ⎟=

n!

k!(n − k)!

4

k-tuples and k-selections

k-tuple (or k-string): Like a permutation, but can have duplicates.

aaabacad

babbbcbd

cacbcccd

dadbdcdd

n choices for 1st itemn choices for 2nd item, etc.

Total possible k-tuples for k items taken from a group of n items is nk

k-selection: Like combination, but can have duplicates.

aaabacad

bbbcbd

cccd dd

Formula:

€

(n −1+ k)!

k!(n −1)!

5

Sample Spaces

A sample space is the set of all possible outcomes for an experiment.

Experiment A: flip 3 coins

Sample space = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}

# of possibilities: ?

Experiment B: Flip a coin until you get heads.

Sample space = {H, TH, TTH, TTTH, . . .}

23

6

Diagramming sample spaces

Experiment A:(flip 3 coins)

7

Diagramming sample spaces

Experiment A:(flip 3 coins)

H3 H1H2H3

T3 H1H2T3

H3 H1T2H3

T3 H1T2T3

H3 T1H2H3

T3 T1H2T3

H3 T1T2H3

T3 T1T2T3

T1

T2

H2

T2

H2

H1

Experiment B:(flip until heads)

Samplespace(the set of all possible outcomes)

8

Diagramming sample spaces

Experiment A:(flip 3 coins)

H3 H1H2H3

T3 H1H2T3

H3 H1T2H3

T3 H1T2T3

H3 T1H2H3

T3 T1H2T3

H3 T1T2H3

T3 T1T2T3

T1

T2

H2

T2

H2

H1

Experiment B:(flip until heads) H1 T1H2 T1T2H3 T1T2T3H4

H1 H2 H3 H4

T1 T2 T3 T4 . . .

Samplespace(the set of all possible outcomes)

...

9

Events

An event is a subset of the sample space:Experiment A (flip 3 coins):

Event A1: First flip is head = {HHH, HHT, HTH, HTT}Event A2: Second flip is tail = {HTH, HTT, TTH, TTT}Event A3: Exactly 2 tails = {HTT, THT, TTH}Event A4: Two consecutive flips are the same=

{HHH, HHT, HTT, THH, TTH, TTT}

Experiment B (flip until get heads):Event B1: First flip is head = {H}Event B2: First flip is tail = {TH, TTH, TTTH ...}Event B3: Even number of flips = {TH, TTTH, TTTTTH ...}

10

Definitions

If A and B are events in sample space S, then

"A and B" is translated

€

A∩ B

"A or B" is translated

€

A∪B

"not A" is translated

€

S − A

Two events are mutually exclusive if

€

A∩ B =∅

11

Examples

1. Of the four events in Experiment A, which pairs are mutually exclusive?

2. Of the three events in Experiment B, which pairs are mutually exclusive?

12

Recall events

An event is a subset of the sample space:Experiment A (flip 3 coins):

Event A1: First flip is head = {HHH, HHT, HTH, HTT}Event A2: Second flip is tail = {HTH, HTT, TTH, TTT}Event A3: Exactly 2 tails = {HTT, THT, TTH}Event A4: Two consecutive flips are the same=

{HHH, HHT, HTT, THH, TTH, TTT}

Experiment B (flip until get heads):Event B1: First flip is head = {H}Event B2: First flip is tail = {TH, TTH, TTTH ...}Event B3: Even number of flips = {TH, TTTH, TTTTTH}

13

Probability DistributionA probability distribution Pr{ } on sample space S is any mapping from events of S to [0...1] such that the following axioms hold:

1) Pr{A} >= 0 for any event A

2) Pr{S} = 1

€

3) If A∩ B =∅ then Pr{A∪B} = Pr{A} + Pr{B}

Example:Flip two coins: S= {HH, HT, TH, TT}Event A = {HT} Event B = {TH}

What is

€

Pr{A∪B}?

14

Some useful theorems

€

1) Pr{∅ } = 0

€

2) Pr{S − A} =1− Pr{A}

€

3) If A⊆B then Pr{A} ≤ Pr{B}

€

4) Pr{A∪B} = Pr{A} + Pr{B} - Pr{A∩ B}

Example: Flip two coins.Event A = {HH, HT, TH}Event B = {HT, TH, TT}

What is

€

Pr{A∪B}?

15

Working with probability treesIf events at distinct stages of a probability tree are independent, then the probability of a leaf is the product of the probabilities on the path to the leaf.Experiment A (flip 3 weighted coins): H1=1/3, T1 = 2/3; H2=1/4, T2=3/4; H3=1/5, T3 = 4/5

16

Working with probability treesIf events at distinct stages of a probability tree are independent, then the probability of a leaf is the product of the probabilities on the path to the leaf.

H3 H1H2H3

T3 H1H2T3

H3 H1T2H3

T3 H1T2T3

H3 T1H2H3

T3 T1H2T3

H3 T1T2H3

T3 T1T2T3

T1

T2

H2

T2

H2

H1

Experiment A (flip 3 weighted coins): H1=1/3, T1 = 2/3; H2=1/4, T2=3/4; H3=1/5, T3 = 4/5

(1/3)(1/4)(1/5) = 1/60

(1/3)(1/4)(4/5) = 4/60 = 1/15

(1/3)(3/4)(1/5) = 3/60 = 1/20

(1/3)(3/4)(4/5) = 12/60 = 1/5

(2/3)(1/4)(1/5) = 2/60 = 1/30

(2/3)(1/4)(4/5) = 8/60 = 2/15

(2/3)(3/4)(1/5) = 6/60 = 1/10

(2/3)(3/4)(4/5) = 24/60 = 2/5

17

Another example

Experiment B (flip a weighted coin until heads): Hi = 1/3, Ti = 2/3

H1 T1H2 T1T2H3 T1T2T3H4

H1 H2 H3 H4

T1 T2 T3 T4 . . .

1/3(2/3)(1/3)= 2/9

(2/3)(2/3)(1/3)= 4/27

(2/3)(2/3)(2/3)(1/3)= 8/81

Probability of S =

€

1

3

2

3

⎛ ⎝ ⎜

⎞ ⎠ ⎟k

=k= 0

∞

∑1

3

2

3

⎛ ⎝ ⎜

⎞ ⎠ ⎟k

=k= 0

∞

∑1

3

1

1− 2/3

⎛ ⎝ ⎜

⎞ ⎠ ⎟=

1

3

1

1/3

⎛ ⎝ ⎜

⎞ ⎠ ⎟=1

1/3 + 2/9 + 4/27 + 8/81 + ... =