COMP9020 Lecture 9bSession 2, 2015

Counting and Likelihood

Revision: 1.2

1

Counting Techniques

General idea: find methods, algorithms, and occasional preciseformulae to count the number of elements in various sets orcollections derived, in a structured way, from some basic sets.Count may mean a precise value or an estimate. The formershould be a useful, comprehensible expression.The latter should be asymptotically correct or at least give a goodasymptotic bound, whether upper or lower. If S is the base set,|S | = n and we denote by c(S) some collection of objectsconstructed on the basis of S then we seek constants a, b such that

a ≤ limn→∞

est(|c(S)|)|c(S)|

≤ b.

2

Ideally we would like to know the precise constant

limn→∞

est(|c(S)|)|c(S)|

= a.

In practice, we sometimes have to settle for

limn→∞

est(|c(S)|)|c(S)|

≤ φ(n)

for some slowly growing function φ(n).

3

Combinatorial Objects

Example

Single base set S = s1, . . . , sn, |S | = n; find the number of

all subsets

ordered selections of r different elements of S

unordered selections of r different . . .

selections of r not necessarily different . . .

partitions of S

partitions of S in k equivalence classes

graphs with S as labelled vertices

trees with S . . .

binary trees . . .

4

Example

Several base sets S = S1,T = S2, . . . ; |Si | = ni ;find the number of

functions S −→ T

functions from S onto T

partial functions

trees on T with labels from S

5

Importance of Counting Methods in CS/CE

Direct information

complexity of algorithms

complexity of decision processes

data storage requirements

Existence and nonexistence of objects

various inter-networking problems (existence of graphs withspecial properties)

various security issues (absence of security leaks)

Likelihood and inference

basis of all discrete probability models, esp. over finite domains

reliability and quality assurance

6

Basic Counting Rules

Union rule — S and T disjoint

|S ∪ T | = |S |+ |T |

S1, S2, . . . ,SN pairwise disjoint (Si ∩ Sj = ∅ for i 6= j)

|S1 ∪ . . . ∪ Sn| =∑|Si |

7

Product rule

|S1 × . . .× Sk | = |S1| · |S2| · . . . · |Sk | =n∏

i=1

|Si |

If all Si = S (the same set) and |S | = m then |Sk | = mk .If F(S, T ) denotes all the functions from S to T then

|F(S, T )| = |T ||S|

8

Basic Inferences

For arbitrary sets S ,T , . . .

|S ∪ T | = |S |+ |T | − |S ∩ T ||T \ S | = |T | − |S ∩ T |

|S1 ∪ S2 ∪ S3| = |S1|+ |S2|+ |S3|− |S1 ∩ S2| − |S1 ∩ S3| − |S2 ∩ S3|+ |S1 ∩ S2 ∩ S3|

9

Corollaries

If |S ∪ T | = |S |+ |T | then S and T are disjoint

If |⋃n

i=1 Si | =∑n

i=1 |Si | then Si are pairwise disjoint

If |T \ S | = |T | − |S | then S ⊆ T

These properties can serve to identify the cases when the sets aredisjoint (resp. contained).Hint for proofs:|S |+ |T | = |S ∪ T | means |S ∩ T | = |S |+ |T | − |S ∪ T | = 0|T \ S | = |T | − |S | means |S ∩ T | = |S | means S ⊆ T

10

Combinatorial Objects — How Many?

Permutations — ordering of all objects;equivalently: selecting all objects when recognising the order ofselection.Their number

n! = n × (n − 1)× . . .× 1, 0! = 1! = 1

r -permutations:selecting any r objects without repetition when recognising theorder of selection.Their number

Π(n, r) = P(n, r) = n(n − 1) . . . (n − r + 1) =n!

(n − r)!

11

r -selections ⇔ r -combinations:Collecting any r distinct objects without repetition;Equivalently: selecting r objects and not recognising the order ofselection.Their number(

n

r

)= C r

n =n!

(n − r)!r !=

n(n − 1) . . . (n − r + 1)

1 · 2 · · · r

12

These numbers are usually termed binomial coefficients due to theformula

(a + b)n = an + nan−1b +

(n

2

)an−2b2 + . . .+ bn =

n∑i=0

(n

i

)an−ibi

13

Examples (of simple counting problems)

number of edges in a complete graph Kn

number of diagonals in a convex polygon

number of diagonals in a regular n-dimensional polyhedron,either n-cube or n-octahedron

all diagonalsmain (interior) diagonals

number of segments when dividing a circle with all the chordson n points

number of poker hands; ‘correct’ ranking of poker hands

decisions in games, lotteries etc.

14

Example

Suppose we want to find the number of committees selected from12 men and 16 women. If the committee is required to have:

1 7 members: then there are(12+16

7

)different committees.

2 3 men and 4 women:(123

)(164

)3 7 men or 7 women:

(127

)+(167

)

15

Example

Draw 10 cards with replacement from a 52-card deck.(a) No. of ways that the 10’th card is not a repetition:Draw the tenth card first — 52 ways; draw the other nine cardsnot matching the 10’th: 51× . . .× 51, for the total 52 · 519.(b) That the 10’th card is a repetition:5210 − 52× 519 = 52(529 − 519)

16

Counting Poker handsA poker hand is 5 cards drawn without replacement from astandard deck of 52 cardsA, 2− 10, J,K ,Q × ♣,♠, ,♥.(a) Number of “4 of a kind” hands (e.g. 4 J’s)

|rank of 4-of-a-kind| × |odd card| = 13× (52− 4)

(b) Number of non-straight flush (all cards of same suit but not aconsecutive block (e.g. 9,10, J,K,Q))

|non straight flush| = |all flush| − |straight flush|= |suit| × |5-hand in a given suit|

−|suit| × |rank of a straight flush in a given suit|

= 4×(

13

5

)− 4× 10

17

Probability — very elementary

(the general story is more complicated but well worth the study;check our web site for pointers)Sample space

Ω = ω1, . . . , ωn

where each point ωi represents an outcome, and each outcome isequally likely.Likelihood of any given outcome ωi is 1

n where n = |Ω|

P(ω1) = P(ω2) = . . . = P(ωn) =1

n

It is a called a uniform probability distribution over the space ofoutcomes.

18

Tossing a coin:Ω = H,T

P(H) = P(T ) =1

2

Throwing a die:Ω = 1, 2, 3, 4, 5, 6

P(1) = P(2) = P(3) = P(4) = P(5) = P(6) =1

6

(Note: one die, two or more dice.)

19

Slight modification is needed to define non-uniform probabilitydistributions:

Ω = ω1, . . . , ωn

P(ω1) = p1,P(ω2) = p2, . . . ,P(ωn) = pn

p1 + . . .+ pn = 1

(This is still not the full story because we only model finite andmaybe countably infinite sample spaces.)

20

An event is a collection of outcomes, a subset of Ω.

P(E ) =∑ω∈E

P(ω)

E = ωi1 , . . . , ωik ⇒ P(E ) = pi1 + . . .+ pik

P(∅) = 0, P(Ω) = 1, P(E ) = 1− P(E )

21

Computing probabilities with respect to a uniform distributioncomes down to counting the size of the event.If E = e1, . . . , ek then

P(E ) =k∑

i=1

P(ei ) =k∑

i=1

1

|Ω|=|E ||Ω|

Most of the counting rules carry on to probabilities with respect toa uniform distribution.The expression “selected at random”, when not further qualified,means:“subject to / according to / . . . a uniform distribution.”

22

Example

Given Σ = a, b, c, d , e a word of four letters is selected “atrandom” from |Σ4| = 54 = 625. Consider some events.

1 E - event consisting of words with distinct letters:|E | = Π(5, 4), P(E ) = 5·4·3·2

54= 120

625 ' 19%

2 E - event consisting of words without vowels (”a,e,i,o,u”):|E | = 34, P(E ) = 81

625 ' 13%

3 E - event consisting of words beginning with a vowel:|E | = 2× 53, P(E ) = 2

5

23

Example

An urn has 3 red and 4 black balls. 3 balls are drawn withoutreplacement. Balls of the same colour are still distinguishable, say,by having numbers printed on them.Probabilities of drawing various colour patterns in the sets of threeballs are computed using as the sample space all ways of drawingthree balls w/o replacement.Their number is

7× 6× 5

3!= 35

1 E = All balls are red: 1 combination

2 E = All balls are black:(43

)= 4 combinations

3 E = One red and two black:(31

)×(42

)= 18 combinations

24

Example

Given that P(A) = 0.5,P(B) = 0.8,P(A ∩ B) = 0.4, what are thefollowing?

1 P(B) = 1− P(B) = 0.2

2 P(A ∪ B) = P(A) + P(B)− P(A ∩ B) = 0.9

3 P(A ∪ B) = 1− P(A ∪ B) = 1− P(A ∩ B) = 0.6

25