chapter 3: a little combinatoricsbazuinb/ece3800/b_notes03.pdf · 3.1 basics of counting 3.2 notes...

Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet,

Probability, Statistics, and Random Signals, Oxford University Press, February 2016.

B.J. Bazuin, Fall 2020 1 of 19 ECE 3800

Charles Boncelet, “Probability, Statistics, and Random Signals," Oxford University Press, 2016. ISBN: 978-0-19-020051-0

Chapter 3: A LITTLE COMBINATORICS

Sections 3.1 Basics of Counting 3.2 Notes on Computation 3.3 Combinations and the Binomial Coefficients 3.4 The Binomial Theorem 3.5 Multinomial Coefficient and Theorem 3.6 The Birthday Paradox and Message Authentication 3.7 Hypergeometric Probabilities and Card Games Summary Problems




Combinatorics

Some important math before doing more probability …

From Merriam Webster’s Dictionary. http://www.merriam-webster.com/dictionary/combinatorial

Combinatorial - of or relating to the arrangement of, operation on, and selection of discrete mathematical elements belonging to finite sets or making up geometric configurations.

For a population of size n … the set contains n elements (a deck of 52 playing cards)

A subpopulation of size r can be defined (draw 5 cards at random from the deck)

How many unique subpopulations of r can we expect (notice that the same r elements can be selected in numerous ways).

(i) Sampling with replacement is the easy way …

Possible combinations rnnnnn

(ii) Sampling without replacement

Possible combinations !!

121rn

nrnnnn

Next considerations … how many ways can the r things be selected …

Possible selections !121 rrrr

Now we can consider the “unique” combinations

Unique combinations !!

!

rrn

n

lectionPossibleSe

mbinationPossibleCo

We have now defined an operator to determine unique values for “n choose r”

!!

!

rrn

n

r

nC n

r

also sometimes shown as !!

!,

rrn

nrnCCrn

This equation also defines binomial coefficients. There are also some important “definitions”

1!0!

!

0

n

nnC n

r and 1!0!

!

n

n

n

nC n

r




Why are they called binomial coefficients? https://en.wikipedia.org/wiki/Binomial_coefficient

Binomial Powers nx1 or nyx - the coefficients for the various power resulting from 2 summed elements to the nth

power.

kknn

k

n yxk

nyx

0

The coefficients can also be selected using Pascal’s Triangle https://en.wikipedia.org/wiki/Pascal%27s_triangle also called a Binomial Expansion.

1 1 1

1 2 1 1 3 3 1

1 4 6 4 1 1 5 10 10 5 1

1 6 15 20 15 6 1

Each row starts (and ends) with 1 and then sums the adjacent coefficients from the next higher row.

So, by inspection 431221344 14641 bbababaaba .

Now, if you flip a coin 4 times, what are the possible combinations and how many times do they occur?

What if I said that: a=Heads and b=Tails

1 – H^4 4 – H^3xT^1 6 – H^2xT^2 4 – H^1xT^3 1 – T^4

Based on this concept, we can define a summation

kknn

k

n yxk

nyx

0

There are some interesting results in manipulating this summation …




The basic equation kknn

k

n yxk

nyx

0

Letting x=y=1 𝑥 𝑦 2 ∑𝑛𝑘 ∙ 1 ∙ 1 ∑

𝑛𝑘

Math equality nn

k k

n2

0

Also note for 𝑥 𝑝 and 𝑦 1 𝑝 𝑞

𝑝 1 𝑝 1 1𝑛𝑘 ∙ 𝑝 ∙ 1 𝑝

𝑝 𝑞 1 1𝑛𝑘 ∙ 𝑝 ∙ 𝑞

It is a proper probability mass function (pmf definitions coming soon)

Finally for 𝑥 𝑝

1𝑛𝑘 ∙

12

∙12

12

∙𝑛𝑘

Anothercombinatorialmathidentity

𝑛𝑘

𝑛!𝑘! ∙ 𝑛 𝑘 !

𝑛𝑛 𝑘

Notice the symmetry of Pascal’s triangle.




5‐CardDrawCombinatorial

How many ways can 5 cards be drawn from a deck of 52 playing cards? A combinatorial analysis!

𝐶𝑛𝑟

𝑛!𝑟! ∙ 𝑛 𝑟 !

𝐶 525

52!5! ∙ 52 5 !

2,598,960

If you are poker player … see

https://en.wikipedia.org/wiki/Poker_probability

Now you have to figure out the number of ways that each poker hand can be dealt to determine the probabilities. (And if any easy formula exists to do it).




3.1 Basics of Counting

1. Selections are ordered or in an arbitrary order.

2. Selections are made with replacement or not with replacement.

Define a set of 4 elements {a, b, c, d}

The possible random selection, with replacement, done twice has the following results:

a. Ordered, with replacement

For n elements selected this way there are 𝑛 4 16 possible results.

b. Ordered without replacement

For n elements selected this way there are 𝑛 ∙ 𝑛 1 4 ∙ 3 12 possible results.

c. Unordered, with replacement

For n elements selected this way there are ∙ ∙ 10 possible results.

d. Unordered, without replacement

For n elements selected this way there are ∙ ∙ 6 possible results.




Generalized Formula

a. Ordered, with replacement of n elements r time

𝑛 ∙ 𝑛 ∙ 𝑛⋯𝑛 𝑛

b. Ordered without replacement of n elements r times

𝑛 ∙ 𝑛 1 ∙ 𝑛 2 ⋯ 𝑛 𝑟 1𝑛!

𝑛 𝑟 !𝑛

where the textbook has defined new notation.

d. Unordered, without replacement

𝑛 ∙ 𝑛 1 ∙ 𝑛 2 ⋯ 𝑛 𝑟 1𝑟!

𝑛!𝑛 𝑟 ! ∙ 𝑟!

𝑛𝑟!

𝑛𝑟

c. Unordered, with replacement

𝑛 𝑟 1 !𝑛 1 ! ∙ 𝑟!

𝑛 𝑟 1𝑟

This one is not intuitively obvious.

The results can be summarized as …




Computer computation of the combinatorial. function [result]=bon_nchoosek(n,k) % Boncelet textbook nchoosek function % Check for valid, k and n if (mod(n,1)~=0) error('n not an integer'); end if (mod(k,1)~=0) error('k not an integer'); end if ( (k>n) || (n<0) || (k<0) ) error('illegal k or n'); end % select the smaller of k and n-k if k > n/2 k = n-k; end result = 1; for j=1:k result = result * (n-j+1); result = result/j; end

There is a numerical limit to a Matlab factorial function in terms of bit precision. Using appropriate algorithm/multiplication ordering, numerical accuracy can be maintained.

Otherwise, the following is a simple notation ….

nchoosek = factorial(n)/(factorial(k)*factorial(n-k));




More Math Equalities

Another combinatorial math identity (useful in deriving the recursive Binomial Theorem) 𝑛𝑘

𝑛 1𝑘

𝑛 1𝑘 1

Manipulating the first term to be similar to the second 𝑛 1𝑘

𝑛 1 !

𝑘! ∙ 𝑛 1 𝑘 !

𝑛 𝑘𝑘

∙𝑛 1 !

𝑘 1 ! ∙ 𝑛 1 𝑘 1 !

𝑛 1𝑘

𝑛 1 !

𝑘! ∙ 𝑛 1 𝑘 !

𝑛 𝑘𝑘

∙ 𝑛 1𝑘 1

But then adding the two terms 𝑛 1𝑘

𝑛 1𝑘 1

𝑛 𝑘𝑘

∙ 𝑛 1𝑘 1

𝑛 1𝑘 1

𝑛 1𝑘

𝑛 1𝑘 1

𝑛 𝑘𝑘

1 ∙ 𝑛 1𝑘 1

𝑛𝑘∙ 𝑛 1𝑘 1

𝑛 1𝑘

𝑛 1𝑘 1

𝑛𝑘∙ 𝑛 1𝑘 1

𝑛𝑘∙

𝑛 1 !

𝑘 1 ! ∙ 𝑛 1 𝑘 1 !

𝑛 1𝑘

𝑛 1𝑘 1

𝑛!𝑘! ∙ 𝑛 𝑘 !

𝑛𝑘

This can be used for proving

𝑎 𝑏 𝑎 𝑏 ∙ 𝑎 𝑏

for

𝑎 𝑏𝑛𝑘 ∙ 𝑎 ∙ 𝑏

Starting with

𝑛𝑘 ∙ 𝑎 ∙ 𝑏 𝑎 𝑏 ∙ 𝑛 1

𝑘∙ 𝑎 ∙ 𝑏




3.5 Multinomial Coefficients (for math lovers)

Theorem 1.8-1: Let n consists of multiple subsets, each of ri elements such that

K

iirn

1

The number of ways in which the population of n elements can be partitioned into K subpopulations of which each contains ri element is

!!!!

!

321 Krrrr

n

Discussion and development in the textbook …




3.6 Birthday Paradox and Message Authentication

For a class of how many students, k, will the probability of two or more students having the same birthday be greater than 50%?

Compute as a negative probability … the likelihood of no matching birthdays.

By person development

k=1 𝑃𝑟 1 𝑞 1 1.0

k=2 𝑃𝑟 2 𝑞 1 ∙ ∙

k=3 𝑃𝑟 3 𝑞 2 ∙ ∙ ∙

k=k 𝑃𝑟 𝑘 𝑞 𝑘 1 ∙ ∙ !

!

5 10 15 20 25 30 35 40 45 500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

X: 23Y: 0.4927

Pro

babi

lity

Number of Students

No Matching Birthday Probability




Making an approximation … using n instead of 365

k-1→k 𝑃𝑟 𝑘 𝑞 𝑘 1 ∙ 𝑞 𝑘 1 ∙ 1

Any time there is a large product sequence, a logarithmic equivalent may be used

k-1→k 𝑙𝑛 𝑃𝑟 𝑘 𝑙𝑛 𝑞 𝑘 1 ∙ 𝑙𝑛 𝑞 𝑘 1 𝑙𝑛 1

k-1→k 𝑙𝑛 𝑃𝑟 𝑘 ∑ 𝑙𝑛 1

Using an approximation for small x…

𝑙𝑛 1 𝑥 𝑥 or 𝑙𝑛 1 𝑥 𝑥

k-1→k 𝑙𝑛 𝑃𝑟 𝑘 ∑

Since we prefer summations from zero and can compute the summation …

𝑙𝑛 𝑃𝑟 𝑘𝑗𝑛

1𝑛∙ 𝑗

1𝑛∙𝑘 ∙ 𝑘 1

2

5 10 15 20 25 30 35 40 45 500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Number of Students

Pro

babi

lity

No Matching Birthday Probability

Prob

Approx




Estimating 50%: 𝑙𝑛 𝑃𝑟 𝑘 0.5 0.693 ∙ ∙

252.9987𝑘 ∙ 𝑘 1

2

252.9987𝑘 ∙ 𝑘 1

2

0 𝑘 𝑘 505.997

Finding the positive root 𝑘 22.9999

MessageAuthentication…

Your wireless communication devices use a “message authentication code” …

… nominally called “Media Access Control” (MAC) addresses.

The Birthday paradox defines a means for hackers to crack generated secure access information and “spoof” another user.

Pick a number of bits for your “Code”: possible values 2

How long until at least two codes match?

Estimating 50%:

𝑙𝑛 𝑃𝑟 𝑘 0.5 0.6931

2∙𝑘 ∙ 𝑘 1

2

0.693 ∙ 2 𝑘 ∙ 𝑘 1

0 𝑘 𝑘 0.693 ∙ 2

Finding the positive root 𝑘 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒𝑑 𝑎𝑡𝑡𝑒𝑚𝑝𝑡𝑠 𝑛𝑒𝑒𝑑𝑒𝑑

𝑟𝑜𝑜𝑡𝑠𝑏2

𝑏2

𝑐

𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑟𝑜𝑜𝑡12

14

0.693 ∙ 2 ~2 ∙ √0.693~0.83 ∙ 2

For a 10-bit code, about 38 trials … for 16 bits, about 302 … etc.

The text describes the concept …




3.7 Hypergeometric Probabilities

From: http://en.wikipedia.org/wiki/Hypergeometric_distribution

In probability theory and statistics, the hypergeometric distribution is a discrete probability distribution (probability mass function) that describes the number of successes in a sequence of n draws from a finite population without replacement.

A typical example is the following: There is a shipment of N objects in which D are defective. The hypergeometric distribution describes the probability that in a sample of n distinctive objects drawn from the shipment exactly x objects are defective.

n

N

xn

DN

x

D

nDNXx ,,,Pr

for DnxNnD ,min,0max

The equation is derived based on a non-replacement Bernoulli Trials …

Where the denominator term defines the number of trial possibilities, the 1st numerator term defines the number of ways to achieve the desired x, and the 2nd numerator term defines the filling of the remainder of the set.

The text extends this to multiple selections

𝑃𝑟 𝑘 , 2,⋯ ,𝑘 ,

𝑛𝑘 ∙

𝑛𝑘 ∙ ⋯ ∙

𝑛𝑘

𝑛𝑘

where nj or subsets of n and kj selected from nj




Quality Control Example

A batch of 50 items contains 10 defective items. Suppose 10 items are selected at random and tested. What is the probability that exactly 5 of the items tested are defective?

The number of ways of selecting 10 items out of a batch of 50 is the number of combinations of size 10 from a set of 50 objects:

!40!10

!50

10

505010

C

The number of ways of selecting 5 defective and 5 nondefective items from the batch of 50 is the product N1 x N2 where N1 is the number of ways of selecting the 5 items from the set of 10 defective items, and N2 is the number of ways of selecting 5 items from the 40 nondefective items.

!35!5

!40

!5!5

!10405

105 CC

Thus the probability that exactly 5 tested items are defective is the desired ways the selection can be made divided by the total number of ways selection can be made, or

0.01614 01027227817

658008252

!40!10

!50!35!5

!40

!5!5

!10

5010

405

105

C

CC

Suppose 10 items are selected at random and tested. What is the probability that exactly 1 of the items tested are defective?

𝐶 ∙ 𝐶

𝐶

10!9! ∙ 1! ∙ 40!

31! ∙ 9!50!

40! ∙ 10!

10 ∙ 27343888010272278170

0.2662

In Matlab

nchoosek(10,1)*nchoosek(40,9)/nchoosek(50,10)

Now what is the probability that 10 items are selected at random and there is more than 1 item bad … you need to sum the exactly 1, 2, 3, 4, 5, … to 10.




Textbook box with Marbles Example

A box contains 9 marbles (4 red and 5 blue). Six marbles are blindly selected from the box. What is the probability of two red marbles and four blue marbles?

𝐶 ∙ 𝐶𝐶

4!2! ∙ 2! ∙ 5!

4! ∙ 1!9!

6! ∙ 3!

6 ∙ 59 ∙ 8 ∙ 7/6

6 ∙ 53 ∙ 4 ∙ 7

514

0.3571

nchoosek(10,1)*nchoosek(40,9)/nchoosek(9,6)= 0.3571

Textbook Poker Full House in 5 cards

Draw 5 cards from a deck of 52. For s specific full house, assume 3 queens and 2 eights.

The probability becomes

𝐶 ∙ 𝐶

𝐶

4!3! ∙ 1! ∙ 4!

2! ∙ 2!52!

47! ∙ 5!

4 ∙ 4 ∙ 32

52 ∙ 51 ∙ 50 ∙ 49 ∙ 48120

𝐶 ∙ 𝐶

𝐶

242598960

1108290

9.2345𝑒 06

The number of possible unique full houses is … 13 3-card groups and 12 2 card groups for a total of 13 x 12 = 156.

Therefore, the probability of being dealt a full house of any kind is

13 ∙ 12 ∙𝐶 ∙ 𝐶

𝐶

156108290

1.4406𝑒 03

(Note that this type of probability is always a rational fraction.)




Another Use: State Lottery Games

There are N objects in which D are of interest. The hypergeometric distribution describes the probability that in a sample of n distinctive objects drawn from the total set exactly x objects are of interest.

Lotteries …

N= number of balls to be selected at random D = the balls that you want selected n = the number of balls drawn x = the number of desired balls in the set that is drawn

Example: Michigan’s Classic Lotto 47

Prize Structure For Classic Lotto 47 (web site data)

Match Prize Odds of Winning

6 of 6 Jackpot 1 in 10,737,573

5 of 6 $2,500 (guaranteed) 1 in 43,649

4 of 6 $100 (guaranteed) 1 in 873

3 of 6 $5 (guaranteed) 1 in 50 Overall Odds: 1 in 47.6

Matlab Odds

Match Odds of Winning 1 in Percent Probability

6 of 6 10737573 <1x10-5%

5 of 6 43648.7 0.0023%

4 of 6 872.97 0.1146%

3 of 6 50.36 1.9856%

2 of 6 7.07 14.1471%

1 of 6 2.39 41.8753%

0 of 6 2.39 41.8753% Chance of winning 2.1024%

ROI per dollar without jackpot ~ $0.2711

see Matlab simulation “MI_Lotto.m”

Keno anyone? Another Michigan gambling game 80 balls, 20 drawn, you need to match k of n selected for n=2 to 20.




MI Keno

A Keno ticket with the payouts is shown!

Another hypergeometric density function

N= number of balls to be selected at random (80) D = the balls that you want selected (D) n = the number of balls drawn (20) x = the number of desired balls in the set that is drawn (0:D)

In general, you get $0.65 back for every $1 played. I did not include a “kicker” bet. The overall odds of a Kicker (1, 2, 3, 4, 5, 10) number being 2 or higher are 1:1.67. see MI_Keno.m on the web site for more information and results.

1 2 3 4 5 6 7 8 9 100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1MI Keno ROI and Pr[win]

ROI per $

Pr[win]




PowerBallLottery

The lottery is a 69 choose 5 game combined with a 26 choose 1 game.

see MATLAB code Power Ball total combinations = 292201338. Prob of 0+0 balls is 1 in 1.53296 min payout is $0. Prob of 0+1 balls is 1 in 38.3239 min payout is $4. Prob of 1+0 balls is 1 in 3.6791 min payout is $0. Prob of 1+1 balls is 1 in 91.9775 min payout is $4. Prob of 2+0 balls is 1 in 28.0531 min payout is $0. Prob of 2+1 balls is 1 in 701.328 min payout is $7. Prob of 3+0 balls is 1 in 579.765 min payout is $7. Prob of 3+1 balls is 1 in 14494.1 min payout is $100. Prob of 4+0 balls is 1 in 36525.2 min payout is $100. Prob of 4+1 balls is 1 in 913129 min payout is $50000. Prob of 5+0 balls is 1 in 1.16881e+07 min payout is $1000000. Prob of 5+1 balls is 1 in 2.92201e+08 min payout is $40000000. Prob of winning something is 1 in 24.8671. Expected Winnings per $2 without Jackpot = $0.32 Expected Winnings per $2 min $40M Jackpot = $0.46 Single Winner Break Even Jackpot = $490,936,628.00 Total computed without considering taxes. Max US tax rate 39.6%, MI tax rate 4.25%)!! Single Winner Break Even Jackpot with taxes= $848,886,670.24

chapter 3: a little combinatoricsbazuinb/ece3800/b_notes03.pdf · 3.1 basics of counting 3.2 notes...

Documents