rules of probability. the additive rule p[a b] = p[a] + p[b] – p[a b] and if p[a b] = p[a b]...

Rules of Probability

The additive rule

P[A B] = P[A] + P[B] – P[A B]

and

if P[A B] = P[A B] = P[A] + P[B]

The additive rule for more than two events

then

and if Ai Aj = for all i ≠ j.

11

n n

i i i ji i ji

P A P A P A A

i j ki j k

P A A A

1

1 21n

nP A A A

11

n n

i iii

P A P A

The Rule for complements

for any event E

1P E P E

Conditional Probability,Independence

andThe Multiplicative Rue

Then the conditional probability of A given B is defined to be:

P A BP A B

P B

if 0P B

if 0

if 0

P A P B A P AP A B

P B P A B P B

The multiplicative rule of probability

and

P A B P A P B

if A and B are independent.

This is the definition of independent

1 2 nP A A A

The multiplicative rule for more than two events

1 2 1 3 2 1P A P A A P A A A

1 2 1n n nP A A A A

and continuing we obtain

1 2 1 2 1n n nP A A A P A A A A

1 2 1 1 2 1n n nP A A A P A A A A

1 2 2 1 1 2 2n n nP A A A P A A A A

1 2 1n nP A A A A

1 2 1 3 2 1P A P A A P A A A

1 2 1n n nP A A A A

Proof

Example

What is the probability that a poker hand is a royal flush

i.e.

1. 10 , J , Q , K ,A

2. 10 , J , Q , K ,A

3. 10 , J , Q , K ,A

4. 10 , J , Q , K ,A

SolutionLet A1 = the event that the first card is a “royal flush” card.

1. 10 , J , Q , K ,A

2. 10 , J , Q , K ,A

3. 10 , J , Q , K ,A

4. 10 , J , Q , K ,A

Let Ai = the event that the ith card is a “royal flush” card. i = 2, 3, 4, 5.

20 341 2 1 3 2 152 51 50, , ,P A P A A P A A A

2 14 3 2 1 5 4 3 2 149 48,P A A A A P A A A A A

Another solution is by counting

4 4Royal Flush

52 52 51 50 49 485 4 3 2 15

P

Royal FlushP

20 34 2 11 2 3 4 5 52 52 50 49 48P A A A A A

4 5 4 3 2 1 20 4 3 2 1

52 51 50 49 48 52 51 50 49 48

The same result

Independencefor more than 2 events

Definition:

The set of k events A1, A2, … , Ak are called mutually independent if:

P[Ai1 ∩ Ai2 ∩… ∩ Aim

] = P[Ai1] P[Ai2

] …P[Aim]

For every subset {i1, i2, … , im } of {1, 2, …, k }

i.e. for k = 3 A1, A2, … , Ak are mutually independent if:

P[A1 ∩ A2] = P[A1] P[A2], P[A1 ∩ A3] = P[A1] P[A3],

P[A2 ∩ A3] = P[A2] P[A3],

P[A1 ∩ A2 ∩ A3] = P[A1] P[A2] P[A3]

0.12

P[A1] = .4, P[A2] = .5 , P[A3] = .6

P[A1∩A2] = (0.4)(0.5) = 0.20

P[A1 ∩ A3] = (0.4)(0.6) = 0.24

P[A2 ∩ A3] = (0.5)(0.6) = 0.30

P[A1 ∩ A2 ∩ A3] =

(0.4)(0.5)(0.6) = 0.12

A1 A2

A3

0.08 0.08

0.12

0.12

0.18

0.18

0.12

Definition:

The set of k events A1, A2, … , Ak are called pairwise independent if:

P[Ai ∩ Aj] = P[Ai] P[Aj] for all i and j.

i.e. for k = 3 A1, A2, … , Ak are pairwise independent if:

P[A1 ∩ A2] = P[A1] P[A2], P[A1 ∩ A3] = P[A1] P[A3],

P[A2 ∩ A3] = P[A2] P[A3],

It is not necessarily true that P[A1 ∩ A2 ∩ A3] = P[A1]

P[A2] P[A3]

0.14

P[A1] = .4, P[A2] = .5 , P[A3] = .6

P[A1∩A2] = (0.4)(0.5) = 0.20

P[A1 ∩ A3] = (0.4)(0.6) = 0.24

P[A2 ∩ A3] = (0.5)(0.6) = 0.30

P[A1 ∩ A2 ∩ A3] = 0.14

≠ (0.4)(0.5)(0.6) = 0.12

A1 A2

A3

0.06 0.10

0.10

0.14

0.20

0.16

0.10

Bayes Rule• Due to the reverend T.

Bayes• Picture found on website:

Portraits of Statisticians• http://www.york.ac.uk/

depts/maths/histstat/people/welcome.htm#h

P A P B AP A B

P A P B A P A P B A

Proof:

P A B

P A BP B

P A P B A

P A P B A P A P B A

P A B

P A B P A B

Example:

We have two urns. Urn 1 contains 14 red balls and 12 black balls. Urn 2 contains 6 red balls and 20 black balls.An Urn is selected at random and a ball is selected from that urn.

If the ball turns out to be red what is the probability that it came from the first urn?

Urn 1 Urn 2

Solution:

Note: the desired conditional probability is in the reverse direction of the given conditional probabilities. This is the case when Bayes rule should be used

14 6,

26 26P B A P B A

A

Let A = the event that we select urn 1

= the event that we select urn 2

1

2P A P A

Let B = the event that we select a red ball

We want .P A B

Bayes rule states

P A P B AP A B

P A P B A P A P B A

1 142 26

61 14 12 26 2 26

14 140.70

14 6 20

Example: Testing for a disease

Suppose that 0.1% of the population have a certain genetic disease.

A test is available the detect the disease.

If a person has the disease, the test concludes that he has the disease 96% of the time. It the person doesn’t have the disease the test states that he has the disease 2% of the time.

Two properties of a medical testSensitivity = P[ test is positive | disease] = 0.96

Specificity = P[ test is negative | disease] = 1 – 0.02 = 0.98

A person takes the test and the test is positive, what is the probability that he (or she) has the disease?

Solution:

Note: Again the desired conditional probability is in the reverse direction of the given conditional probabilities.

0.96, 0.02P B A P B A

A

Let A = the event that the person has the disease

= the event that the person doesn’t have the disease

0.001, 1 0.001 0.999P A P A

Let B = the event that the test is positive.

We want .P A B

Bayes rule states

P A P B A

P A BP A P B A P A P B A

0.001 0.96

0.001 0.96 0.999 0.02

0.000960.0458

0.00096 .01998

Thus if the test turns out to be positive the chance of having the disease is still small (4.58%).Compare this to (.1%), the chance of having the disease without the positive test result.

Let A1, A2 , … , Ak denote a set of events such that

1 1

i ii

k k

P A P B AP A B

P A P B A P A P B A

An generlization of Bayes Rule

1 2 and k i jS A A A A A

for all i and j. Then

If A1, A2 , … , Ak denote a set of events such that

1 2 and k i jS A A A A A

for all i and j. Then A1, A2 , … , Ak is called a partition of S.

A1Ak

A2

…

S

1 kP B P B A P B A

Proof

1 kB B A B A for all i and j.

A1Ak

A2

B

and i jB A B A

Then

1 1 k kP A P B A P A P B A

i

i

P A BP A B

P B

and

1 1

i i

k k

P A P B A

P A P B A P A P B A

Example:We have three urns. Urn 1 contains 14 red balls and 12 black balls. Urn 2 contains 6 red balls and 20 black balls. Urn 3 contains 3 red balls and 23 black balls.An Urn is selected at random and a ball is selected from that urn.

If the ball turns out to be red what is the probability that it came from the first urn? second urn? third Urn?

Urn 1 Urn 2 Urn 3

Solution:

Note: the desired conditional probability is in the reverse direction of the given conditional probabilities. This is the case when Bayes rule should be used

1 2 3

14 6 3, ,

26 26 26P B A P B A P B A

1 2 3S A A A

Let Ai = the event that we select urn i

1 2 3

1

3P A P A P A

Let B = the event that we select a red ball

We want for 1, 2,3.iP A B i

Bayes rule states

1 11

1 1 2 2 3 3

P A P B AP A B

P A P B A P A P B A P A P B A

1 143 26

6 31 14 1 13 26 3 26 3 26

14 14

14 6 3 23

613 26

2 6 31 14 1 13 26 3 26 3 26

6 6

14 6 3 23P A B

313 26

3 6 31 14 1 13 26 3 26 3 26

3 3

14 6 3 23P A B

Example:

Suppose that an electronic device is manufactured by a company.During a period of a week

– 15% of this product is manufactured on Monday,– 23% on Tuesday, – 26% on Wednesday , – 24% on Thursday and – 12% on Friday.

Also during a period of a week – 5% of the product is manufactured on Monday is

defective– 3 % of the product is manufactured on Tuesday is

defective, – 1 % of the product is manufactured on Wednesday

is defective , – 2 % of the product is manufactured on Thursday is

defective and – 6 % of the product is manufactured on Friday is

defective.

If the electronic device manufactured by this plant turns out to be defective, what is the probability that is as manufactured on Monday, Tuesday, Wednesday, Thursday or Friday?

Solution:

LetA1 = the event that the product is manufactured on

Monday

A2 = the event that the product is manufactured on Tuesday

A3 = the event that the product is manufactured on Wednesday

A4 = the event that the product is manufactured on Thursday

A5 = the event that the product is manufactured on Friday

Let B = the event that the product is defective

Now

P[A1] = 0.15, P[A2] = 0.23, P[A3] = 0.26, P[A4] = 0.24 and P[A5] = 0.12Also

P[B|A1] = 0.05, P[B|A2] = 0.03, P[B|A3] = 0.01,

P[B|A4] = 0.02 and P[B|A5] = 0.06We want to find

P[A1|B], P[A2|B], P[A3|B], P[A4|B] and P[A5|B] .

1 1 5 5

i i

i

P A P B AP A B

P A P B A P A P B A

We will apply Bayes Rule

i P[Ai] P[B|Ai] P[Ai]P[B|Ai] P[Ai|B]

1 0.15 0.05 0.0075 0.2586

2 0.23 0.03 0.0069 0.2379

3 0.26 0.01 0.0026 0.0897

4 0.24 0.02 0.0048 0.1655

5 0.12 0.06 0.0072 0.2483

Total 1.00 0.0290 1.0000

The sure thing principle and Simpson’s paradox

The sure thing principleSuppose andP A C P B C

thenP A C P B C

P A P B

Example – to illustrate

Let A = the event that horse A wins the race.

B = the event that horse B wins the race.

C = the event that the track is dry

= the event that the track is muddyC

Proof:

implies P A C P B C

P A C P B CP C P C

P A P A C P A C

or P A C P B C

implies P A C P B C

P A C P B CP C P C

or P A C P B C

P B C P B C P B

Simpson’s ParadoxDoes

andP D S C P D S C implyP D S C P D S C

?P D S P D S Example to illustrate

D = death due to lung cancerS = smokerC = lives in city, = lives in countryC

P D S C P D S CP D SP D S

P S P S

similarly

P D S C P S CP D S C P S C

P S C P S P SP S C

P D S C P C S P D S C P C S

P D S P D S C P C S P D S C P C S

Solution

is greater than

P D S P D S C P C S P D S C P C S

P D S P D S C P C S P D S C P C S

if andP D S C P D S C

P D S C P D S C

whether

, 1 ,P C S P C S P C S

depends also on the values of

and 1P C S P C S P C S

thanP D S P D S C P C S P D S C P C S

P D S P D S C P C S P D S C P C S

Suppose 0.90 =0.60 andP D S C P D S C

0.40 0.10P D S C P D S C

whether.10, 1 .90,P C S P C S P C S

and

=.80 and 1 0.20P C S P C S P C S

.90 .10 .40 .90 .45

.60 .80 .10 .20 .50