in today’s lecture…

In today’s lecture…

Probability Counting methods- Permutations &

Combinations Independence Non-independence/Bayes’ Rule

Example: Prostate Cancer Study

Thompson et al. (2006) Prostate Specific Antigen (PSA) evaluation leads

to early detection of prostate cancer Study looked at 5519 men who underwent

prostate biopsy Characteristics looked at: age, race, history of

prostate cancer, previous biopsies/screening

Table 1:Racial characteristics of study participants (n=5519)

n %

Race

White 5310 96.2 ~ 96

Black 209 3.8~ 4

Table 2: Number of prostate cancers and high grade prostate cancers

Probability: Chance of something happening (from 0-1) 0: cannot happen 1: sure to happen

P(A) = probability that event “A” will occurP(PSA 0-1) = probability that PSA level is from 0-1

P(~A) = probability that event “A” will NOT occur [complement]P(~PSA 0-1)= probability that PSA level is NOT from 0-1

P(A & B) = the probability that both A and B happen [joint probability]P(PSA 0-1 & white) = the probability of being a white male with PSA 0-1

P(A|B) = the probability that A occurs, given that B occurred[conditional probability]P(PSA 0-1|white) = the probability that PSA is 0-1, given that the patient is white

*Sainani K., Stanford

A B

P(A&B)

A B

P(A/B)

Assessing Probability1. Theoretical/Classical probability—based on theory (a priori

understanding of a phenomena)e.g.: theoretical probability of rolling a 2 on a standard die is 1/6

theoretical probability of choosing an ace from a standard deck is 4/52 theoretical probability of getting heads on a regular coin is 1/2

2. Empirical probability—based on empirical datae.g.: you toss an irregular die (probabilities unknown) 100 times and find that

you get a 2 twenty-five times; empirical probability of rolling a 2 is 1/4empirical probability of an Earthquake in Bay Area by 2032 is .62 (based on historical data)empirical probability of a lifetime smoker developing lung cancer is 15 percent (based on empirical data)

*Sainani K., Stanford

Computing theoretical probabilities:counting methods

Great for gambling! Fun to compute!

If outcomes are equally likely to occur…

outcomes of # total

occurcan A waysof #)( AP

Note: these are called “counting methods” because we have to count the number of ways A can occur and the number of total possible outcomes.

*Sainani K., Stanford

Applying our example…P (PSA level 0-1) = (# cases with PSA 0-1)

(total number of cases) = (1963)/(5519) = 0.35

P (PSA level >6) = (# cases with PSA >6) (total number of cases) = (150)/(5519) = 0.03

You randomly pick a patient to test his PSA. What’s the probability that he is white?

P(white) = (# cases who are white) (total number of cases) = (5310)/(5519) = 0.96

…that he is black?P(black) = (# cases who are black)

(total number of cases) = (209)/(5519) = 0.04

Example 2What’s the probability that you pick two patients who are black?

P(1st patient black) = (# cases who are black) (total number of cases) = (209)/(5519) = 0.038 ~ 0.04

P(2nd patient black) = (# cases who are black) (total number of cases) = (208)/(5518) = 0.037 ~0.04

P(black & black) = P(1st patient black) x P(2nd patient black) = 0.0016

This is an example of joint probability…more on this coming up!

Example 3If you have 5 patients (3 white, 2 black), and you want to

test PSA of two randomly chosen patients, what’s the probability that they are white (W) and black (B)?

Considering order of picking,P (1B, 1W patient) = # ways to pick one B, one W pair

# total patient pairs

Numerator = W1B1 W1B2 W2B1 W2B2 W3B1 W3B2

B1W1 B2W1 B1W2 B2W2 B1W3 B2W3 = 12

Denominator = 5x4 = 20

P(1B, 1W) = 12/20 = 0.6

 5 patients 4 patients

Applying our PSA example, using a probability tree…

P(BB=.04)

P(WW=.96)

First pick

P(BB=.04)

P(W=.96)

P(BB=.04)

P(WW=.96)

Second pick Outcome

P(WB)=0.04*0.96 = 0.038

P(BBB)=0.04*0.04 = 0.0016

P(BBW)=0.04*0.96 = 0.038

P(WW)=0.96*0.96 = 0.922

Rule of thumb: in probability, “and” means multiply, “or” means add

P(B)=0.04, From our example 2

P(1B,1W) = P(BW) +P(WB)

= 0.038 + 0.038

= 0.076

Ignoring order of picking:

P (1B,1W patient) = (# ways to pick one B, one W ) (total # ways to pick 2 patients)

Numerator = W1B1 W1B2 W2B1 W2B2 W3B1 W3B2 = 6

Denominator = (5x4)/2

P (picking a B,W patient) = 6 = 12 = 0.6(5x4)/2 20

We divide out the order, by dividing by 2 here

Summary of Counting Methods

Counting methods for computing probabilities

With replacement

Without replacement

Permutations—order matters!

Combinations—Order doesn’t

matter

Without replacement

*Sainani K., Stanford

Permutations—Order matters!A permutation is an ordered arrangement of objects.

With replacement = once an event occurs, it can occur again (after you roll a 6, you can roll a 6 again on the same die).

Without replacement = an event cannot repeat (after you draw an ace of spades out of a deck, there is 0 probability of getting it again).

*Sainani K., Stanford

Permutations with replacement Sample space: the set of all possible outcomes.

Example: in genetics, if both the mother and father carry one copy of a recessive disease-causing mutation (d), there are three possible outcomes (the sample space): child is not a carrier (DD) child is a carrier (Dd) child has the disease (dd).

Probabilities: the likelihood of each of the possible outcomes (always 0 P 1.0). P(genotype=DD)=.25 P(genotype=Dd)=.50 P(genotype=dd)=.25.

*Sainani K., Stanford

Summary: order matters, with replacement

Formally, “order matters” and “with replacement” use powers

Equation for total number of possible outcomes:

revents of # the n event)per outcomes possible (#

*Sainani K., Stanford

Example 1:

P(♀♀D=.5)

P(♀♀d=.5)

Mother’s allele

P(♂♂D=.5)

P(♂♂d=.5)

P(♂♂D=.5)

P(♂♂d=.5)

Father’s allele

______________ 1.0

P(DD) =.5*.5 = .25

P(Dd) = .5*.5 = .25

P(dD) = .5*.5 = .25

P(dd) = .5*.5 = .25

Child’s outcome

What’s the chance of having a child with the disease(dd) if both parents are heterozygote (Dd)?

P(dd) = 1 way to get (dd) = 1/4 = 0.25

22 possible outcomes *Sainani K., Stanford

Permutations without replacementExample 1:Suppose you want to test PSA levels of 4 patients:

A,B,C,D.How many ways can you test them?

A B C DB A C DC B A DD B C A.. # permutations = 4x3x2x1 = 4! = 24..

OR

Reminder!

Factorial notation:

n! =n x (n-1) x (n-2) x……….x1

AB

C

D

A

BC

D

DC

D

C

So there are 4! ways of doing 4 tests for 4 patients

Example 2: What if you had 3 different tests and 5 people?

345 xx

E

BA

C

D

E

AB

D

AB

C

D

Test 1:5 possible

Test 2:Only 4 possible

E

BD

Test 3:

only 3 possible

!2

!5

12

12345

x

xxxx

)!35(

!5

*Sainani K., Stanford

Summary: order matters, without replacement

Formally, “order matters” and “without replacement” use factorials

)1)...(2)(1(or

)!(

!

draws)!or chairs cardsor people (

cards)!or people (

rnnnn

rn

n

rn

n

Note: This formula also worked for Example 1. We were picking 4 people for 4 spots. So,

4!/ (4-4)! = 4!/0! = 4! = 24

*Sainani K., Stanford

Recall Permutation Theory… If you want to see if there is a difference between

the mean PSA scores for black ( n=209) and white patients (n=5310) in PSA example:

1. Calculate mean scores of black patients & white patients

2. Shuffle scores of 5000 random patients, 3. Number of possible permutations of shuffling

are: (5519!) = A huge number of permutations(5519-5000)!4. Compare original mean scores to mean scores

of each permutation.

2. Combinations—Order doesn’t matter

A combination helps determine the number of ways “r” objects can be chosen from “n” larger group of objects

Introduction to combination function, or “choosing”

n

rrn C or

Spoken: “n choose r”

Written as:

*Sainani K., Stanford

Example of combinationsIf you have 3 identical tests. What are the #

of ways you can choose 3 out of 5 patients, to be tested?

= 5C3 = 5! = 5 x 4 = 10

3! (5-3)! 2

Example: Distinct vs. Nondistinct objectsSuppose you want to calculate mean PSA scores of 4patients (3 white, 1black): A (White), B (White), C (White),& D (Black). How many ways can we arrange the 4 patientsbased on race?

Total number of arrangements of 4 people = 4! = 24

However, based on race, 3 of them are identical (White- A, B, C) and 1 of them is identical (Black- D).

If you only consider race of the patients, there will be fewer arrangements possible…

For example:arrangement A B C D (W W W B) = arrangement C B A D (W W W B)

In fact, the arrangement (W W W B) can be done in 6 distinct ways:

A B C DA C B DB C A DB A C DC A B DC B A D

= 3! permutations of white patients x 1 permutation of the black patient

= 6 x1 = 6

This is one race based arrangement.

Similarly, the arrangement (W W B W) can be done in 3!x1!=6 ways :

A B D C

A C D B

B C D A

B A D C

C A D B

C B D A

Since we don’t care about order, 4! ways of arranging the 4patients is reduced to:

4! = 4! = 4 3! 1! 6

Hence, number of ways of arranging n objects, of which k are white and m are black:

= n! k! m!

( 1. White or black are just examples of being nondistinct 2. Can be extended to any number of nondistinct sets)

This is also a “choosing” problem since we are choosing 3 tests for white patients & 1 for the black patient:

4C3 = 4C1

= 4! (3!)(1!) = 4

Summary: combinations

If r objects are taken from a set of n objects without replacement and disregarding order, how many different samples are possible?

Formally, “order doesn’t matter” and “without replacement” use choosing

!)!(

!

rrn

nn

r

*Sainani K., Stanford

Summary of Counting MethodsCounting methods for computing probabilities

With replacement: nr

Permutations—order matters!

Without replacement:n(n-1)(n-2)…(n-r+1)=

Combinations—Order doesn’t

matter

Without replacement:

)!(

!

rn

n

!)!(

!

rrn

nn

r

*Sainani K., Stanford

IndependenceFormal definition: A and B are independent if and only if

P(A&B)=P(A)*P(B)

Going back to our Genetics example:The mother’s and father’s alleles are segregating

independently.P(♂D|♀D)=.5 and P(♂D|♀d)=.5

What father’s gamete looks like is not dependent on the mother’s –doesn’t depend which branch you start on!

Formally, P(DD)=.25=P(D♂)*P(D♀)

Conditional Probability: Read as “the probability that the father passes a D allele given that the mother passes a d allele.”Joint Probability: The probability

of two events happening simultaneously.

Marginal probability: This is the probability that an event happens at all, ignoring all other outcomes.

*Sainani K., Stanford

On the tree

P(♀♀D = .5)

P(♀♀d=.5)

Mother’s allele

P(♂♂D/ ♀♀D )=.5

P(♂♂d=.5)

P(♂♂D=.5)

P(♂♂d=.5)

Father’s allele

______________ 1.0

P(DD)=.5*.5=.25

P(Dd)=.5*.5=.25

P(dD)=.5*.5=.25

P(dd)=.5*.5=.25

Child’s outcome

Conditional probabilityMarginal probability: mother

Joint probability

Marginal probability: father*Sainani K., Stanford

Independent mutually exclusive Events A and ~A are mutually exclusive, but

they are NOT independent. P(A&~A)= 0 P(A)*P(~A) 0

Conceptually, once A has happened, ~A is impossible; thus, they are completely dependent.

*Sainani K., Stanford

Practice problem

If HIV has a prevalence of 3% in San Francisco, and a particular HIV test has a false positive rate of .001 and a false negative rate of .01, what is the probability that a random person selected off the street will test positive?

Answer

______________ 1.0

P (+, test +)=.0297

P(+, test -)=.003

P(-, test +)=.00097

P(-, test -) = .96903

P(test +)=.0297+.00097=.03067

Marginal probability of carrying the virus.

Joint probability of being + and testing +

P(+&test+)P(+)*P(test+)

.0297 .03*.03067 (=.00092)

Dependent!

Marginal probability of testing positive

Conditional probability: the probability of testing + given that a person is +

P(+)=.03

P(-)=.97

P(test +)=.99

P(test - )= .01

P(test +) = .001

P(test -) = .999

*Sainani K., Stanford

Law of total probability

)P(HIV-)HIV|P(test ))P(HIVHIV|P(test )P(test

.97)(001.)03(.99.)P(test

One of these has to be true (mutually exclusive, collectively exhaustive). They sum to 1.0.

*Sainani K., Stanford

Law of total probability Formal Rule: Marginal probability for event A=

)P(B)B|P(A)P(B)B|P(A)P(B)B|P(A P(A) kk2211

exclusive)(mutually 0) and 0.11

ji

k

ii &BP(BB

B2

B3 B1

Where:

%25%)25%)(50()%50)((0(50%)(25%) P(A)

A

*Sainani K., Stanford

Non independent events/Conditional Probability

When two events are not independent, the occurrence of one event depends on whether the other has occurred

Bayes’ Rule

Bayes’ Rule

Definition:

Let A and B be two events with P(B) 0. The conditional probability of A given B is:

)(

)&()|(

BP

BAPBAP

*Sainani K., Stanford

Bayes’ Rule:

)(

)()/()/(

BP

APABPBAP

From the “Law of Total Probability”

OR

)(~)~/()()/(

)()/()/(

APABPAPABP

APABPBAP

*Sainani K., Stanford

In-Class Exercise

If HIV has a prevalence of 3% in San Francisco, and a particular HIV test has a false positive rate of .001 and a false negative rate of .01, what is the probability that a random person who tests positive is actually infected (also known as “positive predictive value”)?

*Sainani K., Stanford

Answer: using probability tree

______________ 1.0

P(test +)=.99

P(+)=.03

P(-)=.97

P(test - = .01)

P(test +) = .001

P (+, test +)=.0297

P(+, test -)=.003

P(-, test +)=.00097

P(-, test -) = .96903P(test -) = .999

A positive test places one on either of the two “test +” branches. But only the top branch also fulfills the event “true infection.” Therefore, the probability of being infected is the probability of being on the top branch given that you are on one of the two circled branches above.

%8.9600097.0297.

0297.

)(

)&()|(

testP

truetestPtestP

*Sainani K., Stanford

Answer: using Bayes’ rule 

%8.96)97(.001.)03(.99.

)03(.99.

)()|()()|(

)()|()|(

truePtruetestPtruePtruetestP

truePtruetestPtesttrueP

*Sainani K., Stanford

Conditional probability in epidemiology: Odds and Risk (probability)Definitions:Risk = P(A) = cumulative probability (you specify the time period!)

For example, what’s the probability that a person with a high sugar intake develops diabetes in 1 year, 5 years, or over a lifetime?

Odds = P(A)|P(~A)

For example, “the odds are 3 to 1 against a horse” means that the horse has a 25% probability of winning.

Note: An odds is always higher than its corresponding probability, unless the probability is 100%.

*Sainani K., Stanford

Introduction to the 2x2 Table

  Exposure (E) No Exposure (~E)

Disease (D) a b a+b = P(D)

No Disease (~D) c d c+d = P(~D)

  a+c = P(E) b+d = P(~E)

Marginal probability of disease

Marginal probability of exposure

*Sainani K., Stanford

Coming soon…(Applications of today’s lecture) More on odds ratios, risk ratios Patterns of categorical data/distributions Frequency tables Chi square Logistic regression Kaplan Meier, survival analysis

Special thanks to Dr. Cobb for her great slides from last year!