lecture 1. uncertain events and probability 2020

Lecture1. Uncertain events and probability

2020

1 Uncertain outcomes

Consider an experiment with an uncertain outcome.

Example: I toss a coin twice, each time notingwhether it lands heads (H) or tails (T).

The four possible outcomes are: HH, HT, TH, TT.

2 Events

An event represents a set of outcomes. For example,in the coin-tossing experiment,

Event Outcomes

First throw gives a head: HH, HTSame result on both throws: HH, TTAt least one head: HT, TH, HHNo heads: TT

Mutually exclusive and exhaustive events:

Event no heads one head two headsOutcomes TT HT, TH HH

3 Combining events

A _ B is the event ‘either A or B occurs or both’

A ^ B is the event ‘both A and B occur’.

For example, in the coin-tossing experiment, take

A = (HH, HT) ‘˛rst toss is H’,B = (HT, TH) ‘one H, one T’,

Then

A _ B = (HH, HT, TH) ‘at most one T’A ^ B = (HT) ‘H then T’

4 A Venn diagram

HH

TH

HT

TT

5 Probability from randomisation

A box contains R red balls and B black balls. Oneball is selected from the box at random. What is theprobability that the ball is red?

The probability of an event is the sum of probabilitiesof outcomes associated with the event.

Probability that any one ball is selected is 1=(R+B).

Probability that selected ball is red is the sum of Rprobabilities, each equal to 1=(R + B), so

Pr(selected ball is red) =R

(R + B)

6 Probability from symmetry

If the coin is fair, symmetry between head and tailsuggests that all four outcomes of the coin-tossingexperiment are equally probable.

The probabilities of getting 0, 1, or 2 heads are then1/4, 1/2, and 1/4.

Note that it would be wrong to argue that there are 3possible values for the number of heads and they areall equally probable.

7 Probability of A _ B

For mutually exclusive events A and B,

Pr(A _ B) = Pr(A) + Pr(B)

In general,

Pr(A _ B) = Pr(A) + Pr(B)` Pr(AB)

For three events, Pr(A _ B _ C) is

Pr(A) + Pr(B) + Pr(C)

`Pr(AB)` Pr(AC)` Pr(BC) + Pr(ABC)

(writing AB for A ^ B, etc.)

8 Sampling without replacement

A population consists of n objects. A sample of size sis selected without replacement. How many di¸erentsamples can be drawn?

If we distinguish between samples which comprise thesame objects, but are selected in a di¸erent order, theanswer is

n(n` 1) ´ ´ ´ (n` s+ 1)

For the case s = n, the answer is n!

(n factorial, the number of ways of arranging nobjects).

9 Sampling without replacement

Usually we regard two samples as identical if theydi¸er only in the order in which the objects wereselected.

What is the total number of samples in this case?

Samples previously regarded as di¸erent now form anumber of groups each of size s!, with samples in thesame group regarded as identical.

The number of groups is

„ns

«=n(n` 1) ´ ´ ´ (n` s+ 1)

s!

This is ‘n choose s’, the number of ways of choosings objects from n.

10 Example

A box contains 4 balls numbered 1 to 4. Select twoballs without replacement.

´ ´ ´ (1,2) (1,3) (1,4)(2,1) ´ ´ ´ (2,3) (2,4)(3,1) (3,2) ´ ´ ´ (3,4)(4,1) (4,2) (4,3) ´ ´ ´

If we take account of ordering, there are 12 simpleoutcomes. If we disregard ordering, there are only six:

(1,2), (1,3), (1,4), (2,3), (2,4), (3,4),

where, e.g., (1,2) now represents either (1,2) or (2,1).

11 A property of ’choose’ numbers

A box contains 5 balls numbered 1 to 5. Select twoballs (without replacement).

How many samples of size 2?Answer: (5ˆ 4)=2 = 10.

How many samples of size 3?Answer: (5ˆ 4ˆ 3)=(3ˆ 2) = 10.

Why are these numbers the same?

Each selection of two balls to be included is also aselection of three balls to be excluded from thesample.

12 Binomial expansion

(p+ q)n = (p+ q)(p+ q) ´ ´ ´ (p+ q) (n terms)

The result is a sum of terms of the form psqn`s. Themultiplier for this term is the number of ways ofchoosing s terms from n, so that

(p+ q)n =nXs=0

„ns

«psqn`s

13 Pascal’s triangle

11 1

1 2 11 3 3 1

1 4 6 4 11 5 10 10 5 1

. . . and so on . . .

END OF LECTURE

Lecture2. Random Variables

2020

14 Random variables

A random variable is a numerical summary of eachoutcome.

Example: a ‘fair’ coin is tossed twice. We assignequal probabilities of 1/4 to each outcome.

Consider the mutually exclusive and exhaustive events‘no heads’, ’one head’, and ‘two heads’.

No. of heads 0 1 2Probability 1/4 1/2 1/4

The number of heads is a random variable. Theprobability distn is given in the table above.

15 Scottish soldiers

Chest sizes (inches) for 5738 Scottish soldiers:

33 34 35 36 ´ ´ ´ 46 47 48 Total3 18 81 185 ´ ´ ´ 21 4 1 5738

One individual is chosen randomly from thepopulation.

Let X be the chest size of the selected individual.

X is a random variable: its value is known only afterselection.

16 Scottish soldiers

Before selection, all that can be said about X withcertainty is that its value will be one of the values 33,34, etc. However, probabilities can be assigned:

Pr(X = 33) = 3=5738; Pr(X = 34) = 18=5738; : : :

The values 33, 34, . . . , 48 and correspondingprobabilities de˛ne the probability distn of X.Obviously, these probabilities add up to 1.

17 A probability distribution

0 4 8 12 160.00

0.05

0.10

0.15

0.20

The number of heads obtained when a fair coin istossed 16 times.

18 Cumulative probability function

0 4 8 12 16

0.0

0.2

0.4

0.6

0.8

1.0

Cumulative probabilities for the number of headsobtained when a fair coin is tossed 16 times.

19 Expectation

X takes values x1, x2, . . . with probabilities p1, p2,. . . The ‘expectation’ of X is

E(X) = p1x1 + p2x2 + ´ ´ ´

You pay me 1 dollar (the stake). A fair coin is tossed.If it falls heads, I return the stake and pay you anextra dollar. If tails, I keep the stake and pay nothing.Is this a fair game?

Your return is a random variable, determined by thetoss of the coin. Your ’expected’ return is2ˆ 0:5 + 0ˆ 0:5 = 1 dollar, exactly equal to thestake. The game is fair.

20 Mean and variance of a random variable

The mean value is another name for E(X). It is ameasure of location, somewhere near the ‘middle’ ofthe probability distn. On this course it will usually bedenoted by m.

The variance (ff2) is E(X `m)2, and the standarddeviation (ff) is the square root of the variance. Theseare measures of dispersion, or spread of the distn.

When X is derived by sampling from a population, mand ff2 are the mean and variance of the values in thepopulation.

An alternative formula for variance is E(X2)`m2.

21 Calculating population mean and variance

Variance depends on the spread of the x values, andalso on the probabilities. Variance is 0.5 in example 1,and 0.2 in example 2.

Example 1 Example 2x p xp x2p p xp x2p

0 0.25 0.00 0.00 0.10 0.00 0.001 0.50 0.50 0.50 0.80 0.80 0.802 0.25 0.50 1.00 0.10 0.20 0.40

Total 1.00 1.00 1.50 1.00 1.00 1.20

22 Two measures of shape

Skewness measures departure from symmetry. Apositive value indicates an extended right tail.

Kurtosis measures the thickness of the tails of adistribution. Large values indicates a heavy-taileddistribution.

A distribution can be both skew and kurtotic.

23 A skew distribution

0 4 8 12 160.00

0.05

0.10

0.15

0.20

Distn of the number of heads obtained when a biasedcoin (p = 1=4) is tossed 16 times.

END OF LECTURE

Lecture3. Conditional probability and

independence2020

24 Conditional probability

P (A jB) denotes the conditional probability of theevent A, given B. Note that

Pr(AB) = Pr(A jB) Pr(B)

The conditioning event B can have a strong in‚uenceon the probability. For example, the probability ofdying within a year is not the same for a 20-year oldand a 70-year old.

Often the unconditional probability applies to anentire population, and the conditional probabilityapplies to a sub-population.

25 Independent events

Events A and B are independent if P (AjB) = P (A),

or equivalently, if P (AB) = P (A)P (B).

Example: the probabilities that a coin lands ‘heads’ or‘tails’ are p and q (p+ q = 1). The coin is tossedtwice.

Independence between the outcomes of the ˛rst andsecond toss leads to the following probabilities:

Outcome TT HT TH HHProbability q2 pq pq p2

What is probability distn of the number of heads?

26 Independent events

The assumption of independence is often based onknowledge of the mechanism generating the randomevents. For example, the result of a coin toss doesnot depend on what happened previously: the coinhas no ‘memory’ of previous events.

Non-independence may be described as a statistical‘association’ between the events.

Positive association: Pr(AB)` Pr(A) Pr(B) > 0Negative association: Pr(AB)` Pr(A) Pr(B) < 0

27 Covariance

Random variables X and Y are independent if, for allpossible values a and b,

Pr(X = a; Y = b) = Pr(X = a) Pr(Y = b):

If X and Y are not independent, the degree ofdependence is measured by the covariance

cov(X; Y ) = E[(X `mX)(Y `mY )]

If X and Y are independent, cov(X; Y ) = 0.

Covariance a¸ects variance of the sum X + Y :

var(X + Y ) = var(X) + var(Y ) + 2 cov(X; Y )

28 Covariance

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

● ●

●

● ● ●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

29 Covariance

−5 0 5 10

0

100

200

300

400

30 Correlation coe‹cient

The correlation between X and Y is cov(X; Y ) dividedby the product of the standard deviations.

A covariance can take any value, but the correlationcoe‹cient is always less than 1 in magnitude. It isunchanged by change in scale of X or Y .

31 Genetic covariance

A positive covariance is often the result of a sharedrandom term.

For example, for measurements X, Y on two siblings,

X = m+ U + e 1; Y = m+ U + e 2;

where U, e 1, and e 2 are independent r.v.s.

In this case cov(X; Y ) = var(U).

(U represents genetic inheritance from shared parents.)

32 Law of total probability

A1 : : : Ak are mutually exclusive and exhaustiveevents. For any event B,

Pr(B) = Pr(BjA1) Pr(A1) + ´ ´ ´+ Pr(BjAk) Pr(Ak)

A population is one-third male and two-thirds female.Half the males and three in eight of the females areleft-handed. Proportion of the population which isleft-handed is

(1=2)(1=3) + (3=8)(2=3) = 1=6 + 1=4 = 5=12

33 Bayes formula

Under the same conditions as on the previous slide

Pr(AijB) = Pr(BjAi) Pr(Ai)=Pr(B)

where Pr(B) is calculated as on the previous slide:

Pr(B) = Pr(BjA1) Pr(A1) + ´ ´ ´+ Pr(BjAk) Pr(Ak)

34 Bayes formula

The table below shows the proportions of thepopulation for all four combinations of gender and’handedness’, with marginal totals.

Left Right Total

Male 1/6 1/6 1/3Female 1/4 5/12 2/3

Total 5/12 7/12 1

Among left-handed individuals, the proportion whichare male is

1=6

1=6 + 1=4= 2=5

END OF LECTURE