probability distributions - cs.ioc.ee · ioc.pdf next section 1 probability densit,y distribution 2...

ioc.pdf

PROBABILITY DISTRIBUTIONS

David M. Lane. et al. Introduction to Statistics : Chapter 9

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 1 / 18

ioc.pdf

Contents

1 Probability density, distribution

2 Probability distributions

Binomial distribution

Multinomial distribution

Normal distribution

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 2 / 18

ioc.pdf

Next section

1 Probability density, distribution

2 Probability distributions

Binomial distribution

Multinomial distribution

Normal distribution

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 3 / 18

ioc.pdf

Probability densityDe�nition

A probability density function or density of a continuous random variable, is a function thatdescribes the relative likelihood for this random variable to take on a given value.

The analogue for discrete random variable:

De�nition

A probability mass function is a function that gives the probability that a discrete randomvariable is exactly equal to some value.

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 4 / 18

ioc.pdf

Probability as area under density curve

Any continuous probability distribution, areas under the density curve representprobabilities.

For any numbers a and b, the probability P(a< x < b) equals the area under the curvebetween a and b;

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 5 / 18

ioc.pdf

Next section

1 Probability density, distribution

2 Probability distributions

Binomial distribution

Multinomial distribution

Normal distribution

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 6 / 18

ioc.pdf

Next subsection

1 Probability density, distribution

2 Probability distributions

Binomial distribution

Multinomial distribution

Normal distribution

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 7 / 18

ioc.pdf

Bernoulli trials

Independent repeated trials of an experiment with exactly two possibleoutcomes are called Bernoulli trials.

Call one of the outcomes "success" and the other outcome "failure".Let p be the probability of success in a Bernoulli trial, and q = 1−p bethe probability of failure: "success" and "failure" are mutually exclusiveand exhaustive.

For example, a series of n trials consisting of m "successes" (event A) and n−m "failures"(event A) is a sum of elementary events:

B = A∩A∩ . . .∩A∩ A∩ A∩ . . .∩ A = AmAn−m

Due to independence of these events, the probability of one such series is:

P(B) = P(A) ·P(A) · . . . ·P(A) ·P(A) ·P(A) · . . . ·P(A) = pmqn−m

The event B is equally likely to the series of n trials containing m "successes" in di�erent order.There are as many such series as many possibilities to choose �positions� for "successes", that isin

(nm

)ways. According to the sum rule of independent events we obtain the Bernoulli's formula:

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 8 / 18

ioc.pdf

Bernoulli's formula (also called BinomialProbability Formula)

Theorem

The probability Pm,n of achieving exactly m "successes" in n trials is

Pm,n =

(n

m

)pmqn−m =

n!

m!(n−m)!pmqn−m

where:

p is a probability of "success" in one trial;

q = 1−p is a probability of "failure" in one trial.

Example: We toss a coin 10 times. What is the probability we get 4 heads?

Here p = q = 0,5 and according to the Bernoulli's formula

P4,10 =10!

4!6!

(1

2

)4(1

2

)6

≈ 0,205

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 9 / 18

ioc.pdf

Bernoulli's formula; example 2

We toss a coin 12 times. What is the probability we get 0 to 3 heads?

P = P0,12 +P1,12 +P2,12 +P3,12 =

=12!

0!12!

(1

2

)0(1

2

)12

+12!

1!11!

(1

2

)1(1

2

)11

+12!

2!10!

(1

2

)2(1

2

)10

+12!

3!9!

(1

2

)3(1

2

)9

=

=

(1

2

)12 [1+12+

12 ·112

+12 ·11 ·10

6

]=

=299

4096≈ 0,073

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 10 / 18

ioc.pdf

Bernoulli's formula; example 3

Students are taking a 40 question multiple choice test. Each question has four choices. Astudent who had no time to prepare is going to guess all answers at random. What ismore likely: he will answer all questions correctly or he will give no right answers?

The probability that all choices are correct:

P40,40 =

(40

40

)·(1

4

)40

= 0,2540

and the probabilit that all answers are wrong:

P0,40 =

(40

0

)·(3

4

)40

= 0,7540.

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 11 / 18

ioc.pdf

Binomial Distribution B(n,p)

Probability mass function:

f (x) =

(n

x

)pxqn−x =

(n

x

)px

(1−p)n−x,

where x = 0,1,2, . . . ,n.Mean: µ = np.

Variance: σ2 = np(1−p).

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 12 / 18

ioc.pdf

Binomial Distribution B(n,p)

Probability mass function:

f (x) =

(n

x

)pxqn−x =

(n

x

)px

(1−p)n−x,

where x = 0,1,2, . . . ,n.Mean: µ = np.

Variance: σ2 = np(1−p).

Normal approximation:

N (np,np(1−p))

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 12 / 18

ioc.pdf

Next subsection

1 Probability density, distribution

2 Probability distributions

Binomial distribution

Multinomial distribution

Normal distribution

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 13 / 18

ioc.pdf

Multinomial Distribution

The multinomial distribution is a generalization of the binomial distribution.

For n independent trials each of which leads to a success for exactly one of k categories,with each category having a given �xed success probability, the multinomial distributiongives the probability of any particular combination of numbers of successes for the variouscategories.

When n = 1 and k = 2 the multinomial distribution is the Bernoulli distribution.

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 14 / 18

ioc.pdf

Multinomial Distribution

The multinomial distribution is a generalization of the binomial distribution.

For n independent trials each of which leads to a success for exactly one of k categories,with each category having a given �xed success probability, the multinomial distributiongives the probability of any particular combination of numbers of successes for the variouscategories.

When n = 1 and k = 2 the multinomial distribution is the Bernoulli distribution.

Example

For example, suppose that two chess players had played numerous games and it was determinedthat the probability that Player A would win is 0.40, the probability that Player B would win is0.35, and the probability that the game would end in a draw is 0.25. The multinomialdistribution can be used to answer questions such as: �If these two chess players played 12games, what is the probability that Player A would win 7 games, Player B would win 2 games,and the remaining 3 games would be drawn?�

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 14 / 18

ioc.pdf

Multinomial Distribution

The multinomial distribution is a generalization of the binomial distribution.

For n independent trials each of which leads to a success for exactly one of k categories,with each category having a given �xed success probability, the multinomial distributiongives the probability of any particular combination of numbers of successes for the variouscategories.

When n = 1 and k = 2 the multinomial distribution is the Bernoulli distribution.

Example

For example, suppose that two chess players had played numerous games and it was determinedthat the probability that Player A would win is 0.40, the probability that Player B would win is0.35, and the probability that the game would end in a draw is 0.25. The multinomialdistribution can be used to answer questions such as: �If these two chess players played 12games, what is the probability that Player A would win 7 games, Player B would win 2 games,and the remaining 3 games would be drawn?�

Probability for this case:

p =n!

(n1!)(n2!)(n3!)pn11pn22pn33

where n is the total number of eventsni is the number of times Outcome i occurs,p1 is the probability of Outcome i .

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 14 / 18

ioc.pdf

Multinomial Distribution

The multinomial distribution is a generalization of the binomial distribution.

For n independent trials each of which leads to a success for exactly one of k categories,with each category having a given �xed success probability, the multinomial distributiongives the probability of any particular combination of numbers of successes for the variouscategories.

When n = 1 and k = 2 the multinomial distribution is the Bernoulli distribution.

Example

For example, suppose that two chess players had played numerous games and it was determinedthat the probability that Player A would win is 0.40, the probability that Player B would win is0.35, and the probability that the game would end in a draw is 0.25. The multinomialdistribution can be used to answer questions such as: �If these two chess players played 12games, what is the probability that Player A would win 7 games, Player B would win 2 games,and the remaining 3 games would be drawn?�

For this example:

p =12!

(7!)(2!)(3!)0.470.3520.253 = 0.0248

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 14 / 18

ioc.pdf

Multinomial Distribution

The multinomial distribution is a generalization of the binomial distribution.

For n independent trials each of which leads to a success for exactly one of k categories,with each category having a given �xed success probability, the multinomial distributiongives the probability of any particular combination of numbers of successes for the variouscategories.

When n = 1 and k = 2 the multinomial distribution is the Bernoulli distribution.

Example

For example, suppose that two chess players had played numerous games and it was determinedthat the probability that Player A would win is 0.40, the probability that Player B would win is0.35, and the probability that the game would end in a draw is 0.25. The multinomialdistribution can be used to answer questions such as: �If these two chess players played 12games, what is the probability that Player A would win 7 games, Player B would win 2 games,and the remaining 3 games would be drawn?�

Formula for k outcomes:

p =n!

(n1!)(n2!) · · ·(nk !)pn11pn22· · ·pnkk

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 14 / 18

ioc.pdf

Next subsection

1 Probability density, distribution

2 Probability distributions

Binomial distribution

Multinomial distribution

Normal distribution

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 15 / 18

ioc.pdf

Introduction to Normal DistributionsThe normal distribution is the most important and most widely used distribution. It is alsocalled the �bell curve,� or �Gaussian curve�.

The density of the normal distribution is given by the function

f (x) =1√2πσ2

e−(x−µ)2

2σ2

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 16 / 18

ioc.pdf

Features of normal distributions

1 Normal distributions are symmetric around their mean.

2 The mean, median, and mode of a normal distribution are equal.

3 The area under the normal curve is equal to 1.0.

4 Normal distributions are denser in the center and less dense in the tails.

5 Normal distributions are de�ned by two parameters, the mean (µ) and the standarddeviation (σ).

6 68% of the area of a normal distribution is within one standard deviation of the mean.

7 Approximately 95% of the area of a normal distribution is within two standard deviationsof the mean

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 17 / 18

ioc.pdf

Central Limit Theorem

The normal distribution is important because of the following

Theorem (Central Limit Theorem)

Given a population with a �nite mean µ and a �nite non- zero variance σ2, the sampling

distribution of the mean approaches a normal distribution with a mean of µ and a variance of

σ2/N as N, the sample size, increases.

Most statistical procedures for testing di�erences between means assume extremelyimportant central limit theorem, the topic of a later section of this normal distributions.Because the distribution of means is very close to normal,

Quételet was the �rst to apply the normal distribution to human characteristics. He notedthat characteristics such as height, weight, and strength were normally distributed.

margarita.spitsakova@ttu.ee ICY0006: Lecture 6 18 / 18