Discrete random variables and probability distributions

random variable is a mapping from the sample space to realnumbers. notation: X, Y, Z,...Example: Ask a student whether she/he works part time or not.S= {Yes,No}.X=1 if Yes, X=0 if No. A random variable whose only possiblevalues are 0 and 1 is called a Bernoulli random variable.Y = number of car accidents in a week.Flip a coin three times. Let Z=number of heads in three flips.{TTT} → Z = 0{HTT ,THT ,TTH} → Z = 1{HHT ,HTH,THH} → Z = 2{HHH} → Z = 3.W = the weight of a randomly selected athlete.

discrete random variable: takes finite or countably infinite values.continuous random variable: takes all values in an interval orunion of intervals.

probability distribution or probability mass function (pmf) of adiscrete rv is defined byp(x) = P(X = x) = P(all s ∈ S: X (s) = x). Note p(x) ≥ 0 and∑

x p(x) = 1.Example: Suppose 20% of JMU students work part time.X = 1 if yes, X = 0 if no.p(0) = P(X = 0) = P(no) = 0.8p(1) = P(X = 1) = P(yes) = 0.2.

Example: Flip a fair coin 3 times.X = number of heads in 3 flips.p(0) = P(X = 0) = P{TTT} = 1/8,p(1) = P(X = 1) = P{HTT ,THT ,TTH} = 3/8,p(2) = P(X = 2) = P{HHT ,HTH,THH, } = 3/8,p(3) = P(X = 3) = P{HHH} = 1/8.

parameter of a distribution

Example : The probability of getting a head in each flip in p, thenp(0) = P(X = 0) = P{TTT} = (1− p)3,p(1) = P(X = 1) = P{HTT ,THT ,TTH} = 3 ∗ p ∗ (1− p)2,p(2) = P(X = 2) = P{HHT ,HTH,THH, } = 3 ∗ p2(1− p),p(3) = P(X = 3) = P{HHH} = p3.p is a parameter of this distribution.

CDF

The cumulative distribution function (cdf) F (x) of a discrete rvX with pmf p(x) is defined for any number x byF (x) = P(X ≤ x) =

∑y :y≤x p(y).

Example 1: The pmf of a rv X is given below:

x 1 2 3 4

p(x) 0.4 0.3 0.2 0.1

Note thatF (1) = P(X ≤ 1) = P(X = 1) = 0.4F (2) = P(X ≤ 2) = P(X = 1or2) = 0.4 + 0.3 = 0.7F (3) = P(X ≤ 3) = P(X = 1or2or3) = 0.4 + 0.3 + 0.2 = 0.9F (4) = P(X ≤ 4) = P(X = 1or2or3or4) = 0.4 + 0.3 + 0.2 + 0.1 =1.0

The cdf of X is

F (x) =

= 0, x < 1= 0.4, 1 ≤ x < 2= 0.7, 2 ≤ x < 3= 0.9, 3 ≤ x < 4= 1.0, x ≥ 4

plot of F (x):

Expected values

The expected value or the mean of a discrete rv X isE (X ) = µx =

∑x xp(x).

For example 1, E (X ) = 1 ∗ 0.4 + 2 ∗ 0.3 + 3 ∗ 0.2 + 4 ∗ 0.1 = 2.0

Expected value of a function of X , h(X ):E (h(X )) =

∑x h(x)p(x).

Example 1 continued: Let h(x) = 1x .

E (h(X )) = 1 ∗ 0.4 + 12 ∗ 0.3 + 1

3 ∗ 0.2 + 14 ∗ 0.1 = 0.641.

Proposition: E (aX + b) = aE (X ) + b.Proof: E (aX + b) =

∑x(ax + b)p(x) =

a∑

x xp(x) + b∑

x p(x) = aE (X ) + b.Two special cases of the proposition: For any constants a and b,E (aX ) = aE (X ),E (X + b) = E (X ) + b.

Variance of X

Let X have pmf p(x) with mean µ. The variance of X isV (X ) = σ2

X =∑

x(x − µ)2p(x) = E (X − µ)2 = E (X 2)− µ2.

The standard deviation of X is σX =√σ2

X .

Example 1: continued.E (X 2) = 12 ∗ 0.4 + 22 ∗ 0.3 + 32 ∗ 0.2 + 42 ∗ 0.1 = 5V (X ) = E (X 2)− µ2 = 5− 22 = 1.Rules of variance:V (h(X )) =

∑x [h(x)− E (h(x))]2p(x).

V (aX + b) = V (aX ) = a2V (X ).proof:V (aX ) = E (aX − aµ)2 = Ea2(X − µ)2 = a2E (X − µ)2 = a2V (X ).

Moments and moment generating functions

moments: expected values of X r or (X − µ)r , where r is aninteger.Moments (about 0): E (X r ). e.g., EX : first momentMoments about mean: E (X − µ)r .e.g.,E (X − µ)2 = V (X ).

The moment generating function (mgf) of a discrete rv isdefined to beMX (t) = E (etX ) =

∑x etxp(x). We say the mgf exists if it is

defined for t ∈ (−t0, t0) where t0 > 0.

compute mgf

Example: Let X be a Bernoulli rv with p(0)=1/3 and p(1)=2/3,thenMX (t) = E (etX ) =

∑x etxp(x) = et∗0p(0) + et∗1p(1) = 1

3 + et 23 .

proposition: If the mgf exists and is the same for twodistributions, then the two distributions are the same.

Theorem: If the mgf exists, E (X r ) = M(r)X (0), where M

(r)X (t) is

the rth derivative of MX (t).

Example: MX (t) = 0.4et + 0.3e2t + 0.2e3t + 0.1e4t .M ′X (t) = 0.4et + 0.6e2t + 0.6e3t + 0.4e4t ,and M ′X (0) = 0.4 + 0.6 + 0.6 + 0.4 = 2 = E (X ).M ′′X (t) = 0.4et + 1.2e2t + 1.8e3t + 1.6e4t .M ′′X (0) = 0.4 + 1.2 + 1.8 + 1.6 = 5 = E (X 2).so V (X ) = E (X 2)− µ2 = 5− 22 = 1.

The Binomial distribution

Binomial experiment:1. The experiment consists of n trials.2. Each trial has two possible outcomes, success or failure.3. The trials are independent.4. The probability of success, p, is constant from trial to trial.Binomial random variable X is defined to be the number ofsuccesses out of n trials. Note the possible vales X can take are 0,1, 2, ...n.

Find binomial probabilities

The multiple problem has 4 possible answers and only one of themis correct. If a student randomly guesses the answers for threeproblems, what is the probability that he can get two problemsright?n = 3, p = 0.25.The three ways he can get two problems right are CCI ,CIC , ICC .Each way has a probability 0.25 ∗ 0.25 ∗ 0.75 = 0.047. Theprobability of the three ways is: 3 ∗ 0.25 ∗ 0.25 ∗ 0.75 = 0.14.

Suppose he has to answer 5 problems. What is the probability thathe can get 3 problems right?One particular way he can get 3 right is CCCII , its probability is0.253 ∗ 0.752 = 0.0088.There are 10 ways he can get 3 right. The probability of these tenways is 10 ∗ 0.253 ∗ 0.752 = 0.09.

The formula for binomial distribution

P(X = x) = n!x!(n−x)!p

x(1− p)n−x , x = 0, 1, 2, ...n.

EX =∑

xp(x) = np,V (X ) = np(1− p).It is known that the 80% of patients who receive heart transplantsurgery will survive. A hospital just performed heart transplantsurgeries on 6 patients.Find the probability thata). All of them will survive.b). Exactly 4 of them will survive.

p(X = 6) = 0.86 = 0.262.p(X = 4) = 6!

4!2!0.840.22 = 0.246.

Binomial theorem:(a + b)n =

∑nx=0

(nx

)axbn−x .

mgf of X :MX (t) =

∑x etxp(x) =

∑nx=0 etx

(nx

)px(1− p)n−x =∑n

x=0

(nx

)(pet)x(1− p)n−x = (pet + 1− p)n.

Use mgf to verify that EX = np,V (X ) = np(1− p).

Geometric distribution

In Bernoulli repeated trials, some times we are interested in theX=number of trials needed to have the first success.A random variable is said to have a geometric distribution if itspmf is given byp(x ; p) = p(1− p)x−1, x = 1, 2, 3, · · · .E (X ) = 1

p ,V (X ) = 1−pp2 .

Example: A family decides to have children until it has a son.What the probability that the family will have 4 children ifP(B) = 0.51,P(G ) = 0.49?P(X = 4) = P(GGGB) = 0.493 ∗ 0.51 = 0.06.

Hypergeometric distribution

The population consists of N individuals/elements.Each element is either a Success (S) or Failure (F). There are Msuccesses in the population.A random sample of n elements is selected without replacement.X = number of S’s in the sample.e.g. A class consists of 10 female and 15 male students. Acommittee made of 4 students is to be randomly chosen from theclass.X = number of females in the committee. x = 0, 1, 2, 3, 4.

P(X = 0) =(15

4 )(25

4 )= 0.1079.

P(X = 2) =(10

2 )(152 )

(254 )

= 0.3735.

In general,

P(X = x) =(M

x )(N−Mn−x )

(Nn)

,max(0, n − N + M) ≤ x ≤ min(n,M).

E (X ) = nMN ,V (X ) = N−n

N−1nMN (1− M

N ).

The Poisson distribution

A random variable is said to have a Poisson distribution withparameter λ(λ > 0) if the pmf of X is

p(x ;λ) = e−λλx

x! , x = 0, 1, 2, 3, · · · .The mean and variance of a Poisson distribution:E (X ) = λ,V (X ) = λ.proposition: The binomial pmf approaches the Poisson pmf if welet n→∞, p → 0 in such a way that np → λ > 0.

Example: Let X be the number of creatures of a certain typecaptured in a trap during a day. Suppose X has a Poissondistribution with λ = 4.5. i.e., on average the trap will capture 4.5creatures a day. The probability that a trap capture 5 creatures ina day is

P(X = 5) = e−4.5(4.5)5

5! = 0.1708.The probability that the trap will capture 0 creatures is

P(X = 0) = e−4.5(4.5)0

0! = e−4.5 = 0.011.The probability that the trap will capture 12 creatures is

P(X = 20) = e−4.5(4.5)12

12! = 0.0016.

Approximate binomial probability

If 2 percent of the books bound at a certain factory have defectivebindings, find the probability the five of 400 books bound by thisfactory will have defective bindings.Poisson approximation:λ = 400 ∗ 0.2 = 8,P(X = 5) = e−885

5! = 0.093.

Binomial probability: n=400, p=0.02,P(X = 5) = 400!

5!395!0.025(0.98)395 = 0.091.

1. Let X be a discrete rv with P(X = −2) = P(X = 2) = p2 , and

P(X = 0) = 1− p, where p is a number between 0 and 1.Find E (X ),V (X ) and E (log(X + 3)).

E (log(X + 3)) =∑

log(x + 3)p(x) =log(−2+3)p

2 +log(2+3)p2 +log(0+3)∗(1−p) = p

2 log 5+(1−p) log 3.

2. An automobile safety engineer claims that 1 in 10 automobileaccidents is due to driver fatigue. If this is true, what is theprobability that at least 3 of 5 automobile accidents are due todriver fatigue?Let X be the number of auto accidents due to driver fatigueamong 5 auto accidents, then X has a binomial distribution withn = 5, p = 0.1.P(X ≥ 3) = P(X = 3) + P(X = 4) + P(X = 5) = 0.00856.

Exercise

Customers at a gas station pay with a credit card (A), debit card(B), or cash (C). Assume that successive customers makeindependent choices with P(A) = 0.5,P(B) = 0.2 and P(C ) = 0.3.1). Among the next 10 customers what is the probability thatexactly 4 will pay with a debit card?2). Among the next 100 customers, what are the mean andvariance of the number who pay with a debit card?

Exercise

Suppose small aircraft arrive at an airport according to a Poissonprocess with rate 8 per hour, so that the number of arrivals duringa time period of t hours is a Poisson rv with λ = 8t.1). What is the probability that exactly 6 small aircraft will arriveduring a 1 hour period?2). What are the expected value of the number of small aircraftthat arrive during a 90 min period?

Exercise

In testing of circuit boards, the probability that any particulardiode will fail is 0.01. A circuit board contains 200 diodes.1). How many diodes would you expect to fail on a circuit board?2). What is the probability that zero diodes will fail on a randomlyselected board?3). If five boards are shipped to a customer, what is the probabilitythat at least 4 of them work properly? ( A board works only if allits diodes work.)