stochastic processes

Stochastic Processes - lesson 3

Bo Friis Nielsen

Institute of Mathematical Modelling

Technical University of Denmark

2800 Kgs. Lyngby – Denmark

Email: bfn@imm.dtu.dk

Bo Friis Nielsen – 12/9-2000 2C04141

OutlineOutline

• Discrete random variables (from last lesson)

OutlineOutline

� Geometric distribution

OutlineOutline

? Lack of memory property

OutlineOutline

• Continuous random variables

OutlineOutline

� Exponential distribution

OutlineOutline

� Moments for continuous random variables

OutlineOutline

� Moments for continuous random variables

• Reading recommendations

• Bernoulli distribution

� Two possibilities succes(X = 1)/failure(X = 0)

• Binomial distribution

� Number of succeses in a sequence of independent

Bernoulli trials

� pdf (Probability Density Function)

? f(x) = P{X = x} =

px(1− p)n−x

� Mean and variance

? E(X) = np V (X) = np(1− p)

Poisson distributionPoisson distribution

• Number of faults/accidents/occurrences

• X ∈ P(λ) ⇔ P{X = x} = f(x) = λx

x!e−λ

• E(X) = V (X) = λ

Geometric distributionGeometric distribution

• How many items produced to get one passing the quality

control

• Number of days to get rain

control

• Sequence of failures until the first success - sequence of

Bernoulli trials

control

Bernoulli trials

• Possible values

control

Bernoulli trials

• Possible values

� If we count all trials

control

Bernoulli trials

• Possible values

� If we count all trials 1, . . . ,∞

control

Bernoulli trials

• Possible values

� If we only count the failures

control

Bernoulli trials

• Possible values

� If we only count the failures 0, . . . ,∞

Derivation of probability density function -

counting all trials

• Let’s look at the sequence FFFFS

counting all trials

• Let’s look at the sequence FFFFS with probability

(1− p)4p

counting all trials

(1− p)4p

• A general sequence will be like FFF. . .FFS

counting all trials

(1− p)4p

• The probability of having x− 1 failures before the first

succes is

counting all trials

(1− p)4p

succes is

P{x trials} =

counting all trials

(1− p)4p

succes is

P{x trials} = P{x− 1 failures and then a succes} =

counting all trials

(1− p)4p

succes is

(1− p)x−1p

counting all trials

(1− p)4p

succes is

(1− p)x−1p

• The cumulative distribution can be found to be

� F (x) =∑x

t=1(1− p)t−1p = 1− (1− p)x

• E(X) = 1p

V (X) = 1−p

• E(X) = 1p

V (X) = 1−p

The expressions when counting only failuresThe expressions when counting only failures

• f(x) = (1− p)xp

• F (x) = 1− (1− p)x+1

• E(X) = 1−p

• V (X) = 1−p

The memoryless propertyThe memoryless property

• What will happen to the distribution knowing that n

failures already occured?

• That is we have been waiting for an empty cab and have

experienced 7 occupied

• Formally

P{X > x + n|X > n}

• Formally

P{X > x + n|X > n} = P{X>x+n∩X>n}

• Formally

P{X > x + n|X > n} = P{X>x+n∩X>n}P{X>n}

• Formally

= P{X>x+n}

• Formally

= P{X>x+n}P{X>n}

• Formally

= P{X>x+n}P{X>n}

=1−(1−(1−p)x+n)1−(1−(1−p)n)

• Formally

= P{X>x+n}P{X>n}

=1−(1−(1−p)x+n)1−(1−(1−p)n)

= (1−p)x+n

(1−p)n

• Formally

= P{X>x+n}P{X>n}

=1−(1−(1−p)x+n)1−(1−(1−p)n)

= (1−p)x+n

(1−p)n= (1− p)x

• Formally

= P{X>x+n}P{X>n}

=1−(1−(1−p)x+n)1−(1−(1−p)n)

= (1−p)x+n

(1−p)n= (1− p)x = P{X > x}

• Formally

= P{X>x+n}P{X>n}

=1−(1−(1−p)x+n)1−(1−(1−p)n)

= (1−p)x+n

(1−p)n= (1− p)x = P{X > x}

• That is, the probability of exceeding x + n having reached

n is the same as the property of exceeding x starting from

the beginning.

• Formally

= P{X>x+n}P{X>n}

=1−(1−(1−p)x+n)1−(1−(1−p)n)

= (1−p)x+n

(1−p)n= (1− p)x = P{X > x}

• That is, the probability of exceeding x + n having reached

n is the same as the property of exceeding x starting from

the beginning. In other words no aging

Continuous random variablesContinuous random variables

• Cumulative distribution function, once again

� P{X ≤ x} = F (x)

• The probability density function in the continuous case

� P{X ≤ x} = F (x)

� f(x) = F ′(x)

� P{X ≤ x} = F (x)

� f(x) = F ′(x)

� Probabilistic interpretation

� P{X ≤ x} = F (x)

� f(x) = F ′(x)

P{x ≤ X ≤ x + dx} = f(x)dx

� P{X ≤ x} = F (x)

� f(x) = F ′(x)

P{x ≤ X ≤ x + dx} = f(x)dx or F (x) =∫ x−∞ f(t)dt

The continuous parallel to the geometric

distribution

the exponential distribution

distribution

• F (x) = 1− e−λx

distribution

• F (x) = 1− e−λx ⇔ X ∈ exp(λ)

distribution

• F (x) = 1− e−λx ⇔ X ∈ exp(λ)

• f(x) = λe−λx

distribution

• F (x) = 1− e−λx ⇔ X ∈ exp(λ)

• The exponential is without memory like the geometric

distribution

• F (x) = 1− e−λx ⇔ X ∈ exp(λ)

• The exponential is without memory like the geometric

distribution

• The geometric/exponential distributions are the unique

memoryless distributions

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

Exponential density with mean=1

’exppdf.lst’

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

Exponential density with mean=1

’exppdf.lst’

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

Exponential cumulative distribution with mean=1

’expcdf.lst’

The normal densityThe normal density

-4 -3 -2 -1 0 1 2 3 4

Normal density with mean=0 and variance=1

’normpdf.lst’

Moments revisitedMoments revisited

• The mean for continuous random variables

� E(X) =∫∞−∞ xf(x)dx

� E(X) =∫∞−∞ xf(x)dx = µ

�∫∞0 xλe−λxdx

� E(X) =∫∞−∞ xf(x)dx = µ

�∫∞0 xλe−λxdx = 1

λby partial integration

� E(X) =∫∞−∞ xf(x)dx = µ

• The variance of continuous random variables

� E(X) =∫∞−∞ xf(x)dx = µ

� V (X) =∫∞−∞(x− µ)2f(x)dx

� E(X) =∫∞−∞ xf(x)dx = µ

� V (X) =∫∞−∞(x− µ)2f(x)dx = σ2

� E(X) =∫∞−∞ xf(x)dx = µ

� V (X) =∫∞−∞(x− µ)2f(x)dx = σ2

� X ∈ exp(λ) V (X) = 1λ2

Small examples using the exponential

distribution

distributionThe time between two consecutive attempts access attempts

to a popular web site can be adequately described by an

exponential distribution with mean 5 seconds.

distribution

• What is the variance of the time between two consecutive

access attempts?

distribution

access attempts?

• What is the probability that the time between two

consecutive attempts will be less than 1 second?

distribution

access attempts?

• What is the probability that the time between two

consecutive attempts will be less than 1 second?

• Knowing that no attempt has been made within the last

minute, what is the probability that there won’t be a new

attempt within the next second?

ModelModel

• X : the time between two consecutive attempts

ModelModel

• X ∈ exp(λ)

ModelModel

• X ∈ exp(λ)

• E(X) = 1λ

ModelModel

• X ∈ exp(λ)

• E(X) = 1λ

= 5s ⇒ λ = 0.2s−1

ModelModel

• X ∈ exp(λ)

• E(X) = 1λ

= 5s ⇒ λ = 0.2s−1

• X ∈ exp(0.2)

What is the variance of the time between

two consecutive access attempts?

• V (X) = 1λ2

• V (X) = 1λ2 = 0.04

To exponential distribution

What is the probability that the time

between two consecutive attempts will be

less than 1 second?

• P{X ≤ 1}

less than 1 second?

• P{X ≤ 1} = F (1) = 1− exp−0.2·1

less than 1 second?

• P{X ≤ 1} = F (1) = 1− exp−0.2·1 = 0.1813

Knowing that no attempt has been made

within the last minute, what is the

probability that there won’t be a new

• P{X > 61|X > 60}