EEE 461 1

Probability and Random Probability and Random VariablesVariables

Huseyin BilgekulEEE 461 Communication Systems II

Department of Electrical and Electronic Engineering Eastern Mediterranean University

Why Probability in Communications Probability Random Variables Probability Density Functions Cumulative Distribution Functions

EEE 461 2

Why probability in Why probability in Communications?Communications?

• Modeling effects of noise– quantization – Channel– Thermal

• What happens when noise and signal are filtered, mixed, etc?

• Making the “best” decision at the receiver

EEE 461 3

SignalsSignals• Two types of signals

– Deterministic – know everything with complete certainty

– Random – highly uncertain, perturbed with noise

• Which contains the most information? Information content is determined from the amount of uncertainty and unpredictability. There is no information in deterministic signals

Information = Uncertainty

x(t) y(t)

(t)

0 100 200 300 400 500 600 700 800 900 1000-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5y[n]x[n]

Let x(t) be a radio broadcast. How useful is it if x(t) is known? Noise is ubiquitous.

EEE 461 4

Need for Probabilistic Need for Probabilistic AnalysisAnalysis

• Consider a server process – e.g. internet packet switcher, HDTV frame decoder, bank teller line,

instant messenger video display, IP phone, multitasking operating system, hard disk drive controller, etc., etc.

Rejected customer, Queue full

Customers arrive at random times

Queue,Length L

Server:1 customer

per seconds

Satisfied customer

EEE 461 5

Probability DefinitionsProbability Definitions

• Random Experiment – outcome cannot be precisely predicted due to complexity

• Outcomes – results of random experiment• Events – sets of outcomes that meet a criteria,

roll of a die greater than 4• Sample Space – set of all possible outcomes,

EE (sometimes called the Universal Set)

EEE 461 6

ExampleExample• B={, , }• Complement

– BC={, , }• Union

• Intersection

• Null Set (), empty set

1

2

3

4 6

5

EEAo

B

Ae

1 3 4 5 6, , , ,oA B

5o oA B A B

EEE 461 7

Relative FrequencyRelative Frequency• nA – number of elements in a set, e.g. the

number of times an event occurs in N trials

• Probability is related to the relative frequency

• For N small, fraction varies a lot; usually gets better as N increases

Relative Frequency

lim Probability

0 1

0 Never Occurs

1 Always Occurs

A

A

n

nf A

nn

P An

P A

P A

P A

EEE 461 8

Joint ProbabilityJoint Probability• Some events occur together

– Sum of two dice is 6– Chance of drawing a pair of jacks

• Events can be – mutually exclusive (no intersection) – tossing a coin– Intersect and have common elements

• The probability of a JOINT EVENT, AB, is

lim Joint Probability

Let then

AB

n

nP AB

n

E A B

P E P A B P A P B P AB

EEE 461 9

Bayes Theorem and Independent Bayes Theorem and Independent EventsEvents

/ = / Bayes Theorem

/ Probability that A occurs given that B has occured

/ Probability that B occurs given that A has occured

P AB P A P B A P B P A B

P A B

P B A

1 2

1 2 1 2

Two events are INDEPENDENT if

/ =

/ =

If a set of events , , ...... are INDEPENDENT

, , ...... = ..... n

n n

P A B P A

P B A P B

A A A

P A A A P A P A P A

EEE 461 10

Axioms of ProbabilityAxioms of Probability

• Probability theory is based on 3axioms– P(A) >0– P(E) = 1– P(A+B) = P(A) + P(B) If P(AB) =

EEE 461 11

Random VariablesRandom Variables

• Definition: A real-valued random variable (RV) is a real-valued function defined on the events of the probability system

Event RV

Value

P(x)

A 3 0.2

B -2 0.5

C 0 0.1

D -1 0.2

E

A

DC

BP(x)

x3-1 0-2

0.5

1

EEE 461 12

Cumulative Density FunctionCumulative Density Function

• The cumulative density function (CDF) of the RV, x, is given by Fx(a)=Px(x<a)

Fx(a)

a3-1 0-2

0.5

1

P(x)

x3-1 0-2

0.51

0.2 0.20.1

EEE 461 13

Probability Density FunctionProbability Density Function

• The probability density function(PDF) of the RV x is given by f(x)

• Shows how probability is distributed across the axis

x xx

a x a x

dF a dP x af x

da da

fx(x)

x3-1 0-2

0.51

0.2 0.20.1

EEE 461 14

Types of DistributionsTypes of Distributions• Discrete-M discrete values at x1, x2, x3,. . . , xm

• Continuous- Can take on any value in an defined interval

fx(x)

x10-1

0.5

1

Fx(a)

x10-1

0.5

1

Fx(a)

a3-1 0-2

0.5

1

fx(x)

x3-1 0-2

0.51

0.2 0.20.1DISCRETE

Continuous

EEE 461 15

Properties of CDF’sProperties of CDF’s• Fx(a) is a non decreasing function• 0 < Fx(a) < 1• Fx(-infinity) = 0• Fx(infinity) = 1• F(a) is right-hand continuous

x x0

lima

F a f x dx

x x0

limF a F a

EEE 461 16

PDF PropertiesPDF Properties

• fx(x) is nonnegative, fx(x) > 0

• The total probability adds up to one

x 1xf x dx F

fx(x)PDF

10-1

2

Fx(a)CDF

1-1

1

EEE 461 17

Calculating ProbabilityCalculating Probability

• To calculate the probability for a range of values

x x x

x x

x0 lim

b

a

P a x b P x b P x a

F b F a

f x dx

fx(x)

10-1

2

1-1ba ba

F(b)F(a)

AREA= F(b)- F(a)

EEE 461 18

Discrete Random VariablesDiscrete Random Variables

1

1

If x is discretely distributed and represents a discrete event

( ) ( ) ( - )

F( ) ( )

i

M

i ii

L

ii

x

f x P x x x

a P x

• Summations are used instead of integrals for discrete RV.• Discrete events are represented by using DELTA

functions.

EEE 461 19

PDF and CDF of a Triangular PDF and CDF of a Triangular WaveWave

-A

A

• Calculate Probability that the amplitude of a triangle wave is greater than 1 Volt, if A=2.

• Sweep a narrow window across the waveform and measure the relative frequency of occurrence of different voltages.

s(t)fx(x)

A

-A

fx(x)

0

1/2A

A-A

EEE 461 20

PDF and CDF of a Triangular PDF and CDF of a Triangular Wave Wave

fV(v)

0

• Calculate Probability that the amplitude of a triangle wave is greater than 1 Volt, if A=2.

1/4

2-2 1

2

V 1 1

V V V

1 11

4 43 1

1 1 14 4

VP v f v dv dv

P v F F

FV(v)

0

3/4

2-2 1

1

EEE 461 21

fV(v)

0

• Calculate Probability that the amplitude of a triangle wave is in the range [0.5,1] v, if A=2.

1/4

2-2 1

1 1

V 0.5 0.5

V V V

1 10.5 1

4 81

0.5 1 1 0.58

VP v f v dv dv

P v F F

FV(v)

0

3/4

2-2 1

1

5/8

PDF and CDF of a Triangular PDF and CDF of a Triangular WaveWave

EEE 461 22

PDF and CDF of a Square PDF and CDF of a Square WaveWave

-A

A

• Calculate Probability that the amplitude of a square wave is at +A.

• Sketch PDF and CDF

s(t)fx(x)

0 A-A

EEE 461 23

PDF and CDF of a Square PDF and CDF of a Square WaveWave

• Calculate Probability that the amplitude of a square wave is at +A. 1/4

• Sketch PDF and CDF

-A

A

s(t) fx(x)

0 A-A

Fx(x)

0 A-A

EEE 461 24

Ensemble AveragesEnsemble Averages• The expected value (or ensemble average) of

y=h(x) is:

x

x

x

[ ] [ ]

For Discrete distributions

[ ( )] i ii

y E y h x f x dx

f x dx

y h x E y h x f x

EEE 461 25

MomentsMoments• The r th moment of RV x about x=xo is

x

o

x

2

2 2 2x

( ) ( )

MEAN is the first moment taken about x =0

VARIANCE is the second moment around the mean

( ) ( )

STANDARD DEVIATION - the second moment around

r ro ox x x x f x dx

m x x f x dx

x x x x f x dx

2

2 2x

2 2

mean

( )

( )

x x f x dx

x x