2© Tallal Elshabrawy
Review of Probability Theory
Experiments, Sample Spaces and Events Axioms of Probability Conditional Probability Bayes’s Rule Independence Discrete & Continuous Random Variables
3© Tallal Elshabrawy
Random Experiment
It is an experiment whose outcome cannot be predicted with certainty
Examples:
Tossing a Coin Rolling a Die
4© Tallal Elshabrawy
Random Experiment in Communications
Why is this a random experiment? We do not know
The amount of noise that will affect the transmitted bit Whether the bit will be received in error or not
Transmission of Bits across a Communication Channel
Waveform
Generator
Waveform
Detection
Channel
v rx y
0 T
0 T
+A V.
-A V.
vivi=1
vi=0
xi
0yi>0
yi<0
ri=1
ri=0
ri+
zi ]-∞, ∞[
yi
5© Tallal Elshabrawy
Random Experiment in Networks
Why is this a random experiment? We do not know
Whether the packet will reach the destination or not If the packet reaches the destination, how long would it take to get
there?
Transferring a Packet across a Communication Network
Packet
Packet
6© Tallal Elshabrawy
Sample Space
The set of all possible outcomes Tossing a coin
S = {H,T}
Rolling a die S = {1,2,3,4,5,6}
The AWGN in a Communication Channel S = ] -∞, ∞ [
Heads Tails
xi +
zi
yi
7© Tallal Elshabrawy
Event
An event is a subset of the sample space S
Examples Let A be the event of observing one head in a coin
flipped two times A = {HT,TH}
Let B be the event of observing two heads in a coin flipped twice B = {HH}
8© Tallal Elshabrawy
Axioms of Probability
Probability of an event is a measure of how often an event might occur
no. of sample pts in P( )
no. of sample pts in
AA
S
Axioms of Probability
1. 0 P 1
2. P 0,P 1
3. P P +P -P ,
A
S
A B A B A B
9© Tallal Elshabrawy
Example
Let Event A characterize that the outcome of rolling the die once is smaller than 3 A = {1,2} P(A) = 2/6 = 1/3
Let Event B characterize that the outcome of rolling the die once is an even number B = {2,4,6} P(B) = 3/6 = 1/2
12
4
6
P , 1/ 6
P 1/3 1/ 2 1/ 6
A B
A B
A B
S
35
10© Tallal Elshabrawy
Conditional Probability
Probability of event B given A has occurred
P ,P
P
A BB A
A
P ,P
P
A BA B
B
Probability of event A given B has occurred
11© Tallal Elshabrawy
Example
Two cards are drawn in succession without replacement from an ordinary (52 cards) deck. Find the probability that both cards are aces
Let A be the event that the first card is an ace Let B be the event that the second card is an
ace
P , =P P
4 3 1P , =
52 51 16 17
A B A B A
A B
12© Tallal Elshabrawy
Conditional Probability in Communications
Conditioned on v=1, what is the probability of making an error?
-5 -4 -3 -2 -1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
r=0Decision
Zone
0 T
0 T
+A V.
-A V.
vivi=1
vi=0
xi
0yi>0
yi<0
ri=1
ri=0
ri+
zi ]-∞, ∞[
yi
[ ] [ ][ ] [ ][ ] [ ]101
101
101
x=x+z<Pr=v=errorPr
x=y<Pr=v=errorPr
v=r=Pr=v=errorPr
14© Tallal Elshabrawy
Theorem of Total Probability
Let B1, B2, …, Bn be a set of mutually exclusive and exhaustive events.
( ) ( ) ( )∑ n
1=i PP=P ii BBAA
15© Tallal Elshabrawy
Bayes’s Theorem
Let B1, B2, …, Bn be a set of mutually exclusive and exhaustive events.
( ) ( ) ( )( ) ( )∑ n
1=i PP
PP=P
ii
ii
iBBA
BBAAB
16© Tallal Elshabrawy
Independent Events
A and B are independent if P(B|A) = P(B)P(A|B) = P(A)P(A,B) = P(A)P(B)
17© Tallal Elshabrawy
Example
Let A be the event that the grades will be out on Thursday P(A)
Let B be the even that I will get A+ in Random Signals and Noise P(B)
So What is the probability that I get A+ if the grades are out on Thursday P(B|A) = P(B)
18© Tallal Elshabrawy
Random Variable
Characterizes the experiment in terms of real numbers
Example X is the variable for the number of heads for a coin tossed three
times X = 0,1,2,3
Discrete Random Variables The random variable can only take a finite number of values
Continuous Random Variables The random variable can take a continuum of values
19© Tallal Elshabrawy
Bernoulli Discrete Random Variable Represents experiments that have two possible outcomes.
These experiments are called Bernoulli Trials
Associates values {0, 1} with the two outcomes such that P[X = 0] = 1-p P[X = 1] = p
Examples Coin tossing experiment maps a ‘Heads’ to X = 1 and a ‘Tails’ to
X = 0 (or vice versa) such that p=0.5 for a fair coin
Digital communication system where X = 1 represents a bit received in error and X = 0 corresponds to a bit received correctly. In such system p represents the channel bit error probability
20© Tallal Elshabrawy
Binomial Discrete Random Variable
A random variable that represents the number of occurrences of ‘1’ or ‘0’ in n Bernoulli trials
The corresponding random variable X may take and values from {0, 1, 2, …, n}
The probability mass function PMF for having k ‘1’ in n Bernoulli trials isP[X = k] = nCk pk(1-p)n-k
Examples In a digital communication system, the number of bits in error in a
packet depicts a Binomial discrete random variable
21© Tallal Elshabrawy
Geometric Discrete Random Variable Geometric distribution describes the number of Bernoulli
trials in succession are conducted until some particular outcome is observed (lets say ‘1’)
The corresponding random variable X may take and values from {1, 2, 3, …, ∞}
The probability mass function PMF for having k Bernoulli trials in succession until an outcome of ‘1’ is observedP[X = k] = (1-p)k-1p
Examples: In a communication network, the number of transmissions until a
packet is received correctly follows a Geometric distribution