1 ee571 part 3 random processes huseyin bilgekul eeng571 probability and astochastic processes...
DESCRIPTION
3 EE571 Kinds of Random ProcessesTRANSCRIPT
![Page 1: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/1.jpg)
1EE571
PART 3Random Processes
Huseyin BilgekulEeng571 Probability and astochastic Processes
Department of Electrical and Electronic Engineering Eastern Mediterranean University
![Page 2: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/2.jpg)
2EE571
Random Processes
![Page 3: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/3.jpg)
3EE571
Kinds of Random Processes
![Page 4: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/4.jpg)
4EE571
• A RANDOM VARIABLE X, is a rule for assigning to every outcome, of an experiment a number X(. – Note: X denotes a random variable and X( denotes
a particular value. • A RANDOM PROCESS X(t) is a rule for
assigning to every a function X(t, – Note: for notational simplicity we often omit the
dependence on .
Random Processes
![Page 5: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/5.jpg)
5EE571
Conceptual Representation of RP
![Page 6: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/6.jpg)
6EE571
The set of all possible functions is called the ENSEMBLE.
Ensemble of Sample Functions
![Page 7: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/7.jpg)
7EE571
• A general Random or Stochastic Process can be described as: – Collection of time functions
(signals) corresponding to various outcomes of random experiments.
– Collection of random variables observed at different times.
• Examples of random processes in communications: – Channel noise, – Information generated by a source, – Interference.
t1 t2
Random Processes
![Page 8: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/8.jpg)
8EE571
Let denote the random outcome of an experiment. To every such outcome suppose a waveform is assigned.The collection of such waveforms form a stochastic process. The set of and the time index t can be continuousor discrete (countably infinite or finite) as well.For fixed (the set of all experimental outcomes), is a specific time function.For fixed t,
is a random variable. The ensemble of all such realizations over time represents the stochastic
),( tX
}{ k
Si
),( 11 itXX
),( tX
t1
t2
t
),(n
tX
),(k
tX
),(2
tX
),(1
tX
),( tX
0
),( tX
Random Processes
![Page 9: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/9.jpg)
9EE571
Random Process for a Continuous Sample Space
![Page 10: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/10.jpg)
10EE571
Random Processes
![Page 11: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/11.jpg)
11EE571
Wiener Process Sample Function
![Page 12: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/12.jpg)
12EE571
![Page 13: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/13.jpg)
13EE571
Sample Sequence for Random Walk
![Page 14: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/14.jpg)
14EE571
Sample Function of the Poisson Process
![Page 15: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/15.jpg)
15EE571
Random Binary Waveform
![Page 16: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/16.jpg)
16EE571
Autocorrelation Function of the Random Binary Signal
![Page 17: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/17.jpg)
17EE571
Example
![Page 18: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/18.jpg)
18EE571
![Page 19: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/19.jpg)
19EE571
Random Processes Introduction (1)
![Page 20: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/20.jpg)
20EE571
Introduction
• A random process is a process (i.e., variation in time or one dimensional space) whose behavior is not completely predictable and can be characterized by statistical laws.
• Examples of random processes– Daily stream flow– Hourly rainfall of storm events– Stock index
![Page 21: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/21.jpg)
21EE571
Random Variable• A random variable is a mapping function which assigns outcomes of a
random experiment to real numbers. Occurrence of the outcome follows certain probability distribution. Therefore, a random variable is completely characterized by its probability density function (PDF).
![Page 22: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/22.jpg)
22EE571
STOCHASTIC PROCESS
![Page 23: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/23.jpg)
23EE571
STOCHASTIC PROCESS
![Page 24: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/24.jpg)
24EE571
STOCHASTIC PROCESS
![Page 25: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/25.jpg)
25EE571
STOCHASTIC PROCESS
• The term “stochastic processes” appears mostly in statistical textbooks; however, the term “random processes” are frequently used in books of many engineering applications.
![Page 26: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/26.jpg)
26EE571
STOCHASTIC PROC ESS
![Page 27: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/27.jpg)
27EE571
DENSITY OF STOCHASTIC PROCESSES
• First-order densities of a random process A stochastic process is defined to be completely or totally characterized if the joint densities for the random variables are known for all times and all n. In general, a complete characterization is practically impossible, except in rare cases. As a result, it is desirable to define and work with various partial characterizations. Depending on the objectives of applications, a partial characterization often suffices to ensure the desired outputs.
)(),(),( 21 ntXtXtX
nttt ,,, 21
![Page 28: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/28.jpg)
28EE571
• For a specific t, X(t) is a random variable with distribution .
• The function is defined as the first-order distribution of the random variable X(t). Its derivative with respect to x
is the first-order density of X(t).
])([),( xtXptxF
),( txF
xtxFtxf
),(),(
DENSITY OF STOCHASTIC PROCESSES
![Page 29: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/29.jpg)
29EE571
• If the first-order densities defined for all time t, i.e. f(x,t), are all the same, then f(x,t) does not depend on t and we call the resulting density the first-order density of the random process ; otherwise, we have a family of first-order densities.
• The first-order densities (or distributions) are only a partial characterization of the random process as they do not contain information that specifies the joint densities of the random variables defined at two or more different times.
)(tX
DENSITY OF STOCHASTIC PROCESSES
![Page 30: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/30.jpg)
30EE571
• Mean and variance of a random process The first-order density of a random process, f(x,t), gives the probability density of the random variables X(t) defined for all time t. The mean of a random process, mX(t), is thus a function of time specified by
ttttX dxtxfxXEtXEtm ),(][)]([)(
MEAN AND VARIANCE OF RP
• For the case where the mean of X(t) does not depend on t, we have
• The variance of a random process, also a function of time, is defined by
constant). (a )]([)( XX mtXEtm
2222 )]([][)]()([)( tmXEtmtXEt XtXX
![Page 31: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/31.jpg)
31EE571
• Second-order densities of a random process For any pair of two random variables X(t1) and X(t2), we define the second-order densities of a random process as or .
• Nth-order densities of a random process The nth order density functions for at times
are given by or .
),;,( 2121 ttxxf ),( 21 xxf
)(tX
nttt ,,, 21
),,,;,,,( 2121 nn tttxxxf ),,,( 21 nxxxf
HIGHER ORDER DENSITY OF RP
![Page 32: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/32.jpg)
32EE571
• Given two random variables X(t1) and X(t2), a measure of linear relationship between them is specified by E[X(t1)X(t2)]. For a random process, t1 and t2 go through all possible values, and therefore, E[X(t1)X(t2)] can change and is a function of t1 and t2. The autocorrelation function of a random process is thus defined by
),()()(),( 122121 ttRtXtXEttR
Autocorrelation function of RP
![Page 33: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/33.jpg)
33EE571
Autocovariance Functions of RP
![Page 34: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/34.jpg)
34EE571
• Strict-sense stationarity seldom holds for random processes, except for some Gaussian processes. Therefore, weaker forms of stationarity are needed.
nnnn tttxxxftttxxxf ,,,;,,,,,,;,,, 21212121
Stationarity of Random Processes
![Page 35: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/35.jpg)
35EE571
Time, t
PDF of X(t)X(t)
Stationarity of Random Processes
![Page 36: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/36.jpg)
36EE571
. allfor constant)( )( tmtXE
. and allfor ,),( 21121221 ttttRttRttR
Wide Sense Stationarity (WSS) of Random Processes
![Page 37: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/37.jpg)
37EE571
• Equality
• Note that “x(t, i) = y(t, i) for every i” is not the same as “x(t, i) = y(t, i) with probability 1”.
Equality and Continuity of RP
![Page 38: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/38.jpg)
38EE571
Equality and Continuity of RP
![Page 39: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/39.jpg)
39EE571
• Mean square equality Mean Square Equality of RP
![Page 40: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/40.jpg)
40EE571
Equality and Continuity of RP
![Page 41: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/41.jpg)
41EE571
![Page 42: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/42.jpg)
42EE571
Random Processes Introduction (2)
![Page 43: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/43.jpg)
43EE571
Stochastic Continuity
![Page 44: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/44.jpg)
44EE571
Stochastic Continuity
![Page 45: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/45.jpg)
45EE571
Stochastic Continuity
![Page 46: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/46.jpg)
46EE571
Stochastic Continuity
![Page 47: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/47.jpg)
47EE571
Stochastic Continuity
![Page 48: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/48.jpg)
48EE571
Stochastic Continuity
![Page 49: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/49.jpg)
49EE571
• A random sequence or a discrete-time random process is a sequence of random variables {X1(), X2(), …, Xn(),…} = {Xn()}, .
• For a specific , {Xn()} is a sequence of numbers that might or might not converge. The notion of convergence of a random sequence can be given several interpretations.
Stochastic Convergence
![Page 50: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/50.jpg)
50EE571
• The sequence of random variables {Xn()} converges surely to the random variable X() if the sequence of functions Xn() converges to X() as n for all , i.e.,Xn() X() as n for all .
Sure Convergence (Convergence Everywhere)
![Page 51: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/51.jpg)
51EE571
Stochastic Convergence
![Page 52: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/52.jpg)
52EE571
Stochastic Convergence
![Page 53: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/53.jpg)
53EE571
Almost-sure convergence (Convergence with probability 1)
![Page 54: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/54.jpg)
54EE571
Almost-sure Convergence (Convergence with probability 1)
![Page 55: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/55.jpg)
55EE571
Mean-square Convergence
![Page 56: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/56.jpg)
56EE571
Convergence in Probability
![Page 57: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/57.jpg)
57EE571
Convergence in Distribution
![Page 58: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/58.jpg)
58EE571
• Convergence with probability one applies to the individual realizations of the random process. Convergence in probability does not.
• The weak law of large numbers is an example of convergence in probability.
• The strong law of large numbers is an example of convergence with probability 1.
• The central limit theorem is an example of convergence in distribution.
Remarks
![Page 59: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/59.jpg)
59EE571
Weak Law of Large Numbers (WLLN)
![Page 60: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/60.jpg)
60EE571
Strong Law of Large Numbers (SLLN)
![Page 61: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/61.jpg)
61EE571
The Central Limit Theorem
![Page 62: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/62.jpg)
62EE571
Venn Diagram of Relation of Types of Convergence
Note that even sure convergence may not imply mean square convergence.
![Page 63: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/63.jpg)
63EE571
Example
![Page 64: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/64.jpg)
64EE571
Example
![Page 65: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/65.jpg)
65EE571
Example
![Page 66: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/66.jpg)
66EE571
Example
![Page 67: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/67.jpg)
67EE571
Ergodic Theorem
![Page 68: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/68.jpg)
68EE571
Ergodic Theorem
![Page 69: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/69.jpg)
69EE571
The Mean-Square Ergodic Theorem
![Page 70: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/70.jpg)
70EE571
The above theorem shows that one can expect a sample average to converge to a constant in mean square sense if and only if the average of the means converges and if the memory dies out asymptotically, that is , if the covariance decreases as the lag increases.
The Mean-Square Ergodic Theorem
![Page 71: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/71.jpg)
71EE571
Mean-Ergodic Process
![Page 72: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/72.jpg)
72EE571
Strong or Individual Ergodic Theorem
![Page 73: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/73.jpg)
73EE571
Strong or Individual Ergodic Theorem
![Page 74: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/74.jpg)
74EE571
Strong or Individual Ergodic Theorem
![Page 75: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/75.jpg)
75EE571
Examples of Stochastic Processes
• iid random process A discrete time random process {X(t), t = 1, 2, …} is said to be independent and identically distributed (iid) if any finite number, say k, of random variables X(t1), X(t2), …, X(tk) are mutually independent and have a common cumulative distribution function FX() .
![Page 76: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/76.jpg)
76EE571
• The joint cdf for X(t1), X(t2), …, X(tk) is given by
• It also yields
where p(x) represents the common probability mass function.
)()()(
,,,),,,(
21
221121,,, 21
kXXX
kkkXXX
xFxFxF
xXxXxXPxxxFk
)()()(),,,( 2121,,, 21 kXXXkXXX xpxpxpxxxpk
iid Random Stochastic Processes
![Page 77: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/77.jpg)
77EE571
Bernoulli Random Process
![Page 78: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/78.jpg)
78EE571
Random walk process
![Page 79: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/79.jpg)
79EE571
• Let 0 denote the probability mass function of X0. The joint probability of X0, X1, Xn is
)|()|()()()()(
)()()(,,,
),,,(
10100
10100
101100
101100
1100
nn
nn
nnn
nnn
nn
xxPxxPxxxfxxfx
xxPxxPxXPxxxxxXP
xXxXxXP
Random walk process
![Page 80: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/80.jpg)
80EE571
)|()|()|()(
)|()|()|()(),,,(
),,,,(),,,|(
1
10100
110100
1100
111100
110011
nn
nn
nnnn
nn
nnnn
nnnn
xxPxxPxxPx
xxPxxPxxPxxXxXxXP
xXxXxXxXPxXxXxXxXP
Random walk process
![Page 81: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/81.jpg)
81EE571
The property
is known as the Markov property.A special case of random walk: the Brownian motion.
)|(),,,|( 1110011 nnnnnnnn xXxXPxXxXxXxXP
Random walk process
![Page 82: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/82.jpg)
82EE571
Gaussian process• A random process {X(t)} is said to be a
Gaussian random process if all finite collections of the random process, X1=X(t1), X2=X(t2), …, Xk=X(tk), are jointly Gaussian random variables for all k, and all choices of t1, t2, …, tk.
• Joint pdf of jointly Gaussian random variables X1, X2, …, Xk:
![Page 83: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/83.jpg)
83EE571
Gaussian process
![Page 84: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/84.jpg)
84EE571
Time series – AR random process
![Page 85: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/85.jpg)
85EE571
The Brownian motion (one-dimensional, also known as random walk)
• Consider a particle randomly moves on a real line. • Suppose at small time intervals the particle jumps a small
distance randomly and equally likely to the left or to the right.
• Let be the position of the particle on the real line at time t.
)(tX
![Page 86: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/86.jpg)
86EE571
• Assume the initial position of the particle is at the origin, i.e.
• Position of the particle at time t can be expressed as where
are independent random variables, each having probability 1/2 of equating 1 and 1. ( represents the largest integer not exceeding
.)
0)0( X
]/[21)( tYYYtX ,, 21 YY
/t/t
The Brownian motion
![Page 87: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/87.jpg)
87EE571
Distribution of X(t)
• Let the step length equal , then
• For fixed t, if is small then the distribution of is approximately normal with mean 0 and variance t, i.e., .
]/[21)( tYYYtX )(tX
tNtX ,0~)(
![Page 88: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/88.jpg)
88EE571
Graphical illustration of Distribution of X(t)
Time, t
PDF of X(t)X(t)
![Page 89: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/89.jpg)
89EE571
• If t and h are fixed and is sufficiently small then
1 2 [( ) / ] 1 2 [ / ]
[ / ] 1 [ / ] 2 [( ) / ]
2
( ) ( )
t h t
t t t h
t t t h
X t h X t
Y Y Y Y Y Y
Y Y Y
Y Y Y
![Page 90: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/90.jpg)
90EE571
Graphical Distribution of the displacement of
• The random variable is normally distributed with mean 0 and variance h, i.e.
)()( tXhtX
)()( tXhtX
duhu
hxtXhtXP
x
2exp
21)()(
2
![Page 91: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/91.jpg)
91EE571
• Variance of is dependent on t, while variance of is not.
• If , then , are independent random variables.
)(tX)()( tXhtX
mttt 2210 )()( 12 tXtX ,),()( 34 tXtX )()( 122 mm tXtX
![Page 92: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/92.jpg)
92EE571
t
X
![Page 93: 1 EE571 PART 3 Random Processes Huseyin Bilgekul Eeng571 Probability and astochastic Processes Department of Electrical and Electronic Engineering Eastern](https://reader035.vdocuments.net/reader035/viewer/2022070605/5a4d1b027f8b9ab05998718c/html5/thumbnails/93.jpg)
93EE571
Covariance and Correlation functions of )(tX
t
YYYE
YYYYYYYYYE
YYYYYYE
htXtXEhtXtXCov
t
httttt
htt
2
21
2121
2
21
2121
)()()(),(
Cov ( ), ( )Correl ( ), ( )
=
X t X t hX t X t h
t t h
tt t h