elements of stochastic processes
DESCRIPTION
TRANSCRIPT
![Page 1: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/1.jpg)
Stochastic Processes
Elements of Stochastic ProcessesBy
Mahdi Malaki
![Page 2: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/2.jpg)
Outline
Basic Definitions Stationary/Ergodic Processes Stochastic Analysis of Systems Power Spectrum
![Page 3: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/3.jpg)
Basic Definitions Suppose a set of random variables
indexed by a parameter Tracking these variables with respect to
the parameter constructs a process that is called Stochastic Process.
i.e. The mapping of outcomes to the real (complex) numbers changes with respect to index.
,...,...,,, 21 nXXXtX
![Page 4: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/4.jpg)
Basic Definitions (cont’d)
In a random process, we will have a family of functions called an ensemble of functionsensemble of functions
t
1,tX
t
2,tX
t
3,tX
![Page 5: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/5.jpg)
Basic Definitions (cont’d)
With fixed “beta”, we will have a “time” “time” function called sample path.
Sometimes stochastic properties of a random process can be extracted just from a single sample path. (When?)
![Page 6: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/6.jpg)
Basic Definitions (cont’d)
With fixed “t”, we will have a random variable.
With fixed “t” and “beta”, we will have a real (complex) number.
![Page 7: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/7.jpg)
Basic Definitions (cont’d)
Example I Voltage of a generator with fixed frequency
Amplitude is a random variables
tAtV cos.,
![Page 8: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/8.jpg)
Basic Definitions (cont’d)
Equality Ensembles should be equal for each “beta”
and “t”
Equality (Mean Square Sense) If the following equality holds
Sufficient in many applications
,, tYtX
0,, 2 tYtXE
![Page 9: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/9.jpg)
Basic Definitions (cont’d)
First-Order CDF of a random process
First-Order PDF of a random process
xtXtxFX Pr,
txFx
txf XX ,,
![Page 10: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/10.jpg)
Basic Definitions (cont’d)
Second-Order CDF of a random process
Second-Order PDF of a random process
22112121 Pr,;, xtXandxtXttxxFX
212121
2
2121 ,;,.
,;, ttxxFxx
ttxxf XX
![Page 11: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/11.jpg)
Basic Definitions (cont’d)
nth order can be defined. (How?)
Relation between first-order and second-order can be presented as
Relation between different orders can be obtained easily. (How?)
ttxftxf XX ,;,,
![Page 12: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/12.jpg)
Basic Definitions (cont’d)
Mean of a random process
Autocorrelation of a random process
Fact: (Why?)
dxtxfxtXEt X ,.
212121212121 ,;,..., dxdxttxxfxxtXtXEttR X
2)(
2)(, tXtXttR
![Page 13: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/13.jpg)
Basic Definitions (cont’d)
Autocovariance of a random process
Correlation Coefficient
Example
2211
212121
.
.,,
ttXttXE
ttttRttC
2221112
21 ,,2, ttRttRttRtXtXE
2211
2121
,.,
,,
ttCttC
ttCttr
![Page 14: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/14.jpg)
Basic Definitions (cont’d)
Example
Poisson Process
Mean
Autocorrelation
Autocovariance
kt
tk
etK .
!
.
tt .
21212
1
21212
221
...
...,
ttttt
tttttttR
2121 ,min., ttttC
![Page 15: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/15.jpg)
Basic Definitions (cont’d)
Complex process Definition
Specified in terms of the joint statistics of two real processes and
Vector Process A family of some stochastic processes
tYitXtZ .
tX tY
![Page 16: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/16.jpg)
Basic Definitions (cont’d)
Cross-Correlation
Orthogonal Processes
Cross-Covariance
Uncorrelated Processes
2121 ., tYtXEttRXY
212121 .,, ttttRttC YXXYXY
0, 21 ttRXY
0, 21 ttCXY
![Page 17: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/17.jpg)
Basic Definitions (cont’d)
aa-dependent processes
White Noise
Variance of Stochastic Process
0, 2121 ttCatt
0, 2121 ttCtt
tttC X2,
![Page 18: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/18.jpg)
Basic Definitions (cont’d)
Existence Theorem For an arbitrary mean function For an arbitrary covariance function
There exist a normal random process that its mean is and its covariance is
t
t
21, ttC
21, ttC
![Page 19: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/19.jpg)
Outline
Basic Definitions Stationary/Ergodic Processes Stochastic Analysis of Systems Power Spectrum
![Page 20: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/20.jpg)
Stationary/Ergodic Processes Strict Sense Stationary (SSS)
Statistical properties are invariant to shift of time origin
First order properties should be independent of “t” or
Second order properties should depends only on difference of times or
…
xftxf XX ,
;,,;, 21212112 xxfttxxftt XX
![Page 21: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/21.jpg)
Stationary/Ergodic Processes (cont’d)
Wide Sense Stationary (WSS) Mean is constant
Autocorrelation depends on the difference of times
First and Second order statistics are usually enough in applications.
tXE
RtXtXE .
![Page 22: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/22.jpg)
Stationary/Ergodic Processes (cont’d)
Autocovariance of a WSS process
Correlation Coefficient
2 RC
0CC
r
![Page 23: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/23.jpg)
Stationary/Ergodic Processes (cont’d)
White Noise
If white noise is an stationary process, why do we call it “noise”? (maybe it is not stationary !?)
aa-dependent Process
aa is called “Correlation Time”
.qC
aC 0
![Page 24: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/24.jpg)
Stationary/Ergodic Processes (cont’d)
Example SSS
Suppose aa and bb are normal random variables with zero mean.
WSS Suppose “ ” has a uniform distribution in
the interval
tbtatX sin.cos.
tatX cos.
,
![Page 25: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/25.jpg)
Stationary/Ergodic Processes (cont’d)
Example Suppose for a WSS process X(8) and X(5) are random variables
eAR .
3200
8.525858 222
RRR
XXEXEXEXXE
![Page 26: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/26.jpg)
Stationary/Ergodic Processes (cont’d)
Ergodic Process Equality of timetime properties and statisticstatistic properties.
First-Order Time average Defined as
Mean Ergodic ProcessMean Ergodic Process
Mean Ergodic Process in Mean Square SenseMean Ergodic Process in Mean Square Sense
T
TTt dttX
TtXtXE
2
1lim
tXEtXE
02
1.1
lim02
0
2
T
XTt dT
CT
tXE
![Page 27: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/27.jpg)
Stationary/Ergodic Processes (cont’d)
Slutsky’s TheoremSlutsky’s Theorem A process X(t) is mean-ergodicmean-ergodic iff
Sufficient Conditions
a)
b)
01
lim0
T
T dCT
0
dC
0lim CT
![Page 28: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/28.jpg)
Outline
Basic Definitions Stationary/Ergodic Processes Stochastic Analysis of Systems Power Spectrum
![Page 29: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/29.jpg)
Stochastic Analysis of Systems Linear Systems
Time-Invariant Systems
Linear Time-Invariant Systems
Where h(t) is called impulse responseimpulse response of the system
babtxaSystembtyatxSystemty ,..
txSystemtytxSystemty
txthtytxSystemty *
![Page 30: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/30.jpg)
Stochastic Analysis of Systems (cont’d)
Memoryless Systems
Causal Systems
Only causal systems can be realized. (Why?)
00 tttxSystemty
00 tttxSystemty
![Page 31: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/31.jpg)
Stochastic Analysis of Systems (cont’d)
Linear time-invariant systems Mean
Autocorrelation
txEthtxthEtyE **
2*21121 *,*, thttRthttR xxyy
![Page 32: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/32.jpg)
Stochastic Analysis of Systems (cont’d)
Example I System:
Impulse response:
Output Mean:
Output Autocovariance:
t
dxty0
).(
tUdtttht
0
).(
t
dttxEtyE0
.
0 0
2121 ..,, ddttRttR xxyy
![Page 33: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/33.jpg)
Stochastic Analysis of Systems (cont’d)
Example II System:
Impulse response:
Output Mean:
Output Autocovariance:
txdt
dty
tdt
dth
txEdt
dtyE
2121
2
21 ,.
, ttRtt
ttR xxyy
![Page 34: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/34.jpg)
Outline
Basic Definitions Stationary/Ergodic Processes Stochastic Analysis of Systems Power Spectrum
![Page 35: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/35.jpg)
Power Spectrum Definition
WSS process Autocorrelation
Fourier Transform of autocorrelation
deRS j.
tX R
![Page 36: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/36.jpg)
Power Spectrum (cont’d)
Inverse trnasform
For real processes
deSR j.2
1
0
0
.cos.1
.cos.2
dSR
dSS
![Page 37: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/37.jpg)
Power Spectrum (cont’d)
For a linear time invariant system
Fact (Why?)
xx
xxyy
SH
HSHS
.
..
2
*
dHStyVar xx2
.2
1
![Page 38: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/38.jpg)
Power Spectrum (cont’d)
Example I (Moving Average) System
Impulse Response
Power Spectrum
Autocorrelation
Tt
Tt
dxT
ty .2
1
.
.sin
T
TH
22
2
.
.sin
T
TSS xxyy
T
Txxyy dR
TTR
2
2
..2
12
1
![Page 39: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/39.jpg)
Power Spectrum (cont’d)
Example II
System
Impulse Response
Power Spectrum
txdt
dty
.iH
xxyy SS .2
![Page 40: Elements Of Stochastic Processes](https://reader034.vdocuments.net/reader034/viewer/2022051013/547f1bdfb4af9f70048b45a9/html5/thumbnails/40.jpg)
Question?Question?