cmsc726: hmms material from: slides from sebastian thrun, and yair weiss
Post on 22-Dec-2015
231 views
TRANSCRIPT
![Page 1: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/1.jpg)
.
cmsc726: HMMs
material from: slides from Sebastian Thrun, and Yair Weiss
![Page 2: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/2.jpg)
Outline
Time Series Markov Models Hidden Markov Models Learning HMMs with EM Applying HMMs Summary
![Page 3: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/3.jpg)
Audio Spectrum
Audio Spectrum of the Song of the Prothonotary Warbler
![Page 4: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/4.jpg)
Bird Sounds
Chestnut-sided Warbler Prothonotary Warbler
![Page 5: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/5.jpg)
Questions One Could Ask
What bird is this? How will the song
continue? Is this bird sick? What phases does this
song have?
Time series classification Time series prediction
Outlier detection Time series segmentation
![Page 6: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/6.jpg)
Other Sound Samples
![Page 7: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/7.jpg)
Another Time Series Problem
Intel
Cisco General Electric
Microsoft
![Page 8: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/8.jpg)
Questions One Could Ask
Will the stock go up or down?
What type stock is this (eg, risky)?
Is the behavior abnormal?
Time series prediction
Time series classification
Outlier detection
![Page 9: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/9.jpg)
Music Analysis
![Page 10: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/10.jpg)
Questions One Could Ask
Is this Beethoven or Bach? Can we compose more of
that? Can we segment the piece
into themes?
Time series classification Time series
prediction/generation Time series
segmentation
![Page 11: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/11.jpg)
CiteSeer.Com (Citation Index)
Dave Rumelhart
Takeo Kanade Tom Mitchell
Raj ReddyJim Morris
![Page 12: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/12.jpg)
Questions One Could Ask Shall UMD give tenure? Shall UMD hire? Shall UMD fire?
Time series classification Time series prediction Outlier detection
Disclaimer: This is a joke!
![Page 13: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/13.jpg)
The Real Question
How do we model these problems?
How do we formulate these questions as a inference/learning problems?
![Page 14: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/14.jpg)
Outline For Today
Time Series Markov Models Hidden Markov Models Learning HMMs with EM Applying HMMs Summary
![Page 15: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/15.jpg)
Weather: A Markov Model
Sunny
Rainy
Snowy
80%
15%
5%
60%
2%
38%
20%
75% 5%
![Page 16: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/16.jpg)
Ingredients of a Markov Model
States:
State transition probabilities:
Initial state distribution:
Sunny
Rainy
Snowy
80%
15%
5%
60%
2%
38%
20%
75% 5%
][ 1 ii SqP
},...,,{ 21 NSSS
)|( 1 jtitij SqSqPa
![Page 17: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/17.jpg)
Ingredients of Our Markov Model
States:
State transition probabilities:
Initial state distribution:
)05.25.7.(
},,{ snowyrainysunny SSS
2.05.75.
02.6.38.
05.15.8.
A
Sunny
Rainy
Snowy
80%
15%
5%
60%
2%
38%
20%
75% 5%
![Page 18: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/18.jpg)
Probability of a Time Series
Given:
What is the probability of this series?
)05.25.7.(
2.05.75.
02.6.38.
05.15.8.
A
0001512.02.002.06.06.015.07.0
)|()|()|()|()|()(
snowysnowyrainysnowy
rainyrainyrainyrainysunnyrainysunny
SSPSSPSSPSSPSSPSP
![Page 19: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/19.jpg)
Outline For Today
Time Series Markov Models Hidden Markov Models Learning HMMs with EM Applying HMMs Summary
![Page 20: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/20.jpg)
Hidden Markov Models
Sunny
Rainy
Snowy
80%
15%
5%
60%
2%
38%
20%
75% 5%
Sunny
Rainy
Snowy
80%
15%
5%
60%
2%
38%
20%
75% 5%
60%
10%
30%
65%
5%
30%
50%0%50%
NOT OBSERVABLE
![Page 21: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/21.jpg)
Sunny
Rainy
Snowy
80%
15%
5%
60%
2%
38%
20%
75% 5%
60%
10%
30%
50%0%50%
65%
5%
30%Ingredients of an HMM
States: State transition probabilities:
Initial state distribution:][ 1 ii SqP
},...,,{ 21 NSSS
)|( 1 jtitij SqSqPa
Observations:
Observation probabilities:
},...,,{ 21 MOOO
)|()( jtktj SqOvPkb
![Page 22: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/22.jpg)
Sunny
Rainy
Snowy
80%
15%
5%
60%
2%
38%
20%
75% 5%
60%
10%
30%
50%0%50%
65%
5%
30%
Ingredients of Our HMM States: Observations: State transition probabilities:
Initial state distribution: Observation probabilities:
)05.25.7.(
},,{ snowyrainysunny SSS
5.5.0
65.3.05.
1.3.6.
B
},,{ umbrellacoatshorts OOO
2.05.75.
02.6.38.
05.15.8.
A
![Page 23: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/23.jpg)
Probability of a Time Series
Given:
What is the probability of this series?
)05.25.7.(
2.05.75.
02.6.38.
05.15.8.
A
5.5.0
65.3.05.
1.3.6.
B
),...,(),...,|()()|( 7171,..., all 71
qqPqqOPQPQOPqqQ
),...,,,()( umbrellaumbrellacoatcoat OOOOPOP
...6.01.03.08.07.0 426
![Page 24: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/24.jpg)
Calculating Data Likelihood
Problem: exponential in time Is there a more efficient way?
![Page 25: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/25.jpg)
The Forward Algorithm (1)
S2
S3
S1
S2
S3
S1
O2 O3O1O2 O3O1
S2
S3
S1
O2 O3O1
S2
S3
S1
O2 O3O1
S2
S3
S1
O2 O3O1
…
),,...,()( 1 ittt SqOOPi
)()(
)()|,(
),,...,(),,...,|,,...,(
),,...,()(
11
111
111111
1111
iaOb
iSqSqOP
SqOOPSqOOSqOOP
SqOOPj
tijt
N
ij
titjtt
N
i
N
iittittjtt
jttt
)()( 11 Obi ii
![Page 26: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/26.jpg)
Question
Does this solve our problem of calculating ? )(OP
YES NO
Count
![Page 27: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/27.jpg)
Answer
And the answer is…
Yes!
N
i
N
iTiT iSqOPOP
1 1
)(),()(
![Page 28: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/28.jpg)
Exercise
What is the probability of observing AB?
a. Initial state s1:
b. Initial state chosen at random:
s2s1
0.60.4
0.30.7
0.3
B
0.7
A
0.2
B
0.8
A
0.2 (0.4 0.8 + 0.6 0.7) = 0.148
0.5 0.148 + (0.5 0.3 (0.3 0.7 + 0.7 0.8)) = 0.1895
![Page 29: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/29.jpg)
Next Question: What is the probability that the state at time t was Si?
Can we answer this?
S2
S3
S1
S2
S3
S1
O2 O3O1O2 O3O1
S2
S3
S1
O2 O3O1
S2
S3
S1
O2 O3O1
S2
S3
S1
O2 O3O1
…
),,...,()( 1 ittt SqOOPi
No!
![Page 30: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/30.jpg)
The Backward Algorithm (2)
S2
S3
S1
S2
S3
S1
O2 O3O1O2 O3O1
S2
S3
S1
O2 O3O1
S2
S3
S1
O2 O3O1
S2
S3
S1
O2 O3O1
…
)|,...,,()( 21 itTttt SqOOOPi
1)( iT
N
jttjijt jObai
111 )()(...)(
![Page 31: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/31.jpg)
The Forward-Backward Algorithm (3)
S2
S3
S1
S2
S3
S1
O2 O3O1O2 O3O1
S2
S3
S1
O2 O3O1
S2
S3
S1
O2 O3O1
S2
S3
S1
O2 O3O1
…
)(it)(it
)()()()( iiSqPi ttitt
![Page 32: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/32.jpg)
Summary (So Far)
For a given HMM , we can compute
),|( OSqP it
![Page 33: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/33.jpg)
Finding the best state sequenceWe would like to the most likely path (and not just the most likely state at each time slice)
The Viterbi algorithm is an efficient method for finding the MPE:
and we to reconstruct the path:
)O,Q(P)O|Q(P maxargmaxargQQ
jitj
1tjitj
1ti1t
1ii1
a)j(maxarg)i(a)j(max)O(b)i(
)O(bq)i(
)Q(Q
)i(maxargQ)i(max)Q(P
tt1t
Ti
TTi
![Page 34: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/34.jpg)
Outline
Time Series Markov Models Hidden Markov Models Learning HMMs with EM Applying HMMs Summary
![Page 35: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/35.jpg)
Hidden Markov Models
Sunny
Rainy
Snowy
80%
15%
5%
60%
2%
38%
20%
75% 5%
Sunny
Rainy
Snowy
80%
15%
5%
60%
2%
38%
20%
75% 5%
60%
10%
30%
65%
5%
30%
50%0%50%
NOT OBSERVABLE
![Page 36: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/36.jpg)
s2s1
0.60.4
0.30.7
0.3
B
0.7
A
0.2
B
0.8
A
Summary So Far
HMMs: generative probabilistic models of time series with hidden state
Forward-backward algorithm: Efficient algorithm for calculating
),|( OSqP it
![Page 37: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/37.jpg)
What about learning?
![Page 38: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/38.jpg)
EM
Problem: Find HMM that makes data most likely
E-Step: Compute for given
M-Step: Compute new under these expectations (this is now a Markov model)
),|( OSqP it
![Page 39: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/39.jpg)
E-Step
Calculate
using the forward-backward algorithm, for fixed model
),|()( OSqPi itt
),|,(),( 1 OSqSqPji jtitt
![Page 40: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/40.jpg)
The M Step: generate =(, a, b))(1 at time statein timesofnumber expected 1 iSii
T
tt
T
tt
i
jiji
i
ji
S
SSa
1
1
)(
),(
state from ns transitioofnumber expected
state to state from ns transitioofnumber expected
T
tt
T
ttvO
i
kii
i
iI
S
vSkb
kt
1
1
)(
)(
statein timesofnumber expected
observing statein timesofnumber expected)(
![Page 41: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/41.jpg)
Summary (Learning)
Given observation sequence O Guess initial model Iterate:
Calculate expected times in state Si at time t (and in Sj at
time t) using forward-backward algorithm Find new model by frequency counts
![Page 42: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/42.jpg)
Outline For Today
Time Series Markov Models Hidden Markov Models Learning HMMs with EM Applying HMMs Summary
![Page 43: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/43.jpg)
Three Problems
What bird is this?
How will the song continue?
Is this bird abnormal?
Time series classification
Time series prediction
Outlier detection
![Page 44: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/44.jpg)
Time Series Classification
Train one HMM l for each bird l Given time series O, calculate
'' )'()|(
)()|()|bird(
ll
l
lPOP
lPOPOlP
![Page 45: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/45.jpg)
Outlier Detection
Train HMM Given time series O, calculate probability
If abnormally low, raise flag If high, raise flag
)|( OP
![Page 46: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/46.jpg)
Time Series Prediction
Train HMM Given time series O, calculate distribution over final
state (via )
and ‘hallucinate’ new states and observations according to a, b
),|( OSqP iT
![Page 47: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/47.jpg)
Typical HMM in Speech Recognition
20-dim frequency spaceclustered using EM
Use Bayes rule + Viterbi for classification
Linear HMM representing one phoneme
[Rabiner 86] + everyone else
![Page 48: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/48.jpg)
Typical HMM in Robotics
[Blake/Isard 98, Fox/Dellaert et al 99]
![Page 49: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/49.jpg)
Problems with HMMs
Zero probabilities Training sequence: AAABBBAAA Test sequence: AAABBBCAAA
Finding “right” number of states, right structure Numerical instabilities
![Page 50: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/50.jpg)
Outline
Time Series Markov Models Hidden Markov Models Learning HMMs with EM Applying HMMs Summary
![Page 51: cmsc726: HMMs material from: slides from Sebastian Thrun, and Yair Weiss](https://reader038.vdocuments.net/reader038/viewer/2022102800/56649d775503460f94a58806/html5/thumbnails/51.jpg)
HMMs: Main Lessons
HMMs: Generative probabilistic models of time series (with hidden state)
Forward-Backward: Algorithm for computing probabilities over hidden states
Learning models: EM, iterates estimation of hidden state and model fitting
Extremely practical, best known methods in speech, computer vision, robotics, …
Numerous extensions exist (continuous observations, states; factorial HMMs, controllable HMMs=POMDPs, …)