deep learning: a theoretical introduction · deep learning: a theoretical introduction –episode 2...
TRANSCRIPT
![Page 1: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/1.jpg)
[1]Deep Learning: a theoretical introduction – Episode 2
Deep Learning:a theoretical introductionEpisode 2The Turning Point
Università degli Studi di Pavia
![Page 2: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/2.jpg)
[2]Deep Learning: a theoretical introduction – Episode 2
A hard AI winter then ensued …
… then, the turning point
![Page 3: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/3.jpg)
[3]Deep Learning: a theoretical introduction – Episode 2
Deep Belief Networks and Autoencoders
![Page 4: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/4.jpg)
[4]Deep Learning: a theoretical introduction – Episode 2
Deep Belief Networks and Autoencoders
![Page 5: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/5.jpg)
[5]Deep Learning: a theoretical introduction – Episode 2
Deep Belief Networks and Autoencoders
![Page 6: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/6.jpg)
[6]Deep Learning: a theoretical introduction – Episode 2
Deep Belief Networks and Autoencoders
![Page 7: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/7.jpg)
[7]Deep Learning: a theoretical introduction – Episode 2
An aside question:probabilities or just numbers?
![Page 8: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/8.jpg)
[8]Deep Learning: a theoretical introduction – Episode 2
•
•
Restricted Boltzmann Machine (RBM)
![Page 9: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/9.jpg)
[9]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
![Page 10: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/10.jpg)
[10]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)h hi
hi
hi = 1 x
W i
![Page 11: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/11.jpg)
[11]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
xj = 1 h
![Page 12: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/12.jpg)
[12]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
hi hj x
xi xj h
x
hi
![Page 13: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/13.jpg)
[13]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
![Page 14: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/14.jpg)
[14]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
![Page 15: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/15.jpg)
[15]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
![Page 16: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/16.jpg)
[16]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
S
![Page 17: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/17.jpg)
[17]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
S
![Page 18: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/18.jpg)
[18]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
(i)
![Page 19: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/19.jpg)
[19]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
![Page 20: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/20.jpg)
[20]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
![Page 21: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/21.jpg)
[21]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
x
![Page 22: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/22.jpg)
[22]Deep Learning: a theoretical introduction – Episode 2
•
•
•
Restricted Boltzmann Machine (RBM)
![Page 23: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/23.jpg)
[23]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
T q
![Page 24: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/24.jpg)
[24]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
•
•
•
P
![Page 25: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/25.jpg)
[25]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
x(0)
h(t+1)
x(t+1)
![Page 26: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/26.jpg)
[26]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
x(0)
h(t+1)
x(t+1)
•
•
![Page 27: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/27.jpg)
[27]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
![Page 28: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/28.jpg)
[28]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
![Page 29: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/29.jpg)
[29]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
![Page 30: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/30.jpg)
[30]Deep Learning: a theoretical introduction – Episode 2
Deep Boltzmann Machines (DBM)
![Page 31: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/31.jpg)
[31]Deep Learning: a theoretical introduction – Episode 2
Deep Boltzmann Machines (DBM)
x
hi
![Page 32: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/32.jpg)
[32]Deep Learning: a theoretical introduction – Episode 2
Deep Boltzmann Machines (DBM)
h(2)
![Page 33: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/33.jpg)
[33]Deep Learning: a theoretical introduction – Episode 2
Deep Belief Network (DBN)
![Page 34: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/34.jpg)
[34]Deep Learning: a theoretical introduction – Episode 2
Deep Belief Network (DBN)
![Page 35: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/35.jpg)
[35]Deep Learning: a theoretical introduction – Episode 2
Deep Belief Network (DBN)
![Page 36: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/36.jpg)
[36]Deep Learning: a theoretical introduction – Episode 2
Deep Belief Network (DBN)
hi = 1
![Page 37: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/37.jpg)
[37]Deep Learning: a theoretical introduction – Episode 2
Deep Belief Network (DBN)
![Page 38: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/38.jpg)
[38]Deep Learning: a theoretical introduction – Episode 2
Deep Belief Network (DBN)
![Page 39: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/39.jpg)
[39]Deep Learning: a theoretical introduction – Episode 2
Deep Belief Network (DBN)
![Page 40: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/40.jpg)
[40]Deep Learning: a theoretical introduction – Episode 2
Deep Belief Network (DBN)
![Page 41: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/41.jpg)
[41]Deep Learning: a theoretical introduction – Episode 2
Deep Belief Network (DBN)
![Page 42: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/42.jpg)
[42]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
![Page 43: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/43.jpg)
[43]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
![Page 44: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/44.jpg)
[44]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
![Page 45: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/45.jpg)
[45]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
![Page 46: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/46.jpg)
[46]Deep Learning: a theoretical introduction – Episode 2
Restricted Boltzmann Machine (RBM)
![Page 47: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/47.jpg)
[47]Deep Learning: a theoretical introduction – Episode 2
Deep Belief Networks and Autoencoders
![Page 48: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/48.jpg)
[48]Deep Learning: a theoretical introduction – Episode 2
Auto-encoders
![Page 49: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/49.jpg)
[49]Deep Learning: a theoretical introduction – Episode 2
Auto-encoders
![Page 50: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/50.jpg)
[50]Deep Learning: a theoretical introduction – Episode 2
Deep Belief Networks and Autoencoders
![Page 51: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/51.jpg)
[51]Deep Learning: a theoretical introduction – Episode 2
The Turning Point
![Page 52: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/52.jpg)
[52]Deep Learning: a theoretical introduction – Episode 2
The Turning Point
![Page 53: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/53.jpg)
[53]Deep Learning: a theoretical introduction – Episode 2
What happened after that
![Page 54: Deep Learning: a theoretical introduction · Deep Learning: a theoretical introduction –Episode 2 [3] Deep Belief Networks and Autoencoders](https://reader033.vdocuments.net/reader033/viewer/2022060522/605114fdb3ec3d7e206ae26d/html5/thumbnails/54.jpg)
[54]Deep Learning: a theoretical introduction – Episode 2
What happened after that