deep learning · machine learning meetup michal illich. michal illich. obsah ... intermezzo 1...
TRANSCRIPT
![Page 1: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/1.jpg)
Deep learning(a strojové učení obecně)
Czech Internet Forum 2014Tech Open Day
DataConf.czMachine Learning Meetup
Michal Illich
![Page 2: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/2.jpg)
Michal Illich
![Page 3: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/3.jpg)
Obsah
● Strojové učení– Kdo ho používá?
– Základní princip
● Deep learning– Běžné neuronové sítě
– Co je podstatou „deep“?
– Proč a kde je skvělý?
![Page 4: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/4.jpg)
Produkty strojového učení používáte každý den.
![Page 5: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/5.jpg)
Kde se používá na webu - I
![Page 6: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/6.jpg)
Kde se používá na webu - II
![Page 7: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/7.jpg)
Kde se používá na webu - III
![Page 8: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/8.jpg)
Mimo web
Siri
OCR
Self-driving cars
![Page 9: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/9.jpg)
Kde si s ním hrajeme my
Magictable
Brandiozo
Golem
a jeden neveřejný projekt
![Page 10: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/10.jpg)
… a Brandiozo
a 2 další, zatím neveřejné projekty
![Page 11: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/11.jpg)
Označkovanádata
![Page 12: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/12.jpg)
Naučíte model:
Známá data → Model → Známý výsledek
![Page 13: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/13.jpg)
Používáte model:
Nová data → Model → Nový výsledek
![Page 14: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/14.jpg)
II. Deep learning
![Page 15: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/15.jpg)
1970s
![Page 16: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/16.jpg)
1970s
![Page 17: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/17.jpg)
1975 - backpropagation
![Page 18: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/18.jpg)
Co chceme, aby uměly?
![Page 19: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/19.jpg)
Co je od tehdy nového?Čím je učení „deep“?
![Page 20: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/20.jpg)
1. Hloubka
Umožněná vyšším výkonem CPU/GPU a pár triky.
![Page 21: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/21.jpg)
2. Využití neoznačených dat
![Page 22: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/22.jpg)
![Page 23: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/23.jpg)
Intermezzo 1
● Semantic hashing● Aneb zpátky k Josefově problému
(hledání podobných dokumentů)
![Page 24: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/24.jpg)
Tři přístupy
1. Podle podobnosti řetězce písmen?
2. Podle podobnosti významu slov?
3. Deep learning?
![Page 25: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/25.jpg)
Salakhutdinov & Hinton, 2006
![Page 26: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/26.jpg)
3. Triky
● odolnost proti přeučení DropOut, DropConnect
● jiné aktivační funkce ReLu místo sigmoidy
● učení po vrstvách● konvoluční sítě
![Page 27: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/27.jpg)
Odolnost proti přeučeníDropOut, DropConnect, Denoising...
![Page 28: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/28.jpg)
Výsledky
Kategorizace obrázků (22000 kategorií)● 9,5% (nejlepší v 2011)
● 18,3% (deep learning ze Stanfordu)
![Page 29: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/29.jpg)
Výsledky
Rozpoznávání řeči (Switchboard, 300 hodin)● chybovost 27,4 % (tradiční metody)
● chybovost 18,5 % (deep learning)
![Page 30: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/30.jpg)
Výsledky
Rozpoznávání číslic (MNIST)● chybovost 0,39 % (2006)
● chybovost 0,21 % (deep learning)
![Page 31: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/31.jpg)
Open source knihovny
Neuronové sítě:
theano, cuda-convnet, fann
Další algoritmy:
gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans
word2vec, gensim, NLP knihovny
Nástroje s GUI:
weka, orange, rapid miner
![Page 32: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/32.jpg)
Intermezzo 2
![Page 33: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/33.jpg)
Máte projekt?
● Z oblasti dat nebo analytics?● Nejméně dva zakladatelé?● Potřebujete €30k?
![Page 34: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/34.jpg)
Přihlašte se do StartupYardu
● Deadline už za 4 týdny 15.12.
● Tříměsíční program od března 2015
● Desítky mentorů● Demo Day pro investory● €30k cash investice za 10%● Plus až $250k v „perks“
kredit do AWS, Rackspace, Google App Engine,...
![Page 35: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/35.jpg)
Závěr
![Page 36: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/36.jpg)
1/3
Mít data nestačí. Používejte je.kreslit barevné grafy je základ, nikoliv plné využití dat
![Page 37: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/37.jpg)
Strojové učení.nechte stroje samotné najít si vztahy v datech
2/3
![Page 38: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/38.jpg)
3/3
Kde může student pracovat s ML na reálných problémech:- komunitní/soutěžní weby jako Kaggle.com- vlastní projekty – sami nebo v akcelerátoru- na univerzitě- v pár českých firmách
![Page 40: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/40.jpg)
Apendix: Výsledky podrobněji
Kategorizace obrázků (22000 kategorií)● 9,5% (nejlepší v 2011, Weston+Bengio)● 18,3% (deep learning ze Stanfordu)
Rozpoznávání řeči (Switchboard, 300h)● chybovost 27,4 % (GMM-HMM)● chybovost 18,5 % (deep learning: DNN-HMM)
Rozpoznávání číslic (MNIST)● chybovost 0,39 % (energy based model, 2006)● chybovost 0,21 % (deep learning s DropConnectem)
![Page 41: Deep learning · Machine Learning Meetup Michal Illich. Michal Illich. Obsah ... Intermezzo 1 Semantic hashing ... gbm, libsvm, vowpal wabbit, sofia-ml, sofia-kmeans word2vec, gensim,](https://reader033.vdocuments.net/reader033/viewer/2022050209/5f5b99ad62aedd1c2c55d019/html5/thumbnails/41.jpg)
Zdroje ilustračních obrázků:
http://www.cs.toronto.edu/~kriz/cifar.htmlhttp://cl.naist.jp/~kevinduh/notes/duh12deeplearn.pdfhttp://www.positscience.com/media-gallery/detail/161/94http://commons.wikimedia.org/wiki/File:Haystack_-_geograph.org.uk_-_462934.jpghttp://www.retrowaste.com/1970s/http://www.moderni-dejiny.cz/clanek/exteriery-z-obdobi-normalizace-soubor-fotografii-bohdana-holomicka/http://zlin.cz/503516n-zlinske-jaro-odstartovalo-vystavu-ritualy-normalizacehttp://www.cs.stanford.edu/people/ang//slides/DeepLearning-Mar2013.pptx
+ pozadí koupené ve fotobance
Semantic hashing: https://www.cs.toronto.edu/~hinton/absps/sh.pdf