machine learning and azure machine learning

30
MACHINE LEARNING & AZURE ML SERVICES

Upload: alexander-konduforov

Post on 15-Jul-2015

1.763 views

Category:

Data & Analytics


5 download

TRANSCRIPT

Page 1: Machine Learning and Azure Machine Learning

MACHINE LEARNING & AZURE ML SERVICES

Page 2: Machine Learning and Azure Machine Learning

ОБО МНЕ

Александр КондуфоровData Science Group Leader,

Software Architect @ AltexSoft

В прошлом – .NET разработчик, Software Architect,

Project Lead

Kharkov AI club

Page 3: Machine Learning and Azure Machine Learning

СОДЕРЖАНИЕ

1. Что такое Machine Learning, зачем он нужен

2. Примеры применений ML

3. Типы обучения

4. Популярные ML-алгоритмы

5. Supervised Learning workflow

6. ML в .NET

7. Обзор Azure ML Services

8. Azure ML demo

Page 4: Machine Learning and Azure Machine Learning

MACHINE LEARNING

“Field of study that gives computers the ability to learn without being explicitly programmed”

Arthur Samuel

Page 5: Machine Learning and Azure Machine Learning

Для некоторых задач невозможно написать алгоритм

Page 6: Machine Learning and Azure Machine Learning

КОГДА НЕ ОБОЙТИСЬ БЕЗ ML

• Человек не может объяснить свой опыт и создать алгоритм –

распознавание образов, голоса, координация движений и т.д.

• Человеческий опыт не существует как таковой – навигация на

Марсе

• Обстановка меняется со временем и нужно адаптироваться –

биометрия людей, биржевые индексы, цены на недвижимость

Page 7: Machine Learning and Azure Machine Learning

ТИПЫ ОБУЧЕНИЯ

• Supervised Learning

• Semi-supervised Learning

• Unsupervised Learning

• Hybrid (Supervised + Unsupervised)

• Reinforcement Learning

• Online Learning

Page 8: Machine Learning and Azure Machine Learning

SUPERVISED LEARNING

Данные – (input, correct output)

Page 9: Machine Learning and Azure Machine Learning

UNSUPERVISED LEARNING

Данные – (input, ???)

Page 10: Machine Learning and Azure Machine Learning

REINFORCEMENT LEARNING

• Роботы

• Игровой AI

Page 11: Machine Learning and Azure Machine Learning

ОСНОВНЫЕ ТИПЫ ЗАДАЧ

Supervised learning

• Классификация

• Регрессия

Unsupervised learning

• Кластеризация

• Определение выбросов

Гибридные

• Коллаборативная фильтрация (рекомендации)

Page 12: Machine Learning and Azure Machine Learning

ПОПУЛЯРНЫЕ АЛГОРИТМЫ

• Регрессия

• Деревья решений

• Искусственные нейронные сети

• k-means кластеризация

• Collaborative Filtering

Page 13: Machine Learning and Azure Machine Learning

РЕГРЕССИЯ

Y = b0+b1x1+b2x2+…+bkxk Y = b0+b1x+b2x2+b3x3

Page 14: Machine Learning and Azure Machine Learning

DECISION TREE / RANDOM FOREST

Создание дерева на основании данных и определение

условий в каждом узле дерева.

Page 15: Machine Learning and Azure Machine Learning

ИСКУССТВЕННЫЕ НЕЙРОННЫЕ СЕТИ

Построены на базе нейронов мозга

Множество видов:

• многослойный перцептрон

• сеть Хопфилда

• сеть Кохонена

• неокогнитрон

• спайковые сети

• сверточные сети

Page 16: Machine Learning and Azure Machine Learning

K-MEANS КЛАСТЕРИЗАЦИЯ

Page 17: Machine Learning and Azure Machine Learning

COLLABORATIVE FILTERING

User-based recommendation:

Page 18: Machine Learning and Azure Machine Learning

SUPERVISED LEARNING WORKFLOW

• Data analysis and preprocessing (cleaning, transformation)

• Feature engineering (new, correlated)

• Fit selected model (training data)

• Evaluate model (test data, choose error type)

• Change data, algorithm and model parameters until

satisfied

• Use model for predictions

Page 19: Machine Learning and Azure Machine Learning

GENERALIZATION VS. MEMORIZATION

Overfitting – features of the model that arise from relations that are in

the training data, but not representative of the general population.

Page 20: Machine Learning and Azure Machine Learning

BIAS-VARIANCE TRADEOFF

Page 21: Machine Learning and Azure Machine Learning

VALIDATION / CROSS-VALIDATION

Validation:

60%/40% 50%/50% 80%/20%

Cross-validation:

• K-fold CV

• Leave-one-out CV

Page 22: Machine Learning and Azure Machine Learning

CLASSIFICATION ERRORS

Page 23: Machine Learning and Azure Machine Learning

ML В .NET

• Infer.NET (MS Research)

• Accord.NET

• AForge

• Numl

•Math.NET Numerics

• Azure ML Services

Page 24: Machine Learning and Azure Machine Learning

AZURE ML SERVICES

Cloud-based predictive analytics from Microsoft

Page 25: Machine Learning and Azure Machine Learning

ОСНОВНЫЕ ВОЗМОЖНОСТИ

• ML Studio

• Built-in data preprocessing and ML algorithms

• Integration with Azure data storages and other services

• Plots for data analysis

• R support

• Fast conversion to web-service

Page 26: Machine Learning and Azure Machine Learning

ЦЕНЫ

Page 27: Machine Learning and Azure Machine Learning

РЫНОК АЛГОРИТМОВ И ДАННЫХ

Page 28: Machine Learning and Azure Machine Learning

TIME FOR SOME DEMO

Page 29: Machine Learning and Azure Machine Learning

ПОЛЕЗНЫЕ ССЫЛКИ

• http://azure.microsoft.com/en-us/services/machine-

learning/

• https://www.coursera.org/courses?orderby=upcoming&

cats=cs-ai

• http://bit.ly/1zTTKrF - Data Science Dojo tutorials!

• http://kaggle.com

Page 30: Machine Learning and Azure Machine Learning

СПАСИБО !

• Blog: http://merle-amber.blogspot.com

• Kharkiv AI Club: http://aikharkov.wordpress.com/

• Skype: alex_konduforov

• @konduforov