191123 deeplearning day1 copy - blog.kakaocdn.net

딥-러닝딥러닝,�첫�번째�날�

2019.11.23.

Deep�Learning

2019.11.23.�|�한이음�ICT�멘토링�|�딥러닝�특강1�|�전미정

첫�번째�날

딥러닝�이론�

케라스�

Azure�Cloud�설정�

MLP�생성�(Multi�Layer�Perceptron)

이론�

실습�

두�번째�날

이미지�분류�이론�

Azure�Cloud�GPU�활용�

cifar10�이미지�분류

이론�

실습�

실습�


딥-러닝�개론/용어를�이해한다�

MLP�모델을�이해/활용할�수�있다�

이미지�분류�문제를�해결할�수�있다


Deep�Learning�

Deep�Neural�Network�

Artificial�Neural�Network(ANN)�

Multi�Layer�Perceptron


인공지능

머신러닝

딥러닝

http://bisintek.com/science/2017/12/27/knowing-basic-artificial-intelligence/


https://miro.medium.com/max/1200/1*c_fiB-YgbnMl6nntYGBMHQ.jpeg

Data�Driven�Machine�Learning


https://www.extremetech.com/wp-content/uploads/2015/12/Brain-Machine-2-640x356.jpghttps://www.alzheimersreadingroom.com/2016/07/alzheimer-tau-protein-spreads-through-brain.html


https://blog.siodigital.com/artificial-neural-networks-and-marketing



Perceptron


Frank�Rosenblatt


인공�신경

입력1

입력2

입력3

Perceptron

가중치2

가중치1

가중치3

함수출력


출력f(x)

인공�신경

x1

x2

x3

w1

w2

w 3

Perceptron

출력


출력f(x)인공�신경

x1

x2

x3

w1

w2

w3

Perceptron

출력


f(x)

x1

x2

x3

w1

w2

w3

w1x1 + w2x2 + w3x3 + b = ∑ wnxn + b

f∑ wnxn + b

인공�신경 출력


f∑ wnxn + b ≥ θ

f∑ wnxn + b < θ

1

0

θ =threshold*


https://qbi.uq.edu.au/brain/brain-anatomy/axons-cable-transmission-neurons


x

y

1

θ

f∑ wnxn + b

Step�Function

0


f∑ wnxn + b ≥ θ

f∑ wnxn + b < θ

1

0

Linear�Binary�Classifier

θ =threshold*


https://miro.medium.com/max/1624/1*xsR57_PO8U7PB_ItLslLmA.png


0�or�10�or�1

AND�Function

AND�Gatex2

0�or�1

0�or�1


AND�Function

0 0

AND

0

0 1 0

1 0 0

1 1 1

x1 x2 f(x) y


x1

x2

1

1

0


XOR�Function

0 0

XOR

0

0 1 1

1 0 1

1 1 0

x1 x2 f(x) y


0�or�1

XOR�Function

XOR�Gatex2

0�or�1

0�or�1


x1

x2

1

1

0


0�or�1ANDx2

0�or�1

0�or�1ORx2

0�or�1

0�or�1 NOT�ANDx2

0�or�1

0�or�1

0�or�1

0�or�1


Multi�Layer�Perceptron(MLP)

x2

0�or�1

0�or�1

0�or�1


https://blog.siodigital.com/artificial-neural-networks-and-marketing



How�it�works?


Weight�Initialization�

Forward�Propagation�

Back�Propagation


0�or�11x2

0

1


Weight�Initialization

wj = [[−10,10], [20, − 20], [5, − 10]] wk = [−20,10, − 20]

x2

0

1

1

wj1

wj2

wj3

wj4

wj5

wj6

wj7

wj8

wj9


Forward�Propagation

wj = [[−10,10], [20, − 20], [5, − 10]] wk = [−20,10, − 20]

x2

0

1

10

-20

-10

1

wj1

wj2

wj3

wj4

wj5

wj6

wj7

wj8

wj9


x

y

1

θ

f∑ wnxn + b

Step�Function

0



wj = [[−10,10], [20, − 20], [5, − 10]] wk = [−20,10, − 20]

1

x20

0

0

1

10

-20

-10

1

wj1

wj2

wj3

wj4

wj5

wj6

wj7

wj8

wj9



wj = [[−10,10], [20, − 20], [5, − 10]] wk = [−20,10, − 20]

1

x20

0

0

1

10

-20

-10

1

0-20

wj1

wj2

wj3

wj4

wj5

wj6

wj7

wj8

wj9


차이�비교

wj = [[−10,10], [20, − 20], [5, − 10]] wk = [−20,10, − 20]

1

x20

0

0

1

10

-20

-10

1

0-20

wj1

wj2

wj3

wj4

wj5

wj6

wj7

wj8

wj9


1

x20

0

0

1

Back�Propagation

wj = [[−10,10], [20, − 20], [5, − 10]]

10

-20

-10

1

wk = [−20,10, − 20]

0-20

wk = [10, − 10,15]wj = [[5, − 20], [−30,10], [−15,20]]

wj1

wj2

wj3

wj4

wj5

wj6

wj7

wj8

wj9

UPDATE


Weight�Initialization�

Forward�Propagation�

비교�

Back�Propagation


차이�비교

wj = [[−10,10], [20, − 20], [5, − 10]] wk = [−20,10, − 20]

1

x20

0

0

1

10

-20

-10

1

0-20

wj1

wj2

wj3

wj4

wj5

wj6

wj7

wj8

wj9


Cost�Function


예측�값과�실제�값의�차이를�기반으로��

모델의�정확도(성능)을�판단하기�위한�함수

Cost�Function *�loss�function�

*�error�function�

*�objective�function


J(w, b) =1n

n

∑i

(h(xi)) − yi)2

mean�squared�error


w, b

J(w, b)

0

Gradient�Descent�


w, b

J(w, b)

0

Gradient�Descent�

w = w − αδ

δwJ(w)


Optimizer어떻게?


http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf


Optimizer

https://ruder.io/optimizing-gradient-descent/



Optimizer


https://www.slideshare.net/yongho/ss-79607172


Optimizer

Perceptron MLP

Weight�InitializationForward�Propagation

Back�PropagationCost�Function


Overfitting

Vanishing�Gradient


f(x)

x1

x2

x3

w1

w2

w3

w1x1 + w2x2 + w3x3 + b = ∑ wnxn + b

f∑ wnxn + b

인공�신경 출력


x

y

1

θ

f∑ wnxn + b

Step�Function

0

https://theffork.com/activation-functions-in-neural-networks/

Sigmoid�Function

https://ayearofai.com/rohan-4-the-vanishing-gradient-problem-ec68f76ffb9b

http://skymind.ai/wiki

http://skymind.ai/wiki

https://medium.com/@anishsingh20/the-vanishing-gradient-problem-48ae7f501257


Sigmoid


ReLU�Rectified�Linear�Unit


tanh


Leakey�ReLU

https://www.quora.com/Why-is-ReLU-the-most-common-activation-function-used-in-neural-networks


Overfitting

Vanishing�Gradient


Model�Capacity

x

yy = ax + b


x

yy = ax + b

y = k + ax0 + bx1 + cx2

https://youtu.be/_sz3KTyB9Lk

https://hackernoon.com/memorizing-is-not-learning-6-tricks-to-prevent-overfitting-in-machine-learning-820b091dc42

Regularization



https://i.stack.imgur.com/CewjH.png

Dropout

https://deepestdocs.readthedocs.io/en/latest/004_deep_learning_part_2/image/0042_fig0.jpg


케라스, 뭔가요?

Keras


AI�접근성과�민주화� 딥러닝의�대중화�

Keras�

François�Chollet


- 파이썬으로�구현된�high-level�deep�learning�API�

- TesnsorFlow,�Theano,�CNTK�

- 쉬운�사용법과�간단한�문법,�빠른�설계�가능


김태영의 케라스 블로그: https://tykimos.github.io/

뿔 진실

상아 거짓



김태영의 케라스 블로그: https://tykimos.github.io/

우리가 만든 모델이 진실을 말해주길…



https://t1.daumcdn.net/cfile/tistory/99BB78335993E64726


Keras

https://towardsdatascience.com/deep-learning-framework-power-scores-2018-23607ddf297a

Model�모델

블록과 함께하는 파이썬 딥러닝 케라스, (주)인스페이스 기술이사 김태영

네트워크�network

목표함수�objective�function

최적화기�optimizer

Model�모델


Network�네트워크

Objective�Function�목표함수

Optimizer�최적화기



x2

0

1

1

wj1

wj2

wj3

wj4

wj5

wj6

wj7

wj8

wj9

256

1024

512

784

1

1

0

네�트�워�크


0

1


https://keras.io/applications/

● Xception (88 MB, 126) ● VGG16 (528 MB, 23) ● VGG19 (549 MB, 26) ● ResNet50 (99 MB, 168) ● InceptionV3 (92 MB, 159) ● InceptionResNetV2 (215 MB, 572) ● MobileNet (17 MB, 88) ● DenseNet121 (33 MB, 121) ● DenseNet169 (57 MB, 169) ● DenseNet201 (80 MB, 201) ● NASNet

https://keras.io/applications/

InceptionResNet

V2

Xception VGG16 VGG19

ResNet50

InceptionV3

MobileNet

Dense Net

NASNet


예측�값과�실제�값의�차이를�기반으로��

모델의�정확도(성능)을�판단하기�위한�함수

Cost�Function *�loss�function�

*�error�function�

*�objective�function

https://keras.io/losses/�

● mean_squared_error�● mean_absolute_error�● mean_absolute_percentage_error�● mean_squared_logarithmic_error�● squared_hinge�● hinge�● categorical_hinge�● logcosh�● categorical_crossentropy�● sparse_categorical_crossentropy�● binary_crossentropy�● kullback_leibler_divergence�● poisson�● cosine_proximity


https://keras.io/losses/

mean�squared�

categorical� binary�




Optimizer

https://keras.io/optimizers/�

● SGD�● RMSprop�● Adagrad�● Adadelta�● Adam�● Adamax�● Nadam�● TFOptimizer


https://keras.io/optimizers/


https://keras.io/optimizers/�

● SGD�● RMSprop�● Adagrad�● Adadelta�● Adam�● Adamax�● Nadam�● TFOptimizer

https://keras.io/optimizers/

SGD RMSprop Adagrad Adam



Compile�컴파일




네트워크�network

목표함수�objective�function

최적화기�optimizer

Model�모델


adam

binary


sgd

categorical



𝑿 𝒀


𝑿 𝒀𝒀 '


𝑿

𝒀 ' 𝒀

𝒀𝒀 '


첫�번째�날

딥러닝�이론�

케라스�

Azure�Cloud�설정�

MLP�생성�(Multi�Layer�Perceptron)

이론�

실습�


AI�Service�On�Azure�Cloud�


DSVM

https://azure.microsoft.com/en-us/services/virtual-machines/data-science-virtual-machines//?WT.mc_id=AI-MVP-5003262

https://azure.microsoft.com/en-us/services/virtual-machines/data-science-virtual-machines/


https://azure.microsoft.com/en-us/services/virtual-machines/data-science-virtual-machines//?WT.mc_id=AI-MVP-5003262


https://azure.microsoft.com/en-us/services/virtual-machines/data-science-virtual-machines/


Before�we�run,


Resource�Group�

Resource�

Subscription�


Resource�


Region�


https://azure.microsoft.com/en-us/global-infrastructure/regions/


Region East�US


Resource�Group�

Resource�

Subscription�


Region Korea�Central


Resource�Group�

Resource�

Subscription�


Pay�As�You�Go


Resource�Group�

Resource�

Subscription�

Virtual�Machine

AzureClass2324

AI_GroupKorea�Central

East�West2


Let’s�Strat�HOL�💻


portal.azure.com

https://portal.azure.com


데이터�셋�준비

문제�정의

모델�설정

모델�훈련�/�평가


AND��Function

1번�문제�정의


0�or�10�or�1

AND�Function

AND�Gatex2

0�or�1

0�or�1


AND�Function

0 0

AND

0

0 1 0

1 0 0

1 1 1

x1 x2 f(x) y


�XOR��Function



XOR�Function

0 0

XOR

0

0 1 1

1 0 1

1 1 0

x1 x2 f(x) y


0�or�1

XOR�Function

XOR�Gatex2

0�or�1

0�or�1


�당뇨병�발병�예측�



Pima�Indian�Diabets

https://www.researchgate.net/figure/Pima-Indian-Diabetes-dataset-attributes_tbl1_325653625


Activation�Layers


은닉층�(Hidden�Layer)

ReLU Leaky�ReLu ELU

Rectifier�linear Leaky�ReLUExponential�Linear�Units


출력층�(Output�Layer)

linear sigmoid softmax

특정�값��예측

이진�클래스��예측

다중�클래스��예측


Data


batch_size

X_train Y_train

문제�몇개를�풀고�답을�확인�할�것인가?


epochs


epochs문제를�총�몇번�풀�것인가?

………


만일,�

epoch=100,000�

정확도가�100%�되지�않을까??��

😏

Overfitting


https://static1.squarespace.com/static/5213a664e4b01a5565dc90f1/t/5bc4e0c4e4966bc550291202/1546736393368/Machine+Learning+Generalization


Data

X_train

X_test

Y_train

Y_test


Data

X_train

X_test

Y_train

Y_test

X_val Y_val



epoch

loss

Loss�Function


우리도�할�수�있어요,�

딥-러닝�🥳


keras.io

https://keras.io


Keras

케라스�문서�

같은�결과가�나오도록�학습하는�방법�

30초만에�케라스�시작하기�

케라스�튜토리얼(한글)�

김태영님의�케라스�강의(한글)�

케라스�코리아(페이스북)

https://keras.io/getting-started/faq/

https://keras.io/getting-started/faq/#how-can-i-obtain-reproducible-results-using-keras-during-development

https://keras.io/#getting-started-30-seconds-to-keras

https://keraskorea.github.io

https://tykimos.github.io/lecture/

https://www.facebook.com/groups/KerasKorea/


Data�Science�Virtual�Machine

DSVM�이란?�

DSVM�DeepLearning�VM�생성하기�

Linux�및�Windows용�DSVM�소개�

DSVM�딥러닝�프레임워크�

�DSVM�데이터�수집도구�

Linux�DSVM�사용하기

https://azure.microsoft.com/en-us/services/virtual-machines/data-science-virtual-machines/?WT.mc_id=AI-MVP-5003262

https://azuremarketplace.microsoft.com/en-us/marketplace/apps/microsoft-ads.dsvm-deep-learning?tab=Overview/?WT.mc_id=AI-MVP-5003262

https://docs.microsoft.com/ko-kr/azure/machine-learning/data-science-virtual-machine/overview/?WT.mc_id=AI-MVP-5003262

https://docs.microsoft.com/ko-kr/azure/machine-learning/data-science-virtual-machine/dsvm-deep-learning-ai-frameworks/?WT.mc_id=AI-MVP-5003262

https://docs.microsoft.com/ko-kr/azure/machine-learning/data-science-virtual-machine/dsvm-tools-ingestion/?WT.mc_id=AI-MVP-5003262

https://docs.microsoft.com/ko-kr/azure/machine-learning/data-science-virtual-machine/linux-dsvm-walkthrough/?WT.mc_id=AI-MVP-5003262


What the hell is perceptron

From perceptron to deep neural nets

Neural�Networks�6:�solving�XOR�with�a�hidden�layer

Optimizing-gradient-descent

Activation�functions�in�neural�networks

DeepLearning

https://towardsdatascience.com/what-the-hell-is-perceptron-626217814f53

https://becominghuman.ai/from-perceptron-to-deep-neural-nets-504b8ff616e

http://www.apple.com

https://www.youtube.com/watch?v=kNPGXgzxoHw




내일�만나요�🕊

[email protected]

mailto:[email protected]

191123 deeplearning day1 copy - blog.kakaocdn.net

Documents