project: back propagation algorithmmcl.korea.ac.kr/.../11/back-propagation-algorithm-matlab.pdf ·...
TRANSCRIPT
Project: Back Propagation Algorithm
2016. 11. 9
• Data prepare
• Cell / Struct
• Feed Forward
• Training schema
• Back Propagation / Weight update
• Result
• Load data
Data prepare
Premable
Clear all data
Load form
‘mnist_uint8_matlab.mat’
MNIST dataset (each row means 1 data)
Test data: 10000
Trainig data: 50000
Input size: 28 × 28 size image
Output size: 10 × 1 size label
• Input & output data
Data prepare
Output
Data
Input
Data
• Input & output data• Test_x(10000 × 784) → one data (1 × 784) → reshape (28 × 28)
Data prepare
2 1 9
0 1 2 3 4 5 6 7 8 9
• Data type of Matlab
Cell / Struct
Most used types Included in the code
• Cell• Array of indexed cells, each capable of storing array of differen
dimension and data type
• Struct• C-like structure, named fields capable of storing of different
dimension and data type
Cell / Struct
𝑎. 𝑛𝑢𝑚 = 12; 𝑎. 𝑐𝑜𝑙𝑜𝑟 =′ 𝑟𝑒𝑑′; 𝑎.𝑚𝑎𝑡 = 𝑜𝑛𝑒𝑠 4 ;
𝑎 = 𝑐𝑒𝑙𝑙 1,3 ; 𝑎 1,1 = 10; 𝑎 1,2 =′ 𝑎𝑏𝑐′ 𝑎 1,3 = 𝑜𝑛𝑒𝑠 4 ;
• 3 struct: init, training, net• init: for initialize weight and bias
• training: for training schema
• net: all information of network (num of layer&neuron, layer, weight, bias …)
Cell / Struct
• Toy example• 3 layer (2, 3, 2 neurons) network
Cell / Struct
• initialize_network.m
Cell / Struct
3 layer
neuron number: [784, 10, 10]
3 layer
neuron number: [2, 3, 2]
Initialize each layer
by zero
net.layer
𝟐 × 𝟏
𝟑 × 𝟏
𝟐 × 𝟏
• initialize_network.m
Cell / Struct
net.weight
[ ]
𝟑 × 𝟐
𝟐 × 𝟑
net.bias
[ ]
𝟑 × 𝟏
𝟐 × 𝟏
• initialize_network.m
Cell / Struct
𝒏𝟏𝟏
𝒏𝟐𝟏
𝒏𝟏𝟐
𝒏𝟐𝟐
𝒏𝟑𝟐
𝒏𝟏𝟑
𝒏𝟐𝟑
net.layer
𝟐 × 𝟏
𝟑 × 𝟏
𝟐 × 𝟏
net.weight
[ ]
𝟑 × 𝟐
𝟐 × 𝟑
net.bias
[ ]
𝟑 × 𝟏
𝟐 × 𝟏
𝑛11
𝑛21
𝑛12
𝑛22
𝑛32
𝑤1,12
𝑤2,12
𝑤3,12
𝑤1,12 𝑤1,2
2
𝑤2,12
𝑤3,12
𝑤2,22
𝑤3,22
𝑏12
𝑏22
𝑏32
𝑏12
𝑏22
𝑏32
• feed_forward.m
Feed foward
𝑛12
𝑛22
𝑛32
= logistic
𝑤1,12 𝑤1,2
2
𝑤2,12
𝑤3,12
𝑤2,22
𝑤3,22
𝑛11
𝑛21 +
𝑏12
𝑏22
𝑏32
𝒏𝟏𝟏
𝒏𝟐𝟏
𝒏𝟏𝟐
𝒏𝟐𝟐
𝒏𝟑𝟐
𝒏𝟏𝟑
𝒏𝟐𝟑
𝑛13
𝑛23 = logistic
𝑤1,13 𝑤1,2
3 𝑤1,33
𝑤2,13 𝑤2,2
3 𝑤2,33
𝑛12
𝑛22
𝑛32
+𝑏13
𝑏23
• training
Training schema
• back_propagation.m• Calculate error in each layer & neuron
• Calculate delta in each layer & neuron
• weight_update.m• Update weight and bias of each neuron by using delta
Back Propagation / Weight update
𝒏𝟏𝟏
𝒏𝟐𝟏
𝒏𝟏𝟐
𝒏𝟐𝟐
𝒏𝟑𝟐
𝒏𝟏𝟑
𝒏𝟐𝟑
• Result file• my_result.mat
• performance.fig
• performance.png
Result