neural network tool box khaled a. al-utaibi. outlines neuron model transfer functions network...

Neural Network Tool Box

Khaled A. Al-Utaibi

ICS583: Pattern Recoginition 2009-2010 2

Outlines

Neuron Model Transfer Functions Network Architecture Neural Network Models Feed-forward Network Training & Simulation Example 1: Majority Function Example 2: Handwritten Digits

Recognition


Neuron Model


Neuron ModelInputs Weights

Bias

WeightedSum

TransferFunction Output


Transfer Functions

Many transfer functions are included in the Neural Network Toolbox

Three of the most commonly used functions are• Hard-Limit Transfer Function• Linear Transfer Function• Log-Sigmoid Transfer Function


Transfer Functions

Hard-Limit Transfer Function


Transfer Functions

Linear Transfer Function


Transfer Functions

Log-sigmoid Transfer Function


Network Architecture

Single Layer of Neurons


Network Architecture

Multiple Layers of Neurons


Neural Network Models

MATLAB contains several models of neural networks (general / special purpose):• Feed-forward back-propagation network• Elman back-propagation network• Cascade-forward back-propagation

network• Pattern recognition network• Fitting network• SOM network (Self-Organizing Map)


Feed-Forward Network

Create feed-forward back-propagation network

Many neural network models in MATLAB are special cases of this model (e.g. pattern recognition, fitting, and SOM)

Syntax

network_name = newff(arguments)



ArgumentsArgument(s) DescriptionP input vectorsT Target vectors[S1 S2 … SN-1] Size of ith layer{TF1, TF2, …, TFN} Transfer function of ith layerBTF Bp network training functionBLF Bp weight/bias learning functionIPF Input processing functions. OPF Output processing functionsDDF Data division function



Output layer size is determined from T Input and output processing functions

transform the inputs and outputs into a better form for network use:• Re-encode unknown input/output values

into numerical values• Remove redundant inputs and outputs

vectors.• Normalizes input/output values.



MATLAB provides several data division functions that divide the input data into three sets (using different strategies and different percentage for each set:• Training Set• Validation Set• Testing Set


Training the Network

Syntax

Arguments[ret_vals] = train(arguments)

Argument(s) Descriptionnet Neural network to be trainedP Network inputsT Network targetsPi Initial input delay conditionsAi Initial layer delay conditions


Training the Network

Returned Values

Argument(s) Descriptionnet Trained neural networkP Training record (iter & performance)Y Network outputsE Network errorsPf Final input delay conditionsAf Final layer delay conditions


Simulating the Network

Syntax

Arguments[ret_vals] = sim(arguments)

Argument(s) Descriptionnet Neural network to be simulatedP Network inputsPi Initial input delay conditionsAi Initial layer delay conditionsT Network targets


Simulating the Network

Returned Values

Argument(s) DescriptionY Network outputsPf Final input delay conditionsAf Final layer delay conditionsE Network errorsPerf Network performance


Example 1: Majority Function

P1 P2 P3 T0 0 0 00 0 1 00 1 0 00 1 1 11 0 0 01 0 1 11 1 0 11 1 1 1

% initialize network inputsinputs = [ 0 0 0 0 1 1 1 1; ... 0 0 1 1 0 0 1 1; ... 0 1 0 1 0 1 0 1]; % initialize network targetstargets = [0 0 0 1 0 1 1 1];


Example 1: Majority Function% initialize network inputsinputs = [ 0 0 0 0 1 1 1 1; ... 0 0 1 1 0 0 1 1; ... 0 1 0 1 0 1 0 1]; % initialize network targetstargets = [0 0 0 1 0 1 1 1]; % create a feed-froward network with a hidden% layer of 3 neuronsnet = newff(inputs, targets, 3, ...{'logsig', 'purelin'}); % train the networknet = train(net,inputs,targets); % simulate the networkoutputs = sim(net,inputs);


Example 2: Handwritten Digits Recognition

Given a set of 1000 samples of different handwritten digits (0,1, …, 9),

Each digit is represented as a binary image of size 28x28 pixels



We would like to use MATLAB Neural Network Toolbox to design a neural network to recognize handwritten digits

Pattern recognition network (newpr) is suitable for this purpose



% initialize network inputsinputs = [0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0; 0 0 0 0 0 0 1 1 1 0 0 1 0 1 0 0 1 1 1 0 0 0 0 0 0];



% initialize network targetsinputs = [ 0 1; ... 1 0; ... 0 0; ... 0 0; ... 0 0; ... 0 0; ... 0 0; ... 0 0; ... 0 0];


Questions

neural network tool box khaled a. al-utaibi. outlines neuron model transfer functions network...

Documents