eddy li eric wong martin ho kitty wong introduction to
TRANSCRIPT
![Page 1: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/1.jpg)
Eddy Li
Eric WongMartin Ho
Kitty Wong
Introduction to
![Page 2: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/2.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Presentation Outline
1) Introduction to Neural Networks
2) What is Neural Networks?
3) Perceptrons
4) Introduction to Backpropagation
5) Applications of Neural Networks
6) Summary
![Page 3: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/3.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Background
- Neural Networks can be :
- Biological models
- Artificial models
- Desire to produce artificial systems capable of sophisticated computations similar to the human brain.
![Page 4: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/4.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
How Does the Brain Work ? (1)
NEURON
- The cell that perform information processing in the brain.
- Fundamental functional unit of all nervous system tissue.
![Page 5: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/5.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Each consists of :SOMA, DENDRITES, AXON, and SYNAPSE.
How Does the Brain Work ? (2)
![Page 6: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/6.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Brain vs. Digital Computers (1)
- Computers require hundreds of cycles to simulate a firing of a neuron.
- The brain can fire all the neurons in a single step. Parallelism
- Serial computers require billions of cycles to perform some tasks but the brain takes less than a second.
e.g. Face Recognition
![Page 7: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/7.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Brain vs. Digital Computers (2)
Future : combine parallelism of the brain with the switching speed of the computer.
![Page 8: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/8.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Definition of Neural Network
A Neural Network is a system composed of
many simple processing elements operating in
parallel which can acquire, store, and utilize
experiential knowledge.
![Page 9: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/9.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Neurons vs. Units (1)
- Each element is a node called unit.
- Units are connected by links.
- Each link has a numeric weight.
![Page 10: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/10.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Neurons vs. Units (2)
![Page 11: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/11.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Computing Elements
A typical unit:
![Page 12: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/12.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Presentation Outline
1) Introduction to Neural Networks
2) What is Neural Networks?
3) Perceptrons
4) Introduction to Backpropagation
5) Applications of Neural Networks
6) Summary
![Page 13: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/13.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Planning in building a Neural Network
Decisions on:
- The number of units to use.
- The type of units required.
- Connection between the units.
![Page 14: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/14.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Learning a Task
- Initializing the weights.
- Use of a learning algorithm.
- Set of training examples.
- Encode the examples as inputs.
- Convert output into meaningful results.
![Page 15: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/15.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Neural Network Example
![Page 16: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/16.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Simple Computations
- 2 components: Linear and Non-linear.
- Linear: Input function- calculate weighted sum of all inputs.
- Non-linear: Activation function- transform sum into activation level.
![Page 17: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/17.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Calculations
Input function:
Activation function:
![Page 18: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/18.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
A Computing Unit
![Page 19: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/19.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Activation Functions
- Use different functions to obtain different models.
- 3 most common choices :
1) Step function2) Sign function3) Sigmoid function
- An output of 1 represents firing of a neuron down the axon.
![Page 20: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/20.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
3 Activation Functions
![Page 21: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/21.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Units in Action
- Individual units representing Boolean functions
![Page 22: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/22.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Network Structures
Feed-forward :
Links can only go in one direction.
Recurrent :
Links can go anywhere and form arbitrarytopologies.
![Page 23: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/23.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Feed-forward Networks
- Arranged in layers.
- Each unit is linked only in the unit in next layer.
- No units are linked between the same layer, back to the previous layer or skipping a layer.
- Computations can proceed uniformly from input to output units.
- No internal state exists.
![Page 24: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/24.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Feed-Forward Example
I2
t = -0.5W24= -1
H4
W46 = 1 t = 1.5
H6
W67 = 1
t = 0.5
I1
I1
t = -0.5W13 = -1
H3
W35 = 1t = 1.5
H5
O7
W57 = 1
W25 = 1
W16 = 1
![Page 25: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/25.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Multi-layer Networks and Perceptrons
- Have one or more layers of hidden units.
- With two possibly very large hidden layers, it is possible to implement any function.
- Networks without hidden layer are called perceptrons.
- Perceptrons are very limited in what they can represent, but this makes their learning problem much simpler.
![Page 26: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/26.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Recurrent Network (1)
- The brain is not and cannot be a feed-forward network.
- Allows activation to be fed back to the previous unit.
- Internal state is stored in its activation level.
- Can become unstable or oscillate.
![Page 27: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/27.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Recurrent Network (2)
- May take long time to compute a stable output.
- Learning process is much more difficult.
- Can implement more complex designs.
- Can model certain systems with internal states.
![Page 28: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/28.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Presentation Outline
1) Introduction to Neural Networks
2) What is Neural Networks?
3) Perceptrons
4) Introduction to Backpropagation
5) Applications of Neural Networks
6) Summary
![Page 29: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/29.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Perceptrons
- First studied in the late 1950s.
- Also known as Layered Feed-Forward Networks.
- The only efficient learning element at that time was for single-layered networks.
- Today, used as a synonym for a single-layer, feed-forward network.
![Page 30: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/30.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Perceptrons
![Page 31: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/31.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
What can Perceptrons Represent ?
- Some complex Boolean function can be represented. For example: Majority function - will be covered in the next presentation.
- Limited in the Boolean functions they can represent.
![Page 32: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/32.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Linear Separability in Perceptrons
![Page 33: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/33.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Learning Linearly Separable Functions (1)
What can these functions learn ?
Bad news:- There are not many linearly separable functions.
Good news:- There is a perceptron algorithm that will learn any linearly separable function, given enough training examples.
![Page 34: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/34.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Most neural network learning algorithms, including the
perceptrons learning method, follow the current-best-
hypothesis (CBH) scheme.
Learning Linearly Separable Functions (2)
![Page 35: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/35.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
- Initial network has a randomly assigned weights.
- Done by making small adjustments in the weights to reduce the difference between the observed and predicted values.
- Main difference from the logical algorithms is need to repeat the update phase several times in order to achieve convergence.
- Updating process is divided into epochs, each epoch updates all the weights of the process.
Learning Linearly Separable Functions (3)
![Page 36: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/36.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Neural-Network-Learning
![Page 37: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/37.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Presentation Outline
1) Introduction to Neural Networks
2) What is Neural Networks?
3) Perceptrons
4) Introduction to Backpropagation
5) Applications of Neural Networks
6) Summary
![Page 38: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/38.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Introduction to Backpropagation
- In 1969 a method for learning in multi-layer network, Backpropagation, was invented by Bryson and Ho.
- The Backpropagation algorithm is a sensible approach for dividing the contribution of each weight.
- Works basically the same as perceptrons
![Page 39: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/39.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Backpropagation Learning
There are two differences for the updating rule :
1) The activation of the hidden unit is used instead of the input value.
2) The rule contains a term for the gradient of the activation function.
![Page 40: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/40.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Backpropagation Algorithm(1)
![Page 41: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/41.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Backpropagation Algorithm(2)
The ideas of the algorithm can be summarized as follows :
- Computes the error term for the output units using the observed error.
- From output layer, repeat propagating the error term back to the previous layer and updating the weights between the two layers until the earliest hidden layer is reached.
![Page 42: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/42.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Examples of Backpropagation Learning
![Page 43: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/43.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Examples of Feed-Forward Learning
![Page 44: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/44.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Examples of Backpropagation Learning
![Page 45: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/45.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Presentation Outline
1) Introduction to Neural Networks
2) What is Neural Networks?
3) Perceptrons
4) Introduction to Backpropagation
5) Applications of Neural Networks
6) Summary
![Page 46: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/46.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Neural Network Applications
- May provide a model for massive parallel computation.
- More successful approach of “parallelizing” traditional serial algorithms.
- Can compute any computable function.
- Can do everything a normal digital computer can do.
- Can do even more under some impractical assumptions.
![Page 47: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/47.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Neural Network in Practice
For classification and function approximation or mappingproblems which are:
- Tolerant of some imprecision.
- Have lots of training data available.
- Hard and fast rules cannot easily be applied.
![Page 48: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/48.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Neural Network Approaches
ALVINN - Autonomous Land Vehicle In a Neural Network
![Page 49: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/49.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Neural Network Approaches
- Developed in 1993.
- Performs driving with Neural Networks.
- An intelligent VLSI image sensor for road following.
- Learns to filter out image details not relevant to driving.
Hidden layer
Output units
Input units
![Page 50: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/50.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Neural Network Approaches
Hidden Units Output unitsInput Array
![Page 51: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/51.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Actual Products Available
Enterprise Miner:
- Single multi-layered feed-forward neural networks.- Provides business solutions for data mining.
Nestor:
- Uses Nestor Learning System (NLS).- Several multi-layered feed-forward neural networks.- Intel has made such a chip - NE1000 for VLSI.
![Page 52: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/52.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Enterprise Miner
- Based on SEMMA (Sample, Explore, Modify, Model, Access) methodology.
- Statistical tools include :Clustering, decision trees, linear and logisticregression and neural networks.
- Data preparation tools include :Outliner detection, variable transformation, random sampling, and partition of data sets (into training,testing and validation data sets).
![Page 53: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/53.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Nestor
- With low connectivity within each layer.
- Minimized connectivity within each layer results in rapid training and efficient memory utilization, ideal for VLSI.
- Composed of multiple neural networks, each specializing in a subset of information about the input patterns.
- Real time operation without the need of special computers or custom hardware.
![Page 54: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/54.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Presentation Outline
1) Introduction to Neural Networks
2) What is Neural Networks?
3) Perceptrons
4) Introduction to Backpropagation
5) Applications of Neural Networks
6) Summary
![Page 55: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/55.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Summary
- Neural network is a computational model that simulate some properties of the human brain.
- The connections and nature of units determine the behavior of a neural network.
- Perceptrons are feed-forward networks that can only represent linearly separable functions.
![Page 56: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/56.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
Summary
- Given enough units, any function can be represented by Multi-layer feed-forward networks.
- Backpropagation learning works on multi-layer feed-forward networks.
- Neural Networks are widely used in developing artificial learning systems.
![Page 57: Eddy Li Eric Wong Martin Ho Kitty Wong Introduction to](https://reader036.vdocuments.net/reader036/viewer/2022062318/551b3dbb550346ae7a8b4875/html5/thumbnails/57.jpg)
Presented by Martin Ho, Eddy Li, Eric Wong and Kitty Wong - Copyright© 2000
References
- Russel, S. and P. Norvig (1995). Artificial Intelligence - A Modern Approach. Upper Saddle River, NJ, Prentice Hall.- Sarle, W.S., ed. (1997), Neural Network FAQ, part 1 of 7: Introduction, periodic posting to the Usenet newsgroup comp.ai.neural-nets, URL: ftp://ftp.sas.com/pub/neural/FAQ.html