what is neural network....???
TRANSCRIPT
Introduction to Neural Networks
Presented by:
Hafiz Syed Adnan Ahmed
Introduction
• Artificial Neural Network is based on the biological nervous
system as Brain
• It is composed of interconnected computing units called
neurons
• ANN like human, learn by examples
Why Artificial Neural Networks?There are two basic reasons why we are interested in building artificial neural networks (ANNs):
• Technical viewpoint: Some problems such as character recognition or the prediction of future states of a system require massively parallel and adaptive processing.
• Biological viewpoint: ANNs can be used to replicate and simulate components of the human (or animal) brain, thereby giving us insight into natural information processing. 3
Science: Model how biological neural systems, like human brain, work?
• How do we see?• How is information stored in/retrieved
from memory?• How do you learn to not to touch fire?• How do your eyes adapt to the
amount of light in the environment?• Related fields: Neuroscience,
Computational Neuroscience, Psychology, Psychophysiology, Cognitive Science, Medicine, Math, Physics. 4
Real Neural Learning
• Synapses change size and strength with experience.
• Hebbian learning: When two connected neurons are firing at the same time, the strength of the synapse between them increases.
• “Neurons that fire together, wire together.”
5
Biological Neurons• Human brain = tens of thousands
of neurons• Each neuron is connected to
thousands other neurons• A neuron is made of:
• The soma: body of the neuron• Dendrites: filaments that provide
input to the neuron• The axon: sends an output signal• Synapses: connection with other
neurons – releases certain quantities of chemicals called neurotransmitters to other neurons
6
Modeling of Brain Functions
7
Modelling a Neuron
• aj :Activation value of unit j
• wj,I :Weight on the link from unit j to unit i
• inI :Weighted sum of inputs to unit i
• aI :Activation value of unit i• g :Activation function
j
jiji aWin ,
What is an artificial neuron ?
• Definition : Non linear, parameterized function with restricted output range
1
10
n
iiixwwfy
x1 x2 x3
w0
y
Simple Neuron
X1
X2
Xn
OutputInputs
b
An Artificial Neuron
x1
x2
xn
…
Wi,1
Wi,2
…
Wi,n
n
jjjii txtwt
1, )()()(net
xi
neuron i
net input signal
synapses
output ))(()(x tnetft iii
Activation functions
0 2 4 6 8 10 12 14 16 18 200
2
4
6
8
10
12
14
16
18
20
xy
-10 -8 -6 -4 -2 0 2 4 6 8 10-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
-10 -8 -6 -4 -2 0 2 4 6 8 10-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
Linear
Logistic
Hyperbolic tangent
)exp(1
1
xy
)exp()exp(
)exp()exp(
xx
xxy
How do NNs and ANNs work?• Information is transmitted as a series of
electric impulses, so-called spikes.
• The frequency and phase of these spikes encodes the information.
• In biological systems, one neuron can be connected to as many as 10,000 other neurons.
• Usually, a neuron receives its information from other neurons in a confined area
13
Navigation of a car
• Done by Pomerlau. The network takes inputs from a 34X36 video image and a 7X36 range finder. Output units represent “drive straight”, “turn left” or “turn right”. After training about 40 times on 1200 road images, the car drove around CMU campus at 5 km/h (using a small workstation on the car). This was almost twice the speed of any other non-NN algorithm at the time.
14
15
Automated driving at 70 mph on a public highway
Camera image
30x32 pixelsas inputs
30 outputsfor steering
30x32 weightsinto one out offour hiddenunit
4 hiddenunits
Computers vs. Neural Networks
“Standard” Computers Neural Networks
one CPU highly parallelprocessing
fast processing units slow processing units
reliable units unreliable units
static infrastructure dynamic infrastructure
16
Neural Network
Input Layer Hidden 1 Hidden 2 Output Layer
Network Layers
The common type of ANN consists of three layers
of neurons: a layer of input neurons connected to
the layer of hidden neuron which is connected to
a layer of output neurons.
Architecture of ANN
• Feed-Forward networksAllow the signals to travel one way from input to
output• Feed-Back NetworksThe signals travel as loops in the network, the
output is connected to the input of the network
Comparison of Brains and Traditional Computers
• 200 billion neurons, 32 trillion synapses
• Element size: 10-6 m
• Energy use: 25W• Processing speed: 100 Hz• Parallel, Distributed• Fault Tolerant• Learns: Yes• Intelligent/Conscious:
Usually
• 1 billion bytes RAM but trillions of bytes on disk
• Element size: 10-9 m• Energy watt: 30-90W (CPU)• Processing speed: 109 Hz• Serial, Centralized• Generally not Fault Tolerant• Learns: Some• Intelligent/Conscious:
Generally No
Neural Networks (Applications)
• Face recognition• Time series prediction• Process identification• Process control• Optical character recognition• Adaptative filtering• Etc…
And Finally….
“If the brain were so simple that we could understand it then we’d be so simple that
we couldn’t”
Introduction is End of Neural Networks