arti cial neural networks and other learning systems filecourse outline exam introduction to anns...
TRANSCRIPT
History and use of ANNsCourse outline
ExamIntroduction to ANNs
Artificial neural networks and other learningsystems
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
1 History and use of ANNsHistoryComponentsUse of ANNsNo free lunch
2 Course outlineLecturesLabsLab review
3 Exam
4 Introduction to ANNsCharacteristicsInspiration from biologyCurrent trends
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
HistoryComponentsUse of ANNsNo free lunch
1 History and use of ANNsHistoryComponentsUse of ANNsNo free lunch
2 Course outlineLecturesLabsLab review
3 Exam
4 Introduction to ANNsCharacteristicsInspiration from biologyCurrent trends
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
HistoryComponentsUse of ANNsNo free lunch
History
Artificial neural network, connectionist network, neurocomputingneuronnat, neuralt nat
Early 40’s first nets1969 Minsky and Paperts critisizm80’s revival with Hopfield and Backprop90’s incorporated into Machine Learning
Early neuromime, Bell
Labs. One neuron with
5 excitatory inputs and
1 inhibitory input, 1
output.
BellCore chip. 32
neurons, 496 synapses,
100.000 patterns/ sec.
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
HistoryComponentsUse of ANNsNo free lunch
Components
topology
nodes
activation function
learning rule
assumptions on data in
type of data out
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
HistoryComponentsUse of ANNsNo free lunch
Use of ANNs
Classification
Pattern recognition
Diagnosis
Signal processing
Data coding
Clustering
Input-output mapping (associator)
Interpolation/generalization
Process modeling
Optimization with soft constraints
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
HistoryComponentsUse of ANNsNo free lunch
No free lunch
The no free lunch theorem states that averaging over all possibleworlds (data sets/problems), there is no single optimal algorithm.But, given a specific domain some algorthms performs better thanother.
Get to know your data by plotting it, do standardstatistical analysis (measure mean, standard deviation,PCA, etc)
Understand what you want to accomplish, what is yourdesired output
Set up your strategy, steps of analysis
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
LecturesLabsLab review
1 History and use of ANNsHistoryComponentsUse of ANNsNo free lunch
2 Course outlineLecturesLabsLab review
3 Exam
4 Introduction to ANNsCharacteristicsInspiration from biologyCurrent trends
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
LecturesLabsLab review
Lectures
Feed-forward networks, Supervised learningframatkopplat nat, lararledd/overvakad inlarning
2 One layer perceptron
3 Multi layer perceptron
4 Multi layer perceptron, cont. (L1), Generalization, thesuport vector machine
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
LecturesLabsLab review
Lectures
Feed-forward, Unsupervised learning/Self-organizationoovervakad inlarning, sjalvorganisation
5 Principal component analysis, independent componentanalysis (L2)
6 Self organizing maps, vector quantization (L3)
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
LecturesLabsLab review
Lectures
Feed-back networks, recurrent networks, supervisedaterkopplade nat
7 Boltzmann machines, Hopfield nets (L4)
8 Processing of temporal data, echo state networks
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
LecturesLabsLab review
Lectures
Domain assumptions, representation etc.
8 Ensemble techniques (boosting, bagging)
9 Regularization, radial basis functions
10 Domain assumptions, representation, inversemodeling, Reinforcement learning
11 Your questions. Old exam questions
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
LecturesLabsLab review
Labs
Feed forward networks - Delta rule and Backpropagation(m+p)
Competetive learning, coding with radial basis functions(m)
Self organizing feature maps (m+p)
Hopfield networks (m+p)
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
LecturesLabsLab review
Lab review
Lab review (labbredovisning)In the lab review you have 10 minutes to convince the instructorthat you should pass the lab exam. Be careful about planning thereview so that you use the time well.You are in charge of the lab review, but we may control who talks(each participant must have knowledge of all parts of the review)and we may ask for clearifications if that is necessary.If you have points that are unclear about the lab, bring that upduring the help time and not during the review. The review is notbeside a computer, so bring all necessary material in terms ofgraphs etc.
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
LecturesLabsLab review
Lab review
Lab review (cont’d)Review main pointsWhat is the lab about? What are the main points of the lab?What are your conclusions based on these main points?What must be known before the lab work can be started?What did you learn by doing the lab?Briefly describe the lab result main points.Show results in terms of calculations, figures, tables etc.For each result, what are your conclusions?Any final questions based on your results and conclusions?
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
Exam
material from lectures, books, labs
grade A-F
look at old exams
know concepts and words, a bit of proofs
simple exercises
geometric/graphical understanding of how it works
how to select algorithm given a problem
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
CharacteristicsInspiration from biologyCurrent trends
1 History and use of ANNsHistoryComponentsUse of ANNsNo free lunch
2 Course outlineLecturesLabsLab review
3 Exam
4 Introduction to ANNsCharacteristicsInspiration from biologyCurrent trends
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
CharacteristicsInspiration from biologyCurrent trends
Characteristics
nodes
(enhet, nod)
activation function
(aktiveringsfunktion)
learning rule
(inlarningsregel)
topology
(topologi)
assumptions on data in
type of data out
ANN node
w
w
w
w
Σ ϕ
input
output
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
CharacteristicsInspiration from biologyCurrent trends
Characteristics
nodes
activation function
learning rule
topology
assumptions on data in
type of data out
activationfunction
Σ
ϕ
Σ
ϕ
Σ
ϕ
Σ
ϕ
piece wise linear sigmoid
linear threshold
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
CharacteristicsInspiration from biologyCurrent trends
Characteristics
nodes
activation function
learning rule
topology
assumptions on data in
type of data out
supervisedin
∆
desired out
correction
w out
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
CharacteristicsInspiration from biologyCurrent trends
Characteristics
nodes
activation function
learning rule
topology
assumptions on data in
type of data out
unsupervised,competitive
out
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
CharacteristicsInspiration from biologyCurrent trends
Characteristics
nodes
activation function
learning rule
topology
assumptions on data in
type of data out
Hebbian,co-active,correlated
0
1
0
1
+w
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
CharacteristicsInspiration from biologyCurrent trends
Characteristics
nodes
activation function
learning rule
topology
assumptions on data in
type of data out
one layerfeed-forward
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
CharacteristicsInspiration from biologyCurrent trends
Characteristics
nodes
activation function
learning rule
topology
assumptions on data in
type of data out
two layersfeed-forward
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
CharacteristicsInspiration from biologyCurrent trends
Characteristics
nodes
activation function
learning rule
topology
assumptions on data in
type of data out
feed-back,recurrent
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
CharacteristicsInspiration from biologyCurrent trends
Inspiration from biology
The neuron
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
CharacteristicsInspiration from biologyCurrent trends
Inspiration from biology
nerve impulse, “spikes”
threshold activation all-or-nothing
travels along the axon
fixed amplitude
activates synapses
The actionpotential
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
CharacteristicsInspiration from biologyCurrent trends
Inspiration from biology
spike releases transmitter
transmitter activates receptor
receptor opens channel
channel gives excitatory EPSP
channel gives inhibitory IPSP
cascade changes during learning
EPSPs andIPSPs
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
CharacteristicsInspiration from biologyCurrent trends
Biological neurons
many simple neurons - nodes
the nerve impulse is all-or-nothing - 1 or 0 out
many inputs and outputs per neuron - many connections
analog transmission in synapse - weight in connection
input summed in cell body - summation of input
sum determines action potential frequency - transfer fkn
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
CharacteristicsInspiration from biologyCurrent trends
Biological neurons
information processing based on local information
memory stored in weights
weights are changed according to learning rules
weights can be positive and negative
parallell processing is fast (but learning takes time)
tolerance to errors in input, in weights and in outputs
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
CharacteristicsInspiration from biologyCurrent trends
Current trends
Machine learning
Mathematical statistics
Information theory
Computability theory
But see also brain inspired algorithms like that of Jeff Hawkins(founder of Palm Computing) and his company Numenta.
Erik Fransen ANN
History and use of ANNsCourse outline
ExamIntroduction to ANNs
CharacteristicsInspiration from biologyCurrent trends
Current use of ANNs
Current contests where ANNs probably will show upCompleted contests ...Some examples of applicationsExamples of companies and projectsExamples of masters thesis (exjobb) ...Conferences about ANNs and learning systemsOrganizations
See also our course DD2431 Machine learning, 6hp, per1.
Erik Fransen ANN