part 5 supervised hebbian learning. outline linear associator the hebb rule pseudoinverse rule...

PART 5Supervised Hebbian

Learning

Outline Linear Associator The Hebb Rule Pseudoinverse Rule Application

Hebb’s Postulate

“When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased.”

D. O. Hebb, 1949

A

B

Linear Associator

a Wp= ai w ijp jj 1=

R

=

Hebb Rule(1/2)

Hebb Rule(2/2)

Batch Operation

Performance Analysis(1/2)

Performance Analysis(2/2)

Example(1/2)

Example(2/2)

Pseudoinverse Rule(1/2)

Pseudoinverse Rule(2/2)

Relationship to the Hebb Rule

Relationship to the Hebb Rule

Example

p1

1–

11–

t1 1–= =

p2

1

11–

t2 1= =

W TP+1– 1

1– 11 1

1– 1– +

= =

P+

PT

P 1–PT 3 1

1 3

1–1– 1 1–1 1 1–

0.5– 0.25 0.25–0.5 0.25 0.25–

= = =

W T P+1– 1

0.5– 0.25 0.25–

0.5 0.25 0.25–1 0 0= = =

Wp1 1 0 01–1

1–

1–= = Wp2 1 0 011

1–

1= =

Autoassociative Memory

Tests50% Occluded

67% Occluded

Noisy Patterns (7 pixels)

Variations of Hebbian Learning

Wnew

Wold

tqpqT

+=

Wnew

Wold

tqpqT

+=

Wnew

Wold

tqpqT

Wold

–+ 1 – Wold

tqpqT

+= =

Wnew Wol d tq aq– pqT

+=

Wnew Wold aqpqT

+=

Basic Rule:

Learning Rate:

Smoothing:

Delta Rule:

Unsupervised:

Solved Problems

Solved Problems

Solved Problems

Solution:

Solved Problems

Solved Problems

Solved Problems

Solved Problems

Solved Problems

Solution:

Solved Problems

Solved Problems

Solved Problems

Solved Problems

Solved Problems

Solved Problems

Solved Problems

Solved Problems

Solution:

Solved Problems

Solved Problems

Solved Problems

Solved Problems

Solved Problems

Solved Problems

Solved Problems

Solved Problems

Solved Problems

Solved Problems

Solved Problems

Solved Problems

Solved Problems