1

Adaptive Resonance Theory

2

INTRODUCTION

Adaptive resonance theory (ART) was developed by Carpenter andGrossberg[1987a]ART refers to the class of self organizing neural architecture that

clusters the pattern space and produce appropriate weight vectorART1 – Clustering binary vectors ART2 – Accepts the continuous – valued vectorUnsupervised learningART nets are designed to be both stable an plastic ART network is a vector classifierChanges in the activation of units and weights are governed by the

differential equations

3

Contd….

Resonance period: Activations are assumed to be change much

more rapidly than the weights Once the acceptable cluster unit is selected for

learning ,the weights may be maintained over an extended period

During that period only weight changes should be done

This period is called ‘resonance period’

Basic Architecture

4

5

ART involves three group of neurons

Input processing(F1 layer) Input portion(F1(a)) Interface portion(F1(b) Comparing the similarity of the input signal to the weight

vector of the cluster unit which is selected for learning F1(b) is connected to F2 through bottom up weights bij F2 is connected to FI(b) through top down weight tij Cluster unit(F2 layer) competitive layer Cluster unit with largest net input is selected

Basic Architecture

6

Contd….

Activation of all other unit is set to zero

Reset Mechanism

Depending upon the similarity between the top-down weight and the input vector , the cluster unit may or may not be allowed to learn the pattern

If the cluster unit is not allowed to learn ,it is inhibited and a new cluster is selected as the candidate

Basic operation

Learning trial- Before the pattern is presented • Activation of all units should be zero• F2 units are made inactive • once a pattern is given to the network ,it continuously send the

input signal Controlling the degree of similarity• controlled by the vigilance parameter Reset mechanism states• Function is to control the state of each node in F2 layer

7

10to

Active- F2 unit is on activation is d=1 Inactive- F2 is off , activation=0

But available to participate in the next competition Inhibit –F2 is off, activation =0,

prevented from participation in further competition

8

Learning in ART

Fast learning

Used in ART1 network

Input is binary

Weight update occur more rapidly during resonance period

Weight reaches the equilibrium on each trial

Weight associated with cluster units are stabilized

9

Slow learning Used in ART2 network Weight changes during resonance period occur

slowly Weight does not reaches the equilibrium in each

trial Many more learning pattern is required Network will not be stablilzed

10

Basic training step

11

Contd…..

12

ART1

ART1 has Computational unit Supplemental unit

13

ART1

14

consist of Computational unit & supplemantal unit

Supplemental unit

15

parameter used inAlgorithm

16

Training algorithm

17

Step 0 : initialize parameters :

nL

Lbij

1)0(0

10

1

L

initialize weights :

1)0( jit

18Adaptive Resonance Theory NN

18

Step 1: While stopping condition is false do Steps 2-13

i

iss

Step 2: For each training input . do steps 3-12

Step 3: Set activations of all F2 units to zero.

Set activations of F1(a) units to input vector s.Step 4: Compute the norm of s:

Step 5: Send input signal from F1(a) to the F1(b) layer

ii sx

ART1 Algorithm (cont.)

19Adaptive Resonance Theory NN

19

1jyStep 6: For each F2 node that is not inhibited:

if . then

i

iijj xby

Step 7: While reset is true. do Steps 8-11.

Step 8: find J such that yJ≥yj for all nodes j.

If yJ then all nodes are inhibited and this pattern cannot be clustered.

Step 9: Recompute activation x of F1(b)

xi = sitJi

ART1 Algorithm (cont.)

20Adaptive Resonance Theory NN

20

i

ixx

s

x

Step 10: Compute the norm of vector x:

Step 11: Test for reset:

if then

yJ=-1 (inhibit node J)(and continue executing step 7 again)

If then proceed to step 12.s

x

ART1 Algorithm (cont.)

21Adaptive Resonance Theory NN

21

Step 12: Update the weight for node J (fast learning)

iJi

iij

xnewt

xL

Lxnewb

)(

1)(

Step 13: Test for stopping condition.

ART1 Algorithm (cont.)

22

23

24

25

26

27

28

29

30

`

THANK YOU

31