22.10.20051 prof. pushpak bhattacharyya, iit bombay cs 621 artificial intelligence lecture 28 –...

22.10.2005 1Prof. Pushpak Bhattacharyya, IIT Bombay

CS 621 Artificial Intelligence

Lecture 28 – 22/10/05

Prof. Pushpak Bhattacharyya

Generating Function, Clustering,SOM, K- Means

22.10.05 IIT Bombay 222.10.2005 2Prof. Pushpak Bhattacharyya, IIT Bombay

Self Organization

Biological Motivation

Brain

Designated area for

specific tasks


Higher brain

Brain

Cerebellum

Cerebrum3- Layers: Cerebrum

Cerebellum

Higher brain


Maslow’s hierarchy

Search for

Meaning

Contributing to humanity

Achievement,recognition

Food,rest

survival


Higher brain ( responsible for higher needs)

Cerebrum(crucial for survival)

3- Layers: Cerebrum

Cerebellum

Higher brain


Mapping of Brain

Side areasFor auditory information processing

Back of brain( vision)


Brain is very Resilient. Lot of Redundancy.

Auditory area can compute visual task if the visual part is damaged


Left Brain and Right Brain

Dichotomy

Left Brain

Right Brain


Left Brain – Logic, Reasoning, Verbal abilityRight Brain – Emotion, Creativity

Music

Words – left Brain

Tune – Right Brain

Maps in the brain. Limbs are mapped to brain


A : A, A, , ,

O/p grid

. . . . I/p neuron


Self Organization or kohonen network fires a group of neurons instead of a single one. The group “some how” produces a “picture” of the cluster.

Fundamentally SOM is competitive learning. But weight changes are incorporated on a neighborhood. Find winner neuron, apply weight change for the winner and its “neighbors”.


Neurons on the contour are the “neighborhood” neurons.

Winner


Weight change rule for SOM

W(n+1) = W(n) + η(n) (I(n) – W(n)) P+δ(n) P+δ(n) P+δ(n)

Neighborhood is also a function of n

Learning rate is a function of n


δ(n) is a decreasing function of n

η(n) learning rate is also a decreasing function of n

0 < η(n) < η(n –1 ) <=1


PictoriallyWinner

δ(n)

. . . .

Convergence for kohonen not proved except for uni-dimension


Ref for “Neural Net part”

BP: Mcclelland and Rumelhart, Parallel and Distributed Processing- V. 1, MIT Press,1986.

Text books on Neural Network by:a) Zurada b) Haykins c) Hassoun d) Schalkoff


Theoretical foundations of learning:

Framework: Probably approximately correct learning(PAC) – L.G. Valiant


Defines learning in a formal way

False missFalse hit

Hypothesis, h

Concept , c


Error Region

C h

Goal of learning is to minimize

C h+

+


Probabilistic Notions to address the question of minimizing C h+

Assume

U – UniverseC Uh U

C h U +


Assume a probability distribution on U.

If U is discrete probability mass function

0 <= P(X belong to U) <= 1

P is the probability that an oracle present’s the point X from out of U and also gives it a label of + / - or 1/ 0


Learning performance tied to probabilityDistribution that generates and labels the points.

C

U


PAC definition of learning is

Pr(P(C h) <= Є ) >= 1 - δ +

Accuracy parameter Confidence parameter

Underlying probability distribution


Pr(P(C h) <= Є ) >= 1 - δ +

C Hypothesis - h

C h+Error region

U


Example

A B

CD

++-

-

x

y

Rectangle learning


+ from within the rectangle

- from outside the rectangle

Algorithm

1. Ignore –ve examples

2. Advance h as the closest enclosing rectangle


+++++

+

+++++

+

Error region

Target region

A

D C

B

22.10.20051 prof. pushpak bhattacharyya, iit bombay cs 621 artificial intelligence lecture 28 –...

Documents