pdp: motivation, basic approach
DESCRIPTION
PDP: Motivation, basic approach. Cognitive psychology or “How the Mind Works”. Information processing. Mental representations. Transformations. Perception / sensation. Action. Key questions:. What are the mental representations? What are the transformations and how do they work? - PowerPoint PPT PresentationTRANSCRIPT
PDP: Motivation, basic approach
Cognitive psychology
or
“How the Mind Works”
Information processing
Perception / sensation Action
Transformations Mental representations
Key questions:
• What are the mental representations?• What are the transformations and how do they work?• Where do these come from?
Answers circa 1979
• Mental representations are data structures (symbols bound together in relational structures)
• “Transformation” processes are rules, like the instructions in a computer program.
• The rules and starting representations are largely innate• Learning is a process of building up new structured
representations from primitives and deriving (and storing) new rules.
Why were (are?) these ideas so appealing?
EG modular, serial processing in word recognition…
Their application is highly intuitive, logical, maybe even obvious!
Explain generalization / generativity, central to human cognition!
EG: Today I will frump the car, yesterday I….______________ the car.
EG: Colorless green ideas sleep furiously!
Can build mechanistic, implemented models of behavior…
Explains modular cognitive impairments!
Pure alexia: Word representations gone!
Prosopagnosia: Face reps gone!
Category-specific impairment: Animals gone!
Broca’s aphasia: Phonological word forms gone!
And so on…
So why isn’t this a class on symbolic cognitive modeling?
W O R K
EG: Today I will meep the car, yesterday I….______________ the car.
Learning and development…
Cognitive impairments not so modular…
Pure alexia: Can’t see high spatial frequencies.
Prosopagnosia: Abnormal visual word recognition.
Category-specific impairment: Can still name half the animals.
Broca’s aphasia: Can still produce single concrete nouns, can’t do grammar.
And so on…
But also Had a lot of Other Problems…
Some issues with modular, serial, symbolic approaches• Constraint satisfaction, context sensitivity• Quasi regularity• Learning and developmental change• Graceful degradation
Rumelhart
• Brains are different than serial digital computers.• Brain:
• Neurons are like little information processors• They are slow and intrinsically noisy…• …but there are many of them and they behave in parallel, not in serial.• It turns out that noisy, parallel computing systems are naturally suited to
some of the kinds of behavioral tasks that challenged symbolic theories of the time.
In other words…
• Rather than thinking of the mind as some kind of unconstrained “universal computer,” maybe we should pay attention to the kinds of computations a noisy, parallel, brain-like system can naturally conduct.
• Paying attention to the implementation might offer some clues about / constraints on theories about how the mind works.
• Added bonus: Such theories might offer a bridge between cognition and neuroscience!
Golgi Stain
Cell body
Dendrites(receive signals)
Axon(transmits signals)
Axon Terminal
-
-
--
--
+
+
+
+
+
++ +
++
+ + + +
- - - - -----
+
--
- -
--
+
+
+
+
+
+
Membrane potential at axon hillock
DepolarizedHyperpolarized
p(Fi
ring
a sp
ike)
Input: Depends on activation of sendingneurons and efficacy of synapses
Output: Train of action potentials at a particular rate
Weight: Effect on downstream neuron
Six elements of connectionist models:
1. A set of units2. A weight matrix3. An input function4. A transfer function5. A model environment6. A learning rule
Six elements of connectionist models:
1. A set of units• Each unit is like a population of cells with a similar
receptive field.• Think of all units in a model as a single vector, with
each unit corresponding to one element of the vector.
• At any point in time, each unit has an activation state analogous to the mean firing activity of the population of neurons.
• These activation states are stored in an activation vector, with each element corresponding to one unit.
Input
Output
Hidden [1 0 .51 .52 .45]
Input
Output
Hidden
Bias
[1 0 .51 .52 .45 1]
Six elements of connectionist models:
2. A weight matrix• Each unit sends and receives a weighted connection
to/from some other subset of units.• These weights are analogous to synapses: they are
the means by which one units transmits information about its activation state to another unit.
• Weights are stored in a weight matrix
Input
Output
Hidden
Bias
[1 0 .51 .52 .45 1]
Sending
Receiving
Six elements of connectionist models:
3. An input function• For any given receiving unit, there needs to be some
way of determining how to combine weights and sending activations to determine the unit’s net input
• This is almost always the dot product (ie weighted sum) of the sending activations and the weights.
Input
Output
Hidden
Bias
[1 0 ?? ?? ?? 1]
-.2
-1.8
0.5 0.2
1.1 -2.2
.1 .2 .3 .4 .2
Sending
Receiving
Six elements of connectionist models:
Input
Output
Hidden
Bias
[1 0 ?? ?? ?? 1]
-.2
-1.8
0.5 0.22
1.1 -2.2
.1 .2 .3 .4 .2
Sending
Receiving
Six elements of connectionist models:
5. A model environment• All the models do is compute activation states over units, given
the preceding elements and some partial input.• The model environment specifies how events in the world are
encoded in unit activation states, typically across a subset of units.
• It consists of vectors that describe the input activations corresponding to different events, and sometimes the “target” activations that the network should generate for each input.
In1 In2 Out
0 0 0
0 1 1
1 0 1
1 1 0
X-OR function
Input
Output
Hidden
Bias
Input1 Input2 Hidden1 Hidden2 OutputInput1Input2Hidden1Hidden2Outputbias
Input1 Input2 Hidden1 Hidden2 OutputInput1Input2Hidden1Hidden2Outputbias
• Note that the model environment is always theoretically important!
• It amounts to a theoretical statement about the nature of the information available to the system from perception and action or prior cognitive processing.
• Many models sink or swim on the basis of their assumptions about the nature of inputs / outputs.
Six elements of connectionist models:
6. A learning rule• Only specified for models that learn (obviously)• Specifies how the values stored in the weight matrix
should change as the network processes patterns• Many different varieties that we will see:
• Hebbian• Error-correcting (e.g. backpropagation)• Competitive / self-organizing• Reinforcement-based
In1 In2 Out
0 0 0
0 1 1
1 0 1
1 1 0
X-OR function
Input
Output
Hidden
Bias
Input1 Input2 Hidden1 Hidden2 OutputInput1Input2Hidden1Hidden2Outputbias
Input1 Input2 Hidden1 Hidden2 OutputInput1Input2Hidden1Hidden2Outputbias
Central challenge
• Given just these elements, can we build little systems—models—that help us to understand human cognitive abilities, and their neural underpinnings?
One early example…