cs 182 lecture 28: neuroeconomics j.g. makin april 27, 2006
Post on 22-Dec-2015
222 views
TRANSCRIPT
CS 182Lecture 28: Neuroeconomics
J.G. Makin
April 27, 2006
Decisions, Uncertainty, and the BrainPaul Glimcher (2003); MIT Press
• Thesis: neuroscience has been dominated by the reflex paradigm
• Alternative: investigations rooted in economics, evolution, game theory, and probability
Reflex Theory
• Model: Input-Association-Output
– (think of trying to explain language this way)
– Even ANNs?
• Methodology: thoroughly constrain the environment
– “Isn’t this how science is done?”
– Obscures a system-level view
• Has this really led researchers astray?
• Why are there so many questions on this slide?
Glimcher 2003
Reflex Theory (con’t)
Challenges to “naïve” reflex theory
• T. Graham Brown and the Half-Center Oscillators [This is not the name of a band, as far as I know, though it should be]– Sherrington: stimulus for walking from
enteroceptive or interoceptive sources only
• Reafference and Efference Copy (Von Holst and Mittelstaedt)– [Glimcher actually has these confused]
Reflex Theory (con’t)
Challenges to “naïve” reflex theory
• T. Graham Brown and the Half-Center Oscillators [This is not the name of a band, as far as I know, though it should be]– Sherrington: stimulus for walking from
enteroceptive or interoceptive sources only
• Reafference and Efference Copy (Von Holst and Mittelstaedt)– [Glimcher actually has these confused]
An Alternative
Behavior is structured
– by goals (cf. shoulder reflex)
– by optimization strategies in the face of uncertainty
– Specification of the problem on the basis of function rather than implementation (Marr)
– In particular, the problem is an optimization problem
– Conclusion: Neuroscience needs probability theory, economics, evolutionary theory, and game theory
Reflex Theory (con’t)
What reflex theory doesn’t address
– the shoulder “reflex” (Paul Weiss)
– foraging
– mate selection
– exploratory behaviors
– Language & thought
An Alternative
Behavior is structured
– by goals (cf. shoulder reflex)
– by optimization strategies in the face of uncertainty
– Specification of the problem on the basis of function rather than implementation (Marr)
– In particular, the problem is an optimization problem
– Conclusion: Neuroscience needs probability theory, economics, evolutionary theory, and game theory
I: Optimization
• Q: Optimization with respect to what?
• A: Inclusive fitness but modularized. Evolution provides the goals, economics the optimization techniques
• Do we have a prayer at specifying the optimum?
– Phototransduction near the quantum limit
– Hair cells can detect individual fluid molecule collisions
– Convergent Evolution: Cichlid fish of Tanzania
II: Uncertainty: Epistemological
• Reflex theory dominated by deterministic responses to input (from a highly constrained set)
• Alternative: in general, we suffer from epistemological uncertainty, so we have to optimized in an indeterminate world
Uncertainty (con’t)
• An empirical test of foraging economics:
the prey model, Parus major
• View foraging as an optimization problem: choose the probability p_i of attacking the prey i that maximizes the rate at which energy is gained
• Solution:
– “zero-one” rule
– “independence from encounter inclusion rate” principle
Uncertainty (con’t)
• Frequencies of large and small mealworms were varied
• Small mealworms always had larger handling time
• Prediction (from optimal sol’n):
– Preference for large worms as their freq. increases, regardless of small worm freq. (by IEIR principle)
– If the bird couldn’t get all the worms, it should give up entirely on the small ones (by the zero-one rule)
• Result: yes and no (only 85% selective)
• Maybe this is an optimal strategy after all…
Epistemological Uncertainty & the Brain:A Series of Studies
• Input-association-output model: sensory-parietal-motor
• Lateral intraparietal area (LIP) and monkey saccades:
– Monkeys trained to perform task w/juice reward
– Invariant to input stimulus (light or button or whatever)
– Position-encoding
– Conclusion: command signal (Mountcastle)
Epistemological Uncertainty & the Brain (con’t)
• Lateral intraparietal area (LIP) and monkey saccades:
– Fixation and saccade tasks w/eccentric light
– Weak activation on fixation, but increasingly active over trials of saccade task
– Conclusion: attentional enhancement (Goldberg)
Epistemological Uncertainty & the Brain (con’t)
• Lateral intraparietal area (LIP) and monkey saccades:
– Memory saccade task: target is extinguished but LIP neuron still fires—until the motor command is executed
– Conclusion: motor intention (Gnadt & Anderson)
Platt & Glimcher: encoding the probability of pay-off
Epistemological Uncertainty & the Brain (con’t)
Epistemological Uncertainty & the Brain (con’t)
Probability experiment
Epistemological Uncertainty & the Brain (con’t)
Value experiment
III: “Irreducible” Uncertainty & Game Theory
• Static environment Dynamic competition with other agents
• Then the optimal approach is given by game- theoretic approaches
• In these cases, the optimum often involves (purposefully) random behavior
Uncertainty & Game Theory (con’t)
• Example 1: Chicken
Uncertainty & Game Theory (con’t)
• Conclusion: Smith is best served by behaving non-deterministically, but with probability 0.647 of being a chicken. (Ditto for Jones.)
• If Jones finds non-randomness in the distribution of Smith’s choices, he can predict above chance which option Smith will pick—and win.
• Random behavior is the optimal solution, so: we shouldn’t expect behavior to look deterministic (contrast w/reflex theory).
Intermezzo: How Random Are We?
• Paper, scissors, rocks
• Dice, viscera divination, etc.: technological breakthrough (Jaynes)
• Unconscious vs. conscious behaviors; natural selection vs. “rational actors”
• Pigeons, babies, and adults: the matching rule and cognitive load (and reward)
Game Theory and Ethology
• Duck foraging
– Two feeders at opposite ends
– 33 ducks
– Rate of food depends on feeder, but the more ducks in an area the worse it is
– Where to sit?
Game Theory & Ethology (con’t)
• Person 1: 2-gram bread ball every 5 sec
• Person 2: 2-gram bread ball every 10 sec
Game Theory & Ethology (con’t)
Game Theory & Human Behavior:Work or Shirk
Insp = -5
Game Theory & Human Behavior:Work or Shirk (con’t)
Insp = -50
• Experiment: subjects play against a computer program which looks for statistical regularities in its opponent’s plays and tries to exploit them
• Subjects are only told that they can make money by playing
• 150 trials, then the pay-off matrix switches (unannounced)
• Guess how human beings played….
Game Theory & Human Behavior:Work or Shirk (con’t)
• 150 trials, one pay-off matrix, vis-à-vis the Nash equilibrium?
Game Theory & Human Behavior:Work or Shirk (con’t)
Game Theory & Human Behavior:Work or Shirk (con’t)
• Work-shirk-work-shirk yields 50% behavior. Shannon entropy of choices?
Game Theory & Human Behavior:Work or Shirk (con’t)
Game Theory & Human Behavior:Work or Shirk (con’t)
• Switching between pay-off matrices?
Game Theory & Human Behavior:Work or Shirk (con’t)
Game Theory & Human Behavior:Work or Shirk (con’t)
Game Theory & the Brain
• Repeat the game, this time with monkeys instead of humans
• Simultaneously record from parietal area LIP
• Prediction: if these neurons encode expected utility, then they will fire at constant rates over various movements and various rewards (contrast Platt & Glimcher 1999)
• Now we have an experiment that yields non-deterministic behavior but about which predictions of lawful actions can nevertheless be made
Game Theory & the Brain (con’t)
• Across trials:
– Monkeys behave (near?) optimally: their behaviors track the Nash equilibrium
– LIP neurons do not track the Nash equilibrium suggesting that they are, in fact, encoding (relative) expected utility
• Play-by-play:
– The relative expected value on any given play does vary slightly, given the randomness of play
– Positive correlation b/n this fluctuating expected value and fluctuations in LIP neurons
Game Theory & the Brain (con’t)
Neuroeconomics & Language
• Skinner’s Verbal Behavior
• Programs that are more than input/output
• Bayes Nets for utility as well as beliefs
• Minimum description length: grammar
• Minimum description length: evolution
Neuroeconomics & Language
• “The paradox disappears only if we make a radical break with the idea that language always functions in one way, always serves the same purpose: to convey thoughts—which may be about houses, pains, good and evil, or anything else you please.” (Sec. 304)