chapter 5 – instrumental conditioning: foundations outline 1 –comparison of classical and...

53
Chapter 5 – Instrumental Conditioning: Foundations Outline 1 Comparison of Classical and Instrumental Conditioning Early Investigations of Instrumental Conditioning. Thorndike Chicks and mazes Cats and puzzle box Modern approaches to the study of instrumental conditioning Discrete trials Procedures Free-operant procedures Magazine training and shaping Response rate as measure of operant behavior Instrumental Conditioning Procedures Positive Reinforcement Negative Reinforcement (Escape or Avoidance) Positive Punishment (Punishment) Negative Punishment (Omission Training/DRO)

Upload: lawrence-bruckman

Post on 15-Dec-2015

233 views

Category:

Documents


4 download

TRANSCRIPT

Page 1: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

Chapter 5 – Instrumental Conditioning: Foundations

• Outline 1– Comparison of Classical and Instrumental Conditioning– Early Investigations of Instrumental Conditioning.

• Thorndike– Chicks and mazes– Cats and puzzle box

– Modern approaches to the study of instrumental conditioning• Discrete trials Procedures• Free-operant procedures

– Magazine training and shaping– Response rate as measure of operant behavior

– Instrumental Conditioning Procedures• Positive Reinforcement• Negative Reinforcement (Escape or Avoidance)• Positive Punishment (Punishment)• Negative Punishment (Omission Training/DRO)

Page 2: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Comparison of Classical and Instrumental Conditioning.– Classical = S-S relationship

• Light-shock – elicits fear• Tone-food – elicits salivation• In Classical Conditioning there is no response

requirement.

– Instrumental = R-S relationship• We will refer to it as R-O• Behavior (Response) is instrumental in producing the

outcome – Press lever – Get food– Pull lever – Get money

• In Instrumental Conditioning a particular response is required

Page 3: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Keep in mind Classical and Instrumental conditioning are approaches to understanding learning. – not completely different kinds of learning.– Many learning situations could be described by either

approach.• Child touches hot stove

– CS, US, UR, CR?– Pavlovian = fear stove– Instrumental = less likely to approach

• Conditioned Taste Aversion– S-S = Taste – LICl– R-O= Drink liquid - get sick (punished)

Page 4: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Early Investigations of Instrumental Conditioning.

• Edward Lee Thorndike (American)– Same time as Pavlov

• Late 1800s; early 1900s

– interested in animal intelligence.• late 19th century - many people believed that

animals reasoned – like people do – Romanes– Stories of amazing abilities of animals.

• biased reporting?– Report interesting behavior– ignore stupid

Page 5: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Thorndike (1898) – “Dogs get lost hundreds of times and no one ever

notices it or sends an account of it to a scientific magazine, but let one find his way home from Brooklyn to Yonkers and the fact immediately becomes a circulating anecdote. Thousands of cats on thousands of occasions sit helplessly yowling, and no one takes thought of it or writes to his friend, the professor; but let one cat claw at the knob of a door supposedly as a signal to be let out, and straightway this cat becomes representative of the cat-mind in all the books…In short, the anecdotes give really the…supernormal psychology of animals.”

Page 6: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Thorndike attempted to understand normal or ordinary animal intelligence.– Chicks in a maze– Cats in a box

Page 7: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental
Page 8: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental
Page 9: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental
Page 10: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Puzzle box video

Page 11: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental
Page 12: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Thorndike tested many animals – chicks, cats, dogs, fish, and monkeys. – little evidence for reasoning.

• Instead learning seemed to result from trial and accidental success

Page 13: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Modern approaches to the study of instrumental conditioning– Discrete trials Procedures

• Like Thorndike’s work• Simpler mazes though

– Figure 5.3» Straight alleyway (running speed)» T-Maze (errors)

– Later» radial arm maze » Morris Water Maze

– Note • each run is separated by an intertrial interval• Just like in Pavlovian Conditioning

Page 14: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental
Page 15: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

8-arm Maze

Page 16: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

Morris Water Maze

Page 17: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Free-operant procedures – There are no “trials”– The animal is allowed to behave freely

• Skinner Box– an automated method for gathering data from

animals

Page 18: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental
Page 20: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Skinner used these boxes to study operant conditioning– operant

• any response that “operated” on the environment

– Defined in terms of the effect it has on the environment

Page 21: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Pressing a lever for food– Doesn’t matter how the rat does it

• Right paw, left paw, tail• As long as it actuated the switch

– Similar to opening a door• Doesn’t matter which hand you use• Or foot (carrying groceries)• Just as long as the result is achieved

Page 22: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Magazine training and shaping– First have to train the animals about

availability of food– training by successive approximations

(shaping)• Shaping a rat

Page 23: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Response rate as measure of operant behavior– In a free-operant situation you do not have

measures such as percent correct, or errors.– Skinner used response rate as a primary

measure– We will see later that various schedules of

reinforcement affect response rate in various ways.

Page 24: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Instrumental Conditioning Procedures• First let’s get some terminology down

– Positive• Behave = Stimulus applied

– Negative• Behave = Stimulus removed

– Reinforcement• Behavior increased

– Punishment• Behavior decreased

• 2 x 2 table

Page 25: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

positive negative

Rein

forc

emen

tPu

nish

men

t

Page 26: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Application– Box 5.2 in book

• Mentally disabled woman• Head banging behavior• Possibly to get attention - reward

– Change contingencies• Ignore head banging• Social rewards when not head banging

– Procedure?• Negative Punishment

– Differential Reinforcement of Other Behavior (DRO)

Page 27: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

ABAB design

Page 28: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Your book uses somewhat different terminology– Positive Reinforcement

• Same

– Escape or Avoidance• Negative Reinforcement

– Punishment • Positive Punishment

– Omission Training/DRO• Negative Punishment

• We will use my terminology

Page 29: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

O

• Outline 2– Fundamental Elements of Instrumental Conditioning

• The Response– Behavioral variability vs. Stereotypy– Relevance or Belongingness– Behavior Systems and Constraints on Instrumental conditioning

• The Reinforcer– Quantity and Quality of the Reinforcer– Shifts in Reinforcer Quality or Quantity

• The Response-Outcome Relation– Temporal Relation– Contingency– Skinner’s Superstition experiment– The Triadic Design– Learned helplessness hypothesis

Page 30: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Behavioral variability vs. Stereotypy– Thorndike emphasized the fact that reinforcement increases the

likelihood of the behavior being repeated in the future• Uniformity – stereotypy• This is often true

– You can increase response variability, however, by requiring it• Page & Neuringer, 1985• Two keys (50 trials per session)

– Novel group» Peck 8 times» Do not repeat a pattern that was used in the last 50 trials» LRLRLRLR» RLRLRLRL» LLRRLLRR» LLLLRRRR

– Control group» RF 8 pecks» Doesn’t matter how they do it.

• Figure 5.8

Page 31: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

Reward and originality (creativity)?Box 5.3

Page 32: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Relevance or Belongingness• We have discussed this in Classical

Conditioning• Bright Noisy Tasty water• Peck differently for water and Grain

• It has also been studied extensively in the Instrumental literature• Originally noted by Thorndike

• Cat – puzzle box• Train cat to yawn to escape• Or scratch themselves• Did not go well

Page 33: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Brelands “The Misbehavior of Organisms” 1961• Play on Skinner’s “Behavior of Organisms” (1938)

• students of Skinner – training animals to do tricks as advertising gimmicks

• raccoon and coin(s)– Shaped to pick up coin– Then to place in bank– Then 2 coins

• pig and wooden coin

Page 34: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Instinctive drift– Arbitrarily established responses drift toward more

innately organized behavior (instinct)

• The arbitrary operant– place coin in bank

• Instinctive drift– species specific behaviors related to food

consumption• Wash food• Root for food

Page 35: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

Behavior Systems and Constraints on Instrumental conditioning

• Behavior Systems (Timberlake)• the response that occurs in a learning episode is

related to the particular behavioral system that is active at the time

• If you are a hungry rat and food is the reward • behaviors related to foraging will increase

• If you are a male quail maintained on a light cycle that indicates mating season and access to a female is being offered • mating behaviors will be elicited

Page 36: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

Behavior systems continued

• The effectiveness of any procedure for increasing an instrumental response will depend on the compatibility of that response with behavioral system currently activated– rats pressing levers for food?– pigeons pecking keys for food?– Very easy to train

• Even works for fish– Easy

• bite a stimulus associated with a rival male• Swim through hoops for a stimulus associated with

female– Difficult

• Bite stimulus associated with access to female• Swim through hoops for access to rival male

Page 37: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

The Reinforcer

• How do qualities of the Reinforcer affect Instrumental Conditioning?– Quantity and Quality of the Reinforcer

• Just like in Pavlovian conditioning. – More intense US Better conditioning

• Trosclair-Lassere et al. (2008)– Taught autistic child to press button for social reward

» Praise, hugs, stories– Social reward

» 10 s» 105 s» 120 s

– Progressive ratio

Page 38: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental
Page 39: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Magnitude of RF and drug abstinence– Perhaps not surprising

• The more you pay the better addicts do.

Page 40: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

.

• Shifts in Reinforcer Quality or Quantity• How well a reinforcer works depends on what

subjects are used to receiving

– Mellgren (1972) -• Straight alleyway

– Phase 1» Half – found 2 pellets (low reward)» Half – found 22 pellets (high reward)

– Phase 2» Half from each group switched to opposite condition» Other half stay the same

Page 41: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• 4 conditions• H-H

• high RF control

• L-L• low RF control

• L-H• Positive contrast

• H-L• Negative Contrast

Results – Figure 5.10

[note Domjan changed it to Small (S) and Large (L) rewards in text]

Page 42: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Response- Outcome Relation– Temporal relation?

• contiguity

– Causal relation?• Contingency

• Independent– Contiguous doesn’t mean contingent

• You wake up – sun rises

– Contingent not always contiguous• Submit tax returns

– Wait a few weeks for the money

Page 43: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Effects of temporal relationship– Hard to get animals to respond if long delay

between response and reward

– Delays are tough experiments to run using free operant procedure• Allow barpressing during “delay?”

– Some barpresses likely close to RF

• Enforce no barpressing after initial barpress?– Delay RF?

Page 44: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• In study in book (Fig. 5.11) each bar press resulted in RF at a specified set time

• For some animals the delay was short– 2 - 4 s

• For some it was long– 64 s

• Example (16 s delay)– Bar press 1 at 1 s (RF1 = 17 s)– Bar press again at 3 s (RF2 at 19s)

• 14 s from RF1

– Bar press again at 12 s (RF3 at 28 s)• 5 s from RF1

– There will still be some “accidental” contiguity• Graph shows how responding is affected by actually experienced delays

Page 45: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental
Page 46: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Why are animals so sensitive to the delay between response and outcome?– Delay makes it difficult to determine which

behavior actually caused RF• Press lever (contingent after 20 s)

– Scratch ear– Dig in bedding on the floor– Rear up– Clean face

» Reward

• All of the other behaviors are more contiguous with RF than is lever pressing

Page 47: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

– A marking procedure can help maintain responding over a long delay• Provide a light or click after “target” response

– Lever press – click .....20 s food

• Helps animals bridge the gap

Page 48: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Response-Reinforcer Contingency– Skinner thought contiguity was more

important than contingency– Superstition experiment

– Superstition and bowling video from u-tube• relevant content begins at 3:12

Page 49: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Reinterpretation of Superstition Experiment• Behavioral systems again

– Different kinds of responses occur when periodic RF is used• Focal search

– Behaviors near food cup as time for RF approaches» “I know its coming”

• Post-food focal search– Again - activity near cup

» “Did I miss any?”

• General search– Move away from cup

» This is probably when skinner saw the turning and head tossing behaviors» “I have to wait, might as well look around”» “I am also a bit frustrated”

– There is evidence for these patterns of responding• Staddon and Simmelhag (1971)

– There is also evidence that slightly different patterns emerge with food vs. water RF.

Page 50: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Effects of the controllability of Reinforcers• Is having control a good thing?

• We briefly mentioned the Brady Executive Monkey study earlier.• That study implied having operant control

over outcomes could be bad for the animal

• That study was confounded • Better evidence for the effects of control over

outcomes comes from the learned helplessness literature

Page 51: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental
Page 52: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental
Page 53: Chapter 5 – Instrumental Conditioning: Foundations Outline 1 –Comparison of Classical and Instrumental Conditioning –Early Investigations of Instrumental

• Learned helplessness hypothesis– Animals learn that they have no control over the

shock– Creates an expectation

• expect shock regardless of their behavior

• Implications– 1. This expectation reduces motivation to make an

instrumental response– 2. Even if they do escape/avoid shock, it is harder

for them to associate their behavior with the outcome.• Shock was independent of behavior in the past• Similar to US preexposure effect