Chapter 5

Chapters 5Instrumental Learning & Operant ReinforcementOperant LearningStimulusResponseOutcomeClassical vs. OperantClassicalRequires reflex actionNeutral stimulus associated with USOutside of subjects controlOperantStrengthening/weakening of voluntary actionSubject responds or doesntCan operate togetherWhats in a Name?Operant learning: subject operates on environmentInstrumental conditioning: subject is instrumental in obtaining outcomeTrial and Error LearningE.L. ThorndikeAnimal intelligenceMaze studies

Puzzle BoxCatsCage with mechanism to open doorEscape latencyDiscrete trial procedureLaw of EffectAny behaviour followed by an appetitive stimulus will increase in frequencyTermsOperant (response): any behaviour that operates on the environment to produce an effectReinforcer: any event that increases the frequency of a behaviourPunisher: any event that decreases the frequency of a behaviour

Operant LearningB.F. SkinnerOperant chamberFree operant procedure

Discrete Trial & Free OperantDiscreteOne trial at a timeApparatus must be re-setMeasure some behavioure.g., mazes

FreeOperant can occur at any timeOperant can occur repeatedlyResponse ratee.g., operant chamberFour ContingenciesPositive reinforcementNegative reinforcementPositive punishmentNegative punishmentPositive and NegativePositive: presents some stimulusNegative: removes some stimulusReinforcers and PunishersReinforcer: increases a behaviourPunisher: decreases a behaviourContingenciesResponse Rate:IncreasesDecreasesRemovedPresentedResponse Causes Stimulus to Be:Positive ReinforcementNegative ReinforcementPositive PunishmentNegative PunishmentLever press --> FoodLever press --> Shock offLever press --> ShockLever press --> Food removedTypes of ReinforcersPrimaryNot dependent on an association with other reinforcersSecondaryInitially neutral stimulusPaired with primary reinforcerConditioned ReinforcerSecondary ReinforcersBridging, clickerSecondary extinction without periodic pairings with primaryGenerally weaker than primaryGeneralized reinforcerPaired with many other kinds of reinforcerse.g., moneyStrength of Operant LearningCan condition practically any behaviourShaping (successive approximations)Shaping a Lever PressGradual processReinforce more appropriate/precise responsesFeedbackResponse ChainsSequences of behaviours in specific orderObjective: primary reinforcerConditioned reinforcersDiscriminative stimuliForward ChainingStart with first response in sequence, then work through to last response in additive stepsBackwards ChainingOften used with complex trainingStart with last response in chainNext, second last responseThird last, etc.

Factors in Operant LearningContingencyCorrelation between behaviour & outcomeStrong contingency --> better learningRandom contingency --> no learningBoth reinforcement and punishmentContiguityTime between behaviour & outcomeShorter = better learningDelays let other behaviours occur, forgetting, extinction (behaviour w/o reinforcement)Learning with delay if stimulus placeholder provided (conditioned reinforcer?)More important for punishmentReinforcer CharacteristicsLarger reinforcers --> stronger learningNot a linear effectQualitative differences in reinforcers and punishersSpecies & individual differencesIntensity of punisher

Task CharacteristicsSome tasks easier to learn than othersSpecies & individual differencesInnate and/or prior conditioningDeprivation LevelsGenerally, the greater the deprivation, the more effective the reinforcerReinforcers can satiateDeprivation can provide motivation to engage in punishable behavioursExtinctionBehavioural does not lead to same outcomeResponse no longer produces same outcomeExtinction burst (with reinforcement)Variability of behaviourAggression and frustrationSpontaneous recoveryResurgenceTheories of ReinforcementHulls Drive Reduction TheoryAnimals have motivational states (drives)Necessary for survivalReinforcers are things that reduce drivesPhysiological valueReduce physiological stateDrive Reduction ReinforcersWorks well with primary reinforcersMany secondary reinforcers have no physiological valueHull: association links secondary to driveSome reinforcers hard to classify as primary or secondarySome increase a physiological stateSome necessities undetectableRoller coastersVitaminsSaccharinRelative Value Theory & Premack PrincipleTreat reinforcers as behavioursIs it the food, or the behaviour of eating that is the reinforcer?Behavioural probability scaleGreater or lesser value of behaviours relative to one anotherNo distinction between primary and secondaryPremack PrincipleOne behaviour will reinforce a second behaviourHigh probability behaviour reinforces low probability behaviourBaseline probability scaleTimeRank orderReinforcement relativityNo absolutesProbabilty of response =Time spent on response

Total timeExampleBehavioursEat ice cream (I), play video game (V), read book (B)Baseline (30 minutes)Student 1: I (2min), V (8min), B (20min)Scale: I -- V -- BStudent 2: I (8min), V (20min), B (2min)Scale: B -- I -- VStudent 1: V reinforces I, B reinforces V & IStudent 2: I reinforces B, V reinforces I & BProblemsBaseline phaseFair rating?How to compare very different behavioursTime problemsWhat if time not important to behaviour?Behaviour duration?Length of baseline period?Response Deprivation TheoryDeprived behaviours = reinforcing behavioursDrop below baseline level of performanceNot relative frequency of one behaviour compared to another (i.e., Premack)Level of deprivation for a behaviourPraise? Yes?Escape and AvoidanceDefinitionsEscapeGet away from aversive stimulus that is in progressAvoidanceGet away from aversive stimulus before it beginsShuttle BoxSolomon & Wynne (1953)DogsChamber with barrier; ShockLight off as signal

Two-Process TheoryClassical and operant conditioningShock = USFear/pain/jump/twitch/squeal = URDarkness = CSFear of dark = CRFear: heart rate, breathing, stomach cramps, etc.Negative reinforcementRemoval of fear (CR)Escape of CS, not avoidance of shockSupport for Two-Process TheoryRescorla & LoLordo (1965)Dog in shuttleboxNo signalResponse gives safe timePair tone with shockTone increases rate of responseCS can amplify avoidanceConditioned inhibition can reduce avoidanceProblems with Two-Process TheoryAvoidance without observable fearHeart rateNot consistentFear diminishes with avoidance learningMeasuring FearKamin, Brimer, and Black (1963)Lever press ---> foodAuditory CS ---> avoidance in shuttle box until: 1, 3, 9, 27 avoidances in a rowCS in Skinner box; check for suppression of lever pressResultsFear decreases during extended avoidance trainingBut, avoidance still strong

Even low fear is enough?Avoidance responses

Responding13927Extinction in Avoidance BehaviourOdd prediction from two-process theoryYo-yo effectAvoidance should toggleBut! Avoidance is extremely persistentsuccessful avoidancetrials# of US receivedOne-Process TheoryClassical conditioning component unnecessaryAvoidance, not fear reduction, is reinforcerSafetySidman Avoidance TaskFree-operant avoidanceCan avoidance be learned if no warning CS?Shock at random intervalsResponse gives safe timeExtensive training --> learn avoidanceBut, usually never perfectHigh variability across subjectsTwo-process theory suggests:Time becomes a CS (time elicits fear)Herrnstein & Hineline (1966)Rapid and slow shock rate schedulesLever press switches schedulesShocks presented randomly, no signalResponses give shock reductionReduction in shock is reinforcerLearned HelplessnessBehaviour has no effect on situationGeneralizesLaboratoryGive inescapable shocksShuttle boxWill not switch sidesExpectation that behaviour has no effectLearned Helplessness in HumansDepressionSituations beyond your controlThree dimensionsSituation: specific or globalAttribute: internal or externalTime: short-term or long-termMaier & Seligman (1976)Motivational impairmentCognitive impairmentEmotional impairmentTherapeutic ApplicationConfidence building (can not fail)Implementation issuesTasks that can be successfully completedProduces immunizationEscapable condition inescapable conditionLearned helplessness less likely to develop