· pdf filecomparison of a stimulus equivalence protocol and ... which is a verbal response...

COMPARISON OF A STIMULUS EQUIVALENCE PROTOCOL ANDTRADITIONAL LECTURE FOR TEACHING SINGLE-SUBJECT DESIGNS

SADIE LOVETT, RUTH ANNE REHFELDT, YORS GARCIA, AND JOHNNA DUNNING

SOUTHERN ILLINOIS UNIVERSITY

This study compared the effects of a computer-based stimulus equivalence protocol to atraditional lecture format in teaching single-subject experimental design concepts toundergraduate students. Participants were assigned to either an equivalence or a lecture group,and performance on a paper-and-pencil test that targeted relations among the names ofexperimental designs, design definitions, design graphs, and clinical vignettes was compared.Generalization of responding to novel graphs and novel clinical vignettes, as well as theemergence of a topography-based tact response after selection-based training, were evaluated forthe equivalence group. Performance on the paper-and-pencil test following teaching wascomparable for participants in the equivalence and lecture groups. All participants in theequivalence group showed generalization to novel graphs, and 6 participants showedgeneralization to novel clinical vignettes. Three of the 4 participants demonstrated theemergence of a topography-based tact response following training on the stimulus equivalenceprotocol.

Key words: stimulus equivalence, college students, tact, verbal behavior, topography-basedresponding

_______________________________________________________________________________

Behaviorally based instructional protocolsgrounded in the principles of stimulus equiva-lence can provide instructors with an alternativemethod of communicating their subject matterof interest to students. The hallmark of stimulusequivalence protocols is that direct training oncertain relations among instructional stimuliwill result in the emergence of untrainedrelations among those stimuli (Sidman, 1994).Early research that examined this phenomenonfocused on teaching reading comprehensionand oral reading skills to individuals withintellectual disabilities (Sidman, 1971). Forexample, Sidman and Cresson (1973) used astimulus equivalence protocol to teach anindividual to read three-letter words, such as‘‘cow.’’ The learner was first trained to relate thespoken word ‘‘cow’’ to a picture of a cow and torelate the spoken word ‘‘cow’’ to the printedword cow. After training, the learner was thenable to orally name the picture of the cow and

the printed word cow, and he was able to relatethe picture of the cow to the printed word cowwithout a direct history of reinforcement forrelating those stimuli. According to the stimulusequivalence paradigm, the oral naming of thepicture and printed word, which involves areversal of the trained relation, is referred to assymmetry. The relation of the printed word andthe picture that had never been previouslypaired is referred to as transitivity.

Instruction using the principles of stimulusequivalence has been applied successfully ina variety of situations. Cowley, Green, andBraunling-McMorrow (1992) taught adults withacquired brain injuries to relate the dictatednames of their therapists to photographs of thetherapists and to relate the dictated therapistnames to the written therapist names. Partici-pants then related the photographs to the writtentherapist names and orally named the therapistswhen shown the photographs without furthertraining. An investigation by Lynch and Cuvo(1995) used a stimulus equivalence protocol toteach fifth- and sixth-grade students relationsbetween pictorial representations of fractions andnumerical fraction ratios and relations between

Address correspondence to Ruth Anne Rehfeldt,Behavior Analysis and Therapy Program, RehabilitationInstitute, Southern Illinois University, Carbondale, Illinois62901 (e-mail: [email protected]).

doi: 10.1901/jaba.2011.44-819

JOURNAL OF APPLIED BEHAVIOR ANALYSIS 2011, 44, 819–833 NUMBER 4 (WINTER 2011)

819

printed decimals and pictorial representations offractions. Following training, students related thenumerical fraction ratios to the appropriate printeddecimals. A recent study by Toussaint and Tiger(2010) examined the use of a stimulus equivalenceprotocol to teach braille literacy to children withdegenerative visual impairments. Participants whowere able to relate a spoken letter name to theprinted letter were trained to relate braille lettersand the corresponding printed letters. Participantsthen were able to select the appropriate brailleletter when provided with the spoken letter nameand orally name the braille letters.

These examples of the application of stimulusequivalence in teaching various skills highlight itsutility as an instructional method. Studies ofstimulus equivalence frequently involve trainingand testing with a conditional discriminationprocedure that relies on selection-based respond-ing, or pointing to one stimulus that is presentedin an array (Michael, 1985). The emergence oftopography-based responding, or responding in adifferent topography than that trained, also hasbeen documented (Michael, 1985). Followingselection-based training, Cowley et al. (1992)demonstrated the emergence of a tact response,which is a verbal response under the control of anonverbal stimulus (Skinner, 1957), by showingthat participants were able to name the photo-graphs of the therapists. Furthermore, Toussaintand Tiger (2010) documented a topography-based tact response by demonstrating the emer-gence of oral naming of the braille letters. Thetopography-based responding shown in thesestudies may reflect the acquisition of moremeaningful verbal repertoires than selection-basedresponding produces (Sundberg & Sundberg,1990), because skills commonly targeted ineducation, such as speaking and writing, requiretopography-based rather than selection-basedresponding. A successful educational curriculumwill result in proficiency in both spoken andwritten topography-based response repertoires.

In recent years, stimulus equivalence proto-cols have been extended to the instruction of

sophisticated learners (e.g., Fienup & Critchfield,2010). Ninness et al. (2005, 2006) usedcomputer-based stimulus equivalence protocolsto teach algebraic and trigonometric mathemat-ical functions. Participants were taught to relatestandard mathematical formulas to factoredformulas and to relate factored formulas tographical representations of those formulas. Aftertraining, participants related the standard for-mulas and the graphs without direct training,and generalization to novel formulas and graphswas shown. Fields et al. (2009) taught statisticalinteraction concepts by training college studentsto relate line graphs and textual descriptions ofinteractions, textual descriptions of interactionsand labels of the interactions, and interactionlabels and definitions of each type of interaction.The emergence of untrained relations among thestimuli was shown, as was generalization ofresponding to novel variations of the trainedstimuli. In addition, a study by Fienup, Covey,and Critchfield (2010) used stimulus equivalenceto teach relations among regions of the brain,anatomical locations of brain regions, andpsychological function of brain regions to collegeundergraduates. Finally, Walker, Rehfeldt, andNinness (2010) taught college students to relatethe names, definitions, causes, and treatments fordisabilities. In addition to demonstrating theemergence of untrained relations, this studyshowed the emergence of written and vocaltopography-based responding following selec-tion-based training.

The aforementioned research indicates thatinstruction with stimulus equivalence protocolsis successful in teaching relations amongstimuli, including complex stimuli that areimportant in training advanced learners. How-ever, there has been little effort to compare theefficacy of stimulus equivalence protocols tostandard educational practices or to assess thesocial validity of the instructional method. Oneexception is Fields et al. (2009), who evaluatedgeneralization to a paper-and-pencil posttestfollowing computer-based training. The use of

820 SADIE LOVETT et al.

worksheets and paper tests, as opposed tocomputer-based procedures, more closely ap-proximates the materials and procedures presentin the average classroom. Fields et al. alsoincluded a questionnaire to assess the socialvalidity of the instructional method (see alsoFienup & Critchfield, 2011).

Further investigations of the efficacy andacceptability of stimulus equivalence protocolsin instruction relative to standard educationalpractices would be beneficial in determining theutility of the method. The objectives of thepresent study were thus as follows: First, wecompared the effectiveness of a stimulusequivalence protocol to that of a lecture inteaching undergraduate students concepts ofsingle-subject experimental design. The proto-col established relations among the names ofdesigns, their definitions, representative graphs,and clinical vignettes in which the use of adesign might be appropriate. Specifically, wecompared performances for the two groups ofparticipants on a paper-and-pencil test. Second,we evaluated the efficacy of the stimulusequivalence protocol by assessing generalizationof the relations to novel graphs and clinicalvignettes and the emergence of a topography-based repertoire. Finally, we investigated thesocial validity of the stimulus equivalenceprotocol and the lecture by administering asatisfaction questionnaire to participants inboth conditions at the end of the experiment.

METHOD

Participants, Setting, and Apparatus

Twenty-four undergraduate students whowere currently enrolled in a research methodscourse participated in this experiment. Allparticipants received extra credit as compensa-tion for participation. Sessions were conductedin an office (3 m by 5 m) that contained twodesks, each with a chair and personal computer.The participant was seated at one desk, and theexperimenter was seated at the second desk,throughout the session. All procedures were

conducted on the computer, with the exceptionof the paper-and-pencil quiz and social validitysurvey. Sessions ranged in length from 75 to140 min.

Equivalence Stimuli

Each of four stimulus classes contained fourstimuli that were presented during trainingand two stimuli that were used to test forgeneralization. Stimuli were developed based onthe definitions, graphical illustrations, andclinical examples of single-subject designs inan undergraduate textbook (Kennedy, 2005).The A stimuli were the names of each of thefour basic single-subject designs. The B stimuliwere definitions of the four correspondingdesigns. The C stimuli were graphs depictingthe implementation of each of the four single-subject designs. Each graph had unique y-axislabels (e.g., percentage aggressive behavior) andunique intervention phase labels (e.g., rein-forcement). The D stimuli were vignettes thatdescribed clinical situations in which the fourcorresponding designs would be appropriate foruse. All vignettes were designed such that thelength of each description was similar. Thevignettes described clinical situations that weredistinct from the behaviors and interventionsillustrated on the graphical (C) stimuli. The Band D stimuli are presented in Figure 1 (seealso Walker & Rehfeldt, in press).

There were four C9 generalization stimuliincluding one novel graphical representation ofeach of the single-subject designs. The novelgraphs showed changes in behavior in theopposite direction from that of the stimulusused in training. For example, the C1 stimulusdepicted a graph for a withdrawal design thatshowed an increase in the percentage correctresponding, and the C91 stimulus was a novelgraph of a withdrawal design that showed adecrease in aggressive behavior. Novel graphsalso included different y-axis and phase labels.The four D9 generalization stimuli were novelclinical vignettes that described situations inwhich each of the four designs would be

STIMULUS EQUIVALENCE PROTOCOL 821

appropriate for use. Length of the descriptionsin the vignettes was similar to that of thevignettes used in training. Like the C9 gener-alization stimuli, the vignettes described achange in behavior opposite of the directionof behavior change on the stimulus used intraining. For example, the D1 stimulus de-scribed a situation in which a teacher wished todecrease talking out in class, and the D91stimulus described a situation in which self-monitoring was used to increase the number ofmath problems completed.

General Procedure

This experiment used a pretest–train–posttestsequence and included two groups of partici-pants. Participants in the equivalence groupwere exposed to a computer-based stimulusequivalence protocol, and participants in thelecture group viewed a video of a lecture thatprovided an overview of the four basic single-subject designs. Participants were assignedrandomly to either the equivalence or lecturegroup by the flip of a coin. Nine participantswere included in the lecture group, and 15participants were included in the equivalencegroup. The number of participants in the twogroups was unequal because several participantsassigned to the lecture group cancelled sessions.Additional participants could not be recruitedfor this group because course material hadprogressed to the point of covering the single-subject design topics targeted in this experimentby that time in the semester. The main depen-dent variable was performance on the paper-and-pencil quiz, and all participants completeda questionnaire to evaluate the social validity ofthe instructional methods.

Paper-and-pencil pretest and posttest evaluation.The paper-and-pencil quiz consisted of 15multiple-choice questions on single-subject de-signs. Each question included four responseoptions (a, b, c, and d). Questions includedadaptations of the stimuli presented in thestimulus equivalence protocol and lecture. Fourquestions tested variations of the definition-to-design-name (B-A) relations by requiring partic-ipants to select the correct design name in thepresence of a modified version of the designdefinition. Four questions required participants toselect the appropriate design name in the presenceof a novel graph (C-A relation). Four questionsrequired participants to select the design name inthe presence of a novel clinical vignette (D-Arelation). Three questions required participants toselect the appropriate definitional design featurein the presence of the design name (A-B relation).A sample quiz item that required the participant

Figure 1. B and D stimuli for each of the fourstimulus classes.


to select the appropriate design name in thepresence of a novel graph is depicted in Figure 2.The quiz was administered at the start of theexperiment as a pretest, and an identical quiz wasadministered as a posttest after completion of thestimulus equivalence protocol or presentation ofthe lecture for the equivalence and lecture groups,respectively. Content of quiz questions wasvalidated by a professor of behavior analysis withexpertise in single-subject design methodology.

Interobserver agreement. Interobserver agree-ment on quiz scores was collected by anindependent reviewer. Agreement was calculat-ed for 33% of pretests and 33% of posttestsfor both the equivalence and lecture groups.Interobserver agreement scores were calculatedby dividing the number of item-by-itemagreements by the total number of agreementsplus disagreements and multiplying that valueby 100%. Item-by-item agreement for bothpretests and posttests for the equivalence groupwas 100%. Mean agreement for pretests for thelecture group was 98% (range, 93% to 100%).Item-by-item agreement for posttests for thelecture group was 100%. Interobserver agree-ment was not conducted on responding duringthe stimulus equivalence protocol because allprocedures and data collection were automated.

Social validity survey. A questionnaire thatassessed participants’ opinions of the respectiveinstructional method was administered at theend of the experiment. The survey included fourquestions that participants rated on a 7-pointLikert-type scale, with higher ratings indicating amore positive evaluation of the instructionalmethod. Questions inquired as to the partici-pant’s confidence in his or her knowledge ofsingle-subject designs, the degree to which he orshe would prefer to be taught using the particularinstructional method, and the participant’sopinions on the time commitment for instruc-tion. The survey is shown in Table 1.

Lecture Group

Following the paper-and-pencil pretest, par-ticipants were seated at desks and were told thatthey were going to view a video on single-subjectdesigns. Up to two participants were presentto view the video during a session; however,participants were seated at separate desks whilecompleting the paper-and-pencil test. The videowas presented on the computer and showed a56-min lecture with an accompanying Power-Point presentation that provided an overview ofthe four basic single-subject designs. Videocontent included an introduction to single-subject

Figure 2. Sample question testing a C-A relation from the paper-and-pencil quiz.


design terminology (e.g. independent variables,dependent variables, and functional relations) andan overview of the basic components of agraphical display (e.g. axes, phases, and datapaths). The majority of the lecture was dividedinto four sections that corresponded to withdraw-al, multiple baseline, alternating treatments, andchanging criterion designs. Each section provideda definition of the design, showed a basic graphicaldisplay of the design, and presented an example ofthe application of the design in an applied settingwith an accompanying graph. Lecture content wasderived from the presentation of single-subjectdesign material in two undergraduate textbooks(Kennedy, 2005; Richards, Taylor, Ramasamy, &Richards, 1999). After the video, the paper-and-pencil posttest was administered, followed by thesocial validity survey.

The paper-and-pencil pretest and posttest aswell as the social validity survey were identicalto those that were used with the equivalencegroup. These two measures were the only pointsof comparison between the two groups.

Equivalence Group

Training for this group consisted of a computer-based stimulus equivalence protocol programmed

using Microsoft Visual Basic 2008 ExpressEdition. Training and test trials were presentedin a match-to-sample format with one samplestimulus at the top of the screen and fourcomparison stimuli at the bottom of the screenon each trial. For example, on a trial thatexamined the A1-B1 relation, the A1 stimuluswas presented at the top of the screen, and all fourB stimuli appeared at the bottom of the screen.Trials in all training and testing phases werepresented in random order, and the order inwhich the comparison stimuli were presented onthe screen was randomized. During training,correct responses were followed by written andauditory feedback in the form of the word‘‘correct’’ and the chime sound from theWindows operating system. Incorrect responseswere followed by written and auditory feedback inthe form of the word ‘‘incorrect’’ and the chordsound from the Windows operating system. Nofeedback was provided during testing. After com-pletion of the paper-and-pencil pretest, partici-pants were seated at the computer and read thefollowing instructions on the screen:

Thank you for participating in this experiment. Yourjob during this experiment is to do the best that youcan at all times. One box will be presented at the topof the screen, and four boxes will be presented below

Table 1

Social Validity Survey

How confident do you feel in your knowledge of single-subject designs?

1 2 3 4 5 6 7

Not at all confident Somewhat confident Very confident

Rate the degree to which you would prefer to be taught using this instructional method.

1 2 3 4 5 6 7

Don’t prefer at all Somewhat prefer Strongly prefer

How appropriate was the time commitment for this instructional method in relation to the amount you feel you have learned?

1 2 3 4 5 6 7

Not at all appropriate Somewhat appropriate Very appropriate

How do you feel about the length of this instructional method?

1 2 3 4 5 6 7

Not at all appropriate Somewhat appropriate Very appropriate


it. Your job will be to choose one of the boxes at thebottom of the screen. During certain portions of theexperiment you will receive feedback as to whetheryour choice was correct or not, but during otherportions of the experiment you will not receivefeedback. Please do the best you can at all timesregardless of whether or not you receive feedback.Click on the button below to start.

Pretest for equivalence and generalization relations.This test consisted of 48 trials that evaluatedequivalence (definition-to-vignette [B-D] andvignette-to-definition [D-B]) and generalization(design-name-to-novel-graph [A-C9] and design-name-to-novel-vignette [A-D9]) relations. Therewere 12 trials for each type of relation (e.g., B-D),and each individual relation (e.g., B1-D1) waspresented three times. No feedback was providedfollowing responses on this test.

Training and symmetry tests. Participants firstwere trained on the design-name-to-definition(A-B) relations. The design names appeared asthe sample stimuli, and the design definitionsappeared as comparison stimuli. A trainingblock consisted of 12 trials, with each individualrelation (e.g., A1-B1) presented three times.Responses were followed by written and audi-tory feedback. Participants repeated the trainingphase until they met the criterion of 11 of 12correct (92%). After meeting the criterion fordesign-name-to-definition (A-B) training, par-ticipants advanced to the definition-to-design-name (B-A) symmetry test, which included 12trials. No feedback was provided after responsesduring this test, and the criterion was 11 of 12correct. If the criterion was not met on thedefinition-to-design-name (B-A) symmetry test,the participant returned to design-name-to-definition (A-B) training. When the criterionin training again was reached, the participantproceeded to the definition-to-design-name (B-A) symmetry test, and this process was repeateduntil he or she achieved the criterion on the test.

After passing the definition-to-design-name (B-A) symmetry test, training began on the design-name-to-graph (A-C) relations. The design namesappeared as the sample stimuli, and graphsappeared as comparison stimuli. Training was

conducted in the same manner as for the design-name-to-definition (A-B) relations, with a trainingblock containing 12 trials and each relation (e.g.,A1-C1) presented three times. After meeting thecriterion of 11 of 12 trials correct, participantsadvanced to the graph-to-design-name (C-A)symmetry test. This test was similar to thedefinition-to-design-name (B-A) symmetry testand included 12 trials, with a mastery criterionof 11 of 12 correct. If the criterion was notattained on the graph-to-design-name (C-A)symmetry test, design-name-to-graph (A-C) train-ing was repeated, and the graph-to-design-name(C-A) symmetry test again was administered untilmastery was achieved.

Following the graph-to-design-name (C-A)symmetry test, the design-name-to-vignette (A-D) relations were trained in the same manner asthe design-name-to-definition (A-B) relations.The design name was presented as the samplestimulus, and the clinical vignettes appeared ascomparison stimuli. A training block contained12 trials with each relation (e.g., A1-D1)presented three times, and the criterion foradvancing was 11 of 12 correct. After achievingmastery on the design-name-to-vignette (A-D)relations, the vignette-to-design-name (D-A)symmetry test was presented in the samemanner as the definition-to-design-name (B-A) symmetry test. The test included 12 trialswith a mastery criterion of 11 of 12 correct. Ifcriterion was not achieved, design-name-to-vignette (A-D) training was repeated, and thevignette-to-design-name (D-A) symmetry testwas presented again until mastery was attained.

Mixed symmetry test. This test included 36trials that evaluated the definition-to-design-name (B-A), graph-to-design-name (C-A), andvignette-to-design-name (D-A) symmetry rela-tions. As in the previous tests, each relation (e.g.,B1-A1) was presented three times. Trials werepresented in random order, and the masterycriterion was 33 of 36 correct (91.7%). Nofeedback was provided following responses.Participants continued to the next test phaseregardless of performance.


Transitivity test. This test consisted of 48trials that evaluated the definition-to-graph(B-C), graph-to-definition (C-B), graph-to-vignette (C-D), and vignette-to-graph (D-C)transitive relations. Each relation (e.g., B1-C1)was presented three times, and the criterion was44 of 48 trials correct (91.7%). Participantscontinued to the next test after completion of all48 trials regardless of performance.

Equivalence test. This test included 24 trialsthat evaluated the definition-to-vignette (B-D)and vignette-to-definition (D-B) equivalencerelations. Each relation (e.g., B1-D1) was pre-sented three times, and the criterion was 22 of 24trials correct (91.7%). Participants continued tothe next test regardless of performance.

Generalization test. The first test for general-ization included 12 trials that evaluated thedesign-name-to-novel-graph (A-C9) relations.The design name was presented as the samplestimulus, and the novel graphs appeared ascomparisons. Each relation (e.g., A1-C91) waspresented three times, and the criterion was 11of 12 trials correct (91.7%). No feedback wasprovided following responses, and participantscontinued to the subsequent generalization testregardless of performance.

The second test for generalization evaluated thedesign-name-to-novel-vignette (A-D9) relationsand was presented in the same manner as the testfor design-name-to-novel-graph (A-C9) relations.The design name appeared as the sample stimulus,and the novel vignettes were presented ascomparisons. This test included 12 trials witheach relation (e.g., A1-D91) presented three times,and no feedback was provided after responses.Following completion of the design-name-to-novel-vignette (A-D9) generalization test, thecomputer program ended, and the paper-and-pencil posttest was administered for 11 of the 15participants in this group. The remaining fourparticipants continued to the tact test.

Tact test. The tact test was conducted in twoblocks using flash cards presented by theexperimenter. Before beginning the tact test,

the experimenter read the following instructionsto the participant: ‘‘I’m going to show you somecards, and I’d like you to tell me whichexperimental design the picture or descriptionon the card represents. I won’t tell you whetheryour responses are correct or incorrect, but tryyour best.’’ An individual trial consisted ofthe experimenter showing the participant onecard and asking, ‘‘What design is this?’’ Theparticipant was allowed 10 s to review thestimulus and respond. No feedback wasprovided after responses, and if the participantfailed to tact the stimulus within 10 s, the nexttrial was presented.

The first tact-test block included 24 trialsthat evaluated the graph-to-design-name (C-A)and vignette-to-design-name (D-A) relations.The graph (C) and vignette (D) stimuli usedduring training on the stimulus equivalenceprotocol were individually presented on cards(7 cm by 10 cm). Trials were presented in apredetermined random sequence, and eachrelation (e.g., C1-A1) was presented threetimes. Criterion for mastery was 22 of 24correct (91.7%), and the second test blockimmediately followed regardless of perfor-mance. The second tact-test block consisted of24 trials that tested the novel-graph-to-design-name (C9-A) and novel-vignette-to-design-name (D9-A) generalization relations, and itwas conducted in the same manner as the tacttest for the trained stimuli. The novel graph(C9) and novel clinical vignette (D9) stimuliwere presented on flash cards. Criterion formastery was 22 of 24 correct (91.7%). Regard-less of performance, the paper-and-pencil post-test was administered after the tact test.

Interobserver agreement on the tact test wascollected by an independent observer for 50%of sessions. Interobserver agreement scores werecalculated by dividing the number of trial-by-trial agreements by the total number ofagreements plus disagreements and multiplyingthat value by 100%. Trial-by-trial agreementwas 100% for all sessions.


RESULTS

Session Length

Participants in the equivalence group spentan average of 85 min completing the stimulusequivalence protocol (range, 62 to 120 min).All participants in the lecture group spent56 min in the instructional portion of theexperiment viewing the recorded lecture.

Paper-and-Pencil Evaluation

Results for the paper-and-pencil quiz forboth the equivalence and lecture groups arepresented in Table 2. For the equivalencegroup, mean quiz scores were 7.4 points (SD 5

1.5) on the pretest and 10.4 points (SD 5 2.3)on the posttest. For the lecture group, meanquiz scores were 7.0 points (SD 5 1.3) on thepretest and 9.4 points (SD 5 2.7) on theposttest. The average increase in scores frompretest to posttest was 2.9 points (SD 5 2.3) forthe equivalence group and 2.4 points (SD 5

2.4) for the lecture group. Bonferroni adjustedalpha levels of .017 per test were used toconduct statistical analyses. An independentgroups t test revealed no significant differencebetween the posttest means of the lecture andequivalence groups, t(21) 5 0.9, p 5.392.Within-group t tests revealed significant differ-ences between pretest and posttest for both theequivalence group, t(13) 5 4.8, p , .001, andthe lecture group, t(8) 5 23.1, p 5 .016.

Social Validity Survey

The average rating regarding participants’confidence in knowledge of single-subject de-signs was 4.3 (range, 3 to 6) and 4.6 (range, 3 to6) for the equivalence and lecture groups,respectively. The degree to which participantswould prefer to receive instruction like that ofthe experimental procedures received an averagerating of 4.3 (range, 2 to 7) for the equivalencegroup and 4.8 (range, 1 to 7) for the lecturegroup. The appropriateness of the time commit-ment for the amount that was learned received anaverage rating of 5.1 (range, 3 to 7) for both

equivalence and lecture groups. The appropri-ateness of the length of the instructional methodreceived an average rating of 4.9 (range, 2 to 7)for the equivalence group and 5.3 (range, 3 to 7)for the lecture group. The average overall sumscore for all items on the rating scale was 18.7(range, 12 to 25) and 19 (range, 13 to 26) for theequivalence and lecture groups, respectively, withhigher ratings signifying a more positive reviewof the instructional method.

Equivalence Class Formation

The number of training and symmetry testblocks to criterion is depicted in Table 3.Fourteen participants in the equivalence groupmet criterion for all training sessions andsuccessfully passed the symmetry tests for theindividual relations. Participant 11 did notcomplete design-name-to-graph (A-C) training,and her participation ended after 54 trialblocks. Therefore, there are no data for thatparticipant for the remaining analyses.

Table 2

Quiz Scores

Group Participant Pretest Posttest

Pretest toposttestchange

Equivalence 1 8 13 52 7 12 53 7 13 64 7 12 55 8 9 16 6 11 57 8 10 28 9 8 219 6 8 2

10 5 5 012 8 9 113 7 12 514 7 11 415 11 12 1

Lecture 16 7 13 617 6 10 418 6 10 419 6 10 420 9 12 321 8 10 222 8 9 123 5 4 2124 8 7 21

Note. The highest possible score was 15.


Results for the stimulus equivalence protocolpretests and posttests for trained and generaliza-tion stimuli are presented in Figure 3. Noparticipants met criterion on the equivalence orgeneralization pretests. All participants reachedcriterion on the mixed test for symmetry(definition-to-design-name [B-A], graph-to-design-name [C-A], and vignette-to-design-name [D-A]relations) after training. Except for Participants4, 9, and 10, all participants reached criterion onthe test for transitivity (definition-to-graph [B-C], graph-to-definition [C-B], graph-to-vignette[C-D], and vignette-to-graph [D-C] relations).Participants 4 and 9 were below criterion by onetrial on the test for transitivity, and Participant10 was below criterion by two trials. Eleven ofthe 14 participants achieved mastery criterion onthe equivalence posttest (definition-to-vignette[B-D] and vignette-to-definition [D-B] rela-tions). All 14 participants met criterion on thedesign-name-to-novel-graph (A-C9) generaliza-tion test. Only 6 of the 14 participants(Participants 1, 2, 3, 4, 12, and 13) achievedcriterion on the design-name-to-novel-vignette(A-D9) generalization test.

Tact test. Results for the tact test are depictedin Figure 4. Three of the four participants metcriterion on the tact test for the graph-to-design-name (C-A) relations using stimuli that

were presented during training. Two partici-pants achieved mastery criterion on the test forthe vignette-to-design-name (D-A) relations.

Three of the four participants attainedcriterion on the novel-graph-to-design-name(C9-A) generalization relations. Two partici-pants met criterion on the novel-vignette-to-design-name (D9-A) generalization relations.Participant 15 was the only one of the fourparticipants exposed to this condition who didnot meet criterion on the tact test using trainedor generalization stimuli.

DISCUSSION

A stimulus equivalence protocol was designedto teach undergraduate students concepts ofsingle-subject experimental designs. Results ofthe current study extend previous findingson the use of equivalence-based instructionwith advanced learners by demonstrating theemergence of relations among stimuli that werenot directly trained (Fields et al., 2009; Fienupet al., 2010; Ninness et al., 2005, 2006; Walkeret al., 2010). As a result of this experiment,participants in the equivalence group were ableto inspect graphical stimuli and make inferencesabout single-subject designs. This study alsoincluded more complex stimuli than the stimuli

Table 3

Number of Training and Symmetry Testing Blocks to Criterion

Participant A-B Train B-A Test A-C Train C-A Test A-D Train D-A Test

1 3 1 2 1 2 12 3 1 2 1 1 13 1 1 3 1 2 15 3 1 3 1 3 14 4 1 4 1 3 16 3 2 3 1 3 17 8 1 4 2 3 18 4 1 3 1 2 19 3 3 4 1 3 1

10 4 1 4 1 4 111 16 1 54a

12 2 2 2 1 3 113 1 1 1 1 1 114 3 1 3 1 4 115 2 1 2 1 3 1

a The participant did not meet criterion.


Figure 3. Posttest scores for symmetry and transitivity relations and pretest and posttest scores on equivalence andgeneralization relations for the equivalence group.


used in stimulus equivalence research withspecial populations. Previous research hasexamined relations that involved simpler stim-uli, such as pictures, letters, numbers, or singlewords (Cowley et al., 1992; Lynch & Cuvo,1995; Toussaint & Tiger, 2010). The lengthierdescriptions provided in the design definitionsand clinical vignettes in this study are moreappropriate for a college-level student and morereflective of complex relational skills. Allparticipants showed generalization of respond-ing to novel graphical stimuli, and six of the 14participants in the equivalence group showedgeneralization to novel clinical vignettes. Threeof four participants were able to tact bothtraining and generalization stimuli at theconclusion of the experiment.

Comparisons of the effectiveness of thestimulus equivalence protocol to a traditional,

lecture-teaching format revealed similar perfor-mance on paper-and-pencil quizzes and similarratings regarding participants’ opinions of thetwo instructional methods, despite the fact thatthere was a 30-min difference between the twoprocedures. Only two prior studies addressedparticipants’ satisfaction with instruction usingstimulus equivalence. Fields et al. (2009) foundthat participants exposed to a stimulus equiv-alence protocol provided higher ratings of boththeir understanding of statistical interactionsand their satisfaction with the instructionalmethod compared to a control group thatreceived no instruction (see also Fienup &Critchfield, 2011). Surprisingly, the presentfindings failed to show significantly higherratings for equivalence protocol instruction overlecture instruction. The Technology of Teaching(Skinner, 1968) leads us to expect thatparticipants would prefer instruction thatinvolves an active response component. Accord-ing to Skinner (1968), learning does not takeplace when a student is merely shown or toldinformation, as is the case in a traditionallecture (p. 103). Results from this study indicatethat, from the student’s perspective, passivelearning (i.e., being shown or told informationin a lecture) is equally as desirable as activelearning (i.e., the stimulus equivalence proto-col). A potential explanation for this finding isthat each participant experienced only oneinstructional method. A lecture format is thetypical teaching procedure used in a collegesetting. Without an alternative with which tocompare it, a student may rate that method asdesirable because it is the standard practice towhich they are accustomed. Skinner alsomaintained that recall of information mustbe directly reinforced for learning to occur.Responding for the equivalence group wasdirectly reinforced during training, yet perfor-mance on the paper-and-pencil quiz did notsurpass that of the lecture group. It is possiblethat a college student with a lengthy learning

Figure 4. Tact test results for Participants 12, 13, 14,and 15. Relations that involved trained stimuli aredepicted in the top graph, and relations that involvedgeneralization stimuli are depicted in the bottom graph.


history of exposure to lecture teaching is betterable to attend to the important aspects of thelecture (i.e., noting the repetition of the lectureror the words in boldface on the slides).

Also of consideration are the possiblelimitations of the lecture condition regardingthe degree to which it approached a trulytraditional lecture. In the typical researchmethods course from which participants wererecruited, approximately 50 students are presentfor lecture. With a class this size, potentiallydistracting stimuli are present (e.g., laptops,comments from peers, less relevant commentsfrom the instructor) that can divert studentattention from lecture material. In this study,lecture sessions included only one or twostudents, and this may have resulted inenhanced effects over the traditional lecturethat is presented to a large group of students.However, online distance learning courses ofteninclude recorded lectures that students watchindividually on the computer, and the lecturecondition in this study closely resembles theconditions under which a student enrolled in adistance education course might view a lecture.The lecture used in this study also differed froma traditional lecture because it was specificallydesigned for this experiment and was tailored topresent the relations targeted in the equivalenceprotocol. Modeling the lecture material on theequivalence protocol may have increased thelikelihood that participants would learn therelations among the stimuli from lecture alone.Considering these limitations of the lecturecondition, a comparison of performance fol-lowing the equivalence protocol with a morenaturalistic lecture delivered to a large classwarrants investigation (see also Critchfield &Fienup, 2010). Gains in performance aftertraining on the equivalence protocol may exceedperformance gains after this form of traditionallecture.

One of the key contributions of this study isthe demonstration of the emergence of atopography-based repertoire after selection-based

training. To date, few studies have examined theemergence of topography-based respondingmore complex than naming pictures or readingsight words in equivalence instruction (Cowleyet al., 1992; Toussaint & Tiger, 2010; Walkeret al., 2010). The topography-based tact re-sponse required in this study was of greatercomplexity than that seen in previous research,in that participants were required to inspectand interpret the graphs and read and inter-pret each clinical vignette. Furthermore, theemergence of a topography-based repertoirewas shown for the generalization stimuli aswell as the trained stimuli. These findings sug-gest that this method of instruction may resultin the emergence of a form of responding thatis more functional in daily life than emergentselection-based responses alone (Michael, 1985;Sundberg & Sundberg, 1990). Selection-basedresponses, such as pointing at or selectingstimuli, are not functional for a college studentwho must be able to read, describe, and discussmaterial to attain the competency to practice inhis or her future profession. This study is a smallstep in the direction of providing scientificinstructional methods that will produce suchcomplex repertoires.

Conclusions that can be drawn from theresults of the tact test in this study are limited,because no pretest for the tact responses wasincluded. However, all participants in theequivalence group received low pretest scoresfor the equivalence and generalization relations,and it is unlikely that they would have namedthe experimental design correctly when shown agraph or clinical vignette if they were not able toselect the appropriate design name during theselection-based pretest. Moreover, the topography-based repertoire was not evaluated for the lecturegroup, and it remains unclear if this procedurewould have resulted in the emergence of a tactrepertoire.

Another interesting aspect of our resultsconcerns the tests for generalization. Allparticipants in the equivalence group showed


generalization to novel graphs, but only sixparticipants showed generalization to novelclinical vignettes. This finding is not surprising,given the physical similarity between thegraphical stimuli. The differences among theappearances of phase-change lines, the presenceof several tiers in the multiple baseline designs,the intervention phase with multiple data pathsin the alternating treatments design, and thegradually changing criteria of the changingcriterion design provide more salient visual cuesthan the descriptions of the clinical vignettes. Inaddition to containing less salient visual cues,the vignettes required the participant to attendto multiple aspects of the description. Choiceof the appropriate design was dependent onattention to the number of behaviors, partici-pants, and settings in the description; themanner in which behavior was changed (i.e.,gradually or not); and the number of interven-tions. Due to the complexity of the vignettes,additional training may have been necessary forparticipants to show generalization to novelstimuli. Training with multiple exemplars ofstimuli previously has been shown to beeffective in promoting generalization. Ninnesset al. (2005) included multiple exemplars ofgraphs depicting mathematical functions duringtraining, and responding generalized to over 40novel graphs. A similar use of training withmultiple exemplars could enhance discrimina-tion of the relevant aspects of the clinicalvignettes used in this study.

Another difficulty with the vignettes is thefact that participants could successfully com-plete training and testing for equivalencerelations without reading the full descriptions.For example, several vignettes included a namefor the client that differed across vignettes. Aparticipant could respond accurately by attend-ing to the client name only. Training withmultiple exemplars of clinical vignettes couldhelp to overcome this difficulty. By requiringthe participant to respond appropriately to

several different examples of vignettes, thefeatures important to choosing the appropriatedesign may become apparent. An alternativemethod of ensuring attendance to the relevantaspects of the stimuli is to include verbal rulesthat describe the stimuli and the relationsamong them (e.g., Ninness et al., 2005).

The paper-and-pencil quiz used as a pre- andposttest also can be viewed as a test forgeneralization (Fields et al., 2009). Each quizquestion included either a variation of the designdefinition, a novel graph, or a novel clinicalvignette. Four of the 15 quiz questions included anovel clinical vignette and required the participantto select the appropriate design name. For theequivalence group, the lack of generalization tonovel vignettes in the stimulus equivalenceprotocol likely was reflected in quiz scores andaffected the differentiation in posttest scores forthe equivalence and lecture groups. Given thegeneralization failures and lack of group differ-ences on the paper-and-pencil quiz, one mightquestion the rationale for continued investigationsof stimulus equivalence protocols in highereducation if this instructional method does notsurpass that of standard educational practices.This issue is particularly important given that thelecture condition presented more information andrequired less time for participants to complete.Nonetheless, the lecture may well have requiredmore preparation time on part of the instructorthan did the equivalence condition. In fact, theadditional preparation time required for thislecture may not differ significantly from the timerequired to design a lecture that incorporates anactive student response component, such asresponse cards (Marmolejo, Wilder, & Bradley,2004) or guided notes (Neef, McCord, & Ferreri,2006). However, the likelihood of an instructoradopting a lecture format that requires greaterresponse effort remains a question for futureresearch. Furthermore, the lack of differencebetween the lecture and equivalence groups neednot devalue the pursuit of research on stimulusequivalence in education. A multitude of different


forms of teaching are likely effective, andinstructors can benefit from having a variety ofteaching methods available. Because each studenthas a unique learning history, it also may be thecase that some students will benefit more fromlecture teaching and others will succeed using anequivalence protocol. In addition, the twoapproaches used in the present study easily couldbe used as complementary teaching strategies.

REFERENCES

Cowley, B. J., Green, G., & Braunling-McMorrow, D.(1992). Using stimulus equivalence procedures toteach name-face matching to adults with braininjuries. Journal of Applied Behavior Analysis, 25,461–475.

Critchfield, T. S., & Fienup, D. M. (2010). Usingstimulus equivalence technology to teach statisticalinference in a group setting. Journal of AppliedBehavior Analysis, 43, 763–768.

Fields, L., Travis, R., Roy, D., Yadlovker, E., de Aguiar-Rocha, L., & Sturmey, P. (2009). Equivalence classformation: A method for teaching statistical interac-tions. Journal of Applied Behavior Analysis, 42,575–593.

Fienup, D. M., Covey, D. P., & Critchfield, T. S. (2010).Teaching brain-behavior relations economically withstimulus equivalence technology. Journal of AppliedBehavior Analysis, 43, 19–33.

Fienup, D. M., & Critchfield, T. S. (2010). Efficientlyestablishing concepts of inferential statistics andhypothesis decision making through contextuallycontrolled equivalence classes. Journal of AppliedBehavior Analysis, 43, 437–462.

Fienup, D. M., & Critchfield, T. S. (2011). Transport-ability of equivalence-based programmed instruction:Efficacy and efficiency in a college classroom. Journalof Applied Behavior Analysis, 44, 435–450.

Kennedy, C. H. (2005). Single-case designs for educationalresearch. Boston: Allyn & Bacon.

Lynch, D. C., & Cuvo, A. J. (1995). Stimulus equivalenceinstruction of fraction-decimal relations. Journal ofApplied Behavior Analysis, 28, 115–126.

Marmolejo, E. K., Wilder, D. A., & Bradley, L. (2004). Apreliminary analysis of the effects of response cards onstudent performance and participation in an upperdivision university course. Journal of Applied BehaviorAnalysis, 37, 405–410.

Michael, J. (1985). Two kinds of verbal behavior plus apossible third. The Analysis of Verbal Behavior, 3, 1–4.

Neef, N. A., McCord, B. E., & Ferreri, S. J. (2006).Effects of guided notes versus completed notes duringlectures on students’ quiz performance. Journal ofApplied Behavior Analysis, 39, 123–130.

Ninness, C., Barnes-Holmes, D., Rumph, R., McCuller,G., Ford, A. M., Payne, R., et al. (2006). Transfor-mations of mathematical and stimulus functions.Journal of Applied Behavior Analysis, 39, 299–321.

Ninness, C., Rumph, R., McCuller, G., Harrison, C.,Ford, A. M., & Ninness, S. K. (2005). A functionalanalytic approach to computer-interactive mathemat-ics. Journal of Applied Behavior Analysis, 38, 1–22.

Richards, S. B., Taylor, R. L., Ramasamy, R., & Richards,R. Y. (1999). Single subject research: Applications ineducational and clinical settings. Belmont, CA: Wads-worth Cengage Learning.

Sidman, M. (1971). Reading and auditory-visual equiv-alences. Journal of Speech and Hearing Research, 14,5–13.

Sidman, M. (1994). Equivalence relations and behav-ior: A research story. Boston: Authors Cooperative.

Sidman, M., & Cresson, O. (1973). Reading andcrossmodal transfer of stimulus equivalence in severeretardation. American Journal of Mental Deficiency,77, 515–523.

Skinner, B. F. (1957). Verbal behavior. New York:Appleton-Century-Crofts.

Skinner, B. F. (1968). The technology of teaching. NewYork: Appleton-Century-Crofts.

Sundberg, C. T., & Sundberg, M. L. (1990). Comparingtopography-based verbal behavior with stimulusselection-based verbal behavior. The Analysis of VerbalBehavior, 8, 31–41.

Toussaint, K. A., & Tiger, J. H. (2010). Teaching earlybraille literacy skills within a stimulus equivalenceparadigm to children with degenerative visual im-pairments. Journal of Applied Behavior Analysis, 43,181–194.

Walker, B. D., & Rehfeldt, R. A. (in press). Using thestimulus equivalence paradigm to teach single-subjectdesigns to college students via Blackboard. Journal ofApplied Behavior Analysis.

Walker, B. D., Rehfeldt, R. A., & Ninness, C. (2010). Usingthe stimulus equivalence paradigm to teach coursematerial in an undergraduate rehabilitation course.Journal of Applied Behavior Analysis, 43, 615–633.

Received July 26, 2010Final acceptance March 28, 2011Action Editor, Joel Ringdahl


· pdf filecomparison of a stimulus equivalence protocol and ... which is a verbal response...

Documents