This article was downloaded by: [University of Toronto Libraries]On: 07 December 2014, At: 12:00Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,37-41 Mortimer Street, London W1T 3JH, UK

Assistive Technology: The Official Journal of RESNAPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/uaty20

Use of Reading Pen Assistive Technology toAccommodate Post-Secondary Students with ReadingDisabilitiesAra J. Schmitt PhD a , Elizabeth McCallum PhD a , Jenna Hennessey MSEd a , Temple LovelacePhD a & Renee O. Hawkins PhD ba School of Education , Duquesne University , Pittsburgh , Pennsylvaniab School of Education , University of Cincinnati , Cincinatti , Ohio , USAAccepted author version posted online: 08 Feb 2012.Published online: 27 Sep 2012.

To cite this article: Ara J. Schmitt PhD , Elizabeth McCallum PhD , Jenna Hennessey MSEd , Temple Lovelace PhD & ReneeO. Hawkins PhD (2012) Use of Reading Pen Assistive Technology to Accommodate Post-Secondary Students with ReadingDisabilities, Assistive Technology: The Official Journal of RESNA, 24:4, 229-239, DOI: 10.1080/10400435.2012.659956

To link to this article: http://dx.doi.org/10.1080/10400435.2012.659956

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of theContent. Any opinions and views expressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon andshould be independently verified with primary sources of information. Taylor and Francis shall not be liable forany losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use ofthe Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Assistive Technology®, 24:229–239, 2012Copyright © 2012 RESNAISSN: 1040-0435 print/1949-3614 onlineDOI: 10.1080/10400435.2012.659956

Use of Reading Pen Assistive Technologyto Accommodate Post-Secondary Students

with Reading DisabilitiesAra J. Schmitt, PhD,1

Elizabeth McCallum, PhD,1

Jenna Hennessey, MSEd,1

Temple Lovelace, PhD,1 andRenee O. Hawkins, PhD2

1School of Education,Duquesne University,Pittsburgh, Pennsylvania2School of Education, Universityof Cincinnati, Cincinatti, Ohio

ABSTRACT Reading pens are a form of assistive technology that may beused to bypass weak word decoding and vocabulary skills of students withreading disabilities. Only two known studies have examined the effects ofreading pens on the comprehension of school-aged students, and no knownstudies have been published regarding post-secondary students. The presentstudy investigated the effects of reading pen assistive technology on thecomprehension accuracy and rate of three post-secondary students with uni-versity-recognized reading disabilities. An alternating treatments design wasimplemented to compare the effects of (1) a reading pen decoding accommo-dation, (2) concurrent reading pen decoding and vocabulary accommodations,and (3) a no-accommodation control condition on the comprehension ofthe three participants when provided college level difficulty reading passages.Results indicate that use of a reading pen did not uniformly improve thecomprehension of the post-secondary students. However, the student with thepoorest reading skills benefitted the most. Discussion focuses on explanationsfor the results, practical implications for post-secondary students with readingdisabilities, and future directions for study.

KEYWORDS reading accommodation, reading assistive technology, reading pen

Address correspondence to Ara J.Schmitt, Duquesne University, SchoolEducation, G3A Canevin Hall,600 Forbes Ave., Pittsburgh, PA 15282.E-mail: schmitta2106@duq.edu

INTRODUCTIONThe number of students with disabilities attending colleges and universities

has dramatically increased across recent decades. Henderson (2001) reportedthat the percentage of full-time post-secondary students who self-identify ashaving any type of disability rose from 2.3% in 1978 to 9% in 2000. Althoughstatistics vary regarding the types of disabilities represented by post-secondarystudents, there is some consensus that more than a third of these students havelearning disabilities (Harbour, 2004; Henderson, 2001). Of those students withdisabilities that attend college, the majority have been formally diagnosed witha reading disability as it is the most common learning disability across gradesand ages (Hudson, High, & Al Otaiba, 2007; Shaywitz, 2003).

229

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

12:

00 0

7 D

ecem

ber

2014

Post-secondary education is an important transi-tional outcome for students with learning disabilities.A college degree may serve as a protective factor againstcommonly experienced poor outcomes for studentswith disabilities, like employment obstacles. Madaus(2006) reported that students with disabilities whograduate from college do not differ from their non-disabled peers in terms of employment rates, benefits,and annual salaries. Accordingly, these graduates expe-rience more favorable outcomes across the lifespanthan individuals with disabilities who do not earncollege degrees (Shaw, Madaus, & Banerjee, 2009).

Section 504 of the 1973 Rehabilitation Act is afederal law designed to protect the rights of indi-viduals with disabilities engaged in any federallyfunded programming, including post-secondaryschools. Although Section 504 entitles elementaryand secondary students with disabilities to a free andappropriate public education (FAPE), post-secondaryschools are not held to this same standard. Instead,Section 504 mandates that post-secondary schoolsmust provide appropriate academic adjustments toqualified students based upon type of disability andindividual needs (Madaus & Shaw, 2004). Theseadjustments may include modifications to academicrequirements and, germane to this study, auxiliaryaids and services that may be applied as learningaccommodations.

Accommodations are not meant to create a compet-itive advantage, but “level the playing field” betweenstudents with and without disabilities by removingdisability-related impediments. Formally defined, anaccommodation is any method that removes construct-irrelevant barriers to accessing material (Ketterlin-Geller, Yovanoff, & Tindal, 2007). In the case of a col-lege student with a basic reading disability (dyslexia),the inability to fluently decode words impedes the stu-dent’s ability to comprehend material in textbooks andwritten course materials (Gillon, 2007; Morris et al.,1998; National Reading Panel, 2000; Shaywitz, 2003;Wolf & Bowers, 1999).

The listening-while-reading (LWR) accommodationallows students to follow along while listening to text inorder to improve comprehension. Several studies havedemonstrated that LWR may be used to improve theperformance of school-aged students with reading dif-ficulties (e.g., Hale et al., 2005; Rose, 1984; Rose &Beattie, 1986). However, these LWR studies requiredlive models (teachers or peers) to read aloud passages

or word lists. This LWR implementation procedure istaxing on human resources and is not likely realistic formost college students who must be independent, self-regulated learners. A means to self-regulate the LWRaccommodation for college students is through theuse of assistive technology devices (Lee & Templeton,2008; Raskind & Higgins, 1998).

Assistive technology devices include any pieceof equipment used to improve or maintain thefunctioning of individuals with disabilities (AssistiveTechnology Act, 1998; Wepner & Bowes, 2004). Theserun the gamut from low-tech tools, such as stopwatchesand metronomes, to high-tech devices, such as comput-erized accommodations (McCallum & Schmitt, 2011;Reed, 2007). An alternative to LWR using live read-ing models involves text-to-speech assistive technologythat converts written material to spoken language.These devices have been demonstrated to increase thecomprehension of primary and secondary studentswith reading disabilities (Elkind, Cohen, & Murray,1993; Izzo, Yurick, & McArrell, 2009; Montali &Lewandowski, 1996; Moorman, Boon, Keller-Bell,Stagliano, & Jeffs, 2010; Schmitt, McCallum, Hale,Obeldobel, & Dingus, 2009).

Few studies have investigated the use of text-to-speech assistive technology with college students.Existing data suggest that for many students, the com-prehension of material is improved when passages areread by the technology compared to a no-treatmentcontrol (Elkind, Black, & Murray, 1996; Higgins &Raskind, 1997). An interesting finding of these collegestudies is that comprehension gains are not uniformlypresent and when participant data are aggregated,changes in comprehension are not statistically present.That being said, data suggest that comprehension with-out use of the technology is inversely related to theamount of comprehension gains that come from use ofthe assistive technology accommodation. Those withthe greatest reading difficulties benefited the most fromthe text-to-speech technology. Similar to findings ofstudies conducted with school-aged students, however,comprehension of text was not raised to the averagerange. In sum, given the time and expense associatedwith text-to-speech software that reads entire passages,reading pens may be a more cost effective means forproviding LWR support.

The reading pen is a handheld optical characterrecognition (OCR) device that allows students to scanprinted text either a word or a line at a time. The

A. J. Schmitt et al. 230

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

12:

00 0

7 D

ecem

ber

2014

scanned text appears on a built-in LCD screen and canbe converted to speech through digital voice. Potentialbenefits of the reading pen as an accommodationfor college students with reading disabilities includeits portability, relatively low cost compared to OCRsystems requiring the use of a computer, and abilityto scan individual words instead of entire pages orchapters. In addition to decoding, and like other text-to-speech tools, reading pens allow students to accessdefinitions of unknown words.

To date, two empirical studies have been conductedevaluating the effectiveness of the reading pen technol-ogy as an accommodation for students with readingdisabilities. Higgins and Raskind (2005) found thatschool-aged students with significant reading disabili-ties were able to comprehend more difficult materialon a reading inventory using reading pens than theycould on their own (comprehension standard score of85 versus 80). This study did not test the hypothe-sis that reading pens could effectively accommodatereaders when provided grade-level material, however.Schmitt, McCallum, Rubinic, and Hawkins (2011)evaluated the use of reading pens on the reading com-prehension accuracy and rates of high school studentswith reading disabilities when provided grade-level pas-sages. Results indicated that reading pen use actuallyimpaired the comprehension of some students. Notonly did the students’ comprehension not improve,by the nature of using a reading pen, it took the stu-dents longer to finish individual passages. The authorssuggested that the use of the reading pens may havedisrupted the students’ reading fluency, which in turn,impeded their comprehension of the text.

Interestingly, these results are in contrast with theresults of the Higgins and Raskind (2005) reading penstudy. A possible explanation for the differences infindings between the two published reading pen stud-ies involves the assessment procedures and specifically,the levels of reading material provided. Higgins andRaskind found reading pens to aid students in compre-hending reduced grade-level material whereas Schmittet al. found reading pens to impede students’ compre-hension of on grade-level material. Despite these differ-ences, neither study found reading pen usage increasedreading performance to a functionally acceptable level.

Although reading pens are currently marketed foruse by college students, and have been suggested tobe appropriate for use in the post-secondary setting(Raskind & Higgins, 1998), research demonstrating the

effectiveness of this technology with college studentscould not be found after searching the peer-reviewedliterature in PsycInfo and ERIC databases. If found tobe effective, reading pens may be particularly benefi-cial for college students who must read college-leveltexts, even with Section 504 accommodations in place.Furthermore, the portability and self-regulated use ofthe reading pen may be particularly conducive to thecollege environment.

METHODParticipants

Participants included three undergraduate studentsenrolled in a Mid-Atlantic United States private uni-versity who were recognized by the school’s disabilityconcerns office as being students with reading disabil-ities. The university’s criteria for learning ability statuswas a significant discrepancy between a student’s abil-ity as measured by an individually administered IQtest, and reading achievement as measured by an indi-vidual achievement test. Each participant received ser-vices and accommodations through a Section 504 plan.In order to recruit these participants, an electronic invi-tation to participate in a study was emailed by thedisability concerns office to all students recognized ashaving a reading learning disability. Using this proce-dure, the students with reading disabilities remainedanonymous to the investigators. All three undergrad-uate students who contacted the primary investigatorwere included as participants in the study. Review of arecent psychological report by the investigators verifiedthe disability status of each participant. All participantsearned IQ scores within the Average range. Becauseindividual achievement testing was completed earlier ineach student’s university career, the researchers admin-istered three reading curriculum-based measurement(CBM) probes to each participant in order to establishthe most current oral reading fluency levels. As collegelevel CBM norms are not available, the performance ofeach was compared to a 12th grade student in the springusing AIMSweb (widely used CBM system) norms(AIMSweb, 2008).

Joseph was a 20-year-old, Caucasian freshman whosemajor involved a helping profession. Most recent eval-uation results revealed the presence of a reading disabil-ity. His median oral reading fluency performance was131 words correct per minute (below 10th percentile).

231 Reading Pen Assistive Technology

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

12:

00 0

7 D

ecem

ber

2014

The second participant, Tim, was a 21-year-oldCaucasian student who was identified as having areading disability, as well as receptive and expressivelanguage disorders. He was sophomore who majoredin a hard science. Tim’s median oral reading fluencyperformance was 144 words correct per minute (below10th percentile).

Liz was a 21-year-old, Caucasian student majoringin a social science. A reading disability was docu-mented in her most recent psychological evaluation.Her median oral reading fluency performance was81 words correct per minute (below 10th percentile).

MaterialsThe reading passages and comprehension questions

used in this study were taken from Timed Readings inLiterature (Spargo, 1989), a series used in previous inter-vention studies involving school-aged students (e.g.,Hale et al., 2005; Schmitt, Hale, McCallum, & Mauck,2011; Schmitt et al., 2011). This series contains a 13th

(college) grade-level book comprised of 50, 400 wordpassages. The difficulty of passages within each gradelevel book was controlled by using Fry’s readabilityindex (1968). This index calculates readability by con-sidering the number of syllables in words and the num-ber of sentences present in each passage. Each passageis approximately one 50th of a grade-level more diffi-cult than the preceding story. Following each passageare 10 multiple choice comprehension questions (5 fac-tual and 5 inferential). Only the first 30 passages andsets of comprehension questions were photocopied foruse by the participants. Passages were counterbalancedacross conditions and sessions to control for passagedifficulty. No participant was exposed to a specificreading passage twice across the study’s procedures.

During the decoding accommodation (RP-D) con-dition and decoding plus vocabulary accommodations(RP-DV) condition, students were provided with aReadingpen® Advanced Edition (2006) as developedby Wizcom Technologies. The handheld Readingpenis placed against the page before the word(s) of inter-est and then swept across the text in a fluid motion.The scanned content then appears in black letters onthe Readingpen’s LCD screen. Buttons (arrows) onthe Readingpen are then used to select word pro-nunciation, syllabication, or definition. The studentalso may elect to have this information, including aword’s definition, read aloud by a digital voice. The

Readingpen also allows the dictionary function to belocked from use. Earbuds can be worn to avoid dis-tracting others and volume may be adjusted to acomfortable level. This study locked the vocabularyfunction and allowed access to the decoding func-tion for the RP-D condition, and permitted use ofthe decoding and vocabulary functions for the RP-DVcondition.

Prior to beginning the experimental sessions, a par-ticipant Readingpen competency checklist was com-pleted. An intervention integrity checklist was used tocollect treatment integrity data and stopwatches wereused to record the amount of time students spentreading the passages. Participants completed an inter-vention acceptability rating form at the conclusion ofthe study.

General Experimental ProceduresData collection was scheduled for 10 sessions across

consecutive days of student availability. Session onewas comprised of a 45 min training protocol designedby the first author. The protocol consisted of anexplanation of Readingpen functions, research assistantmodeling of functions, guided student practice, and achecklist to document participant ability to fluidly usethe technology. Specifically, it was explained that thepen could be used to scan unknown printed words thatwould then appear on the LCD screen. At that point,buttons (arrows) on the pen could be used to selectthe function that would be of assistance. In the case ofthis study, the functions were access to audible pronun-ciation of words and definitions of words, dependingon treatment condition. The participant was then pro-vided a reading passage and prompted to use thereading pen functions until the participant felt com-fortable using the technology. The 45 minute trainingsession culminated with the completion of a perfor-mance checklist to verify each participant was ableto effectively use functions of the Readingpen. Thechecklist included components such as fluidly scannedunknown words, efficiently retrieved the audible pro-nunciation of a scanned word, efficiently retrieved thedefinition of a scanned word, efficiently retrieved theaudible presentation of the definition, and so forth.Researchers also administered the three CBM probesused to determine current reading fluency levels dur-ing the first session. The experimental procedures werethen applied across the remaining nine sessions. All

A. J. Schmitt et al. 232

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

12:

00 0

7 D

ecem

ber

2014

procedures were individually conducted in a distrac-tion free room within the university’s reading clinic.Each intervention session, which lasted approximately20 min, was comprised of 3 counterbalanced condi-tions (i.e., RP-D, RP-DV, and C). A script was usedto ensure uniform and thorough instructions for allparticipants. Scheduled sessions were completed by allparticipants. At the end of the study, rating forms werecompleted by each student to assess acceptability ofReadingpens as an accommodation. To acknowledgestudent participation, each was provided with a giftcard at the conclusion of the study.

Intervention ProceduresNo-accommodation control condition

During the control condition, participants read pas-sages and answered questions without any Readingpenassistance. Participants were given the followinginstructions prior to the start of this condition: “Youare going to read a short story on your own. Read itvery carefully to yourself because when you are fin-ished you are going to answer some questions aboutwhat you read. Are you ready?” After providing anynecessary clarification, the researcher said, “Begin,”and started the stopwatch. When the student finishedreading the passage, the researcher recorded the timespent reading the passage in seconds. The participantthen was provided the corresponding 10 comprehen-sion questions on a sheet of paper. Participants wereinstructed to carefully read each question and circle thebest answer. Unlimited time was available to completethe comprehension questions.

Decoding accommodation condition

At the start of the RP-D condition, participants weregiven a Readingpen accompanied by earbuds with avolume control. Participants were allowed to adjust thevolume as needed. The vocabulary function was lockedfor use so that only the decoding function could beaccessed. The participants were provided the followinginstructions:

“You are going to read a short story using a Readingpen.As you read the passage, you may use the Readingpen to havewords read aloud to you. Read very carefully because when youare finished you are going to answer some questions about whatyou read. Are you ready?”

After answering any questions, the researcher said,“Begin,” and started the stopwatch. The number oftimes the Readingpen was used in the session wastracked to confirm that the participant accessed theavailable accommodation. When the participant fin-ished reading the passage, the stopwatch was stoppedand the time spent reading the passage was recordedin seconds. The associated comprehension questionswere provided on a sheet of paper and participants wereinstructed to carefully read each question and circle thebest answer. Unlimited time to complete the questionswas available. The participants were allowed to use thedecoding function of the Readingpen on the passageand comprehension questions.

Decoding and vocabulary accommodationscondition

At the start of the RP-DV condition, participantswere given a Readingpen accompanied by earbuds witha volume control. Participants adjusted the volume to acomfortable level. Prior to the start of the RP-DV con-dition, the researcher ensured the vocabulary functionwas unlocked so that participants could access both thedecoding and vocabulary functions. The participantswere read the following directions:

“You are going to read a short story using a Readingpen.As you read the passage, you may use the Readingpen to havewords read aloud to you and get the definitions of unknownwords. If you would like for the Readingpen to read the defi-nitions to you, that is allowed too. Read very carefully becausewhen you are finished you are going to answer some questionsabout what you read. Are you ready?”

After providing any necessary clarification, the stop-watch was started. Each instance the participant usedthe Readingpen was recorded to document that theaccommodations were being used. When the partici-pant finished the passage, the stopwatch was stoppedand the time spent reading the passage was recorded inseconds. The corresponding 10 comprehension ques-tions were provided with the instructions to carefullyconsider each question and circle the best answer.Like the other conditions, unlimited time to com-plete the questions was available. The participants wereallowed to use the decoding and vocabulary functionsof the Readingpen on the passage and comprehensionquestions as needed. Comprehension questions werereviewed to ensure that the vocabulary function wouldnot directly provide the correct answer.

233 Reading Pen Assistive Technology

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

12:

00 0

7 D

ecem

ber

2014

Experimental Design andDependent Variables

Similar to previous reading accommodation stud-ies (e.g., Hale et al., 2005; Higgins & Raskind, 1997;Schmitt, McCallum, Hale, Obeldobel, & Dingus,2009), an alternating-treatments design was used tocompare the effects of the two accommodation condi-tions (RP-D and RP-DV) and the no-accommodationcontrol condition on reading comprehension. Theconditions were counterbalanced across sessions tocontrol for order effects. This type of design allows forthe three conditions to be applied to equivalent taskswithin an experimental session that also accounts forhistory and spillover effects (Poncy, Skinner, & Jaspers,2007; Sindelar, Rosenberg, & Wilson, 1985; Skinner &Shapiro, 1989).

The dependent variables of this study includedcomprehension accuracy and comprehension rate.Comprehension accuracy per student, condition, andsession was determined by the number of correctlyanswered multiple-choice comprehension questions(out of 10). Comprehension rate was also computedper student, condition, and session. Comprehensionrate was calculated by dividing the number of cor-rectly answered questions by the number of secondsspent engaged with the passage and then multiplyingby 60 (Skinner, 2008). Effect sizes regarding the depen-dent variables were computed by subtracting the meanperformance of two conditions and then dividing bythe pooled standard deviation of the two conditions(Weiner, Sheridan, & Jenson, 1998).

Treatment Integrity and InterscorerAgreement

A treatment integrity protocol was constructed foreach condition. This checklist determined whetherproper procedures were followed for each condition.For example, the checklist for the RP-D conditionincluded the following: locked the vocabulary func-tion, provided correct passage, read decoding condi-tion script, started timer, stopped timer when studentfinished reading and recorded elapsed time, providedcomprehension questions and read comprehensionscript, and recorded number of times the Readingpenwas used. One session per participant was observed bya second researcher to gather treatment integrity data.As another estimate of treatment integrity, use of the

reading pen was closely monitored and recorded byan experimenter. A second researcher also reviewed 24(30%) of the comprehension assessments for scoringaccuracy.

RESULTSIn order to assess the impact of reading pen accom-

modations on post-secondary students, the readingcomprehension accuracy and rates of three undergrad-uate college students was obtained across two readingpen accommodation conditions (RP-V and RP-DV)and a no-accommodation control condition. Resultsfor comprehension accuracy are presented first, fol-lowed by the results for comprehension rate. Withrespect to treatment integrity, results showed that theprimary researcher implemented the experimental pro-cedures with 100% accuracy. With the exception of asingle RP-D passage (Joseph), each of the three par-ticipants used the assistive technology at least onetime across each of the 18 passages on which use ofthe Readingpen was permitted. Joseph’s median useof the Readingpen was one time per accommoda-tion condition. Tim’s median use was two times peraccommodation condition. Liz’s median use of theReadingpen functions was four times per accommoda-tion condition. One hundred percent agreement waspresent regarding scoring accuracy of the comprehen-sion assessments.

Data Analysis of ComprehensionAccuracy

Consistent across all three students, comprehen-sion accuracy did not incrementally increase from thecontrol condition, to the RP-D condition, and thento the RP-DV condition. Figure 1 displays students’comprehension accuracy as a function of conditionand Table 1 includes the means and standard devia-tions of correctly answered comprehension questions.For Joseph, comprehension accuracy was best in theRP-D condition, followed by the control condition,and then the RP-DV condition. For Tim, the RP-Dcondition resulted in the greatest comprehension accu-racy, trailed by identical performances in the controland RP-DV conditions. Liz’s comprehension accuracywas highest in the RP-D and RP-DV conditions, fol-lowed by the control condition. Also common acrossparticipants, RP-D resulted in greater comprehension

A. J. Schmitt et al. 234

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

12:

00 0

7 D

ecem

ber

2014

FIGURE 1 Comprehension accuracy across participants, con-ditions, and sessions.

accuracy compared to RP-C. Visual analysis of Figure 1reveals that an assistive technology accommodationcondition resulted in greater comprehension accuracyin 6 of 9 sessions for Joseph (4 RP-D, 2 RP-DV), 7 of9 sessions for Tim (4 RP-D, 3 RP-DV) and all 9 sessionsfor Liz (4 RP-D, 5 RP-DV).

Effect size data reported in Table 2 reflect effect sizesranging from negligible (.10 for Joseph and .11 for Tim)to essentially large (.79 for Liz). Effect sizes are inter-preted consistent with the guidelines of Cohen (1992).There were no appreciable differences in performancebetween the control and RP-DV conditions across par-ticipants, for Joseph and Tim. In contrast, an effect sizeof 1.15 indicates that Liz’s comprehension accuracywas meaningfully higher in the RP-DV condition com-pared to the control condition. The RP-D conditionresulted in greater comprehension accuracy comparedto the RP-DV condition for Joseph and Tim (effectsizes of .01 and .09, respectively). No difference waspresent for Liz.

Data Analysis ofComprehension Rate

As stated earlier, comprehension rate is an indexof the number of correctly answered comprehensionquestions given time spent engaged with the pas-sage. Figure 2 displays students’ comprehension rateas a function of condition and Table 1 includes themeans and standard deviations for comprehensionrate. For Tim and Joseph, the greatest comprehensionrate resulted from the control condition, followed bythe RP-D condition, and then the RP-DV condition.For Liz, comprehension rate was equally high in thecontrol and RP-D conditions, followed by the RP-DV condition. Comprehension rate was lowest in theRP-DV condition for all three participants.

TABLE 1 Mean and standard deviation of correctly answered comprehension questions across participantsand conditions

Mean number correct (SD) Mean comprehension rate (SD)

C RP-D RP-DV C RP-D RP-DV

Joseph 5.22 (2.17) 5.44 (2.35) 5.11 (1.90) 1.07 (0.53) 0.93 (0.36) 0.82 (0.39)Tim 6.78 (1.79) 7.00 (2.34) 6.78 (2.44) 1.61 (0.56) 1.13 (0.47) 0.92 (0.44)Liz 4.89 (1.45) 6.44 (2.46) 6.44 (1.24) 0.46 (0.14) 0.46 (0.20) 0.40 (0.09)

Note. C = Control condition. RP-D = Decoding accommodation condition. RP-DV = Decoding and vocabularyaccommodations condition. Maximum mean number correct per condition is 10.

235 Reading Pen Assistive Technology

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

12:

00 0

7 D

ecem

ber

2014

TABLE 2 Comprehension accuracy and rate effect size across participants and between conditions

Effect sizeDependent

variable C vs. RP-D C vs. RP-DV RP-D vs. RP-DV

Joseph Accuracy ES = 0.10 D > C ES = 0.05 C > DV ES = 0.15 D > DVRate ES = 0.31 C > D ES = 0.54 C > DV ES = 0.29 D > DV

Tim Accuracy ES = 0.11 D > C ES = 0 C = DV ES = 0.09 D > DVRate ES = 0.92 C > D ES = 1.38 C > DV ES = 0.46 DV > D

Liz Accuracy ES = 0.79 D > C ES = 1.15 DV > C ES = 0 D = DVRate ES = 0 C = D ES = 0.50 C > DV ES = 0.40 D > DV

Note. C = Control condition. RP-D = Decoding accommodation condition. RP-DV = Decoding and vocabularyaccommodations condition.

FIGURE 2 Comprehension rate across participants, condi-tions, and sessions.

Effect size data reported in Table 2 support thevisual analysis. For Joseph and Tim, and in favor ofthe control condition, effect sizes comparing the con-trol condition to RP-D were small for Joseph (.31) andlarge for Tim (.92). No difference in comprehensionrate was indicated for Liz. With respect to the controlcondition compared to the RP-D condition, compre-hension rate was higher in the control condition forJoseph, Tim, and Liz with effect sizes of .54 (moder-ate), 1.38 (large), and .50 (moderate), respectively. ForJoseph and Liz, the RP-D condition resulted in greatercomprehension rate than the RP-DV condition witheffect sizes of .29 (small) and .40 (moderate). The RP-DV produced higher comprehension rate compared toRP-D with a small effect size of .46.

Accommodation AcceptabilityAll participants were administered a brief treatment

acceptability rating form. Two of the participants,Joseph and Tim, found the Readingpen accommoda-tions enjoyable while Liz indicated that she some-what enjoyed using the Readingpen technology. Allthree students reported the Readingpen was helpful indecoding unknown words and looking up definitionsof unknown words. Two of the participants reportedthat they would use the Readingpen technology again;however, Liz indicated that she would possibly use thistechnology again. Unexpected was that Liz, for whomthe Readingpen provided the most benefit, rated thetechnology somewhat lower that the other two partic-ipants. Plausible is that Liz’s cognitive resources wereso devoted to the task of word decoding that the ben-efit to her comprehension was not fully recognized.On the other hand, Joseph and Tim indicated they

A. J. Schmitt et al. 236

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

12:

00 0

7 D

ecem

ber

2014

would use the Readingpen again, but unknown tothem, it did not provide as much a benefit comparedto Liz. This finding is consistent with previous stud-ies involving text-to-speech assistive technology (e.g.,Schmitt et al., 2011). For some individuals it appearsthat attention is drawn more to the technological capa-bilities of the Readingpen than whether or not anychanges in reading comprehension occurred.

DISCUSSIONPrior to this study, only one known inquiry inves-

tigated the effectiveness of reading pen text-to-speechassistive technology on the comprehension of stu-dents. In their sample of high school students withreading disabilities, reading pen accommodations werefound to increase student performance on a standard-ized measure of reading comprehension (Higgins &Raskind, 2005). The present study sought to exam-ine the effects of reading pen accommodations onthe comprehension of college students when providedcollege level difficulty reading passages. This investi-gation also sought to contribute to the literature byexamining the effects of reading pen accommodationson college student comprehension rates, or compre-hension accuracy given time spent engaged with thepassage. Reading pen assistive technology has beensuggested to be a tool useful for college students toincrease their access to content in courses; however,results of the current study indicate that the effective-ness of the technology may depend on the individualstudent.

For two of the students, access to and use of readingpen accommodations did not meaningfully improvetheir comprehension of college-level passages. On thesurface it would seem that additional decoding andvocabulary assistance would improve the comprehen-sion of the two college students. These results aresimilar to Schmitt et al. (2011) who found readingpen accommodations did not improve the comprehen-sion of high school students with reading disabilities.In fact, the researchers found that the use of the tech-nological accommodations hindered comprehension.Studies have consistently demonstrated that readingfluency is essential for comprehension (e.g., Deno,Marston, Shinn, & Tindal, 1983; Fuchs, Fuchs, &Maxwell, 1988; Meisenger, Bloom, & Hynd, 2010;National Reading Panel, 2000; Rasinski, 1990). For

some students, the act of disengaging from the passageand then manipulating the reading pen appears to dis-rupt fluency to the extent that any benefit providedby the accommodation is lost. One way to quan-tify this phenomenon is by examining comprehensionrates.

Joseph’s and Tim’s learning rates were negativelyimpacted by use of the reading pen accommoda-tions. That is, not only did their comprehension notappreciably improve (or improve at all), the studentsspent more time working through the passages by thenature of the necessity to operate the reading pens.To illustrate, Tim’s comprehension rate was signifi-cantly greater in the control condition than the RP-DVcondition with an effect size of 1.38. This issue isparticularly poignant for college students whose timeis often limited given class, extracurricular, and workschedules. For these two students, use of a reading penmay not make sense as any benefit was minimal and ittook up more of their time to use the technology.

On the other hand, Liz did appear to benefit fromusing the reading pens. Her comprehension accuracyimproved when using the reading pen accommoda-tions as reflected by generally large effect sizes (e.g.,.79 for RP-D and 1.15 for RP-DV compared to the con-trol condition). Although continuous text-to-speechassistive technology was used to read entire passages,this result is consistent with previous research thatfound those with the greatest reading disabilities bene-fitted the most from the technological accommodation(Elkind et al., 1996; Higgins & Raskind, 1997). Eachof the participants’ oral reading fluency was measuredprior to the start of the experiment. Of note, Liz’s oralreading fluency was markedly lower than Joseph’s andTim’s (81, 131, and 144, respectively). It is hypoth-esized that Liz’s reading was already so labored thatthe disruption to fluency caused by use of the read-ing pen was of no consequence. As Table 2 reveals,Liz’s comprehension rates did decrease with increasedaccess to the technology’s functions, and particularlywhen access to the decoding and vocabulary functionswere enabled. Again, this indicates that despite gainsin comprehension accuracy, it took her much longerto progress through the passages using the technol-ogy. For a college student like Liz, one must considerthe benefit of the increased comprehension in lightof the time it takes to complete passages using areading pen.

237 Reading Pen Assistive Technology

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

12:

00 0

7 D

ecem

ber

2014

Practice ImplicationsThe practical implications of this investigation par-

allel previous studies in the text-to-speech assistivetechnology literature. First, individual college studentswith reading disabilities should not assume that read-ing pens will necessarily improve their comprehensionof text. Although variability in response was presentamong the participants, consistent was that the tech-nology did not normalize their reading comprehen-sion. Therefore, the technology did not appear tofacilitate absolute access to curricular content acrosssessions. Other interventions like repeated reading; sur-vey, questions, read, recite, and restate (SQ3R); andpeer tutoring should also be attempted (see Daly,Chafouleas, & Skinner, 2005 for reviews of these inter-ventions). Still, of the three participants, one student’scomprehension did appear to improve compared tothe control condition. That student demonstrated thepoorest oral reading fluency of the three participants.This is consistent with previous text-to-speech litera-ture which suggests that those with the greatest readingdisability may benefit the most from text-to-speechassistive technology. Another practical implication ofthis study is that use of reading pens takes additionaltime compared to silent reading, and no benefit maybe evident. Like school-aged students (McGivern &McKevitt, 2002), college students are strongly encour-aged to use the assistive technology on a trial basis todetermine if it improves their comprehension and ifthe time it takes to read with the reading pen remainsreasonable.

Limitations and Future ResearchAlthough very appropriate for the implemented

experimental design, this study included only threeparticipants and in depth analyses across participantswas not possible. Future studies should strive toinclude a greater number of participants in order totease apart for whom reading pens may meaningfullyimprove comprehension of text. Like Elkind et al.(1996) and Higgins and Raskind (1997), a larger samplewould permit the analysis of treatment gains in light ofreading disability severity. A strength of the researchdesign was that the readability of the passages weretightly controlled and the participants were exposed tonumerous unfamiliar passages. This required the useof a literature series commonly used in interventionstudies. Future studies should consider studying the

effects of reading pens on the comprehension of actualcollege textbooks since reading for memorization, suchas for a test, may influence how a student engages a pas-sage (Linderholm, 2006). Much research is still neededon use of reading pen text-to-speech assistive tech-nology across populations. Continued research shouldfocus using reading pen accommodations in othereducational contexts, such as understanding instructor-provided notes, test preparation, and reading scholarlymanuscripts.

REFERENCESAIMSweb. (2008). AIMSweb growth table: Reading curriculum based

measurement multi-year aggregate. Retrieved from http://aimsweb.com/

Assistive Technology Act (1998). Pub L. No. 105–394.Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159.Daly, E., Chafouleas, S., & Skinner, C. H. (2005). Interventions for reading

problems: Designing and evaluating effective strategies. New York,NY: Guilford Press.

Deno, S. L., Marston, D., Shinn, M. R., & Tindal, G. (1983). Oral read-ing fluency: A simple datum for scaling reading disability. Topics inLearning and Learning Disabilities, 2, 53−59.

Elkind, J., Black, M. S., & Murray, C. (1996). Computer-based compensa-tion of adult reading disabilities. Annals of Dyslexia, 46, 159–186.

Elkind, J., Cohen, K., & Murray, C. (1993). Using computer-based readersto improve reading comprehension of students with dyslexia. Annalsof Dyslexia, 43, 238–259.

Fry, E. (1968). A readability formula that saves time. Journal of Reading,11, 513–516.

Fuchs, L. S., Fuchs, D., & Maxwell, L. (1988). The validity of informalreading comprehension measures. Remedial and Special Education,9, 20−28.

Gillon, G. T. (2007). Phonological Awareness: From Research to Practice.Challenges in Language and Literacy. New York, NY: The GuilfordPress.

Hale, A. D., Skinner, C. H., Winn, B. D., Oliver, R., Allin, J. D., &Molloy, C.C.M (2005). An investigation of listening and listening-while-reading accommodations on reading comprehension levels andrates in students with emotional disorders. Psychology in the Schools,42, 39–51.

Harbour, W. S. (2004). Final report: The 2004 AHEAD survey of highereducation disability services providers. Waltham, MA: The Associationon Higher Education and Disability.

Henderson, C. (2001). College freshmen with disabilities, 2001: ABiennial statistical profile. Washington, DC: American Council onEducation

Higgins, E. L., & Raskind, M. H. (1997). The compensatory effective-ness of optical character recognition/speech synthesis on the readingcomprehension of postsecondary students with learning disabilities.Learning Disabilities: A Multidisciplinary Journal, 8, 75–87.

Higgins, E. L., & Raskind, M. H. (2005). The compensatory effectivenessof the Quicktionary Reading Pen II on the reading comprehensionof students with learning disabilities. Journal of Special EducationTechnology, 20, 31–40.

Hudson, R. F., High, L., & Al Otaiba, S. (2007). Dyslexia and thebrain: What does current research tell us? Reading Teacher, 60,506–515.

Izzo, M., Yurick, A., & McArrell, B. (2009). Supported eText: Effectsof text-to-speech on access and achievement for high school stu-dents with disabilities. Journal of Special Education Technology, 24,9–20.

A. J. Schmitt et al. 238

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

12:

00 0

7 D

ecem

ber

2014

Ketterlin-Geller, L. R., Yovanoff, P., & Tindal, G. (2007). Developinga new paradigm for conducting research on accommodations inmathematics testing. Exceptional Children, 73, 331–347.

Lee, H., & Templeton, R. (2008). Ensuring equal access to technology:Providing assistive technology for students with disabilities. TheoryInto Practice, 47, 212–219.

Linderholm, T. (2006). Reading with purpose. Journal of College Readingand Learning, 36, 70–80.

Madaus, J. W. (2006). Employment outcomes of university gradu-ates with learning disabilities. Learning Disability Quarterly, 29,19–31.

Madaus, J. W., & Shaw, S. F. (2004). Section 504: Differences in the reg-ulations for secondary and postsecondary education. Intervention inSchool and Clinic, 40, 81–87.

McCallum, E., & Schmitt, A. J. (2011). Using technology in schools toenhance student performance. Journal of Evidence-Based PracticesFor Schools, 12, 129–130.

McGivern, J. E., & McKevitt, B. C. (2002). Best practices in working withstudents using assistive technology. In A. Thomas & J. Grimes (Eds.),Best practices in school psychology IV (pp. 1537–1553). Bethesda,MD: National Association of School

Meisinger, E. B., Bloom, J. S., & Hynd, G. W. (2010). Reading fluency:Implications for the assessment of children with reading disabilities.Annals of Dyslexia, 60, 1–17.

Montali, J., & Lewandowski, L. (1996). Bimodal reading: Benefits of a talk-ing computer for average and less skilled readers. Journal of LearningDisabilities, 29, 271–279.

Moorman, A., Boon, R. T., Keller-Bell, Y., Stagliano, C., & Jeffs, T.(2010). Effects of text-to-speech software on the reading rate andcomprehension skills of high school students with specific learn-ing disabilities. Learning Disabilities: A Multidisciplinary Journal, 16,41–49.

Morris, R. D., Stuebing, K. K., Fletcher, J. M., . . . Shaywitz, S. E., Lyon, G.,Shankweiler, D. P., & Shaywitz, B. A. (1998). Subtypes of reading dis-ability: Variability around a phonological core. Journal of EducationalPsychology, 90, 347–373.

National Reading Panel. (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading andits implications for reading instruction. Washington, DC: NationalInstitute for Child Health and Human Development.

Poncy, B. C., Skinner, C. H., & Jaspers, K. E. (2007). Evaluating and com-paring interventions designed to enhance math fact accuracy andfluency: Cover, copy, and compare versus taped problems. Journal ofBehavioral Education, 16, 27–37.

Rasinski, T. V. (1990). Investigating measures of reading fluency.Educational Research Quarterly, 14, 37−44.

Raskind, M. H., & Higgins, E. L. (1998). Assistive technology for post-secondary students with learning disabilities: An Overview. Journal ofLearning Disabilities, 31, 27–40.

Readingpen® Advanced Edition [Apparatus]. (2006). Westford, MA:Wizcom Technologies.

Reed, P. R. (2007). A Resource guide for teachers and administra-tors about assistive technology. Oshkosh, WI: Wisconsin AssistiveTechnology Initiative.

Rose, T. L. (1984). The effects of two prepractice procedures on oralreading. Journal of Learning Disabilities, 17, 544–548.

Rose, T. L., & Beattie, J. R. (1986). Relative effects of teacher-directedand taped previewing on oral reading. Learning Disability Quarterly,9, 193–199.

Schmitt, A. J., Hale, A. D., McCallum, E., & Mauck, B. (2011).Accommodating remedial readers in the general education setting:Is listening-while-reading sufficient to improve factual and inferentialcomprehension? Psychology in the Schools, 48, 37–45.

Schmitt, A. J., McCallum, E., Hale, A. D., Obeldobel, E., & Dingus, K.(2009). Can text-to-speech assistive technology improve the read-ing comprehension of students with severe reading and emotionaldisabilities? Journal of Evidence-Based Practices for Schools, 10,95–115.

Schmitt, A. J., McCallum, E., Rubinic, D., & Hawkins, R. (2011). Readingpen decoding and vocabulary accommodations: Impact on studentcomprehension accuracy and rate. Journal of Evidence Based Practicesfor Schools, 12, 129–130.

Shaw, S. F., Madaus, J. W., & Banerjee, M. (2009). Enhance access topostsecondary education for students with disabilities. Intervention inSchool and Clinic, 44, 185–190.

Shaywitz, S. (2003). Overcoming dyslexia: A new and complete science-based program for reading problems at any level. New York, NY:Alfred A. Knopf.

Sindelar, P. T., Rosenberg, M. S., & Wilson, R. J. (1985). An adapted alter-nating treatments design for instructional research. Education andTreatment of Children, 8, 67–76.

Skinner, C. H. (2008). Theoretical and applied implications of preciselymeasuring learning rates. School Psychology Review, 37, 309–314.

Skinner, C. H., & Shapiro, E. S. (1989). A comparison of taped-wordsand drill interventions on reading fluency in adolescents with behaviordisorders. Education and Treatment of Children, 12, 123–133.

Spargo, E. (1989). Timed readings in literature. Providence, RI:Jamestown.

Weiner, R. K., Sheridan, S. M., & Jenson, W. R. (1998). The effects ofconjoint behavioral consultation and a structured homework programon math completion and accuracy in junior high students. SchoolPsychology Quarterly, 13, 281–309.

Wepner, S. B., & Bowes, K. A. (2004). Using assistive technology for liter-acy development. Reading & Writing Quarterly: Overcoming LearningDifficulties, 20, 219–223.

Wolf, M., & Bowers, P. (1999). The double-deficit hypothesis for thedevelopmental dyslexias. Journal of Educational Psychology, 91,415–438.

239 Reading Pen Assistive Technology

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

12:

00 0

7 D

ecem

ber

2014