an introduct an introduction to educational design research · an introduction to educational...
TRANSCRIPT
![Page 1: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/1.jpg)
An Introduction to Educational Design Research
SLO • Netherlands institute for curriculum developmentSLO
PO box 20417500 CA EnschedeThe Netherlands
T +31(0)53 484 08 40F +31(0)53 430 76 92E [email protected]
www.slo.nl
SLO is the Netherlands institute for curriculum development. We are bridging the contexts of policy, research, and practice. Our expertise focuses on the development of curricular goals and content for various educational levels, from national policy to classroom practices. We closely collaborate with many different stakeholders from policy circles, schools (boards, principals, teachers), research, civic organizations, and the society at large. This allows us to design and validate relevant curriculum frameworks, to elaborate exemplary materials and to evaluate these in school practices. Our products and services support both policy makers and schools and teachers in making substantive curricular decisions and in elaborating these into relevant, inspiring and effective education.
ISBN: 978 90 329 2329 7
An In
trod
uctIo
n to
Edu
cAtIon
Al dEsIg
n rEsEArch
Editors: tjeerd Plomp &
nienke n
ieveenEditors:Tjeerd Plomp & Nienke Nieveen
![Page 2: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/2.jpg)
![Page 3: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/3.jpg)
An Introduction to Educational Design Research
SLO •Netherlands institute for curriculum development
Tjeerd Plomp & Nienke Nieveen (editors)
Proceedings of the seminar conducted at the East China Normal University, Shanghai (PR China), November 23-26, 2007
![Page 4: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/4.jpg)
an introduction to educational design research2
Authors: JanvandenAkker BrendaBannan AnthonyE.Kelly NienkeNieveen TjeerdPlompEditors: TjeerdPlomp NienkeNieveen
Design: AxisMedia-ontwerpers,Enschede
Production: Netzodruk,Enschede
AN: 7.5115.183
ISBN: 9789032923297
Orderaddress: SLO P.O.Box2041 7500Enschede theNetherlands
www.slo.nl/organisatie/international/publications
Enschede,3rdprintMarch2010
Colophon
![Page 5: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/5.jpg)
an introduction to educational design research 3
Contents
Preface 5
1. EducationalDesignResearch:anIntroduction 9 Tjeerd Plomp
2. CurriculumDesignResearch 37 Jan van den Akker
3. TheIntegrativeLearningDesignFramework:AnIllustratedExamplefrom theDomainofInstructionalTechnology 53 Brenda Bannan
4 WhenisDesignResearchAppropriate? 73 Anthony E. Kelly
5 FormativeEvaluationinEducationalDesignResearch 89 Nienke Nieveen
6 ReferencesandSourcesonEducationalDesignResearch 103 Tjeerd Plomp and Nienke Nieveen
Authorbiographies 125
![Page 6: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/6.jpg)
![Page 7: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/7.jpg)
an introduction to educational design research 5
PrefaceThisbookistheresultofaseminaron‘educationaldesignresearch’organizedfromNovem-ber23-26,2007,byProfZhuZhiting(DepartmentofEducationalTechnology)oftheCollegeofEducationalSciencesattheEastChinaNormalUniversityinShanghai(PRChina).Theprimarygoaloftheseminarwasto introduce a group postgraduate students and lectu-ring staff in China to educational design research as a research approach.Thesecondgoaloftheseminarwastoprepare,basedonthecontributionsofanumberinternationalexperts,proceedings of the seminarwritteninsuchawaythattheycanbeusedinpostgraduateseminarsoneducationaldesignresearchacrossChina.
About75peoplewithbackgroundsmainlyininstructionaltechnology,curriculumandinstructionaldesignparticipatedintheseminar.Mostofthemwereworkinginteachereducation,inschoolsasinstructionaltechnologistand/orindistanceeducation.Althoughparticipantshad(throughtheirstudies)alreadyknowledgeandsomeexperienceinin-structionalorcoursedesignandinresearchmethods,theywereeagertobeintroducedtodesignresearchasarelativelynewresearchapproachforaddressingcomplexproblemsineducationalpractice.
TheseminarstaffconsistedofProfsBrenda BannanandEamonn Kelly(bothGeorgeMasonUniversity,Fairfax,VA,USA)andProfJan van den Akker(UniversityofTwenteandNationalInstituteforCurriculumDevelopment[SLO],Enschede,TheNetherlands),andthetwoedi-torsofthisbookDrNienke Nieveen(NationalInstituteforCurriculumDevelopment[SLO],Enschede)andProfTjeerd Plomp(UniversityofTwente,Enschede,TheNetherlands).Ascanbeseenfromthetableofcontentofthisbook,theyarereflectingthebackgroundofthepar-ticipants,astheyrepresentedexperienceinconductingdesignresearchinthedomainsofcurriculumdevelopment,instructionaltechnologyandmathematicsandscienceeducation.ExpertswereconsciouslyinvitedfrombothEurope(TheNetherlands)aswellastheUSA,soastoascertainthatvariationinbackgroundandperspectiveondesignresearchwasrepre-sentedinconductingtheseminar.
Thechaptersinthisbookarebasedonthepresentationsandthesmallgroupdiscussionsduringthisseminar.Althoughthebookdoesnotprovidea‘howtodoguide’fordesigningandconductingdesignresearch,thechaptershavebeenwritteninsuchawaythattheyreflectboththeconceptualunderpinningandpracticalaspectsofthe‘what’and‘how’ofdoingdesignresearch(chaptersbyPlomp,KellyandNieveen),aswellasprovidethereaderaninsightinthespecificsofdoingdesignresearchinthedomainofcurriculum(chapterbyVandenAkker)andinstructionaltechnology(chapterbyBannan).Toassistthereadersinfindingtheirwayintheabundanceofliteratureondesignresearch,wehaveaddedachapterwithreferencesandsourcesoneducationaldesignresearch.This
![Page 8: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/8.jpg)
an introduction to educational design research6
bibliographyisfarfromcompleteandreflectsverymuchthebackgroundandthebiasesoftheeditorsofthisbook.Yetwetrustthatthischapterwillassisttheinterestedreaderingettingintroducedtothisexcitingandpromisingresearchapproach.
WewanttothankProfZhuZhitingfromtheEastChinaNormalUniversityfortakingtheinitiativeforthisseminar.Similarlywewanttothankourcolleaguesforcontributingtothisbook.
Butaboveall,weliketoexpressourhopethatthisbookwillstimulateandsupportmany(future)researcherstoengagethemselvesineducationaldesignresearch.
JanvandenAkkerDirector General SLO
TjeerdPlompandNienkeNieveenEditors
![Page 9: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/9.jpg)
an introduction to educational design research 7
![Page 10: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/10.jpg)
![Page 11: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/11.jpg)
an introduction to educational design research 9
1. EducationalDesignResearch: anIntroduction Tjeerd Plomp
Introduction
Thepurposeofthischapteristoprovideanintroductiontoeducationaldesignresearchasaresearchapproachsuitabletoaddresscomplexproblemsineducationalpracticeforwhichnoclearguidelinesforsolutionsareavailable.Educationaldesignresearchisperceivedasthesystematicstudyofdesigning,developingandevaluatingeducationalinterventions,-suchasprograms,teaching-learningstrategiesandmaterials,productsandsystems-assolutionstosuchproblems,whichalsoaimsatadvancingourknowledgeaboutthecharacteristicsoftheseinterventionsandtheprocessestodesignanddevelopthem.
Theneedforaresearchapproachthataddressescomplexproblemsineducationalpracticehasbeenarguedbyresearchersinvarious‘corners’ofthedomainofeducationfromthelackofrelevanceofmucheducationalresearchforeducationalpractice.Forexample,theDesign-BasedResearchCollective(2003:5)arguesthateducationalresearchisoftendivorcedfromtheproblemsandissuesofeverydaypractice–asplitthatresultedinacredibilitygapandcreatesaneedfornewresearchapproachesthatspeakdirectlytoproblemsofpracticeandthatleadtothedevelopmentof‘usableknowledge’.Fromhisbackgroundinresearchinthedomainofcurriculum development and implementation,VandenAkker(1999:2)arguesthatmany‘traditional’researchapproachessuchasexperiments,surveys,correlationalanalyses,withtheiremphasisondescriptionhardlyprovideprescriptionsthatareusefulfordesignanddevelopmentproblemsineducation.Heclaimsthatanimportantreasonfordesignresearch1stemsfromthecomplexnatureoftheeducationalreformsworldwide.Ambitiousreformscannotbedevelopedatthedrawingtablesingovernmentoffices,butcallforsystematicresearchsupportingthedevelopmentandimplementationprocessesinavarietyofcontexts.Inhisreviewofthestateofeducationalresearchandmorespecificallyeducationaltechnologyresearch,Reeves(2006:57)concludesthatthereis“alegacyofill-conceivedandpoorlyconductedresearchthatresultsinnosignificantdifferencesor,atbest,inmodesteffectsizes”.Healsoarguesforthedomainofeducational technologythateducationaltechnologists,insteadofdoingmore(media)comparisonstudies,shouldundertaketypesofdesignresearch.Inotherwords,ReevesarguesthatinsteadofdoingmorestudiescomparingwhetherinacertaincontextmethodAisbetterthanmethodB,itisbetterto
1) whichhecalls‘developmentresearch’inhis1999publication
![Page 12: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/12.jpg)
an introduction to educational design research10
undertakedesignresearchaimedatdevelopinganoptimalsolutionforaproblemincontext.Inthefieldoflearning sciences,thebeliefthatcontextmattersleadtotheconclusionthatresearchparadigmsthatsimplyexamineslearningprocessesasisolatedvariableswithinlaboratorysettingswillnecessarilyleadtoanincompleteunderstandingoftheirrelevanceinmorenaturalisticsettings(Barab&Squire,2004;withreferencetoBrown,1992).Inthisfield,design-basedresearchwasintroducedwiththeexpectationthatresearcherswouldsystematicallyadjustvariousaspectsofthedesignedcontextsothateachadjustmentservedasatypeofexperimentationthatallowedtheresearcherstotestandgeneratetheoryinnaturalisticcontexts(Barab&Squire,2004:3).
Thesesourcesillustratetheneedfordesignresearchasanalternativeresearchapproach.Beforeelaboratingondesignresearchthispaperwillfirstdiscussmoregenerallypossiblefunctionsofresearchandhowresearchfunctionsarerelatedtoresearchapproaches.Thendesignresearchwillbedefinedandcharacterizedfromvariousperspectives,suchasthetypeofknowledgethedesignresearchersaimfor,thetypeofresearchquestionsthatcanbeaddressed,andtheoutputsofdesignresearch.Thiswillbefollowedbyasectioninwhichdifferentapproachestodesignresearchareintroducedandsectionsdiscussinghowdesignresearchcanorshouldbeconducted,withamorein-depthdiscussionofformativeevaluationasthemostprominentresearchactivityindesignresearch.Conductingdesignresearchputsresearchersinasituationinwhichtheyhavetofaceanumberofdilemmas.Thesewillbediscussedbeforeendingthechapterwithafewconcludingremarks.
Afinalnoteonterminology,followingVandenAkkeretal.(2006:4)weusedesign researchasacommonlabelfora‘family’ofrelatedresearchapproacheswhomayvarysomewhatingoalsandcharacteristics–examplesaredesignstudies,designexperiments,design-basedresearch,developmentalresearch,formativeresearch,engineeringresearch.
Research functions – research approaches
Beforeelaboratingonthemeaningofdesignresearch,itisimportanttopositiondesignresearchasaresearchapproachnexttootherresearchapproaches,whichisthepurposeofthissection.
Thekeyfocusinallscientificresearchisthesearchfor‘understanding’orfor‘knowing’withtheaimofcontributingtothebodyofknowledgeoratheoryinthedomainofresearch.Otherbroad aimsofdoingeducationalresearcharetoprovideinsightsandcontributionsforimprovingpractice,andtoinformdecisionmakingandpolicydevelopmentinthedomainofeducation.
![Page 13: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/13.jpg)
an introduction to educational design research 11
ResearchfunctionsIngeneral,wecandistinguishvariousresearch functions,eachreflectingcertaintypesofresearchquestions.Examplesofresearchfunctions(withexemplaryresearchquestionsfittingthefunction)are:1. to describe: e.g.whatistheachievementofChinesegrade8pupilsinmathematics;
whatbarriersdostudentsexperienceinthelearningofmathematicalmodelling2. to compare:e.g.whatarethedifferencesandsimilaritiesbetweentheChineseandthe
Netherlandscurriculumforprimaryeducation;whatistheachievementinmathematicsofChinesegrade8pupilsascomparedtothatincertainothercountries
3. to evaluate:e.g.howwelldoesaprogramfunctionintermsofcompetencesofgraduates;whatarethestrengthsandweaknessesofacertainapproach;etc
4. to explain or to predict:e.g.whatarethecausesofpoorperformanceinmathematics(i.e.insearchofa‘theory’predictingaphenomenonwhencertainconditionsorcharacteristicsaremet)
5. to design and develop:e.g.whatarethecharacteristicsofaneffectiveteachingandlearningstrategyaimedatacquiringcertainlearningoutcomes;howcanweimprovethemotivationoflearners.
Inmanyresearchprojectstheresearchquestionsaresuchthatinfactvariousresearchfunctionsdoapply.Forexample,iftheresearchquestionpertainstocomparingthemathematicsachievementofChinesegrade8pupilsascomparedtothatincertainothercountries,thenaspartofcomparingtheresearcherswillevaluatetheachievementofgrade8pupilsineachofthecountriesinvolved.Or,asanotherexample,ifonewantstodesign and developateaching-learningstrategyforacquiringthecompetencyofmathematicalmodelling(ingrade11&12),thenresearchersmayfirstwanttounderstandandcarefullydescribewhatbarriersstudentsexperiencewithmathematicalmodelling,whilstalsotheevaluationfunctionisimportantindeterminingwhethertheteaching-learningstrategythathasbeendevelopediseffective.Bothexamplesillustratethatusuallyaresearchprojecthasaprimaryresearchfunction,butthatotherresearchfunctionsarebeingappliedto‘serve’theprimaryresearchfunction.
Atthelevelofaresearch project,startingfromaresearchproblemorquestion,wearesupposedtohavethefollowingsequence:Research question => (primary) research function =>choice of research approach.Inthischapterwefocusonresearchwhichhasdesign and developastheprimaryresearchfunction.
ResearchapproachesMosttextbooksonresearchmethodologypresentanddiscussanumberofresearch approachesorstrategies(seee.g.Denscombe,2007).Usuallyeachresearchapproachcanbe
![Page 14: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/14.jpg)
an introduction to educational design research12
usedforrealizingmorethanoneresearchfunction.Withoutgoingintodetailhere,examplesofresearchapproachesandtheirpossibleresearchfunctionsare:• survey: todescribe,tocompare,toevaluate• case studies:todescribe,tocompare,toexplain• experiments:toexplain,tocompare• action research:todesign/developasolutiontoapracticalproblem• ethnography:todescribe,toexplain• correlational research:todescribe,tocompare• evaluation research:todeterminetheeffectivenessofaprogramTextbooksonresearchmethodologyusuallydonotpresentanddiscussdesignresearch:• design research: todesign/developanintervention(suchasprograms,teaching-learning
strategiesandmaterials,productsandsystems)withtheaimtosolveacomplexeducationalproblemandtoadvanceourknowledgeaboutthecharacteristicsoftheseinterventionsandtheprocessestodesignanddevelopthem.
Inlinewiththeremarkthatmorethanoneresearchfunctionmayhavetobeappliedtoaddressaresearchquestion,itshouldbenoticedthatinaresearchprojectmorethanoneresearchapproachmayhavetobeapplied.Forexample,ifthereisaneedtocomparehowwellChinesegrade8pupilsperforminmathematicsascomparedtoanumberofothercountries,theprimaryresearchfunctionistocompare,leadinginthiscasetoasurveyasthebestresearchapproach.However,aspartofthedevelopmentofavalidandreliablemathematicstest,theresearchersmaydocorrelationalresearchtodeterminewhetherthetestbeingdevelopedisvalid,i.e.correlateswithothermeasuresofmathematicsachievement.
Asafinalremark,itisimportantthatdesignresearchers,likeallresearchers,keepinmindthatalsofortheirresearchtheguidingprinciplesforscientificresearch(Shavelson&Towne,2002)apply,viz:• Posesignificantquestionsthatcanbeinvestigated• Linkresearchtorelevanttheory• Usemethodsthatpermitdirectinvestigationofthequestion• Provideacoherentandexplicitchainofreasoning• Replicateandgeneralizeacrossstudies• Discloseresearchtoencourageprofessionalscrutinyandcritique
![Page 15: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/15.jpg)
an introduction to educational design research 13
What is design research?
Asstatededucationaldesignresearchisthe systematic study of designing, developing and evaluating educational interventions (such as programs, teaching-learning strategies and materials, products and systems) as solutions for complex problems in educational practice, which also aims at advancing our knowledge about the characteristics of these interventions and the processes of designing and developing them. Thetwofoldyieldofdesignresearch,viz.researchbasedinterventionsaswellasknowledgeaboutthem,canalsofoundindefinitionsofdesignresearchbyotherauthors.Forexample,thebroaddefinitionofBarabandSquire(2004)alsoencompassesmostvariationsofeducationaldesignresearch:“aseriesofapproaches,withtheintentofproducingnewtheories,artefacts,andpracticesthataccountforandpotentiallyimpactlearningandteachinginnaturalisticsetting.
Byitsnature,designresearchisrelevantforeducationalpractice(andthereforealsoforeducationalpolicy)asitaimstodevelopresearch-basedsolutionsforcomplexproblemsineducationalpractice.Startingpointfordesignresearchareeducationalproblemsforwhichnooronlyafewvalidatedprinciples(‘howtodo’guidelinesorheuristics)areavailabletostructureandsupportthedesignanddevelopmentactivities2.Informedbypriorresearchandreviewofrelevantliterature,researchersincollaborationwithpractitionersdesignanddevelopworkableandeffectiveinterventionsbycarefullystudyingsuccessiveversions(orprototypes)ofinterventionsintheirtargetcontexts,andindoingsotheyreflectontheirresearchprocesswiththepurposetoproducedesignprinciples.Manyexamplesoftheneedforinnovativeinterventionscanbegivenatsystemlevelandinstitutionallevel.Atsystemlevel,forexample,onemaywanttodevelopasystemfore-learningtoserveaspecifictargetgroupofstudentsinhighereducation,andatthelevelofschoolorclassroomonemaywant,forexample,toaddressthequestionofwhatareeffectivemethodsforcollaborativelearning.SeealsoGustafson&Branch(2002)whodevelopedataxonomyofinstructionaldevelopmentmodelsbasedonaselectedcharacteristics;theydistinguishbetweenmodelswithaclassroomorientation,productorientationandsystemorientation.
Theresearchprocessindesignresearchencompasseseducationaldesignprocesses.Itis–likeallsystematiceducationalandinstructionaldesignprocesses-thereforecyclicalincharacter:analysis,design,evaluationandrevisionactivitiesareiterateduntilasatisfyingbalancebetweenideals(‘theintended’)andrealizationhasbeenachieved.
2) seealsothechapterofKellyinthisbookwherehediscusseswhendesignresearchisappropriate.
![Page 16: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/16.jpg)
an introduction to educational design research14
Thisprocesscanbeillustratedinvariousways.Justafewexamplesarepresentedheretoshowhowdifferentauthorshavevisualizedtheresearchprocess.
Reeves(2006)depictsthedesignresearchapproachasfollows:
Figure 1: Refinement of Problems, Solutions, Methods, and Design Principles (Reeves, 2000, 2006)
McKenney(2001)illustratesinherstudythiscyclicalprocessasfollows:
Figure 2: Display of the CASCADE-SEA study (McKenney, 2001)
Identify and analyse problems by
researchers & practitioners in
collaboration
Development of prototype solutions: informed by state-
of-art theory, existing design principals & technology innova-
tions
Iterative cycles of testing & refinement
of solutions in practice
Reflection to produce ‘design principles’ &
enhance solution implementation in
practice
160
140
120
100
80
60
40
20
0
num
ber o
f par
ticip
ants
needs & contextanalysis
design, development & formative evaluation semi-summativeevaluation
literaturereview &concept
validation
sitevisits
prototype1
prototype2
cycle width is proportional to timeschale: = circa 6 months
prototype3
prototype4
finalevaluation
query
![Page 17: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/17.jpg)
an introduction to educational design research 15
The‘query’asthelastphaseinMcKenney’sdisplaycanbeinterpretedasthereflectionboxinthemodelofReeves(Figure1).AnotherexampleistheIntegrativeLearningDesignFrameworkthatBannan-Ritlandpresentsinchapter5ofthisbook(seealsoBannan-Ritland,2003).
Authorsmayvaryinthedetailsofhowtheypicturedesignresearch,buttheyallagreethatdesignresearchcomprisesofanumberofstagesorphases:• preliminary research:needsandcontextanalysis,reviewofliterature,developmentofa
conceptualortheoreticalframeworkforthestudy• prototyping phase:iterativedesignphase3consistingofiterations,eachbeingamicro-
cycleofresearch4withformativeevaluationasthemostimportantresearchactivityaimedatimprovingandrefiningtheintervention
• assessment phase:(semi-)summativeevaluationtoconcludewhetherthesolutionorinterventionmeetsthepre-determinedspecifications.Asalsothisphaseoftenresultsinrecommendationsforimprovementoftheintervention,wecallthisphasesemi-summative.
Throughoutalltheseactivitiestheresearcherorresearchgroupwilldosystematic reflection and documentationtoproducethetheoriesordesignprinciples(aconcepttakenfromVandenAkker,1999–seealsochapter2)asthescientificyieldfromtheresearch.Onemaystatethatthissystematicreflectionanddocumentationmakesthatsystematicdesignanddevelopmentofaninterventionbecomesdesignresearch.
Authorsaboutdesignresearchalsoagreeanumberofcharacteristicsofthistypeofresearch.ThesearesummarizedbyVandenAkkeretal.(2006:5):• Interventionist:theresearchaimsatdesigninganinterventioninarealworldsetting;• Iterative:theresearchincorporatescyclesofanalysis,designanddevelopment,
evaluation,andrevision;• Involvement of practitioners:activeparticipationofpractitionersinthevariousstages
andactivitiesoftheresearch• Process oriented:thefocusisonunderstandingandimprovinginterventions(ablack
boxmodelofinput–outputmeasurementisavoided);• Utility oriented:themeritofadesignismeasured,inpartbyitspracticalityforusersin
realcontexts;and• Theory oriented:thedesignis(atleastpartly)basedonaconceptualframeworkand
upontheoreticalpropositions,whilstthesystematicevaluationofconsecutiveprototypesoftheinterventioncontributestotheorybuilding.
3) itispossiblethatthedesign/developmentcomponentinasucharesearchprojectwillnotbeginfromscratchbutwiththeevaluationofanexistinginterventionwiththeaimofidentifyingtheneedforimprovement,whichthenisfollowedbyre-designandanumberofdesigncycles.
4) termtakenfromBannan-Ritland,chapter5
![Page 18: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/18.jpg)
an introduction to educational design research16
ThefeaturesandcharacteristicsofdesignresearcharenicelycapturedbyWademan(2005)inwhathecallstheGenericDesignResearchModel(Figure3).Hismodelclearlyillustratesthatthe‘successiveapproximationofpracticalproducts’(whatwecall‘interventions’)isgoinghandinhandwiththe‘successiveapproximationoftheory’(whichhealsocalls‘designprinciples’).
Figure 3: Generic Design Research Model (Wademan, 2005)
Itisimportanttonotethatdesignresearchfollowsaholisticapproach,anddoesnotemphasizeisolatedvariables.VandenAkkeretal.(2006:5)pointtoitthatyetdesignresearchersdofocusonspecificobjectsandprocesses(interventions)inspecificcontexts,buttheytrytostudythoseasintegralandmeaningfulphenomena.Thiscontextboundnatureofmuchdesignresearchalsoexplainswhyitusuallydoesnotstrivetowardscontext-freegeneralizations.Ifanefforttogeneralizingismade,thenitisananalyticalgeneralization(incontrasttostatisticalgeneralizationwheretheresearchermaygeneralizefromsampletopopulation).(Seealsothesection‘outputsofdesignresearch’)
Practitioner and User Participation
Researchers
Other Sources
Collaboratives
Practitioners
ConsultExperts &
Practitioners
FocusedLiterature
Review
AnalyzePromisingExamples
AnalyzePracticalContext
Tentative Product
Approaches
Tentative Design
Principles
Reflection
FormativeEvaluation
PracticalProduct/Results
Contributionto
Theory
Refinement of Problem, Solution and Method
Refinement of Design Theory
Preliminary investigation of Problem, Context,
& Approaches
Problem in Context
Phases
Prototyping & Assessment of Preliminary Products & Theories
Identification of Tentative Products & Design Principes
Problem Identification Tentative Products& Theories
Problem Resolution & Advancing Theory
Successive Approximation of Theory
Successive Approximation of Product
Redesign & Refinementof Products & Theories
![Page 19: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/19.jpg)
an introduction to educational design research 17
A closer look at design research
Asstatedkeycharacteristicsofdesignresearcharethatitisresearchfocusedondesigninginterventionsintherealcontextofeducationortraining(interventionistcharacteristic)combinedwitheffortstounderstandandimproveinterventions(process orientation),utilizingstateofthearttheorieswhilstthefieldtestingandtheevaluationoftheconsecutiveprototypesshouldcontributetotheorybuilding(theory orientation).Inthissectionwewillhavealookatwhatitmeansthatresearchsupportseducationaldesignprocesses,andreverselythateducationaldesignprocessessupportresearch.Thisisfollowedbyabriefdiscussionofthetypeofresearchquestionindesignresearch.Possibleoutputsofdesignresearchwillbediscussedinthenextsection.
Aswealreadystated,oneoftheaimsofdesignresearchisdesigninganddevelopinganinterventionasan(innovative)solutiontoacomplexproblem,andthereforethestartingpointfordesignresearchareeducationalproblemsforwhichnooronlyafewvalidatedprinciples(‘howtodo’guidelines)areavailabletostructureandsupportthedesignanddevelopmentactivities.Ontheotherhand,designresearchisresearchandthereforetheappropriateyieldfordesignresearch(apartfromausableandeffectiveintervention)isempiricallyfoundedtheory,i.e.thechallengefordesignresearchistocaptureandmakeexplicittheimplicitdecisionsassociatedwithadesignprocess,andtotransformthemintoguidelinesforaddressingeducationalproblems(seeEdelson,2006;101;alsoBarab&Squire(2003),andmanyotherauthors).Thisaspectreferstothetheory orientation,mentionedaboveasoneofthecharacteristicsofdesignresearch.VandenAkker(1999,2006,alsochapter2),Reeves(2006;seefigure1)andWademan(2005;seefigure3)usetheconceptof‘designprinciples’whentheyrefertothetheoreticalyieldsofdesignresearch,whereothersspeakofnewtheories(e.g.Barab&Squire,2003;Edelson,2006).
However,itisnotself-evidenthowthedesignofinterventionsmaycontributetotheorybuilding.WithreferencetothegenericmodelofWademen(Figure3)andtheexemplaryschemesofReeves(2006)inFigure1andMcKenney(2001)inFigure2,onemaystatethattheresearcher(orbetter:thecollectiveofresearchersandpractitioners)-basedonanalysisoftheproblemincontext,andutilizingrelevant,state-of-the-arttheories–designsanddevelops(inaniterativeway)theinterventionwiththeaimthatafteranumberofcyclestheintendedoutcomesarerealized,i.e.asatisfyingsolutiontotheproblemidentified.Eachiterationorcycleisamicro-cycleofresearch,i.e.astepintheprocessofdoingresearchandwillincludesystematic reflectiononthetheoreticalaspectsordesignprinciplesinrelationshiptothestatusoftheintervention,resultingintheendindesignprinciplesortheoreticalstatements.
![Page 20: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/20.jpg)
an introduction to educational design research18
Inotherwords,intheendtheresearcher(orresearchgroup)willconcludeabouthisintervention:
Given my context, if I do <intervention (theory based) > then I expect <intended outcomes>.
Thiscanbedisplayedschematicallyas:
Twopointsareimportantinthisscheme:• theoutcomesoftheinterventionareindicatedasY1,Y2,…,Yn,becauseoftenan
interventionisdesignedtorealizemultipleoutcomes(e.g.betterachievement,improvedstudentattitude,increasedteachersatisfaction,etc).
• theinterventionispresentedas‘inputprocess’,becausedesigningaprocess(e.g.learningenvironment)hastotakeintoaccountalsotheinputsnecessarytomaketheprocessfunction(e.g.certaininstructionallearningmaterials,teacherdevelopment).
Sointheend,theresearchgrouphasnotonlyatitsdisposaltheinterventionresultinginthedesiredoutcomes,butalsobasedonasystematicreflectionandanalysisofthedatacollectedduringthiscyclicalprocessanunderstandingofthe‘howandwhy’ofthefunctioningoftheinterventionintheparticularcontextwithinitwasdeveloped.Thedesignresearcherwillsummarizethisunderstandingofthe‘howandwhy’oftheinterventioninoneormore‘designprinciples’ifwewouldusetheterminologyofVandenAkker(1999,2006)andReeves(2000,2006).Asotherauthors,e.g.Barak&Squire(2004)andEdelson(2006),useof‘theory’astheyieldofdesignresearch,onemayalsospeakof‘interventiontheory’or‘designtheory’(Wademan,2005;Figure3)asasecondgenerictermtorefertotheknowledgegeneratedfromthisresearchendeavour(seebelowforspecificexamples).
Indesignresearch,interventionsaredevelopedinacyclical processofsuccessiveprototypes:
Intervention XInput Process
OutcomesY1, Y2, ...., Yn
Intervention XInput Process
OutcomesY1, Y2, ...., Yn
design principles or intervention theory
![Page 21: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/21.jpg)
Akeyideaisthatwheninacertaincycletheprototypeoftheinterventiondoesnotresultinthedesiredoutcomes,onemayconcludethatthedesignprinciples(orinterventiontheory)appliedarenot(yet)effective(or,inotherwords,thattheinterventiontheory‘fails’).Thishastoresultinare-designorrefinementoftheintervention,whichgoeshand-in-handwiththerefinementoftheinterventiontheoryordesigntheory.Whenafteranumberofiterationstheresearcher(orresearchgroup)concludesthatbasedontheanalysisoftheevaluationdatathe‘realizedoutcomes’arecloseenoughtothe‘intendedoutcomes’thenhecanbesatisfied:thedesignprinciplesappeartobeeffective.Or,inotherwords,theresearcher(orresearchgroup)hasdevelopeda‘local’(intervention)theory(i.e.forthecontextinwhichhe/sheworks):in context Z the intervention X (with certain characteristics) leads to outcomes Y1, Y2, …, Yn.
Twoexamplesaregiventoillustratethis–ratherabstract–phrasingoftheyieldofdesignresearch.TheDesign-BasedLearningResearchCollective(2003:5)statethat“thedesignofinnovationsenablesustocreateleaningconditionsthatlearningtheorysuggestsareproductive,butthatarenotcommonlypracticedorarenotwellunderstood”–inotherwordsincludedintheinnovationsisknowledgeabouthowtocreateconditionsforlearning.Thesecondexampleistakenfromscienceeducation.Lijnse(1995:192)arguesthatdesignresearch(hecallsitdevelopmentalresearch)is“acyclicprocessoftheoreticalreflections,conceptualanalysis,small-scalecurriculumdevelopment,andclassroomresearchoftheinteractionofteaching-learningprocesses.Thefinal,empiricallybaseddescriptionandjustificationoftheseinterrelatedprocessesandactivitiesconstituteswhatwecallapossible“didacticalstructure”forthetopicunderconsideration.”Inotherwords,thelocaltheoryconsistsofadidacticalstructureforteaching-learningprocessesforacertaintopic.
Theresearchquestionindesign/developmentresearchBynowitisclearthatdesigninganddevelopinganinterventionisinitselfnotyetdesignresearch.Butonemayconductadesign/developmentprojectasaresearchprojectbyemployingrigorouslysocialscienceresearchmethodology.Astheresearcherisstrivingtofinddesignprinciples(oranintervention theory)thatarevalidinacertaincontext,theresearchquestioncanbephrasedas:
what are the characteristics of an <intervention X> for the purpose/outcome Y (Y1, Y2, …, Yn) in context Z
Design/develop Implement/try-0ut
Evaluate (formative)
an introduction to educational design research 19
![Page 22: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/22.jpg)
an introduction to educational design research20
Examplesofresearchquestionsare:(i) whatarethecharacteristicsofaneffectivein-serviceprogrammeformathematics
teachersthroughwhichtheydeveloptheabilitytoapplystudent-centredpedagogicalmethods,and
(ii) whatarethecharacteristicsofanin-servicearrangementthatfacilitatestheimplementationofMBL5-supportedlessonactivitiesinphysicseducation(Tecle,2003)?
Obviously,notallresearchersareusingthistypeofphrasing,butthewordingofthemainresearchquestionindesignresearchalwaysimpliesasearchforcharacteristics.Anexampleis:Whatisanadequatelearningandteachingstrategyforgeneticsinuppersecondarybiologyeducationinordertocopewiththemaindifficultiesinlearningandteachinggenetics,andtopromotetheacquisitionofameaningfulandcoherentunderstandingofhereditaryphenomena?(Knippels,2002)
The outputs of design research
Wealreadyconcludedthatdesignresearchresultsininterventions(programs,products,processes)andindesignprinciplesorinterventiontheory.Athirdoutputofdesignresearchisprofessionaldevelopmentoftheparticipantsinvolvedintheresearch.Eachoftheseoutputsisbrieflydiscussed.
OndesignprinciplesorinterventiontheoryDesignresearchaimsatproducingknowledgeaboutwhetherandwhyaninterventionworksinacertaincontext.Intheprevioussectionthistypeofoutputhasbeencalleddesignprinciplesorinterventiontheory.Otherauthorsusetermslikedomainspecifictheories(Gravemeijer&Cobb,2006),designtheory(Wademan,2005;Figure3),heuristicsorjustlessonslearned(seeVandenAkkeretal.2006).Wewillusethetermdesign principlesintheremainingofthispaper.
Designprinciplesareheuristic statementsforwhichVandenAkker(1999)developedthefollowingformat:
“If you want to design intervention X for the purpose/function Y in context Z, then you are best advised to give that intervention the characteristics A, B, and C [substantive emphasis], and to do that via procedures K, L, and M [procedural
emphasis], because of arguments P, Q, and R.” (Van den Akker, 1999)
5) MBL=MicrocomputerBasedLaboratory.
![Page 23: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/23.jpg)
Theheuristicprinciplesaremeanttosupportdesignersintheirtasks,butcannotguaranteesuccess-theyareintendedtoassist(inotherprojects)inselectingandapplyingthemostappropriate(substantiveandprocedural)knowledgeforspecificdesignanddevelopmenttasks.Substantiveknowledgeisknowledgeaboutessentialcharacteristicsofaninterventionandcanbeextracted(partly)fromaresultinginterventionitself.Proceduralknowledgereferstothesetofdesignactivitiesthatareconsideredmostpromisingindevelopinganeffectiveandworkableintervention.Asknowledgeisincorporatedininterventions,itisprofitablefordesignresearchersintheearlystageoftheirresearchtosearchforalreadyavailableinterventionsthatcanbeconsideredusefulexamplesorsourcesofinspirationfortheproblematstake.Carefulanalysisofsuchexamplesincombinationwithreviewingrelevantliterature)willgenerateideasforthenewdesigntask.Thevalueofknowledgeresultingfromadesignresearchprojectwillstronglyincreasewhenitisjustifiedbytheoreticalarguments,well-articulatedinprovidingdirections,andconvincinglybacked-upwithempiricalevidenceabouttheimpactofthoseprinciples.Itisforthisreasonthatauthors(e.g.VandenAkker1999,2006;Reeves,2000,2006)statethatthefinalstageofeachdesignresearchprojectshouldconsistofsystematicreflectionanddocumentationtoproducedesignprinciples.
GeneralizabilityindesignresearchHeuristicdesignprincipleswillbeadditionallypowerfuliftheyhavebeenvalidatedinthesuccessfuldesignofmoresimilarinterventionsinvariouscontexts.Chancesforsuchknowledgegrowthwillincreasewhendesignresearchisconductedintheframeworkofresearchprograms,becausethenprojectscanbuildupononeanother.Herewetouchonthequestiontowhatextentdesignprinciplescanbegeneralizedfromonecontexttoothers.ItisinthiscontextthatEdelson(2006)statesthatdesignresearchshouldresultingeneralizabletheory.Indesignresearch,likeincasestudiesandexperimentalstudies,thefindingscannotbegeneralizedtoalargeruniverse–thereisnostatisticalgeneralizationfromsampletopopulation,likecanbethecaseinsurveyresearch.Yin(2003)pointstoitthatincasestudiesandexperimentalstudies,theinvestigatorisstrivingtogeneralizeaparticularsetofresultstoabroadertheory.Thisisalsothecaseindesignresearch,theresearchershouldstrivetogeneralize‘designprinciples’tosomebroadertheory.Yin(2003:37)pointstoitthatgeneralizationisnotautomatic.Designprinciplesmustbetestedthroughreplicationsofthefindingsinasecond,thirdormorecasesinvariouscontextswiththepurposethatthesameresultsshouldoccur.Oncesuchreplicationshavebeenmade,theresultsmightbeacceptedforamuchlargernumberofsimilarcontexts,eventhoughfurtherreplicationshavenotbeenperformed.Thisreplication logicisthe
an introduction to educational design research 21
![Page 24: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/24.jpg)
an introduction to educational design research22
samethatunderliestheuseofexperimentsandallowsexperimentalscientiststogeneralizefromoneexperiment‘toanother’:Yin(2003)callsthisanalyticalgeneralizability.Butawarningshouldbephrasedhere.Wheredesignprinciplesmayhavebeensupportedbyanumberofreplications,andanewcontextmaybesimilartotheonesfromwhichdesignprincipleshaveemerged,yeteachcontexthasuniquecharacteristicsthatjustifiesthatthedesignprinciplesshouldbeusedas‘heuristic’statements:theyprovideguidanceanddirection,butdonotgive‘certainties’.ItisinthiscontextthatReeves(2006)citesLeeCronbachoneofthemostinfluentialresearchersofthe20thcentury:“When we give proper weight to local conditions, any generalization is a working hypothesis, not a conclusion.”(Cronbach,1975:125)
OninterventionsDesignresearchbyitscharacteraimstobepracticallyrelevant.Itisinitiatedtodesignanddevelopinnovativeinterventionstomeetaneedfeltinacomplex,practicalsituationforwhichnoready-madesolutionsorguidelinesareavailable.Thereforedesignresearchersaimatdevelopinginterventions(suchasprograms,teaching-learningstrategiesandmaterials,productsandsystems)thatcanbeusedinpracticeandareempiricallyunderpinnedsolutionstotheproblemsidentified.
OnprofessionaldevelopmentOneofthefeaturesofdesignresearchisthecollaborationofresearchersandpractitioners.Thiscollaborationincreasesthechancethattheinterventionwillindeedbecomepracticalandrelevantfortheeducationalcontextwhichincreasestheprobabilityforasuccessfulimplementation.Buttheparticipationofpractitionersshouldalsobeseenasanimportantformofprofessionaldevelopment.Anextraspin-offmaybethatpractitionerswilldevelopanawarenessofhowresearchmaycontributetoimprovingtheirprofessionalcontext.
Design research differentiation
Designresearchisconductedthroughanumberofcyclesofdesignanddevelopmentresultingintheinitialimplementationoftheinterventioninalimitednumberofcontexts.Asstatedabove,designresearchhasusuallyanumberofstagesorphases(seealsoFigures1,2and3):• needsandcontentanalysis• prototypingphase(iterativecyclesofdesignandformativeevaluation)• assessmentphase(semi-summativeevaluation)
Nieveenetal.(2006)suggestthatdesignresearchthathasresultedinavalidatedandeffectiveintervention(asasolutionfortheproblemunderstudy),andindesignprinciplescanbefollowedbyeffect studies(notnecessarilypartofthesameresearchproject)withan
![Page 25: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/25.jpg)
an introduction to educational design research 23
emphasisonupscalingtheinterventiontoawidercontext,andindoingsoaimingatdesignprinciplestestedinawiderdomain.Effectstudiesmayrangefromsmall-scalelearningexperimentstolarge-scalecomparativetestingofimpact(e.g.viarandomizedcontrolledtrials).
Afurtherdifferentiationindesignstudiesispossibleonthebasisofvariationsingoalsofdesignresearchvizvalidationstudiesversusdevelopmentstudies(seeVandenAkker,Gravemeijeretal.,2006;chapters5and10).
Validation studieshaveafocusondesigninglearningenvironmentsortrajectorieswiththepurposetodevelopandvalidatetheoriesabouttheprocessoflearningandhowlearningenvironmentscanbedesigned.Validationstudiesaimatadvancinglearningandinstructiontheories,suchas(Gravemeijer&Cobb,2006):• micro-theories:atthelevelofinstructionalactivities• localinstructiontheories:atthelevelofinstructionalsequence;• domain-specificinstructiontheories:atthelevelofpedagogicalcontentknowledge.Invalidationstudies,researchersdonotworkincontrolled(laboratoryorsimulated)settings,buttheychoosethenaturalsettingofclassroomas‘testbeds’(althoughtheytendtoworkwithabove-averagenumberofteachingstaff).Usually,thestagesinvalidationstudiesare(Gravemeijer&Cobb,2006):• environment preparation:elaboratingapreliminaryinstructionaldesignbasedonan
interpretativeframework;• classroom experiment:testingandimprovingtheinstructionaldesignorlocal
instructionaltheoryanddevelopinganunderstandingofhowitworks;• retrospective analysis:studyingtheentiredatasettocontributetothedevelopmentofa
localinstructionaltheoryand(improvementof)theinterpretativeframework.DiSessaandCobb(2004:83)warnthat“designresearchwillnotbeparticularlyprogressiveinthelongrunifthemotivationforconductingexperimentsisrestrictedtothatofproducingdomainspecificinstructionaltheories”.Butthepracticalcontributionliesindevelopingandimplementingspecificlearningtrajectoriesthatwereimplementedtotestthetheoreticalbasisofthedesign.(Nieveenetal,2006:153)
Development studiesaimtowardsdesignprinciplesfordevelopinginnovativeinterventionsthatarerelevantforeducationalpractice.“Developmentstudiesintegratestate-of-the-artknowledgefrompriorresearchinthedesignprocessandfine-tuneeducationalinnovationsbasedonpilotinginthefield.…Byunpackingthedesignprocess,designprinciplesthatcaninformfuturedevelopmentandimplementationdecisionsarederived.”(Nieveenetal.,2006:153).Twomaintypesofdesignprinciplescanbedistinguished(VandenAkker,1999):1. proceduraldesignprinciples:characteristicsofthedesignapproach;2. substantivedesignprinciples:characteristicsofthedesign(=intervention)itself.
![Page 26: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/26.jpg)
an introduction to educational design research24
Figurethreesummarizesthecharacteristicsofaresearchcycleconsistingofdesignstudiesandeffectstudies(asdevelopedbyNieveenetal.;2006:155):
Design researchEffectiveness research
Validation studies Development studies
Design aim To elaborate and validate theories
To solve educational problems
-
Quality focus of design
Theoretical quality of design
Practicality of intervention
Effectiveness of intervention
Knowledge claim/ scientific output
Domain-specific instruction theories
Broadly applicable design principles
Evidence of impact of intervention
Methodological emphasis
Iterative design with small scale testing in research setting
Iterative development with formative evaluation in various user settings
Large scale, comparative field experiments
Practical contribution Specific learning trajectories for a specific classroom
Implemented interventions in several contexts/classrooms
Evidence-basedChange at large scale
Figure 4: Educational engineering research cycle (from Nieveen et al., 2006)
Itisimportanttonotethatthisdistinctionbetweenvalidationanddevelopmentstudiesisconceptuallyimportant,butthatinpracticemanyresearchprojecthaveaimsthatareacombinationofsolvingproblemsineducationalpracticeandelaboratingandvalidatingtheories(designprinciples).
Afurtherdifferentiationofdesignresearchisconceivable.Forexample,onecanimaginethatthedisseminationandimplementationofaparticularprogramissupportedbydesignresearch–theresultinginterventionisthesuccessfullydisseminatedandimplementedprogram,whilstthesystematicreflectionanddocumentationoftheprocessleadstoasetofproceduresandconditionsforsuccessfuldisseminationandimplementation(thedesignprinciples).
Asafinalnote,thedifferentiationbetweentypesofdesignresearch,suchasvalidationstudiesversusdevelopmentstudies,servesmainlyconceptualpurposes.Inpractice,designresearchersmaycombinethetwoorientationsintheirresearch.Forexample,startingfromacomplexandpersistentproblemine.g.scienceeducation,theresearchgroupmaydecidetoapplythedesignprinciples(localtheories)resultingfromotherstudiesintheirresearch.Indoingsotheyarenotonlydevelopinganintervention,butatthesametimeexploringthevalidityofdesignprinciples(theory)developedinanothercontextfortheirownproblemcontext.
![Page 27: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/27.jpg)
an introduction to educational design research 25
Howisdesignresearchconducted?Designresearchisconductediterativelyasacollaborationofresearchersandpractitionersinareal-worldsetting.Onlythenthetwoprincipaloutputs(designprinciplesandempiricallyunderpinnedinnovativeinterventions)canberealized.Doingresearchinsuchasettingischallenginganddemandsacarefulresearchdesign.Itisthereforeimportanttoreflectnotonlyonthecyclical,iterativecharacterofthesystematicdesignoftheintervention,butalso–becauseitisresearch-tomakeexplicitthetenetsthatformthefoundationofthistypeofresearch(McKenneyetal.,2006)
McKenneyetal.(2006:77)definethreetenetstoshapedesignresearchforthecurriculumdomain(butthetenetsalsoapplytootherdomains):• Rigor–fordesignresearchtobeabletoresultinvalidandreliabledesignprinciples,the
researchhastomeetrigorousstandardsandapplytheguidingprinciplesforscientificresearchasmentionedbyShavelson&Towne(2002;mentionedabove).Muchliteratureisavailabletoguideresearchinnaturalsettingsthatofferssupporttoissueslikeinternalandexternalvalidity,reliabilityandutililizationoftheresearch.
• Relevance:Designresearchaimstoberelevantforeducationalpractice(andpolicy).Anecessaryconditionforthisisthattheresearchgroupmusthaveagoodworkingknowledgeofthetargetsettingandbeinformedbyresearchanddevelopmentsactivitiestakingplaceinnaturalsettings(ortestbeds).
• Collaboration:fordesignresearchtoberelevantforeducationalpractice,thedesignanddevelopmentactivitiesmustbeconductedincollaborationwithandnotjustforprofessionalsfromeducationalpractice.
Asexplainedinthebeginningofthischapter,designresearchiscyclicalandeachiterationorcyclecontributestosharpeningtheaimsandtobringingtheinterventionsclosertothedesireddesignoutcomesandresearchoutputs.AsisillustratedinFigures1-3,designresearchusuallygoesthroughseveralstageswhichNieveenetal.(2006:154)phraseasfollows(seealsop.15):• preliminary research:thoroughcontextandproblemanalysisalongwiththe
developmentofaconceptualframeworkbasedonliteraturereview;• prototyping stage:settingoutdesignguidelines,optimizingprototypesofthe
interventionthroughcyclesofdesign,formativeevaluation,andrevision–itisimportanttonotethateachcycleinthestudyisapieceofresearchinitself(i.e.havingitsresearchorevaluationquestiontobeaddressedwithaproperresearchdesign);
• assessment stage (summative evaluation):oftenexplorestransferabilityandscaling,alongwith(usuallysmall-scaleevaluationof)effectiveness;and
• systematic reflection and documentation:thisarecontinuousactivities(asillustratedinFigure3)thattakesplaceduringallcyclesintheresearch–however,attheendthe
![Page 28: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/28.jpg)
an introduction to educational design research26
researcherportraystheentirestudytosupportretrospectiveanalysis,followedbyspecificationofdesignprinciplesandarticulationoftheirlinkstotheconceptualframework.
Itisbeyondthescopeofthischaptertodiscussindetailhowtoperformthesestages.Butanexceptionismadeforformativeevaluation,becausethisisthekeyresearchactivityindesignresearchaimedatimprovingthequalityoftheconsecutiveprototypesoftheintervention.
Formativeevaluationindevelopmentresearch6
BasedonpriorworkNieveen(1999;seealsoChapter5)proposesfourgenericcriteriaforhighqualityinterventions(seeTable1).Sheexplainsthesecriteriaasfollows:Thecomponentsoftheinterventionshouldbebasedonstate-of-the-artknowledge(content validity)andallcomponentsshouldbeconsistentlylinkedtoeachother(construct validity).Iftheinterventionmeetstheserequirementsitisconsideredtobevalid.Anothercharacteristicofhigh-qualityinterventionsisthatend-users(forinstancetheteachersandlearners)considertheinterventiontobeusableandthatitiseasyforthemtousethematerialsinawaythatislargelycompatiblewiththedevelopers’intentions.Iftheseconditionsaremet,wecalltheseinterventionspractical.Athirdcharacteristicofhighqualityinterventionsisthattheyresultinthedesiredoutcomes,i.e.thattheinterventioniseffective.
Criterion
Relevance (also referred to as content validity)
There is a need for the intervention and its design is based on state-of-the-art (scientific) knowledge.
Consistency (also referred to as construct validity)
The intervention is ‘logically’ designed.
Practicality The intervention is realistically usable in the settings for which it has been designed and developed.
Effectiveness Using the intervention results in desired outcomes.
Table 1: Criteria for high quality interventions (from Nieveen, 1999; Chapter 5)
Giventhecharacterofdesignresearch,thesefourcriteriamaygetdifferentemphasisindifferentstagesoftheresearchasisillustratedbyFigure5.Forexample,duringthepreliminaryresearchwheretheemphasisisonanalyzingtheproblemandreviewingtheliterature,thecriterionofrelevance(contentvalidity)isthemostdominant,withsomeattentionforconsistency(constructvalidity)andpracticality,whilstinthatstatenoattentionisyetgiventoeffectiveness.Ontheotherhand,intheprototypingstagemuch
6) SeealsoNieveen’schapter5inthisbookinwhichshediscusseshowtodotheformativeevaluationindesignresearch
![Page 29: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/29.jpg)
an introduction to educational design research 27
attentionhastobepaidintheformativeevaluationtothecriterionofpracticality,whilsteffectivenesswillbecomeincreasinglyimportantinlateriterations.Finally,inassessmentstageofsummativeevaluation,thefocuswillbeonpracticalityandeffectiveness(seeFigure5,andFigure2forthestages).
Stage Criteria Short description of activities
1 Preliminary research Emphasis mainly on content validity, not much on consistency and practicality
Review of the literature and of (passed and/or present) projects addressing questions similar to the ones in this study. This results in (guidelines for) a framework and first blueprint for the intervention.
2 Prototyping stage Initially: consistency (construct validity) and practicality. Later on mainly practicality and gradually attention for efficiency.
Development of a sequence of prototypes that will be tried out and revised on the basis of formative evaluations. Early prototypes can be just paper-based for which the formative evaluation takes place via expert judgments.
3 Assessment phase practicality and efficiency
Evaluate whether target users can work with intervention (practicality) and are willing to apply it in their teaching (relevance & sustainability). Also whether the intervention is effective.
Figure 5: Evaluation criteria related to stages in design research
Formativeevaluationtakesplaceinallphasesanditerativecyclesofdesignresearch.AsillustratedbyFigure5,formativeevaluationservesdifferentfunctions,or-inotherwords-isaimedatdifferentcriteria(orcombinationsofthese)inthevariousdevelopmentcycles,eachbeingamicro-cycleofresearchwithitsspecificresearch/evaluationquestionandrelatedresearch/evaluationdesign.OnemaysaythatformativeevaluationhasvariouslayersinadesignresearchprojectasisillustratedinFigure6,takenfromTessmer(1993):frommoreinformalintheearlystagesofaproject(self-evaluation,one-to-oneevaluation,expertreview)tosmallgroupevaluationaimedattestingthepracticalityandeffectiveness,toafullfieldtest(ifapplicable).Theresearch/evaluationdesignforeachcycleshouldreflectthespecificfocusandcharacterofthecycle–seeChapter5byNieveenformoredetails.
![Page 30: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/30.jpg)
an introduction to educational design research28
Figure 6: Layers of formative evaluation (taken from Tessmer, 1993)
Figure6alsoillustratesthatmanypossiblemethodsofformativeevaluationcanbechosen,suchas7
• expertreviewand/orfocusgroups(importanttoconsider‘expertsinwhat’)• self-evaluationorscreening(usingchecklistofimportantcharacteristicsordesign
specifications)• one-to-oneevaluationorwalkthrough(withrepresentativeoftargetaudience)• smallgroupormicro-evaluation• fieldtestortry-outDesignresearchersshouldchooseforeachphaseandforeachprototypeformativeevaluationapproachesthataresuitableforthepurposeofthatparticularstageoftheresearch.Designresearchhastomeetcriteriaforgoodresearch.Itisthereforeimportantthatforeachdevelopmentcycletheresearcher(orresearchgroup)appliesthemethodological‘rules’fordoingresearch,i.e.foridentifyingthetargetaudienceandsampling,forinstrumentdevelopmentandapplytriangulationtoobtaingoodqualityinformation.But
7) seealsoChapter5byNieveen
HighResistanceto Revision
LowResistanceto Revision
Revise
Revise
Revise
Field TestUser Acceptance, Implementability
Organizational Acceptance
Small GroupEffectiveness, Appeal
Implementability
Expert ReviewContent, Design,Techical Quality
One-to-OneCiarity, AppealObvious Errors
Self-Evaluationobvious errors
![Page 31: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/31.jpg)
an introduction to educational design research 29
giventhelayersofformativeevaluationindesignresearch,intheearlycyclesofdevelopmenttheevaluationdesigncanbelessrigorousthaninlaterphases.Figure7adaptedfromNieveen(1999)presentsanexamplethatillustrateshowvariousformativeevaluationmethodsareusedfortherespectiveprototypesinaprojectaimedatdevelopingacomputerassistedsupportsystemforcurriculumdevelopers.
prelimcomp.based
paper-based computer-based versions
final version
Users (n=5)
experts (n=3)
users (n=5)
experts (n=6)
users (n=4)
users (n=4)
users (n=17)
Validity content *) √ ea √ ea
interface √ ea
Practicality content √ wt √ wt √ ea √ me √ to √ ft
interface √ wt √ wt √ ea √ me √ to √ ft
Effectiveness entire system √ to √ ft
*): Content refers to the content of the support system
√ = primary attention of prototype and of formative evaluation
Methods of formative evaluation: me = micro evaluation; wt = walk through; ea = expert appraisal;
ft = field trial; to = try-out
Figure 7: Focus of design and formative evaluation of the prototypes for computer assisted support system for curriculum development (adapted from Nieveen, 1999)
Afinalnoteonthecriteriaofpracticalityandeffectiveness.Itmayoccurincertainstudiesthattheresearcher(orresearchcollaborative)cannotdoafinalfieldtrialoftheinterventionwiththefull(orasampleofthe)targetgroup,buthastorestricthimselftoexpertappraisaland/ormicro-evaluationofthefinalprototypeoftheintervention.Itisobviousthatinsuchasituationtheactual practicalityandtheactual effectivenessoftheinterventioncannotbedemonstrated,butonlyconclusionsabouttheexpected practicalityandtheexpected effectivenesscanbedrawn.Moreevaluationwillthenbeneededtodemonstratetheactual practicalityandtheactual effectiveness.ThiscanbeillustratedwithanexampleadaptedfromMafumiko(2006)whoconducteddesignresearchtoinvestigatewhethermicro–scaleexperimentationcancontributetoimprovingthechemistrycurriculuminTanzania.HisresearchdesignhasbeensummarizedinFigure8.
![Page 32: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/32.jpg)
an introduction to educational design research30
Figure 8: Example of research research design (adapted from Mafumiko, 2006)
SupposearesearcherwouldrestricthimselftothedevelopmentofprototypesoftheinterventionasillustratedinFigure8,anddoesnotplantoinvestigatewhetherVersionIVworksinthetargetcontext.Insuchasituationthemosthecanconcludeiswhetherhisinterventionisexpectedtobepracticalandeffectiveforthetargetcontext.Onlywhenhewouldconductafieldtest,hewillbeinthepositiontodecideuponactualpracticalityandactualeffectiveness(whichiswhatMafumikodid).
Design research dilemmas
Designresearchisconductedinclosecollaborationwitheducationalpractice.Notonlytheproblemaddressedissituatedineducationalpractice,butakeyfeatureofthisresearchisthateducationalpractitionersareactivelyinvolved,oftenasmembersoftheresearchteam.Thisleadstoanumberofchallengesthataretypicalforthistypeofresearch.McKenneyetal.(2006:83,84)havediscussedsomeoftheseandprovidesuggestionsforhowtoaddressthem.Theirpointsarebrieflysummarizedhere.
1. theresearcherisdesignerandoftenalsoevaluatorandimplementer.Severalmeasurescanbetakentocompensateforthispotentialconflictofinterest:• makeresearchopentoprofessionalscrutinyandcritiquebypeopleoutsidetheproject• theresearcherappliesthefollowingruleofthumb:shiftfromadominanceof‘creative
designer’perspectiveintheearlystage,towardsthe‘criticalresearcher’perspectiveinlaterstages(thisisreflectedinTessmer’slayersofformativeevaluation,Figure6)
Summative
evaluation
Appraisel by3 experts
Tryout in3 classrooms
Panel session with experts
Field test infour schools
Tryout with teacher educ students
Version I Version II Version III Version IV
Development of prototypes
Design guidelines &specifications
![Page 33: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/33.jpg)
an introduction to educational design research 31
• haveagoodqualityofresearchdesign,e.g. - strong chain of reasoning(Krathwohl,1998)-themetaphorexpressestheideathat
eachpartoftheresearchdesignisequallyimportant - triangulation–toincreasethequalityofdataandofanalysistriangulationofdata
sourcesanddatacollectionmethodsshouldbeapplied,aswellasinvestigatortriangulationtoavoidtheinfluenceofanyspecificresearcher(seee.g.Denscombe,2007;136)
- empirical testingofboththeusabilityandtheeffectivenessoftheintervention - systematic documentation, analysisandreflectionofthedesign,development,
evaluationandimplementationprocessandtheirresults - haveattentionforvalidityandreliabilityofdataandinstruments - applyavarietyofmethodsandtactics:e.g.usepractitionersandotherresearchersas
‘criticalfriends’;usemultipleobservers/ratersandcalculateinter-observer/raterreliability,etc.
2. real-worldsettingsbringreal-worldcomplicationsDesignresearchisconductedinreal-worldsettingsbecauseitaddressescomplexproblemsineducationalpractice.Oneoftheproblemsisthattheresearchercanbea‘culturalstranger’(Thijs,1999)inthesettingoftheresearchandthatparticipants(e.g.principals,teachersnotinvolvedintheresearch,etc)arehesitanttobecompletelyopentoaresearchercomingfromtheoutside.McKenneyetal.(2006:84)pointstotheimportanceofcollaborationandmutualbeneficialactivitiestogainparticipants’trustandthoroughunderstandingofthecontext(i.e.insiderperspective).Ontheotherhand,theyalsopointtotheadvantagestobeanoutsiderasthismayallowtheresearchertodevelopadegreeofobjectivityand“freedom(orforgiveness)forhonestythatisnotpermittedtothosewithinaparticulargroup”(o.c.85)
3. adaptabilityDesignresearchiscyclicalandtakesplaceinreal-worldsettings.Eachcyclehastotakethefindingsofthepreviousonesintoaccount.Soontheonehandtheresearchdesignhastochange(ordevelop)fromonecycletotheother,whilstontheotherhandanever-changingresearchdesigncanbeweak.Inthiscontext,McKenneyetal.(2006:84)refertothenotionofevolutionaryplanning,i.e.“aplanningframeworkthatisresponsivetofielddataandexperiencesasacceptablemomentsduringthecourseofthestudy”.Thisisalreadyalludedtointhediscussionofformativeevaluation(seeFigure6fromTessmerandtheexampletakenfromNieveen,1999).Theneedforadaptabilitypertainsalsototheroleoftheresearcher.AccordingtoVandenAkker(2005,inMcKenneyetal.,2006),thesynergybetweenresearchandpracticecanbemaximizedwhenresearchersdemonstrateadaptabilityby:
![Page 34: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/34.jpg)
an introduction to educational design research32
(i) beingprepared,wheredesirable,totakeontheadditionalroleofdesigner,advisor,andfacilitator,withoutlosingsightoftheirprimaryroleasresearcher,
(ii) beingtolerantwithregardtotheoftenunavoidablyblurredroledistinctionsandremainingopentoadjustmentsintheresearchdesignifprojectprocesssodictates,
(iii)allowingthestudytobeinfluenced,inpart,bytheneedsandwishesofthepartners,duringwhatisusuallyalong-termcollaborativerelationship.
Suchadaptabilityrequiresstrongorganizationalandcommunicativecapabilitiesonbehalfoftheresearcher,aswellassoundunderstandingtheresearchprocesssothatcarefulchangesandchoicesthatmaximizevalueandminimizethreatstoqualityaremade.(McKenneyetal.,2006:84).
Toaddressthechallengesmentioned,McKenneyetal.(2006:85,86)presentafewguidelines for conducting design researchthatmayhelpresearchersmonitoringthescientificcharacterofhis/herresearch:- haveanexplicitconceptualframework(basedonreviewofliterature,interviewsof
experts,studyingotherinterventions)- developcongruentstudydesign,i.e.applyastrongchainofreasoningwitheachcycle
havingitsresearchdesign- usetriangulation(ofdatasource,datatype,method,evaluatorandtheory)toenhance
thereliabilityandinternalvalidityofthefindings- applybothinductiveanddeductivedataanalysis- usefull,context-richdescriptionsofthecontext,designdecisionsandresearchresults- membercheck,i.e.takedataandinterpretationsbacktothesourcetoincreasethe
internalvalidityoffindings.Itisbeyondthescopeofthispapertoelaborateontheseguidelinesfurther–seeMcKenneyetal.(2006;85,86)andresearchmethodologybooks.
Concluding remarks
Inthefieldofeducationthereismuchneedforresearchrelevantforeducationalpractice.Wehavearguedthatforcomplexpracticalproblemsandforresearchquestion(s)callingforthedesignanddevelopmentofaninterventiondesignresearchistheappropriateresearchapproach.
Givenitsfocusonpracticalproblemsanditsnatureofconductingtheresearchinareal-worldsettingwithactiveinvolvementofpractitioners,designresearchmaylooklikeactionresearch.Soonemaywonderhowdesignresearchisrelatedtoactionresearch.Indeed,actionresearchisalsodealingwithreal-worldproblems,aimingatimprovingpractice,cyclicalinnatureandparticipative(Denscombe,2007),buttheessentialdifferenceisthat
![Page 35: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/35.jpg)
an introduction to educational design research 33
actionresearchisnotaimedatgeneratingdesignprinciples–ithasaparticularnicheamongprofessionalswhowanttouseresearchtoimprovetheirpractices(o.c.:122).
Wediscussedhowdesignresearchersshouldstriveforgeneralizabledesignprinciplesinthemeaningofgeneralizingtoabroadertheory.Whendesignresearchisconductedwithintheframeworkofaprogramofresearchaddressingfundamentalproblemsineducationalpractice,itwillresultinaspecificbodyofknowledge,vizsubstantiveandproceduraldesignprinciplesthatmaycontributetoimproveeducation.Ontheotherhandmanyquestionsarestilltobeaddressedastherearemanytypesofpracticalproblemsandthereforemanytypesofresearchgoalsforwhichdesignresearchmaybethebestapproach(e.g.,Reeves(2000)mentionssixdifferenttypesofgoals).
VandenAkker,Gravemeijer,McKenneyandNieveen(2006)reportthepresentationsanddiscussionsataseminardedicatedtoeducationaldesignresearch.Theirbookpoints-nexttodiscussinganumberofapproachestodesignresearchbyGravemeijerandCobb(2006),Reeves(2006)andMcKenneyetal.(2006)-toissueslikeassessingthequalityofdesignresearchproposals(chaptersbyPhillips,2006,andbyEdelson,2006)andthequalityofdesignresearch(chapterbyKelly,2006)whichneedfurtherreflectionandelaboration.
Finally,anumberofresearchreportsanddissertationshavebeenpublishedwhichareexemplaryforhowdesignresearchcanbeconducted(seechapter6forexamples).Butfordesignresearchtomaturefurthermoreresearchprojectsinavarietyofcontextsshouldnotonlybeconducted,butalsoreportedanddiscussedinresearchjournalsandatconferences.
OurhopeisthatthecommunityofeducationaltechnologistsinChinawillembarkonthisresearchendeavorandwillactivelycontributetothefurtherdevelopmentofeducationaldesignresearch.
Acknowledgement: in preparing this chapter, much use has been made of Van den Akker, Gravemeijer, McKenney and Nieveen (2006).
![Page 36: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/36.jpg)
an introduction to educational design research34
References
Bannan-Ritland,B.(2003).TheRoleofDesigninResearch:TheIntegrativeLearningDesignFrameworkEducational Researcher, 32(1),21-24.
Barab,S.&Squire,K.(2004).Design-basedresearch:puttingastakeintheground.Journal of the Learning Sciences, 13(1),1-14.
Brown,A.L.(1992).Designexperiments:Theoreticalandmethodologicalchallengesincreatingcomplexinterventionsinclassroomsettings.The Journal of The Learning Sciences, 2(2),141-178.
Cronbach,L.J.(1975).Beyondthetwodisciplinesofscientificpsychology.American Psychologist, 30(2),116-27.
Denscombe,M.(2007,3rded)).The good research guide for small-scale social research projects. Buckingham(UK):OpenUniversityPress.
Design-BasedResearchCollective(2003).Design-basedresearch:Anemergingparadigmforeducationalinquiry.Educational Researcher, 32(1),5-8.
diSessa,A.A.&Cobb,P.(2004).Ontologicalinnovationandtheroleoftheoryindesignexperiments.Educational Researcher, 32(1),77-103.
Edelson,D.C.(2006).Balancinginnovationandrisk:assessingdesignresearchproposals.In:VandenAkker,J.,Gravemeijer,K,McKenney,S.&Nieveen,N.(Eds).(2006).Educational design research.London:Routledge,100-106.
Gravemeijer,K.&Cobb,P.(2006).Designresearchfromalearningdesignperspective.In:VandenAkker,J.,Gravemeijer,K,McKenney,S.&Nieveen,N.(Eds).(2006).Educational design research. London:Routledge,17-51.
Gustafson,K.L.&Branch,R.M.(2002).Survey of instructional development models(4thed).Syracuse(NY,USA):ERICClearinghouseonInformation&Technology.
Kelly,A.E.(2006).Qualitycriteriafordesignresearch:evidenceandcommitments.In:VandenAkker,J.,Gravemeijer,K,McKenney,S.&Nieveen,N.(Eds).(2006).Educational design research.London:Routledge,107-118.
Knippels,M.C.P.J.(2002).Coping with the abstract and complex nature of genetics in biology education.Utrecht(TheNetherlands):UniversityofUtrecht–CD-BètaPress.
Krathwohl,D.R.(1998,2nded).Methods of Educational and Social Science Research: An integrated approach.NewYork:Longman
Lijnse,P.L.(1995).“DevelopmentalResearch”asawaytoanempiricallybased“DidacticalStructure”ofScience.Science Education, 29(2),189-199.
Mafumiko,FideliceSimbagungileMbaruku(2006).Micro-scale experimentation as a catalyst for improving the chemistry curriculum in Tanzania.Doctoralthesis.Enschede(TheNetherlands)UniversityofTwente.Availablefrom:http://purl.org/utwente/55448
McKenney,S.(2001).Computer-based support for science education materials developers in Africa: exploring potentials.Doctoraldissertation.Enschede:UniversityofTwente.
![Page 37: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/37.jpg)
an introduction to educational design research 35
McKenney,S.,Nieveen,N.&VandenAkker,J.(2006).Designresearchfromacurriculumperspective.In:VandenAkker,J.,Gravemeijer,K,McKenney,S.&Nieveen,N.(Eds).(2006).Educational design research. London:Routledge,62-90.
Nieveen,N.(1999).Prototypingtoreachproductquality.InJ.vandenAkker,R.M.Branch,K.Gustafson,N.Nieveen,&T.Plomp(Eds),Design approaches and tools in education and training.Boston:KluwerAcademic,125-136.
Nieveen,N.,McKenney,S.&VandenAkker,J.(2006).Educationaldesignresearch:thevalueofvariety.In:VandenAkker,J.,Gravemeijer,K,McKenney,S.&Nieveen,N.(Eds).(2006).Educational design research.London:Routledge,151-158.
Phillips,D.C.(2006).Assessingthequalityofdesignresearchproposals:somephilosophicalperspectives.In:VandenAkker,J.,Gravemeijer,K,McKenney,S.&Nieveen,N.(Eds).(2006).Educational design research.London:Routledge,93-99.
Reeves,T.(2000).Enhancingtheworthofinstructionaltechnologyresearchthrough“designexperiments”andotherdevelopmentalstrategies.AERA.AERA.RetrievedOct.20,2006fromhttp://it.coe.uga.edu/~treeves/AERA2000Reeves.pdf
Reeves,T.C.(2006).Designresearchfromatechnologyperspective..In:VandenAkker,J.,Gravemeijer,K,McKenney,S.&Nieveen,N.(Eds).(2006).Educational design research.London:Routledge,52-66.
Shavelson,R.&Towne,L.(2002).Scientific research in education.WashingtonDC:NationalAcademicPress(website:http://books.nap.edu/books)
Tecle,A.T.(2006).The potential of a professional development scenario for supporting biology teachers in Eritrea.Doctoraldissertation.Enschede(TheNetherlands):UniversityofTwente.
Tessmer,M.(1993).Planning and conducting formative evaluations.London:KoganPage.Thijs,A.(1999).Supporting science curriculum reform in Botswana: The potential of peer
coaching.Doctoraldissertation.Enschede:UniversityofTwente.VandenAkker,J.(1999).PrinciplesandMethodsofDevelopmentResearch.InJ.vanden
Akker,R.M.Branch,K.Gustafson,N.Nieveen,&T.Plomp(Eds),Design approaches and tools in education and training.Boston:KluwerAcademic,1-14.
VandenAkker,J.(2005).Hoekansamenwerkingleidentotmeersuccesenwijsheidinonderwijsontwikkeling?[Howcancollaborationleadtomoresuccessandwisdomineducationdevelopment?].Pedagogische Studiën, 82(4),343-7.
VandenAkker,J.,Gravemeijer,K,McKenney,S.&Nieveen,N.(Eds).(2006).Educational design research.London:Routledge.ISBN10:0-415-39635-2(pbk)
Wademan,M.(2005).Utilizing Development Research to Guide People Capability Maturity Model Adoption Considerations.Unpublisheddoctoraldissertation,SyracuseUniversity(NewYork,USA).
Yin,R.K.(2003,3eded).Case study research: design and methods.NewburyPark(CA,USA):Sage–AppliedSocialResearchMethodsSeries,volume5.
![Page 38: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/38.jpg)
![Page 39: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/39.jpg)
an introduction to educational design research 37
2. CurriculumDesignResearch Jan van den Akker
Introduction
Thetitleofthischapter (Curriculum Design Research)intentionallycombinestwofields:‘curriculumdesign’and‘designresearch’.Itsymbolizestheaimofthistexttodiscussthefunctionandformsofdesignresearchfromacurricularperspective.Inparticular,itfocusesonhowdesignresearchcanincreasethequalityofcurriculumdesignanddevelopment.Also,itillustrateshowtherelevanceofeducationalresearch-awidelydebatedissue-canbenefitfromaconnectiontocurriculumpoliciesandpractices.Giventhisaimithelpstohaveanumberofbasicconceptsandanalyticalperspectivesavailablethatcanstructurecurriculardeliberationsandreducethecomplexityofcurriculumtasks.Thusmyinitialfocusinthischapter(buildingonvandenAkker,2003)isonsummarizingasetofconceptsandperspectivesthathelptoincreasethetransparencyandbalanceofcurriculumanalysis,developmentanddiscourse.Then,thefocuswillshifttowards(curriculum)designresearch(buildingonvandenAkker,1999,2006,andonvandenAkker,Gravemeijer,McKenneyandNieveen,2006).First,Iwillsketchthepotentialandcharacteristicsofdesignresearchinaddressingcomplexcurriculumchallenges.Second,Iwilladdressanumberofmethodologicalissues.Finally,Iwillpayattentiontoaclassicprobleminalleducationalresearch:generalizationoffindings.
Curriculum, what’s in a name?
Whenthereisamyriadofdefinitionsofaconceptintheliterature(aswithcurriculum),itisoftendifficulttokeepaclearfocusonitsessence.Inthosecasesitoftenhelpstosearchfortheetymologicaloriginoftheconcept.TheLatinword‘curriculum’(relatedtotheverb‘currere’i.e.running)referstoa‘course’or‘track’tobefollowed.Inthecontextofeducation,wherelearningisthecentralactivity,themostobviousinterpretationofthewordcurriculumisthentoviewitasacourse,trajectory,or‘plan for learning’(cf.Taba,1962).Thisveryshortdefinition(reflectedinrelatedtermsinmanylanguages)limitsitselftothecoreofallotherdefinitions,permittingallsortsofelaborationsforspecificeducationallevels,contexts,andrepresentations.Obviously,contextualspecificationisalwaysneededincurriculumconversationstoclarifytheperspective.Giventhissimpledefinition,adifferentiationbetweenvariouslevelsofthecurriculumhasproventobeveryusefulwhentalkingaboutcurricularactivities(policy-making;designanddevelopment;evaluationandimplementation).Thenextdistinctionappearstobehelpful:• International/comparative(orsupralevel)• System/society/nation/state(ormacro)level(e.g.nationalsyllabiorcoreobjectives)
![Page 40: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/40.jpg)
an introduction to educational design research38
• School/institution(ormeso)level(e.g.school-specificcurriculum)• Classroom(ormicro)level(e.g.textbooks,instructionalmaterials)• Individual/personal(ornano)level.
Thesupralevelusuallyreferstointernationaldebatesoragreementsonaimsandqualityofeducation,sometimesfuelledbyoutcomesofinternationallycomparativestudies(cf.PISAorTIMSS1).Curriculumdevelopmentatthesupralevelisusuallyofa‘generic’nature,while‘site-specific’approachesaremoreapplicableforthelevelsclosertoschoolandclassroompractice.Moreover,theprocessofcurriculumdevelopmentcanbeseenasnarrow(developingaspecificcurricularproduct)orbroad(alongterm,ongoingprocessofcurriculumimprovement,oftenincludingmanyrelatedaspectsofeducationalchange,e.gteachereducation,schooldevelopment,testingandexaminations).Inordertounderstandproblemsofcurriculumdecision-makingandenactment,abroaderdescriptionofcurriculumdevelopmentisoftenmostappropriate:usuallyalongandcyclicprocesswithmanystakeholdersandparticipants;inwhichmotivesandneedsforchangingthecurriculumareformulated;ideasarespecifiedinprogramsandmaterials;andeffortsaremadetorealizetheintendedchangesinpractice.
Moreover,curriculacanberepresentedinvariousforms.Clarificationofthoseformsisespeciallyusefulwhentryingtounderstandtheproblematiceffortstochangethecurriculum.Acommonbroaddistinctionisbetweenthethreelevelsofthe‘intended’,‘implemented’,and‘attained’curriculum.Amorerefinedtypology(vandenAkker,2003)isoutlinedinbox1.
INTENDED Ideal Vision (rationale or basic philosophy underlying a curriculum)
Formal/Written Intentions as specified in curriculum documents and/or materials
IMPLEMENTED Perceived Curriculum as interpreted by its users (especially teachers)
Operational Actual process of teaching and learning (also: curriculum-in-action)
ATTAINED Experiential Learning experiences as perceived by learners
Learned Resulting learning outcomes of learners
Box 1: Typology of curriculum representations
Traditionally,theintendeddomainreferspredominantlytotheinfluenceofcurriculumpolicymakersandcurriculumdevelopers(invariousroles),theimplementedcurriculum
1) PISAistheOECDProgrammeforInternationalStudentAssessment,asurveyeverythreeyearsofthe15-year-olds.TIMSSistheTrendsInMathematicsandSciencesStudy,conductedevery4yearsbytheInternationalAssociationfortheEvaluationofEducationalAchievement(IEA)inprimaryandsecondaryeducation.
![Page 41: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/41.jpg)
an introduction to educational design research 39
relatesespeciallytotheworldofschoolsandteachers,andtheattainedcurriculumhastodowiththestudents.
Besidesthisdifferentiationinrepresentations,curriculumproblemscanbeapproachedfromvariousanalyticalangles.Forexample,Goodlad(1994)distinguishesthefollowingthreedifferentperspectives:• substantive,focusingontheclassicalcurriculumquestionaboutwhatknowledgeisof
mostworthforinclusioninteachingandlearning;• technical-professional,referringtohowtoaddresstasksofcurriculumdevelopment;• socio-political,referringtocurriculumdecision-makingprocesses,wherevaluesand
interestsofdifferentindividualandagenciesareatstake.Somemightarguethatthislististoolimitedasitrefersespeciallytocurriculumissuesfor‘traditional’planningforlearninginschools,anddoesnotincludethemore‘critical’perspectivesthatareamplypresentincurriculumtheoryliterature(e.g.Pinar,Reynolds,Slattery&Taubman,1995).However,fromaprimaryinterestincurriculumimprovement,thethreeperspectivesseemusefulandappropriate.
The vulnerable curriculum spider web
Oneofthemajorchallengesforcurriculumimprovementiscreatingbalanceandconsistencybetweenthevariouscomponentsofacurriculum(i.e.planforlearning).Whatarethosecomponents?TherelativelysimplecurriculumdefinitionbyWalker(2003)includesthreemajorplanningelements:content,purposeandorganizationoflearning.However,curriculumdesignandimplementationproblemshavetaughtusthatitiswisetopayexplicitattentiontoamoreelaboratedlistofcomponents.Elaboratingonvarioustypologies,wehavecometoadheretoaframework(seeBox2)oftencomponentsthataddresstenspecificquestionsabouttheplanningofstudentlearning.
Rationale or Vision Why are they learning?
Aims & Objectves Toward which goals are they learning?
Content What are they learning?
Learning activities How are they learning?
Teacher role How is the teacher facilitating learning?
Materials & Resources With what are they learning?
Grouping With whom are they learning?
Location Where are they learning?
Time When are they learning?
Assessment How to measure how far learning has progressed?
Box 2: Curriculum components
![Page 42: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/42.jpg)
an introduction to educational design research40
The‘rationale’(referringtooverallprinciplesorcentralmissionoftheplan)servesasmajororientationpoint,andthenineothercomponentsareideallylinkedtothatrationaleandpreferablyalsoconsistentwitheachother.Foreachofthecomponentsmanysub-questionsarepossible.Notonlyonsubstantiveissues(seethenextsection),but,forexample,alsoon‘organizational’aspectsas:• Grouping: - Howarestudentsallocatedtovariouslearningtrajectories? - Arestudentslearningindividually,insmallgroups,orwhole-class?• Location: - Arestudentslearninginclass,inthelibrary,athome,orelsewhere? - Whatarethesocial/physicalcharacteristicsofthelearningenvironment?• Time: - Howmuchtimeisavailableforvarioussubjectmatterdomains? - Howmuchtimecanbespentonspecificlearningtasks?Therelevanceofthesecomponentsvariesacrossthepreviouslymentionedcurriculumlevels(supra,macro,meso,micro,nano)andrepresentations.Afewexamplesmayillustratethis.• Curriculumdocumentsatthemacro-levelwillusuallyfocusonthefirstthree
components(rationale,aims&objectives,content;ofteninratherbroadterms),sometimesaccompaniedbyanoutlineoftimeallocationsforvarioussubjectmatterdomains.
• Whenonetakestheoperationalcurriculuminschoolsandclassroomsinmind,alltencomponentshavetobecoherentlyaddressedtoexpectsuccessfulimplementationandcontinuation.
• Thecomponentsoflearningactivities,teacherrole,andmaterials&resourcesareatthecoreofthemicro-curriculumintheclassroom.
• Thecomponentofassessmentdeservesseparateattentionatalllevelsandrepresentationssincecarefulalignmentbetweenassessmentandtherestofthecurriculumappearstobecriticalforsuccessfulcurriculumchange.
Ourpreferentialvisualizationofthetencomponentsistoarrangethemasaspiderweb(Figure1),notonlyillustratingitsmanyinterconnections,butalsounderliningitsvulnerability.Thus,althoughtheemphasisofcurriculumdesignonspecificcomponentsmayvaryovertime,eventuallysomekindofalignmenthastooccurtomaintaincoherence.AstrikingexampleisthetrendtowardintegrationofICTinthecurriculum,withusuallyinitialattentiontochangesinmaterialsandresources.Manyimplementationstudieshaveexemplifiedtheneedforamorecomprehensiveapproachandsystematicattentiontotheothercomponentsbeforeonecanexpectrobustchanges.
![Page 43: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/43.jpg)
an introduction to educational design research 41
Thespiderwebalsoillustratesafamiliarexpression:everychainisasstrongasitsweakestlink.Thatseemsanotherveryappropriatemetaphorforacurriculum,pointingtothecomplexityofeffortstoimprovethecurriculuminabalanced,consistentandsustainablemanner.
Figure 1: Curricular spider web
Perspectives on substantive choices
Aclassicapproachtotheeternalcurriculumquestionofwhattoincludeinthecurriculum(orevenmoredifficultaswellasurgent:whattoexcludefromit)istosearchforabalancebetweenthreemajorsourcesororientationsforselectionandprioritysetting:• Knowledge:whatistheacademicandculturalheritagethatseemsessentialfor
learningandfuturedevelopment?• Society:whichproblemsandissuesseemrelevantforinclusionfromtheperspectiveof
societaltrendsandneeds?• Learner:whichelementsseemofvitalimportanceforlearningfromthepersonaland
educationalneedsandinterestsofthelearnersthemselves?
Asse
ssm
ent
Aims & Objectives
Content
Learning activitiesTe
ache
r rol
e
Materials &
Resources
Location
Time
Rationale
Grouping
![Page 44: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/44.jpg)
an introduction to educational design research42
Answerstothesequestionsusuallyconstitutetherationaleofacurriculum.Inevitably,choiceshavetobemade,usuallyinvolvingcompromisesbetweenthevariousorientations(andtheirrespectiveproponentsandpressuregroups).Oftentimes,effortsfailtoarriveatgenerallyacceptable,clearandpracticalsolutions.Theresultofaddingupallkindsofwishesisthatcurriculatendtogetoverloadedandfragmented.Implementationofsuchincoherentcurriculaeventuallytendstoleadtostudentfrustrations,failure,anddropout.Howtocreateabettercurriculumbalance?Easyanswersarenotavailable,butafewalternativesseemtohavesomepromise.First,inviewofthemultitudeof(academic)knowledgeclaims,itsometimeshelpstoreducethebignumberofseparatesubjectdomainstoamorelimitednumberofbroaderlearningareas,combinedwithsharperprioritiesinaimsforlearning(focusingonbasicconceptsandskills).Second,referringtotheavalancheofsocietalclaims,moreinteractionbetweenlearninginsideandoutsidetheschoolmayreducetheburden.However,themosteffectiveresponseisprobablytobemoreselectiveinreactingtoallsortsofsocietalproblems.AsCuban(1992)phraseditclearly:schoolsshouldnotfeelobligedtoscratchthebackofsocietyeverytimesocietyhasanitch.Andthird,aboutthelearners’perspective:worldwide,manyinterestingeffortsareongoingtomakelearningmorechallengingandintrinsicallymotivatingbymovingfromtraditional,teacher-andtextbook-dominatedinstructiontowardsmoremeaningful,activity-basedandautonomouslearningapproaches.
Development strategies
Tosketchcurriculumdevelopmentasaproblematicdomainisactuallyanunderstatement.Fromasocio-politicalstance,itseemsoftenmoreappropriatetodescribeitasawarzone,fullofconflictsandbattlefieldsbetweenstakeholderswithdifferentvaluesandinterests.Problemsmanifestthemselvesinthe(sometimesspectacularandpersistent)gapsbetweentheintendedcurriculum(asexpressedinpolicyrhetoric),theimplementedcurriculum(reallifeinschoolandclassroompractices),andtheattainedcurriculum(asmanifestedinlearnerexperiencesandoutcomes).Atypicalconsequenceofthosetensionsisthatvariousfrustratedgroupsofparticipantsblameeachotherforthefailureofreformorimprovementactivities.Althoughsuchblaminggamesoftenseemratherunproductive,therearesomeseriouscriticalremarkstobemadeonmanycurriculumdevelopmentapproachesworldwide.Firstofall,manycurriculumreformeffortscanbecharacterizedbyoverlybiginnovationambitions(especiallyofpoliticians)withinunrealisticallyshorttimelinesandwithverylimitedinvestmentinpeople,especiallyteachers.Second,oftentimesthereisalackofcoherencebetweentheintendedcurriculumchangeswithothersystemcomponents(especiallyteachereducationandassessment/examinationprograms).And
![Page 45: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/45.jpg)
an introduction to educational design research 43
lastbutnotleast,timelyandauthenticinvolvementofallrelevantstakeholdersisoftenneglected.Fromastrategicpointofview,theliteraturehasofferedusmany(technical-professional)modelsandstrategiesforcurriculumdevelopment.ThreeprominentapproachesareTyler’srational-linearapproach,Walker’sdeliberativeapproach,andEisner’sartisticapproach.Asitdoesnotfitwithinthepurposeofthischaptertoexplainthosemodelsinparticular,thereaderisreferredtoeducativetextsasfromMarshandWillis(2003).Obviously,thecontextandnatureofthecurriculumdevelopmenttaskathandwilldeterminetoalargeextentwhatkindofstrategyisindicated.Itisnoteworthythatwearebeginningtoseemoreblendedapproachesthatintegratevarioustrendsandcharacteristicsofrecentdesignanddevelopmentapproachesinthefieldofeducationandtraining(foranoverviewandaseriesofexamples:seevandenAkker,Branch,Gustafson,Nieveen&Plomp,1999).Somekeycharacteristics:• Pragmatism:Recognitionthatthereisnotasingleperspective,overarchingrationaleor
higherauthoritythatcanresolvealldilemmasforcurriculumchoicestobemade.Thepracticalcontextanditsusersareintheforefrontofcurriculumdesignandenactment.
• Prototyping:Evolutionaryprototypingofcurricularproductsandtheirsubsequentrepresentationsinpracticeisviewedasmoreproductivethanquasi-rationalandlineardevelopmentapproaches.Gradual,iterativeapproximationofcurriculardreamsintorealitiesmaypreventparalysisandfrustrations.Formativeevaluationoftentative,subsequentcurriculumversionsisessentialtosuchcurriculumimprovementapproaches.
• Communication:Acommunicative-relationalstyleisdesirableinordertoarriveattheinevitablecompromisesbetweenstakeholderswithvariousrolesandinterestsandtocreateexternalconsistencybetweenallpartiesinvolved.
• Professional development:Inordertoimprovechancesonsuccessfulimplementation,thereisatrendtowardsmoreintegrationofcurriculumchangeandprofessionallearninganddevelopmentofallindividualsandorganizationsinvolved.
Designordevelopment(al)researchisaresearchapproachthatincorporatessomeofthesecharacteristics,anditbecomesevenmorepromisingbyaddingtheelementofknowledgegrowthtoit(vandenAkker,1999).Suchresearchcanstrengthentheknowledgebaseintheformofdesignprinciplesthatofferheuristicadvicetocurriculumdevelopmentteams,when(morethanincommondevelopmentpractices)deliberateattentionispaidtotheoreticalembeddingofdesignissuesandempiricalevidenceisofferedaboutthepracticalityandeffectivenessofthecurricularinterventionsinrealusersettings.However,thereareseveralpersistentdilemmasforcurriculumdevelopmentthatcannoteasilyberesolved,letalonethroughgenericstrategies.Forexample:howtocombineaspirationsforlarge-scalecurriculumchangeandsystemaccountabilitywiththeneedfor
![Page 46: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/46.jpg)
an introduction to educational design research44
localvariationsandownership?Thetensionbetweentheseconflictingwishescanbesomewhatreducedwhenoneavoidsthealltoocommon‘onesizefitsall’approach.Moreadaptiveandflexiblestrategieswillavoiddetailedelaborationandover-specificationofcentralcurriculumframeworks.In-stead,theyoffersubstantialoptionsandflexibilitytoschools,teachers,andlearners.Althoughstrugglesaboutprioritiesinaimsandcontentwillremaininevitable,theprincipleof‘lessismore’shouldbepursued.However,whatisincorporatedinacorecurriculumshouldbeclearlyreflectedinexaminationandassessmentapproaches.The‘enactment’perspective(teachersandlearnerstogethercreatetheirowncurriculumrealities)isincreasinglyreplacingthe‘fidelity’perspectiveonimplementation(teachersfaithfullyfollowcurricularprescriptionsfromexternalsources).Thistrendputsevenmoreemphasisonteachersaskeypeopleincurriculumchange.Bothindividualaswellasteamlearningisessential(Fullan,2001).Teachersneedtogetoutoftheircustomaryisolation.Collaborativedesignandpilotingofcurricularalternativescanbeveryproductive,especiallywhenexperiencesareexchangedandreflecteduponinastructuredcurriculumdiscourse.Interactionwithexternalfacilitatorscancontributetocarefulexplorationsofthe‘zoneofproximaldevelopment’ofteachersandtheirschools.Cross-fertilizationbetweencurriculum,teacher,andschooldevelopmentisaconditiosinequanonforeffectiveandsustainablecurriculumimprovement.Theincreasinglypopularmissionstatementsofschoolstobecomeattractiveandinspiringenvironmentsforstudentsandteacherscanonlyberealizedwhensuchintegratedscenariosarepractised.
The potential of curriculum design research
Variousmotivesforinitiatingandconductingcurriculumdesignresearchcanbementioned.Abasicmotivestemsfromtheexperiencethatmanyresearchapproaches(e.g.experiments,surveys,correlationalanalyses),withtheirfocusondescriptiveknowledge,hardlyprovideprescriptionswithusefulsolutionsforavarietyofdesignanddevelopmentproblemsineducation.Probablythegreatestchallengeforprofessionaldesignersishowtocopewiththemanifolduncertaintiesintheircomplextasksinverydynamiccontexts.Iftheydoseeksupportfromresearchtoreducethoseuncertainties,severalfrustrationsoftenarise:answersaretoonarrowtobemeaningful,toosuperficialtobeinstrumental,tooartificialtoberelevant,and,ontopofthat,theyusuallycometoolatetobeofanyuse.Curriculumdesignersdoappreciatemoreadequateinformationtocreateasolidgroundfortheirchoicesandmoretimelyfeedbacktoimprovetheirproducts.Moreover,theprofessionalcommunityofdevelopersasawholewouldbehelpedbyagrowingbodyofknowledgeoftheoreticallyunderpinnedandempiricallytesteddesignprinciplesandmethods.
![Page 47: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/47.jpg)
an introduction to educational design research 45
Anotherreasonforcurriculumdesignresearchstemsfromthehighlyambitiousandcomplexnatureofmanycurriculumreformpoliciesineducationworldwide.Thesereformendeavorsusuallyaffectmanysystemcomponents,areoftenmulti-layered,includingbothlarge-scalepoliciesandsmall-scalerealization,andareverycomprehensiveintermsoffactorsincludedandpeopleinvolved.Thoseradical‘revolutions’,ifpromisingatall,cannotberealizedonthedrawingtable.Thescopeofdiverseneedsisoftenverywide,theproblemstobeaddressedareusuallyill-specified,theeffectivenessofproposedinterventionsismostlyunknownbeforehand,andtheeventualsuccessishighlydependentonimplementationprocessesinabroadvarietyofcontexts.Therefore,suchcurriculumreformeffortswouldprofitfrommoreevolutionary(interactive,cyclic,spiral)approaches,withintegratedresearchactivitiestofeedtheprocess(bothforwardandbackward).Suchanapproachwouldprovidemoreopportunitiesfor‘successiveapproximation’oftheidealsandformorestrategiclearningingeneral.Inconclusion:curriculumdesignresearchseemsawiseandproductiveapproachforcurriculumdevelopment.
Features of curriculum design research
Curriculumdesignresearchisofteninitiatedforcomplex,innovativetasksforwhichonlyveryfewvalidatedprinciplesareavailabletostructureandsupportthedesignanddevelopmentactivities.Sinceinthosesituationstheimageandimpactoftheinterventiontobedevelopedisoftenstillunclear,theresearchfocusesonrealizinglimitedbutpromisingexamplesofthoseinterventions.Theaimisnottoelaborateandimplementcompleteinterventions,buttocometo(successive)prototypesthatincreasinglymeettheinnovativeaspirationsandrequirements.Theprocessisoftencyclicorspiral:analysis,design,evaluationandrevisionactivitiesareiterateduntilasatisfyingbalancebetweenidealsandrealizationhasbeenachieved.
Towhatextentdothesedesignresearchactivitiesdifferfromwhatistypicalfordesignanddevelopmentapproachesinprofessionalpractices?Whataretheimplicationsoftheaccountabilityofresearcherstothe‘scientificforum’?Attheriskofexaggeratingthedifferences,letusoutlinesomeofthem,basedonwhatisknownaboutroutinizedstandard-patternsincurriculumdevelopmentpractices.Ofcourse,alotofactivitiesaremoreorlesscommonforbothapproaches,sothefocuswillbeonthoseadditionalelementsthataremoreprominentindesignresearchthanincommondesignanddevelopmentpractices.
(1)PreliminaryinvestigationAmoreintensiveandsystematicpreliminaryinvestigationofcurriculumtasks,problems,andcontextismade,includingsearchingformoreaccurateandexplicitconnectionsofthat
![Page 48: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/48.jpg)
an introduction to educational design research46
analysiswithstate-of-the-artknowledgefromliterature.Sometypicalactivitiesinclude:literaturereview;consultationofexperts;analysisofavailablepromisingexamplesforrelatedpurposes;casestudiesofcurrentpracticestospecifyandbetterunderstandneedsandproblemsinintendedusercontexts.
(2) TheoreticalembeddingMoresystematiceffortsaremadetoapplystate-of-the-artknowledgeinarticulatingthetheoreticalrationaleforcurriculumdesignchoices.Moreover,explicitfeedbacktoassertionsinthedesignrationaleaboutessentialcharacteristicsoftheintervention(substantivedesignprinciples)ismadeafterempiricaltestingofitsquality.Thistheoreticalarticulationcanincreasethe‘transparency’and‘plausibility’oftherationale.Becauseoftheirspecificfocus,thesetheoreticalnotionsareusuallyreferredtoas‘mini’-or‘local’theories,althoughsometimesconnectionscanalsobemadeto‘middle-range’theorieswithasomewhatbroaderscope.
(3) EmpiricaltestingClearempiricalevidenceisdeliveredaboutthepracticalityandeffectivenessofthecurriculumfortheintendedtargetgroupinrealusersettings.Inviewofthewidevariationofpossibleinterventionsandcontexts,abroadrangeof(direct/indirect;intermediate/ultimate)indicatorsfor‘success’shouldbeconsidered.
(4) Documentation,analysisandreflectiononprocessandoutcomesMuchattentionispaidtosystematicdocumentation,analysisandreflectionontheentiredesign,development,evaluationandimplementationprocessandonitsoutcomesinordertocontributetotheexpansionandspecificationofthemethodologyofcurriculumdesignanddevelopment.
Morethanmostotherresearchapproaches,designresearchaimsatmakingbothpracticalandscientificcontributions.Inthesearchforinnovative‘solutions’forcurriculumproblems,interactionwithpractitioners(invariousprofessionalroles:teachers,policymakers,developers,andthelike)isessential.Theultimateaimisnottotestwhethertheory,whenappliedtopractice,isagoodpredictorofevents.Theinterrelationbetweentheoryandpracticeismorecomplexanddynamic:isitpossibletocreateapracticalandeffectivecurriculumforanexistingproblemorintendedchangeintherealworld?Theinnovativechallengeisusuallyquitesubstantial,otherwisetheresearchwouldnotbeinitiatedatall.Interactionwithpractitionersisneededtograduallyclarifyboththeproblematstakeandthecharacteristicsofitspotentialsolution.Aniterativeprocessof‘successiveapproximation’or‘evolutionaryprototyping’ofthe‘ideal’interventionisdesirable.Directapplicationoftheoryisnotsufficienttosolvethosecomplicatedproblems.Onemightstate
![Page 49: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/49.jpg)
an introduction to educational design research 47
thatamore‘constructivist’developmentapproachispreferable:researchersandpractitionerscooperativelyconstructworkableinterventionsandarticulateprinciplesthatunderpintheeffectsofthoseinterventions.Anotherreasonforcooperationisthatwithoutinvolvementofpractitionersitisimpossibletogainclearinsightinpotentialcurriculumimplementationproblemsandtogeneratemeasurestoreducethoseproblems.Newinterventions,howeverimaginativetheirdesign,requirecontinuousanticipationatimplementationissues.Notonlyfor‘social’reasons(tobuildcommitmentandownershipofusers)butalsofor‘technical’benefits:toimprovetheirfitnessforsurvivalinreallifecontexts.Therefore,rigoroustestingofpracticalityisaconditiosinequanonindesignresearch.
Emphasis on formative evaluation
Ashasbecomeclearintheprevioussections,formativeevaluationholdsaprominentplaceincurriculumdesignresearch.Themainreasonforthiscentralroleisthatformativeevaluationprovidestheinformationthatfeedsthecycliclearningprocessofcurriculumdevelopersduringthesubsequentloopsofadesignanddevelopmenttrajectory.Itismostusefulwhenfullyintegratedinacycleofanalysis,design,evaluation,revision,etcetera,andwhencontributingtoimprovementofthecurriculum.Thebasiccontributionofformativeevaluationistoqualityimprovementofthecurriculumunderdevelopment.Quality,however,isanabstractconceptthatrequiresspecification.Duringdevelopmentprocesses,theemphasisincriteriaforqualityusuallyshiftsfromrelevance,toconsistency,topracticality,toeffectiveness2.Relevancereferstotheextentthattheintendedcurriculumisperceivedtobearelevantimprovementtopractice,asseenfromthevariedperspectivesofpolicymakers,practitionersandresearchers.Consistencyreferstotheextentthatthedesignofthecurriculumisbasedonstate-of-the-artknowledgeandthatthevariouscomponentsoftheinterventionareconsistentlylinkedtoeachother(cf.thecurricularspiderweb).Practicalityreferstotheextentthatusers(andotherexperts)considertheinterventionasclear,usableandcost-effectivein‘normal’conditions.Effectivenessreferstotheextentthattheexperiencesandoutcomeswiththeinterventionareconsistentwiththeintendedaims.
Themethodsandtechniquesforevaluationwillusuallybeattunedtothatshiftincriteria.Forexample,validitycanadequatelybeevaluatedthroughexpertappraisal,practicalityviamicro-evaluationsandtry-outs,andeffectivenessinfieldtests.Inlaterstagesofformativeevaluation,methodsofdatacollectionwillusuallybelessintensivebutwithanincreasingnumberofrespondents(e.g.achievementtestformanystudentscomparedtoin-depthinterviewwithafewexperts).
2) Seeforthesecriteriaalsochapters1and5.
![Page 50: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/50.jpg)
an introduction to educational design research48
Formativeevaluationwithindevelopmentresearchshouldnotonlyconcentrateonlocatingshortcomingsoftheinterventioninitscurrent(draft)version,butespeciallygeneratesuggestionsonhowtoimprovethoseweakpoints.Richnessofinformation,notablysalienceandmeaningfulnessofsuggestionsinhowtomakeaninterventionstronger,isthereforemoreproductivethanstandardizationofmethodstocollectandanalyzedata.Also,efficiencyofproceduresiscrucial.Thelowerthecostsintimeandenergyfordatacollection,processing,analysisandcommunicationwillbe,thebiggerthechancesonactualuseandimpactonthedevelopmentprocess.Forexample,samplesofrespondentsandsituationsfordatacollectionwillusuallyberelativelysmallandpurposivecomparedtosamplingproceduresforotherresearchpurposes.Theaddedvalueofgetting‘productive’informationfrommoresourcestendstodecrease,becausetheopportunitiesfor‘rich’datacollectionmethods(suchasinterviewsandobservations)arelimitedwithbignumbers.Toavoidanoverdoseofuncertaintyindatainterpretation,oftentriangulation(ofmethods,instruments,sources,andsites)isapplied.Theseargumentsespeciallyholdtrueforearlystagesofformativeevaluation,whentheinterventionisstillpoorlycrystallized.
Generalization of curriculum design research findings
Thepracticallymostrelevantoutcomeofcurriculumdesignresearchisitscontributiontowardsoptimizationofthecurricularproductanditsactualuse,leadingtobetterinstructionalprocessesandlearningresults.However,amajorcontributiontoknowledgetobegainedfromdesignresearchisintheformof(bothsubstantiveandmethodological)‘designprinciples’tosupportdevelopersintheirtask.Thoseprinciplesareusuallyheuristicstatementsofaformatsuchas:“IfyouwanttodesigncurriculumX[forthepurpose/functionYincontextZ],thenyouarebestadvisedtogivethatcurriculumthecharacteristicsA,B,andC[substantiveemphasis],andtodothatviaproceduresK,L,andM[proceduralemphasis],becauseoftheoreticalandempiricalargumentsP,Q,andR.”Obviouslythoseprinciplescannotguaranteesuccess,buttheyareintendedtoselectandapplythemostappropriate(substantiveandprocedural)knowledgeforspecificdesignanddevelopmenttasks.Itisnotuncommonindesignresearchthatsuchknowledge,especiallythesubstantiveknowledgeaboutessentialcurriculumcharacteristics,canpartlybeextractedfromaresultingprototypeitself.Thatisoneofthereasonsthatmakeitsoprofitabletosearchforandcarefullyanalyzealreadyavailablecurriculatogenerateideasfornewdesigntasks.However,thevalueofthatknowledgewillstronglyincreasewhenjustifiedbytheoreticalarguments,well-articulatedinprovidingdirections,andconvincinglybacked-upwithempiricalevidenceabouttheimpactofthoseprinciples.Moreover,thoseheuristicprincipleswillbeadditionallypowerfuliftheyhavebeenvalidatedinsuccessfuldesignofmoreinterventionsinmorecontexts.Chancesforsuchknowledgegrowthwillincrease
![Page 51: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/51.jpg)
an introduction to educational design research 49
whendesignresearchisconductedintheframeworkofresearchprograms,becauseprojectscanthenbuildupononeanother.Sincedatacollectionindesignresearchisoftenlimitedtosmall(andpurposive)samples,effortstogeneralizefindingscannotbebasedonstatisticaltechniques,focusingongeneralizationsfromsampletopopulation.Insteadonehastoinvestin‘analytical’formsofgeneralization(cf.Yin,2003):readers/usersneedtobesupportedtomaketheirownattemptstoexplorethepotentialtransferoftheresearchfindingstotheoreticalpropositionsinrelationtotheirowncontext.Reportsondesignresearchcanfacilitatethattaskofanalogyreasoningbyacleartheoreticalarticulationofthedesignprinciplesappliedandbyacarefuldescriptionofboththeevaluationproceduresaswellastheimplementationcontext.Especiallya‘thick’descriptionoftheprocess-in-contextmayincreasethe‘ecological’validityofthefindings,sothatotherscanestimateinwhatrespectsandtowhatextenttransferfromthereportedsituationtotheirownispossible.Anotheroptionthatmaystimulateexplorationofpossibilitiesfor(virtual)generalizationistoorganizeinteractivemeetingswithexpertsfromrelatedcontextstodiscusstheplausibilityoftheresearchfindingsandrecommendationsforrelatedtasksandcontexts.Lastbutnotleast,designresearchmayofferdraftsofvariousrelevantcurriculumversions(withprovenconsistencyandpracticality)thatcanbecomparedinmorequantitative,large-scalestudies.
References
Cuban,L.(1992).Curriculumstabilityandchange.InP.Jackson(Ed.),Handbook of research on curriculum(pp.216-247).NewYork:Macmillan.
Goodlad,J.(1994).Curriculumasafieldofstudy.InT.Husén,&T.Postlethwaite(Eds.),The international encyclopedia of education(pp.1262-1276).Oxford:PergamonPress.
Fullan,M.(2001).The new meaning of educational change.NewYork:TeachersCollegePress.Marsh,C.,&Willis,P.(2003).Curriculum: Alternative approaches, ongoing issues(third
edition).UpperSaddleRiver,NJ:Merrill/PrenticeHall.Pinar,W.,Reynolds,W.,Slattery,P.,&Taubman,P.(1995).Understanding curriculum. An
introduction to the study of historical and contemporary curriculum discourses.NewYork:PeterLang.
Taba,H.(1962).Curriculum development: Theory and practice.NewYork:Harcourt,Brace&World.
vandenAkker,J.(1999).Principlesandmethodsofdevelopmentresearch.InJ.vandenAkkeretal.(Eds.),Design approaches and tools in education and training(pp.1-14).Dordrecht:KluwerAcademicPublishers.
![Page 52: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/52.jpg)
an introduction to educational design research50
vandenAkker,J.(2003).Curriculumperspectives:Anintroduction.InJ.vandenAkker,W.Kuiper&U.Hameyer(Eds.),Curriculum landscapes and trends(pp.1-10).Dordrecht:KluwerAcademicPublishers.
vandenAkker,J.,Branch,R.,Gustafson,K.,Nieveen,N.&Plomp,T.(Eds.)(1999).Design approaches and tools in education and training.Dordrecht:KluwerAcademicPublishers.
vandenAkker,J.,Gravemeijer,K.,McKenney,S.,&Nieveen,N.(Eds.)(2006).Educational design research.London:Routledge.
Walker,D.(2003).Fundamentals of curriculum: Passion and professionalism.Mahwah,NJ:LawrenceErlbaumAssociates.
Yin,R.K.(2003).Case Study Research: Design and Methods.London:Sage.
![Page 53: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/53.jpg)
an introduction to educational design research 51
![Page 54: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/54.jpg)
![Page 55: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/55.jpg)
an introduction to educational design research 53
3. TheIntegrativeLearningDesign Framework:AnIllustratedExample fromtheDomainofInstructional Technology Brenda Bannan
Articulatingacleardefinitionandprocessofdesignresearchisacurrentandprominenttopicamongeducationalresearchers(Kelly,Lesh&Baek,2008;vandenAkker,Gravemeijer,McKennyandNieveen,2006).Designresearchstudiesinvolvecomplexinteractionsandfeedbackcyclesthatcansignificantlyblurtherolesofresearchers,teachers,curriculumdevelopers,instructionaldesignersandassessmentexperts(Kelly,Lesh,Baek&Bannan-Ritland2008).Aseducationalresearchersstruggletoclarifythisresearchmethod,theycontinuetoraisesignificantquestionssuchashowisdesignresearchdifferentfromtheprocessofdesign?Whatareappropriatemethodsandprocessesthatcanbeusedindesignresearch?Howdowesystematicallycreate,testanddisseminatedesignorteachinginterventionsthatwillhavemaximumimpactonpracticecapitalizingondesignresearch?Howdowegenerateboththeoreticalandpracticalknowledgerelatedtocomplexeducationalsettings?Kelly(2006)andothers(Fishman,Marx,Blumenfeld,Krajcik&Soloway,2004;Zaritsky,Kelly,Flowers,Rogers&O’Neill,2003;Rogers2003;Collins,1999;Design-basedResearchCollective,2003)advocatethattheseemergingmethodscallforthearticulationofnewprocessesandcriteriaincludingfactorssuchastheusefulnessandusabilityofknowledge,itsshareability,andmarketability,howwellitdisseminatesandtheextenttowhichitpositivelyimpactspractice.Sabelli(personalcommunication,May16,2002)citesaneedfororganizationalstructureandprotocolforthediffusionofresearchintopracticeandstatesthateducationalresearchsituationsareextremelycomplexsystemsthatcanbenefitfromintegratedsystemresearchstrategies.Thereisaneedforcomprehensivemodelstoguidedesignresearchaddressingtheprocessofdesigning,developingandassessingtheimpactofaneducationalinnovation.Inthischapter,Ipresentanintegrationofexistingdesignandresearchprocessesofferingaguidingframeworkthatgoesbeyondtheindividualdomainsofsocialscience,behavioralscienceandcommunicationtheoryandattemptstointegratethesystematicprocessesoftherelatedfieldsofinstructionaldesign,softwareengineering,productdesign,hencethenameIntegrativeLearningDesignFramework(ILDF).Buildingontheintegrationofprocessesfrommultiplefieldssuchasinstructionaldesign,objectorientedsoftwaredevelopment,productdevelopmentanddiffusionofinnovationsandeducationalresearch,theILDFpresenta“meta-methodological”viewthatattemptstointegratethebestofdesign,researchanddiffusionofeducationalinnovations.Thisframeworkconsistingoffourphases(seeFigure1)challengesresearcherstoprovideimprovedarticulationofdesignresearchprocessesbyphaseandtoconsidertheentirescopeofresearchfrominitialconceptualizationtodiffusionandadoption.
![Page 56: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/56.jpg)
an introduction to educational design research54
Figure 1: Questions and Methods for Design Research by ILDF Phase
Web-enabled Proto-D
iffusion
ConsequensesD
iffusion Adoption
Adaptation
Publish Results
Evaluate Results
Implem
ent.Theory/System
Refinem.
Formative
TestingD
etailed D
esignArticulated Prototype
Research/System
D
esign
AudienceCharacteriza-
tion
TheoryD
evelop.Survey
LiteratureN
eedsAnalysis
Informed Exploration
Enactment
Evaluation: Local Impact
Evaluation: Broader Impact
Questions:
- What are identified gaps/problem
s in theory, practice and/or the m
arketplace?- W
hat information can be gleaned from
existing data or research?- H
ow can w
e characterize the problem or
leaner need?- W
hat are the systemic social, cultural, and
organizational influences or constraints on design?- W
hat are characteristics of the audience?
Methods:
- Benchmarking
- Performance/needs analysis
- Interviews
- Survey of Experts- Focus G
roups- O
bservation/Role Modeling
- Case Studies
Questions:
- What are the learning targets for
innovation?- W
hat design principles or strategies may
be applicable?- H
ow to identify and operationalize
cognitive and performance processes
in design?- To w
hat extent does the design embody
the theoretical model?
Methods:
- Task Analysis- Contextual Analysis- D
esigner Logs- Expert Review- Audience Review
Questions:
- Is the enacted design usable, valid and
relevant?- Is the design instance accessible and efficient in delivering instruction or supporting learning?- W
hat is the local impact or effectiveness
of the design instance?- H
ow effective is the design solution in
achieving learning targets at its highest fideliy in full context?
Methods:
- Usability Testing
- Expert Review- O
bservation or Video records- Interview
s- Form
ative Evaluation- Pre-post Com
parative Studies- Q
uasi-experimental studies
Questions:
-What factors influence diffusion, adoption
and adaption of innovation?- W
hat are the pragmatic dem
ands of the learning environm
ent that influences adoption of design?- W
hat policies and cultures shape participants use of innovation?
Methods:
- Analysis of computer log files
- Multi-site Interview
s, Surveys and O
bservations- D
ata mining
- Correlational studies- Q
uasi-experimental studies
ILD
Guiding
Questions
for Research
ApplicableResearchM
ethods
Adaptation
![Page 57: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/57.jpg)
an introduction to educational design research 55
ThefourphasesofInformedExploration,Enactment,LocalEvaluationandBroadEvaluationpresentedintheILDFencompassaprocessmodelforconductingdesignresearchbasedonseveralyearsofattemptstoincorporateprogressivelymorerigorous,research-basedcycleswithinatechnology-basedinstructionaldesigneffort.Thistypeofeffortisdifferentthantraditionalinstructionaldesignastheiterativecyclesareessentiallymicro-cyclesofresearch(morecomprehensivequalitativeandquantitativeresearcheffortsthanformativeevaluationcycles)conductedtolearnmorethanhowtoimprovethetechnologysystem,althoughthestudiesmayalsoresultinthatoutcome.ThereferencetolearningwithintheILDFistoplaceemphasisonthelearningthatcanresultinthecontextandactivityofdesign.Forexample,asresearchersorinstructionaldesignerswemaygenerateinformationabouttheteachingandlearningprocess,participants,context,andculturethatisoftennotattendedto,discardedandcapturedinarigorousmannerforotherstolearnfromandcapitalizeon.Whetherourdesignactivityinvolvesclassroom-basedinterventions,technologyorsomecombinationofboth,theinterconnecteddesignresearchcyclescangenerateknowledgeaboutdesignprinciplesbutalsoproviderichinformationonaspectsoflearning,cognition,expertandnoviceperspectives,aswellasstakeholderpositionstodirectdesignanddesigndecision-making.Thecoreissueathandisthattherich,complex,designprocessmayoffermultipleopportunitiestogenerateresearch-basedknowledgehowever,muchofitislostandnotdocumentedinthecreativedesignprocess.Ourchallenge,asdesignresearchers,istotrytosystematicallygather,analyze,reportandcodifythisinformationinarigorousmannerthatstrivestowardsometypeoflogical,argumentativegrammarworthyofstringentresearchprocesses(Kelly,2006).
Connectedcyclesofresearchcyclesanddesignprocessesresultinimproveddecision-makingbasedondata-drivenresultsfordesign,developmentandresearchpurposes.Thoughclearlyinterventionistandprimarilyformativeinnature,theILDFprocessstandsapartfromtraditionalinstructionaldesignandresearchefforts.Throughoutthemultiplephasesandcyclesofintegratedresearchanddesignprocessesvaluableknowledgeinthecontextofuseisgenerated.Weneedtominewhatislearnedaboutimportantfactorsrelatedtolearning,context,culture,andtechnologywithinthedesignprocess(notseparatefromitinacontrolledsettingasevidencedintraditionalresearch).Ifdesignresearcherscanarticulateanintegrativeresearchanddesignprocess,itmayhavethepotentialtosignificantlyimproveourunderstandingofteaching,learningandtrainingin-situ.Themultiplemacroandmicro-cyclesofdatacollection,analysisandmostimportantly,results-drivendesigndecision-makingiswhatsetsdesignresearchapartfromtraditionalformativeevaluationininstructionaldesignwhichisoftenconductedinaverylimitedmannerorasinglecycleofdata-gatheringandanalysis.
![Page 58: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/58.jpg)
an introduction to educational design research56
Tessmer(1993)referstoformativeevaluationasa“judgmentofthestrengthsandweakenessesofinstructioninitsdevelopingstages,forpurposesofrevisingtheinstructiontoimproveitseffectivenessandappeal(p.11)”.Althoughmultiplemethodsmaybeusedincludingexpertreview,one-to-oneevaluation,smallgroupandfieldtesting,formativeevaluationcyclesintraditionalinstructionalsystemsdesignmaynotalwaysemployresearchmethodsthatarespecifictoparticularphasesofanintegrativeandconnecteddesignresearchcycle.Formativeevaluation,despiteitsmostrigorousandcomprehensiveapplicationdoesnotprogressivelygenerateknowledgeaboutcognition,contextandcultureofusebutprovidesalimitedfocusonaparticulartechnologysystemofinstructionandjudgesitseffectiveness,appealandefficiency.Incontrast,designresearchcyclesarebasedonathorough,systematicprocessintegratedmultipledesignandresearchprocessestoprogressivelyimproveunderstandingaboutlearners,learning,context,orcultureaswellasiterativelyimproveanintervention.Therefore,formativeevaluationmethodsaresubsumedasoneselectedmethodinwhatcouldbedescribedasa“meta-methodological”orinvolvingmultipleresearchmethodsacrossthedesignresearchprocess.Whatiscriticalindesignresearchisthetheoreticalyieldoftheefforttobeviewedasimportantastheimprovementoftheintervention(seePlompchapter1).
Designresearchcyclesaredynamicandintegratemultipleexploratory,constructiveand/orempiricalresearchmethodsaswellasmultipledesign/developmenttechniques(seeFigure1).Exploratoryresearchmethodsstructureandidentifynewproblemssuchasfeasibilitytesting,benchmarkingandqualitativeresearchapproaches.Constructiveresearchdevelopssolutionstoproblemsandmayincludetestingofaconstructortheoryagainstapredefinedcriteriaandmay,forexample,includeformativeevaluationtestingofaninstructionaltechnologysystem.Incontrast,empiricalresearchteststhefeasibilityofasolutionusingempiricalordirectorindirectobservationorevidenceinthetraditionofthescientificmethod.Designresearchmayemployallthreeformsofresearchmethodsaswellasincorporateformativeevaluationmethodsatdifferentphasesintheprocess.However,traditional,formativeevaluationperspectiveswhileofferingvaluableiterativeprocesses,donotinisolation,addressthecomplexityinherentineducationalpractice.Mosteducationalresearchprojectsadvocateonlyonecycleofqualitativeorquantitativeempiricaltestingatafixedpointintimeforagiveninstructionalinterventionforthesolepurposeofgeneratingknowledge.Incontrast,designresearchattemptstoprogressivelyanddynamicallygenerate(exploratoryresearch),improve(constructiveresearch)andlearnabout(empiricalresearch)aparticularphenomenonfrominterconnectedresearchanddesigncycles.
Inresponsetothischallenge,theILDFmodelattemptstoprovideacomprehensiveyetdynamicandflexibleguidingframeworkthatpositionsmultiple,microandmacrodesign
![Page 59: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/59.jpg)
an introduction to educational design research 57
researchcyclesasprimarilysocially-constructed,contextualizedprocessofproducingthemosteducationallyeffectiveproductthathasthebestchancetobeusedintheclassroomwhilealsogeneratingknowledgeaboutteachingandlearningwithintheactivityofdesign.Themodelorframeworkattemptstomovepastisolated,individualeffortsofeducationalresearchbyclearlyarticulatingalogically-orderedstructuralframethatconsidersthefullspectrumofresearchmethodologyinadvancingtowardsystemicimpactineducationandmaybeappliedinavarietyofcontexts.Collins(1990;1993)advocatesforasimilarovert,systematicmethodologyforconductingdesignexperimentsandstates:
“When designing a learning environment, whether computer based or not, there are a multitude of design decisions that must be made. Many of these
design decisions are made unconsciously without any articulated view of the issues being addressed or the tradeoffs involved. It would be better if these
design decisions were consciously considered, rather than unconsciously made (1993, p.1).”
TheILDFprocesspresentsonesteptowardasystematicframeworkforthearticulationanddocumentationofcommonphasesandcomplementarystagesbasedonmultipledesignandresearchprocessespromotingmoreconsciousdesignresearch(Collins,1990;1999).Althoughtherearethousandsofdecisionsmadeinadesignresearchcontext,themajorconjectures,learningtargets,taskanalysis,designprinciplesandevaluationorresearchdecision-makingresultingfromexploratory,constructiveand/orempiricalresearchcyclesmaybeuncoveredbyexaminingarichcasestudyaspresentedhereentitledtheLiteracyAccessOnline(LAO)project.TheILDFispresentedhereasastartingpointforresearcherstoconsideraswiththegoalofelicitingquestions,suggestions,limitationsandcriteriathatmayneedtobeconsideredasresearchersstrugglewiththeimplicationsofthisemergingformofeducationalresearch.Inthischapter,IbrieflydescribetheprogressionoftheLAOdesign-basedresearchstudythatencompassedfouryearsofeffortandillustratestheapplicationoftheILDF.TheLAOcasestudyexampleisdescribedaccordingtobroadphasesincluding1)theinformedexplorationphase;2)theenactmentphase;3)thelocalimpactphase;and4)thebroadimpactphaseaswellasthemultiplepotentialappliedandempiricalresearchprocessesthatalignwitheachphase(seeFigure1).ItishopedthattheLAOexamplewillprovideenoughdetailtopotentiallyimproveunderstandingofconductingcyclesofdesignresearchrelatedtoatechnology-basededucationalintervention.
![Page 60: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/60.jpg)
an introduction to educational design research58
LiteracyAccessOnline - An integrative learning design study
TheLiteracyAccessOnline(LAO)project1providesanexampleofanintegrativelearningdesignstudybasedontheILDFspecificallyillustratingtheintersectionandsystematicexpressionofmultipledesignandresearchmethods.LiteracyAccessOnlineisanefforttoutilizeWeb-basedtechnologytoprovidesupportforteachers,tutors,andparents(literacyfacilitators)inaddressingliteracygoalsforallchildrenwithaparticularfocusonthosewithdisabilities.Afterfouryearsofdesignresearchanddevelopment,LAO(http://literacyaccessonline.com)nowprovidesatechnology-basedlearningenvironmentthatpromotestheuseofspecificliteracystrategiesfortheimprovementoftutoringandreadingperformanceasthechildandliteracyfacilitatorcollaborativelyengageintheprocessofreadingonline.
TheInformedExplorationPhaseTheexploratoryresearchobjectivesoftheLAOintegrativelearningdesignstudyweretwo-fold;1) toinvestigatethenatureandeffectivenessofaconsistenttechnology-based,
collaborativeliteracyenvironmentaswellas;2) togenerateknowledgeabouthowliteracyfacilitatorsandchildrenunderstandand
employreadingsupportstrategies.Theseobjectiveswereoriginallyconceivedasresearch/evaluationquestionsandevolvedfromanextended,progressiveinvestigationintotheprovisionofliteracysupportforfacilitatorsandchildren.This“meta-methodological”designresearchprocessconsistingofmultipleresearchmethods(e.g.survey,focusgroups,interviews,expertreviews,etc)resultedinclearlyarticulatedlearningtargets,taskanalysesoflearningobjectives,theoreticalmodelembeddedinatechnologysystemdesignandcongruentresearch/evaluationquestionsthatdrovemorerigorousqualitativetestingoftheinterventionwhoseresultscontributedtotheoryofliteracysupportforchildrenwithdisabilitiesfurtherelaboratedinsectionsbelow.Tobeginexploratoryresearchcycles,initialexplorationsintotargetaudienceandstakeholderperceptions,relatedproductsandliteratureanddocumentationofthecomplexnatureofsupportingliteracyrevealedmanyplausiblepathsfordesignresearch.TheinterdisciplinaryresearchteaminvolvedintheLAOprojectwerechargedwithdeterminingtheresearchdirectionandconsistedofeducationalresearchers,teachers,graduatestudents,contentexpertsinliteracy,specialeducationandassistivetechnologyaswellasparentsinvolvedinanadvocacygroupforchildrenwithdisabilities.Thebroaddesignresearch
1) TheLiteracyAccessOnline(LAO)projectissupportedbytheOfficeofSpecialEducationProgramsintheDepartmentofEducationSteppingstonesofTechnologyInnovationforStudentswithDisabilitiesGrantCFDA84.327A
![Page 61: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/61.jpg)
an introduction to educational design research 59
focusevolvedfromtheteam’sperceivedlackofsupportforchildrenwhowerestrugglingwiththeliteracyprocess,basedondirectobservationsofthisprobleminbothclassroomandhomeenvironmentswhichthenmanifesteditselfintoseveralindividualbutconnectedresearchstudies.Forexample,weconductedmultipleinterviewswithparentswithchildrenwithdisabilitieswhowerestrugglingwiththereadingprocess.Wealsoinvitedseveralparentstoparticipateonourdesignresearchteam.Theteams’analysisoftheinterviewdataaswellasthedesignresearchteamdiscussionsrevealedourinitialapproach.Wehadinitiallydecidedtodesignatutorial-basedinterventiononlyforthechild’sbenefit,however,acommentinateammeetingdramaticallychangedourdesigndirection.InlinewithCollins’notionofconsciousconsiderationsofdesignasdemonstratingcoreunderlyingdesigndecision-making,oneparentmember/stakeholderontheteamstatedthatsheprimarilyreadinconjunctionwith(notto)hersonandwantedtodosoonlinebutwithadditionalsupportofhigherlevelreadingstrategies.Basedonthatinputandfollow-upmicro-cyclesofinterviewdatacollectionandanalysistoconfirmationtheviabilityofthisdesignapproachwithparents,weconductedaseriesofinterviewsandsurveystodeterminethefeasibilityofthisdesigndirection.Theresultsofourinvestigationanddiscussionsevolvedintoanonlinecollaborativeperformancesupportsystemtosupportbothliteracyfacilitatorsandtheirchildrenintheliteracyprocessasthedetermineddesigndirection.Thiswasadesigndecisionbasedondataanalyzedfrommultipleinterviewsandsurveyprocedures.AlignedwithConfreyandLachance’s(2000)notionofdrawingkeyinferencesfromdissatisfactionwithcurrenteducationalpracticesanddirectexperienceswithchildren,initialtheoreticalconjecturesweredevelopedbasedontheanalyzeddatathatadvocatedforreading,writingandassistivetechnologysupportforchildrenwithorwithoutdisabilitiestoincreasetheirengagementandperformanceinliteracy.
Whiletheseinitialtheoreticalconjecturesprovidedacentralpremiseandbroaddirectionfordesignresearch,moreinformationwasneededtorefinetheseconjecturesresultinginacomprehensiveneedsanalysisandliteraturereviewthatprovidedafirmandcomplementarytheoreticalfoundationfortheintendeddesign.Extensiveexplorationintoappropriateliteracystrategies,tutorialprogramsandprocesses,surveysofexperts,teachersandparentsaswellasqualitativeobservationofchildrenandfacilitatorsengagedinaliteracyexperienceallinformedthisphaseoftheresearch.Thisprovidednotonlywell-defineddesigndirectionsbutalsoaddedtotheresearchliteratureregardingchildrenwithdisabilitiesandtheirparentsunderstandingofassistivetechnologyandliteracylearning(seeJeffs,Behrman&Bannan-Ritland,2006).Manypotentialdesignresearchdirectionswereconsideredbasedontheinitialconjectures,however,datadrawnfromconductedinterviews,directexperiencewithpotentialresearchparticipantsandliteraturereviewconvergedandpointedtheteaminaparticulardirection.
![Page 62: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/62.jpg)
an introduction to educational design research60
Aprominentthemethatemergedacrossinitialinterviews,surveysandobservationswithexperts,parents,teachersandchildrenrevealedthatliteracyfacilitatorshadacrucialroleinprovidingsupportforchildrenstrugglingtogainliteracyskillsandthequestionremainedhowtobestsupportthisrole.Thesefindingsandrelatedliteratureprovidedinsightforinformedtheoryandimprovedconjecturesbasedontheaforementionedqualitativeinterviewsandliteraturereviews.Resultsfromdatacollectionandliteraturereviewmethodsintheinformedexplorationphaseindicatedthat:1) childrencan,butoftendonotuseeffectivemetacognitivereadingstrategies;2) explicitlyteachingthesestrategiescangreatlyenhancechildren’scomprehensionof
text;3) teachers(aswellasotherliteracyfacilitators)needtobetrainedinhowtoprovide
cognitivestructurefortheirstudentssothatchildrencanlearntoguidetheirowngenerativeprocessesinreading;and
4) one-to-onetutoringisoneofthemosteffectiveformsofinstructionforimprovingreadingachievementbutincreasedsuccessoftendependsupontheskillofthetutororfacilitatorandtheestablishmentofconsistentrolesandexpectations(Wittrock,1998;Wasik,1998).
Thisexplorationintotheliteratureandperspectivesofthoseinvolvedintheseissuesgreatlyrefinedourinitialtheoreticalconjecturesandresultedinadramaticchangeofourintendeddesigndirectionforthisresearchfromadidactic,tutorial,child-focusedinterventiontoacollaborative,story-basedreadingexperienceprovidingembeddedmetacognitivestrategysupportforboththeliteracyfacilitatorandthechild’suse.Therationaleforthisresearchdirectionwasdocumentedinacomprehensiveneedsanalysisthatdetailedthedatacollection,conclusionsandrelatedliteraturereview.
ThenextstageofourdesignresearchinvolvedtheanalysisanddescriptionoftherangeoflearnersandfacilitatorsthatwouldpotentiallyusetheLAOsystem.Directexperiencewith4th-8thgradechildrenwithorwithoutdisabilities,teachers,tutors,andparentsprovideddatathatcharacterizedouraudience.Thesedescriptionsweredepictedasrolemodels(Constantine&Lockwood,1999)orpersonas(Cooper,1999)thatcomprisedabstractcompositeprofilesofaudiencecharacteristicsgleanedfromactualinterviewsandobservationsandprovidedafocalpointfordesign.RolemodelsorpersonasaresimilartoGraueandWalsh’s(1998)qualitativevignettesthatstrivetocapturethesubstanceofasetting,personoreventtocommunicateacentralthemeofqualitativedata,basedonmultipledirectobservationsandareemployedhereasalsoafocalpointfordesign.Exploringthenatureoftheidentifiededucationalproblem,relatedproductsandliteratureaswellascreatingandrefiningtheoreticalconjecturesanddescriptionsoftheaudienceprovidedaninformedperspectiveforgroundeddesignofalearningenvironmentbasedon
![Page 63: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/63.jpg)
an introduction to educational design research 61
articulatedtheory.Theseactivitiesresultedinspecificresearchartifactsincludinganeedsanalysisthatcontainedanextensiveliteraturereview,anarticulatedandcongruentdesignandresearchdirectionanddetailedaudienceanalysisbasedonqualitativeandquantitativedata.ThesedocumentswerehousedonaprojectWebsitethatprovidedacommunicationmechanismbetweenteammembersaswellasanarchiveofshareabledesignresearchprocesses,productsandevidentiarydata.
TheEnactmentPhaseTheembodimentoftheresultsofourinformedexplorationandtheoriesaboutprovidingliteracysupportforchildrenandliteracyfacilitatorsinausablelearningenvironmentwerecollaborativelyconstructedacrossseveralstagesandconstructiveresearchcyclesthatdevelopsolutionstoproblemsculminatinginaWeb-basedprototype.TheinitialdesignoftheLAOlearningenvironmentresulteddirectlyfromthedesignimplicationsarticulatedinthepreviousphaseofexploratoryresearch,analysesandreview.Theseimplicationsweretranslatedintoanarticulatedprototypeinitiallydevelopedbybuildinganabstract,paper-basedmodelofthesystemforresearcherandteacherinputaccordingtoproceduresadaptedfromusage-centereddesignprocessespreviouslymentionedasrolemodels(Constantine&Lockwood,1999).Rolemodelsareatechniquetocharacterizeprimaryandsecondarytargetaudiencesforthepurposesofdesign.Forexample,wecreatedrolemodelsandpersonasforchildrenwithlearningdisabilities(suchasattentiondeficitdisorder)basedonourdirectexperiencewithachildwhowasstrugglinginthereadingprocessandhismotherwhodidnothaveanyknowledgeofadvancedreadingsupportstrategies.Thesetechniquesarebasedonreal-worldexperienceswithrepresentativesofthetargetaudiencesyourinterventionorsystemisbeingdesignedforbutevolveintoaarchetypalcompositeoftheattributesofmanyindividuals.Therefore,rolemodelsandpersona’sbecomeaqualitativeprofiletocontinuallytargetdesigneffortstomaintaintheaudience(s)oruser(s)perspectives.
Abstractorlow-fidelitymodeling/prototypingoftheinstantiatedorenacteddesignprovidedopportunitiesforinputandco-constructionofLAOwithseveralaudiencememberspriortothemoretime-intensivecomputer-basedproductionofthelearningenvironment.WeutilizedConstantineandLockwood’s(seeforuse.com)proceduresofusage-centereddesignthatencompassedlow-fidelityrepresentationandorganizationofallthefeaturesofthedatabase-drivenWebsite.ForLAO,wedeliberatelyultimatelydesignedaWebdatabasesystemthatwouldpermitperformancesupportfortheparent-childdyadinprovidingmeta-cognitivepromptsforbothparticipantsbasedonresearch-basedreadingstrategiesthroughoutacollaborativeandgenerativeprocessofengagingwithtext.
![Page 64: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/64.jpg)
an introduction to educational design research62
Inthecontextofaconstructiveresearchapproachthatattemptstovalidateaparticularconstruct(e.g.theory,model,softwareorframework)againstidentifiedcriteriaorbenchmarks,theteamconductedseveraliterativecyclesofdata-gatheringandanalysisofexpertreviewsandtargetaudiencereviews.Theseprogressive,micro-cyclesofdatacollectionandanalysisresultedindata-drivencyclicalrevisionsofthearticulatedprototypewhichwerereflectedindetaileddesigndocumentationincludingtheproductionofflowcharts,technicalspecificationsandstoryboards.Thedesignresearchprocessofemployingmicro-cyclesofconstructiveresearchdatagatheringandanalysiselicitedfeedbackateachcycleanddesignrevisionsagreeduponbytheteamwhichresultedintheinitialcreationandthenprogressiveimprovementofaWeb-basedprototypevalidatedbydatacollectedinaconstructiveresearchapproach.Asateam,weconstructedspecificcriteriarelatedtousabilityofsystemandobservationsandvideoanalysisofactualuseofthesystembyliteracyfacilitatorsandchildren.ThespecificmethodsofdatacollectionemployedatthisstageincludeddesignerlogspostedontheprojectWebsite,expertpanelreviewsofthedesignanddocumentedreviewsofthedesignbycontentexperts,audiencemembersandtheresearchteam.
TheLocalImpactPhaseOnceaphysicalWeb-basedprototypewasinplace,theincorporationofformativeevaluationandqualitativemethodsinanempiricalmannercouldcommenceandbegantocharacterizetherich,highlyiterativenatureofthelocalimpactphaseasitprogressivelyinformed,revisedandrefinedourtheoreticalconstructsaswellastheWeb-basedinstructionaldesignapproachandredesignefforts.Thecomplexinteractionsbetweenfacilitatorsandchildrenthatcanoccurinmultiplesettingsformedtheseriesofmicro-cyclesinLAOexaminingthesespecificconstructsthatgroundedrelatedresearchquestions:1)parent-childdyadsinaninformalsettingwithextensiveinvolvementbyresearchers;2)parent-childdyadsinastructuredworkshopexperiencesupportedbyresearchersand;3)pre-service-teacherdyadsinafieldtrialprogressingtowardmorecloselymodelingauthenticconditionsexperiencedwiththeprototype.Whenafullyfunctioningprototypewasnotyetavailable,studiesattemptedtocloselymimicthetasksthatwouldbeembeddedinLAO.Thedatagatheringacrossthesethreestudiesincorporatedobservations,interviews,childandparentjournalentries,videotapeduseofsystemandpre-andpost-onlinesurveys(seeJeffs,Behrmann&Bannan-Ritland,2006).Thismulti-tiered,multi-methodevaluationschemegeneratedusefulknowledgeandsubsequentresultsfromeachstageofinquirywerethencycledintochangesofourtheoreticalconjectures,researchdesignaswellassystemdesign.Thisprocessrevealedinsightsintothecoredesignprinciples(vandenAkker,etal.,2006)thatmaysupportthecollaborativelearningandimplementationofmetacognitiveprocessesbyliteracyfacilitatorsandchildreninatechnology-basedenvironment.
![Page 65: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/65.jpg)
an introduction to educational design research 63
OurcoredesignprinciplesthatevolvedandwererefinedincludedthefollowingthatwhenengagedinacollaborativeliteracyprocessthatprovideshighlevelreadingstrategymetacognitivesupportinaWeb-basedcontext:1) parentliteracyfacilitatorscoulddevelopgreaterawarenessandskillinimplementing
readingactivitiesandidentifysupportsfortheirchildinastructuredsetting;2) childrenshowedimprovementinliteracyskillsusingtechnology-basedsupportwhen
participatinginaguidedworkshopenvironment;and3) pre-serviceteachersfeltthatthestrategiesandactivitiesembeddedintheLAO
environmentfacilitatedchildren’scomprehension,motivationandinterestwhenworkingwiththeminthisenvironment(seeJeffs,et.al.2006).
MorerigorousevaluationsareplannedforthefuturetosystematicallyincreasingnumberofparticipantsandvaryingcontextsfortheuseofLAOinschool,homeandtutoringenvironments.Thesestudiesinvolvedetailedtrackingofcomputer-basedactivitiesofthedyadsinschoolandhomesettings,assessmentoffacilitatorandchilduseofmetacognitivestrategiespriortousingLAOandpre-andpostcomprehensionmeasuresafterseveralweeksofusingthesystem.
Inaddition,aseriesofexpertreviews,usabilitytesting,one-to-to-one,smallgroup,andfieldtestingwereimplementedinprogressivelymoreauthenticsettings.Themoreintensivestudiesfocusedprimarilyonqualitativestudiesthatcharacterizedthetargetaudienceinteractionwiththeenactedtheoreticalmodelinrelationtothelearningtargets.Specifically,theteamwasinterestedinhowfacilitatorsandlearnersperceivedandinteractedwithWeb-basedsupportinthecollaborativeliteracyprocess(whichincludedbothreadingandwritingtasks).ApilotstudywasinitiallyconductedthatsimulatedsometaskswithinLAOandprovidedfeedbackontheemergingsitewithfivedyadsofmostlyparentfacilitatorsandonesiblingfacilitator.Methodsincludedcollectingdatathroughsemi-structuredinterviewsandobservationsofparent-childinteractionwiththeprototypeandcomplementaryassistivetechnologies(e.g.text-to-speech,etc.)thatpromotedinreadingandwritingactivities.
ThepreliminarystudyrevealedthatthechildrenweremotivatedtocompletereadingandwritingactivitiesontheWebandthatfacilitatorsdevelopedawarenessforimplementingreadingactivitiesinacollaborativeprocessbutdesiredadditionalsupportforchildren’sdisabilities.WhiletheWeb-basedactivitiesandsupportsforthereadingprocesswereusefulforprovidingmoreauthenticandself-initiatedreadingandwritingactivities,theresearchalsorevealedthatinteractionbetweenparentandchilddyadsduringtheseactivitiesoftencreatedtensionsthatwerenotpresentwhenchildrenwereworkingwithnon-familymembers.RevisionstothetheoreticalmodelandenacteddesignofLAObased
![Page 66: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/66.jpg)
an introduction to educational design research64
onthiscycleofevaluationincludedamongothers,behavioralpromptsdirectedtowardtheparent-childdyadtopotentiallyreleasetension(suchaspromptstotakeabreak,positivereinforcementtechniques,etc.)whenengagedincollaborativereadingandwritingtasksandadditionalreadingstrategysupportsandactivities.
Tofurtherinvestigatetheenactedtheoreticalmodel,afollow-upsmallgroupqualitativestudywasconductedwitheightparent/childdyadsthatrepresentedavarietyofskilllevelsanddisabilities(Jeffs,2000).Thespecificgoalsofthiscycleofresearchwastoidentifythecharacteristicsofparent/childdyadsworkingtogetherspecificallyinliteracyskilldevelopment,depicttheinteractionsofthedyadandinvestigatetheimpactofvariousformsoftechnology(Internet,EPSSandanyassistivetechnology)onattitudesoftheparticipants.Participantsincludedparentsandchildrenwithvariousdisabilitiesingrades4ththrough6thwhowerereadingatleasttwogradesbelowgradelevelandhadatendencytoavoidreadingandwritingtaskspriortoparticipationinthestudy.Thestudyrevealedthatparentsrecognizedtheimportanceofimmediatefeedbackandassistivetechnologyfeaturesintheprovidedtools.Otherresultsrevealedthatwiththesupportoftheirparents,childrencanselectappropriatetechnologiesandwithintegrateduseoftheInternetandassistivetechnologies,children’swritingsamplesimprovedinbothquantityandquality.SuggestedrevisionsfortheLAOprototypebasedontheseresultsincludedbuilt-inassistivetechnologyfeatures(insteadofmerelyreferencestooutsideresources)suchastext-to-speechcapabilitiesandreadingselectionsreflectingvaryingabilitiesandareasofinterest–featuresthatweresubsequentlyincorporatedintotheLAOdesign.
Ineachofthesecyclesofproblem-state,datacollection,analysisandsubsequentdesignmoveorformativeevaluationprocess,thetheoreticalmodelenactedwithintheLAOprototypeexpandedtoincorporatenewandrevisedelementsbasedontargeteddatacollectionandresearchresults.Atthispoint,traditionalresearchanddesignprocessessomewhatdivergeinthattheanalyzedresultsarenotanendinandofthemselves,butareusedfordata-drivendecisionmakingorproblemsolvingtobuilduponorrevisetheoreticalassumptionsandimprovedesign.Often,basedontestingresults,wewouldneedtothrowoutpreviousprototypefeaturesandtotallyredesign,reviseoraddnewfeatures.Theteam’sinformeddesignjudgmentandcollaborativesocialnegotiationwaskeytothisdecision-making.
Thelocalimpactphaseisatime-intensivephasewithmultiplecyclesthatstrivestoyieldausableandinternallyvalidintervention.Testingtheinterventioninprogressivelymorerealisticsettingsprovidesvaluableinformationtoinformtheoreticalassumptionsrelatedtothedesignbutalsotobegintoisolatevariablesthatmightbefurtherempiricallytested.IntheLAOresearchconductedtodate,theintegrationofreadingstrategyscaffoldsand
![Page 67: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/67.jpg)
an introduction to educational design research 65
assistivetechnologysupportsinthecollaborativeliteracyprocessbetweenfacilitatorsandchildrenwitharangeofdisabilitieswasidentifiedasonefactor,ofmany,thatseemtoholdpromiseforimprovingliteracyskills.Conductingadditionalresearchtofurtherinvestigatethecollaborativeprocesspromotedbythetechnologicalenvironmentaswellasisolatingtheeffectsofthemultiplereadingsupportsandassistivetechnologiesaffordedbytheprototyperemainsanimportantobjectiveinthisresearch.
AlthoughthefundingcycleforLAOhasceased,inordertoprogressfromlocaleffectstomoreexternallygeneralizableeffects,additionalcyclesoftestingareneededtoisolateandtestparticularvariablesusingmultiplesites,diverseparticipantsandsettingsprogressivelylimitingtheresearcher-participantinteraction.Basedonavailablefunding,fieldtestsortrialsareplannedforLAOtocollectsignificantamountsofquantitativeandqualitativedatafromseveralsitesandover50participantdyadsusingselectedmeasurements,onlinesurveysandinterviewsincludingparentsandchildreninhomeschoolenvironments,pre-serviceteachersandin-serviceteachersthatcouldrepresentotherliteracyfacilitatorsinseveralgeographicallocationsinteractingwithchildrenwitharangeofdisabilities.Thisdatawouldprovideadditionalevidencefortheeffectivenessofenactedtheoreticalassumptionsforthecollaborativereadingandliteracyprocessaswellasprovideevidencefortheeffectivenessoftheprototypeatitshighestfidelityinfullcontextoftheintendeduse.
TheBroadEvaluationPhaseThelastphaseofthisdesign-basedresearcheffortinvolvesdisseminatingLAOintothebroadeducationalsystem.AlthoughtheLAOresearchhasnotyetfullyprogressedthroughthisstage,initialexplorationsinthisareahaveyieldedsomeuniqueinsightsintothedisseminationprocess.However,thereadershouldnotethatthedisseminationprocesscanencompassanentireresearcheffortinitself.Forexample,Fishman(2006)hasappliedadesignresearchframeworkrelatedtothesustainabilityoftechnology-basedcurriculuminterventionswithinanentireschooldistrictorsystem.LAO,asaWeb-basedlearningenvironment,affordstheopportunitytopublishcurrentworkingprototypesonlineforopenuseandinputthathasresultedinanearlyanduniquediffusionandadoptionprocessbegunpriortothecompletionofafullyfunctioningsystem.Whilestillindevelopment,wehavetrackedover100potentialadoptersthathavediscoveredandexploredtheLAOsite.Theprofilinganddata-basecapabilitiesofthesitepermittrackingandanalysisofthisinformationthathasprovideddetailedinformationonpotentialadoptersofthesystemprovidingsignificantinsightandimpactonsourcesforourlaterdiffusionefforts.Weplantoincorporatemoresophisticatedcomputer-baseddatacollectionandanalysistechniquessuchasdatamining(Tsantis&Castellani,2001)thatmayyieldevenmoreinsightsintoearlyadopters’behaviors,profilesanduseofthisnewtool.We
![Page 68: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/68.jpg)
an introduction to educational design research66
havejustbeguntopublishourresultsofthedesignbasedresearchconductedrelatedtoLAOintraditionalacademicjournalsandnon-traditionalWebpublishingthatprovideavenuesforadditionalformsofreviewandevaluation.TheresultsofourinitialstudieshavepromptednewresearchdirectionssuchasexploringtheinteractionofanonlinecommunityforparentsofchildrenwithdisabilitiesincorporatedintheLAOenvironment.Giventheiterativenatureofthistypeofresearch,itishighlylikelythatdeterminingtheconsequencesoftheLAOdesignresearcheffortwillyieldnewtheoreticalandappliedquestionsthatwillprompttheentireprocessonceagain.
Theoretical yield of literacy access online design research study
GiventhedesignresearchprocessbasedontheIntegrativeDesignLearningFrameworkdescribedabove,whatdidwelearn?ThecharacteristicsofaninterventionorasvandenAkker,et.al.(2006)describethe“designprinciples”areanimportantyieldofdesignresearch.IntheLAOproject,thesedesignprinciplesincludedprovidingmetacognitivereadingstrategysupportwhileaparent,teacherortutorisengagedinthecollaborativereadingprocesswiththechilddeliveredthroughacomprehensiveWeb-basedperformancesupportsystem.
Designresearchisoftenemployedtobegintogeneratetheory(Design-basedResearchCollective,2003).WithLAO,therewerenoliteraturesources,theoreticalprinciplesorresearchstudiesdirectlyapplicabletoaWeb-supportedcollaborativereadingprocesssotheteamintegratedinsightsfromtutoring,readingstrategiesandreal-timeperformancesupport.Zaritskyet.al.(2003)speaktogoing“…beyondsimpledevelopmentofaninterventionandbeyondstandardcognitiveanalysesallowingtheoryandmodelingthataccountsforthecontent,thecognitionandtheenactmentbyrealpeopleinrealandrichcontextswithreallimitsonresources(p.11).TheLAOdesignresearchteamwentbeyondtraditionaldevelopmentwithintensivecyclesofinterviews,surveys,observationalstudiesaswellasdeepinvestigationoftheone-on-onetutoringandreadingstrategiesliteraturetobuildanewtheoreticalmodelofreal-timemetacognitivereadingstrategyandassistivetechnologysupportforboththeliteracyfacilitatorandthechildwithdisabilities.Muchoftheseinsightswereanintegrationofdataanalyses,directexperiencewithtargetaudiencemembersandagroundedliteratureinreadingprocesses,tutoringandcollaborativeperformancesupport.Thedesignresearchprocesswasconductedsystematicallyto:1) uncovertheinitialconjecturesabouthowlearningmightoccurinthistypeofsetting;2) statedlearningtargets,taskanalyses(inthiscasebasedonActivityTheory);3) thedesignedinterventionwhichembodiesthecoredesignprinciples(metacognitive
readingstrategysupportinacollaborativeperformancesupportcontext);
![Page 69: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/69.jpg)
an introduction to educational design research 67
4) localimpactorevaluationquestionsthatdrovethemoreintensiveresearchcycles(seeFigure2).
Thisprogressiondemonstratesanalignmentorcongruencyfrominitialconjecturesthroughlocalimpactorevaluationquestionsthatevolvedduringthedesignresearchstudy.ThespecifictheoreticalinsightsthatweretestedandrevealedbasedonthisprocessareincludedinFigure2.ThemultiplephasesoftheIDLFprocessuncoveredmanyinformalandformaltheoreticalinsightsbasedonmacroandmicrodatacollectionandanalysiscyclesconductedwithintheprocessofdesignthatcanbetypicallyoverlookedinthetraditionalinstructionaldesignprocess.Forexample,extendingbeyondatraditionallearneranalyses,weconductedmultiplecyclesofsurveys,interviews,andobservationsoftargetaudiencememberinteractionthatrevealedtheoreticalinsightsthatgobeyondjustthedesignoftheintervention.Ourstudiesrevealedthatparentshavelittleformalknowledgeanduseofgoodreadingstrategieswhenengagedwiththeirchildinthereadingprocess.ThisinsightparlayedintothedesignprinciplesofLAObutalsostandapartfromitasafindingthatmaycontributetotheliteratureinthereadingfield.Byformalizingandextendingthemethodsoftraditionalinstructionaldesigntopromoterichcyclesofdatacollectionthatthencaninformourknowledgeofparticularaudiences,learningcontextsandprocesses–separatebutconnectedtothedesignofaparticularintervention,wecanbegintoprogresstowardgeneratingknowledgeandusefultheoreticalinsightsthataretypicallyoverlookedindesign.Thisbecomesaninformation-lossprocessoflearningaboutlearners,contexts,andprocesseswithintheactofdesignthatdesignresearchcanrecapture,whichreferstothenotionthatinthecontextofbothisolateddesignandresearchefforts,wedonottakeadvantageofformalizingmuchofourlearninginanexploratory,confirmatoryorempiricalmanner(Bannan-Ritland&Baek,2008).
![Page 70: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/70.jpg)
an introduction to educational design research68
Figure 2: Initial Progressive Formulation from Conjectures to Local Evaluation Questions in LAO
ConjecturesLearning Targets
Task Analysis of Learning Task,
(based on Activity Theory)
Designed
InterventionLocal Evaluation
Questions
Provinding a consistent environm
ent and reading support strategies for literacy facilitators
and children would collaboratively
engage the dyad in higher level literacy processes
1) Literacy facilitators w
ill acknowledge the im
portance of and dem
onstrate their ability to im
plement reading strategies w
hen provided technology-based support
in a collaborative reading session w
ith a child
2) Children with of w
ithout disabilities w
ill demonstrate their
abilities to access information,
activities and assisitive technology support related to reading as w
ell as interact w
ith literacy facilitators in a collaborative reading session
3) The facilitator-child dyad will
be able to explore and select appropriate assistive technology intergrated w
ith Internet-based supports that can facilitate
performance in reading and w
riting
4) Children, regardless of disability w
ill be able to capitalize on technology-based supports and a collaborative process to im
prove their literacy skills
Subject: facilitator-child dyad
Object: LAO
system
Tools: literacy strategies, assisitive technologies
Division of Labor
Comm
unity: dyads in school, hom
e and tutoring contexts
Theoretical Model Em
bedded in Electronic Perform
ance Support System
(EPSS)
How
do facilitators and learners perceive and interact w
ith W
eb-based support in the collaborative literacy process?
What are the specific
characteristics of parent/child dyads w
orking toward literacy
skill development?
What im
pact does various forms
of technology support have on the attitudes of facilitators and children w
hen engaged in the collaborative literacy process?
How
should the interaction betw
een literacy facilitators and children w
hen engaged in the collaborative literacy process
be depicted?
![Page 71: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/71.jpg)
an introduction to educational design research 69
LimitationswillcertainlyalsoexistfortheILDFframework,astheknowledgegeneratedisonlyasgoodastherigoroftheresearchmethodsemployed.Integratingbasicqualitativeandquantitativeresearchcyclestoinformdesignatparticularpointsandgeneratebothdesignprinciplesbutalsoknowledgeaboutlearners,learningandlearningcontextsistheultimategoal.Limitationsmayexistintime,qualityofinformationuncoveredindatacyclesthatmayimpactdesign,smallNtoprovidemostlyqualitativeinsightsinitiallyandthefailureinherentinthegenerationoftheoryinthediscoveryresearchprocess.However,itisthroughapplicationindifferentdesignresearchcontextsthatmoreformalizedprocesseswillbegintobeunveiled.TheIDLFandLAOexampleareonecaseofafewcurrentlyfordesignresearcherstouncoverthelogicandwarrantsofthisnewformofresearch(Kelly,2006).Therearemanychallengesthatremainbutcapitalizingonthedesignprocesstogenerateresearch-baseddata-driveninsightsisaworthygoal,indeed.
Conclusion
ThischapterhaspresentedabriefexampleandintroductiontotheILDFframeworkthatcomprisesameta-methodologicalviewofthedesignresearchprocessinanattempttoelucidatecommonphasesandstagesinthisspecificresearchmethodology.Theframeworkispresentedtobegintoestablishcommonterminologyandprocessesthatcanpromoteconsciousdesignresearch.Mostimportantly,theILDFframeworkisanattempttoprovidearoadmapforfuturedesignresearcherstoinvestigate,articulate,documentandinformeducationalpractice.
* IamgreatlyindebtedtoDr.AnthonyE.Kellywhoseinsightsandfeedbackonthischapterwereinvaluableinextendingmythinkinginthisarea.MyappreciationalsogoestoDr.TjeerdPlomp,Dr.NienkeNieveenandDr.JanvandenAkker,esteemedcolleaguesandreviewersofthismanuscriptfortheirsuggestionsforrevision.
References
Akker,J.vanden,Gravemeijer,K,McKenney,S.&Nieveen,N.(Eds).(2006).Educational design research.London:Routledge.
Bannan-Ritland,B.&Baek,J.(2008).Investigatingtheactofdesignindesignresearch:Theroadtaken.InA.E.Kelly,R.A.Lesh,&J.Baek(Eds.),Handbook of design research methods in education: Innovations in science, technology, mathematics and engineering.Mahway,NJ:Taylor&Francis.
Cooper,A.(1999).The inmates are running the asylum: Why high-tech products drive us crazy and how to restore the sanity.Indianapolis,IN:SAMS.
![Page 72: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/72.jpg)
an introduction to educational design research70
Collins,A.(1990).Toward a design science of education. Technical Report Issue No. 1 Center for Technology in Education.NewYork,NY.ResourcesinEducation(ERIC)ED326179.
Collins,A.(1993).Design issues for learning environments.TechnicalReportNo.27CenterforTechnologyinEducation,NewYork,NY.ResourcesinEducation(ERIC)EDED357733.
Collins,A.(1999).Thechanginginfrastructureofeducationresearch.InE.C.Lagemann&L.S.Shulman(Eds.),Issues in education research: Problems and possibilities (pp.289-298).SanFrancisco:Jossey-Bass.
Confrey,J.&Lachance,A.(2000).Transformativeteachingexperimentsthroughconjecture-drivenresearchdesign.InA.E.Kelly&R.ALesh(Eds.),Handbook of research design in mathematics and science education (pp.231-265).Mahway,NJ:LawrenceErlbaumAssociates.
Constantine,L.L.&Lockwood,L.A.(1999)Software for use: A practical guide to the models and methods of usage-centered design.Addison,WesleyLongman,Inc.
TheDesign-BasedResearchCollective(2003).Design-basedresearch:Anemergingparadigmforeducationalinquiry.Educational Researcher 32(1) 21-24.
Fishman,B.,Marx,R.,Blumenfeld,P.,Krajcik,J.S.&Soloway,E.,(2004).Creatingaframeworkforresearchonsystemictechnologyinnovations.Journal of the Learning Sciences, 13(1),43-76.
Graue,M.E.&Walsh,D.(1998).Studying children in context.ThousandOaks,CA:SageJeffs,T.(2000).Characteristics, interactions, and attitudes of parent/child dyads and their use
of assistive technology in literacy experience on the internet.Unpublisheddissertation,GeorgeMasonUniversity,Fairfax,VA.
Jeffs,T.,Behrmann,M.&Bannan-Ritland,B.(2006).Asssistivetechnologyandliteracylearning:Reflectionsofparentsandchildren.Journal of Special Education Technology 21(1), 37-44.
Kelly,A.E.,Lesh,R.A.&Baek,J.Y.(Eds).(2008).Handbook of Design Research Methods in Education Innovations in Science, Technology, Engineering, and Mathematics Learning and Teaching.NewYork:LawrenceErlbaumAssociates.
Kelly,A.E.(2006).Qualitycriteriafordesignresearch.In:J.J.H.vandenAkker,K.Gravemeijer,S.McKenney,&N.Nieveen(Eds.).Educational design Research.London:Routledge.
Kelly,A.E.,Baek,J.,Lesh,R.,&Bannan-Ritland,B.(2008).Enablinginnovationsineducationandsystematizingtheirimpact.InA.E.Kelly,R.Lesh,andJ.Baek(Eds.),Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching.NewYork:Routledge.
LiteracyAccessOnlinePhaseFourProjectWebSite(2000).RetrievedAugust30,2002,fromGeorgeMasonUniversity,ImmersionProgramWebSite:http://chd.gse.gmu.edu/immersion/lao/deliver/needs.htm
Rogers,E.M.(2003).Diffusion of innovations.NewYork:TheFreePress.Sabelli,N.(personalcommunication,May15,2002).
![Page 73: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/73.jpg)
an introduction to educational design research 71
Tessmer,M.(1993).Planning and conducting formative evaluations.London:KoganPage.Tsantis,L.&Castellani,J.(2001).Enhancinglearningenvironmentsthroughsolutionbased
dataminingtools:Forecastingforself-perpetuatingsystemicreform. Journal of Special Education Technology, 16(4).
Wasik,B.A.(1998).Usingvolunteersasreadingtutors:Guidelinesforsuccessfulpractices.Reading Teacher,51(7),562-70.
Wittrock,M.C.(1998).Cognitionandsubjectmatterlearning.InN.M.LambertandB.L.McCombs(Eds.),How Students Learn(pp.143-152).Washington,DC:AmericanPsychologicalAssociation.
Zaritksy,R.,Kelly,A.E.,Flowers,W.,Rogers,E.,&O’Neill,P.(2003).Clinicaldesignsciences:Aviewfromsisterdesignefforts.Educational Researcher, 32(1), 32-34.
![Page 74: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/74.jpg)
![Page 75: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/75.jpg)
an introduction to educational design research 73
4. WhenisDesignResearchAppropriate Anthony E. Kelly
Introduction
Designresearchhasbeendescribedindetailinmanypublications,mostrecentlybytheDutch(VandenAkker,Gravemeijer,McKenney&Nieveen,2006,withe.g.,Kelly,2006),andtheKelly,LeshandBaek(2008)collectionofpapers(e.g.,Kelly,Lesh,Baek&Bannan-Ritland,2008;Middleton,Gorard,Taylor&Bannon-Ritland,2008).Plomp(chapter1ofthisbook)alsoprovidesanoverview.Forthatreason,Iwillnotreiteratethedescriptionhere.Rather,Iwillassumethatthereaderisfamiliarwiththesesourcesandthespecialissuesofjournals(e.g.,Barab&Squire,2004;Kelly,2003,2004)thathaveappeared.
Instead,Iwishtoplacedesignresearchwithintheframeofalargercontextforresearchoninterventions.Inherseminalpiece,Bannan-Ritland(2003)describedaportfolioofresearchactivitiesusingthefollowingcategories:
• InformedExploration• Enactment• Evaluation:LocalImpact - Quasi-experimentaldesigns - Randomizedtrials - HierarchicalLinearModeling• Evaluation:BroaderImpact - Implementationinnewcontexts(DesignandResearch) - ImplementationatScale - ScalingupDesignandResearch - Web-enabledprotodiffusion - DiffusionofInnovations(Rogers)• Adoption,adaptation,acceptance,rejection
Ofcourse,thislargerframeworkcallsformanydifferentresearchmethods.Inhispaper(Plomp,chapter1),brieflycapturesthefunctionsofresearchmethods:
• survey:todescribe,tocompare,toevaluate• case studies: todescribe,tocompare,toexplain• experiments: toexplain,tocompare• action research:todesign/developasolutiontoapracticalproblem
![Page 76: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/76.jpg)
an introduction to educational design research74
• ethnography:todescribe,toexplain• correlational research:todescribe,tocompare• evaluation research: todeterminetheeffectivenessofaprogram
HethenprovidesexamplesrelatedtotheChinesecontext:
1. to describe:e.g.whatistheachievementofChinesegrade8pupilsinmathematics;whatbarrierstostudentsexperienceinthelearningofmathematicalmodelling
2. to compare:e.g.whatarethedifferencesandsimilaritiesbetweentheChineseandtheNetherlandscurriculumforprimaryeducation;whatistheachievementinmathematicsofChinesegrade8pupilsascomparedtothatincertainothercountries
3. to evaluate:e.g.howwelldoesaprogramfunctionintermsofcompetencesofgraduates;whatarethestrengthsandweaknessesofacertainapproach;etc
4. to explain or to predict:e.g.whatarethecausesofpoorperformanceinmathematics(i.e.insearchofa‘theory’predictingaphenomenonwhencertainconditionsorcharacteristicsaremet)
5. to design and develop:e.g.whatarethecharacteristicsofaneffectiveteachingandlearningstrategyaimedatacquiringcertainlearningoutcomes;howcanweimprovethemotivationoflearners.
BothBannan-RitlandandPlompprovideabroadercontextforresearch.Withinthislargerframework,wemayask,therefore:Whenisdesignresearchappropriate?Wemayapproachananswerbyasking,first,whenisdesignresearchinappropriate?
When is design research inappropriate?
Areviewofthemanypublishedexamplesofdesignresearch(e.g.,Kelly,Lesh&Baek,2008)demonstratetheheavyinvestmentoftimeandresourcesnecessarytomakeprogressinthefaceofsometimesdauntingcircumstances.Designresearchrequiresinvestmentofsubstantialresourcesatmanylevels:schooldistrictadministrators,teachers,students,andthedesignresearchteam(whichmayincludeeducationresearchers,softwaredevelopers,curriculumspecialists,andsoforth).Thus,designresearchisinappropriateiftheeducationalproblemisfairlysimple.
Iftheproblemhasaknownorstandardsolution,andthereisgeneralagreementonwhentoapplythesolution,andthesolutionhasbeenregularlysuccessfullyappliedinvarioussettings,designresearchisprobablyapooruseofresources.
![Page 77: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/77.jpg)
an introduction to educational design research 75
Evenformorechroniclearningproblemssuchasreading,ifthereareadequatetrainingprograms,andclearmeasuresofsuccessorprogress(e.g.,useofphonicstoteachdecodingskills),designresearchisprobablynotindicated.If,however,newresearchsuggestsapowerfulinnovation,designresearchmaybeareasonablechoice(seebelow,andMcCandliss,Kalchman&Bryant,2003)
Generally,designresearchisprobablynotrecommendedforclosedproblems(e.g.,improvingmathematicscalculationfluency),wherethe:
• Initialstate(s)areknown(e.g.,twonumbersaretobemultiplied;achessboardisreadytoplay).
• Goalstate(s)areknown(e.g.,aproductoftwonumbersistoproduced;checkmateorstalemateinchess).
• Operatorstomovefrominitialstatestogoalstatesareknownandcanbeapplied.(e.g.,theproceduresofmultiplication;therulesofchess).
When is design research appropriate?
Designresearchisrecommendedwhentheproblemfacinglearningorteachingissubstantialanddauntinghow-to-doguidelinesavailableforaddressingtheproblemareunavailable.Further,asolutiontotheproblemwouldleadtosignificantadvancesinlearningoratleastasignificantreductioninmalfunctionintheeducationalsystem.
Thereshouldbelittleagreementonhowtoproceedtosolvetheproblem,andliteraturereviewstogetherwithanexaminationofothersolutionsappliedelsewhere(i.e.,benchmarking)shouldhaveprovenunsatisfactory.Designresearchisfurthersuggestedifpriortrainingorinterventionshaveconsistentlyprovenunsuccessful.Designresearchisoftenindicatedforcriticaleducationalgoals,evenwhenthereisnotacleardefinitionofsuccess,ordesigningadequateindicatorsofsuccessispartoftheoverallproblem.
Inotherwords,designresearchismostappropriateforopen,ormoreappropriately,wickedproblems.TheconceptofawickedproblemwasdescribedbyRittelandWebber(1977)todescribeproblemsthatsharethefeaturesofopenproblems,butthatalsoengageelementsthatmaketheirsolutionfrustratingorpotentiallyunattainable.
![Page 78: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/78.jpg)
an introduction to educational design research76
Followingfromthedescriptionofclosedproblems,above,inopenproblems,someormoreofthefollowingapply:
• Initialstate(s)areunknownorareunclear.• Goalstate(s)areunknownorareunclear.• Operatorstomovefrominitialstatestogoalstatesareunknownorhowtoapplythe
operatorsisunclear.
Forwickedproblems(e.g.,Camillus,2008;Horn&Weber,2007;Richey,2007),thecharacterofopenproblemspertain.Plus,therearetypicallyinadequateresources,unclear“stoppingrules”(conditionsthatindicateasolutionisathandortheprojectshouldbeabandoned),uniqueandcomplexcontexts,andinter-connectedsystemicfactorsthatimpingeonprogress.Mostfrustrating,theseotherfactorsmaythemselvesbesymptomsofproblemsofassociatedwickedproblems.Forexample,attemptingtoteachnumeracyinasocietywithhighpovertyandHIVrates.
Therefore,oneofthebroadgoalsofdesignresearchistodynamicallyclarifytheinitialandgoalstatesandtheoperators,andtoilluminatethenatureoftheproblem–i.e.,to“tame”awickedproblembybetterspecifyingitscharacterandmakingitopentointervention.Ineducationalsettings,designresearchisrecommendedwhenoneormoreofthefollowingconditionsoperatetomaketheproblemmorewickedandopenthansimpleandclosed,forexample:
• Whenthecontent knowledgetobelearnedis new or being discoveredeven by the experts.
• Whenhowtoteachthecontentisunclear:pedagogical content knowledge is poor.• Whentheinstructional materialsarepoorornotavailable.• Whentheteachers’ knowledgeandskillsareunsatisfactory.• Whentheeducational researchers’ knowledgeofthecontentandinstructionalstrategies
orinstructionalmaterialsarepoor.• When complex societal, policy or political factors may negatively affect progress.
AnumberofexamplesofmaybefoundinKelly,LeshandBaek(2008).Someotherexamplesfrommathematics,science,andreadingarebrieflypresentedinthenextsection.
Examples from mathematics, science and reading
Thissectionpresentsbrieflyanumberofexamplesofwhenapplyingdesignresearchistheappropriateresearchapproach.
![Page 79: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/79.jpg)
an introduction to educational design research 77
1. IntroducingExistingScienceorMathematicsatEarlierGradeLevelsForexample,someeducationauthoritieshaveadvocatedtheteachingofalgebrainearliergrades(asearlyasthe8thgradeintheUS),seeFoundationsforSuccess:ReportoftheNationalMathematicsAdvisoryPanel(http://www.ed.gov/about/bdscomm/list/mathpanel/index.html).Afewpolicymakershaveevenadvocatedstartingalgebrainstructionintheearlyelementary.
Howshouldoneproceedtointroduceideasofalgebraicreasoningintheearlyelementarygrades?Isthisrecommendationadvisable?Thisissueclearlymeetsthecriteriasetout,above.SomeofthecomplexitiesassociatedwithansweringthisquestioncanbegaugedbyreadingsomeoftherecentworkonthistopicbyCarraherandcolleagues(e.g.,Carraher&Schliemann,2007;Carraher,Schliemann,&Schwartz,2007;Peled,&Carraher,2007;Schliemann,Carraher&Brizuela,2007).
2. Learningneworemergingsciencecontent(e.g.,genetics)Researchinmicrobiologyisinrevolutionwithstunningfindingsappearingonfrontpagesofnewspapers,almostdaily,worldwide.Howcanhistoricalscienceeducationbeupdatedtopreparehighschoolteachersandstudentstomeetthischallengeandopportunity?Moreover,howcanhighschoolspreparestudentstobesuccessfulinemergingintegratedbiologyprogramssuchastheoneatPrincetonUniversity(http://www.princeton.edu/integratedscience/)?RutgersUniversityhasexploredthischallengethroughitsmicrobiologyprogram(http://avery.rutgers.edu/WSSP/Begin/index.html).Areviewofitsvariedsolutionstothischallengeexemplifiesthisrichcontextfordesignresearch.
3. UncoveringthePotentialContributionsofNeuroscienceforMathematicsLearningTheauthorhasjoinedotherwriters(e.g.,Varma,McCandliss&Schwartz,2008)inoutliningthecaseforcultivatingtheintersectionofneuroscienceandmathematicslearning.(e.g.,Kelly,2002,2008).
Whyisthereagrowinginterestinneuro-mathematicseducation?Anumberoffactorshavecoincidedtosupportasurgeininterestinbrain-basedmathematicseducationresearch(seeOECD,2007foracomprehensivereviewofbrain-relatedresearchineducation):• Confidenceduetorecentgainsinunderstandingthebrainbasesforprocessesof
decodinginreading.• Emergentfindingsintheneuralbasesformathematicalthought.• Decadesofbehavioralandcognitivesciencefindingsonlearningmathematicsand
relatedhigher-orderprocessesfromwhichtodraw.
![Page 80: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/80.jpg)
an introduction to educational design research78
• Adesiretodisambiguateandconstrainresearchhypothesesatthebehavioral,cognitiveandsociallevelsofanalysis.
• Adesiretosharpenandgrounddiagnosisandremediationofmathematicallearningdifficultieswithimprovedassessments.
• Adesiretoconstructnewmixed-methodsresearchmethodologiesforthesocialsciences.
• Adesiretoscientificallydebunklearningandteaching“neuromythologies”.• Asenseofurgencytobringscientificdiscourse,evidenceandreasoningtotheslateof
ethicalissuesthatareemergingthatpertainbothtolearningandteaching.• Agoaltoimprovemethodsofteachingofmathematics.• Agoaltoimproveeducationalmaterials,includingthosethatusecomputerhardware
andsoftware.• Morecomprehensiveandtestablemodelsoflearningemergingfromcognitivescience
(e.g.,Bruer,1997).• Adesiretounderstandandpromotesignificantmathematicalcreativity.• Tochallengeneuroscientiststocontinuetopushtheboundariesofimaging
technologies,andtoco-formulateclinicallearningtasks.Thepointtobedrawnhereisthatthecoincidenceofthesefactors,alone,doesnotdictateteachingorlearningstrategiesorevenprovideprinciples,materials,curricula,interventions,orassessmentapproachestosupporteitherlearningorteaching.How,then,shouldresearchersproceedtobringthelaboratoryfindingsofcognitiveneuroscienceintotheclassroominviableways?Again,theproblemmeetstheaboverequirementsforusingdesignresearch.
4. CyberinfrastructureCyberinfrastructureencompassestheuseofdistributedinternetresourcessuchascomputingsystems,data,informationresources,networking,digitallyenabled-sensors,instruments,virtualorganizations,andobservatories(NSF,2007).Itallowstolinkgroupsofscientiststoattackmulti-levelcomplexproblems.Theseproblemswillhaveassociatedchallengesforlearning,teaching,andassessment.Importantquestionsarehoweducationshouldcapitalizeoncyberinfrastructureresources.Whatitmeanstostudysciencecontentwithinacyberinfrastructureframework,andwhatthecurricular,instructionaldesign,assessment,teacherprofessionaldevelopment,andpolicyquestionsthatareraised,andhowtheymustbeansweredtofullyexploitthehigh-technologyinvestmentinscienceatthislevel.Asimportant,whatarethemethodologicalchallengesinstudyinglearningwithinacyberinfrastructureproject?Forexample,howareclaimsofcausalityhandledinacomplexnetworkedandnestedlearningenvironment,andwhatevidencewouldmakesuchclaimscredible(e.g.,Kelly&Yin,2007)?Thisisaclearexample,spanningmanysciencedisciplines,forwhichdesignresearchisanappropriateinvestment.
![Page 81: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/81.jpg)
an introduction to educational design research 79
TheAppendixtothischapterdiscussesinmoredetailthemeaningandpossibilitiesofcyberinfrastructureore-scienceingeneralandforeducation.
5. ReadingandInquiryScienceThereader’sattentionisdrawntotwoexamplesfromBrendaBannan-Ritland,currentlyatGeorgeMasonUniversity.Heranalysisofhowdesignresearchworkswithinherintegrativelearningdesignframework(Bannan-Ritland,2003;Bannan-Ritland&Baek,2008;seealsopaperinthisvolume)providesexamplesinnarrower,ifnolessimportant,applications.
Forexample,designresearchisappropriatewhendevelopingcreativeorinnovativeeducationalproducts,blueprintsordesignsthataredirectedatchroniceducationalproblems.Inanumberofpapers(seeLiteracyAccessOnline,Bannan-Ritland&Baek,2008;http://immersion.gmu.edu/lao/spring2003/projectResources.htm),andBannan-Ritland’schapter(thisvolume)describestheprocessesundertakentoaddressachronicprobleminmostcountries,howtoteachreadingtostrugglingreaders.
FollowingherworkonLiteracyAccessOnline,Bannan-Ritlandextendedherworkonreadingdesignintothelearningofinquiryscienceatthe4thgrade.Basedonthisexperience,Bannan-Ritlandsignificantlyaddedtothebroadeninguseofdesignresearchprinciplesbymethodologicallyincorporatingteachersasdesignersintheoveralldesignresearchparadigm.Thisexcitingnewdirection,calledteacherdesignresearch(whichdovetailswithworkbyZawojewskietal.,2008),isdescribedinBannan-Ritland(2008).Theareaofapplicationinthereportisearthsciencesintheearlyelementaryschool.
The growing need for design assessment research
Arecentreviewofcontributionstodesignresearchshowanincreasingawarenessoftheneedfortacklingtheproblemofhowtoassesslearninginemergingareasoflearning,particularlywhenthereisanemphasisoninnovationininstructionalpractices(Kelly,Baek,Lesh&Bannan-Ritland,2008).Theynote:
In design research as currently practiced, assessment is not directed at some summative sense of learning, though a summative measure of student
learning would be central to later attempts at confirmatory studies, i.e. to show local impact (Bannan-Ritland, 2003). . . . Design research also differs from forma-
tive assessment with regard to the student’s knowledge end state and how feedback loops are enacted. Formative assessment is the gathering of data
relative to some predetermined fixed point, providing feedback that informs the students and teacher of their current knowledge state in relation to some
![Page 82: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/82.jpg)
an introduction to educational design research80
end state (see Black & Williams, 1998). In design research, assessment may be used formatively in order to dynamically determine progress toward mastery of disciplinary knowledge (e.g., Cobb & Gravemeijer, [Kelly, Lesh & Baek, 2008])
or to guide the design of a prototype and to inform its iterative re-design as necessary or both. In fact, sensitivity to assessment practices themselves may
inform changes to the act of assessment itself (e.g., Lobato, [Kelly, Lesh & Baek, 2008]; Lesh et al., [Kelly, Lesh & Baek, 2008]). Ultimately, design researchers are
challenging the assumptions about learning, teaching, and knowing that underlie available assessment techniques, not only in terms of the psychome-tric assumptions (like item response theory), but also the function of assess-
ment itself within and across the stages of design research (see Sloane & Kelly, [Kelly, Lesh & Baek, 2008]).
Inotherwords,whenasuitablecontextfordesignresearchisidentified,totheextentthattheapplicationisnovel(e.g.,teachingalgebraicconceptsintheearlyelementarygrades,readingcomprehension)ortheknowledgeunfolding(e.g.,genomics,cyberinfrastructure),therewillbearequirementandaresponsibilityforresearchersnotonlytoiterativelyinvestigatetheimpactoflearningprototypes,butalsotoaddressdirectlythequestionofhowthisimpactwillbemeasured.Thepointhereisnotthatassessmentisnecessary,ratherthatthetargetsforassessmentmayarisedynamicallyinthecourseofdesignresearchandmeasuresmaynotbeavailableapriori.Asaresult,manyofthequestionsaboutthevalidityandreliabilityofmeasureshavetobeactivelyreconsidered.Inpractice,toooften,prototypesareredesignedwithoutspecifyingtheevidencebase(viaassessmentdesign)fortheredesign.Inmanycases,designresearchersappeartorelyonjudgmentorsubjectivefactors.Addingtotheunfoldingneedfornewmethodsindesignassessmentwillbeamajorchallengeandopportunityforscholarsinthenextdecade(e.g.,Kelly,2005a,2005b).
Whatistheevidencetosupportclaimsofeffectivenessduringiterations,andlater,astheinnovationissubjecttomorerigoroustests?
Design research in general practice
Thegoalofthischapterwastocharacterizedesignresearchatabroadlevel,andtoprovidesomeexamplesofwherethesignificantresourcesassociatedwithdesignresearchmightbespent.Iwillfinishwithageneraloutlineofhowdesignresearchcyclesunfoldwithinalargerframeworkofresearch(Bannan-Ritland,2003;Plomp,thisvolume).Usingcognitivescience,cognitivepsychologyandothersocialsciencemethodssuchassurveys,casestudies,clinicalinterviews,ethnography:
![Page 83: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/83.jpg)
an introduction to educational design research 81
• Identifyorcharacterizetheinitialstates.Clarifytheinitialknowledgeandgoalknowledgestates(ofstudents,teachers,researchers,experts)usingtheinterventions.
• Identifyorcharacterizethegoalstates.Designformativeassessmentstomonitorprogresstowardthegoalstate.
• Identifyorcharacterizetheoperators.Dynamicallyusingthecognitiveandotheranalyses,iterativelydesignandspecifytheoperators(interventions,supports,environments)tosupportlearning.See,inparticular,theworkofBannan-Ritland(2008)andZawojewskietal.,(2008).
• Informre-designcyclesoriterationsusingdatagatheredfromunfolding,andparallelworkindesignassessment.
• Worktowarddevelopingamatureprototypethatcanbesubjecttoamoredefinitivetest(e.g.,randomizedclinicaltrial),seeBannan-Ritland’s(2003)localimpactphase.
One final note: Prototyping and theory building
Byperturbingthesystemusingtheinterventionsinthisiterativeresearchprocess,designresearchtranscendseachofthelocalmethodsused.Inotherwords,designresearchinvolvesnotonlytheuseofdifferentmethods(e.g.,surveys,casestudies,clinicalinterviews),butcombinesthefruitsofeachmethod,overtime,tospecifytheoryandmodelsrelatedtolearning,teachingandassessingthetargetknowledge(seeCobb&Gravemeijer,2008).Thus,designresearchgoes beyond simple development of an interventionandgoes beyond standard cognitive analysesandallowstheoryandmodelingthataccountsforthecontent,thecognition,andtheenactmentbyrealpeopleinrealandrichcontextswithreallimitsonresources(seeZaritskyetal.,2003).
Thequestionofthe“theoreticalyield”ofdesignresearchisnotasimpleone.Notethatthischapterwasframedintermsofcomplex,openandwickedproblems.Forsuchproblems,thereexistsnosimpletheoreticalmodel(atleastnoneisperceivedatthetime).Forthatreason,if“theory”issomethingthatisassumedtobeinformedbyhypothesistestingofasomewhatdefinitivequestion,thendesignresearch(inearlystages)willlikelynotposeoreasilyanswersimplehypotheses,andthusnothavesimpletheoreticalyield.Schwartz,ChangandMartin(2008;inKelly,Lesh&Baek)viewsthedesignresearchcyclesaspreparatoryfortheoreticalyieldfromlaterrandomizedclinicaltrialsorotherlaboratorytests.Iftheobservationisborneoutthatmuchofeducationalinterventionoccursincomplexsystems,thenthetheoreticalyieldwillnotbeassociatedwithonetheory,butmany(perhapsinterdependent)subtheories.Ifso,thentheyieldmaybediffuseandobfuscatedbytheinfluenceofmanyfactorsthatarenotcontrolledindesignresearchsettings.Someresearchershaveattemptedtoframedesignresearchwithinanoverarchingtheory(say,“variation”theory,Holmqvist,Gustavsson,&Wernberg,2008).Thepay-offofthisapproachwillinformusgreatlyabouttheroleoftheoryindesignresearch.
![Page 84: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/84.jpg)
an introduction to educational design research82
Somewritersusetheword“theory”moregenerallytoencompass“designprinciples,”anditmaybethecasethatsuchprinciplescanindeedbeidentified(seeKali,2008).Suchrecommendationsfordesignpracticeareusefulheuristics.Iftheseheuristicsshowevidenceofdurableapplicabilityacrossmanyprojectsandcontexts,itislikelythatsomenecessary(asopposedtocontingent)principlesarebeingevoked(seeKelly,2004),whichwouldopentheseheuristicstotheoreticalanalysis.
References
Akker,J.J.H.vanden,Gravemeijer,K,McKenney,S.&Nieveen,N.(Eds.)(2008).Educational design Research.London:Routledge.
Bannan-Ritland,B.(2003).Theroleofdesigninresearch:Theintegrativelearningdesignframework.Educational Researcher 32(1) 21-24.
Bannan-Ritland,B.(2008).Teacherdesignresearch:Anemergingparadigmforteachers’professionaldevelopment.InA.E.Kelly,R.Lesh,andJ.Baek(Eds.),Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching.NewYork:Routledge.
Bannan-Ritland,B.,&Baek,J.Y.(2008).Investigatingtheactofdesignindesignresearch.InA.E.Kelly,R.Lesh,andJ.Baek(Eds.),Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching.NewYork:Routledge.
Barab,S.&Squire,K.(2004).Design-basedresearch:puttingastakeintheground.Journal of the Learning Sciences, 13(1),1-14.
Camillus,JohnC.,“StrategyasaWickedProblem,”HarvardBusinessReview,May2008.Carraher,D.W.&Schliemann,A.D.(2007).Earlyalgebraandalgebraicreasoning.InF.Lester
(ed.)Second handbook of research on mathematics teaching and learning: A project of the National Council of Teachers of Mathematics.VolII.Charlotte,NC:InformationAgePublishing,pp.669-705.
Carraher,D.W.,Schliemann,A.D.&Schwartz,J.(2007).Earlyalgebraisnotthesameasalgebraearly.InJ.Kaput.D.Carraher,&M.Blanton(Eds.),Algebra in the early grades. Mahwah,NJ,Erlbaum,pp.235-272(nowTaylor&Francis).
Chin,G.,Jr.,&Lansing,C.S.(2004).Capturingandsupportingcontextsforscientificdatasharingviathebiologicalsciencescollaboratory,Proceedings of the 2004 ACM conference on computer supported cooperative work,409-418,NewYork:ACMPress.
Cobb,P.&Gravemeijer,K.(2008).Experimentingtosupportandunderstandlearningprocesses.InA.E.Kelly,R.Lesh,andJ.Baek(Eds.),Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching. NewYork:Routledge.
![Page 85: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/85.jpg)
an introduction to educational design research 83
Cogburn,D.L.(2003).HCIintheso-calleddevelopingworld:what’sinitforeveryone,Interactions,10(2),80-87,NewYork:ACMPress.
Holmqvist,M.,Gustavsson,L.,&Wernberg,A.(2008).Variationtheory:Anorganizingprincipletoguidedesignresearchineducation.InA.E.
Kelly,R.Lesh,andJ.Baek(Eds.),Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching. NewYork:Routledge.
Horn,RobertE.,Knowledge Mapping for Complex Social Messes,apresentationtothe“FoundationsintheKnowledgeEconomy”conferenceattheDavidandLucilePackardFoundation,July16,2001.
Kali,Y.(2008).TheDesignPrinciplesDatabaseasmeansforpromotingdesign-basedresearch.InA.E.Kelly,R.Lesh,andJ.Baek(Eds.),Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching. NewYork:Routledge.
Kelly,A.E.(2003).Theroleofdesigninresearch.Educational Researcher, 32(1),3-4.Themeissueguesteditor.
Kelly,A.E.(2004).Designresearchineducation:Yes,butisitmethodological?Journal of the Learning Sciences, 13(1),115-128.
Kelly,A.E.(2005a).“How do modeling perspectives inform program assessment?”Presentationatthe27thannualmeetingoftheNorthAmericanChapteroftheInternationalGroupforthePsychologyofMathematicsEducation.Roanoke,VA.
Kelly,A.E.(2005b).“Theintersectionofdesignandassessmentformathematicslearning.”Presentation,NationalScienceFoundationWorkshop,ParkCity,UT.
Kelly,A.E.(2006).Qualitycriteriafordesignresearch.In:J.J.H.vandenAkker,K.Gravemeijer,S.McKenney,&N.Nieveen(Eds.).Educational design Research.London:Routledge.
Kelly,A.E.(2008).Brainresearchandeducation:Potentialimplicationsforpedagogy.InEducation, sciences cognitives et neurosciences.Paris:PressesUniversitariesdeFrance.
Kelly,A.E.,Baek,J.,Lesh,R.,&Bannan-Ritland,B.(2008).Enablinginnovationsineducationandsystematizingtheirimpact.InA.E.Kelly,R.Lesh,andJ.Baek(Eds.),Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching. NewYork:Routledge.
Kelly,A.E.&Yin,R.(2007).Strengtheningstructuredabstractsforeducationalresearch:Theneedforclaim-basedstructuredabstracts.Educational Researcher.133-138.
McCandliss,B.D.,Kalchman,M.,&Bryant,P.(2003).Designexperimentsandlaboratoryapproachestolearning:Stepstowardcollaborativeexchange.Educational Researcher, 32(1),14-16.
Middleton,J.,Gorard,S.,Taylor,C.,&Bannan-Ritland,B.(2008).The“compleat”designexperiment:Fromsouptonuts.InA.E.Kelly,R.Lesh,andJ.Baek(Eds.),Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching.NewYork:Routledge.
![Page 86: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/86.jpg)
an introduction to educational design research84
NationalScienceFoundation(2007).Cyberinfrastructure vision for 21st Century discovery.OfficeofCyberinfrastructure.March.
OECD(2007).Understanding the brain: The birth of a learning science.Paris:OECD.Olson,G.M.,Teasley,S.,Bietz,M.J.,&Cogburn,D.L.(2002).Collaboratoriestosupport
distributedscience:theexampleofinternationalHIV/AIDSresearch,Proceedings of the 2002 annual research conference of the South African institute of computer scientists and information technologists on enablement through technology,44–51.
Peled,I.&Carraher,D.W.(2007).Signednumbersandalgebraicthinking.InJ.Kaput.D.Carraher,&M.Blanton(Eds.),Algebra in the early grades.Mahwah,NJ,Erlbaum,pp.303-327(nowTaylor&Francis).[608k]
Richey,Tom;Wicked Problems: Structuring Social Messes with Morphological Analysis,SwedishMorphologicalSociety,lastrevised7November2007
Schliemann,A.D.,Carraher,D.W.,&Brizuela,B.(2007).Bringing out the algebraic character of arithmetic: From children’s ideas to classroom practice.LawrenceErlbaumAssociates(nowTaylor&Francis).
Schwartz,D.L.,Chang,J.,&Martin,L.(2008).Instrumentationandinnovationindesignexperiments:Takingtheturntowardsefficiency.InA.E.Kelly,R.Lesh,andBaek(Eds.),Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching. NewYork:Routledge.
Thomas,J.J.,&Cook,,K.A.(Eds.),(2005).Illuminating the path: The research and development agenda for visual analytics. LosAlamitos,CA:IEEEComputerSociety.
Varma,S.,McCandliss,B.D.,Schwartz,D.L.(2008).Scientificandpragmaticchallengesforbridgingeducationandneuroscience.Educational Researcher, 37(3),140-152.
Wulf,W.(1989,March).Thenationalcollaboratory.InTowards a national collaboratory. UnpublishedreportofaNationalScienceFoundationinvitationalworkshop,RockefellerUniversity,NewYork.
Zaritksy,R.,Kelly,A.E.,Flowers,W.,Rogers,E.,&O’Neill,P.(2003).Clinicaldesignsciences:Aviewfromsisterdesignefforts.Educational Researcher, 32(1),32-34.
Zawojewski,J.,Chamberlin,M.,Hjalmarson,M.A.,&Lewis,C.(2008).Developingdesignstudiesinmathematicseducationprofessionaldevelopment:Studyingteachers’interpretivesystems.InA.E.Kelly,R.Lesh,andJ.Baek(Eds.),Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching. NewYork:Routledge.
![Page 87: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/87.jpg)
an introduction to educational design research 85
Appendix
FortheUSNationalScienceFoundation(NSF,2007),theopportunitiesinthecomplementaryareasthatmakeupcyberinfrastructure:computingsystems,data,informationresources,networking,digitallyenabled-sensors,instruments,virtualorganizations,andobservatories,alongwithaninteroperablesuiteofsoftwareservicesandtoolsprovidechallengesalongthreelines:(a)data,dataanalysis,andvisualization;(b)virtualorganizationsfordistributedcommunities;and(c)learningandworkforcedevelopment.
AmajorparallelactivityincyberinfrastructureisunderwayinEurope,whichislabeled“e-science”.e-sciencedescribessimilaractivitiestotheUScyberinfrastructure.NotunlikeearlyvisionsofUScyberinfrastructure,theUKlaunchingdocument,(http://www.nesc.ac.uk/documents/OSI/report.pdf),didnotexplicitlylisteducationasoneofthekeyareasofconcerninsettingupacyberinfrastructure.Itfocused,rather,onnetworks,middleware,digitallibraries,andcomputationalresources.AsintheUS,thisimbalanceisbeingrecognized.InEurope,itbeingaddressedbythecreationofICEAGE:“Theinternationalcollaborationtoextendandadvancegrideducation”(http://www.iceage-eu.org/v2/partners.cfm).ICEAGE,whileinternational,isprimarilyaEuropeaneffort,withbranchesinEdinburgh,Scotland,UniversityofCatania,Sicily,SPACI(SouthernPartnershipforAdvancedComputationalInfrastructures),anItalianuniversity-basedeffort(http://www.spaci.it/),CERN,nearGeneva(http://public.web.cern.ch/Public/Welcome.html),theRoyalInstituteofTechnologyinSweden(http://www.kth.se/?l=en_UK),andTheComputerandAutomationResearchInstitute,HungarianAcademyofSciences(http://www.sztaki.hu/institute).
Cyberinfrastructuredescribestheuseofdistributedinternetresourcestolinkgroupsofscientiststoattackmulti-levelcomplexproblems.Theseproblemswillhaveassociatedchallengesforlearning,teaching,andassessment.Forexample,adesignresearchproblemwouldbehowtodescribeandcreditastudent’slearninginacyberinfrastructureresearchcollaboratoryingeosciences:Scientifically,acrucialconcernindetectingearthquakesistomeasureminutechangesinelevation.Traditionalradar,whichusesradiowavesasthemeansofdetectingdistancesfromthesource,areoflimitedvalueinprecisemeasurementsduetothelengthoftheradiowaves.TheuseofLiDAR(LightDetectionandRanging)technologyallowstheuseofwavelengthsintheultraviolet,visible,ornearinfraredrange(fromabout10micrometerstotheUV(ca.250nm).Theseshorterwavelengthsallowdetectionofsmokeandotherdiffuseparticulates,whichhasledtotheuseofLiDARinmeteorology.Forearthquakeprediction,LiDARcanbeusedtolocatefaults,andtomeasureuplift.Faultsdescribethelineoffractureanddemarcationbetweenplates(McKnight&Hess,2000).
![Page 88: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/88.jpg)
an introduction to educational design research86
Upliftistypicallyduetotectonicplateactivity(Kearney&Vine,1990),technically“orogenicuplift”orduetotheremoval(duetoerosion)ofheavymaterial,technically“isostaticuplift.”ThesignificantadvantageofLiDARoverradaristhatLiDARcangeneratedigitalelevationmodels(DEMs)oftheshapetheearth’ssurfaceatresolutionsnotpreviouslypossible.Complexifyingtheproblem,anearthquakeissometimesassociatedwithvolcanicactivity.Forexample,the“PacificRimofFire”isassociatedwithcollidingtectonicplates.Insuchcases,LiDARmaybeusednotonlytomakeprecisemeasurementsofelevation,butalsotocharacterizethedensityandeventhechemicalmakeupofthegasesandashemittedbyavolcano.LiDARdataonMountSt.Helen’svolcanomaybefoundathttp://wagda.lib.washington.edu/data/type/elevation/lidar/st_helens/.
LearningaboutgeomorphologyusingLiDARiscomplex,andsomepubliclyavailablewebsiteshaveattemptedtoprovideinstruction(e.g.,http://lidar.cr.usgs.gov/andhttp://gisdata.usgs.net/website/lidar/viewer.php).ThemostcomprehensiveactivityhasbeenconductedbytheGEONnetwork(http://www.geongrid.org/).Thisnetworkispartofacyberinfrastructureresearchcollaborator.TutorialsontheuseofLiDARwithinandoutsideofgeoscience(e.g.,coastalerosion,flooding,rivercourses,forestmappingandmining)maybefoundhere;http://home.iitk.ac.in/~blohani/LiDAR_Tutorial/Airborne_AltimetricLidar_Tutorial.htm.
WecannowseejustafractionoftheassociatedscientificconceptsthatarepertinentinunderstandingtheuseofLiDARinunderstandinggeoscience:e.g.,radartechnologyvsLiDARtechnology,thescienceofplatetectonics,digitalelevationmodels,readingandunderstandingcomputervisualizations,modelingcomplexinter-relatedscientificprocesses,reasoningaboutimplicationsforhumanactivity,includingurbangrowth,andsoforth.Whichofthese(orotherrelatedconcepts)aremostpertinentforscientistsinacyberinfrastructureresearchcollaboratorywillbeanempiricalquestion.Howtoidentifythecentralconstructspertinenttoahigh-schoolscienceeducationwillprovideasignificantmeasurementchallenge,includinghowtodesignauthenticassessmentstomeasureunderstandingoftheseconcepts.Identifyingandmappingoutthecontentandcognitivedemandsofsuchmeasurementcouldbeamajorfocusofthedesignresearchwork.Ofparticularinterestwillbehowtoestablishcontent,construct,predictive,concurrentandotherformsofvalidityforthesemeasures.
Factorsconvergingtosupportthedevelopmentofcyberinfrastructure.
1. ExistingcomputingdatagridsintheUSandoverseas a. TheTeraGridproject(http://www.teragrid.org/about/)combinesthepowerofNCSA,
SDSC,ArgonneNationalLaboratory,CACR,PSC,ORNL,TACC,andvariousuniversity
![Page 89: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/89.jpg)
an introduction to educational design research 87
partnersintegratedbytheGridInfrastructureGroupattheUniversityofChicago.Europeane-sciencelinksfacilitiesontheContinentwiththoseintheUK.SimilaractivitiesoccurinJapan.IndustrypartnersincludeIBM,Intel,Hewlett-PackardandOracle.
2. Theavailabilityofmassivedatastoragecapacityandspeed a. TheTeraGridcurrentlyoffersover100teraflopsofcomputingpower;andover3
petabytesofrotatingstorage3. Thedevelopmentofmiddlewareandsoftwaretogatherandanalyzestoreddata a. TheTeraGridsupportsdataanalysisandvisualizationproductioninterconnectedat
10-30gigabits/second.4. Theemergenceoflargeteamsofscientistsdedicatedtosolvingsharedscience problems(actingthroughscience“collaboratories”and“gateways”) a. Acollaboratory(Wulf,1989)is“morethananelaboratecollectionofinformationand
communicationstechnologies;itisanewnetworkedorganizationalformthatalsoincludessocialprocesses;collaborationtechniques;formalandinformalcommunication;andagreementonnorms,principles,values,andrules”(Cogburn,2003,p.86).Collaboratoriesexistinmanyareasofscience,includingbiology,chemistry,geoscienceandastronomy(e.g.,Chin&Lansing,2004;Olson,Teasley,Bietz,&Cogburn,2002).
b. Sciencegatewaysareweb-basedportalsorinterfacesforthestructuresanddataofthecyberinfrastructureinmanyscienceareas(foralistingof24gateways,seehttp://www.teragrid.org/programs/sci_gateways/).
5. Developmentsinscientificvisualization. Scientificvisualizationdrawsonhumanspatialandvisualprocessinginordertomodel
andanalyzecomputationallyintensethegraphicdisplayofcomplexdata(foracomprehensivereview,seeThomas&Cook,2005).Existingmethodsandmodelsforscientificvisualizationaresignificantlychallengedbycyberinfrastructure(e.g.,http://www.teragrid.org/userinfo/data/vis/vis_gallery.php;Chinetal.,2006).
6. Funding. Theestablishmentandfundingofnationalandinternationaleffortstocoordinateand
developtheinfrastructuretobetterservescienceand,morerecently,education(e.g.,theOfficeofCyberinfrastructure–NSF;CERN,Dutch(VL-e)andUKinitiatives).ThepromiseofcyberinfrastructureforeducationisthatthevastinvestmentbyUSagencies(upwardsof$250Moverthenext5years,alone)willprovidetest-bedsforexploration.
![Page 90: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/90.jpg)
![Page 91: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/91.jpg)
an introduction to educational design research 89
5. FormativeEvaluationinEducational DesignResearch Nienke Nieveen
Introduction
Inthischapterandinlinewiththegeneralintroductionofthisbook,wedefineeducationaldesignresearchas:the systematic study of analyzing, designing and evaluating educational interventions in order to solve complex educational problems for which no ready-made solutions are available and to gain insight in key design principles.Designresearchprojectsstriveaftertwotypesofmainresults.Thefirstaimcompriseshigh-qualityinterventions(suchasprograms,productsandprocesses)designedtosolvecomplexeducationalproblems.Thistypeofoutputputsforwardthepracticalrelevanceofdesignresearch.Itisforthatreasonthatdesignresearchisalsolabeledasbeinguse-inspired,appliedorientedand/orsociallyresponsibleresearch(vandenAkker,1999;Reeves,2000).Thesecondmainoutputofdesignresearchistheaccompanyingsetofwell-articulateddesignprinciples(Linn,Davis&Bell,2004;vandenAkker,1999)thatprovideinsightinthe:• purpose/functionoftheintervention;• keycharacteristicsoftheintervention(substantiveemphasis);• guidelinesfordesigningtheintervention(proceduralemphasis);• itsimplementationconditions;• theoreticalandempiricalarguments(proof)forthecharacteristicsandprocedural
guidelines.Thesecomprehensivedesignprinciplesserveseveralpurposesforavarietyoftargetgroups.Fromaresearchperspective,theseprinciplesshowthecontributionofdesignresearchtotheexistingknowledgebasewithinformationonhowtheinterventionworksinpractice,theeffectsofusingtheinterventionandexplanationoftheworkingmechanisms.Foreducationaldesigners,theseprinciplescarryrichinformationonhowtodesignsimilarinterventionsforsimilarsettings.Fromtheperspectiveoffutureusers,theprinciplesprovideinformationneededforselectingandapplyinginterventionsinthespecifictargetsituationandprovideinsightsintherequiredimplementationconditions.Finally,forpolicymakers,theseprinciplesassistinmakingresearch-baseddecisionsforsolvingcomplexeducationalproblems.
Inordertoreachbothtypesofoutput(highqualityinterventionsanddesignprinciples),designresearcherscarefullycombinedesignandresearchactivitiesresultinginaniterativedevelopmentapproach.Inthiscontributionwewillfirstexplorethisiterativenatureofdesignresearch(herelabeledwiththetermprototypingapproach)andthenelaborateon
![Page 92: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/92.jpg)
an introduction to educational design research90
therolethatformativeevaluationplaysindesignresearchprojectsinordertooptimizeinterventionsanddesignprinciples.Thecontributionwillendwithsomeremarksontheroleofdesignresearchersconcerningformativeevaluationactivities.
Prototyping approach
Designresearchisbynaturehighlyiterative(Design-basedresearchcollective,2003;vandenAkker,1999).Eachiterationhelpstoimproveprototypesofbothendresultsofdesignresearchefforts:1.theeducationalinterventionunderdevelopment;and2.itsaccompanyingtentativedesignprinciples.Thissectiondiscussesbrieflythenotionofaniterativeorprototypingapproach.
Aprototypeisapreliminaryversionofthewholeorapartofaninterventionbeforefullcommitmentismadetoconstructandimplementthefinalproduct.Prototypesmaybeusedintwoways(cf.Smith,1991).Ontheonehand,aprototypemaybecontinuallyrefined(basedonformativeevaluationresultsandreflectionsofdevelopersontheprototype)andevolvetowardsafinaldeliverable.Thisrefiningapproachcanbereferredtowiththetermevolutionaryprototyping.Ontheotherhand,developerscandesignthrow-awayprototypes,suchasscenariosorpaper-basedmock-ups(Nieveen,1999).Ascenarioisanarrativedescriptionoftypicalandcriticalsituationsthatprospectiveusersparticipatein.Scenariosmaybeusedtomakethetentativedesignspecificationsmoreconcrete.Thismakesiteasiertocommunicatethepotentialsofasystemwiththetargetgroup.Apaper-basedmock-upcomprisesapileofpapersrepresentingallscreenswhichmayappearduringtheuseoftheintervention.Thiskindofprototypeisoftenusedinsoftwaredevelopmentprojects.Usersmay‘walkthrough’thescreenstogetanideaoftheintentionsofthesoftwareapplication.Paper-basedprototypesfocustheattentionoftheusermoreoncontentandoverallstructurethanonappearance.Afterbeingevaluated,athrow-awayprototypewillbediscardedanditsevaluationresultsaretakenintoaccountinthenextprototype.Thisprocesswillcontinueuntilalluncertaintiesarecoveredandthefinalproductorinterventioncanbedelivered.
Especiallyindesignresearchprojectsthataimatinnovativeandcomplexproducts,withfewexperiencesordesignprinciplesfromwhichtodraw,suchaprototypingapproach(eitherevolutionaryorthrow-away)isrecommendable.Somedesignresearchprojectscombinebothkindsofprototypes,forinstancebyfirstdesigningandevaluatingthrow-awayprototypesandthenshiftingtoanevolutionaryapproach(Nieveen,1999).Tomaketheprototypingapproachwiththrow-awayand/orevolutionaryprototypesfeasible,thenotionof‘thinkbig,butstartsmall’ishelpful.Byfirstdevelopingasmallpartoftheproposedintervention,onekeepsthedevelopmentprocessmanageableandonecan
![Page 93: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/93.jpg)
an introduction to educational design research 91
learnfromfailuresandapplysuccesseswhendesigningthesubsequentparts.Inordertokeepanoverviewontheentiredevelopmentprocessitisoftenfunctionaltodecomposetheinterventionintoseveralcomponentsthatcouldbebuiltseparately.Educationalinterventionscanbedecomposedintoatleasttwokeyaspectswhichwillrequiremajorattentionduringthedesignprocess(cf.Nieveen,1999;Nieveen&vandenAkker,1999):• theconceptual framework oftheintervention,referringtoallnotionsthatare
underlyingtheintervention.Incaseofaneducationalintervention,itrefersforinstancetotheconceptualizationofall10curriculumcomponentspresentedinthecurricularspiderweb(vandenAkker,2003);
• thepresentation-modeoftheintervention,referringtotheformatthatassuresthattheinterventionisusableforitsusers.Toassistusersfindingthecontentoftheirpreference,allinterventions(paper-basedandcomputer-based)needasounduser-interfaceincludingconsistentlayoutandtransparantnavigation.
Thevariousconceptualandpresentationelementsmaybeindifferentstagesofdevelopmentineachprototype.However,towardsthefinaldeliverable,allelementsneedtobeconsistentwithoneanother.Forexample,inamathprojectaimedatsolvingproblemsoflow-achievingstudentswithmeasuringquantities,interventionsweredevelopedtohelpthesestudentstoacquiretherequiredmathematicalproblemsolvingskills.Thedesignresearchteamputmuchemphasisonthedevelopmentofinnovativelearningandteachingactivities(twocomponentsofthecurricularspiderweb/theconceptualframework)gearedtotheproblemsofthesestudents.Subsequently,thefirstversionofthelessonmaterialswasdesignedaccordingtothisspecificpedagogy.Duringtheformativeevaluationofthisfirstprototype,thedesignresearchteamwasespeciallyinterestedinthequalityofthenewlearningandteachingactivities(beingpartoftheconceptualframework)andlessinthelayout(beingpartofthepresentation-mode).However,towardstheendofthestudy,thelayoutofthematerialsgotspecificattentioninordertoimprovetheoverallpracticalityofthematerials.
Formative evaluation
Inaprototypingapproachempiricaldataareneededtogaininsightintothequalityofthetentativeinterventionanddesignprinciples.Forthatreason,formativeevaluationisacrucialfeatureofeachprototypingapproachandthusofeachdesignresearchproject.Itprovidesinsightinthepotentialsoftheinterventionanditskeycharacteristics.Resultsoftheformativeevaluationgivegroundforboth1.improvingtheprototypeoftheinterventiontowardsahigh-qualityfinaldeliverableand2.sharpeningtheunderlyingtentativedesignprinciplestowardsanelaboratedsetofdesignprinciples.Inthisway,eachprototypingcyclecontributestosuccessiveapproximationofbothoutputsofadesignresearchproject.IntheGenericDesignResearchModelofWademan(2005),seeFigure1,thisisnicelyillustratedintheprototypingandassessmentphase.
![Page 94: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/94.jpg)
an introduction to educational design research92
Figure 1: Generic Design Research Model (Wademan, 2005)
Inthissection,theconceptofformativeevaluationwillbefurtherelaboratedandplacedinadesignresearchcontext.Asfarasthetermevaluationisconcerned,theJointCommitteeonStandardsforEducationalEvaluation(1994)usesthefollowingdefinition:“Evaluationisthesystematicassessmentoftheworthormeritofsomeobject.”Meritreferstotheobject’sinherent,intrinsicvalue,whileitsworthisdefinedasitscontextuallydetermined,place-boundvalue(Lincoln&Guba,1979).Scriven(1967)wasthefirstauthorwhomadethedistinctionbetweenformativeandsummativeevaluation.Formativeandsummativeevaluationsservedifferentfunctions.Thefunctionofformative evaluationis‘toimprove’.Itfocusesonuncoveringshortcomingsofanobjectduringitsdevelopmentprocesswiththepurposetogeneratesuggestionsforimprovingit.Thefunctionofsummative evaluationis‘toproof’.Asummativeevaluationiscarriedouttogainevidencefortheeffectivenessoftheinterventionandfindargumentsthatsupportthedecisiontocontinueorterminatetheproject.Summativeevaluationsarebeingcarriedoutwithoutthedirectintentiontoreveal
Practitioner and User Participation
Researchers
Other Sources
Collaboratives
Practitioners
ConsultExperts &
Practitioners
FocusedLiterature
Review
AnalyzePromisingExamples
AnalyzePracticalContext
Tentative Product
Approaches
Tentative Design
Principles
Reflection
FormativeEvaluation
PracticalProduct/Results
Contributionto
Theory
Refinement of Problem, Solution and Method
Refinement of Design Theory
Preliminary investigation of Problem, Context,
& Approaches
Problem in Context
Phases
Prototyping & Assessment of Preliminary Products & Theories
Identification of Tentative Products & Design Principes
Problem Identification Tentative Products& Theories
Problem Resolution & Advancing Theory
Successive Approximation of Theory
Successive Approximation of Product
Redesign & Refinementof Products & Theories
![Page 95: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/95.jpg)
an introduction to educational design research 93
pointsofimprovement.However,itisnotalwayspossibletodrawasharplinebetweenformativeandsummativeevaluation.Theresultsofsummativeevaluationsareusuallytakenintoaccountwhiledevelopingasecondreleaseoftheproduct.
Basedoncomparingandsynthesizingdefinitionsofvariousscholarsinthefieldofformativeevaluation(cf.Brinkerhoff,Brethouwer,Hluchyj&Nowakowski,1983;Flagg,1990;Scriven,1967,Tessmer,1993)wedefineformativeevaluationinthecontextofdesignresearchas:a systematically performed activity (including research design, data collection, data analysis, reporting) aiming at quality improvement of a prototypical intervention and its accompanying design principles.Asstatedbefore,adesignresearchprojectusuallyneedsseveraliterationsbeforeanoptimalsolutionforthecomplexproblemcanbeenreached.Eachdesignresearchcycleoriterationconcentratesonspecificresearchquestionsandneedsanappropriateresearchdesign.Theremainderofthissectionwillelaborateonissuesrelatedtotheresearchdesignofformativeevaluationactivities.
FormulatingresearchquestionsThemainresearchquestionofaformativeevaluationisbuiltaroundthekindofvaluejudgmentthatisexpectedfromevaluatingtheprototypeandtwokeyattributesoftheprototypicalintervention:1.thestageofdevelopmentoftheprototype;and2.themainelementoftheprototypethattheevaluationwillfocuson.
Firstofall,itisnecessarytomakeclearthetypeofvaluejudgmentthattheevaluationneedstoresultin.Inthisrespect,wedistinguishfourqualitycriteriathatareapplicabletoawidearrayofeducationalinterventions(seeTable1).Attheendofadesignresearchproject,theinterventionshouldsufficeallofthesecriteria.However,usuallyeachiterationconcentratesononeortwoofthesecriteria.
![Page 96: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/96.jpg)
an introduction to educational design research94
Criterion
Relevance (also referred to as content validity)
There is a need for the intervention and its design is based on state-of-the-art (scientific) knowledge.
Consistency (also referred to as construct validity)
The intervention is ‘logically’ designed.
Practicality ExpectedThe intervention is expected to be usable in the settings for which it has been designed and developed. ActualThe intervention is usable in the settings for which it has been designed and developed.
Effectiveness ExpectedUsing the intervention is expected to result in desired outcomes. ActualUsing the intervention results in desired outcomes.
Table 1: Criteria for high quality interventions
Itisimportanttopointheretothedistinctionbetweenexpectedandactualpracticalityandeffectiveness.Onlywhenthetargetusershavehadpracticalexperiencewithusingtheinterventiononewillbeabletogetdataontheactual practicalityoftheprototype.Similarly,onlywhentargetusershavehadtheopportunitytousetheinterventioninthetargetsetting,theevaluatorwillgetdataontheactual effectiveness.Inallotherinstances,suchasagroupdiscussionsbasedonthematerials,theresearcherwillonlygetdataontheexpected practicality and/or effectiveness.
Moreover,whenpreparingaformativeevaluationitisimportanttodescribetheboundariesoftheprototypethatwillbeevaluated.Inadesignresearchprojecta(throw-awayorevolutionary)prototypeisusuallyinoneofthefollowingdevelopmentstages:• Designspecifications:Afirstandgeneraldescriptionoftheinterventioninwhich
attentionispaidtoitssubstantiveparts.Thissketchhasbeenbasedonpreliminaryresearchactivities(includingproblemandcontextanalysisandliteraturereview).
• Globalintervention:Someorallcomponentsoftheinterventionaregivensomedetail.Thiscouldbetermedasahorizontalprototype.Itgivesanideaofhowtheinterventionwilleventuallyappear,howeveritcannotyetbeusedinpractice.Forexample,inthecaseofthedevelopmentofanewcurriculumatthisstagetheinterventioncouldtaketheformofatableofcontentswithabriefdescriptionofsub-componentsormodules.
• Partoftheinterventionindetail:Atthisstage,apartorcomponentoftheinterventionhasbeenelaboratedtoaconcretelevelforusebythetargetgroup.Thiscouldbecalledaverticalprototype.Onecanimaginevarioussub-stageswitheachofthemaddressingonlyaspecificpartofthetotalinterventionforuseinpractice.
![Page 97: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/97.jpg)
an introduction to educational design research 95
• Completeintervention:Thetotalinterventionissufficientlydetailedthatitcouldbeusedintheintendeduser-setting.
Anotherissuethatneedstobeclearbeforestartingaformativeevaluationisthemainelementsoftheprototypethattheevaluationwillfocuson.Thesecanbealloroneelementsrelatedtotheconceptualframeworkandpresentation-modeoftheintervention.
Thethreecharacteristics(qualitycriteria,elementsandthestageofdevelopment)giveinputforthemainresearchquestions.Thesyntaxoftheseresearchquestionsis:‘Whatisthe[qualitycriteriuma,b,cand/ord]of[elementoftheconceptualframeworkorpresentationmode]oftheinterventionthatisin[developmentstagew,x,y,z].Instancesofthesequestionsare:• Whatistherelevancy[qualitycriterium]ofthecontent[conceptualelement]ofaquick
referencemanualforusingChinesecharactersthatisinaglobalstage[developmentstage]?
• Whatistheinternalconsistency[qualitycriterium]oftheattainmenttargets[conceptualelement]forscienceinuppersecondaryeducationofwhichthreeoutofsevendomainsareelaboratedindetail[developmentstage]?
• Whatisthepracticality[qualitycriterium]ofthelayout[elementofpresentationmode]oftheMathtextbookmodulesthatisinacompletedstage[developmentstage]?
SelectingappropriatemethodsDesignresearchersneedtoselectthoseformativeevaluationmethodsthatfittheresearchquestions.Buildingonearlierresearch(Nieveen,1997,1999),Table2providesanoverviewoftherelationshipbetweentheresearchquestions(withonthevericalaxisthequalitycriteriaconcerningtheelementsofaninterventionandonthehorizontalaxisthedevelopmentstages,seeprevioussection)andsuitableformativeevaluationmethods(indicatedinthecells).Herewedistinguishthefollowingmethods.PleasereferforanextensiveoverviewforinstancetoTessmer(1993)orBrinkerhoff,etal.(1983):• Screening:membersofthedesignresearchteamcheckthedesignwithsomechecklists
onimportantcharacteristicsofcomponentsoftheprototypicalintervention.• Expertappraisal:agroupofexperts(forinstance,subjectmatterexperts,instructional
designexperts,teachersreviewthematerials)reactsonaprototypeofanintervention,usuallyonthebasisofaguidelinewithcentralquestionsofthedesignresearchteam.Usuallythisisdonebyinterviewingtherespondents.
• Walkthrough:thedesignresearcherandoneorafewrepresentativesofthetargetgrouptogethergothroughthesetupoftheintervention.Usuallythisiscarriedoutinafacetofacesetting.
![Page 98: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/98.jpg)
an introduction to educational design research96
• Micro-evaluation:asmallgroupoftargetusers(e.g.learnersorteachers)usespartsoftheinterventionoutsideitsnormalusersetting.Here,themainactivitiesoftheevaluatorareobservingandinterviewingtherespondents.
• Try-out:alimitednumberoftheusergroup(e.g.teachersandlearners)usesthematerialsinthedaytodayusersetting.Iftheevaluationfocusesonpracticalityoftheintervention,thefollowingevaluationactivitiesarecommon:observation,interviewing,requestinglogbooks,administeringquestionnaires;iftheevaluationhasitsfocalpointontheeffectivenessoftheintervention,evaluatorsmaydecidetorequestlearningreportsand/orgiveatest.
Summativeevaluationmethods,suchas(quasi-)experiments,surveysandaccompanyingcase-studies,followtheseformativeevaluationactivitiesassoonastheinterventionhasbecomefullygrownandhasbeenimplementedineducationalpractice(seeforinstanceRossi,Freeman&Lipsey,1999).
Table 2: Table for selecting formative evaluation methods
Wheninterventionsbecomemoredetailed,thefocusoftheformativeevaluationwillgraduallyshiftwithrespecttotheaforementionedqualitycriteria.Inanearlystage,themainfocuswillbeontherelevancyandconsistencyofaprototype.Assoonasaglobalinterventionhasbeendesigned,designresearchersalsowouldliketoassesstheexpectedpracticalityoftheintervention.Whentheinterventionisevenmoreelaborated,thenthefocuswillshifttowardstheactualpracticalityandeffectiveness.Intable2,itisindicatedingreythatwiththisshiftinfocusalsoother,moresuitable,evaluationmethodswillcomeintoplay.Moreover,eachdevelopmentstagemayconsistofseveralcyclesofanalysis,designandformativeevaluationbeforetheprototypewillgrowintoanextdevelopmentstage.
Table 6: Table for selecting formative evaluation methods
Design stage
Quality criterion
Designspecifications
Global design Partly detailedintervention
Completeintervention
Implementedintervention
Relevance - Screening- Expert appraisal
- Screening- Expert appraisal
- Screening- Expert appraisal
- Screening- Expert appraisal
Consistency - Screening- Expert appraisal
- Screening- Expert appraisal
- Screening- Expert appraisal
- Screening- Expert appraisal
Practicality expected - Screening- Expert appraisal
- Screening- Expert appraisal
- Expert appraisal- Walkthrough
- Expert appraisal- Walkthrough
actual - Micro-evaluation - Micro-evaluation- Try-out
Survey, (Quasi)experiment, Case-study
Effective-ness
expected - Screening- Focus group
- Screening- Focus group
- Expert appraisal - Expert appraisal
actual - Micro-evaluation - Micro-evaluation- Try-out
Survey, (Quasi)experiment, Case-study
![Page 99: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/99.jpg)
an introduction to educational design research 97
Sampling-selectingrespondentsTobeabletoanswertheresearchquestionswiththechosenevaluationmethods,therequiredtypeandnumberofrespondentsneedtobediscussed.Thetypeandsamplesizedependontheresearchquestions.Withrespecttothetypeofrespondents,oneneedstoselectthoserespondentsthatcanhelpansweringtheresearchquestions.Forinstance,incasedesignresearcherswanttogaininsightintherelevancyofthedesignfromasubjectmatterperspectivetheywillselectanumberofexpertsinthatspecificdomaintodoanexpertappraisal.Incaseinsightsareneededintheactualpracticalityofalearningpackageforlearnersbyperformingamicro-evaluation,studentsneedtobesampledwhowillhavetoworkwiththeintervention.Moreover,themainpurposeoftheevaluationalsoinfluencesthesamplesize.Incaseofaformativeevaluationduringearlystagesoftheproject,themainpurposeistolocateshortcomingsintheinterventionandtogeneratesuggestionsforimprovement(seealsodefinitionofformativeevaluation),thenumberofrespondentsislesscritical:aremarkofonlyonerespondentcouldbehighlyvaluablebecauseofitssalience.Smallsamplesofrespondentsareusuallysufficientiftheyarecarefullyselected.Samplesareusuallydeliberatelychosen(alsoreferredtoaspurposivesamplingwheresubjectsareselectedbecauseofsomecharacteristic),insuchawaythatthecommentsandreactionswillbeasinformation-richaspossible.Thismeansthatforinstancefororganizingamicro-evaluationinordertogaininsightsintothepracticalityofaprototypeofsomelearnermaterials,nexttohigh-achievingstudentsalsoagroupoflow-achievingandagroupofavaragestudentsneedtobeselected.Triangulationisimportanthereinordertoenhancethereliabilityandinternalvalidityofthefindings(cf.Miles&Huberman,1994).Onecouldtriangulatebyusingdifferenttypeofpersons,differenttimes,differentplaces.Theeffectivenessoftriangulationrestsonthepremisethattheweaknessesineachsingledatasourcewillbecompensatedbythecounterbalancingstrengthofanother.Incaseofasummativeevaluation,whenthemainpurposeofanevaluationisto prooftheactualpracticalityandeffectiveness,(quasi-)experimentalresearchdesignswithexperimentalandcontrolsettingsarerequiredwithlargesamplesizes.FormoreinformationonsamplingseeforinstanceCreswell(2008),Denscomb(2007)andMills,Gay,AirasianandAirasian(2008).
Wheninvitingrespondentsforaformativeevaluationitisnecessarytoilluminatetheirrole.Theycouldfulfilltheroleoflearner,criticand/orrevisor(Weston,McAlpine&Bordonaro,1995).Respondentswithalearnerrolearenotspecificallyexpertinthesubjectmatterwhichiscoveredbythematerials.Onecouldthinkofstudentswholearnanewsubject;butalsoteacherswhohavenottaughtinacertainmannerbefore.Inmanycasesexpertsrepresentthiscategoryaswell.Forinstance,educationaltechnologyexpertsdonotalwayshaveexpertiseinthesubjectmatterdomainoftheeducationalintervention.Theywilltaketheroleofalearnerfirst,beforetheywillgivecommentsonmattersrelatedto
![Page 100: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/100.jpg)
an introduction to educational design research98
educationaltechnology(inwhichtheyareexperts).Criticsarerespondentswhoareaskedtocommentonthematerialsfromtheperspectiveoftheirexpertise.Thisgroupconsists,forinstance,ofsubjectmatterexpertsandteacherswhoareinvitedtomakestatementsaboutthedifficultyorreadabilityoflearnermaterials.Revisorswillnotonlygivecommentsonthematerials(likecriticsdo),buttheywillalsoprovidesuggestionsforimprovements.Forinstance,asubjectmatterexpertmayindicatewhattypeof‘state-of-the-artknowledge’ismissinginthelearnermaterialsandwherethisknowledgecouldbefound.Itisimportanttonotethatindividualsmayplayseveralrolessimultaneouslyduringtheformativeevaluation.Thenextsectionwillelaborateontheroleoftheresearchersduringaformativeevaluation.
Researchers’ role during formative evaluation
Sinceadesignresearchprojectcomesintoplaywhenaneedarisestosolveacomplexeducationalproblemforwhichnoready-madesolutionsareavailable,oftentimesamulti-disciplinedteamisbroughttogethertoworkonit.Suchteamsusuallycompriseofexpertsindomainsthatweredistinguishedwhendecomposingtheintervention(e.g.fromaconceptualpointofview:subjectmatterexperts,pedagogicalexperts,instructionaldesigners;fromapresentation-modepointofview:user-interfacedesigners)aswellasmembersofthetargetgroup.Monk,Wright,HaberandDavenport(1993,p.5)stressthat“Itrequiresaccesstopeopletypicalofthosewhowillactuallyusethesystem,nottheirrepresentativesormanagement.”Involvingfutureusersinadesignresearchteamhasseveraladvantages(cf.Moonen,1996;Shneiderman,1992):moreaccurateinformationaboutcomplexityoftheproblemathand,moreintensivediscussionsabouttherequirementsoftheintervention,increaseofusercommitmentandownershipofthefinaldeliverable,increaseofinsightsintotherequirementsofthecontextinwhichtheinterventionwillbeused,andstimulationoftheprofessionaldevelopmentofallparticipants.
Oneofthekeyresponsibilitiesofthedesignresearchteamistoworkontheformativeevaluationoftheprototypes.Forreasonsofscientificrigor,itisoftenrecommendedtolookforexternalevaluators.However,certainlyintheearlystagesofadesignresearchprojectitseemslegitimateorevenadvisablethatdesignresearchersthemselvescarryouttheformativeevaluationoftheprototype.Engaginginformativeevaluationactivitiestendtoleadtoimportantlearningexperiencesofthedesignresearchers.Theywillexperiencethemselvestheproblemsthatoccurandhearoutoffirsthandthesuggestionsforimprovementthatrespondentscomeupwithduringtheiruseofaprototype(forexample,byobservingorinterviewingteachersorstudents).Thisusuallyhasstrongerandmore
![Page 101: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/101.jpg)
an introduction to educational design research 99
directimpactontheirthinkinganddesignactivities,comparedtocaseswhereexternalevaluatorsreporttheresultstothedevelopers.Ofcourse,designresearchersneedtobeawareofseveralpitfallswhentheyareinvolvedintheformativeevaluationoftheinterventiontheyarealsodesigning(cf.McKenney,Nieveen&vandenAkker,2006).Theymayeasilybecometoo‘attached’totheirprototypewhichcouldleadtoalessobjectiveviewtowardproblemsandcommentsfromtherespondents.Inthisrespect,Scriven(1991)warnsofa(co-)authorshipbias.Moreover,respondentscouldbebiasedduringtheevaluation,aswell.Forinstance,iftheyknowhowmucheffortthedesignresearchteamhasputintotheprototype,theymayhesitatetobefullycriticalofit.Toovercomethesebiases,itseemsessentialtoincludeformativeevaluationearlyoninthedevelopmentprocessandtoapplytriangulationofdatasources,methods(observation,interview,questionnaires,etc.),evaluators(differentevaluators)andtheories(differentconceptualframeworks).
Closing remark
Thischapterfocusesontheiterativenatureofeducationaldesignresearch.Eachiterationorcycleofanalysis,designandformativeevaluationgivesthedesignresearchteamfirmergroundandargumentsfortheinterventiontheteamisworkingoninordertosolveacomplexeducationalproblem.Theempiricaldatatheteamcollectsduringaformativeevaluationwillnotonlyprovidesuggestionsforimprovingtheintervention,butwillalsoassistinsharpeningtheaccompanyingdesignprinciples.Proceedingthroughseveraloftheseiterationswillendinafinalstageofthescientificcycleinwhichclaimsofcausalitycanbestudiedinsummativeevaluationsettings(cf.Nieveen,McKenney&vandenAkker,2006).Inthiscontributionweconcentratedontheresearchdesignforeachformativeevaluationperformedwithinsuchaniterativeorprototypingapproach.Weelaboratedontheresearchquestions,selectionofappropriatemethodsandrespondents.Weareawarethatthereismuchmoretosayaboutformativeevaluationingeneral,andintegratedindesignresearchprojectsinparticular.Forinstance,wecouldhavepaidattentiontoevaluationinstruments,datacollection,dataanalysisandreporting.Severalhelpfulbooksandarticlesareavailabletoassistinsystematicallyconductingformativeevaluationineducation(cf.Brinkerhoff,etal.,1983;Flagg,1990;Tessmer,1993).Althoughthesesourceswerenotwrittenwiththespecificneedsandwishesofdesignresearchersinmind,theycanprovideampleinspiration.
![Page 102: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/102.jpg)
an introduction to educational design research100
References
Brinkerhoff,R.O.,Brethouwer,D.M.,Hluchyj,T.,&Nowakowski,J.R.(1983).Program evaluation: A practitioner’s guide for trainers and eduacators. Boston:Kluwer-Nijhoff.
Creswell,J.W.(2008).Educational research: planning, conducting and evaluating quantitative and qualitative research.UpperSaddleRiver,NJ:Pearson.
Denscomb,M.(2007). The good research guide for small-scale social research projects. Maidenhead(UK):OpenUniversityPress.
Design-basedresearchcollective.(2003).Design-basedresearch:Anemergingparadigmforeducationalinquiry.Educational Researcher, 32(1),5-8.
Flagg,B.N.(1990).Formative evaluation for educational technologies.Hillsdale,NJ:LawrenceErlbaumAssociates.
JointCommitteeonStandardsforEducationalEvaluation(1994).The program evaluation standards. ThousandOaks,CA:SagePublications.
Lincoln,Y.&Guba,E.G.(1979).Thedistinctionbetweenmeritandworthinevaluation.Paper presented at the Annual Meeting of the Evaluation Network(5th,Cincinnati,OH,September24-26,1979)retrievedOctober20,2008,http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/32/b5/c8.pdf
Linn,M.,Davis,E.A.,&Bell,P.(2004).Internet environments for science education.Hillsdale:LawrenceErlbaum.
McKenney,S.Nieveen,N.&Akker,J.vanden(2006).Designresearchfromacurriculumperspective.InJ.vandenAkker,K.Gravemeijer,S.McKenney,&N.Nieveen(Eds.),Educational design research(pp.67-90).London:Routledge.
Miles,M.B.,&Huberman,A.M.(1994).Qualitative data analysis: An expanded source book.London:Sage.
Mills,G.,Gay,L.R.,Airasian,P.&Airasian,P.W.(2008).Educational research: competencies for analysis and applications.UpperSaddleRiver,NJ:PrenticeHall.
Monk,A.,Wright,P.,Haber,J.,&Davenport,L.(1993).Improving your human-computerinterface: A practical technique.NewYork:PrenticeHall.
Moonen,J.(1996).Prototypingasadesignmethod.InTj.Plomp,&D.P.Ely(Eds.),International encyclopedia of educational technology(pp.186-190).Oxford:Pergamon.
Nieveen,N.M.(1997).Computer support for curriculum developers: A study on the potential of computer support in the domain of formative curriculum development,Doctoraldissertation.Enschede:UniversityofTwente.
Nieveen,N.(1999).Prototypingtoreachproductquality.InJ.vandenAkker,R.Branch,K.Gustafson,N.Nieveen,andT.Plomp(Eds.),Design approaches and tools in education and training(pp.125–36).Dordrecht:KluwerAcademicPublishers.
Nieveen,N.M.,&Akker,J.J.H.vanden(1999).Exploringthepotentialofacomputertoolforinstructionaldevelopers.Educational technology research & development, 47(3),77-98.
![Page 103: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/103.jpg)
an introduction to educational design research 101
Nieveen,N.,McKenney,S.,&Akker,J.vanden(2006).Educationaldesignresearch:Thevalueofvariety.InJ.vandenAkker,K.Gravemeijer,S.McKenney,andN.Nieveen(Eds.),Educational design research(pp.151-158).London:Routledge.
Reeves,T.(2000).Enhancingtheworthofinstructionaltechnologyresearchthrough“designexperiments”andotherdevelopmentalstrategies.RetrievedOct.20,2006fromhttp://it.coe.uga.edu/~treeves/AERA2000Reeves.pdf
Rossi,P.H.,Freeman,H.E.&Lipsey,M.W.(1999).Evaluation: A systematic approach.ThousandOaks,CA:Sage.
Scriven,M.(1967).Themethodologyofevaluation.InR.W.Tyler,R.M.Gagné,&M.Scriven(Eds.),Perspectives of curriculum evaluation.AERAMonographseriesoncurriculumevaluation.nr.1.Chicago,MI:RandMcNally.
Scriven,M.(1991).Beyondformativeandsummativeevaluation.InM.W.McLaughlin,&D.C.Phillips(Eds.),Evaluation and education: At quarter century(pp.19-64).Chicago:UniversityofChicago.
Shneiderman,B.(1992).Designing the user interface: Strategies for effective human-computer interaction.Reading,MA:Addison-Wesley.
Smith,M.F.(1991).Software prototyping: Adoption, practice and management.London:McGraw-Hill.
Tessmer,M.(1993).Planning and conducting formative evaluations: Improving the quality of education and training.London:KoganPage.
VandenAkker,J.(1999).Principlesandmethodsofdevelopmentresearch.InJ.vandenAkker,R.Branch,K.Gustafson,N.Nieveen,andT.Plomp(Eds.),Design approaches and tools in education and training (pp.1–15).Dordrecht:KluwerAcademicPublishers.
VandenAkker,J.(2003).Curriculumperspectives:Anintroduction.InJ.vandenAkker, W.Kuiper&U.Hameyer(Eds.),Curriculum landscapes and trends(pp.1-10).Dordrecht:
KluwerAcademicPublishers.Wademan,M.(2005).Utilizing development research to guide people capability maturity
model adoption considerations. Unpublisheddoctoraldissertation. Syracuse:SyracuseUniversity.Weston,C.,McAlpine,L.,&Bordonaro,T.(1995).Amodelforunderstandingformative
evaluationininstructionaldesign.Educational training research & technology, 43(3),29-48.
![Page 104: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/104.jpg)
![Page 105: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/105.jpg)
an introduction to educational design research 103
6. ReferencesandSourcesonEducational DesignResearch Tjeerd Plomp and Nienke Nieveen
Introduction
Thisbibliographyhasbeencompiledtosupportresearchersandgraduatestudentsingettingaccesstokeypublicationsondesignresearch.Wedonotclaimthattheselectionofsourcesincludedinthischapteriscompleteandexhaustive–it is coloured by our background and bias as well as our knowledge and familiarity with publications.Importantcriteriaforustoincludetitlesinthisbibliographyare(i)provenusefulnessofsourcesforourownwork,and(ii)representingimportantperspectivesandgroupsthatare(orhavebeen)activelyworkinginthisdomain.Inthefirstsectionwepresentjustanoverviewofrelevantsourcesavailable.Thisisfollowedbyasectioninwhichwepresentthestructureandcontentoftheexcellentwebsite‘Design-basedResearchEPSS’(http://projects.coe.uga.edu/dbr/index.htm)–createdbyInstructionalTechnologyPh.D.studentsatTheUniversityofGeorgia(lastupdateNovember2006).Giventhequalityandcompletenessofthiswebsite(atleasttillNovember2006),wedecidedtointroduceitinaseparatesectionandintheothersectionsofthischapterwewillrefertopartsofthiswebsite,butalsointroduceanumberofothersources.Inthefollowingtwosectionswepointthereadertoselectedjournalarticlesandbookchaptersontheconceptandmethodologyofdesignresearchandondesignresearchindomainssuchascurriculum,instructionaltechnology,andthelearningofreadingandwriting,mathematicsandscience.Inthefinalsection,welisttheURLsofanumberofdoctoralthesesthathavebeendefendedinTheNetherlandsutilizingdesignresearchasaresearchapproach.Asstated,ourselectioniscolouredbyourbiasandexperience,butallthesepublicationsrefertoawiderangeofwritingsondesignresearchandwetrustthattheythereforeserveasausefulintroductiontothereader.
Overview of sources
Thissectionpresentstitlesandreferencestovariousspecialissuesofjournalsandbooksthathavebeenpublishedaboutdesign(-based)research.Besidesanumberofwebsiteswillbelisted.
![Page 106: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/106.jpg)
an introduction to educational design research104
Specialissuesofjournals
• EducationalPsychologist,39(4),2004 Specialissue‘Design-basedresearchmethodsforstudyinglearningincontext’,edited
byW.Sandoval&P.Bell,including: - Sandoval,W.A.,&Bell,P.L.(2004).Design-BasedResearchMethodsForStudying
LearningInContext:Introduction.Educational Psychologist, 39(4),199-201. - Hoadley,C.(2004).Methodologicalalignmentindesign-basedresearch.Educational
Psychologist, 39(4),203-212. - Sandoval,W.A.(2004).Developinglearningtheorybyrefiningconjecturesembodied
ineducationaldesigns.Educational Psychologist, 39(4),213-223. - Tabak,I.(2004).ReconstructingContext:NegotiatingtheTensionbetween
ExogenousandEndogenousEducationalDesign.Educational Psychologist, 39(4),225-233.
- Joseph,D.(2004).ThePracticeofDesign-BasedResearch:UncoveringtheInterplayBetweenDesign,Research,andtheReal-WorldContext.Educational Psychologist, 39(4),235-242.
- Bell,P.L.(2004).Onthetheoreticalbreadthofdesign-basedresearchineducation.Educational Psychologist, 39(4),243-253.
- AlsoincludescommentarybyAngelaO’Donnell.
• EducationalResearcher32(1),January/February2003 SpecialissuepreparedbyA.E.Kelly,including: - Kelly,A.E.(2003).ThemeIssue:TheRoleofDesigninEducationalResearch.
Educational Researcher, 32,3-4. - TheDesign-BasedResearchCollectiveDesign-BasedResearch(2003).AnEmerging
ParadigmforEducationalInquiry.Educational Researcher, 32,5-8. - Cobb,P.,Confrey,J.,diSessa,A.,Lehrer,R.&Schauble,L.(2003).DesignExperimentsin
EducationalResearch.Educational Researcher, 32,9-13. - McCandliss,B.D.,Kalchman,M.&Bryant,P.(2003).DesignExperimentsand
LaboratoryApproachestoLearning:StepsTowardCollaborativeExchange.Educational Researcher, 32,14-16.
- Lobato,J.(2003).HowDesignExperimentsCanInformaRethinkingofTransferandViceVersa. Educational Researcher, 32,17-20.
- Bannan-Ritland,B.(2003).TheRoleofDesigninResearch:TheIntegrativeLearningDesignFrameworkEducational Researcher, 32,21-24.
- Shavelson,R.J.,Phillips,D.C.,Towne,L.,&Feuer,M.J.(2003).OntheScienceofEducationDesignStudies.Educational Researcher, 32,25-28.
![Page 107: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/107.jpg)
an introduction to educational design research 105
- Sloane,F.C&Gorard,S.(2003).ExploringModelingAspectsofDesignExperiments.Educational Researcher, 32,29-31.
- Zaritsky,R.,Kelly,A.E.,Flowers,W.,Rogers,E.,&O’Neill,P.(2003).ClinicalDesignSciences:AViewFromSisterDesignEfforts.Educational Researcher, 32,32-34.
• EducationalTechnology,45(1),2005 SpecialissuepreparedbyC.Dede,including: - Dede,C.(2005).Whydesign-basedresearchisbothimportantanddifficult.
Educational Technology, 45(1),5-8. - Squire,K.D.(2005).Resuscitatingresearchineducationaltechnology:Usinggame-
basedlearningresearchasalensforlookingatdesign-basedresearch.Educational Technology, 45(1),8-14.
- Barab,S.A.,Arici,A.,&Jackson,C.(2005).Eatyourvegetablesanddoyourhomework:Adesign-basedinvestigationofenjoymentandmeaninginlearning.Educational Technology, 45(1),15–21.
- Nelson,B.,Ketelhut,D.J.,Clarke,J.,Bowman,C.,&Dede,C.(2005).Design-basedresearchstrategiesfordevelopingascientificinquirycurriculuminamultiuservirtualenvironment.Educational Technology, 45(1),21–28.
- Kafai,Y.B.(2005).Theclassroomas“livinglaboratory”:Design-basedresearchforunderstanding,comparing,andevaluatinglearningsciencethroughdesign.Educational Technology, 45(1),28–34.
- Hay,K.E.,Kim,B.,&Roy,T.C.(2005).Design-basedresearch:Morethanformativeassessment?AnaccountoftheVirtualSolarSystemProject.Educational Technology, 45(1),34-41.
- Hoadley,C.(2005).Design-basedresearchmethodsandtheorybuilding:AcasestudyofresearchwithSpeakEasy.Educational Technology, 45(1),42-47.
- Reeves,T.C.(2005).Design-basedresearchineducationaltechnology:Progressmade,challengesremain.Educational Technology, 45(1),48-52
• JournaloftheLearningSciences,13(1),2004 Specialissue,including: - Barab,S.,&Squire,K.(2004).Design-BasedResearch:PuttingaStakeintheGround.
Journal of the Learning Sciences, 13(1),1-14. - Collins,A.,Joseph,D.,&Bielaczyc,K.(2004).DesignResearch:Theoreticaland
MethodologicalIssues.Journal of the Learning Sciences, 13(1),15-42. - Fishman,B.,Marx,R.W.,Blumenfeld,P.,Krajcik,J.,&Soloway,E.(2004).Creatinga
FrameworkforResearchonSystemicTechnologyInnovations.Journal of the Learning Sciences, 13(1),43-76.
![Page 108: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/108.jpg)
an introduction to educational design research106
- diSessa,A.A.,&Cobb,P.(2004).OntologicalInnovationandtheRoleofTheoryinDesignExperiments.Journal of the Learning Sciences, 13(1),77-103
- Dede,C.(2004).IfDesign-BasedResearchistheAnswer,WhatistheQuestion?ACommentaryonCollins,Joseph,andBielaczyc;diSessaandCobb;andFishman,Marx,Blumenthal,Krajcik,andSolowayintheJLSSpecialIssueonDesign-BasedResearch.Journal of the Learning Sciences, 13(1),105-114.
- Kelly,A.(2004).DesignResearchinEducation:Yes,butisitMethodological?Journal of the Learning Sciences, 13(1),115-128.
Books• VandenAkker,J.,Gravemeijer,K,McKenney,S.&Nieveen,N.(Eds).(2006).Educational
design research.London:Routledge.ISBN10:0-415-39635-2(pbk)(163pages) Availableathttp://www.taylorandfrancis.co.uk/shopping_cart/products/product_
detail.asp?sku=&ppid=118302&isbn=9780415396356 ThisbookcomprisesthepaperspresentedataseminarorganizedbytheNetherlands
OrganizationforScientificResearch,inparticularbytheProgramCouncilforEducationalResearch.Theseminar,conductedinDecember2003,hasbeenameetingplaceofdesignresearchersfromtheUSAandTheNetherlands.Thebookreflectsthevariousanglesfromwhichresearchersinthedomainsofcurriculum,instructionaltechnologyand(mathematicsandscience)educationaddresstheneedtodevelopresearchbasedsolutions(interventions)toproblemsforwhichnoguidelinestosolutionsareavailable.Thebookillustratesthatauthorswithvariousbackgroundshaveclearlyacommongroundwhenreflectingondesignresearchasaresearchapproach.
Thebookhasfourparts: Part1.Whatandwhy 1. IntroducingEducationalDesignResearch-JanvandenAkker,KoenoGravemeijer,
SusanMcKenney,NienkeNieveen 2. TowardProductiveDesignStudies-DeckerWalker Part2.Examplesfromthefield 3. DesignresearchfromtheLearningDesignPerspective-KoenoGravemeijer,Paul
Cobb 4. DesignResearchfromtheTechnologyPerspective-ThomasReeves 5. DesignResearchfromaCurriculumPerspective-SusanMcKenney,NienkeNieveen,
JanvandenAkker
![Page 109: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/109.jpg)
an introduction to educational design research 107
Part3.Quality 6. AssessingtheQualityofDesignResearchProposals:SomePhilosophicalPerspectives
-D.C.Phillips 7. BalancingInnovationandRisk:AssessingDesignResearchProposals-DanielC.
Edelson 8. QualityCriteriaforDesignResearch:EvidenceandCommitments-AnthonyE.Kelly Part4.Movingahead 9. FromDesignResearchtoLarge-ScaleImpact:EngineeringResearchinEducation-
HughBurkhardt 10. EducationalDesignResearch:TheValueofVariety-NienkeNieveen,SusanMcKenny,
JanvandenAkker
• Kelly,A.E.,Lesh,R.A.&Baek,J.Y.(Eds).(2008).Handbook of Design Research Methods in Education Innovations in Science, Technology, Engineering, and Mathematics Learning and Teaching.NewYork:LawrenceErlbaumAssociates.ISBN:978-0-8058-6059-7(pbk)(560pages)
Availableathttp://www.routledgeeducation.com/books/Handbook-of-Design-Research-Methods-in-Education-isbn9780805860597
Theannouncementofthebookstatesthatthehandbookpresentsthelatestthinkingandcurrentexamplesofdesignresearchineducation.Design-basedresearchinvolvesintroducinginnovationsintoreal-worldpractices(asopposedtoconstrainedlaboratorycontexts)andexaminingtheimpactofthosedesignsonthelearningprocess.Designedprototypeapplications(e.g.,instructionalmethods,softwareormaterials)andtheresearchfindingsarethencycledbackintothenextiterationofthedesigninnovationinordertobuildevidenceoftheparticulartheoriesbeingresearched,andtopositivelyimpactpracticeandthediffusionoftheinnovation.
TheHandbook of Design Research Methods in Educationismeanttofillaneedinhowtoconductdesignresearchbythosedoingsorightnow.Thechaptersrepresentabroadarrayofinterpretationsandexamplesofhowtoday’sdesignresearchersconceptualizethisemergentmethodologyacrossareasasdiverseaseducationalleadership,diffusionofinnovations,complexitytheory,andcurriculumresearch.
Thehandbookhaseightsections: - Designresearchanditsargumentativegrammar - Modelingstudentlearningduringdesignresearch - Modelingteacherlearningusingdesignresearch - Modelingstakeholderscommitmentsusingdesignresearch - Reflectingondesignresearchattheprojectlevel
![Page 110: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/110.jpg)
an introduction to educational design research108
- Reflectingondesignresearchattheprogramlevel - Extendingdesignresearchmethodologically - Trackingthediffusionofdesignresearch.
• Reinking,D.&Bradley,B.A.(2008).On Formative and Design Experiments: Approaches to Language and Literacy Research.NewYork&London:TeachersCollege,ColumbiaUniversity.ISBN:978-0-8077-4841-1(pbk)(134pages)
Thisbookletprovidesaniceintroductionintoformativeanddesignexperiments,atermsynonymousforwhatwecalldesignresearchandothersdesign-basedresearch.Itprovidesathorough,butpracticalandusefuloverviewofdesignresearchaddressingthefollowingquestions:
- Whatareformativeanddesignexperiments(Ch1)? - Whatarethemethodsofformativeanddesignexperiments(Ch2)? - Whataresomegoodexamplesofformativeanddesignexperiments(Ch3)? - Isthereaformativeordesignexperimentinyourfuture(Ch4)?
• Richey,R.&Klein,J.D.(2007).Design and development research: methods, strategies, and issues.London:Routledge.ISBN080585732X,9780805857320(180pages)
Thisvolumediscussesmethodsandstrategiesappropriateforconductingdesignanddevelopmentresearch.Richwithexamplesandexplanations,thebookdescribesactualstrategiesthatresearchershaveusedtoconducttwomajortypesofdesignanddevelopmentresearch:1)productandtoolresearchand2)modelresearch.Commonchallengesconfrontedbyresearchersinthefieldwhenplanningandconductingastudyareexploredandproceduralexplanationsaresupportedbyawidevarietyofexamplestakenfromcurrentliterature.
Websites• http://projects.coe.uga.edu/dbr/index.htm(lastupdateNovember2006): titled‘Design-basedResearchEPSS’–createdbyInstructionalTechnologyPh.D.students
atTheUniversityofGeorgiaundersupervisionofTomReeves(comprehensivetilllastupdateofNovember2006).Thiswebsiteissummarizedinthenextsection.
• http://cider.athabascau.ca/CIDERSIGs/DesignBasedSIG/dbrreferences(lastupdateearly2005):
ThisbibliographyisdrawnupbyTerryAndersonoftheUnivesityofAthabasca(Edmonton,Alberta,Canada).Andersoncallsitasnapshotofmostcurrent(early2005)literaturerelatedtodiscussion,explorationandexamplesofdesign-basedresearch.ThereferencesarepresentedwithURLs(ifavailable)alongwithabstractsandoccasionallyquotationsorannotationsbyAnderson.HasmuchoverlapwiththeUniversityofGeorgiawebsite.
![Page 111: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/111.jpg)
an introduction to educational design research 109
• http://www.designbasedresearch.org/index.html(lastupdatenotclear,butnoreferenceslaterthan2004)
ThisisthewebsiteoftheDesign-BasedResearchCollective,asmallgroupofresearcherswhoengageindesign-basedresearch,oftenintechnologyenhancedlearningenvironments.Itcontainsreferencesofanumberofpublications,aswellasanumberoflinkstorelevantrelatedwebsites.OtherpublicationsApartfromthesourcesmentionedabove,manyarticlesandbookchaptershavebeenpublisheddealingwithconceptualand/ormethodologicalaspectsofdesignresearch,orreportingaboutdesignresearchprojects.Manyofthesereferences(plusabstracts)canbefoundonthewebsitesmentionedinthissection,butwehaveselectedanumberwhicharesummarizedinthefinalsectionofthischapter.
UGA Website ‘Design-based Research EPSS’, November 2006
TheURLoftheUGAwebsiteElectronicPerformanceSupportSystem(EPSS)is:http://projects.coe.uga.edu/dbr/index.htm.Thewebsite(November2006),createdbyInstructional Technology Ph.D. students at The University of Georgia(UGA),supervisedbyTomReeves1,hasthreeparts:1. PEERTutorial2. Webliography3. ExpertInterviews
1.PEERTutorialThisusefultutorialiscomposedoffourprimarysections:(i) tutorialsurvey,(ii) explanation,(iii)enactment,and(iv)reflection.Parts(ii)and(iii)areusefulandinformativetobecomefamiliarwithdesignresearchandhowtogetstarted.(ii)Explanation:Thepurposeoftheexplanationpartofthetutorialistoprovidetheuserwithfundamentalknowledgeandinsightaboutdesignresearchcomposedoffivesections:• WhatisDesign-basedResearch(DBR)?• HowdoesDBRdifferfromotherapproaches?• HowdidDBRgetstarted?• WhatarethebenefitsofDBR?
1) TheauthorsofthiswebsiteusethetermDesignBasedResearch(DBR),whilstweusethroughoutthisbookletthe term‘DesignResearch’:bothtermsshouldbeseenassynonymous.
![Page 112: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/112.jpg)
an introduction to educational design research110
• Whataresomecriticalperspective?Eachsectionisconciseandprovidescoreinformationwithamplereferencestoliterature.(iii)Enactment:Thispartconsistsofthreesections:- HowdoIgetstartedwithDesign-BasedResearch(DBR)?- SomeexamplesofDBR- WhatarethechallengesofdoingDBR?EachsectiondiscussesanumberofrelevanttopicsforthosewhowanttoconductDesignResearch(DR)orDesign-BasedResearch.Asstated,thisisausefultutorial.Butkeepinmindthatauthorsmaydifferinemphasis,approachand/oruseoftermsandconcepts.Butwhenyouareanopen-minded,criticalreaderyouwillfindmanyusefulideasandsuggestionsinthistutorial.
2.WebliographyThepurposeofthispartofthewebsiteis–accordingtoitscreators–toprovidevarioustypesofresourcesthatmayprovideinterestedresearchersabeginningpointforinvestigatingandpursuingthetopicofdesign-basedresearch.Warning:Asthewebsitehasbeenpreparedin2006,itmaybepossiblethatsomeoftheURLsreferredtoarenolongeraccessibleoractive.Thewebliographyhasthefollowingsections:(i) GlossaryAlimitednumberofkeyconceptsaredefined,themostimportantbeingdesign-basedresearch.(ii)PrintedresourcesOnebookislisted(VandenAkker,etal.,2006)andquiteanumberofjournalarticles(andtheirabstracts),dividedintomethodologicalarticlesandresearchexamples.Thisisausefullistofpublicationsup‘tillNovember2006,andespeciallythosepublishedinNorthAmerica.(iii)OnlineresourcesThissectionhasanumberofsub-sections:anumberofusefulwebsites,twoonlinejournals(toolimitedtobereallyuseful),andtheURLsofanumberofusefularticles.(iv)OrganizationsTwoorganizationsarementioned,viz.Design-basedResearchCollective(DBRC)andtheDesign-basedResearchSIGoftheCanadianInstituteofDistanceEducationResearch,butthewebsitesofbothorganizationsseemnottobeuptodate.Nevertheless,thewebsiteofDBRC(http://www.designbasedresearch.org/index.html)givesusefulreferencestotwospecialissuesofjournals:
![Page 113: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/113.jpg)
an introduction to educational design research 111
• Kelly,E.A.(Ed.).(2003).Theroleofdesignineducationalresearch[specialissue].Educational Researcher, 32 (1).• Sandoval,W.&Bell,P.(Eds.).(2004).Design-basedresearchmethodsforstudying
learningincontext[SpecialIssue].Educational Psychologist, 39(4)2.
(v)Expertsindesign-basedresearchContainsshortbiographicalnotesandapictureofanumberofexpertsindesignresearch(amongstthemallexceptthefirstauthorofthisbooklet).(vi)FrequentlyAskedQuestions(FAQ):Threequestionsareaddressed,vizwhatisdesign-basedresearch(DBR),howtobeginwithDBR,andhowdoesDBRdifferfromotherresearchmethodologies.
3.ExpertinterviewsThispartofthewebsitecontainsanumberofvideotapedinterviewswithanumberofexpertsindesignresearch.
Selected journal articles and book chapters on concept and methodology
Therearesomanypublicationsoneducationaldesignresearchthatitisimpossibletodrawupacomprehensivebibliography.Anumberofpublicationshavealreadybeenlisted(withabstracts)ontheUGAwebsite(see6.2).Howeverwewanttopointthereadertoanumberofarticlesandchapterthathavehelpedustogetinvolvedindesignresearchandtounderstandthemainissuesinourfield.Giventhisrationaleforselectingthesetitles,thereaderwillfindthatsomeofthetitlesarealsoreferredtoontheUGAwebsite.
Akker,J.vanden(1999).Principlesandmethodsofdevelopmentresearch3.InJ.vandenAkkeretal.(Eds.),Design approaches and tools in education and training(pp.1-14).Dordrecht:KluwerAcademicPublishers. Abstract:Thischapterdiscussestheroleofresearchinrelationtoeducationaldesign
anddevelopmentactivities.Thefirstpartofthechapterfocusesontherationaleandbasicprinciplesofdevelopmentresearchbyoutliningmotivesforconductingformativeresearch,analyzingdefinitionsandaimsofvarioustypesofdevelopmentresearch,anddiscussingseveralofitskeycharacteristics.Thesecondpartofthechapterdealswith
2) Thewebsiteofthisspecialissueofferstheopportunitytopurchasearticlesforb22.00plusVAT.3) Theconceptdevelopmentresearch,usedinsometitles,issynonymoustodesignresearch.
![Page 114: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/114.jpg)
an introduction to educational design research112
methodsofdevelopmentresearch,exploringsomeofitstypicalproblemsanddilemmas,anddiscussingseveralchallengesforfurtheractionandreflection.
Akker,J.vanden&Plomp,Tj.(1993).Developmentresearchincurriculum:propositionsandexperiences.PaperpresentedattheannualconferenceoftheAmericanEducationalResearchAssociation,April1993,Atlanta(GA,USA). Abstract:Reasontoincludethispaperisthatitisthefirstpaperfromthegroupatthe
UniversityofTwenteonwhattheycalledatthattimedevelopmentresearch.Basedontheassertionthatbothcurriculumdevelopmentandcurriculumresearchhavemuchrelevancetogainfromacloseliaison,theauthorssuggestthatboundariesbetweenthetwoshouldfade,whichcanbedoneinanewresearchstrategycalleddevelopmentresearch.Thepaperpresentsthepurpose,aconceptualframeworkandsomecharacteristicsofdevelopmentresearchincurriculum
See:www.leerplanevaluatie.slo.nl/taakhulp/lezen
Bannan-Ritland,B.(2003).Theroleofdesigninresearch:Theintegrativelearningdesignframework. Educational Researcher, 32(1),21-24. Abstract(fromUGAwebsite):Inthisarticle,ageneralmodelisproposedfordesign
researchineducationthatgrowsoutoftheauthor’sresearchandworkinrelateddesignfields.Themodelemphasizesthestagesensitivityof(a)researchquestions,(b)dataandmethods,and(c)theneedforresearcherstodesignartifacts,processes,andanalysesatearlierstagesintheirresearchthatcanthenbeprofitablyused(perhapsbydifferentresearchers)inlaterstages.
Barab,S.A.,andSquire,K.D.(2004).Design-BasedResearch:PuttingaStakeintheGround.Journal of the Learning Sciences, 13(1),1-14. Abstract(fromUGAwebsite):Thearticlehighlightsandproblematizessomechallenges
thatarefacedincarryingoutdesign-basedresearch.Itstatesthattheemergingfieldoflearningsciencesisonethatisinterdisciplinary,drawingonmultipletheoreticalperspectivesandresearchparadigmssoastobuildunderstandingsofthenatureandconditionsoflearning,cognitionanddevelopment.Afundamentalassumptionofmanylearningscientistsisthatcognitionisnotathinglocatedwithintheindividualthinkerbutisaprocessthatisdistributedacrosstheknower,theenvironmentinwhichknowingoccursandtheactivityinwhichthelearnerparticipates.Inotherwords,learning,cognition,knowingandcontextareirreduciblyco-constitutedandcannotbetreatedasisolatedentitiesorprocesses.
![Page 115: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/115.jpg)
an introduction to educational design research 113
Brown,A.L.(1992).Designexperiments:Theoreticalandmethodologicalchallengesincreatingcomplexinterventionsinclassroomsettings.Journal of the Learning Sciences, 2,141-178. Abstract(fromUGAwebsite):Thisistheseminalarticleondesignresearch.Discusses
theoreticalandmethodologicalchallengesincreatingcomplexinterventionsinclassroomsettings.Movementfromtheclassicalpsychologicalpositionofconcentratingonatheoreticalstudyofthelearningprocessesofindividualstudentstoaconcentrationonconceptualchangeinteachersandstudents;Classroomrestructuring;Designexperiments;Experiencesonlearningtheory.
Design-BasedResearchCollective.(2003).Design-basedresearch:Anemergingparadigmforeducationalinquiry.Educational Researcher, 32(1),5-8. Abstract(fromUGAwebsite):Theauthorsarguethatdesign-basedresearch,which
blendsempiricaleducationalresearchwiththetheory-drivendesignoflearningenvironments,isanimportantmethodologyforunderstandinghow,when,andwhyeducationalinnovationsworkinpractice.Designbasedresearchers’innovationsembodyspecifictheoreticalclaimsaboutteachingandlearning,andhelpusunderstandtherelationshipsamongeducationaltheory,designedartifact,andpractice.Designiscentralineffortstofosterlearning,createusableknowledge,andadvancetheoriesoflearningandteachingincomplexsettings.Designbasedresearchalsomaycontributetothegrowthofhumancapacityforsubsequenteducationalreform.
Kelly,A.E.(2006).Qualitycriteriafordesignresearch.In:J.vandenAkker,K.Gravemeijer,S.McKenney,&N.Nieveen(Eds.).Educational design Research.London:Routledge. Abstract:thischapterdiscussesforeachofthreedifferentusesfordesignresearchin
educationanumberofcharacteristicsandexemplaryexamples.Itintroducesthenotionofthecommissivespaceofdesignresearch,meaningthat(amongstothercharacteristics)designresearchdoesnotstriveforcontext-freeclaimsbutseescontextsascentraltoitsconceptualdomain,thatdesignresearchisexperimentalbutnotanexperiment,andthatdesignresearcherschoosetoworkinthe“contextofdiscovery”,ratherthaninthe“contextofverification”utilizingrandomizedtrials.
Reeves,T.(2000).Enhancingtheworthofinstructionaltechnologyresearchthrough“designexperiments”andotherdevelopmentalstrategiesPaperpresentedattheAnnualMeetingoftheAmericanEducationalResearchAssociation,April2000,NewOrleans(LA,USA).RetrievedOct.20,2006fromhttp://it.coe.uga.edu/~treeves/AERA2000Reeves.pdf
![Page 116: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/116.jpg)
an introduction to educational design research114
Abstract:Theauthorarguesthatingeneralresearchintheareaofinstructionaltechnologyispoor,notprovidingpractitionerswithsufficientguidance.Hediscussesvarioustypesofinstructionaltechnologyresearchgoalsandmethodsandsuggeststhat‘use-inspiredbasicresearch’isneededinthedomainofinstructionaltechnologyreferringtoapproacheslikedevelopmentresearchanddesignexperiments.Hepresentsaframeworkandcharacteristicsfordevelopmentresearchintheareaofinstructionaltechnology.
Selected journal articles and book chapters on design research in domains
Overthelastfewyears,increasinglyexamplesofdesignresearchhavebeenpublished.Thissectioncontainsjustafewexemplaryreferencestoarticlesandchaptersinbooksofdesignresearchinvariousdomains,ofwhichafewaretakenfromtheUGAwebsiteWehaveaddedinthenextsectionreferencestosomePhDdissertationsreportingondesignresearchconductedatDutchuniversitiesthatcaneasilybeaccessedthroughtheWorldWideWeb.
DomainofmathematicseducationAseminalchapteris:Gravemeijer,K.&Cobb,P.(2006).Designresearchfromthelearningdesignperpective,invandenAkker,K.Gravemeijer,S.McKenney,&N.Nieveen(Eds.)EducationalDesignresearch:Thedesign,developmentandevaluationofprograms,processesandproducts.London:Routledge,17-51. Abstract:thischapterpresentsanapproachtodesignresearchthathasbeenusedand
refinedinaseriesofdesignresearchprojectsinwhichthetwoauthorscollaboratedoveraten-yearperiod.Theirapproachisfallingwithinthebroadercategoryofdesignresearchthataimsatcreatinginnovativelearningecologiesinordertodeveloplocalinstructiontheoriesontheonehand,andtostudytheformsoflearningthatthoselearningecologiesareintendedtosupportontheotherhandinthedomainofmathematicseducation(includingstatisticseducation).
Theapproachtodesignresearchhasitsrootsinthehistoryofthetwoauthorswhichistheworkonrealisticmathematicseducation(RME)thatiscarriedoutintheNetherlands(firstauthor)andthatofsocio-constructivistanalysisofinstruction(secondauthor).
Somereferences4fordesignresearchcasesinthedomainofmathematicseducationconductedintheUSAare:
4) WewanttoexpressourthankstoPaulCobbforprovidingthesereferences.
![Page 117: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/117.jpg)
an introduction to educational design research 115
Bowers,J.S.,Cobb,P.,&McClain,K.(1999).Theevolutionofmathematicalpractices:Acasestudy.Cognition and Instruction, 17,25-64.Cobb,P.(1999).Individualandcollectivemathematicallearning:Thecaseofstatisticaldataanalysis.Mathematical Thinking and Learning, 1,5-44.Cobb,P.,McClain,K.,&Gravemeijer,K.(2003).Learningaboutstatisticalcovariation.Cognition and Instruction, 21,1-78.Confrey,J.,&Smith,E.(1995).Splitting,covariation,andtheirroleinthedevelopmentofexponentialfunctions.Journal for Research in Mathematics Education, 26,66-86.Lehrer,R.,&Schauble,L.(2004).Modelingnaturalvariationthroughdistribution.American Educational Research Journal, 41,635-679.Lobato,J.(2003).Howdesignexperimentscaninformarethinkingoftransferandviceversa.Educational Researcher, 32(1),17-20.Simon,M.A.(1995).Reconstructingmathematicspedagogyfromaconstructivistperspective.Journal for Research in Mathematics Education, 26,114-145.Stephan,M.,Bowers,J.,&Cobb,P.(Eds.).(2003).Supporting students’ development of measuring conceptions: Analyzing students’ learning in social context. Journal for Research in Mathematics Education Monograph No. 12. Reston,VA:NationalCouncilofTeachersofMathematics.
Anillustrativeexampleofdesignresearchinthecontextofadevelopingcountryis:Vos,P.,Devesse,T.G.,andPinto,A.A.R.(2007).DesigningMathematicsLessonsInMozambique:StartingFromAuthenticResources.African Journal of Research in SMT Education, 11(2),pp.51-66 Abstract:Thisarticledescribesresearchonthedesignofstudent-centredinstructionin
Mozambique.Thestartingpointwastheuseofreal-liferesources,suchastraditionalartcraftobjectsandauthenticnewspaperclippings.Theresearchwasbasedonaninstructionaldesignmodel,whichattemptstoaligntheorywithpracticeandwhichisgearedtowardsimprovingpractice.Intwoparallelstudies,oneongeometryandoneonstatistics,student-centredinstructionwasfacilitatedthroughtheuseofworksheetswithopen-endedquestionstailoredforgroupwork.Inacyclicprocess,theprototypematerialsandtheassociatedinstructionalmethodwereformativelyevaluated.Theevaluationsshowedthatthedesignswereusefuleveninclassroomspackedwithmorethansixtystudents.
DomainofscienceeducationHoadley,C.M.,&Linn,M.C.(2000)Teachingsciencethroughonline,peerdiscussions:SpeakEasyintheknowledgeintegrationenvironment.International Journal of Science Education,22(8),839-857.
![Page 118: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/118.jpg)
an introduction to educational design research116
Abstract:Thisarticlediscusseswhetherstudentscanlearnsciencefromcarefullydesignedonlinepeerdiscussions.Contraststwoformatsofcontributedcomments--historicaldebateandnarrativetext--andassessestheimpactofanasynchronousdiscussiononstudentunderstandingofthenatureoflight.Italsoreportsthatstudentsgainintegratedunderstandingofthenatureofcolorfrombothdiscussionformats.
Kafai,Y.B.,&Ching,C.C.(2001).Affordancesofcollaborativesoftwaredesignplanningforelementarystudents’sciencetalk.Journal of the Learning Sciences, 10(3),323-363. Abstract:Thisarticleinvestigateswhethersciencepermeatesthedesignenvironment
andisthuscontextedwithintheotheractivitiesofcollaborativemanagementandtechnology.Focusesonwhichcontextsgaverisetosciencetalk.Studiesaclassroomwith(n=33)studentsdividedintoseventeams
Knippels,M.C.P.J.,Waarlo,A.J.,andBoersma,K.Th.(2005).Designcriteriaforlearningandteachinggenetics.Journal of Biological Education,39(3),108-112. Abstract:Whilelearningandteachingdifficultiesingeneticshavebeenabundantly
exploredanddescribed,therehasbeenlessfocusonthedevelopmentandfield-testingofstrategiestoaddressthem.Toinformthedesignofsuchastrategyareviewstudy,focusgroupinterviewswithteachers,acasestudyofatraditionalseriesofgeneticslessons,studentinterviews,andcontentanalysisofschoolgeneticsteachingwerecarriedout.SpecificdifficultiesreportedintheliteraturewerecomparabletothoseperceivedbyDutchteachersandfoundinthecasestudyandthestudentinterviews.Theproblemsassociatedwiththeabstractandcomplexnatureofgeneticswerestudiedinmoredetail.Theseparationofinheritance,reproductionandmeiosisinthecurriculumaccountsfortheabstractnatureofgenetics,whilethedifferentlevelsofbiologicalorganisationcontributetoitscomplexnature.Finally,fourdesigncriteriaaredefinedforalearningandteachingstrategytoaddresstheseproblems:linkingthelevelsoforganism,cellandmolecule;explicitlyconnectingmeiosisandinheritance;distinguishingthesomaticandgermcelllineinthecontextofthelifecycle;andanactiveexplorationoftherelationsbetweenthelevelsoforganisationbythestudents.
Key words:Biologyeducation;Genetics;Learningandteachingdifficulties;Designcriteria
Lijnse,P.L.(1995).“DevelopmentalResearch”asawaytoanempiricallybased“DidacticalStructure”ofScience.Science Education, 29(2),189-199. Abstract:Theauthorarguesthatdevelopmentalresearch(inthisbookcalled´design
research’)isneededinwhichsmall-scalecurriculumdevelopmentiscyclicallycoupledtoin-depthclassroomresearchofteaching-learningprocesses.Suchresearchshouldresultinworkedoutexamplesofsuccessfulwaysofteaching,accordingtonew
![Page 119: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/119.jpg)
an introduction to educational design research 117
conceptualcurriculumstructures.Designingsuch`didactical´structuresconstitutesalongertermresearchprogram,whichasksforinternationalexchangeandcooperatition.
Domainofreading-writingAbbott,S.P.,Reed,E.,Abbott,R.D.,&Berninger,V.W.(1997).Year-longbalancedreading/writingtutorial:Adesignexperimentusedfordynamicassessment.Learning Disability Quarterly,20(3),249-263. Abstract:Sixteenchildrenwithseverereadingproblemsinfirstgradereceivedayear-
longindividualtutorialintervention.Growthcurveanalysesfoundsignificantgainsonmeasuresoforthographicandphonologicalcoding,wordidentification,wordattackskills,readingcomprehension,letterautomaticity,andspellingandmarginallysignificantgainsinwritingcomposition.
DeCorte,E.,Verschaffel,L.,&vandeVen,A.(2001).Improvingtextcomprehensionstrategiesinupperprimaryschoolchildren:Adesignexperiment.The British Journal of Educational Psychology,71,531-559. Abstract:Withrespecttotheacquisitionofcompetenceinreading,newstandardsfor
primaryeducationstressmorethanbeforetheimportanceoflearningandteachingcognitiveandmetacognitivestrategiesthatfacilitatetextcomprehension.Therefore,thereisaneedtodesignaresearch-basedinstructionalapproachtostrategicreadingcomprehension.Thedesignexperimentaimedatdeveloping,implementingandevaluatingaresearch-based,butalsopracticallyapplicablelearningenvironmentforenhancingskilledstrategyuseinupperprimaryschoolchildrenwhenreadingatext.Thisdesignexperimentshowsthatitispossibletofosterpupils’useandtransferofstrategicreadingcomprehensionskillsinregularclassroomsbyimmersingtheminapowerfullearningenvironment.Butthisinterventiondoesnotautomaticallyresultinimprovementofperformanceonastandardizedreadingcomprehensiontest.
Neuman,S.B.(1999).Booksmakeadifference:Astudyofaccesstoliteracy.Reading Research Quarterly,34(3),286-311. Abstract:Thisarticleexaminestheimpactofaninterventiontargetingeconomically
disadvantagedchildrenthatfloodedover330child-carecenterswithhigh-qualitychildren’sbooksandprovided10hoursoftrainingtochild-carestaff.Itexaminestheproject’simpactandgivessupportforthephysicalproximityofbooksandthepsychologicalsupporttochild-carestaffonchildren’searly-literacydevelopment.
DomainofinstructionaltechnologyBannan-Ritland,B.(2003).Theroleofdesigninresearch:Theintegrativelearningdesignframework.Educational Researcher,32(1),21-24.
![Page 120: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/120.jpg)
an introduction to educational design research118
Abstract:(fromUGAwebsite):Inthisarticle,ageneralmodelisproposedfordesignresearchineducationthatgrowsoutoftheauthor’sresearchandworkinrelateddesignfields.Themodelemphasizesthestagesensitivityof(a)researchquestions,(b)dataandmethods,and(c)theneedforresearcherstodesignartifacts,processes,andanalysesatearlierstagesintheirresearchthatcanthenbeprofitablyused(perhapsbydifferentresearchers)inlaterstages.
Herrington,J.,&Oliver,R.(1997).Multimedia,magicandthewaystudentsrespondtoasituatedlearningenvironment.Australian Journal of Educational Technology,13(2),127-143.Availableat:http://www.ascilite.org.au/ajet/ajet13/herrington.html Abstract:Thisarticlepresentsadesignofaninteractivemultimedialearning
environmententitledInvestigatingassessmentstrategiesinmathematicsclassrooms,whichrepresentstheoperationalizedcharacteristicsofsituatedlearning.Theauthorsalsosuggestthecriticalguidelinesforthedesignofthemultimediasoftwaretoenableittosupportasituatedlearningenvironment.Theythenreportastudythatinvestigatespatternsofbehaviorofstudentsimmersedinthismultimediasituatedlearningenvironment.Thefindingssuggestthattheuseofthesituatedlearningmodelissuccessfulinprovidingguidelinesforthedevelopmentofaninteractivemultimediaprogram.Theyalsorevealthatininstanceswherelearnersareempoweredandareenabledtoassumehigherdegreesofresponsibilityfortheiractivityandconductinalearningsetting,theresearchersneedtobecognizantofthevariousdesignfactorswhichcanimpedeorenhancelearning.Inmultimediaenvironments,theseincludesuchelementsasthemotivationalaspectsoftheenvironment,theinterfacedesign,andthenavigationelementsemployed.Inconclusion,theauthorssuggestthatitisalsoimportanttopracticeresearchwhichexplorestheimpactofthemoretangibleaspectsofmultimediadesignsuchasthoseexploredinthisstudy.
Herrington,J.,&Oliver,R.(2000).Aninstructionaldesignframeworkforauthenticlearningenvironments.Educational Technology Research and Development,48(3),23-48.Availableat:http://edserver2.uow.edu.au/~janh/Assessment/Authentic%20Assessment_files/ETR%26D.pdf Abstract:Theinstructionaltechnologycommunityisinthemidstofaphilosophical
shiftfromabehaviouristtoaconstructivistframework,amovethatmaybegintoaddressthegrowingriftbetweenformalschoollearningandreal-lifelearning.Onetheoryoflearningthathasthecapacitytopromoteauthenticlearningisthatofsituatedlearning.
Thepurposeofthisthreepartstudywasfirstly,toidentifycriticalcharacteristicsofasituatedlearningenvironmentfromtheextensiveliteraturebaseonthesubject;secondly,tooperationalisethecriticalcharacteristicsofasituatedlearningenvironmentbydesigningamultimediaprogramwhichincorporatedtheidentifiedcharacteristics;
![Page 121: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/121.jpg)
an introduction to educational design research 119
andthirdly,toinvestigatestudents’perceptionsoftheirexperiencesusinganmultimediapackagebasedonasituatedlearningframework.
Thelearningenvironmentcomprisedamultimediaprogramforpreserviceteachersonassessmentinmathematics,togetherwithrecommendedimplementationconditionsintheclassroom.Eightstudentswereobservedandinterviewedtoexploretheirperceptionsofthesituatedlearningenvironment.Findingssuggestthattheuseofthesituatedlearningframeworkappearedtoprovideeffectiveinstructionaldesignguidelinesforthedesignofanenvironmentfortheacquisitionofadvancedknowledge.
Reeves,T.(2006).Designresearchfromatechnologyperspective.In:J.vandenAkker,K.Gravemeijer,S.McKenney,&N.Nieveen(Eds.).Educational design Research.London:Routledge. Abstract:Theeffectivenessofthefieldknownaseducationaltechnologyin
fundamentallyenhancingteachingandlearninghasincreasinglybeencalledintoquestion,ashastheefficacyofeducationalresearchingeneral.Doubtsabouteducationaltechnologyresearchprimarilystemfromdecadesofanarguablyflawedresearchagendathathasbeenbothpseudoscientificandsociallyirresposible.Itisproposedthatprogressinimprovingteachingandlearningthroughtechnologymaybeaccomplishedusingdesignresearchasanalternativemodelofinquiry.Designresearchprotocolsrequireintensiveandlong-termcollaborationinvolvingresearchersandpractitioners.Itintegratesthedevelopmentofsolutionstopracticalproblemsinlearningenvironmentswiththeidentificationofreusabledesignprinciples.Examplesofdesignresearchendeavorsineducationaltechnologyaredescribedhere.Thechapterendswithacallfortheeducationaltechnologyresearchcommunitytoadoptdesignresearchmethodsmorewidely.
Reeves,T.C.,Herrington,J.,&Oliver,R.(2004).Adevelopmentresearchagendaforonlinecollaborativelearning.Educational Technology Research and Development, 52(4),53-65. Abstract:Althoughimportant,traditionalbasic-to-appliedresearchmethodshave
providedaninsufficientbasisforadvancingthedesignandimplementationofinnovativecollaborativelearningenvironments.Itisproposedthatmoreprogressmaybeaccomplishedthroughdevelopmentresearchordesignresearch.Developmentresearchprotocolsrequireintensiveandlong-termcollaborationamongresearchersandpractitioners.Inthisarticle,weproposeguidelinesforimplementingdevelopmentresearchmodelsmorewidely,andconcludewithaprescriptionforanonlinecollaborativelearningresearchagendaforthenextfivetotenyears.
Reinking,D.,&Watkins,J.(2000).Aformativeexperimentinvestigatingtheuseofmultimediabookreviewstoincreaseelementarystudents’independentreading.Reading Research Quarterly, 35(3),384-419.
![Page 122: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/122.jpg)
an introduction to educational design research120
Abstract:Thisstudyinvestigateshowacomputer-basedinstructionalintervention(creatingmultimediareviewsofbooks)mightincreasefourthandfifthgraders’independentreading.Thestudyfindsthatthesuccessoftheinterventionwasrelatedtothemediatingeffectsofusingtechnology,changesintheinteractionsamongstudentsandteachers,andstudents’engagementinrelationtotheirreadingability.Italsonotesseveralotherfactors.
DomainofcurriculumMcKenney,S.&vandenAkker,J.(2005).Computer-basedsupportforcurriculumdesigners:Acaseofdevelopmentalresearch.Educational Technology Research & Development, 53(2)41-66. Abstract:Inthisarticle,weexplorethepotentialofthecomputertosupportcurriculum
materialsdevelopmentwithinthecontextofsecondarylevelscienceandmathematicseducationinsouthernAfrica.Duringthefour-yearcourseofthestudy,acomputerprogramwasdevelopednamedCASCADE-SEA,whichstandsforComputerAssistedCurriculumAnalysis,DesignandEvaluationforScience(andmathematics)EducationinAfrica.Bycarefullydocumentingtheiterativeprocessofanalysis,prototypedesign,evaluation,andrevision,wesoughtinsightintothecharacteristicsofavalidandpracticalcomputer-basedtoolthatpossessesthepotentialtoaffecttheperformanceofitsusers.TheresultsofthisstudyincludetheCASCADE-SEAprogramitself,whichassistsusersinproducingbetterqualitymaterialsthantheyotherwisemight,whiletheyalsolearnfromthedevelopmentprocess.Further,thisresearchhascontributedtothearticulationofdesignprinciplesandrelateddevelopmentalresearchmethods.Thisarticlehighlightstheresearchanddevelopmentthattookplace,andonlybrieflyaddressesthetoolitself.
McKenney,S.,Nieveen,N,&vanderAkker,J.(2002).Computersupportforcurriculumdevelopers:CASCADE.Educational Technology Research and Development,50(4),25-35. Abstract:Thisarticleexaminesresearchonacomputer-basedtool,CASCADE(Computer
AssistedCurriculumAnalysis,DesignandEvaluation),thatwasdevelopedattheUniversityofTwente(Netherlands)toassistincurriculumdevelopment.Thearticlediscusseselectronicperformancesupportsystemsandtheneedforincreasedattentiontoimplementationandimpactstudies.
Nieveen,N.M.(1999).Prototypingtoreachproductquality.In:J.J.H.vandenAkker,R.Branch,K.Gustafson,N.M.Nieveen,&Tj.Plomp(Eds.),Design approaches and tools in education and training(pp.125-136).Dordrecht:Kluwer. Abstract: Thischapterprovidesaframeworkforproductqualityconsistingofthe
followingthreecriteria:validity,practicalityandeffectiveness,andprovidesinsightinto
![Page 123: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/123.jpg)
an introduction to educational design research 121
theapplicabilityoftheframeworkinvariousdomainsofeducationalproductdevelopment.Inordertoreachproductquality,theprototypingapproachisseenandunderstoodasasuitableapproach.Thischapterdiscussesthreesignificantcharacteristicsofaprototypingapproach:extensiveuseofprototypes,highdegreeofiterationandtheroleofformativeevaluation,andtheparamountimportanceofuserinvolvement.Thechapterillustratesthewaytheprototypingapproachhasbeeninstrumentalindevelopingacomputersupportsystemforinstructionaldevelopers.Duringtheprototypingprocess,theframeworkassistedindecidingthefocusofeachprototypeandenhancedthetransparencyoftheentireprocess.
Nieveen,N.M.&Akker,J.J.H.,vanden(1999).Exploringthepotentialofacomputertoolforinstructionaldevelopers. Educational Technology Research & Development,47(3),77-98. Abstract:Informationandcommunicationtechnologytoolscurrentlypermeatealmost
everyprofessionaldomain.Thosegearedtowardthefieldofinstructionaldevelopmenthaveemergedinrecentyears.Thisarticleexploresthepotentialforlinkingthedomainsofcomputersupportandinstructionaldevelopment.ThisarticlereportsonthedesignandevaluationofCASCADE(ComputerAssistedCurriculumAnalysis,DesignandEvaluation),acomputersystemthatsupportsinstructionaldevelopersduringformativeevaluationefforts.Fiveprototypesofthesystemwerecreatedandevaluatedonthebasisoftheirvalidity(reflectionofstate-of-the-artknowledgeandinternalconsistency);practicality(abilitytomeettheneeds,wishesandcontextualconstraintsofthetargetgroup);andeffectiveness(improvedusertaskperformance).TheresultsofthisstudysuggestthattheuseofCASCADEcould:(a)improvetheconsistencyofformativeevaluationplansandactivities;(b)motivatedevelopersbyelevatingtheirconfidenceinusingformativeevaluationactivities;(c)savetime;and(d)helptoprovidejustificationsfordecisionsmade.
Some PhD theses utilizing design research as a research approach
Overtheyears,variousPhDthesehavebeenwritteninwhichdesignresearchhasbeenappliedasthemainresearchapproach.InthissectionwejustmentionafewthathavebeendefendedinTheNetherlandsattheUniversityofTwenteandtheUniversityofUtrecht.Undoubtly,manymoredissertationscanbefoundviasearchmachinesontheWWWeb,butwejustwanttopointthereadertoafewexamplesofdesignresearchwearefamiliarwith.
DomainofcurriculumdevelopmentNieveen,N.(1997).Computer support for curriculum developers.Doctoralthesis.Enschede(TheNetherlands)UniversityofTwente,.
![Page 124: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/124.jpg)
an introduction to educational design research122
Availablefrom:http://projects.edte.utwente.nl/cascade/original/McKenney,S.(2001).Computer-based support for science education materials developers in Africa: exploring potentials. Doctoralthesis.Enschede(TheNetherlands):UniversityofTwente.Availablefrom:http://projects.edte.utwente.nl/cascade/seastudy/
Kouwenhoven,W.(2003).Designing for competence in Mozambique: towards a competence-based curriculum for the Faculty of Education of the Eduardo Mondlane University. Doctoralthesis.Enschede(TheNetherlands):UniversityofTwente.Availablefrom:http://doc.utwente.nl/41442/1/thesis_Kouwenhoven.pdfNote:thisisanexampleofdesignresearchinwhichtheresearcherwasnotactivelyinvolvedinallphasesofthedesignprocess.
DomainofprofessionaldevelopmentofteachersTeclaiTecle,Andemariam(2006). The potential of a professional development scenario for supporting biology teachers in Eritrea.Doctoralthesis.Enschede(TheNetherlands):UniversityofTwente.Availablefrom:http://purl.org/utwente/55985
DomainofmathematicseducationArmanto,Dian(2002).Teaching multiplication and division realistically in Indonesian primary schools : a prototype of local instructional theory.Doctoralthesis.Enschede(TheNetherlands)UniversityofTwente.Availablefrom:http://purl.org/utwente/58710
Bakker,A.(2004).Design research in statistics education : on symbolizing and computer tools.Doctoralthesis.Utrecht(TheNetherlands:UniversityofUtrecht.Availablefrom:http://igitur-archive.library.uu.nl/dissertations/2004-0513-153943/inhoud.htm
Fauzan,Ahmad(2002).Applying realistic mathematics education (RME) in teaching geometry in Indonesian primary schools. Doctoralthesis.Enschede(TheNetherlands)UniversityofTwente.Availablefrom:http://purl.org/utwente/58707
DomainofscienceeducationKnippels,M.C.P.J.(2002).Copingwiththeabstractandcomplexnatureofgeneticsinbiologyeducation:Theyo-yolearningandteachingstrategy.Doctoralthesis.Utrecht(TheNetherlands:UniversityofUtrecht.Availablefrom:http://igitur-archive.library.uu.nl/dissertations/2002-0930-094820/inhoud.htm
![Page 125: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/125.jpg)
an introduction to educational design research 123
Mafumiko,FideliceSimbagungileMbaruku(2006).Micro-scale experimentation as a catalyst for improving the chemistry curriculum in Tanzania.Doctoralthesis.Enschede(TheNetherlands)UniversityofTwente.Availablefrom:http://purl.org/utwente/55448Ottevanger,W.(2001).MaterialsdevelopmentasacatalystforsciencecurriculumimplementationinNamibia.Doctoralthesis.Enschede:UniversityofTwente.
Tilya,FrankNicodem(2003).Teacher support for the use of MBL in activity-based physics teaching in Tanzania. Doctoralthesis.Enschede(TheNetherlands)UniversityofTwente.Availablefrom:http://purl.org/utwente/41462
Westbroek,H.B.(2005).Characteristics of meaningful chemistry education - The case of water quality.Doctoralthesis.Utrecht(TheNetherlands:UniversityofUtrecht.Availablefrom:http://igitur-archive.library.uu.nl/dissertations/2005-0922-200121/index.htm
![Page 126: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/126.jpg)
![Page 127: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/127.jpg)
an introduction to educational design research 125
AuthorbiographiesJanvandenAkkerJanvandenAkkerisdirectorgeneralofNetherlandsInstituteforCurriculumDevelopment[SLO].Besides,heispart-timeprofessor(chaironCurriculumDesign)attheUniversityofTwente.Inhiswideteaching,research,supervision,andconsultancyexperiences(bothintheNetherlandsandabroad)hetendstoapproachcurriculumdesignchallengesfromabroadereducationalinnovationperspective.Overtheyearshispreferencefordesignresearchhasincreasedbecauseofitsstrongcombinationofpracticalrelevanceandknowledgegrowth.Somebooksoverthelastdecade(editedwithsomecolleagues)thatrepresenthisorientation:Design Approaches and Tools in Education and Training (1999), Curriculum Landscapes and Trends (2003),and Educational Design Research (2006).
BrendaBannanBrendaBannanisanassociateprofessorintheinstructionaltechnologyprogramatGeorgeMasonUniversityinFairfax,Virginia,USA.Herresearchinterestsprimarilyinvolvetheintegrationofdesignandresearchprocessesrelatedtoeducationaltechnologydesignanddevelopment.ShehasauthoredseveralarticlesandchaptersondesignresearchintheHandbookforDesignResearchMethodsinEducation(Eds.Kelly,Lesh&Baek,2008)aswellastheEducational Researcher,vol32,2003.Dr.BannanwasawardedanNSFCAREERgrantawardin2003toconductafive-yearcycleofdesignresearchandwasinvitedasavisitingscholaratStanfordUniversityfortheacademicyearof2005-2006.
AnthonyE.KellyAnthonyE.KellyisaprofessorofeducationalpsychologyatGeorgeMasonUniversityinVirginia,USA.Hisresearchinterestsextendtoresearchmethodologydesign,andresearchattheintersectionofcognitiveneuroscienceandeducation.HecurrentlyhastwoNationalScienceFoundationgrants,oneonmodelinglearningincyberinfrastructurenetworksinearthsciences(withBrendaBannan-Ritland),andoneonlearninginmegacities.HisNSFgrantondesignresearchmethodswithRichardLesh,producedtheHandbookofDesignResearchMethods(Kelly,Lesh&Baek,2008).Kellyeditedthespecialissueondesignresearch,EducationalResearcher,vol32,2003.HeistheeditorofthespecialissueontheUSNationalMathematicsAdvisoryPanelReport,Educational Researcher,forthcoming.HeservedasaprogrammanagerattheNationalScienceFoundationfrom1997-2000,and2006.
NienkeNieveen(authorandeditor)NienkeNieveenworksattheresearchdepartmentofNetherlandsInstituteforCurriculumDevelopment[SLO].HerworkcentersaroundcoordinatingandsupportingtheInstitute’sdesignresearchactivitiesbytakingseriesofconcertedactionstoassistcurriculum
![Page 128: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/128.jpg)
an introduction to educational design research126
developersininterweavingdesignandformativeevaluationactivities.Indoingso,SLOaimsatknowledgegrowthandprovenqualityforallofitsproducts,i.e.curriculumframeworksandteachingandlearningmaterials.Herdissertation,in1997,wasbasedonafouryeardesignresearchprojectand,alsoafterthisperiod,shecontinuedworkingondesignresearchprojectsattheUniversityofTwente.Shehasauthoredseveralarticlesandchaptersondesignresearchandco-editedthebookEducational design research.
TjeerdPlomp(authorandeditor)TjeerdPlompisemeritusprofessorofcurriculumoftheUniversityofTwenteinEnschede,TheNetherlands.Hehasbeeninchargeofteachingeducationaldesignmethodologyinthe(atthattime)FacultyofEducationalScienceandTechnology.HewaschairofIEA,theInternationalAssociationfortheEvaluationofEducationalAchievement,from1989-1999.IntheIEAheservedaschairforthe‘ComputersinEducation’study(Comped),theThirdInternationalMathematicsandScienceStudy(TIMSS)andtheIEASecondInternationalTechnologyinEducationStudy(SITES).Hisresearchinterestsareeducationaldesignanddesignresearch,internationalcomparativeresearch,andinformationtechnologyinthecurriculumandteachereducation.Hehasbeenrecentlyinvolvedasadvisorinvariousresearchprojectsandprogramsutilizingdesignresearch,bothinTheNetherlandsandinternationally.
![Page 129: An Introduct An Introduction to Educational Design Research · An Introduction to Educational Design Research SLO • Netherlands institute for curriculum development SLO PO box 2041](https://reader030.vdocuments.net/reader030/viewer/2022040405/5e9e07b7ed32b478fd2d2543/html5/thumbnails/129.jpg)
An Introduction to Educational Design Research
SLO • Netherlands institute for curriculum developmentSLO
PO box 20417500 CA EnschedeThe Netherlands
T +31(0)53 484 08 40F +31(0)53 430 76 92E [email protected]
www.slo.nl
SLO is the Netherlands institute for curriculum development. We are bridging the contexts of policy, research, and practice. Our expertise focuses on the development of curricular goals and content for various educational levels, from national policy to classroom practices. We closely collaborate with many different stakeholders from policy circles, schools (boards, principals, teachers), research, civic organizations, and the society at large. This allows us to design and validate relevant curriculum frameworks, to elaborate exemplary materials and to evaluate these in school practices. Our products and services support both policy makers and schools and teachers in making substantive curricular decisions and in elaborating these into relevant, inspiring and effective education.
ISBN: 978 90 329 2329 7
An In
trod
uctIo
n to
Edu
cAtIon
Al dEsIg
n rEsEArch
Editors: tjeerd Plomp &
nienke n
ieveen
Editors:Tjeerd Plomp & Nienke Nieveen