University of Nebraska - LincolnDigitalCommons@University of Nebraska - Lincoln

DBER Speaker Series Discipline-Based Education Research Group

11-14-2013

Steps in Conducting a Scholarly Mixed MethodsStudyJohn W. CreswellUniversity of Nebraska - Lincoln, jcreswell1@unl.edu

Follow this and additional works at: http://digitalcommons.unl.edu/dberspeakers

This Presentation is brought to you for free and open access by the Discipline-Based Education Research Group at DigitalCommons@University ofNebraska - Lincoln. It has been accepted for inclusion in DBER Speaker Series by an authorized administrator of DigitalCommons@University ofNebraska - Lincoln.

Creswell, John W., "Steps in Conducting a Scholarly Mixed Methods Study" (2013). DBER Speaker Series. Paper 48.http://digitalcommons.unl.edu/dberspeakers/48

http://digitalcommons.unl.edu?utm_source=digitalcommons.unl.edu%2Fdberspeakers%2F48&utm_medium=PDF&utm_campaign=PDFCoverPageshttp://digitalcommons.unl.edu/dberspeakers?utm_source=digitalcommons.unl.edu%2Fdberspeakers%2F48&utm_medium=PDF&utm_campaign=PDFCoverPageshttp://digitalcommons.unl.edu/dber?utm_source=digitalcommons.unl.edu%2Fdberspeakers%2F48&utm_medium=PDF&utm_campaign=PDFCoverPageshttp://digitalcommons.unl.edu/dberspeakers?utm_source=digitalcommons.unl.edu%2Fdberspeakers%2F48&utm_medium=PDF&utm_campaign=PDFCoverPageshttp://digitalcommons.unl.edu/dberspeakers/48?utm_source=digitalcommons.unl.edu%2Fdberspeakers%2F48&utm_medium=PDF&utm_campaign=PDFCoverPages

AbstractforDBERGroupDiscussionon20131114Presenter(s),Department(s):JohnCreswellProfessorDepartmentofEducationalPsychologyUniversityofNebraskaLincolnTitle:StepsinConductingaScholarlyMixedMethodsStudyAbstract:

MixedmethodsresearchisarapidlyexpandingmethodologyinthesocialandhumansciencesintheUSandaroundtheworld.InthispresentationIwillfirstdefinemixedmethodsresearch(combiningbothquantitativeandqualitativemethodsofresearch)anddiscusswhatitisandwhatitisnot.ThenIwillreviewabriefhistoryofitsdevelopment,andwhyitisimportanttoday.Iwilldiscussseveralofthescientificdevelopmentsinmixedmethodsthathaveoccurredoverthelasttenyears,suchasthespecificationoftypesofdesigns,theformationofmixedmethodsquestions,andtheuseofinnovativeapproachestojointlydisplayquantitativeandqualitativeresults.Finally,Iwilltalkaboutthefutureofthismethodologywhereitisheadedandsomeimportantworldwidedevelopmentsthathaveencouragedmixedmethodsresearch.

StepsinConductingaScholarlyMixedMethodsStudy

JohnW.CreswellUniversityofNebraskaLincoln

Numbers PersonalExperiences

HowWeMakeSense

AMixedMethodsApproach

ASimpleIdea

Mixedmethodsisaresearchapproach,popularinthesocial,behavioral,andhealthsciences,inwhichresearcherscollect,analyze,andintegratebothquantitativeandqualitativedatainasinglestudyorinasustainedlongtermprogramofinquirytoaddresstheirresearchquestions.

Letsusequantitativeandqualitativedata(orquantitativeresearchandquantitativeresearch)together togainamorecompleteunderstandingofourresearchquestions.

5

New NIH funding awards that self-identify as mixed methods (N = 226) (1997-2008)

Source:CRISPdatabase;PlanoClark (2010)

0

10

20

30

40

50

60

70

1996 1998 2000 2002 2004 2006 2008 2010

Year

Nu

mb

er o

f N

ew A

war

ds

POPULARITY FEDERALFUNDING

Haines,C.,2011unpublisheddissertation,UniversityofNebraskaLincoln

DISSERTATIONS

MixedMethodsSocialScienceArticles

MixedMethodsHealthScienceArticles

SelectMixedMethodsBooks

Jennifer Greene USA- University of Illinois evaluator

John Hunter and Allen Brewer USA -Northwestern and Boston College sociologists

Alan Bryman - England management

John Creswell USA education

Nigel and Jane Fielding sociologist in UK

Jan Morse nursing in Canada

When did mixed methods develop in the social sciences and health/behavioralsciences?(1985-90)

Ifyouwalkedintomyoffice,withtheideaofconductingamixedmethodsstudy,Iwouldtakeyouthroughaseriesofquestionstohelpyoudesignyourproject.

Iwilltakethestepsoutoforderofwhatyoutypicallyseeinaresearchproject,butwewillputtheminorderattheend.

Question1.Doyouhaveamixedmethodsstudy?(orsimplyaquantitativestudyoraqualitativestudy)

WhatIamlookingfor(corecharacteristicsofamixedmethodsstudy):

1. Doyouhaveaquantitativedatabase?(closedended)

2.Doyouhaveaqualitativedatabase?(openended)

3. Doyouplanonbringingthetwodatabasestogether?(integration)

4.Whatprocedures(design)willyouuse?

WhatIamNOTlookingfor:

Keepingthedatabasesseparate ThecollectionofBOTHquantitative

andqualitativedata(notcontentanalysis) Multipleformsofquantitativedataormultiple

formsofqualitativedata(multimethod) Onlythetransformationofqualitative

dataintoquantitative(inanalysis) Onlyformative/summativeevaluation Onlyaddinginqualitativedataintoan

experiment Doingmixedmethodsbecauseitistrendy

Question2.Doesyourresearchproblemorquestionmeritamixedmethodsstudy?

Theysay,chooseyourmethodbasedonyourproblem.

Whenismixedmethodssuitablefora problemorquestion?

Whenqualitativeresearchorquantitativeresearchisinsufficienttofullyunderstandtheproblem

Weneedtoexplorebeforeweadministerinstruments

Weneedtoexplainourstatisticalresultsbytalkingtopeople

Weneedtoseeifourquantitativeresultsandourqualitativeresultsmatch

Weneedtoenhanceourexperimentsbytalkingwithpeople

Weneedtodevelopnewinstrumentsbygatheringqualitativedata

Othertypesofproblems,when:

Question3:Whatistheoverallintentofyourstudy?

Mygoal:

Tolearnwhetheryouarequantitatively orqualitativelyorientedbythewayyoustateyourintent

Keywords

in

yo

Quantitative Qualitative

factors explore

determinants generate

relationship develop

causes create

influences meaning

effects personalexperiences

1. Willhelpmethinkaboutwhatmixedmethodsdesignyouwilllikelyuse

2. Willhelpmethinkaboutyourdisciplinefieldbackgroundthatyoubringtomixedmethods

3. Willhelpmethinkaboutwhatskillsyouhavetobeginconductingamixedmethodsstudy

YourKeywords:

ExamplesofGeneralIntentStatements

ThisarticlereportsonthestudyconductedtounderstandstudentspersistenceintheDistributedDoctoralPrograminEducationalLeadershipinHigherEducation(ELHE)offeredbytheUniversityofNebraskaLincoln(UNL).(Ivankova &Stick,2007)

Weconductedaninterviewbasedstudyofterminallyillindividualswhowerereceivingpalliativecare.(Kutner,Steiner,Corbett,etal.,1999)

tounderstandpatientsexperiencesoftransitionsfromhospitaltoahomelessshelteranddetermineaspectsoftheseexperiencesassociatedwithperceivedqualityofthesetransitions.(Greysen,Allen,Lucas,etal.,2012)

Question4:Whatquantitativeandqualitativedatawillyoucollecttoaddressthisintent?

Iwillaskyoutolistyourquantitativeandqualitativedatasources:

QuantitativeData QualitativeData

Siteandsample?

Howmany?

Typesofdatatocollect

Specificinstruments/protocols

Typesofscales/questions

Iwillaskyoutolistyourquantitativeandqualitativedataanalysissteps:

QuantitativeAnalysis

QualitativeAnalysis

Getting datareadyforanalysis

Preliminaryanalysis

Stepsinanalysis(including statistics,themes)

ListMultipleTypesofQuantitativeandQualitativeDatatobeCollectedandAnalyzed

QUANTITATIVERESEARCH Quantitativedata

collection(closedended) Instruments Behavioralchecklists Records

Quantitativedataanalysis Usenumericdata,

Fordescription Forcomparinggroups Forrelatingvariables

QUALITATIVERESEARCH Qualitativedatacollection

(openended) Interviews Observations Documents Audiovisualmaterials

Qualitativedataanalysis Usetextandimagedata,

Forcoding Forthemedevelopment Forrelatingthemes

27

AreTheseProceduresRealisticandFeasible?

Time Economically Skills Usefulforstakeholders

Question5.Whyareyoucollectingbothquantitativeandqualitativedata?

Basicreasonsforusingmixedmethods(calledtherationaleinmixedmethods):

Needdifferent,multipleperspectives,ormorecompleteunderstandings

Needtoconfirmourquantitativemeasureswithqualitativeexperiences

Needtoexplainquantitativeresults Needbettercontextualizedinstruments,measures,or

interventionstoreachcertainpopulations Needtoenhanceourexperiments Needtogathertrenddataandindividualperspectivesfrom

communitymembers Needtoevaluatethesuccessofaprogrambyusinganeeds

assessmentANDatestofthesuccessoftheprogram

Questions6.Doyouplanonusingatheory?Doyouplanonadvancingyourphilosophicalstance?

32

Frame the study within theory/philosophy:

Paradigm/Worldview (epistemology, ontology, axiology, methodology)

Theoretical lens(e.g., feminist, racial, social science theories)

Methodological approach (e.g., ethnography, experiment, mixed methods)

Methods of data collection (e.g., interviews, checklists, instruments)

Adapted from Crotty M. (1998)

Whatisaphilosophy?

Whichphilosophiesaretypicallyusedasthefoundationinmixedmethodsresearch?

Furtherquestionsaboutphilosophy:

Pragmatism Transformativeworldview Criticalrealism Dialecticpluralism

Multipleworldviews/paradigms Matchworldviewtodesign Worldviewcomesfromyourscholarly

community

Philosophiestoconsiderinmixedmethods:

Whatisatheory?

Howdoyoufindatheory?

Howwillyouuseitinyourmixedmethodsstudy?

Furtherquestionsabouttheory:

Question7.Whatmixedmethodsdesignwillyouuse?

Parsimonioussetofdesigns:BasicandAdvanced

Basic DesignsConvergent DesignExplanatory Sequential DesignExploratory Sequential Design

Advanced DesignsIntervention DesignTransformative DesignMultiphase Design

BasicDesigns

AdvancedDesigns

Basicmixedmethodsdesigns

ConvergentParallelDesign

QuantitativeDataCollectionandAnalysis

DetermineQuantitativeResultstoExplain

QualitativeResults

QualitativeDataCollectionandAnalysis

MergeResultsForComparison

InterpretorExplainConvergence/Divergence

ExplanatorySequentialDesign

QuantitativeDataCollectionandAnalysis

QuantitativeResults

QuantitativeResults

QualitativeDataCollectionandAnalysis

QualitativeResults

InterpretHowQualitativeDataExplainsQuantitativeResults

ExploratorySequentialDesign

QualitativeDataCollectionandAnalysis

QualitativeResults

UseResultstoFormVariables,Instruments,Interventions

QuantitativeDataCollectionandAnalysisBasedonVariables,Instruments,Interventions

QuantitativeResults

InterpretHowQuantitativeResultsProvideNewResults,New,BetterInstruments,andBetterInterventions

AdvancedmixedmethodsdesignsInterventionMixedMethodsDesign

SocialJusticeDesign(usinganExplanatorySequentialDesignexample)

MultistageEvaluationDesign(usinganExploratorySequentialDesignexample)

ExperimentalStudy

ExperimentGroup

ControlGroup

Pretest

Pretest

PostTest

PostTest

InterventionQualitativeDataCollection,Analysis,andResults(Exploratory)

QualitativeDataCollection,Analysis,andResults(Convergent)

QualitativeDataCollection,Analysis,andResults(Explanatory)

Theory QuantitativeDataCollectione.g.,survey

QuantitativeResults

QualitativeDataCollectionandAnalysis

QualitativeResults

InterpretHowQualitativeDataExplainsQuantitativeResultsAndCallsforAction

ResearchQuestions

PromoteSocialJustice

FormativeNeedsAssessment(qualitativedatacollection)

Theory/ConceptualFramework(basedonqualitativeresults)

InstrumentDevelopment(basedonquantitativetests)

FormativeProgramAssessment(qualitativedatacollection)

SummativeProgramEvaluation(prepostquantitativetests)

ProgramRevision

SingleProgramObjective

Firstselectyourbasicdesign,thenaddifyouhaveanadvanceddesign

Consideryourscholarlycommunity:quantitativeorqualitative?(howtobeginyourstudy)

Consideryourskills:quantitativelystrongorqualitativelystrong(whatwillbeemphasized)

Consideryourresources(collectatonetimeorspaceout)

Considerthecomplexityofthedesign(whatistheeasiest?thehardest?)

Howdoyouchooseyourdesign?

AdvancedDesign:Betancourtetal.R24projectexaminingtheuseofafamilystrengthinterventionwithcommunityinputwithrefugeesinBoston

ExploratorySequentialDesign

InterventionDesign

CommunityBasedParticipatoryResearchApproach

Question8.Canyoudrawa diagramofyourdesign?

Convergent Design Wittink et al. (2006)

Source: based on Wittink et al.(2006)

Procedures:Three groups: control group, complianceIntervention group, alliance intervention group group comparisons

Outcome measures: 1) attitudes toward medication 2) adherence to treatment 3) avoidance of relapseDAI measure completed 3 times (pre, post, & follow up)

Products: Numerical item scores Change scores Test statistics

Experimental Methodology

QUAN QUANPre- Intervention Post- &

Follow-up Follow-upMeasure Measures

Procedures: Discuss treatment effectiveness

Discuss themes in context of

interventions and outcomes

Products:Discussion

Procedures: One-on-one semi-

structured interviews exiting the trial, participants from two experimental conditions;

Thematic analysis

Products: Transcripts Themes and quotes

Procedures: One-on-

one semi-structuredinterviews

Thematic analysis

Products: Transcripts Developed intervention

treatment

qual before

intervention

qual after

intervention

Overall results

and interpretation

Flow of the experiment

InterventionDesign(Rogers,Day,Randall,,&Bentall ,2003,studytoimprovethemanagementofantipsychoticmedication)

Question9.Canyouwriteagoodpurposestatement(orstudyaim)thatfitsyourdesign?

Howtowriteamixedmethodsstudyaim:

Thismixedmethodsstudywilladdress[overall contentaim].Aconvergentparallelmixedmethodsdesignwillbeused,anditisatypeofdesigninwhichqualitativeandquantitativedataarecollectedinparallel,analyzedseparately,andthenmerged.Inthisstudy,[quantitativedata]willbeusedtotestthetheoryof[thetheory]thatpredictsthat[independent variables]will[positively,negatively]influencethe[dependentvariables]for[participants]at[thesite].The[typeofqualitativedata]willexplore[thecentralphenomenon]for[participants]at[thesite].Thereasonforcollectingbothquantitativeandqualitativedataisto[themixingreason].

Intent

Design

DataCollection

Rationale

46

Question10.Canyouwriteresearchquestionsforyourmixedmethodsstudy?

Quantitativequestionsorhypotheses

Qualitativequestions

Amixedmethodsquestion

Write3typesofquestions:

Basicdesignmixedmethodsquestions:

ConvergentDesign

ExplanatoryDesign

ExploratoryDesign

Towhatextentdothequantitativeandqualitativeresultsconverge?

Inwhatwaysdothequalitativedatahelptoexplainthequantitativeresults?

Inwhatwaysdothequantitativeresultsgeneralizethequalitativefindings?

Advanceddesignmixedmethodsquestions:

InterventionDesign

TransformativeSocialJusticeDesign

MultistageDesign

Howdothequalitativefindingsprovideanenhancedunderstandingofthequantitativeresults?

Howdothequalitativefindingsprovideanenhancedunderstandingofthequantitativeresultsinordertoexploreinequalities?

Howtothedifferentphasesintheprojecttoaddresstheoverallresearchgoal?

Problem Theoryand/orphilosophy Purpose(studyaim) Rationaleforgatheringboth

quantitativeandqualitativedata Researchquestions(quantitative,

qualitative,mixed) Typesofdata/analysis Definitionofmixedmethods Typeofmixedmethodsdesign Diagramofthedesign

Nowplacethesestepsinorderforagoodmixedmethodsproject:

StepsinConductingaScholarlyMixedMethodsStudy

JohnW.CreswellUniversityofNebraskaLincoln

University of Nebraska - LincolnDigitalCommons@University of Nebraska - Lincoln11-14-2013

Steps in Conducting a Scholarly Mixed Methods StudyJohn W. Creswell

2013-11-14-Steps_in_Conducting_Mixed_Methods_Research-Creswell2013-11-14-Steps_in_Conducting_Mixed_Methods_ResearchPPT