Usability Study of an Online Mobile Apps Instructional ... ?· Usability Study of an Online Mobile Apps…

Download Usability Study of an Online Mobile Apps Instructional ... ?· Usability Study of an Online Mobile Apps…

Post on 04-Jul-2018

212 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • UsabilityStudyofanOnlineMobileAppsInstructionalModuleforBusyInstructors

    CherylIrebaria

    UniversityofHawaiiatMnoaHonolulu,Hawaii

    UnitedStatesofAmericairebaria@hawaii.edu

    http://cheirebaria.wix.com/mobileappsmodule

    Abstract: Considering the accessibility of online educational applications, it is natural to assume that instructors incorporate them into their practice. However, lack of resourceawareness and time prevent instructors from learning about practical apps. To overcome these barriers, an online instructional module that succinctly shares useful information on how to use the Kahoot!, Blendspace, and PowToon apps was developed. These apps were selected because they offer instructors relief from the timeconsuming task of creating entirely new deliverables by providing readymade products that instructors can use. The purpose of this usability study was to assess the ease of use, learnability, and user satisfaction of the module before use with its target audience. Online resources such as PowToon and YouTube were used to develop content while the ARCS Model of Motivation and Gagnes Nine Events of Learning guided design and content organization. The testing sessions, which were recorded using Google Hangouts on Air and ScreencastOMatic 2, exposed navigation, structural, and language issues as participants engaged in specific tasks. At the end of each session, participants completed a survey that provided additional insight and recommendations for improvement. Two rounds of usability testing were done each round uncovered problem areas that were corrected to facilitate quicker navigation of the site. Revisions to the module resulted in a more intuitive and fluid experience. These positive gains lend support for the effectiveness of usability testing in meeting designobjectives.

    IntroductionInstructorsmaybetoooccupiedwiththedemandsoftheirprofessiontolookforandresearcheducationalappsthatcouldbeeffectivelyutilizedintheclassroom.Thisisunfortunatebecausethereisawealthoffreeappsthatwouldbenefitbothinstructorsandstudents.Whileitmaybeeasytoprovideinstructorswithlistsofappsthatcouldbeusedforinstruction,manydonothavethetimetoexploreandevaluateappsthatwouldbeappropriatefortheirindividualdisciplinesorfields.Thedevelopmentofamodulethatshowcasesuniversalandeffectivemobileapplicationswouldrelievethemofhavingtosiftthroughloadsofmobileappsjusttofindonesthatwouldsuittheirneeds.Theshorttermbenefitsofinstructorsexpandingtheirtoolboxwillbeadiversified

    mailto:irebaria@hawaii.edumailto:irebaria@hawaii.eduhttp://cheirebaria.wix.com/mobileappsmodule

  • classroomandnewskillstoaddtotheirrepertoirethelongtermgoalisstudentswillbeexposedtonewappsandleavetheirlearningenvironmentequippedwiththe21stcenturyskillsandknowledgenecessarytonavigateaworldfloodedwithnewtechnologies.Thepurposeofthisusabilitystudyprojectistoevaluateanonlinemobileappsinstructionalmoduleforindividualswhoserveinanyteachingcapacity(e.g.,instructorsinthefieldsofbusiness,education,military,medicine,etc.)TheonlineMobileAppsforBusyInstructorsmodulepresentsthreeapplications:Kahoot!,Blendspace,andPowToon.Kahoot!isawebbased,gamebaseddigitalplatformthatemploystheuseofmobiledevicesasresponsesystemclickers.Inotherwords,studentscanusetheirmobiledevicestoanswerquestionsononlinequizzes(thatwerecreatedusingKahoot!).Blendspaceisaplaceonthewebatwhichinstructorscanstoreusefulinstructionalmaterialsandresourcesforstudentstoaccessoutsideofclass.Lastly,PowToonisanonlinevideoanimationapplicationthatcanbeusedtocreatedynamicvideos.LiteratureReviewOtherinstructionaldesignershavecreatedmodulesforprofessionaldevelopmentwhosetargetaudiencewaseducatorsinHawaiispublicschools.InastudyconductedbyresearchersLin,Iyoda,andHo(2011)teacherswereaskedwhattheywouldsuggesttootherteachersconsideringmobiletechnologyintheirclassroomsandthreethemesformedaroundcommonanswers:1)gettingteachersreadyformobileintegration,2)gettingtheschoolreadyformobileintegration,and3)gettingstudentsreadyformobilelearningtheseresultsrevealthat...formobilelearningtobesuccessful,theschoolsystem,teachers,andstudents,allneedtobereadyfordiverselearningenvironmentsthatsupportdigitalagelearningexperiences(ascitedinTsinakosandAlly,2013,p.223).Theexperiencesharedintheaforementionedstudyindicatesaneedforteachertraininginmobiletechnologies.Formoreadoptionandintegrationoftechnologytoolstooccur,itwilltakemorethanintroducingandbuildingawarenessofsuchtechnologiesprofessionaldevelopmentcoursesneedtobemadeavailablesoteacherswillhaveopportunitiestolearnhowtoeffectivelyusemobileappsandincorporatethemintotheirpractice.AstudyofschoolsnearSiliconValley,Californiashowedthateventhoughthoseschoolshadthetechnologyinfrastructuretosupportmeaningfuluseoftechnology,teachersdidnotusetechnologytoitsfullpotential(Richardson,Flora,&Bathon,2013).Thismeansthathavingtechnologytoolsatonesdisposaldoesnotensurethattheywillbeusedeffectivelytofacilitatelearningandencouragetheirintegrationintocurricula.Developmentofaninstructionalmodulethatguidesinstructorsandprovidesthemwithreadymadeideasforutilizingeducationalappswouldbeapurposefulstepintherightdirection.Thejustificationsforcreatinganonlinelearningmodule(insteadofconductingthecoursefacetoface)includemakinglearningconvenientforbusyinstructorsaswellas

  • promotingtheideasofmobilityandaccessibility,demonstratingtheusesoftechnology,andenhancinglearning.Whilenotcompletelyapplicabletothemobileappsmoduleduetoadifferenceintopicausabilitystudyforaninteractive,webbasedmoduleonmarketingconceptsfoundthatlearningthatinvolvesinteractingwithlessonspresentedonacomputerresultsinavaluableandmeaningfullearningexperience(GirardandPinar,2011).Anotherstudycorroboratesthepreviouslystatedsuggestionwithevidencecollectedfromfocusgroups,whichrevealthatintegrationoftechnologytoolsintheclassroomenhanceslearningandmayleadtopositiveeducationaloutcomes(Devlin,Feldhaus,andBentrem,2013).Researchsuggeststhatthereissupportfortheuseoftechnologytoolsinteachingandtheteachingoftechnologytools.ProjectDevelopmentThecreationofanonlinemodulewouldnotonlydemonstratetheusefulnessandvalueoftechnologiesbutalsomakelearningaccessibleforbusyinstructors.Toaddresstheissueofinstructorreluctancetointegratetechnology,aninstructionalmodulewasdesigned,developed,anddeliveredusingWix.comawebsitebuilderandhostsite.Theonlineinstructionalmodule(functionalprototype)canbeaccessedathttp://cheirebaria.wix.com/mobileappsmodule.Forinstructorswhoarelessfamiliarwithusingmobileappsorknowingwhatmobileappstouse,thisinstructionalmoduleprovidesideasandassiststheminintegratingtechnologyintotheirrespectivelearningenvironments.Essentially,therehavebeenthreeiterationsoftheinstructionalmodulebasedonthecomponentsoftheADDIEmodel(Boulet,2007).Ateverystageofdevelopment,therewasanevaluationoftheproductthatuncoveredareasthatneedimprovement.ThewebsitebuilderWix.comwasusedbecauseithasfeaturesthatallowonetoeditthewebsiteinmobileviewforoptimalviewingonamobiledevicetheresultisamodulesitethatisbothviewableonacomputerandmobiledevices.Wixisalsouserfriendly,allowinganyonewithnocodingexperiencetobuildawebsiteusingthedraganddropfeaturetoaddimages,videos,buttons,andtextboxestoawebpage.Also,unlikeotherwebsitebuilderplatforms,itprovidesmorefreedomintermsofpagedesignverticalmenus,forexampleandhasaverylargelibraryoffreeimages(seeAppendixE).TheADDIEmodelsystemsapproach(Analysis,Design,Development,Implementation,andEvaluation)framedmoduledevelopmentandtheARCSModelofmotivationaldesignguidedcontentdevelopmentanddelivery(Boulet,2007Keller,1999).ItmaybethatparticipantswillnotnoticethedifferentelementsoftheARCSModel,butthemodulelessonsdofollowthesuggestedcomponentsofAttention,Relevance,Confidence,andSatisfaction.Initsentirety,theinterfaceandcontentoftheinstructionalmodulearelooselycraftedaccordingtoGagnesNineEventsofInstructioninrelationtomultimediainterfacedesign.Forexample,selectiveuseofcolorsandabalancedscreendesigncanhelpgainandkeeptheattentionaswellasdirecttheattentionofthelearner(seeFigure1andFigure2).Screendesigncanbestructuredinawaythatgainstheattentionofthestudent(Deubel,2003).Inaddition,Stemler(1997)foundthatchunked

    http://cheirebaria.wix.com/mobileappsmodule

  • informationimprovesscreendesign(ascitedinDeubel,2003).NoticetheseparationsandchunkingofinformationinFigure1.

    Figure1.ScreenshotoftheHomePageInthesamearticle,itisalsostatedthatwhennavigationalelementssuchasmenusandbuttonsareconsistentlyandpurposefullyplacedthroughoutaprogramitprovidesstructureand,inturn,giveslearnerscontrol,whichleadstoconfidence(Deubel,2003).Advicefromexpertswastakenintoconsiderationinthedesignoftheonlinemodulefunctionalprototypesuchas:Designersshouldthinkastheusertodesignamobilelearningexperiencethatisusercentered,objectsonapageshouldbeorganizedandconsistentthroughoutsothatuserscanrealizethepatterninwhichinformationissharedandpresented(Forni,2012,2014).Theconceptsofchunking,consistency,andcuesaredemonstratedonthesitetomakeitasuserfriendlyandassimpleaspossible(Figure2).Additionally,havingaconsistentandpredictabledesignthroughoutbuildsuserconfidenceandcomfortinnavigationofthesite.

    Figure2.Sidebysidecomparisonoftwomodulelessonsthatillustratesconsistency.

  • MethodsTheresearchersentoutanemail,contactedprospectsviaphone,andapproachedpotentialrecruitsinpersonwithinformationaboutthestudy.Sixinstructorsagreedtoparticipateintheusabilitystudy.Allparticipantsarefemaleadults.Theiryearsofteachingexperience,age,knowledgeofonlinetools,andleveloftechnologyskillsvary.Thestudywasconductedintworounds,threeparticipantsperround.Agedistributionsforeachroundareasfollows:Round1,6170(33%)and3140(66%)andRound2,1830(33%),4150(33%),3140(33%).

    Figure3.Agerangeofusabilitystudyparticipants.Participantsofthisstudyareinstructorsemployedindifferentfields(e.g.,themilitary,business,education,etc.).Ofthosewhoareinstructorsineducation,twoareteachersandoneisatechnologycoordinatortheotherparticipantsholdadministrativerolesintheirrespectivefieldsbusiness,retail,andthemilitary.Breakdownperround:Round1,Militaryadministrator(33%),Retailadministrator(33%),andEducationteacher(33%)Round2,Businesssupervisor(33%),Educationteacher(33%),andEducationtechnologycoordinator(33%).

    Figure4.Professionsofusabilitystudyparticipants.

  • Toinforminstructionaldesign,webdevelopment,andfutureiterationsoftheMobileAppsonlineinstructionalmodule,theusabilitystudysoughttoanswerthefollowingresearchquestions:

    1. Howdifficultisitforinstructorstonavigatethroughouttheinstructionalmodulewebsite?

    2. Howdifficultisitforinstructorstobecomefamiliarwiththeorganizationorstructureofthewebsite?

    Tasksandscenariosbasedontheresearchquestionsweredevelopedtohelpidentifynavigational,organizational,structural,andgeneralfunctionalityissueswiththewebsite.

    Table1.TasksandScenarios

    Beforetheusabilitytestingsessionwasinitiated,participantswereinformedofthepurposeofthestudy,method,timerequiredtoparticipate,andpotentialrisksandbenefits,revealedinaconsentform(seeAppendixA).Participationwasvoluntaryandinformedconsentwasobtainedbeforeparticipantsproceededwiththeusabilitysession.ToevaluatetheMobileAppsforBusyInstructorsonlineinstructionalmodule,participantsengagedintheusabilitystudyinoneoftwoways:1)ModeratedcompletelyonlineviaGoogleHangoutsonAir,withtheresearcherandparticipantusingtheirowncomputersandInternetconnectiontoconductthestudyatamutuallyagreedupontimeor2)Moderatedinpersonatamutuallyagreedupontimeandlocation,usingacomputerandInternetconnectionprovidedbytheresearcher.Foronlineandinpersontests,participantswereaskedtonavigatetheonlineinstructionalmoduleandtothinkoutloudwhiledoingso.Theresearcherpresentedtotheparticipantsscenariostoexploredifferentcomponentsofthesite.Astheparticipantscompletedtasks(forthescenarios)theywereaskedtoverbalizetheirthoughts(SeeTable1).TheprotocolwasbasedlargelyontheoutlineprovidedinSteveKrugsworkRocketSurgeryMadeEasy:TheDoItYourselfGuidetoFindingandFixingUsabilityProblems(2009)(seeAppendixD).Screen

  • activityandaudiowererecorded,butnorecognizableimagesoftheparticipantswerecaptured.AtoolcalledScreencastOMatic2wasusedtorecordactivityandcommentaryduringinpersonusabilitytestingsessions.Foronlinesessions,GoogleHangoutsonAirrecordedverbalandscreenactivity.RecordingswereautomaticallypublishedtoYouTubeattheendofeverysession.Allrecordingsweredownloadedontoapasswordprotectedcomputerforsafekeeping,thendeletedfromYouTube.Thecreationofaninstructionalpackageandexecutionofusabilitytestingofthemoduletookapproximately20weekstocomplete.Beforeparticipantstestedthefunctionalprototype,itunderwentanevaluationofpeersduringFallsemester2015,andchangesweremadeaccordingtofeedback.TheactualusabilitystudytookplaceduringthebeginninghalfofSpringsemester2016.Therewasatotalofsixparticipantsforofficialusabilitytestingthisnumberwasneededtogathersufficientdatatoascertainsevereissues.Informationgatheredfromthefirstroundofusabilitytestingwasusedtoimprovethesiteandmakeexploringitasintuitiveaspossible.Revisionsweremadebeforethesecondroundofusabilitytesting.Evaluationoftheinstructionalmodule,analysis,andrevisiontookapproximately10weekstocomplete.ResultsAnonlineprestudysurveygathereddataondemographicsandinstructorsselfperceptionwithrespecttotechnologyuseandintegration(seeAppendixB).Basedonprestudysurveydata,allparticipantsarecomfortablewithusingcomputers,tablets,andsmartphonesandusetheInternetonadailybasis.AllbutoneparticipantselectedNeutralonitemspertainingtocomfortwithusingtheInternetforprofessionaldevelopmentandintegrationofmobiletechnologyintoherpractice.Attheconclusionofeachusabilitytestingsession,anonlinepoststudysurveywasadministered.Openendedquestionsonthesurveygatheredadditionalfeedbackonhowthemodulecanbeimproved(seeAppendixC).Likertscaleitemswereusedtomeasureeaseofuse,learnability,andusersatisfaction.Degreeofagreementwasmeasuredusinga5pointscale(5StronglyAgree,4Agree,3Neutral,2Disagree,and1StronglyDisagree).Theresearcherconsideredincludingquestionsregardingcourseeffectivenessinpromotingtechnologyintegrationandbuildingfamiliaritywithtechnologytoolsbutdecidedtheywerenotpertinenttotheusabilitystudybecauseparticipantswouldnotbeabletoactuallytakethecourseduringthesession.BothpreandpostsurveyswerecreatedusingGoogleFormsforquickrevisionandseamlessdatacollection.DatafromthesurveyswerecompiledautomaticallyinaGoogleSheet.Fromthere,theresearchercleanedandorganizedthedataforstatisticalpurposes.QuantitativedatageneratedmainlyfromLikertscaleitemsweresummarizedusingdescriptivestatistics.XLMinerAnalysisToolpak,anonlineGoogleSheetsaddon,analyzedthedatatoproducemeansandstandarddeviationsforeachsurveyitem.Thestatisticsprovidedmeaningfulinsightsandsuggestedrecommendationsforimprovingmoduledesign.

  • ThepoststudyresultsforEaseofUsesurveyitemsrevealthatparticipantsofthesecondround(comparedtoparticipantsofthefirstround)didnotfindthesiteeasytouse(Figure3).Thefollowingstatements,LinksareeasytoidentifyandThemodulehasacleanandsimplepresentationscoredlowerforRound2.Oddly,thestatement,Thewebsitewaseasytonavigatehada...