LMS Review/Canvas Pilot Project Recommendations - April 2016
Teaching&LearningTechnologies|April2016
TableofContents
ExecutiveSummaryPilotProjectOverview A.Premise B.PilotGoals C.PilotSchedule D.Communication&Outreach E.Training F.Pre-Pilot&MidtermSurveysEvaluation
A. SurveyData–FacultyPerspectivesB. SurveyData–StudentPerspectivesC. FunctionalSupportReviewD. TechnicalReview
SummaryandRecommendationAppendix1:PilotFacultyandClassesAppendix2:FacultyMidtermSurvey:CanvasPilotAppendix3:StudentMidtermSurvey:CanvasPilot
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
1 Teaching&LearningTechnologies|April2016
ExecutiveSummaryInresponsetofeedbackfromstudents,faculty,andstaff,Teaching&LearningTechnologiesbegananeffortinspringof2015toinvestigatepotentialalternativestoBlackboardasourenterpriselearningmanagementsystem(LMS).TherapidgrowthofCanvasinthemarketplaceamongourpeerinstitutionsbroughtitquicklytotheforefrontofourattention,andweassembledagroupoffacultytoperformaninitialassessmentoftheplatformintheirclassroomsinfallof2015.Thatassessmentwasexpandedforspringsemester2016,concludingwithatotalof82pilotcourses,77facultyparticipants,2,450students,andover100sandboxesandorganizationstestingandevaluatingtheCanvasplatform.
AswenearcompletionofourinitialcohortofCanvaspilotfacultyandstudents,wehaveseveralconclusions:
• FacultyandstudentswhorespondedtosurveysfavoramovetoCanvas.81%ofspringpilotfacultyreportpreferringCanvas;59%ofstudents.ReviewofpeerinstitutionsconductingCanvasevaluationsshowsimilarpatternsinpilotsurveyresults.
• StudentsarevocalaboutinconsistentLMSusebyfacultyacrosstheboard,aswellasannoyancewithhavingtousetwoLMS’sduringthepilot.Thisstronglysuggestthatanymovebeexpeditedtolimitconfusionandfrustration.
• Facultyandstudentshistoricallyhavereportedaconsistentfrustrationwiththe“clunky”natureofBlackboard.SomeofthosefrustrationsareobviatedbyCanvas’moremoderninterfaceandunderpinnings,whichtranslatesintosignificantlyincreasedease-of-useratings(74%offacultysurveyedrateCanvasasveryeasytouse,withBlackboard’seaseofuseratingatonly20%).
• FacultysurveydatashowsthetopthreeCanvasfeaturesusedbyfacultywere:Postingannouncements,creatingassignments,andpostinggradesusingthegradebook.ThemostpopularfeaturesusedbypilotfacultywereRollCall,Scheduler,andSpeedGrader.
• CanvasisonasignificantlymoreaggressiveupdateschedulethanBlackboard,anditsroadmapanddirectionareclearerandbetterarticulated.
• Usingacloud-hostedlearningmanagementsystemhasnumerousadvantages,includingreducedliabilityforourlocalinfrastructure,greatervendorengagementand,ideally,theabilitytofocusstaffresourcesonimprovingprocessratherthantroubleshootingproblems.
• Movingtoacloud-hostedLMSalsohassomedownsides,includinggreateropacitywhensolvingproblems,lesscontroloverpermissionsandinterface,andreducedaccesstologsandusagedata.However,thesedownsideswouldlikelybepresentinthefutureroadmapsofbothplatforms,asBlackboard’sproductroadmapisentirelyfocusedontheSaaSspace,withplansforsignificantuser-interfacechangesanddecidedlylessclarityinproductdeliverables.
• Withregardtofunctionality,therearenocriticalfeaturesinBlackboardthatcannotbeprovidedbyCanvas,eitherbythecoreapplicationorwithexternaltools.Infact,manycampuscollaborationtoolsareeasiertointegrateandconnectwiththeCanvascloudinfrastructure(Box,Ares,ECHO).
Insummary,ourrecommendationsareasfollows:
• AdoptCanvasasthecampusLearningManagementSystem.• DevelopandcommunicateacomprehensiveLMStransitionplanthattargetsallnewcoursesitesusingCanvasbyFall
2017.• Createanddocumentabestpracticescoursegallery,completewithcoursetemplatestofacilitatefacultytransition
andencourageexemplarycoursedesign.• DevelopprocessesandpracticesaroundtheevaluationandadoptionofLTItoolsandAPIconnectorsthatextendthe
Canvasproduct.• SecureInstructureTier1supporttoprovide24/7supportforfacultyandstudentsforthedurationofthe
transition/implementation.• Outlineoptionsandbestpracticesfororganizations(non-course)useofCanvas,withguidesandlinkstoadditional
campuscollaborationtools(i.e.Box,Office365Sites,OrgSync,CanvasGroups).
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
2 Teaching&LearningTechnologies|April2016
PilotProjectOverviewA.Premise:Inthefallof2014,LITScompletedtwoBlackboard-relatedprojects(BlackboardCSI–ContinualServiceImprovement–andBlackboardAnalytics)toestablishandmeasurestandardsforcustomersatisfaction,productfunctionality,productusability,andinstructionaltechnologymethodology.WhileBlackboarduse,asmeasuredbynumberofactivecourses,increasedforEmoryandOxford,theuserbaselinemeasuresgatheredrevealedahighrateofdissatisfactionwiththeusabilityandperformanceoftheBlackboardlearningmanagementsystem.Keytake-awaysfromtheprojectsinclude:
• HighratesofdissatisfactionfromstudentsandfacultyabouttheirexperiencesusingBlackboard• AdoptabilityofBlackboardtoolsandfeaturesisrelativelylow.WhileoveralluseofBlackboard
hasincreasedacrosscampustoover60%ofcoursesusingitinsomeway,thatusageismostlycenteredondocumentstorageandclasscommunication,withmoreinteractivetoolusagebeingdecidedlylow.
• FrustrationamongstudentsaboutinconsistencyoffacultytoolusewithintheirBlackboard-enabledcourses.
• ManypeerinstitutionshavemovedfromBlackboardinthelast2years,notingsimilarfacultydissatisfactionwithusabilityandfunctionality.
• ReportsfromourBlackboardtechnicalteamnotethatBbtechnicalsupportisnon-responsiveorverylatetorespondtosoftwareandQAissues.
• GivenBlackboard’shighcostofownershipforin-houseinfrastructure,associatedoperationalrisks,andBlackboardInc.’sincreasedfocusontheirSoftware-as-a-Serviceproducts,weidentifiedaneedtoseriouslyconsidermovingtoBlackboard’sSaaSproductifwearetocontinuewithBlackboardasthecampusLMS.Itwasdeemedprudenttobeginalearningmanagementsystem(LMS)evaluationonashortertimetable.
LMSMarketTrends&PeerInstitutionsTheLMSmarket,likeothertechnologyindustries,isundergoingamajorshiftcurrentlyfromon-premisessolutions,wheretheLMSisphysicallyinstalledandmanagedbytheinstitution,toacloudsolution,wheretheLMSishostedormanagedbytheLMSprovider.Emory’scurrentLMSison-premisesBlackboardLearn9.1andissupportedwithlocalstaff,serversandstorage.OurlicensewithBlackboard9.1expiresinthelatesummerandisrenewableannually.
Blackboard’scurrentroadmapisfocusedontheSaaSspace,withnewproductdevelopmentonlyavailabletoManagedHostedorSaaScustomersandminimalcontinueddevelopmentforon-premises9.1installations.Blackboard’scurrentstrategyisdominatedbyaninterfaceparadigmcalledUltrathatrepresentsasubstantialchangeintheuserexperience.SaaSBlackboardwillsupportalegacyLearn9.1look-and-feel,aswellasanUltra-themedversionoftheLearn9.1interface,butmostofthestrategicfocusisontheUltraCourseExperience.ThisentirelynewBlackboardconcepttakesmanyoftherapiddevelopmentpracticespreviouslyadoptedbyCanvasandisanattempttobringBlackboard’sinterfaceandpaceofinnovationuptomodernstandards.
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
3 Teaching&LearningTechnologies|April2016
TheUltrainterfaceis,asofthiswriting,inaveryearlydevelopmentphaseandsupportsonlythemostbasiccoursefunctionality.Developmentanddeliveryoffundamentaltechnologyrequiredtosupporteventhemostbasiccourseshasbeenslowincoming,andtherehavebeennumerousmisseddeliverydatesandaconfusinganddisjointedrolloutprocessandstrategy.Thecurrenttechnicalpreviewsuggestsauserinterfacedirectionthatcouldprovedifficulttosupportandsocializewithfaculty,andthereissomeconcernaboutBlackboard’sabilitytodeliveraspromised.
Basedonresearchofourpeerinstitutions,theInstructureCanvascloudsolution/hostedplatformistheleadingalternativetoBlackboard,withthemajoritycitingeaseofuse,coreLMSfunctionality,androbustcloud-basedservicesaskeytoselection.
Institution CurrentLMS
Website PreviousLMS
BrownUniversity Canvas http://brown.edu/it/canvas/ Blackboard
Dartmouth Canvas http://sites.dartmouth.edu/canvas/canvas-learning-center/
Blackboard
HarvardUniversity Canvas http://tlt.harvard.edu/canvas iSites
NorthwesternUniversity
Canvas http://www.it.northwestern.edu/education/learning-management/index.html
Blackboard
StanfordUniversity
Canvas http://gocanvas.stanford.edu Sakai
UniversityofPennsylvania
Canvas http://www.library.upenn.edu/news/1194 Blackboard
YaleUniversity Canvas http://canvas.yale.edu/overview.html Sakai
UniversityofTexasatAustin
Canvas http://www.utexas.edu/its/course-mgmt/index.php Blackboard
Berkeley Canvas http://ets.berkeley.edu/bcourses/ Sakai
EmoryLMSHistoryThehistoryofEmoryUniversity’sLMSusehasbeenevolutionary.Learnlink,amessagingsystembasedontheFirstClassplatform,beganasanemail/e-bulletinboardplatformandwasusedunofficiallyasanLMSbymanyfacultymembersforyears.GoizuetaBusinessSchoolalsohadaseparateinstallationofFirstClassthatservedastheirprimaryLMSduringthesametimeperiod.BlackboardhasbeeninuseatEmorysince2002,butwasinitiallyanopt-inserviceforfacultywithinconsistentadoption.DuetomarkettrendsmovingtowardWeb-basedsolutionsratherthanFirstClass’smorearchaicclient-servermodel,bothLearnlinkandtheGBSFirstClassservicesweredecommissionedin2014,makingBlackboardtheonlysupportedLMSforEmory’scampus.
Duetofacultydemandandstudentexpectations,LMSusehasbecomeintegraltostudents’learningexperiences,facultyteachingandresearchefforts,andinstitutionalcommunicationforEmory.SinceLITSbegansupportingBlackboardmorethan12yearsago,wehavenotformallyassesseditseffectivenesscomparedtootherLMSproductsonthemarketnorassessedthebusinessvalueitaddsfor
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
4 Teaching&LearningTechnologies|April2016
ourteachersandlearnersandtheschoolstheysupport.BothobjectiveandanecdotaldatapointtoBlackboardasaservicethatfacultyandstudentshavecometodependupon,evenastheystrugglewithitsarchaicuserinterfaceandinconstantfunction.
B.PilotGoals:Thegoalofthepilotwastoevaluateandrecommendabest-of-breedLMSsolutionthatwillprovidethebestcombinationoffunctionalityandusability,aswellascandeliversoundpedagogicaltoolstoEmory,allwhilegainingthehighestpossibleadoptioninordertoenhanceacademicinstructionforstudentsandfaculty.ThestateofourcurrentBlackboardinfrastructuremakesthiseffortadditionallychallengingbecauseofthebreadthofusecasesandthelackofstandardmethodsforindividualfacultycoursecreation.
OneofthemostcommoncomplaintsaboutBlackboard,bothinobjectivesurveydataaswellassubjectiveconversationswithfacultyandstudents,istheopaqueanddensenatureoftheuserinterface.FacultyandstudentsalikeexpressfrustrationwiththelackofstandardWebUIconventionslikedrag-and-dropfileuploading,aswellasthewaysinwhichfunctionality,wherepresent,isobfuscatedbythecomplexityoftheinterface.OneofthemostcommonissuesweexperiencewithnewBlackboardusersisthedifficultyinfiguringouthowtomakeacourseavailabletostudents.Theprocessisrequiredbyeveryfacultymembertoenabletheircourses,andtakesfiveclicksintothecontrolpanelinBlackboard.ThesameprocesstakesasingleclickinCanvasandispresentedatthetopofthecoursehomescreeninvividcolor.
TheinitialsemesteroftheCanvasPilotprojectwasfocusedonanassessmentofoverallfunctionalityandtoevaluatethedegreetowhichCanvaswouldbeanadequatereplacementforBlackboard.Ourfocuswasonintensivesupportofasmallnumberofcoursestolearnandassesstheplatform.Thesecondsemesterpilotwasmorefar-reaching,withtheintenttoseeifCanvascouldprovideanappropriatelyrobustexperienceforsomeofourmoredemandingLMScustomers.Wetookadvantageoftheopportunitytoexperimentwiththegradebook,communicationandassessmenttools,andtoevaluateCanvas’supportoptions.
C.PilotSchedule:InMarch2015,acallforparticipationwasannouncedthroughtheInstructionalDesign&TechnologyCommunityofPractice(IDTCoP)forfacultyandstaffwhowouldbeinterestedinpilotingCanvasforFall2015.Teaching&LearningTechnologiesthenvettedfacultywhoresponded,toensureadiversesetofcourses,students,andinstructionalcontent.Eighteen(18)courseswereselectedtopilottheCanvasLMSatEmoryforFall2015semester.ThosecourseswererepresentativeofallofthemajoracademicunitsatEmoryUniversityandOxfordCollege.AsimilarcallforparticipationwasannouncedinNovember,2015,foraspring2016expandedpilot.Theexpandedpilotincreasedthenumberoftotalpilotcoursesto82,with77faculty,1,900students,and
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
5 Teaching&LearningTechnologies|April2016
112sandbox/organizationstestingtheCanvasplatform.SeeAppendix1forfulllistofpilotfaculty/courses.ScheduleTheprojectschedulewasdevelopedtopilottheLMSatthebeginningoftheUniversity’sacademicyear(fallsemester).Itwasexpandedforspring2016.
April–May2015 Callforfacultyparticipation&vettingofsubmittedcourses
AnnouncementofPilotcampusesandcoursesSetuppilotenvironmentandgetaccesstoCanvasenvironmentPilotprojectteamkick-offmeeting
June2015 DevelopmentoftraininganddocumentationSetupCanvaspilotenvironmentCreatetestcoursesitesforparticipantsSetupcommunicationmechanisms(listserv,discussionboard)forprojectparticipantsTeamattendedInstructureconferencetogatherdataandlearnaboutbestpractices
July-midAugust2015 Orientationandtrainingforfacultypilotusers(18total)StartdesigncoursesitesforfallcoursesBeginningofconsultationandsupportprovidedtofacultyEnrollmentofstudents
August2015 DeliveryofcoursesutilizingCanvasforFall2015Begindevelopmentofprojectassessmentandevaluationtools
September–October2015 AssessmentandevaluationtoolscompletedContinuousconsultationandsupportprovidedtofacultyCollectdataonfacultyandstudentuseofCanvasCompleteanalysisofdatagatheredfromfacultyandstudents
November2015 CompletefinalreportwithrecommendationtoexpandpilottoEmoryUniversityleadershipCallforspringpilotusers
December2015 ExpandPilotforSpring2016
January–March2016 SpringPilotlaunchedwith59newfacultyparticipantsAssessmentandevaluationtoolscompleteCreatedweeklyvideoshighlightingCanvasfunctionsusingtheCanvasWebcollaborationtoolsTeammetwithITandInstructionalDesignrepresentativesfromacrossthevariousprofessionalschoolstodocumentexperiences,andconcerns
April2016 CompleteanalysisofdatagatheredfromfacultyandstudentsRecommendationreportcompleteandpresentedtoITGovernance
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
6 Teaching&LearningTechnologies|April2016
D.Communication&Outreach:AWordPresssitewasdevelopedtokeepthegreatercampuscommunityawareoftheCanvaspilotprogressandevents:https://scholarblogs.emory.edu/canvaspilot/.Duringthespringsemester,videotutorialswereaddedweeklytothesite,showcasinguniquefeaturesofCanvasandbuildingacollectionofreusabletrainingmaterial.ACanvastoBlackboardcomparisontoolwasalsocreatedandpostedtothesitehighlightingdifferencesbetweenthetwoLMS’sandprovidinglinkstoonlineresourcesonhowtouseeachtoolareawithinCanvas.
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
7 Teaching&LearningTechnologies|April2016
TheUniversitycommunitywasinvitedtolivedemonstrationsoftheCanvasLMStwicemonthlybetweenSeptember2015andApril2016.The“CanvasDrop-inDemo”sessionswereheldinWoodruffLibraryfrom12-2pm.TheTeaching&LearningTechnologiesgroupalsoledLunch&LearnsessionsforfacultyatOxfordCollegeregardingCanvasandthepilotprojectinlateSeptemberandOctober2015,andaCookies&CanvaseventforOxfordstudentsinFebruary2016.Twostudent-focusedoutreachsessionswereheldonmaincampusduringWonderfulWednesday,wherestudentsgavefeedbackaboutBlackboardandtheirLMSpreferences.
E.Training:FacultytrainingforfallpilotparticipantswasdeliveredinlateJuly/earlyAugust2015.TrainingeventswereheldinDecemberandJanuaryforthespringpilot.PilotfacultywereprovidedthreeoptionsforCanvastraining:
• Anonlinetrainingcoursethatincludedstep-by-stepvideoandhandoutdirectionsonhowtobuildouttheirCanvascourse,
• Atwo-dayface-to-facesession,witheachsession3hourseach,and• Anallday,6-hourface-to-facesession.
Eachorientationwasmonitored/facilitatedbyTeaching&LearningTechnologiesstaff.Theonlineorientationwasasynchronous,withnovirtualmeetings;facultywereabletoworkthroughthecontentofthistrainingcourseattheirownpace(printingonlineguidesandwatchingvideo),withcontentthatwasavailable24/7.Allpilotfacultyhadaccesstoablankcourse,wheretheycouldimmediatelytryoutthenewskillstheywerelearningduringtrainingandultimatelybuildafullydevelopedpilotcourse.FacultyrespondedpositivelytomanyCanvasfeatures(suchasSyllabus&Scheduler),buthadlukewarmreactionstothestandardcontentorganizationpracticeofCanvasandtheeffortneededtotransferexistingBlackboardcontentover.EmorystudentswerenotifiedthroughtheirfacultyaboutparticipationintheCanvaspilot.Teaching&LearningTechnologiesstaffwereavailableduringthefirstweekofclassestogiveanoverviewofthepilotandCanvasplatform.ManyfacultytooktheleadingettingtheirstudentsuptospeedonCanvas.FacultyalsoincorporatedstudentsupportresourcesfromInstructuredirectlyintothepilotcourses.Teaching&LearningTechnologiesstaffalsodevelopedastudent-focusedorientationandsupportsitewithinCanvasforthespring2016semester.
F.Pre-Pilot&MidtermSurveys:Thefacultyparticipantsweresentacustomizedlinktotwoelectronicsurveys(usingSurveyMonkey)viaemailtotheiruniversityemailaddress,orwithintheirpilotCanvascoursebyDr.DanaBryant,LeadInstructionalTechnologist,withTeaching&LearningTechnologies.StudentswhowereenrolledinacoursethatpilotedtheCanvasLMSweresentacustomizedlinktotheirsurveysviaCanvasemailbythe
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
8 Teaching&LearningTechnologies|April2016
pilotcoordinator,Dr.Bryant.ThestudentswereinvitedtovoluntarilyparticipateinthePre-PilotandMidtermSurvey.Follow-upemailsfortheMidtermSurveyweresent1weekaftertoallnon-respondents.ThefacultyresponseratefortheMidtermSurveyinFall2015was100%;inSpring2016was56%.ThestudentresponseratefortheMidpointSurveyinFall2015was28%and12%inSpring2016.Thefirstsurveyincludedclosed-endedandopen-endedquestionsrelatedtotheuser’sexperienceswithBlackboardLMS;thesecondsurveywasrelatedtotheirexperienceusingCanvas,specificallywithregardtoproductintuitivenessandeaseofuse.Eachsurveytookapproximately20minutestocomplete.Noindividualsreceivedcompensationforparticipatinginthesurvey.SeeAppendix2:FacultyMidtermSurveyandAppendix3:StudentMidtermSurveyfortheentirelistoftheCanvassurveyquestionsandresponses.Evaluation
A.SurveyData–FacultyPerspectivesDescriptionofPilotFacultyFall2015therewere20facultyparticipants;inSpring2016thenumbersjumpedto80.Positivewordofmouthfromthefirstsemestercontributedtotheincrease,withmanyfacultyinquiringaboutjoiningthepilotbeforetheextensiontoSpringwasconfirmed.ThemajorityoffacultyparticipantscamefromEmoryCollegeofArtsandSciences,followedbySchoolofNursingandSchoolofPublicHealth.
6.25%
29.69%
4.69%14.06%
7.81%
12.50%1.6%
7.81%
7.81%7.81%
WhatisyourprimaryEmoryaffiliation?
CandlerSchoolofTheology EmoryCollegeofArts&Sciences
GoizuetaBusinessSchool NellHodgsonSchoolofNursing
OxfordCollege RollinsSchoolofPublicHealth
LaneyGraduateSchool SchoolofMedicine
SchoolofLaw Other(pleasespecify)
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
9 Teaching&LearningTechnologies|April2016
Themajorityofpilotfacultyreportedteachinghybridcourses(ablendoftraditionalandonlineinstruction),whichisindicativeoftheinstructionalcultureatEmory.However,allinstructionalmethodswererepresented,asillustratedinthechartbelow.ItshouldalsobenotedthattwoentirelyonlineprogramsatEmory-theRollinsSchoolofPublicHealthExecutiveMPH(http://www.sph.emory.edu/departments/emph/index.html)andNellHodgsonSchoolofNursingOnlinePrerequisitesCourses(http://www.nursing.emory.edu/admission-and-aid/online-prerequisites/index.html)-hadfaculty/coursesthatparticipatedintheCanvaspilot.
CanvasFeatures&FunctionalityRegardlessofcourseformat,thetopthreeCanvasfeaturesusedbyfacultywere:Postingannouncements,creatingassignments,andpostinggrades(gradebook).
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
10 Teaching&LearningTechnologies|April2016
BasedonsurveycommentsandfeedbacktoourEducationalAnalysts,thefollowingCanvasfeatureswerealsopopularwithfaculty:
• RollCall:Withthistool,instructorscankeeptrackofcourseattendancebytakingrollelectronically.Instructorscanchoosetoviewthetoolinalistorgridformatandcancustomizetheplacementofeachstudentintheseatingchart.ThetoolalsocreatesanassignmentintheGradebookandcalculatesattendanceasapercentageofastudent'sgrade.
PilotFacultyFeedback:
“RollCallisusefultocreateseatingchartsandquicklyregisterattendancebyhand,soit’sworthkeepingandisastepintherightdirection.”“ILOVEtheattendancemechanism[RollCall].”
• Calendar/Scheduler:ThisispartofthenewCalendarandcreatesappointmentgroups(collectionofindividualappointments)thatstudentscansignupfor.Forexample,youcanuseSchedulerto:createofficehours,scheduleTAsessions,andassignpresentationtimes.
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
11 Teaching&LearningTechnologies|April2016
PilotFacultyFeedback:
“Schedulerisawesome.ItispreciselywhatIcurrentlyuseathird-partypaidappfornow.”[WhatareyourfavoritethingsaboutCanvas?]“Thecalendar,andthefactthatIcansubmitassignmentstheredirectly.”“Thewayitautomaticallyputstests,assignmentsontothecalendarisgreat.”
• Speedgrader:Asaninstructor,thistoolallowsforviewingandgradingstudentassignmentsubmissionsinoneplaceusingasimplepointscaleorcomplexrubric.CanvasacceptsavarietyofdocumentformatsandevenURLsasassignmentsubmissions.Somedocumentassignmentscanbemarkedupforfeedbackdirectlywithinthesubmission.Youcanalsoprovidefeedbacktoyourstudentswithtextormediacomments.
PilotFacultyFeedback:
“Justlookedthroughassignments[forstudents]torespondandsigntheplagiarismandcollaborationpolicy.It[Speedgrader]wasreallyreallyeasyandquickreviewand"grading"(checkingcompleteorincomplete).TotallytheoppositeofmyexperiencereviewingsubmissionsonBlackboardwhereIreallydidhavetoclickatleastfivetimestogettoeachindividualsubmission.”“IlikethatIcangetallgradesforasinglestudentexportedwithoneclick-andIcanPRINTitforthem!”[WhatareyourfavoritethingsaboutCanvas?]“Thespeedgraderwithaudioandvideooptions.Also,thesubmissionofaudioandvideoassignments.”
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
12 Teaching&LearningTechnologies|April2016
CanvasEaseofUseTheCanvastasksduringthepilotrated“veryeasy”weremonitoringcourseactivityandstudentprogress,sending/receivingmessagestostudents,andgivingfeedbackonstudentwork/submissions;thetaskrated“difficult”wascustomizingthelookandfeel/navigationofmycourseat27%.
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
13 Teaching&LearningTechnologies|April2016
FacultySupportofCanvasAdoptionTheMidpointSurveyaskedspecificallyifEmoryUniversityshouldadoptCanvas.Fiftypercent(50%)ofthefacultyrespondentsinFall2015preferredCanvasandwantedtoadoptit;Spring2016responsesinsupportofadoptingCanvasjumpedto81%.Fall2015
Spring2016
81.00%
19.00%
WouldyousupportEmoryUniversityadoptingCanvastoreplaceBlackboard?
Yes No
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
14 Teaching&LearningTechnologies|April2016
PilotFacultyFeedback:
“PleasepleasepleaseswitchtoCanvasassoonaspossible.IhategoingbacktoBlackboardformyothercoursesnow.”(Fall2015)“ItissomuchbetterthanBlackboard,pleasedon'tmakemegoback!”(Fall2015)“Canvasisawelcomechange.IhadbasicallygivenupwithBB[Blackboard].Also-ourstudentsseemtolikeandhavetransitionedwelltoit!”(Spring2016)
CommentsWehadveryengagedfacultyparticipatinginthepilot,asdemonstratedbylevelofcommitmentandsurveyresponserate.Mostfacultywerenotawareofthelevelofworkinvolvedinreworkingtheircourse/instructionalcontentforuseinCanvas;however,manywerepleasedwiththefinalresult.FacultywouldbenefitfromacomprehensiveformaltrainingandoutreacheffortforasuccessfulCanvasimplementation.SeeAppendix2:FacultyMid-termPilotSurvey
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
15 Teaching&LearningTechnologies|April2016
B.SurveyData–StudentPerspectivesDescriptionofpilotstudentsTheincreasedinterestbyfacultyintestingCanvasresultedinmorepilotcourses,whichinturnproducedalargestudentparticipantpool,totaling1900inSpringsemester.ThemajorityofstudentparticipantsurveyresponsescamefromEmoryCollegeofArtsandSciences,followedbySchoolofPublicHealth,andOxfordCollege(whichonlyrepresented1.6%ofpilotcourses).
CanvasFeatures&FunctionalityStudentswerecandidintheirassessmentofCanvasfeatures.Theirfeedbackcollectedintheopen-endedsurveyquestionscanbecategorizedinthefollowingareas:
• CalendarPilotStudentFeedback:
“Ireallylikehowtheassignmentsarelistedtogetherandbydateandthecalendarviewinthecornerthatshowsthehighlighteddaysofwhenassignmentsaredue.”“Thecalendar(alltheday'sassignmentsfromeachclassinoneplace).”
13.01%
21.10%
8.67%
9.54%8.96%
14.16%
12.14%
7.80%4.62%
WhatisyourprimaryEmoryaffiliation?
CandlerSchoolofTheology EmoryCollegeofArts&Sciences GoizuetaBusinessSchool
NellHodgsonSchoolofNursing OxfordCollege RollinsSchoolofPublicHealth
SchoolofMedicine SchoolofLaw Other(pleasespecify)
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
16 Teaching&LearningTechnologies|April2016
• Communications&Notifications
PilotStudentFeedback:
“Ialsolikethatyoucanmessageotherstudentsinyourclassthroughcanvas“Todolistonthesideofthescreensoicanseeallupcomingassignmentsandthereforenotmissanything.”“Ienjoybeingabletoseeresponsesfromprofessorsregardinggradedassignments.Ialsoappreciatehavingausableandintuitiveapp;Blackboard'sappneverworkedforme.Itallowsformorecollaborationsanddiscussions.”
• SubmittingAssignments&GradebookfeaturesPilotStudentFeedback:
“It'sreallyeasytouploadassignments.”“Abilitytoresubmitassignmentworkifnecessary.”“Beingabletotypeinagradeandseehowyouroverallgradewouldchangeautomatically.”
Pilotstudentsalsoprovidedthoughtful,specificsuggestionsforimprovementinCanvas,whichincluded:
“Canvaswouldbemuchmoretolerableifyoucouldsaveadraft.IhavemultiplewindowsopenandIdon'twanttocreatemydocumentatanotherlocation,soIcancutandpastelater.”“Thetimerforquizzesandtestsisonlyatthetopofthepage,soasyouscrolldownforquestionsyoucannotseethetimeranymoreandhavetokeepscrollingbackuptocheckyourtime.”“Acalculator”
CanvasEaseofUseThetoptaskspilotstudentsrated“easiest”inCanvasweresubmittingworkassignments,exporting/viewingcoursecontent,andnavigatingmycoursetofindthingsinalogicalmanner;thetasksrated“difficult”werenavigatingmycoursetofindthingsinalogicalmanner,viewing/usingrubrics,andparticipatinginDiscussions.
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
17 Teaching&LearningTechnologies|April2016
SomepreliminarycommentsaboutCanvasincluded:
“IthinkCanvasisactuallymoreuserfriendly.Ilikethatdocumentscanbeviewedbeforedownloadingandit’sfasterandbetterorganized.”
“Wellorganizedanduserfriendlyinterface.”
“Therejustseemstobemorevarietyinwhatyoucando,likepeeredit.”
0%10%20%30%40%50%60%70%80%90%
100%
Canvashelpsmeto
completecourse
assignments
Canvasiseasyformetouse.
Canvashelpsmesavetimeandcompletemorecourseactivities.
Canvashelpsme
communicatewithmyprofessor.
Canvasexpandsaccesstoavailable
courselearningmaterials
(print,audio,video,etc).
IndicateyouragreementwiththefollowingstatementsaboutusingCanvas
StronglyAgree
Agree
Neutral
Disagree
StronglyDisagree
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
18 Teaching&LearningTechnologies|April2016
StudentSupportofCanvasAdoptionTheMidpointSurveyaskedspecificallyifEmoryUniversityshouldadoptCanvas.Fiftypercent(50%)ofthestudentrespondentsinFall2015preferredCanvasandwantedtoadoptit;Spring2016responsesinsupportofadoptingCanvasroseto59%.Fall2015
Spring2016
50.90%
22.60%
26.40%
WouldyousupportEmoryUniversityadoptingCanvastoreplaceBlackboard?
Yes No Undecided
58.90%
41.10%
WouldyousupportEmoryUniversityadoptingCanvastoreplaceBlackboard?
Yes No
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
19 Teaching&LearningTechnologies|April2016
PilotStudentFeedback:
“IlikecanvasalotmorethanBlackboard.Everythingiseasiertofind,andIreallylikethetodolistonthesideofthescreen.”
“Ihaveusedblackboardforyearsinanothersetting.IfoundCanvastobemoreintuitive,easiertonavigateandmademore"sense"tomethanblackboard.highlyrecommend.”
CommentsOurstudentpilotpopulationhadlesstime(thanfaculty)togetacclimatedtousingCanvassincetheirorientationcoincidedwiththestartofclasses.Feedbackismoremixedfromthestudents’standpoint;forexample,thereisbothpraiseANDdislikenotedfortheModulespages.WhiletheintegratedcourseCalendarfeaturewasanoverallhit,theDiscussionfeatureinCanvaswashardforsometomaneuver.StudentswouldbenefitfromcomprehensiveandextendedtrainingandsupportforanyCanvasimplementation.SeeAppendix3:StudentMid-termPilotSurvey
C.FunctionalSupportReviewTheEducationalAnalysts,LeadInstructionalTechnologist,andLMSAdminsfromTeaching&LearningTechnologiesformedtheprimaryfunctionalteamprovidingsupporttopilotparticipants,whileexploringthefunctionalityandtechnicalaspectsofCanvas.TrainingforsupportstaffwasprovidedbyInstructure(parentcompanyofCanvas)viawebinarsinJuly2015.Afterfacultyorientationconcluded,eachEducationalAnalystwasassignedindividualfacultymembers/coursestoprovidededicatedsupportforthedurationofthepilot.PilotfacultyusedaDiscussionBoardontheOnlineOrientationsitetoaskquestionsandsharesolutionsduringthepilot.KeyObservationsThegroupwasinagreementwiththefacultyonthepositivefeaturesofferedbyCanvasincludingthedynamic,integratedCalendarandAssignmentfunctions;uniformcoursenavigationdesign;andqualityofCanvasresourcematerials.Interestingly,someoftheCanvasfeaturespraisedbyfacultyandcitedcurrentlylacking(inlinegrading/editsandnotifications)arealreadyavailableinBlackboard,butthecomplexityoftheBlackboardinterfacemayhavehamperedadoptionofthosetools.ThismayspeaktoanedgetowardCanvasregardingintuitivenessandeaseofuse.
Consultsduringcourse-buildingwithfacultyalsorevealedsometoolssuchastheGradeBookandDiscussionarenotasfeature-richasthecurrentLMS.However,theessentialLMSfeaturescanservethemajorityoftheusersandasimplecleaninterfacecanbeaplus.TherearealsoexternaltoolsthatcanenhanceCanvasfunctionalitythatareeasytointegrateduetotheopenLTIframeworkCanvasfullysupports.Intheinstanceofdiscussionboards,theexternaldiscussionboardtoolPiazzawaseasilyinstalledandintegratedintopilotcoursesfortesting.
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
20 Teaching&LearningTechnologies|April2016
ItisclearthatfacultyexpectationsshouldnotbetorecreatetheCanvaspilotcourseasareplicaoftheircurrentBlackboardcourse.Adifferentproductcallsfordifferentcoursedesign,especiallywithregardtocontentorganization(breadthvs.depth).Facultywilldefinitelyneedmorethanonesemestertobecreativeintheenvironment.WhiletheessentialfeaturesofCanvasareeasytouse,theTeaching&LearningTechnologiesfunctionalteamrecognizesfacultyneedmoretrainingandtimetoexploreinstructionaldesignbestpracticesthatarespecifictoCanvas.
FeatureComparisonThemaindistinctionsnotedbytheFunctionalTeamare(1)Modules&Pagesusedforcoursedesign(vs.foldersinBlackboard)and(2)nobuilt-inplagiarismdetectiontoolforCanvas.Blackboardincludesanintegratedplagiarism-detectiontool,whereasCanvassupportsthird-partyplagiarismservicesviatheLTIstandard.Aftersomeresearch,wefoundotherinstitutionshavedocumentedotherfeaturedifferencesandincorporatedtipsfornavigatingthesedifferencesintotheirtrainingandoutreachinitiativeswhiletransitioningtoCanvas.Withregardtoananti-plagiarismtool,CanvashasrecentlypartneredwithVericiteandisrollingoutanativeversionoftheiranti-plagiarismtoolthatintestingissimilarinfunctionalitytoBlackboard’sSafeAssign.ThisisaseparatelicenseagreementandwillneedtobeacquiredforEmoryfaculty.
ManyfacultyandsupportstaffalsoexpressedaninterestinseeingcoursesthatexemplifiedbestpracticesfortheuseofCanvasininstruction,aswellasmoredetailsonhowBlackboardfeaturescompareandtranslateintoCanvas.Providingsamplecourselayoutsandtemplates,aswellassupporttoolsfortransitioningfromoneLMStoanother,willbeanimportantcomponentofanyimplementationplan.
EducationalAnalystswillcontinuetolookatthequalitativefeedbackforissuesthatshouldbeaddressedwithCanvastraininganduseasaguidetoimprovetheuserexperienceaspartofanyproposedimplementation.
AccessibilityCanvasstatesitwasdesignedwithaccessibilityinmind,withspecialattentiontoensuringallCanvascoursesscreen-readable,aswellaslimitingcustomizationsofcolorsandschemessothatcontentisaccessibleforallusers.AsofMay2015,WebAIM.org,athirdpartyauthorityinwebaccessibility,certifiesCanvastobesubstantiallyconformantwithLevelAandLevelAAoftheWebContentAccessibilityGuidelinesversion2.0.Canvas/InstructureprovidesaVoluntaryProductAccessibilityTemplate,orVPAT,forLMSadministratorsandinstructionaldesignerstoevaluateCanvas'conformancewiththeaccessibilitystandardsunderSection508oftheRehabilitationActandtheActWCAG2.0AAStandards.Seehttps://community.canvaslms.com/docs/DOC-2061formoreinfoonaccessibilitywithintheCanvasLMSplatform.
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
21 Teaching&LearningTechnologies|April2016
InstructureSupportInstructure(parentcompanyofCanvas)providedPremiumTier124/7UserSupportduringthispilot.AllcoursesfeaturedthefollowingHelpfeatureembeddedwithin:
ThePremiumTier1UserSupportprovidesend-userswithanInstructurephonenumberfordirectsupportfromthevendor,24hoursaday,alongwithenhancedchatandemailsupportavailablearoundtheclock.Teaching&LearningTechnologiesprovidedInstructurewithguidance/exampleresponsesforanyinquiriesthatfelloutsideofthescopeofInstructuresupportsothattheinquirywouldberoutedbacktoEmoryforappropriateresolution.InstructureCommunityInstructure’sproductdevelopmentprocessisfocusedona“student-centered”designmethodologythatintegratesbothusagedataandcommunityfeedbackintoongoingiterationsandfuturedirectionoftheplatform.BecauseofthehostednatureofCanvas,Instructureisabletoreacttousagetrendstobetteranticipatestudentandfacultyneedsandprioritiesthanourcurrenton-premisesBlackboardproduct,whichhasverylong,monolithicupdatecycles.UpcomingproductfeaturesaredirectlyinformedbyfeaturerequestsintheCanvascommunity,andthecompanyactivelyandpubliclyengageswith
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
22 Teaching&LearningTechnologies|April2016
customersinthatforum.Likewise,customershavenumerous“birdsofafeather”groupswithinthecommunitytosharesolutions,scripts,andbestpracticeusesoftheapplication.SupportSurveyResponsesPilotFacultyFeedback
“Theironline(chat)supporttotallyrocks!Everysingleoneofthe(multiple)timesthatIhavehadtoopenachattoaskforhelpfiguringsomethingouttheyhavebeenfriendlyandhelpful.Sometimestheywereabletojustpointmetowardsanonlinehelparticleandothertimeswehaveworkedproblemsouttogetheronlinebutitwasalwaysareallypositiveexperience;whichisnotalwaysthecaseinthecomputerITuniverse.”“IwouldhighlyencouragecolleaguestojustgetonChatLivewithCanvas–theyareamazingbeyondwords!!!ItisfareasierforeveryoneasCanvaspeoplerespondevenat2amonSundays!!Ihaveusedhoursofhelpsofar,andwithoutthemIwouldnothavebeenreadyyesterday[1stdayofclasses]”
AggregateDatashowsamajorityoffaculty(38%)usedsearchofguides(availabletoanyCanvascustomer),butthesecondlargestgroup(21%)usedlivechat(partofTier1support).Manyfacultyexpressedaneedforthisforcoursebuildingandquestionsoutsideofregularbusinesshours—whenfacultyareusuallybuildingcourses.23%offacultystatedtheydidnotuseanysupportatall,whichmayspeaktoeaseofuseandtherelativelyshortlearningcurveforCanvas.
38.33%
1.67%
3.33%
21.67%
8.33%
23.33%
3.33%
PleaseindicatethefollowingCanvassupportresourcesyouusedduringthepilot:
SearchoftheCanvas(online)Guides
ReportaproblemtoCanvas
CanvasSupportHotline(844-765-2516)
LiveChatwithCanvasSupport
AskingquestionwithintheCanvasCommunity
NA(IdidnotneedtouseanyoftheCanvasSupportresourcesabove)
IwasnotawareofanyoftheCanvassupportresources
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
23 Teaching&LearningTechnologies|April2016
AggregateDatashowsthemajorityofstudentsdidnotneedtousesupportresources(61%).Studentstendtobefamiliarwithmostlearningmanagementsystems,somereportinghavingusedCanvasbeforeinhighschooloratotherinstitutions.Afairnumberwerenotawarethatonlinesupportwasavailable(29%),showinganeedtoimprovemessagingtoletstudentsknowthisresourcesisavailable.
OrganizationsForhistoricalreasons,alargenumberoforganizationsandnon-coursecontenthasbeenhostedinBlackboardatEmory.WithchangesincentralITservicesandtheadditionofvariouscloudofferingsoverthepastfewyears,manyorganizationslookingforonlinecollaborationtoolshavemoreoptionsthanwerepreviouslyavailabletothecampus.WhileCanvasdoeshavea“Groups”feature,it’sprimaryfunctionisinsupportofacademiccoursecontent.Organizationsthatsimplyneedaccess-restricteddocumentsharingtoolsshouldbedirectedtootherservicesdesignedtosupportonlinecollaborationthatarenoweasilyavailable(Box,Office365Sites,OrgSync,etc.).Thosewhomaybenefitfromcourse-likefeaturesfoundwithinanLMS(ex.discussionboards)canbeaccommodatedwithintheCanvasplatformasaCanvasGroup.
6%
1% 0%1%
2%
61%
29%
PleaseindicatethefollowingCanvassupportresourcesyouusedduringthepilot:
SearchoftheCanvas(online)Guides
ReportaproblemtoCanvas
CanvasSupportHotline(844-765-2516)
LiveChatwithCanvasSupport
AskingquestionwithintheCanvasCommunity
NA(IdidnotneedtouseanyoftheCanvasSupportresourcesabove)
IwasnotawareofanyoftheCanvassupportresources
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
24 Teaching&LearningTechnologies|April2016
D.TechnicalReviewArchitectureCanvasisaSoftwareasaService(SaaS)applicationhostedonAmazonWebServices.Thisarchitectureprovidesscalability,nodowntimeformaintenance,andpermanentretentionofdata.EachuserlicensedinCanvasreceives0.5GBofdatastorageaggregatedglobally.Canvasoffersitsinstitutionsthreeinstances:Production,Test,andBeta.TheBetainstanceprovidesearlypreviewsofupcomingenhancementsandfeaturestoCanvasbeforetheyarereleasedtoTestandProduction.Whileitisadvantageoustoseetheseearlypreviews,Instructurehasastrictrefreshschedulethatcannotbealteredorcontrolledbyanyinstitution:BetaisrefreshedweeklyandTestisrefreshedevery3weeks.ApplicationAdministrationandCustomizationCanvasisadministeredviaeithertheWebGUIorbyscriptingwiththeWebServicesAPIs.Institutionaladministratorsaregivenaccesstothesystematsub-administratorlevelwithaccesstoalimitedsetoftabsandbuilt-inconfigurationoptions.Thoughthisfreesuptheadministratorstopursueotheravenuesofexternalintegrations(listedinfollowingsubsection)andscripting,itpresentslimitationsrelatedtopermissions,built-infeatures,andcustomization.WhiletheGUIadministrationinterfaceislimitedincustomizationoptions,institutionaladministratorscanaddaglobalJavaScriptandCSSURL.Thisaddsagreatlevelofcustomizationinwhatourusersseeandwhattheyhaveaccesstowithintheircourses.Atpresent,CanvasreliesheavilyonCSStoallowsystemadministratorstocustomizeaspectsofthepermissionsmechanismnotdirectlysupportedbythenativeroleoptions.ThisdependenceonhidingfunctionswithCSSratherthantrulydisablingthemwillrequireadditionalscrutinyonthepartofsystemadministrators.IntegrationsSIS:IntegrationusingfeedfileswithinCanvasappearstobeverysimilartoBlackboard'sprocess,allowingintegrationbetweenCanvasandPeoplesoftwithminimaleffort.However,duetothewayCanvashandlesoverallintegration,includinglackoflogging,alocalservertomanageintegrationfilesandautomationwouldbeneeded.CanvasdoesnotsupporttheindustrystandardIMSLIS,preventingrealtime'outofthebox'integrationwithPeoplesoft,thoughitisontheirroadmap.LTI:InstructureprovidesmyriadthirdpartyapplicationsthroughtheirEduAppCenterthatcanbeinstalleddirectlyintoCanvasviastandardLTI.Duringtheset-upofourenvironmentforthepilot,weutilizedtheEduAppCenterforapplicationssuchasYouTube,Box,andRespondusLockdownBrowsersuccessfully.IntegrationscurrentlyinourBlackboardproductionenvironmentwerealsosuccessfullytested,includingECHOandAresCourseReserves.SomepolicyconsiderationsaroundindividualfacultycreatingtheirownLTIintegrations,aswellasmanaginguserexpectationsaroundLTIintegrationstheymightrequest,willbeneededforfullproductionimplementation.
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
25 Teaching&LearningTechnologies|April2016
API:Canvasoffersarich,well-documentedsetofREST-basedWebservicesAPIsthatallowforautomationandintegrationsthroughoutmultiplelayersofthesoftwarestack.CanvasAPIscriptsareinvokedwithanapplicationaccesskeythatmustbecarefullyprotected,astheaccesskeyprovidesanyaccesstoCanvasadministrationaffordedtothekey’screator.Nearlyanynon-LTICanvasfunctionaccessiblethroughtheGUIcanbeaccessedormodifiedwiththeWebservicesAPIs,andtheyarecriticaltolearningtoautomateandoptimizeuseoftheCanvasinfrastructure.BlackboardwillprovideitsownsetofREST-basedAPIsintheSaaS-basedUltraenvironmentbut,asofthiswriting,thelistoffunctioncallsisminimalandrestrictedtouserandcoursecreationandmodification.SecurityAuthentication:Duringourset-upphaseforthepilot,wewereabletoimplementShibbolethsuccessfullywithCanvassothatallusersinvolvedinthepilotcouldusetheirEmorylogincredentials.Audits:Canvashasprovidedyearlysecurityauditsofitsapplicationandenvironmentatitsexpense.Logging:Providedthroughtheapplication,thoughloggingofCanvasadminaccessisminimal.DataAsnotedabove,Canvasstorescustomerdatapermanently.Thiscanbeadvantageousforrecoveryoptions,butitalsopresentsanissuerelatedtodataretentionpolicies.DatacanbepermanentlydeletedthroughsupportrequeststoCanvas’helpdesk,butInstructure’sexpectationisthatmostinstitutionswillmaintaintheirdataindefinitely.CentraltoourtechnicalreviewisaccesstoourdataintheCanvasdatabase.Canvashasmadeseveraldifferentdataoptionsavailableforusers,includingfreeaccesstodatadownloadsevery24hours.Thedatadownloadshavetobeloadedintoalocaldatabasetobeprocessedandviewed.ThedatabaseschemaisdocumentedintheCanvasDataPortal.Forinstitutionsthatdon’twanttorunalocaldatabase,InstructureoffersanoptionalhosteddatabaseintheAmazonWebServicesspaceusingAmazon’sRedShiftservice.ReliabilityDespiteCanvas’AmazonWebServicesunderpinnings,wehaveseenperiodic,briefissuesshowupbothinactualpracticeaswellasdocumentedontheCanvasstatuspage(http://status.instructure.com/history).Duringthepilot,Canvashasexperiencedbetweenthreeandtwelveservice-impactingissuesduringthecourseofagivenmonth,withissuesrangingfromdegradedperformancetoregional“whitescreen”errorsforcertaincustomers.OneofthefundamentalrealitiesofmovingtoaSaaSenvironmentisthattheperformanceofaservicebecomesentirelyvendor-dependent,versusanon-premisesimplementationwherelocalstaffcanhavecontroloverthehardwareandsoftwarestack.Inmostcases,though,issuesobservedonthestatuspagehavenotresultedinuser-reportedproblemsonourcampusandCanvashasrespondedquicklyandtransparentlywithresolution.
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
26 Teaching&LearningTechnologies|April2016
Summary&RecommendationInsummary,keyfindingsfromthe2015-2016CanvasPilotatEmoryinclude:thesystemisoveralleasytouse;transferofBlackboardcoursecontentcanbechallengingdependingoncomplexity;theuserinterfaceissimpleandmodern,butthelowerdegreeofcustomizationcanbeinterpretedaspositiveornegative;someofthenewfeaturesareverywell-liked;performancethroughthehostedservicewasgenerallyreliable.ThispilotprojectprovidedusadditionalinsightsonfeaturesofferedandtheactualexperienceofusingCanvasfromfacultyandstudents.Whilethefeedbackandexperiencesarelargelypositive,therearecertainprogramsandgroupsthatexpressedconcernswithfunctionalityofCanvasduringthepilot.Thiscanpossiblybeattributedtothefollowingfactors:limitedpersonalizationofCanvascoursenavigation;users’overallresistancetochange;facultynothavingenoughtimetouseCanvasdifferentlythantheircurrentBlackboardpractice(s);studentsneedingmoreinitialorientationtotheCanvasplatform;andgeneralfrustrationwithusingtwolearningmanagementsystemsduringasemester.Evenwiththeseconcerns,81percentofthefacultyparticipatinginthespringpilotstatedtheysupportamovetoCanvas.FeaturesandfunctionsthatweremostvaluedincludetheCalendar,RollCall,Scheduler,SpeedGraderandtheintuitiveandmodernnatureoftheinterface,mostfeaturescurrentlynotavailableinBlackboard.PilotusersalsorespondedpositivelytoInstructure’s24/7Tier1support.Canvasdoesnotprovideaone-to-onereplacementforeveryfeatureandtoolfoundinBlackboard.Tothatpoint,manyfacultyreportedthatthecombinationofTeaching&LearningTechnologiesandInstructuresupport,alongwithhavingextendedtime(1-2semesters)usingCanvas,provedcriticaltorevisingcourseinstructionaldesigntocomplimentCanvas’uniqueenvironmentandfunctionality.Fromatechnicalperspective,thepilotshowedthatthenativecloudarchitectureprovidesasimplerintegrationwithexternaltoolproviders,aswellasstreamlinedupdateswithnosystemdowntimeandcontinuousproductrefinements.ThereisalsosignificantcostsavingsfrommigratingtheLMSservicetoanativecloud-hostedplatformversusmaintainingon-campusserversandsupportinginfrastructure.TheCanvasopenplatformallowsLMSadministratorstofocusmoreontesting,installation,andsupportofLTI/APIenhancementstothelearningmanagementenvironment,insteadofafocusonapplicationadministration,legacytools,andrecurringbreak-fixthatisinherentinthecurrentBlackboardself-hostedmodel.Blackboard’sexistingSaaSmodelisanotheroption,butis,inouropinion,ariskierstrategicdirection.ThecompanyhasyettoarticulateaconsistentandclearstrategyforthefutureofitsUltraproduct,andthereisagreatdealofskepticismamongthosewhotrackthelearningmanagementsystemproductspacethatBlackboard,Inc.cancompletelyrewriteandreinventtheircurrentplatformwhilemaintainingthefewtechnologyadvantagescurrentlyseenwiththatproduct.EithermovingtoCanvasorstayingthecoursewithBlackboardwillinvolvesignificantchangemanagementchallengesforourfacultyandstudents,butInstructurehasmadeamorecredibleandarticulatecasewithCanvasandthemajorityofourpeerinstitutionsagree.
LMS Review/Canvas Pilot Project Recommendations – Spring 2016
27 Teaching&LearningTechnologies|April2016
OurpilotresearchshowsthatCanvasdirectlyaddressestheconcernsofdissatisfactionwithBlackboardexpressedbyfacultyandstudentsovertheyearsinongoingsurveys.AmovetoCanvasalsobringsusinlinewiththemajorityofourpeersandwiththeemergingindustrystandardLMSplatform.Amigrationtoanewlearningmanagementsystemwillnotbetrivial,butthereareclearlong-termbenefitstotheEmorycommunityineaseofuse,systemreliability,andexpandedfunctionality.Basedonpilotfeedbackandafunctionalandtechnicalreview,weproposethefollowingrecommendations:
• AdoptCanvasasthecampusLearningManagementSystem,withfullcampuslicensingstartingJune1,2016toprovideservicecontinuityforpilotfacultyteachingsummercoursesandfourfullsemestersformigration.Blackboardwillcontinuetobelicensedandsupportedforthe2016-2017academicyeartoallowsufficienttimefortrainingontheCanvasplatformandcoursemigration.BlackboardlicensingwouldnotberenewedSeptember1,2017.
• DevelopandcommunicateacomprehensiveLMStransitionplan,detailingtechnicalandfunctionaltasksrequiredtomovetofullproduction;outliningfaculty,student,andinstructionalstaffsupportoptionsformigration;coordinatingeffectivecommunicationtothecampusaboutimplementationtimelines;andfullydocumentingandpromotingCanvastrainingplansforthe2016-2017migrationperiod.
• Createanddocumentabestpracticescoursegallery,withstartercoursetemplatesandpages,toprovideexamplesandready-to-usecourseshellsmodelingexemplarycoursedesignwithintheCanvasplatformandencouragingbothefficientandeffectiveonlineinstructionalcontentcreationanddelivery.
• DevelopprocessesandpracticesaroundtheevaluationandadoptionofLTItoolsandAPIconnectorsthatextendtheCanvasproduct,leveragingthenativecloudarchitectureandstandards-baseddesigntoprovidenewfunctionalityandautomation.
• SecureInstructureTier1supporttoprovide24/7supportforfacultyandstudentsforthedurationofthetransition/implementation.Thiswillcomplementandexpandonexistingcampusinstructionalsupportresourcesforamoreseamlessandexpeditedmigration.
• Outlineoptionsandbestpracticesfororganizations(non-course)useofCanvas,withguidesandlinkstoothercampussolutionsthatmayprovidemoreappropriateandusefulfunctionalityforonlinecollaborativework(Box,Office365Sites,OrgSync,CanvasGroups,etc.).