An introduction to the what, where, who, and what-for of Analytics

Contents (pg 1 of 7)What is Analytics Where is CCCOnline in terms of Learning Analytics?What is the Desire2Learn Analytics product? What can it actually do? What have other institutions done? Where are other institutions going?

What are Learning Analytics to us?Analytics is processing data in some fashion that will help us do our jobs as administrators or instructors.It is similar to and includes earlier fields/fads, such as educational data mining, but implies visualization of data so as to be made more useful to faculty and staff.

What is CCCOnline up toDesire2Learn progress trackingFaculty in-attendance alertsStudent no-show reportsDesire2Learn AnalyticsBehavior analysis

D2L Progress ToolNot graphical, all tables

D2L Analytics Faculty PortalWhat are my students doing at a glance?Tool useGrade patterns

Quiz Consistency AnalysisDoes my quiz measure just one thing?

D2L Analytics Proper

D2L Analytics data domainsSessions When have they been in their course?Tool use When did they go into the discussions?Content access What have they read?Difficulties with contentGrades Various gradebook designsQuiz question grades

What are other institutions doing?What is out there that we want to achieve as well?Who is doing what?Visualizing dataStandard reports - What happened?Ad hoc reports - How many how often and wereQuery/Drill down -Where exactly is the problem? Alerts - What actions are needed?Statistical Analytiss - Why is this happening?Forecasting/Extrapoluation -What if these trends continue?Predictive Modeling - What will happen next?Optimization - Whats the best that can happen?

Katholieke Universiteit LeuvenMonitor WidgetVisually compare your time in class or resources accessed with your peers.Am I doing what I should be in order to be successful?

SNAPPUniversities of Queensland and Wollongong, AustraliaUniversity of British Columbia, Canada

University of BelgradeLOCO-Analyst

Local-AnalystContent Access & Analysis

Loco-AnalystSocial Network Analysis

Minnesota State College and UniversitiesAccountability dashboard

Predictive modeling

Signalshttp://www.itap.purdue.edu/tlt/signals/signals_final/index.htm

Signals illustrated

Signals Faculty DashboardStudent success at a glancePrepare and dispatch custom intervention E-mails

American Public University SystemFor profit university serving over 80k online students.Collects almost a hundred metrics based on student demographics, prior grades, and current course data.Metrics are fed into a Neural Network that compares the metrics to grades in previous semesters, ranking the students from 1-80k in their chances of success.The user can drill down to find out exactly what makes the network think a student will fail.

Recommendation Engine Fruanhofer Insituttion for Applied information Technology at FITDomain Ontology+ Usage patterns of prior users+ Identifying feature of this user a search term, academic status, etc= Recommended resources

Another example of a recommendation engine

Semantic AnalysisOpen University, UKLook into the content of posts to determine what style of communication it is.Challenges eg But if, have to respond, my viewCritiques eg However, Im not sure, maybeDiscussion of resources eg Have you read, more linksEvaluations eg Good example, good pointExplanations eg Means that, our goalsExplicit reasoning eg Next step, relates to, thats whyJustifications eg I mean, we learned, we observedOthers perspectives eg Agree, here is another, take your point

Ultimate GoalModeling/Predicting successStaging the most effective interventionsImproving instructor abilitiesImproving students self awarenessCustomized learningLearning StylesCognitive Load The hierarchy of student success through Action AnalyticsRaising Awareness (Analytics IQ)Data, Information, and Analytics Tools and ApplicationsEmbedded Analytics in student success processesCulture of performance measurement and improvementOptimized student success

DangersAnalytics for learners rather than of learners - Dragan Gasevic, Athabascau U.Trapping students into limiting models of good behavior.Disrupting and Transformative Innovation Institutions resist change

****We can also use the quiz grades from within Analytics to perform our own analysis, KR20, Crombachs Alpha, etc*Graph of Tool use with Time*Talk about the types of data we can draw from Desire2Learn Analytics.Show the tools chart from the POS study*Full Sail has their own LMS custom built. The have all data, but looking to visualize it. 13k studentsAnalytics types by Linda BaeaSummarize here instead*http://ariadne.cs.kuleuven.be/monitorwidget-lak11/

The easiest kind of learning analytic to understand is visualizing the data, such as this monitor widget letting you clearly see your own time in course.*SNAPP allows visualization of discussion interactions; how students are interaction with each other in one easy to read visual chart.Talk about Social Network AnalysisSNA has been demonstrated to assist educators in indentifying instances of: Learner isolation (Mcdonald, Stuckey,,, Noakes, & Nyrop, 2005)Stands for Social Networks Adapting Pedagogical Practice (SNAPP)*Show the picture of *Dashboards can be institutional in scope. Minnesota State College and Universities put together an Accountability dashboard so the public can see, at a glance, how well they are meeting their goals. Note particularly the simple visualization of data and the simplicity of the controls. Another important point you cant see here is the good response speed when settings are changed. Maybe an entire college was overrun by Martians. http://www.mnscu.edu/board/accountability/index.html

*Dashboards that show the current status of a student in a convenient format are nice, but not as far as we can go. Predictive modeling takes it to the next level.In predictive modeling, you create a model of what affects a students success. Then take measurements of (metrics) of the inputs to the model, and have it attempt to predict what the outcome will be, based on your assumptions.Models can be prebuilt and inflexible, or entirely dynamic, constantly adjusting themselves to better match the results they have seen in the past. *Signals

Dashboards provide a high-level view of student behavior, ideally with drill down capability. An instructor can see at a glance if any of their students are falling behind, or have stopped attending entirely, helping them prioritize their time and intervene more effectively.

*http://www.itap.purdue.edu/tlt/signals/signals_final/index.htm

As odd as it may first sound, Students can benefit from a comprehensive view of themselves. In the New Horizons report, it was noted that students are not always aware that they are failing, or in what fashion they are failing. A simple dashboard that shows their overall progress, with drill down to what metrics they are failing, can help them stay on top of things and understand better what to improve.

Of course, if you rely on the metrics for grading purposes, students will learn how to game the system. *The author, Phil Ice, started by running linear regression to determine which factors indentified students most able to be helped by Advising staff. The for-profit university benefited from a greater return on advising dollars. The system advanced dramatically from there. http://www.apus.edu/*Include the learning object repository sorter. OER need to research this and add to it.Usage contexts for object similarity: Exploratory investigations at https://tekri.athabascau.ca/analytics/papersFruanhofer Insituttion for Applied information Technology at FITTalk about what a domain ontology is. *Rebecca Ferguson and Simon Buckingham ShumAt The Open University wrote Learning Analytics To Identify Exploratory Dialogue within Synchronous Text Chathttps://tekri.athabascau.ca/analytics/papersUsed to categorize live chat about a conference and separate social content from exploratory dialog.Could also, in theory, be used to categorize discussion posts and assist an instructor in identifying students who are not participating in exploratory dialog.

*Ralph Smith is two weeks into his course. Can we say with an accuracy, given all we know, whether he is going to pass or fail?

I support personalized systems only within the pizza box, not personalized to me (the student)*