evolving a learning analytics platform
DESCRIPTION
Slides accompanying a talk by Ari Bader-Natal (twitter: @aribadernatal) at the 1st International Conference on Learning Analytics and Knowledge. More information about this work can be found on the Grockit research page: grockit.com/blog/main/researchTRANSCRIPT
![Page 1: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/1.jpg)
Ari Bader-Natal
February 28, 2011
Evolvinga Learning Analytics Platform
Thomas [email protected]
@AriBaderNatal
![Page 2: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/2.jpg)
Ari Bader-Natal
![Page 3: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/3.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
Questions to ask when evolving a platform:
![Page 4: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/4.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The first question
Questions to ask when evolving a platform:
![Page 5: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/5.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The first question
The interesting questions
Questions to ask when evolving a platform:
![Page 6: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/6.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The first question
The interesting questions
The infrastructure question
Questions to ask when evolving a platform:
![Page 7: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/7.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The first question
The interesting questions
The infrastructure question
The subset question
Questions to ask when evolving a platform:
![Page 8: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/8.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The first question
The interesting questions
The infrastructure question
The subset question
The missing data question
Questions to ask when evolving a platform:
![Page 9: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/9.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The first question
The interesting questions
The infrastructure question
The subset question
The missing data question
The pro-active question
Questions to ask when evolving a platform:
![Page 10: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/10.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The first question
The interesting questions
The infrastructure question
The subset question
The missing data question
The pro-active question
The effort question
Questions to ask when evolving a platform:
![Page 11: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/11.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The first question
The interesting questions
The infrastructure question
The subset question
The missing data question
The pro-active question
The effort question
The discovery question
Questions to ask when evolving a platform:
![Page 12: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/12.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The first question
The interesting questions
The infrastructure question
The subset question
The missing data question
The pro-active question
The effort question
The discovery question
The production questions
Questions to ask when evolving a platform:
![Page 13: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/13.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The first question
The interesting questions
The infrastructure question
The subset question
The missing data question
The pro-active question
The effort question
The discovery question
The production questions
The audience question
Questions to ask when evolving a platform:
![Page 14: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/14.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The context questionsWhat is Grockit? Where? Why? How?
![Page 15: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/15.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
![Page 16: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/16.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
E-Learning
Computer-Supported Collaborative Learning
Intelligent Tutoring Systems
![Page 17: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/17.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
![Page 18: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/18.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
System control(adaptive/IRT assessment)
Learner control(custom-specified study)
System control(adaptive study)
![Page 19: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/19.jpg)
![Page 20: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/20.jpg)
Asynchronous Coordination
Synchronous Coordination
![Page 21: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/21.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
Social and motivational value of having a cohort of peers:
• opportunities to ask for assistance (just-in-time guidance)
• opportunities for discussion (both on-task and off-task)
• opportunities to earn recognition (for assisting others)
![Page 22: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/22.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
Social and motivational value of having a cohort of peers:
• opportunities to ask for assistance (just-in-time guidance)
• opportunities for discussion (both on-task and off-task)
• opportunities to earn recognition (for assisting others)
![Page 23: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/23.jpg)
![Page 24: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/24.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
![Page 25: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/25.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
Study plans
Learning goals
![Page 26: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/26.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
![Page 27: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/27.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
Recognition of achievements
Recognition from peer learners
Indicator of helpfulness
![Page 28: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/28.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
The first questionWhat are people actually doing?
![Page 29: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/29.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
Just look around in the database.
![Page 30: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/30.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
Just look around in the database.
•select count(*) from learners;
![Page 31: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/31.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
Just look around in the database.
•select count(*) from learners;
•select * from discussions;
![Page 32: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/32.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
Just look around in the database.
•select count(*) from learners;
•select * from discussions;
•select * from responses;
![Page 33: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/33.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
Just look around in the database.
•select count(*) from learners;
•select * from discussions;
•select * from responses;
•select avg(time) from responses
![Page 34: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/34.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
The interesting questionsExploratory Data Analysis...
![Page 35: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/35.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
![Page 36: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/36.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
What is effective? What is engaging?
![Page 37: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/37.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
What is effective? What is engaging?
• How does frequency and duration of discussions during group study sessions differ between high school students and post-college learners?
![Page 38: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/38.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
What is effective? What is engaging?
• How does frequency and duration of discussions during group study sessions differ between high school students and post-college learners?
• Do students who work in groups spend more time on task than those who choose to work alone?
![Page 39: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/39.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
What is effective? What is engaging?
• How does frequency and duration of discussions during group study sessions differ between high school students and post-college learners?
• Do students who work in groups spend more time on task than those who choose to work alone?
• Which of the various interventions within Grockit lead to the largest learning gains?
![Page 40: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/40.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
What is effective? What is engaging?
• How does frequency and duration of discussions during group study sessions differ between high school students and post-college learners?
• Do students who work in groups spend more time on task than those who choose to work alone?
• Which of the various interventions within Grockit lead to the largest learning gains?
• Do peer-awarded points motivate more discussion in small group settings?
![Page 41: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/41.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
What is effective? What is engaging?
• How does frequency and duration of discussions during group study sessions differ between high school students and post-college learners?
• Do students who work in groups spend more time on task than those who choose to work alone?
• Which of the various interventions within Grockit lead to the largest learning gains?
• Do peer-awarded points motivate more discussion in small group settings?
• How does an Item Response Theory model compare to a Knowledge Tracing model, in terms of predictive accuracy or responses?
![Page 42: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/42.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
What is effective? What is engaging?
• How does frequency and duration of discussions during group study sessions differ between high school students and post-college learners?
• Do students who work in groups spend more time on task than those who choose to work alone?
• Which of the various interventions within Grockit lead to the largest learning gains?
• Do peer-awarded points motivate more discussion in small group settings?
• How does an Item Response Theory model compare to a Knowledge Tracing model, in terms of predictive accuracy or responses?
• Are video explanations more effective than written explanations?
![Page 43: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/43.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
![Page 44: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/44.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
Question:
![Page 45: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/45.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
Question:
Are the questions currently available appropriate for the current population of learners?
![Page 46: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/46.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
Question:
Are the questions currently available appropriate for the current population of learners?
• Are there enough hard questions?
![Page 47: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/47.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
Question:
Are the questions currently available appropriate for the current population of learners?
• Are there enough hard questions?
• Are there enough easy questions?
![Page 48: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/48.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
Question:
Are the questions currently available appropriate for the current population of learners?
• Are there enough hard questions?
• Are there enough easy questions?
Actionable:
![Page 49: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/49.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
Question:
Are the questions currently available appropriate for the current population of learners?
• Are there enough hard questions?
• Are there enough easy questions?
Actionable:
If there isn’t a good fit, ask the content authors to start focusing on writing what is currently most needed.
![Page 50: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/50.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The infrastructure questionHow can we streamline analysis?
![Page 51: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/51.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
![Page 52: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/52.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
collection
![Page 53: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/53.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
collection
selection
![Page 54: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/54.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
collection
selection
analysis
![Page 55: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/55.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
collection
selection
analysis
visualization
![Page 56: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/56.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
collection
selection
analysis
visualization
distribution
![Page 57: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/57.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
collection
selection
analysis
visualization
distribution
![Page 58: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/58.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
collection
selection
analysis
visualization
distribution
![Page 59: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/59.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
collection
selection
analysis
visualization
distribution
![Page 60: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/60.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
collection
selection
analysis
visualization
distribution
![Page 61: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/61.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
![Page 62: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/62.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The subset questionHow can we ask the same question of different subsets of the available data?
![Page 63: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/63.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
Example views
![Page 64: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/64.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
Multiple views at once.
![Page 65: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/65.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
Multiple views at once.
Subject (e.g. Algebra)
Question
Game
Question Session
User (e.g. Ari)
Game Session
Response Chat Message
![Page 66: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/66.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
Multiple views at once.
Subject (e.g. Algebra)
Question
Game
Question Session
User (e.g. Ari)
Game Session
Response Chat Message
![Page 67: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/67.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
![Page 68: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/68.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
![Page 69: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/69.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The missing data questionWhat about actions that leave no trace?
![Page 70: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/70.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
![Page 71: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/71.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The pro-active questionHow can we test out new hypotheses?
The effort questionHow can we reduce friction in
experimentation and hypothesis testing?
![Page 72: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/72.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
http://xkcd.com/552/
![Page 73: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/73.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
R. Kohavi, R. Henne, and D. Sommerfield. Practical guide to controlled experiments on the web. In Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2007.
http://bit.ly/exp-long
http://bit.ly/exp-short
Eric Ries: The one line split-test, or how to A/B test all the time.
http://bit.ly/one-line-split
![Page 74: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/74.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
![Page 75: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/75.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
InitialQuestionsShould new students be given easier questions when playing alone,
until they answer at least 5 questions correctly?This test is being assigned and evaluated for new users only.
EasyFirst NormalSubsequent activity (1657 persons) (1721 persons) (difference)
Participated in further study 73.2% (1213) 67.5% (1161) (2.6% to 8.9%) **Participated in group study 31.4% (520) 27.0% (464) (1.3% to 7.5%) *
Logged in again later 30.8% (510) 28.4% (489)Participated in group discussion 22.1% (366) 18.4% (316) (1% to 6.5%) *
Reviewed past questions 13.9% (230) 12.8% (221)
![Page 76: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/76.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The discovery questionHow can we deliver findings effectively?
![Page 77: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/77.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
![Page 78: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/78.jpg)
![Page 79: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/79.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
![Page 80: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/80.jpg)
![Page 81: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/81.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The production questionsScalability? Performance? Reliability?
![Page 82: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/82.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
![Page 83: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/83.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
The audience questionHow can this help teachers and learners?
![Page 84: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/84.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
![Page 85: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/85.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System Ari Bader-Natal
http://grockit.com
Performance (per-skill)
![Page 86: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/86.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
My unresolved questionIs evolving the right approach?
![Page 87: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/87.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
YAGNI: “You Ain’t Gonna Need It”
Ron Jeffries: “Always implement things when you actually need them, never when you just foresee that you need them.”
![Page 88: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/88.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
In the age of #BigData, how do you choose tools for analysis before your dataset is undeniably Big? Start with Big tools or YAGNI?
YAGNI: “You Ain’t Gonna Need It”
Ron Jeffries: “Always implement things when you actually need them, never when you just foresee that you need them.”
![Page 89: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/89.jpg)
Ari Bader-Natal
Evolving a Learning Analytics System @AriBaderNatal
http://grockit.com
Your unresolved questions???
![Page 90: Evolving a Learning Analytics Platform](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c46b514a795947708b4592/html5/thumbnails/90.jpg)
Ari Bader-Natal
Thanks!
February 28, 2011
Evolvinga Learning Analytics Platform
@AriBaderNatal
Thomas [email protected]