streamlined orchestration: investigating the impact of organised orchestration on teaching
TRANSCRIPT
Streamlined OrchestrationInvestigating the Impact of Organised Orchestration on Teaching
Lighton Phiri
Supervisors:Prof. Dr. Christoph Meinel and A/Prof. Hussein Suleman
August 21, 2015
Introduction
■ Exploring technology-driven orchestration◻ Understand whether and/or how organisation of learning activities
influences educators’ effectiveness■ Thesis goals
◻ Investigate the impact of an orchestration workbench on teaching◻ Investigate successful use of an orchestration workbench
2
Motivation
Supporting educators considered one the of most effective ways of improving the quality of education
However, contemporary orchestration is:1) Challenging
◻ Heterogenous activities—instructional and non-instructional◻ Timing constraints—typical lecture session lasts ~45 minutes
2) Ad hoc◻ Specialised tools used for managing activities◻ Generic tools appropriated to achieve desired goal
3
Inspiration
■ Inspiration drawn from success of organisation in other domains◻ Developer IDEs for tool and service integration◻ Scientific workbenches for managing workflows
4
Streamlining Orchestration (1)
Orchestration workbench concept■ Organisation of session activities■ Facilitate sequencing of activities
5
■ Single unified view
■ Multi-tool and service intergration
Streamlining Orchestration (2)
Thesis statement:Streamlined orchestration—attainable through explicit organisation of enactment activities using an orchestration workbench—could potentially make educators more effective.
Research questions:1) Does an orchestration workbench enable educators to
become more effective?2) What is the impact of an orchestration workbench on
educators’ teaching experience?
6
Evaluation Plan (1)
‘5+3’ Conceptual Orchestration Framework■ Management
◻ Coordination of learning activities
◻ Regulation of activities to guide learning activities
■ Flexibility◻ Flexibility of tool
■ Assessment◻ Learning experience
7
Evaluation Plan (2)
8
# Evaluation Technique
1. Case studies
2. Experiment in the large
3. Targeted group
4. Survey of existing tools
5. Captive audience
6. General random trial
7. Survey of teaching staff
8. UCT Lecture recording analysis
Expert Reviews
9
Expert Reviews
■ Expert review sessions held with potential EdTech early adopters◻ Individuals integrating technology
in the classroom◻ Understand potential study
environment; recruit participants■ Eight teaching staff interviewed■ Main outcomes
◻ Scale of technology integration◻ Varying learning models
10
Department #
Architecture & Planning 2
CILT 1
Chemical Engineering 1
Computer Science 3
Mechanical Engineering 1
8
Flipped Classroom Experiment
11
Orchestrating a Flipped Classroom (1)
■ Computer Architecture second year course◻ 175 registered students◻ Three core-activities conducted during lecture sessions
■ Study aimed at assessing feasibility and potential◻ Extent of prototype tool use◻ Tool effect on learners’ learning experience
12
Activity Orchestration Orchestration Level
Demonstration Live demonstration Class
Discussion Interactive discussions Class/Individual
Timed quiz Timed quiz session Individual
Orchestrating a Flipped Classroom (2)
■ Prototype orchestration tool◻ Standalone Web-based tool◻ Orchestration of three core activities
■ Prototype utilised between September–October 2014◻ Across a total of 12 lecture sessions
■ Review sessions held with lecturer to determine changes■ Direct observations conducted during lecture sessions■ Video analysis on lecture recordings and screencasts■ Learner survey on last day of class
13
Orchestrating a Flipped Classroom (3)
■ Nine orchestration tools used during duration of course ◻ Most tools used to render content◻ Specialised tools once-off tasks
■ On average prototype orchestration tool used most of the time◻ Used 66.72% on average◻ Context switching occurred an
average of two times with noticeable time during switchover
14
Tool Freq. Duration
Prototype 10 00:30:31
VideoGlide 8 00:07:56
Firefox 1 00:21:29
Impress 1 00:38:26
Evince 1 00:00:50
QtSpim 1 00:12:47
Robotic Arm 1 00:01:24
TextEditor 1 00:02:07
VirtualBox 1 00:00:58
Orchestrating a Flipped Classroom (4)
■ Tool helped organise activities
■ Static sequencing (activity listing) found helpful
■ Timer useful to Avg performers
15
■ ”I did not really notice the tool”
■ ”I found the classroom experience fun”
Orchestrating a Flipped Classroom (5)
Feasibility of organised orchestration■ Facilitated neutral flow of activities■ Results from learner survey indicate tool was impact
neutral
Potential to facilitate improved learning outcomes■ Learner survey suggests orchestration workbench has
the potential to positively impact learning experience
16
Future Directions
17
Further explore the correletion between the use of an orchestration workbench and positive teaching and learning experience
■ Gather additional evidence to support core claim■ Evaluate approach in authentic educational settings
Year #2+ Plan
18
Planned Activities
19
Milestone Milestone description Timescale
1. Experiment #2 M1021 case study September 2015
2. Experiment #3 Controlled experiment October 2015
3. Experiment #4 Large-scale experiment January 2016
4. Experiment #5 CSC1010H case study April 2016
5. Experiment N : TBD
6. : : :
7. Thesis draft Draft manuscript February 2017
8. Thesis manuscript Final manuscript April 2017
Bibliography
[1] Jeremy Roschelle et al. “Classroom Orchestration: Synthesis”. Computers & Education, 6a9:523–526, 2013.
[2] Mike Sharples. “Shared orchestration within and beyond the classroom”. Computers & Education, 69:504–506, 2013.
[3] Pierre Dillenbourg. “Design for Classroom Orchestration”. Computers & Education, 69:485–492, 2013.
[4] Pierre Dillenbourg and Patrick Jermann. “Technology for classroom orchestration”. New Science of Learning. 525–552, 2011.
[5] Luis P. Prieto et al. “Orchestrating technology enhanced learning: a literature review and a conceptual framework”. International Journal of Technology Enhanced Learning, 3(6):583, 2011.
[6] Juan A. Munoz-Cristobal et al. “Supporting Teacher Orchestration in Ubiquitous Learning Environments: A Study in Primary Education”. TLT, 83–97, 2014.
20