Streamlining Technology-drivenOrchestration in Formal Learning

SpacesLighton Phiri1 · Christoph Meinel2 · Hussein Suleman1

1Department of Computer Science, University of Cape Town, South Africa2Hasso Plattner Institute, University of Potsdam, Germany

MotivationEducators can be more effective whensupported with orchestration—themanagement of learning activities.

Orchestration is FlawedTechnology-driven orchestration ischallenging [1] and, arguably, ad hoc.It is multi-faceted and not standardised.

Proposed Solution

Streamline orchestration by facilitatingorganisation of learning activities.Organisation could lead to effectiveness.

Streamlined Orchestration WorkflowStreamlining orchestration takes the form of a softwareorchestration workbench which facilitates the integration andcentralised access to tools and services required to orchestratelearning activities.

Orchestration workflow procedure involves four steps.

Step 1: Activities

– Define activitieshierarchical structures formodules and sessions.

Step 2: Resources

– Organised learningresources (e.g. PDFs,videos and URLs).

Step 3: Sequencing

– Create sequence chain todefine orchestration orderof activities.

Step 4: Orchestration

– Playback saved sequencechain during learningsession.

Pre-session management is performed in steps 1–3 andsession management is performed in step 4.

Study 1: Orchestrating a Flipped Class

Flipped class case study conducted using prototpye Workbenchto assess evaluate approach in authentic educational setting [4].

Organisation

Timer Feature

Activity Listing

100 50 0 50 100Percentage

Strongly Disagree Disagree Neutral Agree Strongly Agree

Figure 1. Learners’ Experience Survey Results

Noticeable neutral flow of activities and potential positive effecton learners’ learning experience.

Study 2: Peer-led Guided Orchestration

Controlled study involving 24 Tutors, for first year ComputerScience courses, aimed at measuring orchestration load.

Frustration

Effort

Performance

Temporal Demand

Mental Demand

Physical Demand

0 20 40 60 80 100

Mean Raw Rating

NA

SA

-TL

XS

ubsc

ales

Figure 2. Sequencing Workload Ratings

NASA-TLX used to mea-sure orchestration load.

– High values imply highworkload.

– Width of bars reflectsimportance of weight.

– Height of barsrepresents magnitude ofrating.

Results suggest minimal workload requirement during scripting.

Study 3: Ad Hoc vs. Organised Orchestration

Controlled study conducted with 61 educators in order tocompare ad hoc orchestration and organised orchestration [3].

– Workbench prototype UI simulated organised orchestration.

– PortableApps simulated ad hoc orchestration.

SuperfluousSuperfluous

TooTooSelf-Self-

OrientedOrientedSelf-Self-

OrientedOriented

TooTooTask-Task-

OrientedOriented

NeutralNeutral Task-Task-OrientedOriented

DesiredDesired

Pragmatic Quality (PQ)

Hed

onic

Qua

lity

(HQ

)

PortableApps Workbench

Figure 3. Portfolio Presentation

6.74

5.57

PortableApps

Workbench

0 2 4 6 8 10

Mean Task Time (Min)

Tec

hniq

ue

Figure 4. Time on Tasks

-3

-1

1

3

PQ HQ-I HQ-S ATT

Dimensions

Mea

nS

core

PortableApps Workbench

Figure 5. Dimension Means

AttrakDiff 2 used an measure-ment instrument.Organised orchestration is:

– Desirable and 21% faster.

– Correlated with improveduser experience.

Organised orchestration more effective than ad hoc.

Study 4: Reusable Orchestration OERs

Practical usage scenario assessed through implementation ofreusable virtual orchestration appliances (rVOA) [2].

– rVOA player: Flexible offline UI workflow component.

– rVOA repository: Scalable online repository platform.

Feasibility of integrating orchestration packages with OERs.

Acknowledgements

This work is funded, in part, by the Hasso Plattner Institute.

Bibliography

[1] Dillenbourg, P. Design for Classroom Orchestration.Computers & Education. 69:485–492, November 2013.

[2] Parker, N. and Valentyn, M. Reusable Virtual OrchestrationAppliances. Retrieved from https://goo.gl/qNOF7V

[3] Phiri, L., Meinel, C., and Suleman, H. Ad hoc vs. OrganisedOrchestration. In 8th IEEE International Conference onTechnology for Education. December 2016.

[4] Phiri, L., Meinel, C., and Suleman, H. StreamlinedOrchestration. Computers & Education. 95:231–238, April2016.