fdp301x mentoring educators in educational technology€¦ · evaluation report for fdp design page...

Evaluation Report for FDP Design

FDP301x

Mentoring Educators in Educational

Technology

EVALUATION REPORT FOR

FDP DESIGN ACTIVITY on

“Active Learning: Think-Pair-Share and Peer

Instruction”

Sunita Milind Dol: [email protected]

Neha Sahu: [email protected]

Mercy Shalinie S: [email protected]

Vaibhav Singh: [email protected]

This work is licensed under the Creative Commons Attribution 4.0 International

License. To view a copy of this license, visit

http://creativecommons.org/licenses/by/4.0/.

mailto:[email protected]




http://creativecommons.org/licenses/by/4.0/


Table of Content

Content Page No.

ANALYSIS OF ENGAGEMENT 3

A) What was the gross engagement in the course? [Macro level] 3

B) What was the engagement pattern of individual LeDs, LbDs, LxIs, and LxTs?

[Micro Level]

5

C) What is the transition pattern of engagement of learners across LeD, LbD, LxI, and

LxT?

6

ANALYSIS OF LEARNING 7

A) What is the gross learning in the course?[Macro] 7

B) What is the learning pattern across LbDs and Quizzes? [Micro] 7

C) What is the transition pattern of learning across LbDs and Quizzes? [Meso Level] 8

ANALYSIS OF PERCEPTION 11

What was the perception of learners about the course? [Macro] 11

REFLECTIONS ON ANALYSIS 13

What do gross data about learning, perception and engagement tell about the course? 13

What do the analysis tell about the course experience? 13

What needs improvement in the current course offering? 13

WORKLOAD DISTRIBUTION 14

MENTORS INTERACTION DETAILS 16


ANALYSIS OF ENGAGEMENT

A) What was the gross engagement in the course? [Macro level]

The MOODLE course on Active Learning: Think-Pair-Share and Peer Instruction Activity was

originally designed as part of FDP 301x Mentoring Educators in Educational Technology. There

were around 12 particiapnts assigned by IIT, Bombay who were the participants of this FDP 301x.

These 12 particiapants assessed the course during 20th

June to 27th

June 2018 . The course involved the

following activities that were to be completed by participants during 20th

June to 27th

June 2018:

Topic: Active Learning: Think-Pair-Share and Peer Instruction Activity

Welcome Note

Course Entry Survey

Introduction to Active Learning: Think-Pair-Share and Peer Instruction Activity

Learning Objectives of Active Learning: Think-Pair-Share and Peer Instruction Activity

Key Concepts

Active Learning: Think-Pair-Share and Peer Instruction Activity

o LeD 1- Active Learning

o LbD 1 -Active Learning

o LeD 2 – Think-Pair-Share (TPS) Activity – What

o LbD 2 - Think-Pair-Share (TPS) Activity – What

o LeD 3 – Think-Pair-Share (TPS) Activity – How

o LbD 3 - Think-Pair-Share (TPS) Activity – How

o LeD 4 – Peer Instruction (PI) Activity – What

o LbD 4 - Peer Instruction (PI) Activity – What

o LeD 5 – Peer Instruction (PI) Activity – How

o LbD 5 - Peer Instruction (PI) Activity – How

o Knowledge Quiz

Discussion and Reflection Quiz

o Discussion Forum on TPS Activity

o Reflection Quiz on TPS Activity

o Discussion Forum on PI Activity

o Reflection Quiz on PI Activity

Advancing Your Learning

o LxT Video 1: Think-Pair-Share for Active Learning

o LxT Video 2: Think-Pair-Share Activity Example

o LxT Video 3: Peer Instruction Activity

o LxT Video 4: Peer Instruction Activity Example

o Assimilation Quiz on on LxT Video 1 and LxT Video 2


Designing your own TPS, PI Activity and Assessment

o Designing your own TPS Activity

o TPS Activity Constructor

o TPS Example

https://fdp301x-team120.moodlecloud.com/mod/page/view.php?id=17





o Resource Creation Assignment on TPS

o Designing your own PI Activity

o PI Example

o Resource Creation Assignment on PI

Evaluation and Feedback

Acknowledgement and References

About Team-120

The activity completion report generated from the course Reports section shows the engagement of

participants in the activities as shown in Table 1.

Table 1: Activity Completion Summary

Activity Completion Status

Course Entry

Survey

11

LeDs 1 11

2 11

3 11

4 11

5 11

LbDs 1 11

2 11

3 11

4 11

5 11

KQ 12

Discussion on TPS

activity

11

Discussion on PI

activity

11

RQ1 10

RQ2 10

LxT1 11

LxT2 10

LxT3 10

LxT4 11

AQ1 10

AQ2 10

Design Your Own

TPS

2

Design Your Own PI 3

Peer Review of TPS 2

Peer Review of PI 2


B) What was the engagement pattern of individual LeDs, LbDs, LxIs, and LxTs? [Micro Level]

Looking into the performance and engagement of individuals in the course activities, following is the

engagement pattern of selected participants in the course. Twelve participants were selected for the

analysis so that we have a sample groups showing high engagement, medium engagement and low

engagement pattern.

Table 2 shows the level of engagement of the twelve sampled participants in the course.

Table 2: Participation of individual learners in the course

Activity P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12

Course

Entry

Survey

Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes

LeDs 1 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes

2 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes




LbDs 1 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes





KQ Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

Discussion

on TPS

activity


Discussion

on TPS

activity


RQ1 Yes No Yes Yes Yes Yes Yes Yes No Yes Yes Yes

RQ2 Yes No Yes Yes Yes Yes Yes Yes No Yes Yes Yes

LxT1 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes

LxT2 Yes No Yes Yes Yes Yes Yes Yes No Yes Yes Yes

LxT3 Yes No Yes Yes Yes Yes Yes Yes No Yes Yes Yes

LxT4 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes

AQ1 Yes No Yes Yes Yes Yes Yes Yes No Yes Yes Yes

AQ2 Yes No Yes Yes Yes Yes Yes Yes No Yes Yes Yes

Design

Your Own

TPS

Yes Yes No No Yes Yes Yes No Yes Yes No Yes

Design

Your Own

PI

Yes Yes No No Yes Yes Yes No No Yes No Yes


C) What is the transition pattern of engagement of learners across LeD, LbD, LxI, and LxT?


ANALYSIS OF LEARNING

A) What is the gross learning in the course?[Macro]

Analysis of the learning of the students is done by analysing the marks obtained by the participants in

attempting various quizzes and activities. There were five types of quizzes and activities for the

participants.

The average marks for each of them is given below.

Learning by Doing (LbD) Activities- 5 Nos [10 marks]

Reflection Quiz- 2 Nos [20 marks]

Assimilation Quiz- 2 Nos [20 marks]

Knowledge Quiz [20 marks]

Think-Pair-Share and Peer Instruction Design Activity [30 marks]

Total 100 marks

Table 3: Number of participants who attempted each activity and the average marks and percentage

they scored.

LbDs KQ RQ AQ Activity

(TPS)

Activity

(PI)

Any All

No of

Persons

attempted

11 12 10 10 2 3 10

2

(including

RCA on

TPS and

PI)

Average

marks

6.67 12.33 9.79 7.17 20 10 48.33 70.5

Maximum

marks

10 20 20 20 20 10 100 100

Average

percentage

48.33 70.5

B) What is the learning pattern across LbDs and Quizzes? [Micro]

Table 4: Number of participants who have attempted each LbDs. The total is calculated in 10 as it was

the average given to the LbDs in the whole course scoring.

LbDs LbD1 LbD2 LbD3 LbD4 LbD5 Any of the

LbDs

All of the

LbDs

No of

Persons

attempted

11 11 11 11 11 0 11

Average

marks

1.25 1.42 1.08 1.58 1.33 0 72.7

Maximum

marks

2 2 2 2 2 0 2 (Each

LbD having

2 marks)

Average

percentage

62.5 71 54 79 66.5 0 66.6


Table 5: Number of participants who submitted the Think-Pair-Share (TPS) and Peer Instruction

design (PI) activity, peer reviewed these activities and the marks they have been given. The total is

calculated in 30 as it was the average marks given to the Design activity in the whole course scoring.

PI Design

Activity

Submission

Peer Review

of PI Design

Activity

TPS

Design

Activity

Submission

Peer

Review of

TPS Design

Activity

Total Marks

No of persons

attempted

6 4 6 14 30

Average

Marks

6 4 6 14 30

Maximum

Marks

6 4 6 14 14

Average

Percentage

100 100 100 100 100

C) What is the transition pattern of learning across LbDs and Quizzes? [Meso Level - It would

be good to use iSAT here]


ANALYSIS OF PERCEPTION

What was the perception of learners about the course? [Macro]

We can analyse the overall perception of learners through their feedback in the three surveys[5]

:

Perception of Learning

Perception of Engagement

Perception of Usability

Perception of Learning

For each of the parameters to analyse perception of learning, we can see that the participants seem to

have a good perception about the learning offered by the course.

Talking about the learning avenue offered by additional resources/reference materials, most of the

learners are strongly agree about their learning potential.

Fig 1. Perception of Learning

https://docs.google.com/document/d/e/2PACX-1vQHE33Udsx4pBpc0opRIFuTNFe3_ZBi1nqEfoixkAzb6fFXLdwc1h2wL4tnufNP20RbvtxuypkOmr5T/pub?embedded=true#ftnt5


Perception of Engagment

It is clearly evident that the perception of engagement has been high.

Observing each criteria, we can find that the participants are easily able to get actively involved each

time they access the course and find the additional resources/reference materials engaging enough.

Fig 2. Perception of Engagement

Perception of Usability

Here’s a visual summary of the survey responses of participants.

Fig 3. Perception of Usability

From the graph, we can analyse each of the 9 categories used to evaluate the usability and further drill

down into the response of each participant.

Going by the bar graphs, we can see that most of the participants have agreed in the range of Mildly

and Strongly on the usability of the course.


REFLECTIONS ON ANALYSIS

What do gross data about learning, perception and engagement tell about the course?

The gross data about engagement shows that the number of participants completing the

activities/quizzes are good.

Also, from the data on scores of participants, we can see that the scores of the participants are

gradually fluctuating, although the number of participants attempting the final Knowledge Quiz (KQ)

is also good. This indicates that the perceived learning value of the course is good.

From the participants' perception of the course, it can be observed that the pace of the course and

consistency of course material design is good.

What do the analysis tell about the course experience?

The analysis shows that the course was engaging enough and the levels of usability was also good, and

the levels of learning were not bad.

What needs improvement in the current course offering?

The following sections need considerable improvement:

Engagement:

Engagement in Designing your own activity

Usability

Duration of Resource creation activity

Learning

Provide the additional example related to Designing your own activity


WORKLOAD DISTRIBUTION

The team consisted of 4 participants. The workload is distributed among them as follows:

Member 1: (Team leader – Mrs. Sunita M Dol)

Preparing the workload distribution

Overall Structure of FDP Design course


o LeD 2 – Think-Pair-Share (TPS) Activity – What

o LbD 2 - Think-Pair-Share (TPS) Activity – What

o LeD 3 – Think-Pair-Share (TPS) Activity – How

o LbD 3 - Think-Pair-Share (TPS) Activity – How

o Knowledge quiz

Designing your own TPS, PI Activity and Assessment

o Designing your own TPS Activity

o TPS Activity Constructor

o TPS Example

o Resource Creation Assignment on TPS

o Designing your own PI Activity

o PI Example

o Resource Creation Assignment on PI

Progress Bar

Grading Policies

FDP design analysis report: analysis of course engagement

Member 2 (Neha Sahu)


o LeD 4 – Peer Instruction (PI) Activity – What

o LbD 4 - Peer Instruction (PI) Activity – What

o LeD 5 – Peer Instruction (PI) Activity – How

o LbD 5 - Peer Instruction (PI) Activity – How

o Knowledge Quiz


o Discussion Forum on TPS Activity

o Reflection Quiz on TPS Activity

About Team-120

Progress Bar

Grading Policies

FDP design analysis report: analysis of learning

Member 3 (Mercy Shalinie S.)

Introduction to Active Learning: Think-Pair-Share and Peer Instruction Activity


Learning Objectives of Active Learning: Think-Pair-Share and Peer Instruction

Activity

Key Concepts


o Discussion Forum on PI Activity

o Reflection Quiz on PI Activity

Evaluation and Feedback


Progress Bar

Grading Policies

FDP design analysis report: analysis of learner feedback

Member 4 (Vaibhav Singh)


o LeD 1- Active Learning

o LbD 1 -Active Learning

Advancing Your Learning

o LxT Video 1: Think-Pair-Share for Active Learning

o LxT Video 2: Think-Pair-Share Activity Example

o LxT Video 3: Peer Instruction Activity

o LxT Video 4: Peer Instruction Activity Example




Progress Bar

Grading Policies

FDP design analysis report: Self-Reflection based on analysis of the first three points

(what have the team members learnt from the data collected above and your experience)

‘


MENTORS GROUP INTERACTION DETAILS

Date: 1th

June 2018


Date: 2

nd June 2018


Date: 4th

June 2018


Date: 5th

June 2018


Date: 6th

June 2018


Date: 7th

June 2018


Date: 9th

June 2018


Date: 10th

June 2018


Date: 11th

June 2018


Date: 12th

June 2018


Date: 13th

June 2018


Date: 14th

June 2018


Date: 15th

June 2018

fdp301x mentoring educators in educational technology€¦ · evaluation report for fdp design page...

Documents