the effect of using mobile classroom response system on students’ engagement and performance

The Effects of Using Mobile

Classroom Response System

(MCRS) on Students’

Engagement and

Performance

Khaled Hamdan, PhD Instructor Kwantlen Polytechnic University

Outline

Background: Classroom Response System and Mobile


Goals & Objectives

Methods

Results and Discussion


CRS a set of hardware

and software that

facilitates teaching

activities.

A classroom response system

(CRS)= a personal response

system, a student response system,

an audience response system, or

simpley a clicker.


Instructor poses a

multiple-choice question

to students.

Each student submits an

answer to the question

using a handheld

transmitter.

“on the fly” response bar

chart

htt

ps:

//w

ww

.um

b.e

du

Background literature

CRSs are most effective in large classes as a stimulus to

greater student involvement and interaction.

CRSs promote learning in all kinds of classes: small

and large, graduate and undergraduate, lecture and lab.

Milner-Bolotin, Antimirova, & Petrov (2010)

Sevian & Robinson (2011).

Some studies suggest that use of a classroom response

system improves learning outcomes.

Camey, Gray, & Gray 2008

Mobile Device Penetration in

Canada

The

Canadia

n R

adio

-tele

visi

on a

nd T

ele

com

munic

ations

Com

mis

sion, 2015.

Mobile Device Penetration in BC

by Age

htt

p:/

/ww

w.6

smark

eting.c

om

/blo

g/c

anadia

n-s

mart

phone-u

se-s

tatist

ics/

Mobile Classroom Response

System (MCRS)

MCRS=

student’s

mobile device

is the

classroom

response

system.

htt

p:/

/blo

g.c

engage.c

om

/tech

nolo

gy-

class

room

-dis

tract

ion-a

sset/

Outline



Goals & Objectives

Methods


The Issue

Previous research is in agreement that CRS

enhances the engagement of students.

Little research has addressed the effects of MCRS

on student engagement and performance.

Contradictory findings have been reported on the

influence of a CRS on student performance.

Goals and Objectives

to understand students’ perceptions on using

MCRS in classroom instruction.

to provide insights on the effects of MCRS usage on

students’ engagement and performance.

Explore the associations between gender/age and

perceptions on the effect of MCRS on class

engagement.

Outline



Goals & Objectives

Methods


Methods

Two class sections

Sections 1: control section, no use of a MCRS

(GEOG 1120 Earth Science Fall 2015 term). N=23.

Section 2: MCRS used throughout the entire

semester (GEOG 1120 Earth Science Winter 2016

term). N=22

Gender Distribution (%) in each of the

Study Class Sections

10

15

20

25

30

35

40

45

50

55

60

Control MCRS

Male Female

50% 50%

54%

46%

Age Distribution (%) in each of the

Study Class Sections

0

10

20

30

40

50

60

Under 20 yr-old 20-25 yr-old 25-30 yr-old

Control MCRS

5% 5%

50% 50%

45% 45%

Methods

Both sections were taught by the same instructor.

Same materials (lecture notes, assignments, and

exams) were used for both sections.

Both sections had attendance and participation

worth 5% of the course final grade.

Methods

Students (Winter 2016 term) were surveyed on

their perceptions of how using MCRS in

classroom instruction influenced their learning

experience.

Qualitative Data: class survey

Standard

75%

Qualitative Data Analysis

Association between categorical variables:

Gender and perceptions on the effect of MCRS on class

engagement.

Age and perceptions on the effect of MCRS on class

engagement.

Conditional Percent tables and Chi-square

analysis, C.I. 95%

Quantitative Data: Control and

MCRS sections

Attendance

Midterm exam scores

Final exam scores

Paired Student t-test, C.I. 95%

Outline



Goals & Objectives

Methods


Results: Attendance

0

10

20

30

40

50

60

70

80

90

100

Attendence throughout the term Attendence in the last 2 weeks

of classes

Control MCRS

70%

(P 0.047) (P 0.371)

(a) (a)

(b) (b)75%

90% 86% The use of MCRS

improved attendance

by ~15%.

Student t-test

revealed that

attendance is

significantly

improved.

Results: Attendance

0

10

20

30

40

50

60

70

80

90

100

Attendence throughout the term Attendence in the last 2 weeks

of classes

Control MCRS

The use on MCRS Improved attendance by ~15%.

(P 0.047) (P 0.037)

(a) (a)

(b) (b)

Caused students to attend the class

more regularly?

Strongly Disagree Disagree Neither Agree Agree Strongly Agree

Nor Disagree

Most students agreed or strongly agreed

(84%) that the use of MCRS has caused

them to attend the class more regularly.

Narrative feedback: Attendance

It is a good method of increasing interest in course and

attending lectures regularly.

Narrative feedback: Attendance

Maintains one’s regularity…

Results: participation (control)

Control: Despite my best effort, most classroom

activities did not result in 100% student

participation.

Results: participation (control)

Control: Despite my best effort, most classroom

activities do not result in 100% student

participation.

Typically, when I ask a question in class, only a

few students are called on to respond. Even if 25%

raise their hands, only 5% may get to share their

responses.

Results: participation (MCRS)

MCRS: ~99%

participation

With a MCRS, almost

every student responded to

every question. MCRS

clearly increased student

engagement.

htt

ps:

//w

ww

.bost

onglo

be.c

om

Made students more willing to

participate in class discussions?


Nor Disagree

Most students agreed or strongly agreed (89%)

that the use of MCRS has them more willing

to participate in class discussions.

Maybe the two students who usually answer

all questions even with no MCRS?

Narrative feedback: participation

Students get deserving participation marks.

Makes the class participation engaging!

Consequences of improved

participation rate Strongly Disagree Disagree Neither Agree Agree Strongly Agree

Nor Disagree

74%

79%

79%

92%

Made the class more engaging?


Nor Disagree


that the use of MCRS has made the class more

engaging.

Enhanced the learning experience?


Nor Disagree


that the use of MCRS has enhanced their

learning experience.

Narrative feedback: engagement

I think it was a great learning experience made class

lectures really interesting…


…It helps students to learn topics in depth and improves

their own understanding.


I think it was a cool way to keep us engaged.

…. as I listen very much carefully to professor in order to

answer those questions….

Results: Performance

0

10

20

30

40

50

60

70

80

Control MCRS Control MCRS

Midterm Final Exam

61

P 0.371P 0.55

68

(a)

(a)(a)

Students’

performance

improved in the

midterm exam.

Student t-test

revealed that the

performance

improvement is not

statistically

significant.

70

56

(a)

Results: Performance

0

10

20

30

40

50

60

70

80

Control MCRS Control MCRS

Midterm Final Exam

61

P 0.371 P 0.55

68(a)

(a)(a)

There’s a trend of

improved students’

performance in

both the midterm

and final exam.

Student t-test

revealed that the

performance

improvement is

not statistically

significant.

70

56(a)

Results: Association between categorical variables-

Gender & perceptions on class engagement

68(a)

(a)(a)

Two categorical

variables are related in

the sample if at least

two rows noticeably

differ in the pattern of

row percent.

Chi-square: P=0.037

(a)

Strongly

Agree/Agree

Neither Agree

Nor Disagree Total

Male 10 2 12

Female 7 0 7

Total 19

Strongly

Agree/Agree

Neither Agree

Nor Disagree Row

%

Male 83% 17% 100%

Female 100% 0 100%

Row percent

Data

Results: Association between categorical variables-

Age & perceptions on class engagement

68(a)

(a)(a)

Age is related to the

perceptions on class

engagement because at

the rows are noticeably

differ in the pattern of

row percent.

Chi-square: P=0.021

(a)

Strongly

Agree/Agree

Neither Agree

Nor Disagree Total

<20 yrs 10 0 10

20-25 yrs 6 2 8

>25 yrs 1 0 1

Total 19

Strongly

Agree/Agree

Neither Agree

Nor Disagree Row

%

<20 yrs 100 0 100

20-25 yrs 75 25 100

>25 yrs 0 100 100

Row percent

Conclusions

Using MCRS significantly improved attendance and participation.

There’s a trend of improved performance, however, t-test

revealed it is not statistically significant.

Class is more engaging and overall a better learning experience.

Results may differ depending on class demographics as females

(52% of BC university/college students) and students < 20 year-

old (28% of BC university/college students) seem to have strong

positive opinion about how the MCRS affected class

engagement.

htt

p:/

/ww

w.a

ved.g

ov.

bc.

ca/d

ata

ware

house

/

Limitations and Opportunities

Sample size N=22

An opportunity for Collaboration

[email protected]

mailto:[email protected]

Using Smart Devices as

Mobile Classroom

Response Systems

Khaled Hamdan, PhD Instructor Kwantlen Polytechnic University

Starts at 11 am in this room

the effect of using mobile classroom response system on students’ engagement and performance

Education