using analytics to examine course design and identify effective practices (166234657)

7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)

http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 1/31

Using Analytics to Examine CourseDesign and Identify Effective Practices

Feb. 6, 2013

Karin Readel, University of Maryland, Baltimore County



For Audience Participation:

http://rwpoll.com

Session ID = eli2013

http://rwpoll.com/

http://rwpoll.com/



Which best describes your feelings about theresults of Superbowl 2013?

1. Go Ravens! Woot!2. Total depression over the loss

3. Meh

4. Superbowl, schmooper-bowl. It’s all aboutthe Puppybowl!



About Blackboard @ UMBC

Began using Bb in SP2000 Campus Adoption

95% of all students

75% of all instructors 65% of all courses

350 Organizations



www.umbc.edu/blackboard/reports



Most Active Bb Courses



Bb Analytics for Learn

In 2012 UMBC participated in the LFT for BA4L

Implementation almost complete Interested in what the data could tell us

about the impact of our faculty training

efforts



Alternate Delivery Program (ADP)

Training Program for faculty moving fromF2F to Hybrid or Online format

Provides 1 time $2500 stipend to redesigncourse

1 day “Hybrid Workshop”

Self-reflection on pedagogical problem posed

Interview with IT and FDC staff

Create & present 2 online activities

Teach redesigned course

Evaluate ADP process



Questions:

Are there differences in course measuresbetween the various modes of instruction?

Does training make a difference?

Can we use BA4L data to identify effectivepractitioners?

What other (non-LMS) data can be included

in analysis?



rwpoll.com session ID= eli2013

Which mode of instruction would you expect tofind the highest average level of courseinteraction?

1. F2F

2. Hybrid

3. Online




Would you expect to see higher levels of course interaction with Graduate or Undergraduate students?

1. Graduate students

2. Undergraduate students



F2F Hybrid Online

Undergrad 131.6 306.9 480.6

Grad 125 245.7 532.2

0

200

400

600

800

1000

1200

A v e r a g e # o f I n t e r a c t i o n s

Average # of Interactions - All Faculty



All Trained All Trained All Trained

F2F Hybrid Online

Undergrad 131.6 149.8 306.9 468 480.6 659.6

Grad 125 132.5 245.7 250.8 532.2 401.6

0

200

400

600

800

1000

1200

A v e r a g e #

o f I n t e r a c t i o n s

Ave Interactions- All vs. ADP Trained



0

10

20

30

40

50

60

F2F Hybrid Online

Ave Course Measures by Instruction Mode

for All Faculty

Ave Content

Ave Course Accesses

Ave Min/Access



0

10

20

30

40

50

60

70

80

F2F Hybrid Online

Ave Course Measures by Instruction Mode for ADP

Faculty

Ave Content

Ave Course Accesses

Ave Min/Access



0

10

20

30

40

50

60

70

80

Ave Content Ave Course

Accesses

Ave Min/Access

Effect of Training on Average Course

Measures

F2F All

F2F Trained

Hybrid All

Hybrid Trained

Online All

Online Trained



content% content

accessedgradecenter tools assessments

All 69.6 41.3 37.9 64.3 40

Trained 81 43.3 54 77 58.3

0

10

20

30

40

50

60

70

80

90

% o

f C o u

r s e s U s i n g

Effect of Training on Course Measures



F2F Hybrid Online All Modes

All Instructors 5.9 9.7 6.5 6.3

Trained 7.6 14.4 6.1 8.4

0

2

4

6

8

10

12

14

16

Effect of Training on Folder Depth Ratio



BA4L Built in Reports



Ability to Drill Down to Course




Would you expect to see higher levels of course interaction in the LMS in courses using“clickers in class?

1. Yes

2. No



Trends For Courses Using Clickers…

No difference in average interactions,course accesses, % of items accessed vs.non-clicker courses.

“Clicker courses” were more likely to useassessments (79% vs. 40%)



What About Smart Room Use?

Smart Room = fixed data projector Lecture Rooms aren’t all centrally scheduled

79 Registrar; 55 Departmental

68 of 79 monitored by GVE

Projector use data was crossmapped with

the course schedule data to produce reportsshowing exactly when projectors were usedin specific courses



% Projector Usage for Courses in GVE

Monitored Rooms FA2011- SP2012

0 0-10 10-20 20-30 30-40 40-50 50-60 60-70 70-80 80-90 90-100

FA2011 11.5 14.4 11.1 8.8 5.9 7.7 7.4 6.7 12.0 9.4 5.1

SP2012 8.3 17.7 11.6 9.1 12.6 12.2 12.8 13.3 2.3 0.1 0.0

0.0

2.0

4.0

6.0

8.0

10.0

12.0

14.0

16.0

18.0

20.0

% C

o u r s

e s



Room Size of Courses with <10%

Projector Use

<40 40-60 60-80 80-100 100-150 >150

FA2011 26.6 47.3 10.4 10.0 2.9 2.9

SP2012 39.6 38.3 8.7 4.8 4.3 4.3

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

45.0

50.0

% o

f C o u r s e s




Do you think that hybrid courses would usedata projectors more or less than F2Fcourses?

1. More

2. Less

3. No Difference



0

5

10

15

20

25

30

35

SP2012 FA2012

A v e % P

r o j e c t o r U s e

% Projector Use - All Courses vs. Hybrid

All Courses

Hybrid



0

2

4

6

8

10

12

14

16

0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 91-100

% o

f H y b r i d C o

u r s e s

% Projector Use Hybrid Courses SU2012



Next Steps:

Look at impact of training over time onspecific faculty

Look at impact of “redesign” on specific

courses Add clicker and projector use data to iBBLA

cube



Using Analytics to Examine CourseDesign and Identify Effective Practices

Karin Readel

DoIT, UMBC

[email protected]

using analytics to examine course design and identify effective practices (166234657)

Documents