stinkin' badges: why we need 'em and how to use 'em

49
Stinking Badges: Why We Need Em’ and How to Use Em’ Rudy McDaniel, Joseph Fanfarelli, and Kelvin Thompson Information Fluency Conference University of Central Florida February 27, 2014

Upload: kelvin-thompson

Post on 09-May-2015

1.546 views

Category:

Education


0 download

DESCRIPTION

Listen to session audio while manually viewing slides at: http://ofcoursesonline.com/?p=408. Presentation w/ Rudy McDaniel and Joseph Fanfarelli at 2014 Information Fluency Conference.

TRANSCRIPT

Page 1: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Stinking Badges: Why We Need Em’ and How to Use Em’

Rudy McDaniel, Joseph Fanfarelli, and Kelvin ThompsonInformation Fluency Conference

University of Central FloridaFebruary 27, 2014

Page 2: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Outline• Overview • Four Quick Case Studies of Badging Projects• Toward a Badge Design Taxonomy• Discussion / Q&A

• Note: these slides can be downloaded from: http://goo.gl/ezz3DV or http://tinyurl.com/badgesrock

Page 3: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Achievements and Badges• Achievements, or earned tokens of accomplishment,

often encourage players to spend more time within digital systems (esp. videogames) and to alter their playing habits in order to unlock particular types of challenges (e.g., find every coin in a given area or unlock a particular puzzle within a certain amount of time).

• Badges, or visible markers of achievement, have now made the transition from entertainment media to other forms of scholarship and pedagogy, particularly in online learning environments (Jindal, 2011; Bruckman, 2004; Lindgren & McDaniel, 2011; Lindgren, McDaniel, & Friskics, 2011).

Page 4: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Badge graphics courtesy of Matthew Dunn

Page 5: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Badges Are Not New

Video Games

Military Girl Scouts

Page 6: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

How Do Badges Work?• Badge = Task-reward system.

– Task - Can present a task to complete.

– Reward – Can also serve as a reward for completing the task.

• Rewards for completing goals can be

– Internal to the system (e.g. Points).– External to the system (e.g. Free or

discounted “stuff”).– The badge, itself.

Page 7: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Badges Are Gaining Widespread Attention• A few familiar names that are actively taking part in

badging:

urdue

Page 8: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

What Makes Them So Interesting?• Badges can serve as:

– Goal setters – Motivators – Inspiration to Explore – Creativity Boosters – Progress Trackers

• Connect Badge Criteria to Course Objectives

• Expected vs. Unexpected Badges– Foster different goals. – Expected may help in achieving a specific purpose, while

Unexpected may hurt the purpose (and vice versa).

Page 9: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Implications for Education• Motivate students to do their best work.

– Or additional work.• Help students set goals for clearer routes to

success.• More precisely and creatively track progress, in

comparison to the final grade in a course.• Encourage students to implement creative

thinking to discover unexpected achievements.• We will now discuss some of our

implementations.

Page 10: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Dumb(?) Things I’ve Done with Badges• Badges seen only by recipient• Badges not easily shareable• Badges as “back-handed compliments”• Badges for required activities

Page 11: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

CASE STUDY #1: BLENDKIT2012

Page 12: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

BlendKit2012

Subject Blended learning

Type Professional Development

Level

Size 1230 enrolled

Badge Source Graphic designer

Badge Platform Developer + Mozilla Framework

Focus Competencies

Grades/Badges Badges only

List/Easter Eggs Badge list

Viewable By Self

Status Complete

Page 13: Stinkin' Badges: Why We Need 'Em and How to Use 'Em
Page 14: Stinkin' Badges: Why We Need 'Em and How to Use 'Em
Page 15: Stinkin' Badges: Why We Need 'Em and How to Use 'Em
Page 16: Stinkin' Badges: Why We Need 'Em and How to Use 'Em
Page 17: Stinkin' Badges: Why We Need 'Em and How to Use 'Em
Page 18: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

CASE STUDY #2: EME5050

Page 19: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

EME5050

Subject Ed Tech

Type Academic

Level Grad

Size 15-25

Badge Source Purdue Passport

Badge Platform Credly

Focus 2nd level Competencies

Grades/Badges Grades + Badges

List/Easter Eggs Easter Eggs

Viewable By Class

Status 3rd Iteration Underway

Page 20: Stinkin' Badges: Why We Need 'Em and How to Use 'Em
Page 21: Stinkin' Badges: Why We Need 'Em and How to Use 'Em
Page 22: Stinkin' Badges: Why We Need 'Em and How to Use 'Em
Page 23: Stinkin' Badges: Why We Need 'Em and How to Use 'Em
Page 24: Stinkin' Badges: Why We Need 'Em and How to Use 'Em
Page 25: Stinkin' Badges: Why We Need 'Em and How to Use 'Em
Page 26: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

CASE STUDY #3: AEM

Page 27: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Course Structure• 30 different modules to

choose from at 10 different points in the semester.

• A back story involving a media mogul recruiting new students (the “dream job” scenario) is released via four different animations throughout the course.

Page 28: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Example Module Selection

Page 29: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Assessment Approach• Implementation of course technology and

curriculum with ~100 students in Fall 2010 and ~200 students in Fall 2011 was successful

• Badges were added in Fall 2011 version of the course

Page 30: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Sample Badge

Page 31: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Comparing Badges

Page 32: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Assessing the Effects of Badges

• Several components of assessment including student project analysis, focus groups, and comparisons to other courses

• Focus here is on engagement and learning surveys that asked specifically about the badges

• 206 students completed at least one survey• 127 completed both pre- and post-surveys

Page 33: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Student Surveys

To what extent did you believe that having the ability to choose which module to take throughout the course was a positive feature that helped you to learn? Extremely Mostly Somewhat Not at all Positive Positive Positive Positive

I found myself working harder on assignments/projects in order to acquire achievements. Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree

In the last 8 weeks, how often have you discussed ideas from this course outside of class?

Very Often Often Sometimes Never

• Some questions adapted from the 2010 NSSE

Page 34: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

General Attitudes

Page 35: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

General Attitudes

Page 36: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

0

1

2

3

4

5

6

7

Felt Achievement System Had Pos-itive Impact on Course

7 = Strongly Agree

Males FemalesMales Females0

1

2

3

4

5

6

7

Motivated by Other Students Receiving Achievements

7 = Strongly Agree

Males Females

Attitudes - Badges

Page 37: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Variable 1 Variable 2 Pearson Correlation (r)

Significance (p)

Felt Achievement System Was Positive (1 to 7)

Discussed Ideas Outside of Class (1 to 7) .175 .040*

Worked Harder To Receive Achievements (1 to 7)

Commented On Other Students’ Work (1 to 7) .217 .010*

Worked Harder To Receive Achievements (1 to 7)

Amount Of Time Spent Collaborating With Other Students

.242 .004**

Seeing Others Get Achievements Was Motivating (1 to 7)

Total Number Hours Spend On Course Per Week .158 .064

Attitudes - Badges

• Post-Survey: Positive feelings about badge system was correlated with other positive feelings in the course

Page 38: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Assessment Summary

• Importance of “framing” the achievement system at the outset

• Interesting gender patterns – suggests badge systems may be a productive means of targeting female learners

Page 39: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Case Study #4: Two Current UCF Courses

• 1 Semester, 158 Undergraduates• 2 courses: Graphic Design (99 students) and Web

Design (59 students).– 2 sections of each.

» 1 section of each has badges. » 1 section of each does not.

Graphic Design 1

Graphic Design 2

Web Design 1

Web Design 2

Badges Yes No Yes NoStudents 49 50 30 29

Page 40: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Current Courses: Background

• Courses are meant to prepare students for the Adobe Certified Expert (ACE) Exam in the related software.– Web Design = Dreamweaver– Graphic Design = Photoshop

• Completely web-based 16 week courses.• Balanced emphasis on Quizzes / Exams and

Project-based assignments.

Page 41: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Current Courses: Badges

• Badges are:– Unexpected - No list of possible badges can be

found by students.– Private – Students cannot see the badges others

have earned.– Both Objective and Subjective.

• Objective – Named all layers within a Photoshop project.

• Subjective – Helped a classmate succeed

– Awarded both automatically and by the instructor.

Page 42: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Method of Award

• Checkbox in gradebook for each badge simplifies the process

Page 43: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Viewing Badges

• Badges can be found from the course menu, like other important course information.

Page 44: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Viewing Badges• Click a badge to see how it was earned

Page 45: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Current Courses: The Goals

• Improve:– Motivation– Engagement– Academic Performance

• Identify:– Can number of achievements earned be used to

predict grades in a certification preparation course?

Page 46: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Badging Observations

• Each stakeholder determines value

o Issuer, Earner, “Observer,” (Displayer)

• Potential value in each phase of badging:

o Underlying data/record

o Notification email

o Claiming (“Save and Share”)

o Making public

o Linking to specific badges

Page 47: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Unanswered Questions to Ponder

• Why do badges appeal to some but not others?

• Does badging really engage the unengaged?

• What is the right balance of automation and personal attention for course badging?

• What is the relationship between badges and formal credentials?

• What is the right balance of curricular and co-curricular badging at an institution?

Page 48: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Toward a Taxonomy for Badge Design• Subject (e.g., information literacy; educational tech; interdisciplinary; etc.)• Type (e.g., academic; professional development; etc.)• Level (e.g., undergraduate, graduate; etc.)• Tiers (e.g., single tier; multiple difficulty tiers; three cumulative tiers; etc.)• Issued By (e.g., single issuer/multiple; instructor; organization; etc.)• Scale (e.g., course-level; discipline-specific; institution-wide; public; etc.)• Population Size (i.e., to whom badges are available; e.g., 1230; 35; 217; etc.)• Badge Image Source (e.g., graphic designer; badge making template; etc.)• Platform (e.g., Purdue Passport; Credly; etc.)• Focus (e.g., core competencies; off-topic diversion/fun; secondary competencies; etc.)• Grades/Badges (e.g., badges only; badges = grades; grades & badges separate; etc.)• Fixed/Extensible (e.g., defined list of badges; new badges suggested/added on the fly; etc.)• Expected/Unexpected (e.g., published list (“a priori”); discovered Easter eggs; etc.)• Visibility (e.g., issuer; earner; bounded group (“class”); public; etc.)• Status (e.g., complete; interrupted; underway; planning; etc.)

Version 1.1

Page 49: Stinkin' Badges: Why We Need 'Em and How to Use 'Em

Thank You!

• Rudy McDaniel, [email protected], @rutang5

• Joey Fanfarelli, [email protected]

• Kelvin Thompson, [email protected], @kthompso

Badge graphics courtesy of Matthew Dunn