ride 2016 badges

21
OPEN BADGES TO SUPPORT PEER FORMATIVE ASSESSMENT IN ONLINE LEARNING ENVIRONMENTS STYLIANOS HATZIPANAGOS, KING’S COLLEGE LONDON & STEVEN WARBURTON, UNIVERSITY OF SURREY

Upload: stylianos-hatzipanagos

Post on 18-Feb-2017

114 views

Category:

Education


0 download

TRANSCRIPT

Page 1: RIDE 2016 badges

O P E N B A D G E S T O S U P P O R T P E E R F O R M AT I V E A S S E S S M E N T

I N O N L I N E L E A R N I N G E N V I R O N M E N T S

S T Y L I A N O S H A T Z I PA N A G O S , K I N G ’ S C O L L E G E L O N D O N & S T E V E N W A R B U RT O N , U N I V E R S I T Y O F S U R R E Y

Page 2: RIDE 2016 badges

CONTEXTA project that explored the effectiveness of peer feedback and assessment:

Investigated whether open badges can support learning in online environments through enhanced engagement; Revisited and evaluated badge typologies with the aim of identifying the most appropriate typology that supports peer feedback and assessment; And created a clear correspondence between the typology of badges and peer feedback and assessment practice in online classrooms.

Page 3: RIDE 2016 badges

RATIONALEUse of digital badges to support peer feedback and assessment in higher education.

By the use of open badges in encouraging and rewarding competences, knowledge and skills acquired in digital learning spaces, we aimed to reinforce the value of peer assessment and by extension peer feedback and review activities in supporting student learning.

Page 4: RIDE 2016 badges

PEER ASSESSMENT DEFINITION

Peer assessment is to be understood as an educational arrangement in which students assess the quality of their fellow students’ work and provide one another with feedback.

Dochy et al., 1999

Page 5: RIDE 2016 badges

FORMATIVE E-ASSESSMENT CAN:

• engage learners with tasks that enhance learning outcomes and test the development of higher order capabilities;

• enrich learning approaches by making feedback central to all assessment activities;

• promote a dialogue in relation to feedback, peer and self assessment activities.

Hatzipanagos & Warburton 2009

Page 6: RIDE 2016 badges

PEER ASSESSMENT/PEER REVIEW

• Peer assessment is frequently underused in online learning environments.

• Its informal learning value is not exploited by peers as it is not explicitly linked to summative assessment activities.

• There is a reluctance to participate in such peer review activities and an ambivalence of tutors in relation to their value.

• Informal aspect is not clearly or meaningful embedded in the learning process.

Page 7: RIDE 2016 badges

Peerassessmentattributes(Dochyetal)

Whattheresearchsays Badges

Validity Peersarepronetoproduceratingsbasedonuniformity,raceandfriendship ifthereisnoextensivetraininginpeerrating.Majorityofstudiesshowedanacceptablyhighvalidityandreliabilityinavarietyoffields.

Clearcriteriaembeddedinbadgesdefinition.Validitycanbehighifbadgesarewelldefined.

Fairness. StudentperceptionsoffairnessofpeerassessmentStudentswereverypositiveabouttheeffectsofalternativeassessmentontheirlearning.

Opennessandclarityinmicrocertificationfundamentalrequirementsofafairandvalidassessmentsystem.

Accuracy. Agreementbetweenstudentandtutors.Noconsistentresults.

Therecanbe‘objective’approachesoftestingaccuracyinmicrocertification

Effects. Peerassessmentfosteredanappreciationfortheindividuals' performancewithinthegroupandinterpersonalrelationshipsintheclassroom

Badgescancaptureinterpersonalrelationships(softskills)andperformance(hardskills)

Implicationsforpractice.

valuableasaformativeassessmentmethodbut…

friendshipmarking;collusivemarking;decibelmarking;andparasitemarking..

AppropriatenessofemergenttechnologiessuchasbadgesforformativeassessmentAssessing‘mishaps’ canbepreventedbylearninganalyticsanddigitaltrailinnewenvironments

Page 8: RIDE 2016 badges

METHODOLOGY

The Open Badges used in King’s College London and the University of Surrey.

King’s programme (Postgraduate Certificate in Academic Practice) targets novice lecturers, PhD students with a teaching role, librarians and information specialists who have chosen to study a ‘Supporting technology enhanced learning’ module.

Peer Assessment is an integral part of these modules and learners are strongly encouraged to participate in formative assessment activities. There was an opportunity to link performance and peer review in these activities with open badges.

Page 9: RIDE 2016 badges

METRICSAssigning badges was a semi-automated process. 1. Use of the learning environment triggered automated assignment of

badges for metrics associated with use of the learning environment e.g. when students used discussion fora and the social media communication. Metrics such as number of posts to a discussion forum, number of log-ins, response to other posts, sharing content and reflections were used.

2. Other badges were deployed by application and evidence of achievement, moderated by the project team or by peers on the course. A protocol for awarding a badge was developed by the project team in collaboration with the module tutors and learners.

Evaluation of the project activities was carried out by collecting data from the online learning environment, using focus group interviews and by end of module and programme evaluation questionnaires.

Page 10: RIDE 2016 badges

PROJECTIONS• By rewarding positive behaviours the participants would feel more motivated

to complete learning activities. The aim of this approach was to manage any motivational drops that can commonly be experienced in online/blended courses.

• Participants were encouraged to push their practice beyond the central learning design journey and give opportunities for digital professional identity development.

• The project would provide a mechanism for self-reflection by the academics engaged in the study. They would be able to evaluate the benefit of using a badging system within their own learning designs.

Page 11: RIDE 2016 badges

CONCRETE EXAMPLES

• For example, the ability to award a badge to your peer evaluator if they provide good constructive advice on your work.

Page 12: RIDE 2016 badges

Powered by

Digital Badges evaluation

Page 13: RIDE 2016 badges

Q2: HOW DO BADGES COMPARE WITH OTHER WAYS OF DEMONSTRATING YOUR ACHIEVEMENTS?

Page 14: RIDE 2016 badges

Q5: DO YOU THINK THAT BADGES MOTIVATE LEARNERS IN AN ONLINE ENVIRONMENT?

Page 15: RIDE 2016 badges

Q14: DO YOU SEE BADGES BECOMING PART OF YOUR DIGITAL ACADEMIC IDENTITY?

Page 16: RIDE 2016 badges

PERCEPTIONS OF VALUE• Random orchestrations between learners that are not part of the same

community.

‘Due to the limited time I spent with my fellow students I didn't get to know them well. Therefore I felt that the use of badges was not particularly useful’

• Soft indicator of achievement, used for formative rather than summative purposes.

‘in lieu of marking or registers they could be used as a 'soft' indicator of achievement (e.g. a record of optional on-line courses taken)’

• Overall caution against the use of badges

‘don't think the possibility of winning a digital badge really changes adult behaviour’

‘unless digital badges are established to international- or the very least national standards as a well recognized metric of achievement I wouldn't think they will affect online learning.’

• Student centred approach should work

‘something that the students themselves defined and awarded, rather than it coming from me (also a good way to gauge what they think of the whole thing!)’

Page 17: RIDE 2016 badges

WHAT DO YOU THINK WOULD HELP THE ADOPTION OF BADGES?• I think a clear rationale for the benefits to both the student and staff

• Defining their purpose more clearly, and applying them consistently to something that is meaningful to students' learning andachievements. Instinctively, it's quite hard to take them seriously. Is it the name? or the varied application? Not sure.

• If someone badges feed into the assessment process -i.e. maybe 5 extra marks could be added to the summative assessment to recognize participation and contribution.

• If they were linked to module credit

• As the criteria for digital badges are still very loose I don't think it is likely that they are used in evaluating academic contribution. Stricter and well defined metrics are in place for academic professional development (at least in Life Sciences).

• I might be wrong about this - and it's a bit of a chicken and egg situation - but I don’t see them as an important part of a professional identity, and I gather from friend on corporate e industry (e.g. cloud computing) that they are not taken seriously there either. If I felt that they were something that employment panels cared about and that validated my use of e-learning tools, then I'd swallow my scepticism, adopt them enthusiastically and show off about the ones I have earned. Of course, if none of us decide to adopt thatapproach then the panels won't care, and so on around the circle... it's just really difficult for me not to see them as something rather childish and believe that my students would feel the same.

• I think there's potential use for badges as part of the summative assessment (which would then require further guidelines and transparency).

• Academia is already a game measured by the 'badges' of qualifications, publications, citations and research grants -- further gamification seems unnecessary

Page 18: RIDE 2016 badges

CONCLUSIONS

• Badges as a ‘Trojan horse’ for recognizing and embedding transversal skills in university curricula;

• Perception of value of badges for summative assessment; promotion; recruitment and employability;

• Establishing professional identity linked to generational attitudes; (and digital literacies?), ‘good for our students but not good for us’;

• Motivation is not a key driver in using badges, at least for teaching practitioners.

Page 19: RIDE 2016 badges

RECOMMENDATIONS

• Integration of badges (and ePortfolios) with university systems (e.g. VLE) is necessary for uptake;

• Consider value of badges for summative assessment; promotion; recruitment;

• Integration of badges with external certification systems (Mozilla open badges, Open Badge Factory) and these with social media;

• Embed micro-portfolios of badges in PDP (personal development planning) and portfolio building for staff and students.

Page 20: RIDE 2016 badges

REFERENCES

• Dochy , F. , Segers, M. & Sluijsmans, D. (1999) The use of self-, peer and co-assessment in higher education: A review, Studies in Higher Education, 24:3, 331-350, DOI:10.1080/03075079912331379935.

• Hatzipanagos, S. & Warburton, S. (Eds) (2009). Social Software and Developing Community Ontologies. 598 pp. London: Information Science Reference, an imprint of IGI Global ISBN 978-1-60566-208-4.

Page 21: RIDE 2016 badges

IF YOU HAVE QUESTIONS OR FEEDBACKEmail

[email protected]