Abelardo Pardo (@abelardopardo)Faculty of Engineering and IT slideshare.net/abelardo_pardo

Increasing Student Engagement with Personalised Feedback

Ikhlasul Amal flickr.com

Education Grand Rounds Westmead Hospital Education Centre

8/May/2017

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 2

Simple information transfer is not working

Mazur, E. (2009). Farewell, lecture. Science, 323(5910), 50-51.

Krugazor flickr.com

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 3

Student Engagement

and Feedback

Design for Engagement Examples

theilr flickr.com

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 4

StudentEngagement

andFeedback

theilr flickr.com

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 5

“… robust correlations between student involvement in a subset of ‘educationally

purposive activities’, and positive outcomes of student success and development, including

satisfaction, persistence, academic achievement and social engagement”

Trowler, V. (2010). Student engagement literature review. York, UK: The Higher Education Academy.

Joshua Ganderson flickr.com

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 6

Bonwell, C. C., & Eison, J. A. (1991). Active Learning: Creating Excitement in the Classroom ASHEERIC Higher Education Report No. 1. Washington, DC, USA: George Washington University.

Active Learning

Any instructional method that engages students in the learning process.

Active learning requires students to do meaningful learning activities and think about what they are doing.

WolfVision G

mbH

flickr.com

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 7

Active Learning Works

Abelardo Pardo Increasing Student Engagement With Personalised Feedback

If you could choose one…• More than 500 meta-analyses

of student achievements

• 100 factors with potential influence

• Feedback in top five

• (74 meta-analyses) Most effective form: video, audio, computer-assisted instructional feedback, and/or related goals

8Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses related to achievement. New York: Routledge.

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 9

Boud, D., & Molloy, E. (Eds.). (2013). Feedback in Higher and Professional Education: Understanding it and doing it well. London and New York: Routledge.

Faruk Ateş flickr.com

“Feedback is a process whereby learners obtain information about their work in order to appreciate the similarities and differences between the appropriate standards for any given work, and the qualities of the work itself, in order to generate improved work”

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 10

Black, P., & Wiliam, D. (1998). Assessment and Classroom Learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7-74. doi:10.1080/0969595980050102

Gareth C

hristopher flickr.com

Innovations designed to strengthen the frequent feedback that students receive about their learning yield substantial learning gains

Abelardo Pardo Increasing Student Engagement With Personalised Feedback

Effective feedback…1. encourages contact between student and

instructors

2. develops reciprocity and cooperation among students

3. uses active learning techniques

4. is given promptly

5. emphasizes time on task

6. communicates high expectations

7. respects diverse talents and ways of learning

11

Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. American Association for Higher Education Bulletin, 39, 3-7. Chickering, A. W., & Ehrmann, S. C. (1996). Implementing the seven principles: Technology as lever. American Association for Higher Education Bulletin, 49, 3-6.

Kevin O'M

ara flickr.com

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 12

Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81-112. doi:10.3102/003465430298487

Feedback Levels

1. Task Level (understanding, performance)

2. Process Level (what to do to understand, perform)

3. Self-regulation level (detecting and directing effort)

4. Self level (personal evaluation and affect)

Fabien CAM

BI flickr.com

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 13

Hounsell, D. (2007). Toward more sustainable feedback to students. In D. Boud & N. Falchikov (Eds.), Rethinking Assessment in Higher Education: Learning for the Longer Term. London and New York: Routledge.

Perceived as an administrative chore instead of a pedagogical necessity

Marcin W

ichary flickr.com

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 14

Student Engagement

and Feedback

Design forEngagement

theilr flickr.com

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 15

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 16

Janson Hew

s flickr.com

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 17

“… teaching in higher education will necessarily shift the balance of its efforts towards a greater investment in design

as a way of coping with otherwise intolerable pressures on staff and

resources.”

Goodyear, P. (2015). Teaching as Design. HERDSA Review of Higher Education, 2, 27-50.

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 18

"There is no such thing as a neutral design"

Jeremy Brooks flickr.com

Thaler, R. H., & Sunstein, C. R. (2008). Nudge. Great Britain: Yale University Press.

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 19Thaler, R. H., & Sunstein, C. R. (2008). Nudge. Great Britain: Yale University Press.

“People make good choices in contexts in which they have experience, good information, and prompt feedback"

Derek Bruff flickr.com

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 20

Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: beliefs, techniques, and illusions. Annu Rev Psychol, 64, 417-444. doi:10.1146/annurev-psych-113011-143823

Christian W

eidinger flickr.com

We, as learners…

• May not know how to promote comprehension, retention, transfer.

• May not assess properly our own learning

• May be biased when judging our learning

• May rely too much on social beliefs

• Should become “adaptive learners”

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 21

Students are less likely to engage in pre-class activities if they are not interactive, do not provide formative feedback, and not coherently linked with

the face-to-face activities

O'Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. The Internet and Higher Education, 25, 85-95. doi:10.1016/j.iheduc.2015.02.002

Dan Klim

ke flickr.com

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 22

Beware of technology pushing learners away from rational thinking

Jen R flickr.com

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 23

“38 meta-analyses investigating 105 correlates of achievement, based on 3,330 effect sizes from almost 2 million students”

Schneider, M., & Preckel, F. (2017). Variables Associated With Achievement in Higher Education: A Systematic Review of Meta-Analyses. Psychological Bulletin. doi:10.1037/bul0000098

• The effectivity of courses is strongly related to what teachers do.

• The effectivity of teaching methods depends on how are implemented

• Teachers can improve the instructional quality of their courses by making a number of small changes

- providing detailed task-focused and improvement-oriented feedback

• The combination of teacher-cantered and student-cantered instructional elements is more effective than either form of instruction alone

Variables associated with achievement

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 24

Sam Abraham

flickr.com

Blended Learning

Frontier between physical and virtual spaces is blurring

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 25

1. Think in multiple spaces

Towards higher order skills

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 26

2. Keep outcomes in mind while designing

After this lecture/weekstudents should be able to …

CC

SU N

Z 2013 flickr.com

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 27

training p a h flickr.com

Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: beliefs, techniques, and illusions. Annu Rev Psychol, 64, 417-444. doi:10.1146/annurev-psych-113011-143823

3. Design preparation activities

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 28

hijukal flickr.com

Think outside of the box

4. Design face-to-face activities

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 29

PNN

L Pacific Northw

est flickr.com

• Collect data about how students participate in a learning experience

• Translate observations into actions

• Take into account the instructional design

• Expertise of the instructor

• Create and deploy personalised support actions

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 30

Student Engagement

and Feedback

Design for Engagement Examples

theilr flickr.com

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 31

Simple intuitive structure to find resources

Available: bit.ly/elec1601 (Only Australian Universities)

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 32

Discuss Strategies, Approach, Difficulties

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 33

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 34

Initial contact with video

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 35

Formative assessment to promote practice

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 36

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 37

Tracking

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 38

You should take a more careful look at how symbols are encoded in the video. Would you be able to encode/decode UAL symbols without looking at the video?

Good initial work. However, did you understand the trick to handle encoding with a variable number of bits? Would you be able to provide an example?

Good work. Would you be able to come up with your own machine language and your encoding scheme? Remember that it has to be unambiguous.

Thorough work with the task about machine language encoding. Give it a quick review before the midterm.

Q1 Q2 Q3 Q4

Instructor

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 39

AutomaticEmail

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 40

Effect size (Cohen’s d) = 0.49. Medium positive effect

Effect size (Cohen’s d) = 0.21. Small positive effect

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 41

Liu, D. Y.-T., Bartimote-Aufflick, K., Pardo, A., & Bridgeman, A. J. (2016). Data-driven Personalization of Student Learning Support in Higher Education. In A. Peña-Ayala (Ed.), Learning analytics: Fundaments, applications, and trends: A view of the current state of the art. In preparation: Springer.

Student Relationship Engagement System (SRES)

Instructors define simple rules to create personalised emails from multiple data sources

Not so m

uch flickr.com

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 42

ontasklearning.org

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 43

IF [video 3.9.5 not watched] THEN “…."

ontasklearning.org

Abelardo Pardo Increasing Student Engagement With Personalised Feedback 44

• Support instructors to create personalised feedback

• Simple rule-base knowledge encoding

• Provide appropriate view of data sources

• Scale to large and highly diverse cohorts

• Open-source project

• First pilots in Q1/2 2017

• Tutorial in LAK 2017

• Contact us if interested

ontasklearning.org

Abelardo Pardo Increasing Student Engagement With Personalised Feedback

Conclusions

45

Betsy Weber flickr.com

• Strong evidence connects student engagement and feedback with academic attainment

• (Good) Feedback can have a strong positive effect • Embrace design in a complex environment • Acknowledge student approaches to learning • Collect data and provide frequent, personalised,

situated feedback

Abelardo Pardo (@abelardopardo)Faculty of Engineering and IT slideshare.net/abelardo_pardo

Increasing Student Engagement with Personalised Feedback

Ikhlasul Amal flickr.com

Education Grand Rounds Westmead Hospital Education Centre

8/May/2017