Using group discussion to enhance evaluation of clinical

teaching: A pilot study

Dr Charlotte Mackay –GP ST4 Medical Education Fellow, Queen Mary University of London

Maria Hayfron-Benjamin –Lecturer in Medical Education, Queen Mary University of London

Background• Varying attitudes towards

quality assurance processes

• Teachers view process as an imposition and distraction (Booth et al., 2008)

• Students report ambivalence about whether their views and opinions are taken seriously (Spencer and Schmelkin, 2002)

Where we are currently at QMUL

What was good about this placement?

“Everything”

What did you find useful about this teaching?“Nothing”“Clinical teaching is good”

Was there something you thought should be changed to improve it? If so, what would you suggest?“Not really”

What we stand to gain from improved evaluation and feedback

• For tutors:• Valuable information on teaching methods –what

worked/did not work and why?

• For students:• Development of skill set to use throughout careers for

teaching, continuing professional development and appraisal

• For faculty:• Highlight opportunities for curriculum development

QMUL Pilot: Methods

• Cohort study

• Medicine in Society GP Placement

• 55 year one students (groups of 6-8 per practice)

• Eight GP tutors (and eight community tutors)

QMUL Pilot: Timeline

• October 2014• Students and tutors recruited into study by email

• November/December 2014• Initial practice visits to introduce project in more detail and discuss

evaluation template

• February/March 2015• Student group discussion (observed by MHB and/or CM) and completion of

feedback proforma • Evaluation sent to GP tutors once MedSoc module completed

• April 2015• Focus group interview with students• GP tutors invited to feedback via email

Results: Observing student group discussion

• Discussion lasted 45-60 minutes

• Student chair and scribe

• Lively debate

• Range of voices heard

Results: Focus Group

• Three participants

• Core themes identified

Focus Group Theme: Benefits of group discussion

• “Sometimes you want to say something but if you have to fill in a form you forget it, but if you discuss it with others and they come up with an idea, then you’re like “oh yeah”’

• “It was something fun. It wasn’t something that you write it down and you find it boring”

• “I felt we worked better as a group, finding things to highlight and things that I wouldn’t really pick up on by myself”

• “It makes you appreciate things about the teaching that you might not necessarily have noticed”

Focus Group Theme: Potential pitfalls of group discussion

• “Some people expressed to me by themselves like “Oh I don’t really like that or this” but when we were in the group setting ….they kind of tagged along, so they weren’t as genuine as if you had asked them by themselves”

Focus Group Theme: Loss of anonymity

• “I don’t mind it being anonymous or not, its constructive feedback…it’s something more to improve”

• “As long as you’re polite then I don’t think it should be a problem at all”

Focus Group Theme: The role of the chair

• “Our chair was really good. He was like “what do you think about this?”…and then around the group so that all the voices were heard…he organised everybody and if someone was just saying “I agree” then he made sure they were first to talk at the next point”

Results: Tutor Feedback• Comments from two GP tutors and one community

tutor

• GP tutors:• Mixed response• Recognised that comments had assisted their own

personal development

• Community tutor:• ‘Enlightening’

Suggestions for improvement

• Students:• More detailed explanation of project needed

beforehand

• Tutors:• Proforma more explicit about specific MedSoc

learning objectives

Discussion

• Broadly positive response from participants

• Students:• More enjoyable than traditional feedback methods• Benefits of group discussion to stimulate new ideas

and clarify existing ones

• Tutors:• Concern about time required to implement• Recognise advantages of more descriptive feedback

Next steps

• Extension of pilot over 2015/16 academic year

• All MBBS Year1 and GEP MedSoc students to be involved

References• BOOTH, J. COLLINS, S. HAMMOND, A. 2008. Considered evaluation of clinical placements in a new medical school.

Assessment & Evaluation in Higher Education, 34, 17-29.

• CURRAN, V. CHRISTOPHER, J. LEMIRE, F. COLLINS, A. BARRETT, B. 2003. Application of a responsive evaluation approach in medical education. Medical Education, 37, 256-266.

• GMC 2011. Tomorrow's Doctors. General Medical Council.

• HAMMOND, A. COLINS, S. BOOTH, J. KALIA, S. 2009. Learning from Evaluation: A descriptive, student-informed approach. The Clinical Teacher, 6 (2) 73-78.

• NARASIMHAN K. 2001. Improving the climate of teaching sessions: The use of evaluations by students and instructors. Quality in Higher Education, 7, 179-190.

• RAUPACH, T., SCHIEKIRKA, S., MUNSCHER, C., BEISSBARTH, T., HIMMEL, W., BURCKHARDT, G. & PUKROP, T. 2012. Piloting an outcome-based programme evaluation tool in undergraduate medical education. GMS Zeitschrift fur medizinische Ausbildung, 29, Doc44-Doc44.

• SPENCER, K. J. & SCHMELKIN, L. P. 2002. Student Perspectives on Teaching and its Evaluation. Assessment & Evaluation in Higher Education, 27.

• STAKE, R. E. 1976. EVALUATE AN ARTS PROGRAM. Journal of Aesthetic Education, 10, 115-133.