subverting multiple-choice questions for deep learning
TRANSCRIPT
Subverting MCQs for Deep Learning
Learning outcomes
• Understand how formative MCQs can be used to promote deep learning in face-to-face and online learning
• Take a short formative MCQ quiz to see how it works from a participant’s perspective
• Begin to construct an online formative MCQ quiz
Deep approach v. surface approach
LearnersSurface approach Deep approach
Passive listeners Active participants
Understanding information only as it is presented
Trying to make linkages between information presented with other bits of (previously received) information
Processes and retains information as separate facts or bits of information
Looking for an underlying rationale/overarching structure to connect the (independent) facts
Focuses on breadth of content coverage Focuses on depth of content coverage
Assessment-oriented view of education Academically-inclined
Non-academic priorities or interests Intrinsic curiosity or a preference to engage with material at a conceptual level
Is expecting a predictable learning experience and not looking to be stretched
Is looking to be stimulated and challenged
TeachersSurface approach Deep approach
Presenting and expounding (explain) Eliciting (evoke) and probing
Learners as passive listeners Learners as active participants
Learner’s role: listen, take notes, ask the odd question
Learner’s role: active listening, discussion with peers/lecturer, query and counter-argue?
Breadth of coverage is a priority Depth of coverage is a priority
Views unwieldy questions as a distraction from lecture
Welcomes questions and views them as a teachable moment
Provides answers to students’ questions Facilitates students in seeking out the answers to their questions
The multiple choice question (MCQ) as we know it
• What is it?
• How is it currently used?• Why is it not a favoured form of assessment?
What is the most widely spoken language in the world?
i. Englishii. Chineseiii. Spanishiv. Mandarin
Can MCQs be used to promote deep learning?
Formative MCQs
• Improve conceptual understanding• Target areas of potential confusion• Provide feedback• Prime learners• Activate prior knowledge• As a diagnostic test/JIT• In concert with other active learning methods
– peer instruction, discussion fora, etc.• Enable self-regulated and reflective learning
Anatomy of a formative MCQ
Topic Question set Distractors
Feedback Marking schemeIntegrated learning activity
BLOOC 10
Topic
• A key topic or fundamental concept• Lends itself to MCQ-style questions• Not likely to require frequent changes or updates
8 July 2014
Stem and options
Which of the following statements are true of learners adopting a deep approach?i. They retain information and knowledge longerii. They are able to make more connections
between different pieces of informationiii. They experience confusion and uncertainty
when trying to understand new knowledgeiv. They focus on the details when mastering a
topic
stem
Correct options
distractor
BLOOC 12
Feedback
• Adaptive• Interactive• Deferred
8 July 2014
BLOOC 13
Marking scheme
• Multiple correct answers• Negative marking• Certainty-based marking
8 July 2014
BLOOC 14
Results of the self-assessment
8 July 2014
BLOOC 15
Results – graph of results
8 July 2014
Evaluating the effectiveness of MCQs
BLOOC 17
Trivia v. deep learning
8 July 2014
BLOOC 18
Do MCQs promote deep learning?
• Biggs’ theory of learning• Deep/surface/strategic
• Bloom’s taxonomy• Knowledge, comprehension, application, analysis, synthesis,
evaluation
• Nicol’s principles of feedback• Clarifying goals, criteria and standards; self-assessment and
reflection, high-quality feedback; peer dialogue, feedback and motivation; closing the gap; just-in-time teaching
8 July 2014
How can you increase the power of a formative MCQ?
BLOOC 20
Limitations of MCQs for deep learning
• The deep approach is not often used in everyday teaching• Difficulty finding an appropriate topic• Pedagogical design
• Learning aims• Integrating it with other learning activities
• Constructing the test• Item ambiguity• Technology
• Student expectations need to be managed
8 July 2014
BLOOC 21
Can you use MCQs in your teaching?
• In which part of the curriculum could you use an MCQ?• What is the underlying rationale? • What would be the benefits for the students, for yourself?• How would it promote deep learning and better learning
outcomes?• Could you integrate it with other learning activities? • What potential challenges do you foresee with this
approach?
8 July 2014
DIY
• Get into groups of three and design a formative MCQ quiz
Topic Question set Distractors
Feedback Marking schemeIntegrated learning activity
Wrap up and review
BLOOC 24
References• Biggs, J. and Tang, C., 2007. Teaching for Quality Learning at University. 3rd ed. Berkshire: SRHE
and OUP• Bloom’s Taxonomy. Centre for Teaching. Vanderbilt University. [online] Available at:
http://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/ [Accessed 24 June 2014]• Lublin, J., 2003. Deep, surface and strategic approaches to learning. Good Practice in teaching
and Learning. Centre for Teaching and Learning, University College Dublin. [online] Available at: http://www2.warwick.ac.uk/services/ldc/development/pga/introtandl/resources/2a_deep_surfacestrategic_approaches_to_learning.pdf [Accessed 18 June 2014]
• Nicol, D., 2007. E assessment by design: using multiple choice tests to good effect. ‐ ‐ Journal of Further and Higher Education. 31:1, 53-64. Available at http://ewds.strath.ac.uk/REAP/public/Papers/MCQ_paperDN.pdf [Accessed on 30 May 2014
8 July 2014