Gen Ed AssessmentCritical Thinking Outcome
Multiple Choice Question (MCQ) DevelopmentProject in the Social Sciences
BASED ON SLIDES FROM DEC.
LAURA BLASI, PH.D., DIRECTOR, INSTITUTIONAL ASSESSMENT
Critical Thinking – The Gen Ed Outcome
When testing Critical Thinking in the General Education program at Valencia we are focused on the three indicators addressing – (1) bias, (2) use of evidence, and (3) context. A pattern in faculty work focused on Critical Thinking since 2002.
Accomplishment 1 – Materials and Website –
Valencia College Gen Ed Faculty Resources Specific to Critical Thinking
http://valenciacollege.edu/academic-affairs/institutional-effectiveness-planning/institutional-assessment/loa/ResourcesCriticalThinking.cfm
Purpose of the Multiple Choice Question (MCQ) Project
The current MCQ project :
(1) invests the money in our faculty and a homegrown test item bank that is emerging from our experience with the students in Gen Ed;
(2) increases the college’s capacity for reliability testing (in our IR office) moving away from reliance on consultants;
(3) assures that faculty concerns about external reliability using pilot data are addressed by recognized experts in the field;
(4) provides an option after Social Science faculty discovered that nationally normed example can be $6 per student or higher.
Different forms of questions are possible…. Examples can be taken from the standardized tests used across the country – for example –
excerpt of a study
Applying the StandardsImagine you are a 2nd year student…Notice the question begins with reference to Political Science but it is broad enough to be accessible to students who have not taken Political Science.
Next steps needed
Questions by Jan 31
Pilot February – March
Expert Analysis April
Discussion of Results Assessment Day
50 questions to develop (we can include review of those we have)
Work is distributed and faculty-led
Questions stand up to peer review applying external standards
So our “Self Test” questions hold up when applied to the items (internal standards.)
The MCQ creation strategies – not discipline specific - from the Steve Downing workshop are followed and adhered to (external standards.)
Timeline for the pilot takes into account student administration and validation study by USF
Dr. Steve DowningTips for Developing Multiple Choice Questions Across Disciplines (examples)
Write a clear “testing point” or objective for item [context, bias, evidence]
Pose a clear question - review, edit, rewrite
Focus on important/essential information
Assure that question can be answered without reading options
Write clear, concise items; avoid superfluous information
Include most information in stem, avoiding lengthy options
Don’t use trick questions
Test higher-order cognitive knowledge (he refers to Bloom’s Taxonomy)
Application, problem solving, judgment, synthesis