measuring your success in course redesign
DESCRIPTION
Measuring Your Success in Course Redesign. Sandi Connelly Gosnell School of Life Sciences, RIT, Rochester, NY [email protected] or [email protected]. The Dirty Words in Education. Success Efficacy Assessment Evaluations. Our Topics for Today. - PowerPoint PPT PresentationTRANSCRIPT
Measuring Your Success in Course Redesign
The Dirty Words in Education
1. Success
2. Efficacy
3. Assessment
4. Evaluations
Our Topics for Today
1. How will you measure success?
2. Will you use historical data (common exams, etc.) to support the efficacy of your assessments (MyLab & Mastering)?
Mastering platform in Sciences similar to MyLab platform in other disciplines
3. MyLab & Mastering assignments will contribute what percentage to a students final course grade? = assessment
4. What do you do with the information evaluated?
Today’s Plan
My course structure
Implementation of a few simple techniquesQuick ways to measure successHow to use MyLab / Mastering to maximize student learning
How do other data support MyLab / Mastering use?
How much is too much (or too little!) for a grade item?
How to use MyLab / Mastering for Course Assessment
My Course
General Biology (year-long sequence)
Typical # of students: 250-340 (taught in two sections)This year 408
Typical # of different majors: 55 from all RIT Colleges
~ 50/50 science/non-science
Freshmen -> grad students
Current grade calculation (2013 – 2014):
65-70% exams; 15% Mastering Bio HW; 10% Mastering Bio quizzes; 5-10% in class participation
Before you measure success…
Are Students Prepared to Succeed?
Preparation (lack of!) comes in many formsReading / writing comprehensionEnglish as a Second Language (or 3rd, 4th, ….)Learning disabilities Fundamental knowledge in sciencesStudy skillsDedication to the courseDedication to the degree / collegeEtc……
Preparing Students for Success
Homework & QuizAssign immediately on Day 1Provide timely feedback on the assignment
“Suggestions” from instructor do hold water!
Require regular assignments
In Class Response ActivitiesEvery day! (“mock quiz” type)Immediate or timely feedback critical
Preparing Students for Success
Be clear about the goals of pre-course activitiesE.g. pretests, diagnostic tests, etc.Benefit to students
Not enrolled in a class in which they cannot succeed
Decreases DWF rates
Increase retention – intangibles
Benefit to you / courseLess time on “catch up”, More time on application
Less stresses associated with DWF, office hours, etc.
So you have “prepared” students
… Now what?
Measuring Success – How? Of what?
Retention (D/W/F)
Common exams?
Shared assessments across sections?
Final exam scores
Final course grades
MyLab / Mastering
Alternative learning initiatives
Measuring Success – How? Of what?
What works for …YouYour studentsYour discipline
Your assessment office?Your supervisor?
Measuring Success – How? Of what?
Let’s focus on MyLab / Mastering as a success tool …
We can add in the other things later …
MyLab / Mastering = X %
2008-2009: Not much – trial only; assignments optional
3 attempts per question; unlimited time
2009-2010: Not much – trial only; assignments optional3 attempts per question; unlimited time
2010-2011: 5% of grade; ~20% never did any; nearly all students who completed the assignments had a 100% avg
3 attempts per question; unlimited time
2011-2012: 15% of grade; all complete at least most; avg scores 88-93%
2 attempts per question; unlimited time
MyLab / Mastering = X %
2012: 20% of grade HW and 5% on quizzes; all students complete nearly all (> 97% completion rate)
HW = 2 attempts per question; unlimited timeQuiz = 1 attempt per question; 60 min time limit
2012 ExampleMitosis/Meiosis HW: 96 questions; avg time spent = 44 minutes; avg score = 85.9%Homework due before lecture
Result? No time in class on process – instead, theory!
MyLab / Mastering = X %
2013 – 2014: 15% of grade HW and 10% on quizzes; all students complete nearly all (> 98% completion rate)
HW = 2 attempts per question; unlimited time; weeklyQuiz = 1 attempt per question; 60 min time limit; weekly
Food for thought … is this % too much?Will students cheat, copy, etc.?
Everything has challenges …
What is your biggest challenge?
Time is the untamed obstacle for everyone ..
Take 5 minutes Sit down! (or better yet – while you are shampooing!)What information do you really want your students to retain?
WHY?
What do you want your students to leave your class knowing?
WHY?
Challenges to Assessment
Things that I have considered and toyed with …
How much do I want the students to do outside of class on MyLab/Mastering?
Limited to “Time Estimated” = 1 hourLess if possible! I want their attention!
Challenges to Assessment
Things that I have considered and toyed with …
What is the value of a long assignment if the students don’t do it, copy it from another student, just use all of the hints, etc.?
For students who are failing exams, but have 100%+ on the homeworks, I look at their time spent in MyLab / Mastering
Discuss issue with them in person and make note in the Institute Reporting
Challenges to Assessment
Things that I have considered and toyed with …
Are the students just memorizing answers, or are they applying the information and retaining it?
Have stopped assigning reading / test bank questions
Implement more low stakes quizzes
Forced repeated recall of information in class
You have data … Now what?
Student Data Assessment
Why assess?
My general interestAre the students learning what I think they are learning?
How can I improve this course?
University call to assess Gen Ed courses (Accreditation)
Assessment needs?
My interest Exam and homework scores before and after MyLab / Mastering implementation
University Guidelines: 1 – 4 scale of comprehension
Student Data Assessment
What was assessed?
Kept it simple to start!Exam averages
Add as I have interest / time!Participation in MyLab / Mastering
Scores on MyLab / Mastering
Time spent on MyLab / Mastering
Student Data Assessment
Exam grades (presented herein)
All stored in ExcelTM since 2007
Mastering (not presented)
Export grades & time spent from Mastering to ExcelTM
Processing, Compilation, and Statistical Analyses
ExcelTM
Exam Score Trends
Err = St ErrAvg n = 244
Talk to me data!
Higher exam averages every year – but leveling out
Significant differences only in plants – but trends are obvious
Too many students from too many majors to see huge jumps in scores despite all efforts
Tightening of exam scores
Standard deviations and ranges decreasing
Increasing amount of “average time spent” on MyLab / Mastering assignments
Assessment Challenges
Why do exam grades change?
Things I have consideredWhat changes from year to year other than MyLab / Mastering?
Students?
Accessibility of tutors?
Me?
Order of topics covered?
Lab instructors?
Assessment Challenges
Why do exam grades change?
Things that I want to consider
Students Time spent on studying (other than MyLab / Mastering)?
Major?
College entrance exams? AP Bio?
DWF rates by major and biology “experience”
MeLectures improved?
Lecture structure change with online assignments?
“Napkin Notes”
General observations:
Increased time spent on assignments after 1st exam of term
Increased # questions in office hours on assignments
Increased time spent “off topic” in lecture – not a bad thing!
Changes implemented to date:
Lecture time is more interactiveParticipation activities with the students (worksheets, group work, think-pair-share, flipped lectures, etc.)
Modifications to assignments and quizzes to “challenge” the students in preparation for class and exams
Advice for Instructors Looking to Assess Success
Don’t assess while you are teaching the course!
Assessment can be done any time! Store the data, save it for later
Even if you have time … subconsciously not good for your teaching!
Don’t try to assess everything at once!
Pick one exam in your course, and compare the average ± standard deviation (standard error) between years
Even simpler?
Compare exam averages between class sections. TA DA! You just did an assessment!
Advice for Instructors Looking to Assess Success
Don’t expect MyLab / Mastering to make your students in to geniuses!
These are tools! And how the students implement the tools (accept the tools!) will be highly variable.
You may not see immediate changes in exam grades
Accept the “intangible” as a result!
You can not grade student enthusiasm (which means you can’t assess it – sorry!)
Student enthusiasm is contagious – not just to other students – but also to you!
Advice for Instructors Looking to Assess Success
What can you do with your data?
FLAUNT IT!White papers
Peer-reviewed papers
Teaching portfolio
Promotion dossier
Assessment office
Final thoughts …
What else is assessable?
I got my class back!
Not spending class time on giving quizzes, answering questions about homework format, etc.
What do we do then? Talk about Biology!
AND … most of my assessment is built in to MyLab / Mastering and my course management system!
Don’t reinvent the wheel!
Download grades, assign your assessment strategy, .. TA DA!
What else is assessable?
HUGE time savings in my classes
I have “recovered” 35 minutes per week in class (of 150 min) with no quizzes and homework things in class
Prepared students means < 45 min lecture!
> 105 min biology exploration!
Time saving is so huge … how huge is it?
Enough that I am completely “flipping” one section
DATA! DATA! DATA! DATA! DATA! DATA! DATA! DATA!
But .. The Most Important Thing to
Me?
What the Students Have to Say!
"Mastering Bio helps a lot! The more you have us do, the more I learn!”
"I wish that our exams were as interactive as Mastering! I love learning that way!"
"(Favorite part about Mastering) aspects of the assignments helped me to better grasp the materials through the videos. I would not have done as well on those parts of the test without Mastering.”
"More Mastering - 'nuf said!"
Let’s Recap
1. How will YOU measure success?
2. Will YOU use historical data (common exams, etc.) to support the efficacy of your assessments (MyLab & Mastering)?
3. MyLab & Mastering assignments will contribute what percentage to YOUR students final course grade? = assessment
4. What do YOU do with the information evaluated?
1. Success
2. Efficacy
3. Assessment
4. Evaluations
THOUGHTS!?