learning is a subtle process. students vary enormously!! disciplines & professions require...
TRANSCRIPT
Overcoming Challenges with the SLO process
CHALLENGE - The Business Model
Problems with the “production process” in higher education
Learning is a SUBTLE Process.
Students vary ENORMOUSLY!!
Disciplines & professions require diverse methods of instruction.
The “production process” in higher education is
“More Intricate and complicated
than that in any industrial
enterprise…”
The University is not a business:Treating the University as a business has
damaged the quality of education and undermined its requests for funding.
Given the catastrophes that the business world has imposed on the global economy, I find it insane that the business model has been superimposed on education.
--Marc Epstein
ANTIDOTE Stay informed!
Increase ability to clearly articulate how education is different from business
Remember colleges are not “cookie cutters” of each other
Define what we do; otherwise, somebody will do it for us
CHALLENGE – Complex Mission
ANTIDOTE Define your local priorities
Have conversations about who you serve and why
Plan strategically
Pay attention to legislation
CHALLENGE –Tools Missing
ANTIDOTE Determine “real” tools needed
Identify tasks and path to completion
CHALLENGE #2 RESISTANCE TO CHANGECHALLENGE – Formalized
Assessment is
ANTIDOTEIdentify and chart layers
Identify and chart levels
Diagram connections
Keep it simple!
CHALLENGE - External Factors
ANTIDOTEIdentify external forces for your local college
Prepare counter evidence if needed
Choose to be collegial
Work cooperatively
CHALLENGE - Resistance to Change
Change Happen
s!!
The phases of transition through change
Focusing on the past and denying change
Assessing where they stand and
resisting change
Imagining and exploring ways to
the future
Commitment to work together for a
new future
The phases of transition through change
Focusing on the past and denying
change
Assessing where they stand and
resisting change
Imagining and exploring ways to
the future
Commitment to work together for a
new future
SLOs added to the
accreditation standards
2002
SLOs Proficiency
2012
ANTIDOTE Acknowledge and tolerate resistance
Encourage and support change
For those that are stuck in concrete..
Breathe & Move on!
CHALLENGE - FACULTY M.I.A.
ANTIDOTEIdentify reason for each MIA faculty
Match people with their skills
Make meetings etc. productive AND enjoyable
Publicize accomplishments of faculty work
Streamline faculty work
Schedule meetings at faculty friendly times
Plan for time to discuss our craft!
CHALLENGE - Budget
ANTIDOTESimplify tasks
Develop enrollment management plan (Assert faculty role)
Choose to find the positive
Find the light!
Talk more about students and learning
CHALLENGE - TIME
ANTIDOTERelook at the work of faculty
Use meetings for the “work”
Put minutia in emails
Rethink committee structure
Focus on the 10+1
And, the Current Biggest Challenge
Closing the Loop:
Providing Credible Evidence
Closing the Loop means going back to square one, or…
Assessing the effectiveness of changes to curriculum, pedagogy, resources, assessment methods, or learning
outcomes implemented in response to assessment
results.
GLOSSARY FOR TODAY
CLO Course Learning Outcome or Course Student Learning
Outcome
PLO Program Learning Outcome or Program Student Learning
Outcome
ILO Institutional Learning Outcome or Institutional Student
Learning Outcomes
CASE STUDY #1
Course Learning Outcomes & Closing the Loop
English 49 Success Rates
ENGL 049
58% 57%
65%71%
76%
Spring 2007 Spring 2008 Spring 2009 Spring 2010 Spring 2011
ENGL 049
60%
51%
61%
68%73%
Fall 2006 Fall 2007 Fall 2008 Fall 2009 Fall 2010
Note that English 49 Coordination data showed that a change in assessment during Fall 2008 from a final exam (71% on average) to portfolio (82% on average) showed an 11% increase in subsequent success rates of those students enrolled in English 101/105.
Success Rates of English Lab Students
Fall 2010 Spring 2011 Fall 10 - Spring 11Average
72%
79%75%
79%82% 81%
62% 63% 62%
English Tutoring 1-2 sessions English Tutoring 3 or more sessions No English Tutoring
Case Study #2
Program Learning Outcomes &
Closing the Loop
Trait 1 Trait 2 Trait 3 Trait 4
Courses: Green = FoundationsBlue = First Year, no COEYellow = 2nd Sem, w/ COEBrown =Advanced / Final Course
TotalEnrollment
0-1 2 3 4% at
2 or <0-1 2 3 4
% at 2 or <
0-1 2 3 4% at 2 or <
0-1 2 3 4% at 2 or <
Total at 2 or
Higher
Below
Target
ART-121: 3-D Foundations 26 2 1 13 10 92% 3 2 7 14 88% 7 4 7 8 73% 4 3 8 11 85% 85%
ART-137: Fund. of Drawing 23 2 1 7 13 91% 1 3 9 10 96% 8 5 4 6 65% 2 1 7 13 91% 86%ART-137: Fund. of Drawing 33 4 1 10 18 88% 1 1 15 16 97% 9 7 8 9 73% 3 4 11 15 91% 87%ART-127: Intro Painting 16 2 0 8 6 88% 1 2 8 5 94% 7 4 1 4 56% 2 1 7 6 88% 81% XART-127: Intro Painting 5 1 0 1 3 80% 0 0 3 2 100% 2 1 1 1 60% 1 0 1 3 80% 80% XART-147: Clay Sculpture 16 1 1 6 8 94% 1 1 5 9 94% 5 2 3 6 69% 2 1 4 9 88% 86%ART-154: Survey of Ceramics 10 2 1 2 5 80% 1 2 2 5 90% 4 1 1 4 60% 1 1 2 6 90% 80% XART-155A: Intro Ceramics 16 2 1 3 10 88% 1 1 6 8 94% 5 3 3 5 69% 1 2 4 9 94% 86%ART-155B: Intro Ceramics 10 1 1 4 4 90% 1 1 3 5 90% 5 2 1 2 50% 2 1 3 4 80% 78% XDART-120: Intro Digital Art 23 2 1 9 11 91% 1 2 6 14 96% 3 5 7 8 87% 2 1 14 6 91% 91%PHOT-120: Intro Photo 25 3 1 13 8 88% 1 1 11 12 96% 4 4 7 10 84% 1 1 13 10 96% 91%PHOT-120: Intro Photo 22 2 0 14 6 91% 0 1 9 12 100% 5 3 6 8 77% 1 1 11 9 95% 91%PHOT-120: Intro Photo 26 2 1 7 16 92% 2 1 13 10 92% 4 3 11 8 85% 0 1 9 16 100% 92%PHOT-120: Intro Photo 24 2 1 10 11 92% 1 1 8 14 96% 2 2 13 7 92% 2 1 15 6 92% 93%PHOT-120: Intro Photo 30 3 1 11 15 90% 1 2 17 10 97% 4 3 10 13 87% 3 1 12 14 90% 91%PHOT-150: Intro Dig Photo 17 2 1 7 7 88% 1 1 9 6 94% 2 3 8 4 88% 2 2 5 8 88% 90%ART-128: Intermediate Paint 12 1 0 6 5 92% 1 1 4 6 92% 3 2 4 3 75% 1 0 4 7 92% 88%ART-128: Intermediate Paint 12 1 1 4 6 92% 0 0 7 5 100% 4 3 3 2 67% 3 1 5 3 75% 83% XART-138: Creative Drawing 18 1 1 7 9 94% 1 1 7 9 94% 3 6 4 5 83% 2 1 9 6 89% 90%ART-180: Printmaking 13 1 0 3 9 92% 0 0 8 5 100% 3 2 5 3 77% 1 1 3 8 92% 90%ART-230: Figure Painting 18 0 0 7 11 100% 0 0 3 15 100% 2 1 9 6 89% 1 0 6 11 94% 96%ART-250: Ceramic Workshop 12 1 0 5 6 92% 0 0 0 12 100% 3 1 6 2 75% 1 0 4 7 92% 90%ART-255A: Inter Ceramics 2 0 0 0 2 100% 0 0 0 2 100% 1 0 0 1 50% 0 0 1 1 100% 88%ART-255B: Inter Ceramics 5 1 0 2 2 80% 0 0 1 4 100% 1 0 2 2 80% 0 1 2 2 100% 90%DART-140: Typography 27 2 0 13 12 93% 0 0 12 15 100% 4 2 12 9 85% 1 0 15 11 96% 94%PHOT-121: Basic Lab Photo 10 1 0 6 3 90% 0 0 2 8 100% 3 2 4 1 70% 0 1 7 2 100% 90%PHOT-122: Basic Lab Photo 11 1 0 3 7 91% 0 0 4 7 100% 2 1 3 5 82% 0 1 5 5 100% 93%PHOT-151: Int Digital Photo 7 0 0 3 4 100% 0 0 2 5 100% 1 0 4 2 86% 0 0 3 4 100% 96%ART-270: Pro.Practices 13 0 0 4 9 100% 0 0 1 12 100% 0 0 4 9 100% 0 0 2 11 100% 100%
TOTAL 482 43 15 188 236 91% 19 24 182 257 96% 106 72 151 153 78% 39 28 192 223 92% 89%
3 or < 424 3 or < 439 3 or < 304 3 or < 415
What does it all mean?
•Student scores across the program met and exceeded expectations, but Vocabulary trait score significantly lower.
• Scores increased with course level, but not by a statistically significant amount.
• Lowest scores clumped around 1st year courses without Conditions on Enrollment.
•Results did not tell us about student learning accumulated through course sequencing in the program
Program Response
•Discussed and analyzed how vocabulary is handled by each instructor, looking for effective practices
•Created formative evaluations covering vocabulary, including vocabulary quizzes and vocabulary sheets for specific courses and sections and made these available to all instructors
•Created department Primer on Critiques (Power Point and handout) that was made available to all instructors
•Developed list of program-wide essential vocabulary
•Revised CORs and syllabi to make essential vocabulary an explicit part of instruction in Foundations Courses: 2-D, 3-D Design, and Fundamentals of Drawing
•Reviewed COEs for courses above Foundations level and determined that some realignment of enforced sequencing through COEs was appropriate, pending review of current program degrees and development of AA-Ts
We re-assessed this PLO in Spring 2012.
The results showed improvement across all four
traitswith marked improvement on
the Vocabulary trait.
Case Study #3
Institutional Learning Outcomes &
Closing the Loop
One way…..
Data compilations
ILO Program Program
Communicate ideas clearly and concisely in written, oral, and other forms, using a variety of complementary media.
Acquire and critically evaluate information, solve complex problems, and make informed decisions.
Set goals and develop a plan to achieve those goals.
Another way
Follow the same pattern as you did for CLOs and PLOs
Create a new processes for ILOs
You want EVIDENCE??
Types of EVIDENCE??Updated CORsYearly budget requestProgram Review documentsComprehensive Assessment ReportClass schedulesCollege catalogMeeting minutesEmailsStudent work samplesAssessment samples
Ass
essm
ent
of
cour
ses
LOs
Examples from Monterey Peninsula College Documents
Executing the Plan
2010
2011
Strengthened connections between Program Reflectionsand Program ReviewInstituted time for Program Reflections at all Flex Day events
Equated Institutional SLOs with GEOs.
MPC
Strengthen connections between Program Reflections and Program Review
Adjust language in all resource allocation processes to reflect Program Reflections and/or student learning
And the new Challenge …….
Linking Assessment with Faculty Evaluations
Potential Benefit of Closing the Loop
Consciously Incompetent
Unconsciously incompetent
Consciously Incompetent
✖
Unconsciously Competent
Unconsciously incompetent
Consciously Incompetent
Consciously Competent
Unconsciously Competent
Unconsciously incompetent
Consciously Incompetent
REMEMBER!!
Differences among colleges is NATURAL, EXPECTED, and BENEFICIAL.
The very essence of education and a creative, productive, innovative populous is variance in ways of looking at issues and problems.
REMEMBER!!
Incorporate your dialog in already existing college forums.
OR
Create new forums for dialog as necessary
AND
Eliminate forums/meetings that are not productive
REMEMBER!!
Don’t let the desire to “do it right” make the process foreign to your campus.
Questions??
Thank You!