Improving  Course-­‐Embedded  Assessment  through  Faculty  Development  

Mary  Kay  Jordan-­‐Fleming,  Ph.D.  Professor  of  Psychology  

Coordinator,  Academic  Assessment    

James  H.  Bodle,  Ph.D.  Professor  of  Psychology  

Director,  Center  for  Teaching  and  Learning  Excellence  

Mount St. Joseph University

Our  Focus:  •  Benefits  and  challenges  of  course-­‐embedded  (C-­‐E)  assessment  

•  CriCcal  importance  of  alignment  between  assignments  and  rubrics  

•  Process  for  insuring  alignment  •  Faculty  skill  in  aligning  assignments  to  rubrics:  Research  on  experts  and  novices  

•  Role  of  faculty  development  in  success  of  C-­‐E  assessment  

Mount St. Joseph University

Course-­‐Embedded  Assessment  •  Faculty-­‐designed  and  controlled  • Meaningful  (graded)  •  Customizable  •  Low-­‐cost  •  NaConal  benchmarking:    AACU  VALUE  project,  Lumina  FoundaCon  DQP  

 

MSJU  adopted  course-­‐embedding  in  2004  

Mount St. Joseph University

% Demonstrating Interdisciplinarity By (Mid-Level) Course

0

10

20

30

40

50

60

70

80

Course 1 Course 2 Course 3 Course 4 Course 5 Course 6

Mount St. Joseph University

Backward  Design  of  Core  Curriculum  •  ArCculate  learning  outcomes  of  undergraduate  Core  

•  Require  courses  that  teach  those  competencies  •  Assign  assessment  “arCfacts”  to  required  courses  

ArIfact  Map  (excerpt)  

Mount St. Joseph University

Alignment  of  Assignments  to  Rubrics  •  PublicaCon  of  MSJU  Assignment  Guidelines  •  Requirement  of  dual-­‐approval  for  Core  courses  

Dual Approval Process

Curriculum Committee:

course approval

Assessment Coordinator:

artifact approval

Mount St. Joseph University

ASSIGNMENT:    ReflecCve  Paper  (handout)  • Social  Responsibility  4  •  IntegraCve  Learning  1  •  IntegraCve  Learning  3  

1  1.2  1.4  1.6  1.8  2  

2.2  2.4  2.6  2.8  3  

COMM   CR  THINK   CULT   ETHICS   S  RESP  

FORMATIVE  avg  raCng  SUMMATIVE  avg  raCng  

Core  Curriculum  transiIon  2013-­‐14  FormaIve  =  approved            SummaIve  =  unapproved  

Mount St. Joseph University

So,  if  faculty  are:    experts  in  their  fields    creaCve  in  designing  assignments,  and    proficient  at  grading,  ...  

   

WHY  DO  THEY  FAIL  AT  ALIGNING  ASSIGNMENTS  TO  ASSESSMENT  RUBRICS?  

   

“Expert  blind  spot”  

Mount St. Joseph University

On  experIse…  

“There  are  known  knowns…  There  are  known  unknowns…    But  there  are  also  unknown  unknowns.  There  are  things  we  don't  know  we  don't  know.”

Mount St. Joseph University

Expert  Blind  Spot  Skills  and  steps  taken  for  granted  by  experts  are  not  obvious  to  a  naïve  reader.    For  example:      

“Sauté until the sauce is done. Cook until the sauce is a good consistency. Add spices to taste.”

   The  Expert  Blind  Spot  limits  experts’  ability  to  write  clear  instrucCons  for  novices.   Nickerson, 1999

Mount St. Joseph University

Improving  Alignment  to  Rubrics  •  Give  clear  guidelines  for  wriCng  assignments  Ewell,  2013:    A  good  assignment  should  iden0fy  students’  central  task,  how  to  undertake  it  and  communicate  results,  and  how  extensive/eviden0al  response  should  be.  

•  Give  examples  of  assignments  that  align  to  rubrics  •  Provide  construcCve  criCque  of  assignments  •  Include  faculty  in  the  assessment  process  (raCngs,  criCque  and  feedback,  improvement)  

•  Create  opportuniCes  for  peer-­‐to-­‐peer  faculty  development    

Mount St. Joseph University

Lessons  Learned  •  Expert  Blind  Spot  may  compromise  alignment  of  assignments  with  rubrics,  more  acutely  for  content-­‐  than  process-­‐oriented  outcomes.  

•  Third-­‐party  approval  improves  alignment  with  rubrics  and  validity  of  data.  

•  Course-­‐embedded  assessment  requires    seamless  integraCon  of  curriculum,  assessment,  and  faculty  development.