outcomes training for faculty spring 2014
DESCRIPTION
Slides from Faculty Training on OutcomesTRANSCRIPT
OUTCOMES IN CANVAS:
FACULTY TRAININGSpring II 2014
• Program Overview• Student Learning Assessment • What We Shared with Our Students• Assessment with Rubrics• Two Types of Evaluations• Outcomes in Canvas• Addressing Student Concerns• Faculty Resources• Questions
AGENDA
• Student Learning Assessment (SLA) is the process of determining
whether students are learning/mastering the Competencies and
achieving the Program Learning Outcomes (PLOs).
• Outcomes data is SLA data that is used for program evaluation
purposes.
• Online Programs have been collecting this data for two years. We are
sharing this process with students because our new LMS, Canvas, has
changed the availability of that data to include students.
• Outcomes in Canvas will eventually be in all Online Campus courses.
PROGRAM OVERVIEW
Program Courses in Spring II
MA I/O Psychology All
PhD/PsyD Business Psychology All but dissertation courses
PhD Organizational Leadership None this term (OL539 & OL556)
MA Psychology MP505 & MP520
MA Forensic Psychology None
PhD International Psychology None
MA Public Health None
BCBA None
PROGRAMS WITH OUTCOMES
• SLA is the process of determining whether students are learning/mastering the Competencies and achieving the Program Learning Outcomes (PLOs).
• SLA is more than whether the student passes a course or his/her grade in a course.
• Example PLO in Business Psychology: “The student is able to use SPSS and Microsoft Excel to analyze and present data to technical and non-technical audiences.”– How do we measure this? This is assessed in Dissertation I (B),
Dissertation II (I), Advanced Stats (A), Dissertation III (A), and Comprehensive Exams (A).
STUDENT LEARNING ASSESSMENT
STUDENT LEARNING ASSESSMENT
Overarching Goal• To ensure students
are learning what we promise they will learn
Academic ProgramReview Cycle
Action 11Institutional Effectiveness ReviewJune
Action IStudent Learning Assessment (SLA) Plan
Action 2SLA Data Collection
Action 3SLA Review July
Action 4Document Findings from SLA Review
Action 5Develop Program Profile
Action 6Program EffectivenessReview( PE) January
Action Planitems included in Budget Requests
March
Action 8Develop Action Plan Action 7
Campus Meta Review February
Action 9Cross-Campus Program SLA March
Action 10System-wide Meta May
STUDENT LEARNING ASSESSMENTInstitutional
Learning Goal
ProgramCompetency
ProgramCompetency
ProgramCompetency
Program LearningObjective
PB 611
Program LearningObjective
Program LearningObjective
Program LearningObjective
ProgramCompetency
Program LearningObjective
Program LearningObjective
PB 611
PB 612PB 612
PB 400 PB 455
PB 528
PB 530 PB 565
PB 566
Student Learning Assessment occurs at the PLO level.
• SLA data is used by faculty and administration to gauge the
effectiveness of the program and its curriculum.
• No matter what rating a student receives on an Outcomes
assignment, the primary purpose of Outcomes is for Academic
Program Review, not for individual student performance
assessment.
• The bottom line is that there are no negative consequences for
students with this new assessment tool.
WHAT WE SHARED WITH OUR STUDENTS
• There are no consequences for any score received for any of the PLOs in any course. Students’ grades on assignments and in the course are not affected by Outcomes ratings.
• The data is used to evaluate the Program and its curriculum, not as an individual performance assessment.
• The ratings shared with students are meant to be used for developmental purposes only.
• We encourage you to provide feedback to your students on their strengths and potential areas of improvement related to performance on PLOs.
WHAT THIS MEANS FOR OUR STUDENTS
• The Programs use a four-point rating scale: “1” = Novice, “2” = Internship, “3” = Entry-Level, and “4” = Graduate, such that higher ratings mean a demonstration of mastery on that PLO.
• Students are not expected to get a “4” on all Outcomes assessments. We expect that for most assignments, students will score in the 2-3 range.
ASSESSMENT WITH RUBRICS
• As instructors, you will be making two (2) types of assessments for an Outcomes assignment:1. First and foremost, you will
give the student a grade on the assignment, i.e., how well they met the criteria for the assignment.
2. Next, you will make an evaluation of how well their work matches the rubric for the Program Learning Outcome(s) attached to the assignment.
• These are two separate evaluations of the student’s performance.
• Note that one evaluation does not affect the other because you are evaluating different items:
– one is related to the course learning outcomes;
– the other is related to Program Learning Outcome mastery.
TWO TYPES OF EVALUATIONS
• You and your students will see the Outcomes rubric on the assignment page at the bottom.
• All Outcomes rubrics for that assignment will be listed with each performance level behavioral descriptor.
• Open SpeedGrader to make your ratings.
OUTCOMES IN CANVAS
• In SpeedGrader, after reviewing and grading the assignment (first evaluation), click on “View Rubric” to complete the Outcomes ratings.
• This is the second evaluation you will make for this assignment.
OUTCOMES IN CANVAS
• In the Rubric, move your mouse over the 4 performance levels to review the behavioral descriptors.
• Choose the rating that matches with the student’s performance on that assignment.
• Click the box to make the rating.
• Click “Save” to save your ratings and close the Rubric.
OUTCOMES IN CANVAS
• Each Outcomes assignment is not attached to a student’s grade. There are no points associated with Outcomes. No score will be visible in SpeedGrader.
• What you will see after saving the Rubric is only the behavioral descriptor for the performance level you selected.
• (Ex) If you chose “2” for a student, then you will see the behavioral descriptor for the performance level “2” after you close out of the Rubric.
OUTCOMES IN CANVAS
• This is what students will see in Grades when they click on the Rubric icon for an Outcomes assignment. Outcomes are indicated with the yellow “arrow” next to the criterion label.
• Note that these assessments do not contribute to their grade for that assignment (note the “--” indicated under “Pts.”).
OUTCOMES IN CANVAS
• Add an announcement about Outcomes for your students.• Discuss Outcomes at a Q&A session/conference call with your
students.• When making Outcomes ratings, if possible, provide feedback to
the student if you have suggestions on how they can improve their skill in a specific area measured by the PLOs.
• What do you do if a student has a question about Outcomes?– Refer to the FAQ document posted in each of your courses.– Refer any question you cannot answer to me or your Department
Chair.
ADDRESSING STUDENT CONCERNS
•What are Outcomes? http://guides.instructure.com/s/2204/m/4152/l/67950-what-are-outcomes•How do I grade student work with Rubrics (Outcomes) [make sure Outcome is not used for grading]? http://guides.instructure.com/s/2204/m/4152/l/39726-how-do-i-grade-student-work-with-rubrics•How do I open a Rubric (Outcomes) in SpeedGrader? http://guides.instructure.com/s/2204/m/4152/l/55022-how-do-i-open-a-rubric-in-speedgrader•How do I view Rubrics (Outcomes) results for an assignment? http://guides.instructure.com/s/2204/m/4212/l/67891-how-do-i-view-rubric-results-for-my-assignment
RESOURCES