unm assessment connections retreat dona ana … · plg and/or nmhed core area •verbal non-verbal...
TRANSCRIPT
UNM ASSESSMENT CONNECTIONS RETREAT
Susan Williams
Professor
Dona Ana Community College
OVERVIEW
Definitions
Differentiate between Program and Course Level Student Learning Outcomes
Differentiate between Course Level Student Learning Outcomes and Course Level Instructional Objectives
Differentiate between Program Assessment and Classroom Assessment
Selection of Assessment Methods
Measuring it
Responding to it
WHAT IS ASSESSMENT???
Evaluation and determination of the completion of key critical components.
WHAT DOES THAT MEAN???!!
HOW ABOUT…
Checking to see if they’ve learned what you want them to learn!
So…The first question is…
WHAT DO I WANT THEM TO LEARN?
(Besides EVERYTHING I’m teaching)
FIRST THINGS FIRST
What you do MATTERS!
To whom?
For what?
Why?
FLOW – MISSION STATEMENTS TO CLASSROOM ASSESSMENT
Start with your Institution’s Mission Statement.
Look at the Institutional Learning Goals.
Look at the Program Student Learning Outcomes.
Look at the Course Level Student Learning Outcomes.
What is the student learning in your class that supports those three things?
Pick one of those things to measure (student learning outcomes)
GOING WITH THE FLOW
Institutional Mission Statement
Institutional Student Learning Goals (K, S & R)
Program Mission Statement
Program Learning Goals
Program Student Learning Outcomes
Course Level Student Learning Outcomes
Course Level Instructional Objectives
PROGRAM LEARNING GOALS (PLGS)
The BIG picture (the finished
jigsaw puzzle)
“When the students have
completed (this course of
study), they will be prepared
to (fill in the blank)”
What are the CENTRAL
CLAIMS of the program
Each course contributes
something to reaching that
goal
PLGS
“Upon completion of this Program, student will be able to effectively communicate”
NOW…how do we assess that?
PROGRAM STUDENT LEARNING OUTCOMES (PSLOS)
Flow from the Program Learning Goals
Must be Measureable and Observable
Provides feedback to the faculty about how well they are meeting their Program Learning Goals
Focus on assessing what students are able to KNOW, THINK AND DO upon graduation from a program
“Upon completion of this program, the student will be able to demonstrate effective verbal communication.”
COURSE LEVEL STUDENT LEARNING OUTCOMES (CSLOS)
Flow from the Program Student Learning Outcomes• Things that you want the students to learn that support the Program Student Learning Outcomes
(PSLOs)
Must be Measurable and Observable
Focus on assessing what students are able to KNOW, THINK and DO upon completion of a course.
“By the end of the course, student will be able to effectively use verbal and nonverbal cues when presenting.”
Are they? How do you know? If not, what needs to change for me to achieve this outcome??
COURSE LEVEL STUDENT LEARNING OUTCOMES
• Communication
PLG and/or NMHED Core Area
• Verbal
Non-verbal
Gender Differences
Written
Outcomes• Volume/Pitch
Hands – Posture -Spatial
He said/She said
Spelling – Vocabulary -Grammar
Objectives
COURSE LEVEL INSTRUCTIONAL OBJECTIVES
Flow from the Course Level Student Learning Outcomes (or PSLOs)
Commonly included in syllabi
What is the student expected to demonstrate or perform at the end of a unit of instruction or class.
Must be Measurable and Observable
Helps faculty identify and utilize class-by-class instructions, practices and or assignments to measure or address Course Level Student Learning Outcomes (CSLOs)
“By the end of class on Week 2, student will be able to explain the importance of using volume and pitch when presenting”
SUMMARY
Goal = Students will be able to effectively communicate.
PSLO = Upon completion of this program, the student will be able to demonstrate effective verbal communication.
CSLO = By the end of the course, student will be able to effectively use verbal and nonverbal cues when presenting.
Course Level Instructional Objectives = By the end of class on Week 2, student will be able to explain the importance of using volume and pitch when presenting.
PROGRAM VS. COURSE LEVEL ASSESSMENT: ASSESSING PSLOS AND CSLOS
Program Student Learning Outcomes (PSLOs) Course Level Student Learning Outcomes (CSLOs)
• Measures designated PSLOs • Measures designated CSLOs
• Can be course-embedded • Examples of course level or classroom assessments:
Essays
Portfolios
Rubrics
Course assignments
Final exams
Labs Reports/Assignments
Measurable Classroom Assessment Techniques
(CATS)
• Examples of course-embedded assessments:
Essays
Portfolios
Rubrics
Course assignments
Final exams
Lab Reports/Assignments
Measurable Classroom Assessment Techniques (CATS)
• Examples on other assessments:
Standardized tests
Portfolios
Rubrics
Exit exams
FORMATIVE VS. SUMMATIVE
Formative Assessment
Assessment of student learning that provides information to a teacher, always for the purpose of improving instruction. It helps you formulate a plan of attack to better instruction.
Summative Assessment
Assessment of student learning that provides information about a student’s mastery of specified course or program objectives, often for the purposes of reporting. It summarizes.
DIRECT VS. INDIRECT
Direct Assessment
You can directly assess performance of the Learning Outcomes/Objectives: pre/post test; course-embedded questions; standardized exams; portfolio evaluation/rubrics; videotape/audiotape of performance; capstone course evaluation/rubrics.
Indirect Assessment
Assess opinions or thoughts about student knowledge, skills, attitudes, learning experiences, and perceptions. Examples of indirect measures are: student surveys about instruction; focus groups; alumni surveys; exit surveys; employer surveys.
CURIOSITY MAKES FOR GOOD ASSESSMENT
Change in Curriculum
What are you going to change?
Why are you doing it? (In order to ensure that students are aware of options in conflict resolution, I will…)
How will you measure it?
AND THEN??
Analyze Data
Did the students perform as expected?
Did you notice anything else about the data collected?
Respond
What are you going to do as a result of the analysis?
Make any changes
Make the changes and assess again next year to see if learning was substantially impacted
REMEMBER:
Institutional Mission to Institutional Learning Goals to PSLOs to CSLOs to Course level Instructional Objectives (by way of Classroom Assessment Techniques)
Don’t be afraid to ask the hard questions
Look at the data generated closely! You need good information to make decisions!
Change or modify what you do based on the results.
1
ACT CAAP (Collegiate Assessment of
Academic Proficiency)
Building a Successful Assessment Plan!
Sue Wheeler
ACT Manager, New Mexico
June 25-26, 2015
University of New Mexico
Assessment Connections Retreat
2
3
• Curriculum-based college
entrance exam
• Measures academic
achievement– English
– Math
– Reading
– Science
– Writing (optional)
• Only college entrance exam
based on number of correct
answers– No penalty for guessing
• High-stakes assessment
• New Mexico 2014 ACT High
School Tested Graduates =
12,945
The ACT Overview
5
ACT Collegiate Assessment of Academic Proficiency
(ACT CAAP) is the standardized, nationally normed
assessment program from ACT
that enables postsecondary
institutions to assess,
evaluate, and enhance
student learning and
general education
program outcomes.
ACT CAAP Overview
6
• Based on ACT research and development
• ACT CAAP User Norms
– Data from more than 250 institutions
– Calculated annually for each test
− Three-year rolling basis module
– Broken out by type of institution, year of students tested,
and institutional ownership
ACT CAAP Overview
7
• National, standardized postsecondary
assessment program
• Measures student
progress in the
acquisition of
core general
education skills
ACT CAAP Overview
8
• 6 independent test modules in subject areas most commonly
covered in postsecondary general education programs
– Reading, Writing Skills, Writing Essay, Mathematics, Science, and
Critical Thinking
• Convenient administration
– No national test dates
• Customizable test modules
– Up to 9 local questions can be added per module
• Scored by ACT
• Comprehensive reporting at the cohort and individual student level
ACT CAAP Features
9
ACT CAAP Reading Test• 36 items, 40 minutes
• Measures Reading Comprehension in two categories
– Referring Skills
– Reasoning Skills
• 4 passages, each with a set of multiple-choice questions
• Two subscores and a total test score
− Arts/Literature
• Prose Fiction and Humanities
− Social Studies/Sciences
• Social Studies and Natural Science
ACT CAAP Features
10
ACT CAAP Writing Skills Test• 72 items, 40 minutes
• Measures understanding of the conventions of standard written English
• 6 passages, each with a set of multiple-choice questions
• Two subscores and a total test score
− Usage/Mechanics
• Punctuation
• Grammar
• Sentence Structure
− Rhetorical Skills
• Strategy
• Organization
• Style
ACT CAAP Features
11
ACT CAAP Writing Essay Test
• 2 writing prompts, 20 minutes each
• Measures writing skills most commonly taught in college-level
writing courses and required in upper-division college courses
– Requires examinee to take a position on an issue and explain why the
position taken is the best alternative
• One reported score
– Average of scores on both essays
– Each essay scored by two raters (1–6 scale)
ACT CAAP Features
12
ACT CAAP Mathematics Test• 35 items, 40 minutes
• Multiple-choice questions that require the use of quantitative reasoning skills to solve practical problems
• Two subscores and a total test score
− Basic Algebra
• Pre-Algebra
• Elementary Algebra
• Intermediate Algebra
• Coordinate Geometry
− College Algebra
• College Algebra
• Trigonometry
ACT CAAP Features
13
ACT CAAP Science Test• 45 items, 40 minutes
• Measures skills in scientific reasoning
• 8 passages, each with a set of multiple-choice questions
• One reported score
– Data Representation
− Research Summaries
− Conflicting Viewpoints
ACT CAAP Features
14
ACT CAAP Critical Thinking Test• 32 items, 40 minutes
• Measures skills in clarifying, analyzing, evaluating, and extending arguments.
• 4 passages, each with a set of multiple-choice questions
• One reported score
– Analysis of elements of an argument
– Evaluation of an argument
– Extension of an argument
ACT CAAP Features
15
Reports
• ACT CAAP Standard Reporting Package
– Issued to the institution after each ACT CAAP test administration
• Institutional Summary Report
– Available for schools testing 25 or more students
• Student Score Reports
• Certificates of Achievement
• Student Roster Report
• ACT CAAP Linkage Report
– Links students' scores on the ACT® or ACT Compass® on entry to
college with their ACT CAAP scores after general education work is
completed
• ACT CAAP Content Analysis Reports
– Detailed data of students’ strengths and weaknesses in specific content
areas of each test module
ACT CAAP Features
16
Resources
• Student
– ACT CAAP Student
User Guide
– ACT CAAP
Score Report
Interpretive Guide
for Student
ACT CAAP Features
http://www.act.org/caap/resources/students.html
17
ACT CAAP can be used to:
• Satisfy accreditation and accountability reporting requirements
• Measure students' achievement levels on a group and individual
basis
• Compare students' achievement levels with national user norms
• Evaluate the strengths and weaknesses of general education
programs
• Document the performance gain of students' achievement levels
over time
• Analyze what interventions may be necessary to enhance results
• Determine student eligibility for upper-division studies
• Advise individual students how to achieve academic success
ACT CAAP Benefits
18
Resources
• Institution
– ACT CAAP Guide to Successful General Education Outcomes
Assessment
– ACT College Learning Outcomes Assessment Planning Guide
– ACT CAAP Technical Handbook
– ACT CAAP Score Report Interpretive Guide for Institutions
http://www.act.org/caap/order/reviewing.html
ACT CAAP Features
http://www.act.org/caap/resources/students.html
21
Sue Wheeler
ACT Manager, New Mexico
319 321-9708
Thank you for the opportunity to discuss
ACT’s CAAP program.
Questions