Download - Using CAS Standards and Review Process for Improving Student Learning and Program Outcomes
Using CAS Standards and Review Process for Improving Student Learning and Program Outcomes
Patricia CarrettaGeorge Mason University
CAS Officer-at-LargeNACE Representative to CAS
NASPA 2008 International Assessment & Retention Conference
Purpose
CAS Standards and Review ProcessIntegrating CAS into assessment of
programs and services and student learning outcomes
Role and Use of CAS Standards:
1. Student Affairs / Student Services at the division level?
2. Student Affairs / Student Services at the departmental or program level?
3. Other?4. Have looked at the CAS Standards for at
least one functional area?5. Have participated in a department/program
review process using CAS standards?6. Have used CAS standards for other
purposes?
Participant Outcomes
Define the role of CAS self-assessment / review process within a comprehensive assessment plan
List the seven step process of conducting a CAS review Appreciate the value of engaging a variety of
institutional stakeholders in the review process Strategize how to conduct a comprehensive
assessment of programs, services and student learning using CAS Standards
Formulate an assessment plan Understand the importance of assessing and reporting
on input data as well as student learning and development outcomes as a strategies for improving learning and program quality
About CAS:
The mission of the Council for the Advancement of Standards in Higher Education (CAS) is to promote the improvement of programs and services to enhance the quality of student learning and development.
CAS is a consortium of professional associations who work collaboratively to develop and promulgate standards and guidelines and to encourage self-assessment.
CAS Resources
CAS Professional Standards, 2006 editionSelf-Assessment GuidesCAS Statement of Shared Ethical
PrinciplesCAS Characteristics of Individual
ExcellenceFrameworks for Assessing Learning and
Development Outcomes
CAS Guiding Principles
Students and Their InstitutionsDiversity and MulticulturalismOrganization, Leadership, and Human
ResourcesHealth Engendering EnvironmentsEthical Considerations
Organization of CAS Standards
Mission Program Leadership Human Resources Ethics Legal
Responsibilities Equity and Access Diversity
Organization and Management
Campus and External Relations
Financial Resources Technology Facilities and
Equipment Assessment and
Evaluation
Student Learning & Development
Must promote student learning and development outcomes
Must identify relevant and desirable student learning & development outcomes
Must assess outcomes Must provide evidence of their impact on
student learning and development Must articulate how they contribute to or
support student learning and development
CAS: Student Learning & Development Outcomes
Knowledge acquisition, construction, integration and application
Cognitive complexityIntrapersonal developmentInterpersonal competenceHumanitarianism & civic engagementPractical competence
Student Learning & Development Outcomes
Consistent with the institutional mission, programs and services must identify relevant and desirable student learning and development outcomes from among the six domains. When creating opportunities for student learning and development, programs and services must explore possibilities for collaboration with faculty members and other colleagues.
Programs and services must assess relevant and desirable student learning and development outcomes and provide evidence of their impact on student learning and development. Programs and services must articulate how they contribute to or support student learning and development in the domains not specifically assessed.
Essential Elementsin the CAS Approach
Culture that values involvement of all its members
Quality indicatorsUse of standards and guidelinesWillingness/capacity:
To examine itself and assemble resultsTo report and use the results
Uses of CAS Standards
Credibility, accountability, improvement: Program & service improvement; measures of quality
and effectiveness; measures of impact on learning Design of new programs & services Restructuring / reorganization Change in leadership Institutional self-studies Preparation for accreditation
Staff development Academic preparation
CAS: Standards and Guidelines
STANDARD (BOLD TYPE; AUXILARY VERB MUST)To support the institution’s mission and goals, Career services must provide faculty and staff and administrative units with information, guidance, and support on career development and employment issues and linkages with the broader community.
GUIDELINE (LIGHT-FACED TYPE; VERB SHOULD)Career Services should support faculty and staff and administrative units by: Identifying and disseminating information on employment
trends, top employing organizations, and co-op and internship sponsors.
Comprehensive Assessment Plan
Includes:Assessment of programs and servicesAssessment of student outcomes,
including student learning and development
Program Evaluation
Is the program or service functioning effectively to achieve its mission?
What evidence is available to support the determination?
How is evidence used to make program decisions?
Student Outcomes Assessment
What is the effect of our work on students?
How are they different as a result of interacting with our programs and services?
How do we demonstrate their learning?How do we know? Measure?
The CAS SAG: A Self-Assessment Guide
Translates CAS standards into an effective workbook / evaluative format
Promotes program self-assessment and development
Informs on program strengths and weaknesses
Supports professional staff development Leads to enhanced programs and services
that benefit student learning and development
CAS Self-Assessment Process
1. Establish and prepare the Self-Assessment Team
2. Initiate the Self Study3. Compile and review Documentary and
Evaluative Evidence4. Identify Discrepancies5. Determine appropriate Corrective Action6. Recommend Steps for Program
Enhancement7. Prepare an Action Plan
The Self-Assessment Team
Appoint and Train the Team: Size of team Composition: internal, external Coordinator/Leader Establish team ground rules Discuss meaning of each standard Establish team’s inter-rater reliability Encourage team discussion; expect
disagreements; commit to consensual resolution
Team Actions
Decide whether to include guidelines or other measures that go beyond the standards
Gather and analyze quantitative and qualitative data
Individuals rate each and every criterion measure
Obtain additional documentary evidence if required to make an informed team decision
Complete the assessment, ratings & action plan worksheets
Evaluative EvidenceSTUDENT RECRUITMENT MATERIALS:
Brochures & other program information Participation policies & procedures
PROGRAM DOCUMENTS: Mission statements; program purpose & philosophy statements Catalogs & related materials Staff & student manuals; policies & procedures statements
ADMINISTRATIVE DOCUMENTS: Organization charts; student & staff profiles Financial resource statements & budgets Annual reports
STAFF ACTIVITY REPORTS: Curriculum vitae & resumes; professional activity Service to other programs, departments, or community
STUDENT ACTIVITY REPORTS: Portfolios, developmental transcripts, resumes Reports of student service
RESEARCH & EVALUATION DATA: Needs assessments & self-studies Program evaluation; graduate & follow-up studies
Quantitative Data
Summary of objective responses on a questionnaire or program evaluation
Statistics about use by students and various population subgroups
Needs assessments, follow-up studies, and self-study reports
Institutional research reports and fact books Comparative or benchmark data at local,
regional or national level
Qualitative Data
Focus group informationWritten summaries of responses to
open-ended questions in interviews and on evaluations
Client satisfaction surveys, self-reports, and written comments, both solicited and unsolicited
Using the SAGs
SAGs offer a ready format for evaluationIn most instances, there are multiple
criterion statements for each standardEach criterion measure focuses on a
particular aspect of the standard, allowing raters to express more detailed and specific judgments
Criterion Measure Rating Scale
ND 1 2 3 4 NRNot Done Not met Minimally Well Fully Not Rated
Met Met Met
Using this scale, consider each criterion statement and decide…•The extent to which each criterion measure has been met by the program or service
ND 1 2 3 4 NRNot Done Not met Minimally Well Fully Not Rated
Met Met Met
Assessment Criteria ExamplePart 6: FINANCIAL RESOURCESCounseling Services (CS) must have adequate funding to accomplish its mission and goals. Funding priorities
must be determined within the context of the stated mission, goals, objectives, and comprehensive analysis of the needs and capabilities of students and the availability of internal or external resources.
CS must demonstrate fiscal responsibility and cost effectiveness consistent with institutional protocols.
ND 1 2 3 4 NR Not Done Not Met Minimally Met Well Met Fully Met Not Rated
PART 6. FINANCIAL RESOURCES (Criterion Measures) Rating Scale NOTES6.1 The program has adequate funding to accomplish its mission and goals. ND 1 2 3 4 NR
6.2 Funding priorities are determined within the context of program mission, student needs, and available fiscal resources. ND 1 2 3 4 NR
6.3 The program demonstrates fiscal responsibility and cost effectiveness consistent with institutional protocols. ND 1 2 3 4 NR
Part 6: Financial Resources Overview Questions
A. What is the funding strategy for the program?
B. What evidence exists to confirm fiscal responsibility and cost-effectiveness?
CAS: Work Form AAssessment, Ratings, and Significant Items
CASWork Form A
Assessment, Ratings, and Significant ItemsINSTRUCTIONS:This work form should be completed following individual ratings of the participants. For each of the 13 Parts, identify (circle) the criterion measure item number(s) in the column labeled for which there is a substantial rating discrepancy (two or more ratings apart). Items not circled should reflect consensus among judges that practice in that area is satisfactory. Items where judgment variance occurs need to be discussed thoroughly by team members. Follow this action by determining which practices (criterion measures) can be designated as “excellent” or “unsatisfactory” and record them in the Step One column. In Step Two, list the items requiring follow-up action including any criterion measure rated as being unsatisfactory by any reviewer.
Step One
Part Items Excellent Unsatisfactory
1. Mission 1.1a1.4
1.1b1.5
1.1c1.6
1.2 1.3
2. Program 2.1 2.4.3 2.4.8 2.4.132.6 2.8d2.11
2.2 2.4.4 2.4.9 2.4.142.7 2.8e2.12
2.32.4.52.4.102.4.152.8a2.8f2.13
2.4.12.4.62.4.112.4.162.8b2.9
2.4.22.4.72.4.122.52.8c2.10
CAS: Work Form A continued
Step Two: List item number(s) for each Part determined to merit follow-up and describe the practice weaknesses that require attention
1.
2.
3.
4.
5.
CAS Work Form B: Follow Up Actions
INSTRUCTIONS:The purpose of this work form is to begin the planning for action to be taken on practices judged to merit follow-up (See Step 1, Work Form A). In Step Three, transfer short descriptions of the practices requiring follow-up and detail these items using the table format provided.
Step Three: Describe the current practice that requires change and actions to initiate the change
Practice Description
Corrective Action Sought
Task Assigned To
Timeline Due Dates
CAS Work Form C:Summary Action Plan
Step Four: This form concludes the self-assessment process and calls for action to be taken as a consequence of study results. Write a brief action plan statement in the spaces below for each Part in which action is required.
Part 1: Mission
Part 2: Program
Part 3: Leadership
Part 4: Human Resources
Part 5: Ethics
Closing the loop
Purpose & scope of study is defined Self-study team is trained Self-study is conducted Evaluative evidence is collected Discrepancies are identified Appropriate actions are determined Special actions for program enhancement are
recommended Action plan is developed and communicated
Student Outcomes Assessment
What is the effect of our work on students?
How are they different as a result of interacting with our programs and services?
What have students learned?How have they developed?
Why Focus on Student Learning?
AccountabilityAlignment with institution’s missionConsistent with language of the
academyQuality improvementMore effective learning Better defined and more effective
planning and strategy development
Defining Learning Outcomes
“Outcomes usually identify growth in some dimension of knowing, being or doing.”
Susan Komives & Sarah Schoper, “Developing Learning Outcomes,” Learning Reconsidered 2 (2006)
Knowledge (cognitive)Attitude (affective)Skill (psychomotor)
CAS: Student Learning & Development Outcomes
Knowledge acquisition, construction, integration and application
Cognitive complexityIntrapersonal developmentInterpersonal competenceHumanitarianism & civic engagementPractical competence
Plan for Assessing Student Learning Outcomes
Review/revise mission Identify major services, programs, activities Specify desired student outcomes Identify information needed to show student
learning Identify existing and new data needed Develop process for measuring outcomes Identify ways to use findings to
change/improve
Writing Learning Outcomes
Identify words/phrases that describe what an ideal program participant or college graduate would be able to:Know – Be – Do … as a result of
engaging with your program, activity or service.
What would he/she learn from the experience?
Student Outcomes MatrixMission: The Multicultural Research and Resource Center (MRRC) consolidates and disseminates research on intercultural and cross-cultural inclusion, and collaborates with academicians to develop curriculum based in related theory. Additionally, the Center assists students with research and practice regarding multiculturalism in higher education; and administers an international model of leadership development training university-wide and in partnership with other entities in the regional, national and international community.
CAS Student Outcome Domain: Humanitarianism and Civic Engagement
Major Services, Programs, Activities
Dimensions of Outcome Domains:Understanding and appreciation of cultural and human differences
Social responsibility
Global Perspective
Sense of civic responsibility
•Classroom presentations•Curriculum/faculty development•Campus speakers•Applied research on issues affecting multiculturalism, community and handling controversial issues•Building Bridges partnership•NCBI training
Desired outcomes:Understands one’s own identity and culture; seeks involvement with people different from oneself; articulates the advantages and impact of a diverse society; identifies systematic barriers to equality and inclusiveness, then advocates and justifies means for dismantling them; in interactions with others, exhibits respect and preserves the dignity of others
Desired outcomes:Recognizes social systems and their influence on people; Appropriately challenges the unfair or uncivil behavior of individuals or groups; Participates in service/volunteer activities that achieve reciprocity; Articulates the values and principles involved in personal decision making; Affirms and values the worth of individuals and communities
Desired outcomes:Understands and analyzes the interconnectedness of societies worldwide; Demonstrates effective stewardship of human, economic, and environmental resources
Desired outcomes:Demonstrates consideration of the welfare of others in decision making; Critiques decisions and learns from mistakes; Engages in principled dissent; understands and participates in relevant governance systems
Writing Learning Outcomes
Describe the learner – characteristics of users/participants of programs and services. What knowledge, skills, experiences do they start with?
Determine acceptable levels of performance
Consider how you would provide evidence of achievement
Ways to measure learning outcomes
Surveys, questionnaires, self-report (indirect) Observations Interviews Focus groups Reflective journals Performance evaluations Pre-test/post-test Portfolios Capstone projects Review of project/performance by peers and
internal/external reviewers
Tips, suggestions
Identify all your program outcomes Rank outcomes based on importance
Activity outcome Program outcome Institutional or division outcome
Identify where learning opportunities are occurring (mapping)
Design and implement assessment cycle – one outcome at a time
Remember:
Start smallStart doableStart narrowStart targetedStart!
One program, one workshop, one desired outcome, one measured tool
Additional CAS Resources
www.cas.eduStandards book
Prologue & Context with “How to” information
FALDOs – book and CD formats as package
SAGs – CD or individually