nccea annual conference waynesville, nc assessment basics: implementing and sustaining a...

NCCEA Annual Conference WaynesvillNCCEA Annual Conference Waynesville, NCe, NC

Assessment Basics: Assessment Basics: Implementing and Sustaining a Comprehensive, Implementing and Sustaining a Comprehensive, Outcomes-Based Assessment PlanOutcomes-Based Assessment Plan

October 19, 2006October 19, 2006Melissa Canady Wargo, DirectorMelissa Canady Wargo, DirectorOffice of AssessmentOffice of AssessmentNCCEA Annual ConferenceNCCEA Annual Conference


Questions for today’s session:Questions for today’s session:

• What is assessment and why should What is assessment and why should we care about it?we care about it?

• How do I begin?How do I begin?• What do I need to be doing?What do I need to be doing?• How can I use assessment to improve How can I use assessment to improve

my program?my program?• Others??? Others???


Definition of AssessmentDefinition of Assessment

“ … any effort to gather, analyze, or interpret evidence, which describes institutional, departmental, or agency effectiveness.” (pp. 3-4) *

Schuh, J. and Upcraft, M. (2001). Assessment Practice in Student Affairs: An applications manual. San Francisco: Jossey-Bass.


Why Assess Programs & Services?Why Assess Programs & Services?

• To improve programs through informed decision-making

• To document the quality of service or extent of student learning for all program constituents (faculty, administration, students, alumni, accrediting agencies, public officials, etc.).


Types of AssessmentTypes of Assessment

• Needs Assessment

• Environmental Assessment

• Cost Assessment

• Outcomes Assessment


Why Outcomes-Based Why Outcomes-Based Assessment?Assessment?

• Shifts focus from services provided to results accomplished

• Defines expected end result, assessment of which allows for focused improvement

• Allows YOU to determine what constitutes an effective program


What is outcomes-based What is outcomes-based assessment?assessment?

Propose Improvements

to Program

ImplementChanges

FormulateOutcomes

Develop Means of

Assessment

Gather &Analyze Data

InterpretResults

Program, Division, University Mission,

Vision, Strategic Priorities


How do I go about assessing my How do I go about assessing my program?program?

• Engage all staff in assessment conversations

• Commit to support assessment activities

• Be willing to use assessment information to make programmatic changes


How do I begin?How do I begin?

Develop comprehensive assessment plan:• Define program purpose• Set program goals• Articulate intended outcomes• Identify assessment methods• Implement data collection/analysis strategies• Establish process to disseminate and act on

results


Formulating Program MissionFormulating Program Mission

3 Key Questions to Address in Purpose Statement:

•What is program?

•What does program provide?

•Whom does the program serve?


Sample Mission/Purpose StatementSample Mission/Purpose Statement

The mission of the Career Services office is to enhance the academic, career and personal development of students by helping them better understand themselves and their environments, and to gain the knowledge and skills necessary for academic success, wise career selection, and satisfying interpersonal relationships. To that end, Career Services will consult with University faculty and staff regarding student needs and organizational development issues; and collect and disseminate information about changing student needs and the effectiveness of our services.


Sample Mission/Purpose StatementSample Mission/Purpose Statement

University Career Services, an integral part of the educational process, assists students and alumni in assessing their career possibilities, setting their personal goals and achieving their objectives toward becoming productive citizens in the global community. While assisting its clients in identifying professional employment opportunities, University Career Services also provides the university community with insights into the ever-changing world of work to help develop realistic ways to better educate tomorrow’s leaders. (Texas Christian University)


Formulating Program GoalsFormulating Program Goals

• Focus on what the program provides to achieve the mission and outcomes.

• Keep goals brief and general

• Describe an ideal career/coop/internship program for your type of institution


Sample Program GoalsSample Program Goals

• To challenge and support students as they engage in self-exploration.

• To respond to the critical needs of the University community.

• To provide students with the information, experience and personal attention they require to make informed career choices commensurate with individual strengths and aspirations.


Formulating OutcomesFormulating Outcomes

Type of Outcomes:

• Program (Operational)

• Learning


Program OutcomeProgram Outcome

• Statement of what a program intends to do, achieve, or accomplish through certain activities or experiences; i.e., what a program accomplishes for its students, faculty/staff or institution.


Sample Program OutcomesSample Program Outcomes

• Placement Rate

• # of Coop/Internships

• Coop/Internship Participation Rates

• # of Employers at Job Fair


Learning OutcomeLearning Outcome

• Statement of what a constituent should know, think, be able to do, or value as a result of participation in/completion of your program.


Types of Learning Outcomes*Types of Learning Outcomes*• Complex Cognitive Skills• Knowledge acquisition• Intrapersonal development• Interpersonal development• Practical Competence• Civic Responsibility• Academic Achievement• Persistence

* Adapted from Upcraft, M. (1999). “Assessing Student Learning.” In Stage, F., Watson, L., & Terrell, M. (eds.), Enhancing Student Learning: Setting the campus context.


Learning Outcomes Most Applicable Learning Outcomes Most Applicable to Career/Coop/Internship Programsto Career/Coop/Internship Programs

• Intrapersonal Development: formation of autonomy, values, identity, self-esteem, maturity

• Interpersonal development: valuing diversity, establishing relationships, relating to others

• Practical Competence: career prep, time/money management, self-sufficiency


Formulating OutcomesFormulating Outcomes

Think SMART*

• Specific

• Measurable

• Attainable

• Results-Oriented

• Time-Bound

* Adapted from UCF Assessment Handbook


Sample Learning OutcomesSample Learning Outcomes

• Career counseling participants will be able to describe the steps involved in making career decisions.

• Describe at least three job search strategies based on their career objective.

• Explain to a potential employer how their background and skills match the employer’s needs.

• Co-op students will exhibit the maturity and professionalism commensurate with the work environment.


Group ExerciseGroup ExercisePart IPart I

Develop a learning outcome appropriate for a career/co-op/internship program:

•Think about skills/knowledge/satisfaction related to student development.

•Remember SMART


Formulating Assessment MethodsFormulating Assessment Methods

• What evidence will allow us to determine whether the outcome has been met?

• Are those evidence/data attainable?

• How often should they be collected?

• Who is responsible for collection, analysis, reporting?

• When will the collection, analysis, reporting occur?


Types of Assessment MeasuresTypes of Assessment Measures

• Direct:– An assessment method designed to directly measure

what a participant knows or is able to do (i.e., requires demonstration of the skill or knowledge). Most applicable to learning outcomes.

• Indirect:– An assessment method that indirectly measures what

a target subject knows, can do or values (i.e., what a subject perceives or is satisfied with). Most applicable to process/program outcomes.


Assessment MeasuresAssessment MeasuresEvaluation of:• Experiential activities (e.g. internships, voluntary

activities, etc.)• Student work samples (e.g., case study responses,

essay responses, etc.)• Collection of products demonstrating extent of learning

(portfolios)• Surveys (student, alumni, etc.)• Exit interviews• Employment rates• Retention/graduation data• Demographics of student population• Focus groups


Group ExerciseGroup ExercisePart 2Part 2

Develop two assessment methods appropriate to the outcome. Consider:

– What data are needed?– Where will data be collected? – Are there mechanisms in place already?– How often are data collected?– How will data be disseminated and

discussed?


Discuss your results!Discuss your results!

Act on them!Act on them!

Celebrate your improvements!Celebrate your improvements!


How can this process be managed How can this process be managed more effectively?more effectively?

• Capitalize on activities already in place

• Locate and use resources from around the University

• Use your time and resources efficiently


Questions/CommentsQuestions/Comments

??

nccea annual conference waynesville, nc assessment basics: implementing and sustaining a...

Documents

assessment information

assessment methods

assessment conversations

assessment practice

nc definition of assessment

nc assessment basics

comprehensive assessment

support assessment activities