Connecting Student Learning Connecting Student Learning Outcomes to Teaching, Outcomes to Teaching,

Assessment, CurriculumAssessment, Curriculum

Shirley Ching, Judy Donnelly, Masako Kanai-Pak, Judeen Schulte, Fionca Tse, Cynthia Wu

June 21 and 22, 2007

Workshop OutcomesWorkshop Outcomes

Apply ability-based outcome principles to specific programs and clinical areas

Complete a process of designing outcomes for institutions, programs, and courses

Explore how outcomes could be reflected in courses or assessments

EXPECTED EXPECTED OUTCOMESOUTCOMES

TEACHING

ASSESSMENT

STUDENT LEARNING

INSTITUTIONAL/

PROGRAM ASSESSMENT

INSTITUTION

PROGRAM

COURSE

MISSIONMISSION

EDUCATIONALEDUCATIONAL PRINCIPLESPRINCIPLES

OutcomesOutcomes

A set of statements that describe what a student will be able to do with what he or she knows as a result of a set of learning experiences.

Alverno College Faculty (1994). Student Assessment-as-Learning at Alverno College. Milwaukee, WI: Alverno Institute.

The general statements that refer to characteristics graduates must acquire by the end of the program.Billings, D. M. & Halstead, J. A. (1998). Teaching in nursing: A guide for faculty.

Philadelphia: W. B. Saunders Company (p. 182).

Behavioral Objectives as related to Behavioral Objectives as related to OutcomesOutcomes

There is a distinct difference between behavioral objectives and outcomes from a pedagogical aspect. Faculty take the responsibility for shaping the learning environment so students can meet stated behavioral objectives. Billings, D. M. & Halstead, J. A. (1998). Teaching in nursing: A guide for faculty.

Philadelphia: W. B. Saunders Company (p. 148).

Behavioral Objectives as related to Behavioral Objectives as related to Outcomes, continuedOutcomes, continued

An outcome orientation suggests that the learning process is dynamic not static, that students and faculty share the responsibility for learning, that the faculty role changes from a person who imparts knowledge to one who facilitates learning, and that the faculty and students have the ability for shaping the learning environment to best fit individual needs and learning styles.

Billings, D. M. & Halstead, J. A. (1998). Teaching in nursing: A guide for faculty. Philadelphia: W. B. Saunders Company (p. 148)

Functions of OutcomesFunctions of Outcomes

Provide direction for learning Provide continuity Assure accountability by making

teaching/learning public and explicit Integrate knowledge, skills, attitudes,

dispositions

Sources of OutcomesSources of Outcomes

Rethinking and Implementing Outcome Clarification,” Marlene J. Evans, Georgine Loacker, Ernest G. Palola, New

Directions for Experiential Learning, 12, 1981.

FOR LEARNER

InstitutionalCriteria

Program or Major Criteria

CourseCriteria

Assessment Criteria

Behaviors as Evidence

INSTITUTIONAL OUTCOMES

PROGRAM OR MAJOR OUTCOMES

COURSEOUTCOMES

ASSESSMENTOUTCOMES

MOSTGENERIC

MOSTSPECIFIC

CONTINUUM OF INCREASINGSPECIFICATION OF ABILITY IN CONTEXT

Toward outcomes: Toward outcomes: What will the student look like when she has completed the program? (abilities, knowledge)

Gain autonomy and independence (C & C T; LL) Able to make appropriate judgment on independent nursing

interventions (decision-making) (C & C T, CA) Work together with other health care team members,

including patients and families (C & C T; GO; NSR; EL) Effective communication/ good negotiation skills/conflict

management within team; language (BTL; EL, CA, LL) Advocacy for patients and families (NSR, EL, CA)

[Developing] Be aware of organizational systems and how to use it well

(NSR, EL) Sense of social and national responsibility (NSR) Attain a global and transcultural perspective; cultural

sensitivity to patients and their health care needs (GO, CA, BTL, LL)

Commitment to developing self and professional upgrading (LL, GO, EL)

Subject knowledge Ethical and moral and sensitive to patients (CA, NSR) All roundedness (ALL)

Critical and Creative Thinking (C & C T)

National and social responsibility (NSR)

Lifelong learning (LL)

Global outlook (GO)

Cultural appreciation (CA)

Entrepreneurship and leadership (EL)

Bi-Literacy and trilingual literacy (BTL)

Institutional & Program AlignmentInstitutional & Program Alignment

Hong Kong Poly Attributes

Critical and Creative Thinking (C & C T)

National and social responsibility (NSR)

Lifelong learning (LL)

Global outlook (GO) Cultural

appreciation (CA) Entrepreneurship

and leadership (EL) Bi-Literacy and

trilingual literacy (BTL)

? ADD Communication/Interaction?

All roundedness (ALL)CRITICAL AND CREATIVE THINKING Gain autonomy and independence (C & C T; LL) Able to make appropriate judgment on independent nursing

interventions (decision-making) (C & C T, CA) Work together with other health care team members, including

patients and families (C & C T; GO; NSR; EL)NATIONAL AND SOCIAL RESPONSIBILITY Advocacy for patients and families (NSR, EL, CA) [Developing] Be aware of organizational systems and how to use it well (NSR,

EL) Sense of social and national responsibility (NSR)LIFELONG LEARNING Commitment to developing self and professional upgrading (LL,

GO, EL)GLOBAL OUTLOOK Attain a global and transcultural perspective; cultural sensitivity to

patients and their health care needs (GO, CA, BTL, LL)CULTURAL APPRECIATION Ethical and moral and sensitive to patients (CA, NSR)BI-LITERACY AND TRILINGUAL LITERACY Effective communication/ good negotiation skills/conflict

management within team; language (BTL; EL, CA, LL) Subject knowledge

Course: Community Health Course: Community Health Nursing – third year: Nursing – third year: Toward Outcomes

Content/abilities Integrate principles and practices of primary health care to the

practice of CHN Recognize the presence of cultural health practices in community

care Examine the availability of social health resources for various

nursing interventions Differentiate health promotion strategies for healthy and at-risk

aggregates Justify health care needs for providing appropriate health

interventions in individual, family, and community level Utilize theories of health education and community strategies for

public health education Evaluate effectiveness of various communication models Relate issues of global health concerns and problems Identifying critical factors affecting local and global health care

system, settings and development

Criteria for outcome Criteria for outcome statementsstatements

Is the outcome multidimensional? Is the outcome something students can

use in personal and/or professional situations in life?

Is the outcome appropriate to the mission/ aims of your institution or program?

Does the outcome embody potential levels of development?

Ability as Outcome: Ability as Outcome: Descriptors of Global Outlook [GO] (Cultural Appreciation, C & CT)

Tell difference about PHC in different countries and different regions of countries

Recognize that different regions have different resources available, depending on political, social, economic, geographic elements and level of development

Appreciate the needs of people in different countries and across (cultural) communities, generations, faiths, gender, physical condition, sexual preference

Not ethnocentric or nationalistic Recognizes own beliefs about superiority of country Aware of own assumptions so can understand what

others think Apply meanings of Global Outlook to achieve something

Ability as Outcome: Ability as Outcome: Global Outlook [GO] (for ALL students (Institutional Criteria) [1-2 yr, Beginning; 2-3 yr, Intermediate, Developing; 3-4 (Graduate, Advanced)

Tell difference about PHC in different countries and different regions of countries [Nursing] Recognize issues of local and global health concerns and problems

Recognize that different regions have different resources available, depending on political, social, economic, geographic elements and level of development of region [ALL] [Beginning]- Identify critical factors affecting local and GHC system, settings and development

Appreciate the needs of people in different countries and across (cultural) communities, generations, faiths, gender, physical condition, sexual preference [ALL] [Intermediate] Recognize issues of local and global health concerns and problems

Recognizes ethnocentrism or nationalism and how it affects relationships with others [ALL] [Beginning] Recognizes and identifies own beliefs about superiority of country [ALL]- Identify critical factors affecting local and GHC system, settings and development

Aware of own assumptions and values so can understand what others think [ALL] [Intermediate]- Identify critical factors affecting local and GHC system, settings and development

Apply meanings of Global Outlook to achieve something [ALL] [Advanced] Use relevant evidence to identify critical factors affecting local and GHC system,

settings and development

Criteria for outcome Criteria for outcome statementsstatements

Is the outcome multidimensional? Is the outcome something students can

use in personal and/or professional situations in life?

Is the outcome appropriate to the mission / aims of your institution or program?

Does the outcome embody potential levels of development?

Course: Community Health Nursing – 3rd year:Course: Community Health Nursing – 3rd year:Course Outcome: Course Outcome: Use relevant evidence to identify critical factors affecting local and global health care system, settings and development

Content/abilities Integrate principles and practices of primary health care to the practice

of CHN Recognize the presence of cultural health practices in community care Examine the availability of social health resources for various nursing

interventions Differentiate health promotion strategies for healthy and at-risk

aggregates Justify health care needs for providing appropriate health interventions in

individual, family, and community level Utilize theories of health education and communication strategies for

public health education Evaluate effectiveness of various community participation models Relate issues of global health concerns and problems Identifying critical factors affecting local and global health care system,

settings and development

Revised Course Outcome that Revised Course Outcome that integrates ability and Global integrates ability and Global Outlook AttributeOutlook Attribute

Use relevant evidence to identify critical factors affecting local and global health care system, settings and development

Framework for DesigningFramework for DesigningPerformance AssessmentsPerformance Assessments

Identify expected outcome(s) Generate list of generic criteria Design performance assessment

instrument or process Determine who will judge performance Determine feedback process Evaluate assessment instrument or

process

Operational Definition of Student Operational Definition of Student Assessment-as-LearningAssessment-as-Learning

Student Assessment-as-Learning: A process in operation at Alverno College, integral to learning, that involves observation and judgment of each student’s performance on the basis of explicit criteria, with resulting feedback to the student. It serves to confirm student achievement and provides feedback to the student for the improvement of learning and to the instructor for the improvement of teaching.

Alverno Student Assessment-as-LearningDesign and Implementation Process

OUTCOME

(ABILITY INTEGRATED WITH KNOWLEDGE)

Components(or GeneralCriteria)

PerformanceInstrumentor Process(Stimulus/Context)

Specific Criteria

StudentPerformance

Judgmentby Assessors(Including Self) Feedback Evaluation

Alverno College Faculty (1994). Student Assessment-as-Learning at Alverno College. Milwaukee, WI: Alverno Institute.

Ability as Outcome: Ability as Outcome: Global Outlook (for ALL students (Institutional Criteria) [1-2 yr, Beginning; 2-3 yr, Intermediate, Developing; 3-4 (Graduate, Advanced)

Tell difference about PHC in different countries and different regions of countries [Nursing] Recognize issues of local and global health concerns and problems

Recognize that different regions have different resources available, depending on political, social, economic, geographic elements and level of development of region [ALL] [Beginning]- Identify critical factors affecting local and GHC system, settings and development

Appreciate the needs of people in different countries and across (cultural) communities, generations, faiths, gender, physical condition, sexual preference [ALL] [Intermediate] Recognize issues of local and global health concerns and problems

Recognizes ethnocentrism or nationalism and how it affects relationships with others [ALL] [Beginning] Recognizes and identifies own beliefs about superiority of country [ALL]- Identify critical factors affecting local and GHC system, settings and development

Aware of own assumptions and values so can understand what others think [ALL] [Intermediate]- Identify critical factors affecting local and GHC system, settings and development

Apply meanings of Global Outlook to achieve something [ALL] [Advanced] Use relevant evidence to identify critical factors affecting local and GHC system,

settings and development

Description of Student: Description of Student: Level, Course, Previous Experience in Developing the Ability / Outcome:

Level: Third Year student Course: Community Health Nursing Previous Experience: Beginning, intermediate learning re: Global Outlook Attribute

Course Outcome: Course Outcome: Use relevant evidence to identify critical factors affecting local and GHC system, settings and development

Relevant Content/Experiences of Course: Input re: Role of health promotion Speakers on primary care Articles related to current health concerns, e.g., cancer, obesity

Performance Assessment Stimulus / Performance Assessment Stimulus / Instrument / Process Instrument / Process (Based on Student Context and Experience with Abilities: Input Mode; Response Mode; Student Role; Directions)

Project: Plan, execute, and evaluate a health promotion event

Course input: Role of health promotion; speakers on primary care; articles related to current health concerns, e.g., cancer, obesity

Input: Oral, written, aural, visual

Response: - Group work (interaction)- Mentor to help with plan (interaction)- Written paper- Oral Presentation

Student Role: Member of student group and bigger team, student community nutrition educator

Performance Assessment Performance Assessment DirectionsDirections

Written and oral directions to develop these Products and facilitate processes

- Group work with small group and then with mentors from institution

- Written report of planning, execution, evaluation plus self assessment (next time would keep ___ and change ___)

- Poster regarding content and also publicity- Presentation to publics – school,

employers, alums

Specific Criteria Related to Specific Criteria Related to PerformancePerformance Speaking criteria Elements to include in developing

criteria for assessment of postersAesthetic presentationAccurate informationDocumentation of sources, etc.

Self AssessmentSelf Assessment

Self Assessment is the ability of a student to observe, analyze, and judge her performance on the basis of criteria and determine how she can improve it. The process of self-observation leading to self-judgment leading to self-correction is a multi-dimensional process.

• Alverno College Faculty, 1994

Self Assessment ProcessSelf Assessment Process

Evaluate process and products and write about things would do would keep and change the next time asked to do similar performances

Assessment and Feedback Assessment and Feedback to Studentsto Students

Train alums on criteria and ask that they function as external assessors to provide written and/or oral feedback, based on criteria (as substitute for instructor feedback; faculty determines how assessment contributes toward course grade and course success)

Discuss criteria with students and ask that they pair up to provide feedback (may be written or oral)