Overview

Portfolio Basics• Portfolio purposes• Learning objectives to be addressed• Roles of student and faculty in portfolio development, implementation and use• Institutional strategies for data use

A case study Promise and Pitfalls Lessons learned

What is purpose of the portfolio?

What outcomes are desired from portfolio use?

• Clear purpose• Design of portfolio driven by its use and the knowledge and skills to be assessed

Growth model Showcase Hybrid

What are the MEASURABLE learning objectives?

Facilitates meaningful evaluation Provides a common language Use tools to facilitate the process Involve your key constituents Communicate with students

Program Learning Outcome:

“Students will demonstrate effective oral

communication skills”

Performance Criteria: Acceptable Standard of

Performance Effective oral communication

Personal appearance is appropriateSpeaks clearly and with sufficient volumeAchieves rapport with audienceUses engaging vocalizationResponds effectively to questions and commentsUses audience-appropriate vocabulary, content, and style

Public Speaking Evaluation Sheet

Student: __________________________________ Date: ________Title of Presentation: _____________________________________

Evaluation Scale: Yes, a lot () 4 3 2 1 0 () No, not at all

Criteria Score Totals

Presentation Style:1. Personal appearance is appropriate _____2. Speaks clearly and with effective volume _____3. Achieves rapport with audience _____4. Uses engaging vocalization _____5. Responds effectively to questions and comments _____6. Uses audience-appropriate vocabulary, content, and style _____

Presentation style total: ______

Content:7. Uses the grammar of standard English _____8. Presentation includes introduction, body, and conclusion _____9. Organizes content logically and sequentially _____10. Presents ideas and arguments clearly and logically _____11. Uses appropriate audiovisual materials _____12. Cites sources appropriately _____

Student and Faculty roles in portfolio design and use

Be clear about how faculty are involved in the design and use of portfolio regardless of portfolio platform

• Meet their needs for information• Sensitivity to workload• Non-intrusive as possible

Involve students in meaningful ways• Be clear about what’s in it for students• Process should reinforce and be aligned with the educational process

Assessment

Development of scoring rubrics• Linked to performance criteria• Known to students• Scales consistent with purpose of assessment (i.e., student/program)

Involve students in meaningful ways

• Be clear about what’s in it for students

• Process should reinforce and be aligned with the educational process

What strategies for data use?

• Know what you are going to do with the data

Who is going to review/rate portfolio

contents? Who is going to be responsible for analysis of data? What mechanisms do you have in place to evaluate the data? How do the results relate to your educational delivery strategies – curricular and co-curricular?

Close the loop

● Importance of feedback that provides meaningful information● Target those who can do something about it● Focus on decision-making

• Improvement• Accountability• Celebration

● Inform those who have been involved about how results have been used● Communicate to stakeholders

AAHE Principles of Best Practice for Assessment of Student

Learning Use the “best practices” to guide development of assessment process Recognition of local constraints

INPUTS

OUTCOMES

Rose-HulmanInstitute

Of Technology

Terra Haute, Indiana

1650+ undergraduate students

B.S. degrees in engineering, science, and mathematics

85%+ engineering students

Rose-Hulman’s Mission

To provide students with the world’s best undergraduate education in engineering, science, and mathematics in an environment of individual attention and support.

Recruit highly qualified students, faculty, and staff

Input

Quality

Climate

Outcomes

Resources

Provide an excellent learning environment

Encourage the realization and recognition of the full potential of all campus community members

Instill in our graduates skills appropriate to their professions and life-long learning

Provide resource management & development that supports the academic mission

Instill in our graduates skills appropriate to their professions and life-long learning

Recognition of ethical and professional responsibility Understanding of contemporary issues Recognition of the role of professionals in the global society Understanding of diverse cultural and humanistic traditions Ability to work effectively in teams Ability to communicate effectively Apply the skills and knowledge necessary for mathematical, scientific, and engineering practices Ability to interpret graphical, numerical, and textual data An ability to design and conduct experiments An ability to design a product or process to satisfy a client’s needs subject to constraints

Outcomes

Assessment of student material

Faculty work in teams Each team assesses one learning objective Score holistically Emerging rubrics

• Does the reflective statement demonstrate or argue for the relevance of the submitted material to the criterion?• Is the submitted material at a level expected of a student who will graduate from Rose-Hulman?

Example of Results

1. Reflection relevant to criterion?

2. Expected for R-HIT graduate?

Communication Skills

0%

20%

40%

60%

80%

100%

1 2

Rater Question

Example of Results

1. Reflection relevant to criterion?

2. Expected for R-HIT graduate?

Communication Skills - Criterion 5Gramatically correct

0%

20%

40%

60%

80%

100%

1 2

Rater Question

Example of ResultsIs the submitted material at a level expected of a student who will graduate from Ross-Hulman?

1.Appropriate for audience

2.Organization3.Content

factually correct

4.Test audience response

5.Grammatically correct

Communication Skills -All Criteria

0%

20%

40%

60%

80%

100%

1 2 3 4 5

Rater Question

Linking results to Practice

Development of Curriculum MapLinking curriculum content/pedagogy to knowledge, practice and demonstrating learning outcomes

Show

Me!

Curriculum Map resultsFall 1999-2000 (181 courses/labs)

Communication Skills

86%

70%

7%

69%

0%

20%

40%

60%

80%

100%

Explicit Competence Feedback Not Covered

Curriculum Map resultsFall 1999-2000 (181 courses/labs)

Ethics

26%

17%

59%

29%

0%

20%

40%

60%

80%

Explicit Competence Feedback Not Covered

Closing the loop

DEC

JAN

FEB

MAR

APR

MAY

JUN

JUL

AUG

SEP

OCT

NOV

Eval committee receives and

evaluate all data; makes report and

refers recom-mendation to

appropriate areas.

Institute assessment center prepares

reports for submission to Dept

Heads of the collected data (e.g. surveys, portfolio

ratings)

Institute acts on the recommendations of

the Eval Comm.

Reports of actions taken by the institute

and the targeted areas are returned to the Eval Comm. for iterative evaluation.

Primary focus

It is not about electronic portfolios. It is about:

• teaching and learning• faculty and student development• support for the transformation of the teaching/learning environment

Implement Develop

Revise

Benefits to teaching

Faculty are asked to reflect on learning outcomes in relation to practice

• Consider the value of stated outcomes• Right ones?• Right performance criteria?• Individual faculty ownership in creating the context for learning

Develop a common language and understanding of program/institutional outcomes Promotes interdisciplinary discussions/ collaborations Explicit accountability for learning outcomes (e.g., state mandates/accreditation)

Benefits to learning

Students review their own progress as it related to expected learning. Portfolios provide a way for students to make learning visible and becomes the basis for conversations and other interactions among students and faculty. Learning is viewed as an integrated activity not isolated courses. Students learn to value the contributions of out-of-class experiences. Student reflections are metacognitive as they appraise their own ways of knowing. Promotes a sense of personal ownership over one’s accomplishments.

Students

Rewards Relevance Commitment

Lessons Learned

Assessment language – developing a discourse community Allocate resources Living, iterative process Focus is on improvement and celebration Start early Prepare for the need for “in-service” for faculty and students Decouple from faculty evaluation

DEMO Site

http://www/rose-hulman.edu/ira/reps

http://www.rose-hulman.edu/irpaAAHE/HLC

gloria.rogers@rose-hulman.edu