Writing, Reflection, and Writing, Reflection, and Assessment with Assessment with

ePortfoliosePortfolios

Christy Desmet

Director of First-year Composition

Presented to

2007 Academic Affairs Faculty Symposium

Unicoi State Park, Helen GA

March 31, 2007

Specific Recommendations on Specific Recommendations on “Effective Writing”“Effective Writing”

Specific Recommendations on Specific Recommendations on “Effective Writing”“Effective Writing”

Writing Task Force 2006-2007Writing Task Force 2006-2007

• Fran Teague and Nelson Hilton, Co-Chairs• Recommendations

Writing Task Force 2006-2007Writing Task Force 2006-2007

• Fran Teague and Nelson Hilton, Co-Chairs• Recommendations

Why ePortfolios? What is an Why ePortfolios? What is an ePortfolio?ePortfolio?

• A place to – Collect– Reflect on– Display or publish

multimodal writing

What is an ePortfolio?What is an ePortfolio?

• An opportunity to– Record and reflect on growth– Demonstrate discipline-

specific writing skills and genres– Make connections

(e.g., between coursesor between academics and life)

What is an ePortfolio?What is an ePortfolio?

• An electronic archive that supports– Student ownership– Sharing of writing– Program assessment– Research into the writing process

482,674 documents on March 16, 2007

Evaluation with <emma>Evaluation with <emma>

ePortfolios provide the perfect feedback system:

1. Shared rubric or vocabulary/set of assessment criteria2. Text3. Peer review4. Revision5. Assessment6. Publication in the ePortfolio

Revision and Peer ReviewRevision and Peer Review

• Markup/Revision– Markup makes rhetorical

choices explicit;– Web displays help

students “see” their texts with new eyes;

– Ease of download/upload permits multiple revisions.

• Peer Review– Makes students

intelligent critics of their own and others’ writing;

– Provides students with multiple, authentic audiences;

– Fosters collaboration and responsibility;

It all begins with shared It all begins with shared criteria.criteria.

FYC Program establishes a rubric, which establishes criteria and levels for assessment.

Writers then . . . write.Writers then . . . write.

Writers produce their documents in the OpenOffice word processor.

Peers provide feedback.Peers provide feedback.

Revision focuses on specific Revision focuses on specific rhetorical elements.rhetorical elements.

Writers revise. Teachers comment Writers revise. Teachers comment and assess.and assess.

Students reflect, revise, publish.Students reflect, revise, publish.

ePorts travel with students across ePorts travel with students across the curriculumthe curriculum

People

ePorts

Documents

Assessment and ResearchAssessment and Research

• Files uploaded in the last 7 days: 3663

• Files uploaded in the last 30 days: 14484

• Files uploaded: 489646

Error AnalysisError Analysis

In a preliminary investigation of errors marked in <emma> from Spring 2005, we looked at 748 essays from 10 sections with different instructors:

140Tense

169Paragraph coherence

186Apostrophe

217Wordy

244Other punctuation

244Documentation

294Spelling

307Awkward

368Diction

425Development

643Comma errors

748Number of documents

25Logical fallacy

29Organization

43Wrong preposition

48Passive voice

63Fragment

67Agreement subject-verb

69Vague pronoun reference

70Paragraph unity

72Coherence

78Agreement pronoun-antecedent

82Expletive construction

104Comma splice

117Transition

Sources and CitationsSources and Citations

Revision StudyRevision Study

• Does revision improve writing (as measured by students' scores on essays)?

• Research Method– 450 graded essays submitted for a grade (before);– Select 450 portfolio version of the same essays (after);– Essays graded online and randomly by trained holistic

raters;– 6-point rating scale calibrated to the program grading rubric.

Improvement in ScoresImprovement in Scores

0.46

0.28

0.26

Improvement in Scores

+1 pt. or more

+/- 0 pts.

-1 pt. or more

Cycle CompletedCycle Completed

Pedagogy

Assessment

Research