implementing a modified portfolio assessment approach in esl

Implementing a “Modified” Portfolio Assessment Approach in ESL Writing

Courses

David BrunoAssistant Professor II & Chairperson

ESL DepartmentCamden County College

1996: Academic Skills Writing Department pilots portfolios as final assessment in the last Writing course before ENG-101

2009 – Academic Skills Writing Department developed following “Position Statement on Writing”

“One piece of writing—even if it is generated under the most desirable conditions—can never serve as an indicator of overall writing ability, particularly for high-stakes decisions. Ideally, writing ability must be assessed by more than one piece of writing, in more than one genre, written on different occasions, for different audiences, and responded to and evaluated by multiple readers as part of a substantial and sustained writing process.”

2013 – Academic Writing and ESL Department Co-Presented at NJCCC Best Practices Conference

Portfolio Background at CCC

Prior to 2006Individual instructors

responsible for creating own assessment

measures and final exams

2006Department

creates standardized final

exams for most ESL courses,

including writing courses

2008Department

implements a holistically

graded “timed essay” final exam

for Level 5 writing course - similar to ENG-

101 final assessment

2010Department

expands “timed essay” final exam to Level 4 writing

course

2013 to PresentDepartment implements “portfolio

assessment” approach in all writing courses

ESL Department Writing Course Assessment Timeline

Final exam administered 2 weeks prior to end of semester Graded holistically by ESL Department faculty on 1 – 6 scale by two graders Students must pass “ESL Departmental Exam” in order to pass the course.

If passing, score counts as 30% of final grade (holistic score is converted) If failing, students automatically fail the course

Students offered retest within 2 weeks. Matches current English Composition I final exam method in order to better

prepare students Drawbacks to “ESL Departmental Exam”

No ongoing ESL Department involvement in the learning process One essay was not enough to properly assess the students’ writing ability All Student Learning Outcomes not being assessed in the exam. Students who are not ready for the next level are still being passed in both

lower levels and in courses with “Departmental Exam”

“Timed Essay” Final Exams

Increased uniform approach across sections Increased departmental role in sections taught by

adjuncts Students received a “realistic” assessment mid-

semester in time to make adjustments Increased focus on sentence structure skills – one

complaint of English Department Reduce the number of students who are passed on to

the next level even though they are not ready

Benefits of Portfolio Assessment

Full-time faculty “mentors” are partnered with 5+ adjunct faculty teaching a combined 10+ sections of writing.

Mentors collect and review first-day handouts and 1st day writing samples for verification purposes.

Partners communicate via email or in person about academic concerns, process questions, classroom management, essay topics, teaching tips, and grading.

All partners have at least three face-to-face meetings per semester: review 1st day writing verifications, to review midterm portfolios and to review final portfolios.

Mentors review portfolios at midterm and end of semesters and provide

Portfolio Assessment Procedure

Midterm: Cover letter Revised essay with a draft MyWritingLab (online software) printout

Final: Cover letter Two revised essays (one must be peer edited and one teacher

edited) open-topic reading-based

Timed essay MyWritingLab printout

Academic Writing - Portfolio Contents

Beginning of Semester 1st Day Writing Verification (if completed)

Midterm Portfolio Review Writing Assignment for Chapter A – All drafts (including final draft graded with departmental

rubric) Writing Assignment for Chapter B – All drafts (including final draft graded with departmental

rubric) Timed Writing Assignment 1 – graded with departmental rubric. Initial Reflection Homework Assignment (not graded) Graded Quizzes from Chapters A & B

Final Portfolio Review All Midterm Portfolio documents Writing Assignment for Chapter C – All drafts (including final draft graded with departmental

rubric) Writing Assignment for Chapter D – All drafts (including final draft graded with departmental

rubric) Graded Quizzes from Chapters C and D Timed Writing Assignment 2 – graded with departmental rubric. Final Reflection Homework Assignment – not graded

ESL Department Portfolio Contents

1st Day Verification shows full spectrum of growth Standardized assignments/assessments

All sections have the same required assignments to complete and quizzes/tests to administer

All writing assignments are scored based on departmentally created rubric for each level

Quizzes included to demonstrate skill ability Burden on department, not student

Faculty maintains portfolio Incomplete portfolios based on students not completing assignments Copies of work provided for student review

Required assignments show all work completed, not just best Factors into final grade breakdown

“Modified” Approach to Portfolio Assessment

Standardized 1st day tests to determine appropriate level

Given in all courses (except Oral Communication*)

Writing classes – writing prompt FT faculty mentor evaluates writing sample to

determine level Students will be moved up levels – never

down!

1st Day Verification

Standardized Rubrics Used

ESL Writing V – Departmental Rubric: Spring 2015 Name: __________________________________ Assignment: ____________________________ Date: _________________________________ Content (15 points) Maximum Your Score

The essay fits the assignment. 5 The essay is interesting to read and fully developed. 5 The essay matches on the prewriting & outlining. 5

Total 15 Organization (35 Points)

The essay has an introduction, body, and conclusion. 8 The essay has a funnel or attention-getting introduction with the thesis

statement at the end, and the thesis statement is well written. (it is not too general; it is not just a simple announcement; it avoids stating an obvious fact)

7

Each body paragraph discusses only one main idea, and there are no sentences that are off topic.

10

Each paragraph achieves coherence by repeating key nouns, using transition signals to show relationships among ideas, and using transitions to link paragraphs.

5

The conclusion includes a summary of the main points or paraphrases the thesis statement.

5

Total 35 Grammar & Sentence Structure (40 points)

Verb tenses are used correctly. (form and choice of tense) 5 The sentences use the correct word forms. (nouns, pronouns, verbs,

adjectives & adverbs, singular vs. plural) 5

The sentences are all complete sentences (no fragments) and there are no fused sentences (run-ons, comma splices).

8

The sentences demonstrate subject-verb agreement. 4 There are a variety of sentence types: simple, compound, complex, and

compound-complex. 3

The sentences demonstrate English fluency including standard word order – generally subject-verb-object/complement.

4

Word choice is accurate and descriptive. 3 Phrasal verbs and prepositions are used correctly. 3 Noun determiners (articles, much, many, some, etc.) are used correctly. 3 Modals are used correctly (form & choice of modal). 2

Portfolio Assessment Sheets

ESL Department – Spring 2015 Midterm Portfolio Comment Sheet: Writing

Name: ___________________________________ Course Number: ESL-_____________

Writing Assignment #1 Comments:

Well written! Acceptable

Major errors in paragraph/essay structure Major errors in sentence structure Major errors in grammar/mechanics

Writing Assignment #2 Comments:



Timed Writing Assignment #1 Comments:



Quizzes Comments:

Great job! OK Needs improvement

GENERAL MIDTERM ADVISEMENT (this is NOT included in final grade) This student’s portfolio is:

Passing Failing

Comments (if needed):

All assignments/assessments are scored by the instructor – reviewed by the mentors Midterm and Final Portfolios are due on specific dates

Reviewed by FT Faculty Mentors Midterm Portfolio is for “advisement purposes only”

Allows failing & borderline students sufficient time to take additional steps to improve in time for the final portfolio

Final Portfolio scored as “Passing” or “Failing” If the student passes the Final Portfolio, the grades of the assignments included in the portfolio

will count as 40% of the final grade. The 40% is broken down into the final categories as such: 10% for chapter writing assignments 8% for timed writing #1 12% for timed writing #2 10% for the chapter quizzes.

All categories are averaged into the final grade separately. If the student fails the Final Portfolio, the student will receive 0 points for each of the

above categories in the final grade. As a result, the student will automatically fail the course.

Student AND/OR faculty as ability to challenge

Portfolio Reviews

ESL Department has established a standardized “final grade breakdown” for all courses in order to build a uniform approach to assessment

Students made aware on 1st day of class The final grade breakdown for Writing courses are:

10% - active in-class participation, homework and in-class work 8% - timed writing assignment 1 (included in portfolio assessment) 12% - timed writing assignment 2 (included in portfolio assessment) 10% - chapter writing assignments (included in portfolio assessment) 10% - quizzes (included in portfolio assessment) 20% - MyWritingLab work 30% - ESL Department exam

Addendum included with portfolio instructions and grading

Final Grade Breakdown

If a student/instructor would like to challenge the portfolio, he/she must follow these steps:

The faculty member must approve the challenge request. If the faculty member agrees with the failing portfolio grading, the student will not be permitted to challenge and the failing grade will stand.

If the faculty member agrees with the student that a challenge is warranted, another full-time faculty member (not the faculty member’s current mentor) will review the portfolio.

If a challenge is still warranted, the student will be given the opportunity to complete a timed writing assignment on campus prior to the start of the semester.

The ESL Department Chair will review the assignment and share it with the faculty member so that they can both make the decision together.

Portfolio Challenge Procedure

ADVANTAGES All faculty have one person to direct questions to for quick response

and discussion. For new faculty, mentors help to adjust individual grading ideas with

departmental standards. Midterm portfolios enable partners to review and discuss grading,

comments & feedback on writing, format/organization/grammar issues that appear throughout a section, indicating the need for additional teaching or lessons.

Exceptionally strong students can be identified for possible next level exemption.

Adjuncts feel connected to the department and are informed about policies and changes in a personal manner.

Portfolio Assessment for Mentoring

Student Learning Outcome Alignment with ESL-027

Upon completion of this course, the student will be able to: Demonstrate the ability to write multi-

paragraph essays using accurate, meaningful and appropriate sentences.

Demonstrate knowledge of the writing process

Demonstrate the ability to detect and analyze errors and use correct forms in writing

Demonstrate the ability to recognize and produce various sentence types

Demonstrate the ability to react in writing to a reading selection

Demonstrate the ability to produce grammatical structures relevant to ESL Writing V

Portfolios allow students to meet their SLOs:

Data shows the students are more successful in College-level courses

Revised work allows students who need time to revise show how they can use the writing process to reach their writing goals.

The timed essays show how students can edit on their own work independently. Revised essays allow more time to revision and editing.

The mix of reading-based and timed essays allow students to showcase their best work on meeting a variety of these goals.

The quizzes/tests include grammar-based questions

Passing Rate Per Course

Before Portfolios: 2010 - 2012

ESL-020: ESL Writing I 74% successfully completed

ESL-022: ESL Writing II 73% successfully completed

ESL-023: ESL Writing III 77% successfully completed

ESL-026: ESL Writing IV 80% successfully completed

ESL-027: ESL Writing V 82% successfully completed

Portfolios: 2013 - 2014 ESL-020: ESL Writing I

64% successfully completed ESL-022: ESL Writing II

63% successfully completed ESL-023: ESL Writing III

65% successfully completed ESL-026: ESL Writing IV

65% successfully completed ESL-027: ESL Writing V

68% successfully completed

Students are no longer being passed unless ready for next level

Passing Rate of ESL-027

Before Portfolios: 2010 - 2012

ESL-020: ESL Writing I 72% successfully completed ESL-027

ESL-022: ESL Writing II 73% successfully completed ESL-027

ESL-023: ESL Writing III 73% successfully completed ESL-027

ESL-026: ESL Writing IV 74% successfully completed ESL-027

Portfolios: 2013 - 2014 ESL-020: ESL Writing I

No data – students currently taking ESL-027

ESL-022: ESL Writing II 83% successfully completed ESL-027

ESL-023: ESL Writing III 79% successfully completed ESL-027

ESL-026: ESL Writing IV 80% successfully completed ESL-027

Students are better prepared for higher level ESL courses

Passing Rate of HIS-101 & PSY-101

Students who successfully completed ESL-026 from 2010 - 2012

75% successfully completed HIS-101

68% successfully completed PSY-101

Students who successfully completed ESL-026 from 2013 - 2014

80% successfully completed HIS-101

78% successfully completed PSY-101

Students are better prepared for College-level courses

Portfolio Assessment - Results

Pros Students are better prepared for

higher level ESL courses Students are better prepared for

College-level / Gen-Ed courses More uniform approach of

assessment Full snapshot of ability to assess Increased adjunct preparedness Better departmental communication Better data to analyze

Cons Enrollment

Students losing financial aid or F-1 status

Time/Effort to Assess Due Dates Data Analysis

Difficulties More needed ENG-101

Key to implementation and sustainability FT faculty compensation (release time/stipends)

Work goes above and beyond the normal workload Portfolio documents/requirements updated each semester

Time/effort of adjunct faculty Meeting with mentors periodically Preparing portfolios for review

Secretarial support Handling portfolio supplies Copying of necessary documents

Portfolio Storage (3 years minimum at CCC)

Institutional Support

Electronic Portfolios Online Courses Expand to Oral Communication courses for

“oral portfolios” Digital capturing of assignments and

presentations Attempted in 2013, but removed due to technical

issues

The Future of Portfolios at CCC

implementing a modified portfolio assessment approach in esl

Documents