Beliefs and Practices of EAP Instructors

Elana Spector-CohenTel Aviv University

Ruppin Academic Center

TBLT 2009

Lancaster UniversitySeptember 15, 2009

espector@post.tau.ac.ilespector@ruppin.ac.il

2

EAP: Historical Background, Israel

Course bibliographies in English; language of communication / tasks is Hebrew

Departmentally-dictated EAP course goals: reading comprehension only

Assessment: traditional reading comprehension tests Washback effect classroom teaching / materials Course materials isolated texts” plus “inauthentic

questions” no thematic link to simulate authentic academic reading

High-stakes final exam RC questions, often lower-order thinking skills, not part of thematic unit construct assessed?

Had the quest for ‘authenticity’ backfired?

3

EAP: Background

At the same time, developments in field: Content- and task-based teaching Alternatives of assessment, including task-

based / performance-based assessment Academic literacies World-Wide Web and global communications

Were we preparing our students for the demands of today’s global world?

4

Stage 1: Fall 2006Research Questions

What are instructors’ beliefs regarding goals and assessment in EAP courses?

Do instructors report that they are adopting methods of teaching and assessment that are aligned with integrative content-based teaching and alternatives of assessment?

Do they report that their practices are aligned with their beliefs? If not, why?

5

The StudyEAP instructors in the Division of Foreign Languages at

TAU and in the Center for English Studies at Ruppin Academic Center were asked to complete a questionnaire

Part 1: demographic survey Part 2: original questionnaire on assessment

beliefs and practices (50 questions on 5-point Likert scale)

Part 3: additional comments

6

Part 1: The Demographic Survey

TAURuppinNumber of respondents: 24

(31 staff members, including hourly staff )

77% response rate

Number of respondents: 16

)21 staff members, including hourly staff(

76% response rate

24 female 0 male 13 female 3 male

Native language: Bulgarian )1(, Dutch )1(, English )12(, Hebrew )1(, Russian )4(, Spanish )3(, Turkish )1(, Yiddish )1(

Native language: English )7(, Hebrew )8(, Swedish )1(

Years teaching EAP: ranged from 8 – 36 years; mean = 22 years

Years teaching EAP: ranged from 1/2 – 30 years; mean = 8.28 years

7

Part 2: The Questionnaire: Reading Only?

meanSD%

answered 1%

answered 2%

answered 3%

answered 4%

answered 5

The goal of EAP courses in my context should be reading only.

TAU2.54

1.50 41.67 12.5 4.17 33.33 8.33

RUPPIN

1.60.80 53.33 400 6.670

1 = Strongly disagree / Never 3 = No opinion/Not sure 5 = Strongly agree / Always

0

2

4

6

8

10

12

12345

Ruppin

TAU

8

Part 2: The Questionnaire: Reading Only?

meanSD% answered

1

% answered

2

% answered

3% answered

4% answered

5

Assessment in my context involves reading only.TAU3.17

1.55 25 12.5 8.33 29.17 25

Ruppin 1.94

1.03 43.75 31.25 12.50 12.50 0

1 = Strongly disagree / Never 3 = No opinion/Not sure 5 = Strongly agree / Always

0

1

2

3

4

5

6

7

8

12345

Ruppin

TA U

9

Course Goals: Conclusions

What are instructors’ beliefs regarding goals and assessment in EAP courses?

TAU: Beginning of a paradigm shift? Disagreement (polarization) regarding goals of

program / courses (reading only?) Move toward assessment that integrates four skills

OR still assessing reading only? Ruppin: More receptive to integrative approach?

“Please” the researcher? Exposed to more recent trends in language teaching?

10

The Questionnaire:

Assessment Modes - BeliefsmeanSD

% answered 1

% answered 2

% answered 3

% answered 4

% answered 5

Assessment should integrate language with performance on a "real-world" task.

TAU 4.29

.89 0 4.17 16.67 2554.17

Ruppin

4.38 .70 0 0 12.50 37.5050

Self-assessment and peer-assessment are valuable measures.

TAU3.5

1.29 8.33 16.67 20.83 25 29.17

Ruppin 4.06

.97 0 6.25 25 25 43.75

1 = Strongly disagree / Never 3 = No opinion/Not sure 5 = Strongly agree / Always

12

The Questionnaire:

Assessment Modes – Practices, con’t.I use performance-based assessment involving authentic academic tasks in my course.

meanSD%

answered 1

% answered

2

% answered

3

% answered

4

% answered

5

TAU 4.27

1.09 4.55 4.55 9.09 22.73 59.01

Ruppin 3.73

.7706.6726.67 53.33 13.33

1 = Strongly disagree / Never 3 = No opinion/Not sure 5 = Strongly agree / Always

0

2

4

6

8

10

12

14

12345

Ruppin

TAU

13

The Questionnaire:

Assessment Modes – Practices, con’t.

I regularly use assessment rubrics in my classroom.

meanSD%

answered 1

% answered

2

% answered

3

% answered

4

% answered

5

TAU 2.74

1.29 21.05 31.58 5.26 36.84 5.26

Ruppin 3.13

1.27 12.50 25 12.50 37.50 12.50

1 = Strongly disagree / Never 3 = No opinion/Not sure 5 = Strongly agree / Always

01234

5678

12345

Ruppin

TAU

14

Part 3: Additional Comments

“Some of the questions seemed irrelevant to our courses which do not stress or measure writing and speaking.”

“Some answers are determined by departmental … policy. There is some discrepancy in my answers due to this factor.”

15

Assessment: Conclusions Do instructors report that they are adopting

methods of teaching and assessment that are aligned with integrative content-based teaching and alternatives of assessment?

Beginning of paradigm shift? Many instructors felt that CBI/TBLT and PBA provide

valuable insights into students' abilities to use English in an academic setting

Instructors report using both traditional and alternative modes of assessment

Yet, do self-reports mirror what is really taking place in the classroom? (e.g. confusion regarding rubrics or common language regarding assessment terminology)

16

Model for EAP Courses Applied to Program at Ruppin Academic Center: 2006-2008

UNIT

CBI PBI

Collaborative Learning

(Kol, Schcolnik & Spector-Cohen, 2006)

Simulation of authentic activity

Extended content

Integration of the four skills

Multiple media / resources

Activation / acquisition of background knowledge

Branching / specialization

Integration /application

Sharing information

Multiple modes of assessment

17

18

Stage 2: Formative Evaluation of EAP Program: Spring 2008

“Don’t get too excited. Your idea never really gets off the

ground.”

Ruppin Academic Center:

EAP staff members who had taught in the 2007-2008 academic program post program change

n = 12 (academic program only)

response rate = 78%

19

Results of Formative Evaluation Questionnaire: Spring 2008 What changes, innovations, activities, etc.

that were introduced in the English department in the past year were most beneficial to you and/or your students and/or the department as a whole?

Please explain why. You may list as many items as you feel

are appropriate. Please also rank them if you can,

according to relative importance (from 1 = most beneficial).

20

Results of Formative Questionnaire

First place = 10 pointsSecond place = 8 points

Third place = 6 points Fourth place = 4 points

Fifth place = 2 points Sixth place = 0 points

21

Results of Formative Questionnaire

1. TBLT / PBI (84 pts)

2. Content-based teaching (80 pts)

3. Theoretical framework/clear syllabus/raising standards (30 pts)

3. Blended learning/variety of task modes (30 pts)

3. Official midterm test and fewer classroom tests (30 pts)

4. Empirical research article for teaching and testing (28 pts)

5. Teacher Cooperation (14 pts)

5. Administrative and pedagogical leadership of dept. head (14 pts)

5. Professional development of teachers (14 pts)

5. Improving testing/ the test-writing process (14 pts)

6. Use of library/staff to teach databases and to prepare for presentations (10 pts)

7. Committees (2 pts)

8. Fewer staff meetings (0 pts)

22

Results of Formative Questionnaire

What changes, innovations, activities, etc. that were introduced in the English department in the past year were least beneficial to you and/or your students and/or the department as a whole?

Please explain why. You may list as many items as you feel are appropriate.

Please also rank them if you can, according to relative importance (from 1 = least beneficial).

23

Results of Formative Questionnaire

1. Failed committees

(20 pts)

2. The Ruppin web site

(14 pts)

3. The place of proper linguistic subjects such as grammar and translation skills in EAP (10 pts)

3. CBI in engineering (10 pts)

3. Grades (find solutions to grade calculation on site / Excel)

(10 pts)

3. Integrating empirical research articles in business administration—boring and tiresome (10 pts)

3. No make up midterm exam (10 pts)

4. Accommodations: cassettes (8 pts)

24

Results of Formative Questionnaire

Do you have any recommendations for future directions for the department for next year?

25

Results of Formative Questionnaire: Recommendations (Excerpts)

Not to lower the standards

Cooperation with content teachers

Integrate authentic evaluation

Continue ongoing professional development

Develop more writing tasks and rubrics

26

Stage 3: 2008-2009

Intensive staff development on Moodle and blended learning

EAP staff pilots Moodle…

Based on success of pilot, decision made to offer Moodle to all instructors

Thanks for listening!

28

Reference

Kol, S., Schcolnik, M. & Spector-Cohen, E. (February 2006). Making connections: Content- and task-based EAP. Unpublished paper presented at UTELI (University Teachers of English Language in Israel) Conference, Ben-Gurion University of the Negev, Beer Sheva, Israel.