handbook of screening - university of massachusetts boston - a

Massachusetts Department of Elementary and Secondary Education ABE-DL Programs

HANDBOOK OF SCREENING

For Instructors and Administrators of Online ESOL Programs

June 30, 2009

Prepared By

Berna Kahraman, [email protected] Amelia Mallona, [email protected]

In Consultation with

Donna Haig Friedman Dragana Bolcic-Jankovic

With Assistance from

Yesenia De Los Santos

P h o n e : 6 1 7. 2 8 7 . 5 5 5 0 F a x : 6 1 7 . 2 8 7 . 5 5 4 4 w w w . m c c o r m a c k . u m b . e d u / csp

TABLE OF CONTENTS

INTRODUCTION .......................................................................................................................... 1

I. INITIAL TASKS FOR THE SCREENING PROCESS.............................................................. 1

II. BACKGROUND FOR THE INTERVIEW INSTRUMENT .................................................... 2

What is the Interview Instrument? .............................................................................................. 2

Purpose of the Interview Instrument: .......................................................................................... 2

III. GUIDELINES FOR ADMINISTERING THE INTERVIEW ................................................. 3

IV. INDICATORS OF READINESS ............................................................................................. 5

Section A: Past Schooling and Self-Study Experience .............................................................. 5

Section C: Goals and Motivation ............................................................................................... 6

Section D: Perceptions & Expectations ...................................................................................... 6

Section E: Strengths and Weaknesses........................................................................................ 7

Section F: Time-Management.................................................................................................... 7

Section G: Supports and Challenges .......................................................................................... 7

V. INTERVIEW INSTRUMENT ................................................................................................... 8

VI. APPENDIX ............................................................................................................................. 17

The Center for Social Policy McCormack Graduate School of Policy Studies UMass Boston June 30, 2009 1

INTRODUCTION

This handbook provides guidelines for administering the screening tools for applicants of online ESOL programs funded by the Massachusetts Department of Elementary and Secondary Education. It explains the screening process as well as the indicators for determining applicants’ readiness for online education. The handbook is organized in five sections:

• The first section contains the preliminary tasks for the screening process. • The second section presents the background information about the interview instrument

and its purpose. • The third provides information on how to administer the interview. • The fourth explains each of the indicators for determining applicants’ readiness for online

learning. • The last section contains the interview instrument.

The handbook also includes a demographic information form in the appendix.

It is important to read the whole handbook in order to have an overview of the screening process as well as an understanding of the indicators for readiness and the questions designed to gather information for each indicator.

I. INITIAL TASKS FOR THE SCREENING PROCESS The initial part of the screening will involve: English Level Assessment Computer Skills Assessment

If applicants meet the English and the computer skills requirements, then applicants will be screened to determine their readiness for online learning. This will be achieved through conducting an interview with the applicant using an interview instrument. The interview will last approximately 20-30 minutes. Prior to the interview, the interviewer should explain to applicants that there will be a 20-30 minute conversation about online learning, during which applicants can ask questions. The interviewers should also explain that the purpose of the interview is to determine together if online learning is the appropriate mode of learning for the applicant.


II. BACKGROUND FOR THE INTERVIEW INSTRUMENT

What is the Interview Instrument? The interview instrument is a qualitative and interactive tool which includes a set of questions for prospective students of adult basic education. It covers seven broad areas which could have a role in determining a person’s readiness for online learning. Purpose of the Interview Instrument: The interview instrument provides a means for having a guided and interactive conversation with prospective students. The intention of the interactive approach is to build a foundation of trust in order to increase prospective students’ confidence and ownership of their learning process. In this regard, the interview is not only a means for selecting the applicants who are appropriate for online learning, but also a process to support them as they engage in online learning. It seeks to accomplish the following:

a. To assess readiness of prospective students for online learning. b. To spark a process of reflection in prospective applicants about their goals, motivation,

expectations and the support they will need to succeed in online learning. Success and persistence in this mode of learning can be defined in many different ways. One of the most useful definitions of success is one that takes into regard the context of learning and the life circumstances of an adult learner. Success, in this context, has been defined as persistence in the ESOL program, which leads to increasing the English assessment scores for the ESOL applicants. Persistence is defined, along the lines of the other studies in this area, as continued attendance in programs and, when a need to stop out of the program arises, having a plan to return as soon as the reason to stop out has been eliminated. (Comings, Parrella, & Soricone, 1999). Following are the guidelines on how to administer the instrument.


III. GUIDELINES FOR ADMINISTERING THE INTERVIEW

The Interview Process

One of the most important aspects of the interview is that it should take place in a friendly and informal manner. The atmosphere should feel comfortable. Through this process the interviewer is getting to know the adult learner and the adult learner is getting to know the context of online learning. Hence this should become an exchange after which both parties can make more informed decisions. Most adult learners are new to this mode of learning. This is an opportunity to provide them with accurate information and clarify any misconceptions they might have.

Before the Interview The first step is to communicate the purpose of the interview process to the adult learner in order to build a foundation of trust. The following script is an example of how this can be done. “I am going to ask you a set of questions which will help us determine what type of ESOL program would be right for you and how the program can support you. These questions are about your prior schooling experience, why you want to improve your English, and how you plan to do this. Please be as open as possible as this will help us better serve you. During this process you will also learn about online learning and the ESOL program. You will have opportunities to ask question, too. This interview is going to last approximately 20-30 minutes.”

During the Interview The role of the interviewer is to:

• guide the applicants through the interview questions; • probe for more information where necessary; • present information about online learning and the program; • clarify any misconceptions.

The sequence in the protocol should be followed as there is a logic underlying the sequencing of the sections. However, deviations and improvisations might be necessary to focus back and achieve the purposes of the interview process. Interviewers can take notes during the process with the purpose of helping to prepare a better orientation and to identify any areas that the


applicant might need support with in the future. This information will be useful for the instructors.

A good interviewer has two qualities: being a good listener and a good facilitator. Below is a set of general guidelines for interviewing:

DOs… DON’Ts… ♦ Listen well and encourage applicants to expand

on significant ideas ♦ Force the applicant to give an answer if you

feel that he/she is not comfortable in responding

♦ Clarify misunderstandings or unrealistic expectations by providing useful information on online learning or about the ESOL program

♦ Impose your own personal values and ideas

♦ Provide an opportunity to ask questions ♦ Allow the applicant to go off topic by asking unrelated questions

♦ Pose the questions in your own words; add probes as necessary as long as you do not deviate from the main idea

♦ Ask the probe if the applicant already provided the information after the main question

♦ Ask the question; wait for an answer, then check the yes or no box

♦ Lead the applicant to give a yes or no answer by saying “Yes or No?”

♦ Guide the applicant if he/she is going off topic, by saying something like, “I am sorry to cut you off, but I still have more questions that we should cover. Can we go back to where we left off?”

♦ Allow the applicant to see this as a therapy session

Below is a set of guidelines for taking notes:

DOs… DON’Ts ♦ Record answers to the questions ♦ Try to take notes of everything the applicant

said ♦ Make use of lists if appropriate ♦ Try to write down full sentences ♦ Note down your perceptions, observations,

reflections and comments using brackets [ ] to distinguish them from what the applicant said

♦ Write down your own perceptions, observations reflections and comments in a way that cannot be distinguished from the applicants’ answer

♦ Write legibly because your notes can be used by other people, like the instructors or those who are preparing the orientation


After the Interview After the interview is complete, interviewers should encourage applicants to:

• think further about the information exchanged during the interview • think about their own readiness for online learning • come prepared for the orientation, where they will have more opportunities to ask

questions

The next part includes the background information which explains the purpose of each section of the interview instrument. The instrument is presented after that on page 8. IMPORTANT: Please read the background information for each section while referring to the corresponding questions in the interview instrument.

IV. INDICATORS OF READINESS

Section A: Past Schooling and Self-Study Experience The questions in this section gather background information about applicants’ past schooling experience in their home countries and in the US. Although, questions A1-A3 will be gathered through the SMARTT intake form it is necessary to collect this data during the screening process as well. An examination of ESOL applicants’ backgrounds has revealed that they have very diverse schooling experiences. Some will have education well beyond high school, while some will not have a high school diploma. Students may be better apt for studying on their own with limited support as their previous level of education increases; the literature suggests that students may acquire further learning strategies as they gain more education which will be useful for learning online (Kahraman, Mallona, Friedman, Kahan, Platt, 2008). Therefore ESOL applicants’ experiences in online learning will be very diverse. Questions on applicants’ previous experiences with self-study will help you to unearth any challenges that they might have had studying without a teacher in the past. The questions on self-study can also be used as an opportunity to reinforce some of the good strategies that applicants say they use when they study on their own.


Section B: Knowledge about Characteristics of Online Learning The purpose of this section is to provide basic information about characteristics of online learning and the differences and similarities between this type of learning, traditional mode of learning (classroom learning) and self-study. Previous research has shown that adult applicants of online programs have very little to no knowledge about online learning (Kahraman, Mallona, Friedman & Kuck Jalbert, 2009). The information which compares these different modes of learning is included as a table in the interview instrument on page 10. Interviewers should provide a copy of this table to applicants and discuss the information together. Section C: Goals and Motivation This section is intended to uncover the underlying forces of motivation. Adult education research indicates that having explicit goals can be a leading force in motivating adult learners and help them persist in distance education (Comings et al., 1999). Applicants should be encouraged to ask themselves “Why am I doing this?” early on in the process. The first question in this section intends to get at applicants’ life goals. The clearer the applicants are in defining their goals, the more likely they are to persist (Comings, et al., 1999). The second question asks applicants how long they are willing to commit to the program. This can be done by first stating how long the program actually takes and then asking applicants whether they can commit to the program that long. If for one reason or the other an applicant is not able to make that commitment, this is a good opportunity to plan together with the applicant on how the goal of learning English can be achieved. Section D: Perceptions & Expectations This section aims to present applicants with information on:

• specifics of the curriculum; • what can be achieved studying with this curriculum; • how the program and the curriculum is delivered ( i.e. whether through software or an

internet site).

This section also aims to: • provide applicants an opportunity to ask questions; • further encourage applicants to think whether they can benefit from this mode of

learning; • gain an understanding of applicants’ perceptions and expectations of the ESOL program

and online learning; • clarify any misconceptions they might have about the ESOL program and online

learning.

Having clear and realistic expectations on online learning and on the specifics of the program will help students persist (Schunk 1989). Previous research shows that many candidates of ESOL come to the program with the expectation of improving their oral communication skills


(Kahraman, Mallona, Friedman & Kuck Jalbert, 2009). It is important to inform the applicants how this goal can be achieved with this program and this specific curriculum. The information on the program and the curriculum could be provided in a simple format in a booklet or a handbook. It might also be useful to give a handout on examples from the courses and topics and include information on any additional resources that students can use. Section E: Strengths and Weaknesses The goals of this section are to have applicants discuss:

• their challenges in learning English; • their strengths and weaknesses; • what success means to them.

Other studies have illustrated that students who are confident in their abilities to accomplish the task and who have the ability to self-assess are more likely to persist (Bandura, 1993; Pintrich & Garcia, 1991; Schunk, 1991; Schraw and Dennison, 1994). Section F: Time-Management This section is intended to cover three major areas:

• how applicants plan to fit studying into their lives; • how much time is really available for studying; • whether applicants are able to complete tasks on time.

Successful distance learners are those who have the ability to organize their time in order to accomplish their goals (Schunk & Zimmerman, 1998; Artino 2008). The interviewer can start by telling applicants how much time on average per week is required to study in the program. The interviewer can then help applicants to create a customized time table. The table does not need to be detailed; it could include general time slots such as mornings, afternoons, or evenings. Section G: Supports and Challenges One of the purposes of this section is to inquire about applicants’ awareness of negative and positive forces in their lives, which may hinder or contribute to their persistence. Having such awareness has been presented as one of the factors that help students persist in adult basic education programs (Comings et al., 1999). This section is intended also to identify applicants’ willingness in communicating with instructors and their comfort with asking for help from instructors, peers and others. Willingness and initiative for seeking help is a crucial factor for student success (Comings et al., 1999).


V. INTERVIEW INSTRUMENT

DEPARTMENT OF ELEMENTARY AND SECONDARY

EDUCATION

Face to Face Interview Instrument for Screening Prospective ESOL Applicants for Online Education

SECTION A: PAST SCHOOLING AND SELF STUDY EXPERIENCE

The information gathered in this section will help interviewers in administering Section B. Section B is related to characteristics of online learning. Interviewers can use the information gathered in Section A to explain how online learning is similar to or different from self-study. The responses to questions about self-study could also be used as a basis to point out learning strategies applicants have used and would work well for learning online.

Questions: Schooling

Answers/Observations

A1. Did you finish High School in your country?

[ ] YES LEVEL____________ [ ] NO (IF NO SKIP TO A3)

A2. Did you go to College in your country?

[ ] YES LEVEL____________ [ ] NO

A3. Have you gone to school in the United States?

[ ] YES LEVEL____________ [ ] NO


Questions: Self-Study


A4. Did you ever study English on your own/without a teacher?

[ ] YES LEVEL__________ [ ] NO (IF NO SKIP TO SECTION B)

A5. What did you do to learn?

A6. What did you like about the experience?

A7. What did you not like about the experience?


SECTION B: CHARACTERISTICS OF ONLINE LEARNING

The interviewers should familiarize themselves with the information below prior to the interview. The table should be provided to applicants as a hand-out. Interviewers should go through the table together with the applicants checking whether the applicants are familiar with the information. The interviewers can also skip some of the information depending on the level of knowledge of the applicant. INFORMATION TABLE ON DIFFERENT MODES OF LEARNING Face to Face Online Self-Study Space

In a classroom Anywhere where there is a computer

and internet access Anywhere

Time

An organized schedule defined by the institution offering instruction

Any time, self-planned Any time, self-planned

Teacher Yes, in the classroom Yes, at a distance No teacher

Mode of instruction

Multiple (books, field trips, simulations, labs, technology)

Computer, internet (can be complemented with books)

Computer, internet and books

Communications with the teacher and other learners

Face to face, and sometimes online Online: through forums, e-mail, chat room and blog; and face to face on occasion

N/A

Curriculum Planned curriculum with assignments and tasks and associated deadlines to complete

Planned curriculum with assignments and tasks

Self-planned

Control over learning

Instructor controls learning (activity, space, time, pace, mode of instruction, curriculum)

Learners control learning with assistance from instructor

Learners control learning

Feedback Immediate during classroom interaction

There is some wait time No feedback

Support

Mostly face to face; sometimes with computer, internet; usually direct

Mostly using computer, internet; e-mail, chat, etc…rarely face to face; might be indirect, i.e. instructor can guide one to resources for learning

No support

Assessment

Teacher assesses progress with grades on exams, assignments and tests

Usually self assessment of progress; might be built in the curriculum software; tests at certain points in time; no grades

Self assessment of progress; tests at certain points in time; no grades

Code of conduct

Defined by the institution and the teacher, usually communicated verbally

Defined by the institution and the teacher, some of the same rules in classroom learning will apply

N/A

Cost

Self-pay or tax payers pay Tax payers pay; it is not really free Self-pay

Duration of study Determined by the institution (one semester, two semesters)

Depends on the amount of hours invested in studying, could take as much or longer than face to face

Depends on the amount of hours invested in studying, could take as much or longer than face to face


SECTION C: GOALS AND MOTIVATION

Questions


C1. Can you tell me why you want to learn better English? PROBE: Could you share an example of why it is important for you?

Tell Applicants: “This online English course has __ lessons, and we want you to study 1 lesson each week. You can finish this course in ___ weeks.” C2. Do you think you can stay in this course and finish it in __ weeks?


SECTION D: PERCEPTIONS & EXPECTATIONS

STEP 1

Provide applicants with information* on:

• specifics of the curriculum;

• what can be achieved studying with this

curriculum;

• how the program, and how the curriculum is

delivered whether through a software or an internet

site.

STEP 2

• Provide an opportunity to ask questions;

• Further encourage applicants to think whether they

can benefit from this mode of learning;

• Gain an understanding of applicants’ perceptions

about and expectations of the ESOL program and

online learning;

• Clarify any misconceptions they might have about

the ESOL program and online learning.

*This information could be provided in a simple format in the form of a booklet or a handbook. It might also be useful to give a handout on examples from the courses and topics and include information on any additional resources that applicants can use.


SECTION E: STRENGTHS AND WEAKNESSES

Questions


E1. Is learning English hard for you?

[ ] YES [ ] NO (IF NO SKIP TO SECTION E3)

E2. If it is hard, why?

E3. In English, which are your best skills?

• Reading • Writing • Speaking • Listening

E4. In English, what is hardest for you?

• Reading • Writing • Speaking • Listening

E5. Which of the skills we talked about you think you will be able to do better after you complete this course?


SECTION F: TIME MANAGEMENT

Tell applicants: “This course will require you to study_____hours every week on the computer.”

Questions


F1.Are you planning to study at home or somewhere else?

F2.When do you plan to study? PROBE: What times, on what days?

Tell applicants: “Studying online requires that you are well-organized and that you complete your assignments and lessons on time.” F3. Do you usually do things on time? PROBE: How will you manage your home and work responsibilities with your English studies?

[ ] YES [ ] NO


SECTION G: SUPPORTS AND CHALLENGES

Interviewers should remember that one of the purposes of this section is to assess how comfortable applicants are in asking for help; thus interviewers should be ready to ask appropriate follow-up questions if necessary. Questions


G1. Do your friends and family support you in studying English? PROBES: If yes, how do they support you? Example? If no, how could they support you more?

[ ] YES [ ] NO

G2. Do you think that you will have difficulties studying English online? PROBE: If yes, what are those?

[ ] YES [ ] NO

G3. What would you do if you have difficulties studying online?

G4. How would you like your instructor to support you in your studies?


APPLICANT’S SELF ASSESSMENT:

The purpose of this self assessment section is to further encourage students to think about learning online. The decision regarding students’ readiness should be based on all the information gathered throughout the interview. Now that you know a little bit about learning online, this program and the courses, what do you think about learning this way? Is learning online for you?

INTERVIEWER’S ASSESSMENT OF THE APPLICANT:

On the basis of the academic and computer skills assessment and this interview please indicate how you feel about this applicant’s readiness for ESOL online: ___ IS READY FOR THIS ESOL PROGRAM: ___ IS NOT READY FOR THIS ESOL

PROGRAM:

Please note below areas where this applicant might need extra support during the first lessons of the ESOL program, based on the information learned during the screening: Listening Skills:

Reading Skills:

Writing Skills:

Speaking Skills:

Computer Skills:

Goals and Expectations:

Time Management:

Learning Supports and Challenges:

Other:


VI. APPENDIX

DEMOGRAPHIC INFORMATION FORM

This information will be gathered later during intake. However, it might be useful to gather this information at this point for applicants who are being screened out.

BASIC INFORMATION:

First Name: Middle Initial:

Last Name:

Gender: � Male � Female

Date of Birth: ______ / ______/ _______ MM DD YEAR

Ethnicity: (CHECK ALL THAT APPLY)

� American Indian or Alaskan Native � Asian � Black (not of Hispanic origin) � Cape Verdean � Haitian � Hawaiian or Pacific Islander � Hispanic � Indian Sub-Continent � White (not of Hispanic origin)

Country of Birth:

Immigrant: � Yes � No

Mandated: � Yes � No

Interview Date: _____ / ______/ _______ MM DD YEAR

Native Language:

Interviewer Name:

Language at Home: Other Language:

Release of Information Form Signed: (revised 4/1/08) � Yes � No


EMPLOYMENT AND HOUSEHOLD:

Employment Status: � Employed � Unemployed and Not Looking for Work � Unemployed and Looking for Work � Retired or Otherwise Not Looking for Work � Homemaker

Occupation: (required if employed)

Job Type: � Full Time (30 or more hrs/wk) � Part Time ( less than 30 hrs/wk) � Multiple Jobs

Public Assistance: � TAFDC (Transitional Aid to Families with Dependent Children) � EAEDC (Emergency Aid to the Elderly, Disabled and Children) � Food Stamp Benefits � EA (Emergency Assistance) � SSI (Supplemental Security Income) � None � Other

Dependents:

� Yes (If Yes, Dependent Birth Year Required) � No

Dependent Birth Year:

In School (PreK-12): (Y/N)


VII. REFERENCES Artino, A.R. (June, 2008). Promoting academic motivation and self-regulation: Practical

guidelines for online instructors. TechTrends, Vol. 52 (3), pp. 37 – 45. Bandura, A. (1993). Perceived self-efficacy in cognitive functioning. Educational Psychologist,

28, 117-148. Comings, J.P., Parrella, A. & Soricone, L. (1999). Persistence among adult basic education

students in pre-GED classes. Cambridge, MA: National Center for the Study of Adult Learning and Literacy. Retrieved May, 2008 from http://gseweb.harvard.edu/~ncsall/research/report12.pdf (ERIC Document Reproduction Service No. ED 437 579)

Kahraman, B., Mallona, A., Haig-Friedman, D., Platt, E. & Kahan, M. (2008). Massachusetts’s

Adult Basic Education and ESOL distance learning programs: how to assess student readiness for distance learning? Unpublished document prepared for Adult and Community Learning Services (ACLS), Massachusetts Department of Elementary and Secondary Education. The Center for Social Policy, University of Massachusetts Boston.

Kahraman, B., Mallona, A., Haig-Friedman, D. & Kuck Jalbert, S. (2009). How to Assess

Readiness for Distance Learning: Findings from the Pilot Screening Process. Unpublished document prepared for Adult and Community Learning Services (ACLS), Massachusetts Department of Elementary and Secondary Education. The Center for Social Policy, University of Massachusetts Boston.

Pintrich, P.R. & Garcia, T. (1991). Student goal orientation and self-regulation in the collage

classroom. In M.L. Maerhr & P.R. Pintrich (Eds.), Advances in motivation and achievement: Goals and self-regulatory processes, vol 7, pp. 371-402. Greenwich, CT: JAI.

Schunk, D.H. (1989). Self-efficacy and cognitive skill learning. In C. Ames & R. Ames., (Eds.),

Research on motivation in education, vol.3. San Diego, CA: Academic Press, Inc. Schunk, D. H. (1991). Self-efficacy and academic motivation. Educational Psychologist, 26,

207-231. Schunk, D.H. & Zimmerman, B.J. (Eds.). (1998). Self-regulated learning: from teaching to self-

reflective practice. New York: The Guilford Press. Schraw, G., and Dennison, R. S. (1994). Assessing metacognitive awareness. Contemporary

Educational Psychology, 19, 460-475.

http://gseweb.harvard.edu/~ncsall/research/report12.pdf�

handbook of screening - university of massachusetts boston - a

Documents