TSL 641Chapelle, C. (2001). Computer Applications in

SLA. UK: Cambridge

Three needs must be addressed when evaluating CALL materials (courseware/website, tasks, activities etc. )

1. Evaluation criteria should incorporate findings and theory-based speculation about ideal conditions for SLA

2. Criteria should be accompanied by guidance as to how they should be used i.e. a theory of evaluation needs to be articulated

3. Both criteria and theory need to apply not only to software, but also to the task that the teacher plans and that the learner carries out

CALL Evaluation: Principle 1

Evaluation of CALL is a situation-specific argument.

The outcome of task evaluation for any L2 tasks including those for CALL cannot be a decision about effectiveness.

Instead, an evaluation has to result in an argument indicating in what ways a particular CALL task is appropriate for particular learners at a given time, the use at a particular setting

Implication: CALL developers need to be familiar with criteria for evaluation which should be applied relative to a particular context.

CALL Evaluation: Principle 2

CALL should be evaluated through two perspectives: judgmental analysis of software and planned tasks, and empirical analysis of learners’ performance.

Implication: Methodologies for both types of analyses are needed.

The 3 levels of analysis for CALL evaluation.

• Judgmental analyses - examine characteristics of the software and task in terms of criteria drawn from research on SLA.

• Empirical analyses - address the same criteria but through data gathered to reveal the details of CALL use and learning outcomes.

CALL Evaluation: Principle 3

Criteria for CALL task quality should come from theory and research on instructed SLA.

Implication: CALL evaluators need to keep up with and make links to research on instructed SLA.

Language learning potential = the extent to which the activity can be considered to be a language learning activity rather than simply an opportunity for language use.

Learner fit = the individual differences in linguistic ability level and non-linguistic characteristics.

Meaning focus = the learner’s primary attention is directed toward the meaning of the language that is required to accomplish the task.

Authenticity = the degree of correspondence between an L2 learning task and tasks that the learner is likely to encounter outside the classroom.

Positive impact = its effects beyond its language learning potential. E.g. – help build metacognitive strategies, willingness to seek out opportunities to use TL.

Practicality = how easy it is for the learners and teachers to implement a CALL task within the particular constraints of a class or language program.

CALL Evaluation: Principle 4

Criteria should be applied in view of the purpose of the task.

Implication: CALL tasks should have a clearly articulated purpose.

CALL Evaluation: Principle 5

Language learning potential should be the central criterion (most critical) in evaluation of CALL.

Implication: Language learning potential should be one aspect of the purpose of CALL tasks.

Level of Analysis

Object of Evaluation

Example Question Method of Evaluation

1 CALL software

Does the software provide learners the opportunity for interactional modifications to negotiate meaning?

Judgemental

2 Teacher-planned CALL

activities

Does the CALL activity designed by the teacher provide learners the opportunity to modify interaction for negotiation of meaning?

Judgemental

3

Learners’ performance during CALL

activities

Do learners actually interact and negotiate meaning while they are working in a chat room?

Empirical

Levels of Analysis for CALL EvaluationLevels of Analysis for CALL Evaluation

Implication!

Criteria for CALL Task AppropriatenessCriteria for CALL Task AppropriatenessLanguage learning potential

The degree of opportunity present for beneficial focus on form.

Learner fit

The amount of opportunity for engagement with language under appropriate conditions given learner characteristics.

Meaning focus The extent to which learners’ attention is directed toward the meaning of the language.

Authenticity

The degree of correspondence between the CALL activity and target language activities of interest to learners out of the classroom.

Positive impact The positive effects of the CALL activity on those who participate in it.

Practicality The adequacy of resources to support the use of the CALL activity.

Implication!

Qualities Questions

Language learning potential

Do task conditions present sufficient opportunity for beneficial focus on form?

Learner fit Is the difficulty level of the targeted linguistics forms appropriate for the learners to increase their language ability?Is the task appropriate for learners with the characteristics of the intended learners?

Meaning focus Is learner’s attention directed primarily toward the meaning of the language?

Authenticity Is there a strong correspondence between the CALL task & second language tasks of interest to learners outside the classroom?Will learners be able to see the connection between the CALL task and tasks outside the classroom?

Impact Will learners learn more about the target language and about strategies for language learning through the use of the tasks. Will instructors observe sound second language pedagogical practices by using the task?Will both learners and teachers have a positive learning experience with technology through the use of the task?

Practicality Are hardware, software, and personnel resources sufficient to allow the CALL task to succeed?

Qualities Questions

Language learning potential

What evidence suggests that the learner has acquired the target forms that were the focused on during the CALL tasks/What evidence indicates that learners focused on form during the CALL task?

Learner fit What evidence suggests that the targeted linguistic forms are at an appropriate level of difficulty for the learners?What evidence suggests that the task is appropriate to learners’ individual characteristics (age, learning styles, computer experience)

Meaning focus What evidence suggests that the learner’s construction of linguistic meaning aids language learning?What evidence indicates that the learners use the language during the task for constructing and interpreting meaning?

Authenticity What evidence suggests that learners’ performance in the CALL tasks corresponds to what one would expect to see outside the CALL tasks?What evidence suggests that the learners see the connection between CALL task and tasks outside the classroom?

Impact What evidence suggests that the learners learn more about the target language and about strategies for language learning through the use of the tasks?

Practicality What evidence suggests that hardware, software and personnel resources prove to be sufficient to allow the CALL task to succeed?