gaming the system in the call classroom peter gobel kyoto sangyo university [email protected]
TRANSCRIPT
Gaming the System in the CALL Classroom
Peter GobelKyoto Sangyo [email protected]
Introduction
J. Carrol (1963)- time on task hypothesis: the learner will succeed in learning a given task to the extent that he spends the amount of time that he needs to learn the task.
Baker et. al (2004a)- off-task behavior associated with reduced learning
Cont.
Baker et. al (2004b)- gaming the system (misuse of tutoring systems) associated with substantially lower learning. 1/3 less than those who do not engage in this behavior.
Learned helplessness and ‘performance orientation’ related to gaming.
Basic Question
Is motivation related to misuse of the system or inappropriate help-seeking?
Research QuestionsIn Japanese university students:
Is frequency of off-task behavior a predictor of test performance?Does the kind of off-task behavior make a difference?Are motivational factors predictors of performance behavior and test performance?
Kyoto Sangyo CALL Curriculum
Students lower proficiency non-English majors
System meeting 90 minutes/week
• DynEd• ALC
Course requirements grade linked to completion of level tests and time spent on software system
Participants
Three intact classes - 105 students
All non-English majors
All streamed at low level
Observation
Off-task behavior (physical and virtual)
Non-system related activityInactivityMisuse of system (gaming)
On-task behavior (physical and virtual)Appropriate system-related activitySeeking help from peers, software, and teacher
Observation (continued)
Total amount of study timeRecorded automatically by software system
Study scoreBased on total amount of study time and ‘proper’ use of the software.
Observation (continued)
Test performance Gain scores on general proficiency test
Listening sectionReading sectionAdministered in April and July
Motivation Questionnaire
35seven-point Likert scale items
administered in Japanese
30 items were written referring mainly to motivational theory
5 items referring to CALL activity
Questionnaire Items
Expectancy for successAttainment ValueIntrinsic value
Extrinsic utility value
CostAttitudes toward target group
Effort
CALL related items
Results
ObservationOff-task behaviorOn-task behaviorTotal study timeStudy score
QuestionnaireStructureMotivational predictors of performanceMotivational predictors of behavior
Classroom Observation
0
10
20
30
40
50
60
70
80
90
1stQtr
2ndQtr
3rdQtr
4thQtr
EastWestNorth
on/off-task averagesoff-T nonsoft5%off-T inactive14%
off-T gaming5%off-T software help0%on-T software75%
pn-T t/peer help1%
Off-task behavior
Off-task non software
Off-task inactive
Off-task software
Help
Study record
Off-task gaming
On-task behavior
On-task softwareMicrophones and speech analyzers seldom usedCertain activities overused and recycled
On-task helpRarely usedTranslation rarely used
Behavior Regression Results
Whether students were generally off task or on task was a significant predictor of test performance.
Gaming was not a significant predictor of test performance.
Behavior Regression Results
Listening Gain
Study Score a significant predictor of performance
Reading Gain
Study Score &Total Study Time significant predictors of performance.
Questionnaire results Four factors
Attitudes toward the target group
Perceived usefulness of studying
English
Expectancy for success
Attitude towards CALL
Motivation regression analysis
Factor 1 (Attitudes
towards the target
group) was a predictor
of gain scores in both
listening and reading,
but not for behavior.
Research Questions Revisited
Task behavior was a predictor of test performance.The kind of off-task behavior did not make a difference.Attitude toward the TG was a predictor of test performance, but not off/on-task behavior.
ConclusionOff-task behavior
Evidence of off-task behavior related mostly to inactivity rather than gaming
On-task behaviorAll aspects of the software not fully used
MotivationAttitude towards TG most important for effective CALL use
References
Baker, R. S., Corbett, A. T., Koedinger, K. R., & Wagner, A. Z. (2004). Off-task behavior in the cognitive tutor classroom: When students ‘game the system’. Proceedings of ACM CHI 2004: Computer Human Interaction, 383-390.
Baker, R. S., Corbett, A. T., & Koedinger, K. R. (2004). Detecting student misuse of intelligent tutoring systems. Proceedings of the Seventh International Conference on Intelligent Tutoring Systems, 531-540.
Carrol, J. (1963). A model of school learning. Teachers College Record, 64(8), 723-733.