evaluating learners’ motivational and cognitive processing in an online game-based learning...

11
Evaluating learners’ motivational and cognitive processing in an online game-based learning environment Wen-Hao Huang * Department of Human Resource Education, Department of Educational Psychology, University of Illinois, IL, USA article info Article history: Available online 17 August 2010 Keywords: Motivation Cognitive load Game-based learning Mental effort investment abstract This paper describes the process and results of an evaluation on an online game-based learning environ- ment (GBLE) by focusing on learners’ motivational processing and cognitive processing. The goal is to explore how online GBLE might initiate and support learners’ goal-setting activities and impact learners’ cognitive loads. The study surveyed 144 undergraduate students after their autonomous participation in the online game available at the Nobel Prize Foundation website teaching the Heckscher–Ohlin Theory on international trade. Grounded in the integrative theory of motivation, volition, and performance (MVP), the evaluation indicated that participants felt significantly confident in learning the subject. The per- ceived satisfaction, however, was lower than the rest of motivational components possibly due to heavy cognitive processing. The finding of cognitive load reported that learners perceived a significantly higher level of intrinsic load than the germane load due to the novelty of the subject matter. Data analysis fur- ther indicated a significant canonical correlation between learners’ motivational and cognitive process- ing. This particular finding could inform future research to investigate specific motivational processing components’ effects on learners’ cognitive load levels in online GBLEs. Ó 2010 Elsevier Ltd. All rights reserved. 1. Introduction Studies in education and instructional design have been con- ducted with the intention of finding effective interventions that can increase and sustain learning motivation. With today’s learn- ing technologies, however, insufficient emphasis has been placed on the motivational impact induced by complex learning environ- ments enriched with multimedia and interactions. The complexity of highly interactive learning environments coupled with high le- vel motivational support could pose high demand on learners’ lim- ited cognitive processing capacity, which in turn, might impede the learning process (Warschauer, 2007). Online game-based learning environment (GBLE), as an example, might dissipate excessive motivational support that could very likely overwhelm learners’ cognitive processing capacity (Ang, Zaphiris, & Mahmood, 2007; Huang & Aragon, 2009). A game, regardless of its delivery mechanism, is a context in which individual and teamed players compete to attain game objectives by following rules and principles. The playing process is fun, voluntary, and intended to overcome challenges (Gredler, 1994; Suits, 1978). In GBLE, playing becomes ‘‘serious” activities that require players to achieve the game and learning objectives (Apt, 1970). Avedon and Sutton-Smith (1971) argued that playing instructional games allows learners to control a disequilibrium system, and players continuously devise, implement, evaluate, and revise new strategies to restore the system to the equilibrium state. The game playing process therefore supports the learning process by allowing players to acquire learning experiences in games, encouraging interactions between learners and the game system, and situating learners in complex learning environments (Johnson & Huang, 2008; Pannese & Carlesi, 2007). While the learning process in online GBLE might be promising to engage learners, the inherent complexity of interacting with on- line GBLE might pose problems for learners. Huang and Johnson (2008) identified 10 digital game characteristics that are often seen in computer-based instructional games, all require learners’ signif- icant cognitive investment to process the environmental and social stimuli while identifying essential cues for the performance goals. If managed improperly the learning process could be interrupted early, because learners’ limited motivational processing as well as cognitive processing capacity could be overloaded (Ang et al., 2007; Keller, 2008). Studies have utilized the attention, relevance, confidence, and satisfaction (ARCS) model of motivational design (Keller, 1987a, 1987b) to evaluate GBLEs’ motivational stimuli for learners’ perfor- mance (Chang & Lehman, 2002; House, 2003; Huang, Huang, Diefes-Dux, & Imbrie, 2006; Means, 1997; Song & Keller, 2001; Wongwiwatthananukit & Popovick, 2000). Recently ChanLin (2009, p.101) applied ARCS model in guiding the design of web-based courses. The qualitative analysis on participants’ online discussion postings suggests that online learners could be 0747-5632/$ - see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.chb.2010.07.021 * Tel.: +1 217 333 0807. E-mail address: [email protected] Computers in Human Behavior 27 (2011) 694–704 Contents lists available at ScienceDirect Computers in Human Behavior journal homepage: www.elsevier.com/locate/comphumbeh

Upload: wen-hao-huang

Post on 05-Sep-2016

230 views

Category:

Documents


4 download

TRANSCRIPT

Page 1: Evaluating learners’ motivational and cognitive processing in an online game-based learning environment

Computers in Human Behavior 27 (2011) 694–704

Contents lists available at ScienceDirect

Computers in Human Behavior

journal homepage: www.elsevier .com/locate /comphumbeh

Evaluating learners’ motivational and cognitive processing in an onlinegame-based learning environment

Wen-Hao Huang *

Department of Human Resource Education, Department of Educational Psychology, University of Illinois, IL, USA

a r t i c l e i n f o

Article history:Available online 17 August 2010

Keywords:MotivationCognitive loadGame-based learningMental effort investment

0747-5632/$ - see front matter � 2010 Elsevier Ltd. Adoi:10.1016/j.chb.2010.07.021

* Tel.: +1 217 333 0807.E-mail address: [email protected]

a b s t r a c t

This paper describes the process and results of an evaluation on an online game-based learning environ-ment (GBLE) by focusing on learners’ motivational processing and cognitive processing. The goal is toexplore how online GBLE might initiate and support learners’ goal-setting activities and impact learners’cognitive loads. The study surveyed 144 undergraduate students after their autonomous participation inthe online game available at the Nobel Prize Foundation website teaching the Heckscher–Ohlin Theory oninternational trade. Grounded in the integrative theory of motivation, volition, and performance (MVP),the evaluation indicated that participants felt significantly confident in learning the subject. The per-ceived satisfaction, however, was lower than the rest of motivational components possibly due to heavycognitive processing. The finding of cognitive load reported that learners perceived a significantly higherlevel of intrinsic load than the germane load due to the novelty of the subject matter. Data analysis fur-ther indicated a significant canonical correlation between learners’ motivational and cognitive process-ing. This particular finding could inform future research to investigate specific motivational processingcomponents’ effects on learners’ cognitive load levels in online GBLEs.

� 2010 Elsevier Ltd. All rights reserved.

1. Introduction system, and players continuously devise, implement, evaluate,

Studies in education and instructional design have been con-ducted with the intention of finding effective interventions thatcan increase and sustain learning motivation. With today’s learn-ing technologies, however, insufficient emphasis has been placedon the motivational impact induced by complex learning environ-ments enriched with multimedia and interactions. The complexityof highly interactive learning environments coupled with high le-vel motivational support could pose high demand on learners’ lim-ited cognitive processing capacity, which in turn, might impede thelearning process (Warschauer, 2007). Online game-based learningenvironment (GBLE), as an example, might dissipate excessivemotivational support that could very likely overwhelm learners’cognitive processing capacity (Ang, Zaphiris, & Mahmood, 2007;Huang & Aragon, 2009).

A game, regardless of its delivery mechanism, is a context inwhich individual and teamed players compete to attain gameobjectives by following rules and principles. The playing processis fun, voluntary, and intended to overcome challenges (Gredler,1994; Suits, 1978). In GBLE, playing becomes ‘‘serious” activitiesthat require players to achieve the game and learning objectives(Apt, 1970). Avedon and Sutton-Smith (1971) argued that playinginstructional games allows learners to control a disequilibrium

ll rights reserved.

and revise new strategies to restore the system to the equilibriumstate. The game playing process therefore supports the learningprocess by allowing players to acquire learning experiences ingames, encouraging interactions between learners and the gamesystem, and situating learners in complex learning environments(Johnson & Huang, 2008; Pannese & Carlesi, 2007).

While the learning process in online GBLE might be promisingto engage learners, the inherent complexity of interacting with on-line GBLE might pose problems for learners. Huang and Johnson(2008) identified 10 digital game characteristics that are often seenin computer-based instructional games, all require learners’ signif-icant cognitive investment to process the environmental and socialstimuli while identifying essential cues for the performance goals.If managed improperly the learning process could be interruptedearly, because learners’ limited motivational processing as wellas cognitive processing capacity could be overloaded (Ang et al.,2007; Keller, 2008).

Studies have utilized the attention, relevance, confidence, andsatisfaction (ARCS) model of motivational design (Keller, 1987a,1987b) to evaluate GBLEs’ motivational stimuli for learners’ perfor-mance (Chang & Lehman, 2002; House, 2003; Huang, Huang,Diefes-Dux, & Imbrie, 2006; Means, 1997; Song & Keller, 2001;Wongwiwatthananukit & Popovick, 2000). Recently ChanLin(2009, p.101) applied ARCS model in guiding the design ofweb-based courses. The qualitative analysis on participants’ onlinediscussion postings suggests that online learners could be

Page 2: Evaluating learners’ motivational and cognitive processing in an online game-based learning environment

W.-H. Huang / Computers in Human Behavior 27 (2011) 694–704 695

benefited by the motivational strategies embedded in web-basedcourses based on ARCS principles. Other attempts have been situ-ated in interactive instructional settings to address the motiva-tional processing issue (Dempsey & Johnson, 1998; Klein &Freitag, 1991; Small & Ferreira, 1994). The learning environmentsthat hosted these studies, however, were far less complex and dis-tracting than what is commonly available in online GBLEs today.

With regards to the cognitive demand, studies have concurredthat learners could be cognitively overloaded by highly interactivelearning activities in GBLEs. In massively multiplayer online roleplaying games (MMORPGs), for instance, players need to invest asubstantial amount of mental effort to be able to simultaneouslyinteract with the game environment, game objects, game tasks,and other players. If it is beyond what the player’s limited cogni-tive processing could handle, cognitive overload occurs (Anget al., 2007). Enriched multimedia in educational games also im-pacts the efficiency on learners’ cognitive processing. Nelson andErlandson (2008) argued that the multimedia information process-ing drawn from multiple information sources increases learners’mental effort investment (cognitive load) to process the visual, tex-tual, and audio elements. The design of multimedia messagesshould be based on multimedia learning principles in order toavoid cognitive overload (Mayer & Moreno, 2003).

To emphasize the equal positions of motivational and cognitiveaspects of learning processes in multimedia learning environments,studies have proposed a potential relationship between learners’motivational processing and their mental effort investment. Situ-ated in the theory of multimedia learning (Mayer, 2001), Astleitnerand Wiesner (2004) proposed an integrated model of multimedialearning and motivation to connect learners’ cognitive processing inmultimedia learning environments with their motivational levelsmeasured by the ARCS model of motivational design. Later Deimannand Keller (2006) included learners’ volitional control to explainmotivational learning process in multimedia learning environ-ments. Both theoretical frameworks, however, have not been empir-ically examined in online game-based learning environments.

Therefore, this study aimed to understand (1) how a commononline GBLE might impact learners’ motivational processing capac-ity, (2) how the GBLE might impact learners’ mental effort invest-ment based on the cognitive load theory, and (3) what might be theempirical relationship between learners’ motivational processingand cognitive processing suggested by recent integrative theories(Keller, 2008).

2. Literature review

Learning motivation is complex to measure due to its multipleconstructs inherent within the domain (Driscoll, 2000; Mayer,2003). The increasing complexity of today’s online GBLEs furtherchallenges the current understanding of motivational processing.The following section first discusses the neglected motivationcomponent in instructional design and ARCS model of motivationaldesign (Keller, 1987a, 1987b). Then the paper discusses the inte-grative theory of motivation, volition, and performance (MVP) withspecific focus on motivational processing and cognitive processing(Keller, 2008). A selected group of recent studies then demonstratethe need to implement empirical studies to investigate learners’motivational and cognitive processing in online GBLEs. Finallythe instructional materials motivational survey and cognitive loadscale are discussed as instruments for this study.

2.1. The neglected motivation component in instructional design

Goal-directed behaviors are often stimulated and maintained byan essential process known as motivation (Berliner & Gage, 1998;

Schunk, 1990), which plays a critical role in learning (Weiner,1985). Studies also identified positive correlations between learn-ers’ motivational levels and performance achievements (ChanLin,2009; Sachs, 2001; Sankaran & Bui, 2001). It is, however, often ne-glected in instructional design owing to its inherent complexity in-volved with self-regulatory skills, learner control (Armstrong,1989; Baird & White, 1982; Lee, 1990) and meta-cognitive activi-ties (Zimmerman, 1989; Zimmerman & Martinez-Pons, 1988).The ignored motivational components might discount the finallearning and performance outcomes attained by learners (Cheng& Yeh, 2009; Keller, 1983; Spitzer, 1996).

2.2. ARCS model of motivational design

The ARCS model of motivational design (Keller, 1983, 1987a,1987b), widely applied in instructional design processes that con-nects learning motivation with performance (Ames, 1992; Ander-man & Maehr, 1994; Bandura, 1997; Huang & Johnson, 2002;Keller, 2008; Weiner, 1985), suggests that learning motivation isdependent of four perceptual components: attention, relevance,confidence and satisfaction (Keller, 2008). Attention refers to thelearner’s response to perceived instructional stimuli provided bythe instruction (Keller, 1983). Relevance helps learners associatetheir prior learning experience with the given instruction. Confi-dence stresses the importance of building learners’ positive expec-tation towards their performance on the learning task. Satisfactioncomes near the end of the learning process when learners are al-lowed to practice newly acquired knowledge or skills (Keller,1987b). The model was initially developed as a conceptual toolfor diagnosing motivational problems and prescribing motivationalstrategies (Keller, 1983), which includes pre-measurement oflearners’ motivational level, motivational intervention implemen-tation, and post-measurement of learners’ motivational level.

ARCS model, based on various learning, instructional, and moti-vation theories (Driscoll, 2000; Small & Gluck, 1994; Steers & Por-ter, 1983), focuses on the interactions between learners and theinstructional programs. Its main thesis is rooted in the expec-tancy-value theory (Porter & Lawler, 1968; Vroom, 1964) thatviews human behaviors as evaluative outcomes among expecta-tions (beliefs), perceived probability for success (expectancy), andperceived impact of the success (value) (Palmgreen, 1984). Whatthe model theoretically measures, therefore, is the amount of effortinvested by learners to achieve the learning goal (Small, 2000;Song & Keller, 2001).

2.3. Integrative theory of motivation, volition, and performance

In his latest rendition of motivational learning, the integrativetheory of motivation, volition, and performance (MVP), Keller(2008) argues that a complete motivational learning cycle is con-sisted of several stages: motivational and volitional processing,motivational and information processing interfacing, informationand psychomotor processing, and finally, the outcome processing(p. 94). Motivational processing helps learners set up initial perfor-mance goals that are critical for sustainable learning processes.Learners at first should have sufficient level of curiosity to explorethe learning task (attention); then understand the value of thelearning task (relevance), and evaluate the possibility of attainingsuccessful performance (confidence), to identify and confirm theperformance goal. These processes, in turn, prepare learners forthe follow-up actions of learning. The satisfaction component seenin the ARCS model, however, is only considered at the end of thelearning cycle.

The next stage is the volitional processing that converts learn-ers’ learning intentions into executable learning actions. Learnersat this point should apply action control strategies to implement

Page 3: Evaluating learners’ motivational and cognitive processing in an online game-based learning environment

696 W.-H. Huang / Computers in Human Behavior 27 (2011) 694–704

needed activities that move them towards the performance goal.At the effect of volitional processing learners enter the interfacebetween motivation and information processing. This is wherelearners apply meta-cognitive strategies to actively manage theirlearning processes within the limited cognitive processing capac-ity. The next stage, information and psychomotor processing, fo-cuses on how learners might utilize a variety of mental activitiesto process information that leads to the desired performance.Learners at this stage carry out learning activities that help themcreate and automate transferrable mental models. The processingcapacity, however, is limited by learners’ working memory.

Finally, the outcome processing stage allows learners to evalu-ate the discrepancy between the performance consequence andtheir invested efforts. Learners reflect upon all previous stages’experiences emotionally and cognitively, and develop a collectivesense of satisfaction towards the learning process.

The implication of the theory of MVP is twofold. First, sincemotivational processing is crucial at the early stage of the learningprocess, instructional designers must be cautious to neither over-whelm learners’ processing capacity nor distract them with com-peting stimuli. In online GBLEs this design consideration isparticularly important (Astleitner & Wiesner, 2004). Second, learn-ers’ cognitive processing activities could play a substantial role insustaining the learners’ motivation, because after the motivationaland volitional processing, learners must interact with the learningenvironment cognitively with their limited processing capacity be-fore the final outcome processing. Learners overloaded with cogni-tive stimuli, regardless of their initial attention, confidence, andrelevance levels (motivational processing results), are still vulner-able to be unmotivated by exhausting cognitive information pro-cessing tasks.

2.4. Recent studies on motivational and cognitive processing in onlineinstructional games

GBLEs and many of their derivative forms are motivating to play(Huang & Johnson, 2008). Their effect in promoting meaningfullearning might be due to opportunities for ‘‘learning by doing”(Pannese & Carlesi, 2007). Klein and Freitag (1991) concluded thatinstructional board games had a positive impact on students’attention, relevance, confidence, and satisfaction levels. Thecontext of the instructional board game, however, was not as inter-active and complex as learners can experience in online environ-ments today. Dickey (2007) analyzed a massively multiple-playeronline game environment and reached two conclusions. First, on-line game environments could provide practical design modelsfor creating complex learning environments. Second, the characterdesign and narrative environments of game environments couldfoster players’ intrinsic motivation and sustain their persistent par-ticipation in the game playing process. In a case study, Pannese andCarlesi (2007) too identified factors that might integrate gameplaying with intended motivational learning processes with focuson the reflection aspect of the motivational processing. Althoughstudies have reported that digital games could be repurposed forinstructional applications due to their motivational support (Gee,2003; Papastergiou, 2008; Prensky, 2001; Rieber, 1996), it remainsinconclusive as to how those GBLEs could impact each motiva-tional processing component, which poses challenges for designersto prescribe effective motivational design strategies (Cheng & Yeh,2009).

With regards to the cognitive processing in online GBLEs, manyhave discussed the design of GBLEs with focus on reducing learn-ers’ cognitive load. In a qualitative exploratory study, Ang et al.(2007) found that the game playing process in complex game-based environments could overtax players’ cognitive capacity dueto common factors that are available in online GBLEs in general

(e.g., multiple interactions in the game, user interface activities,and identity construction). The design online GBLEs, as interactivelearning environments, must consider learners’ limited cognitiveprocessing capacity, to ensure efficient learning processes (Kalyu-ga, 2007). Substantial efforts have also been devoted to study indi-vidual design elements in interactive learning environments andtheir effect on cognitive load. For example, the nonlinear fashionof textual information presentation often seen in GBLEs might in-crease learners’ cognitive load (Zumbach & Mohraz, 2008). Ani-mated instructional messages, by posing a higher level ofineffective cognitive load, also might demand more cognitive pro-cessing capacity from learners (Ayres, Kalyuga, Marcus, & Sweller,2005; Ayres & Paas, 2007). While those findings were fruitful forcognitive load studies to a large extent, further investigations areneeded to discuss the collective effect of online GBLE elementson learners’ cognitive processing.

2.5. Instructional materials motivational survey (IMMS) and cognitiveload scale

Keller (1993) developed a measuring instrument, instructionalmaterials motivational survey (IMMS), to complement the imple-mentation of the ARCS model. The instrument itself has raised sev-eral issues concerning its applicability to computer-basedinstructional programs. First, there is a lack of empirical studiessupporting the IMMS’s validity to measure each ARCS component.Most of ARCS model-based research utilized ARCS model as a de-sign guideline to design motivationally sound instructions(Arnone, 2003; Chyung, 2001; Jacobson & Xu, 2002). Second, thereis a lack of studies investigating the motivational processing com-ponents as a whole. Sachs (2001) discussed the impact of confi-dence level on learners’ ability to perform well, while Chang andLehman (2002) and Means (1997) both focused on the relevancecomponent of the ARCS model.

To address these issues, a study was conducted to preliminarilyvalidate the instrument in a computer-based learning environ-ment. Based on the results of exploratory and confirmatory analy-ses from 875 undergraduate students, the study concluded thatIMMS, while valid and applicable for the computer-based setting,its responsiveness to instructional programs’ features makes theinstrument a design-practical diagnostic tool to evaluate learners’motivational processing in similar instructional settings (Huanget al., 2006).

2.6. Cognitive load theory and mental effort measurement

The gap between information structures presented in theinstructional material and human cognitive architecture must bebridged so that learners can use their working memory efficiently(Sweller, van Merriënboer, & Paas, 1998). Based on the assumptionthat learning is supported by schema construction and automation,cognitive load theory (CLT) (Chandler & Sweller, 1991) proposes aframework to connect cognitive learning processes with instruc-tional design (van Merriënboer, Clark, & De Croock, 2002).

CLT defines cognitive load as a multidimensional construct thatincludes task-based mental load induced by task characteristics,learners’ performance, and mental effort invested by learners inthe working memory to process information (Paas, Tuovinen, Tab-bers, & van Gerven, 2003; Paas & van Merriënboer, 1994; Swelleret al., 1998). Mental effort, among the three, is suggested as themeasure that reflects the authentic cognitive load of learners,which indicates the actual cognitive load allocation by learnersas the result of interacting with task characteristics while achiev-ing the desired performance (Kalyuga, 2007; Paas et al., 2003).

Three types of cognitive load which, combined, compose the to-tal cognitive load: intrinsic, extraneous, and germane. The total

Page 4: Evaluating learners’ motivational and cognitive processing in an online game-based learning environment

W.-H. Huang / Computers in Human Behavior 27 (2011) 694–704 697

cognitive load can never exceed a learner’s working memorycapacity. The total extraneous and germane cognitive load, com-bined, is assumed to be equal to the total cognitive load minusthe intrinsic cognitive load. Since the intrinsic cognitive load can-not be manipulated via instructional interventions, instructionaldesign’s main purpose is to optimize the combination of the extra-neous cognitive load and the germane cognitive load. That is to re-duce the extraneous while increasing the germane cognitive load(van Gerven, Paas, van Merriënboer, & Schmidt, 2006).

Intrinsic cognitive load is associated with the element interac-tivity – the degree to which information can be understood alonewithout other elements’ involvement – inherent to the instruc-tional material itself. Information with high element interactivityis difficult to understand thus induces a high intrinsic cognitiveload, since the instruction requires more working memory forinformation processing (Paas et al., 2003). The extraneous cogni-tive load and germane cognitive load, in contrast, can be manipu-lated by instructional design (Brünken, Plass, & Leutner, 2003).

Also known as ineffective cognitive load, as it only involves theprocess of searching for information, extraneous cognitive load canbe influenced by the way information is structured and presented(Paas et al., 2003; Sweller et al., 1998). Considered a necessary cog-nitive cost of processing information, yet not related to the under-standing of the information or the construction of new schema ormental models, extraneous cognitive load must be reduced (Brün-ken et al., 2003). One method found to be successful in reducingextraneous cognitive load is the use of well-structured instruc-tional multimedia components since multimedia representationsare able to lower the cognitive load by utilizing learners’ multiplemodalities to process information (Khalil, Paas, Johnson, & Payer,2005a, 2005b; Mayer & Moreno, 2003).

In contrast to the desired low degree of the extraneous cogni-tive load, instructional materials should be designed to increasethe germane cognitive load. Also known as effective cognitive load,the germane cognitive load indicates the mental effort learners in-vest in learning (Paas et al., 2003). A higher germane cognitive loadlevel is suggested to induce a deeper learning experience, which inturn, supports both near and far transfers of desired performance(Kalyuga, 2009; van Merriënboer et al., 2002). The essential designprinciple for enhancing germane cognitive load is to deliverinstructions that compel learners to constantly reexamine everynew piece of information while accessing their long-term memory(de Crook, van Merriënboer, & Paas, 1998).

CLT provides a framework to allow researchers constantlydeveloping and revising valid and reliable measurements to gaugelearners’ cognitive load and it is an ongoing process (Kalyuga,2009), which focuses on learners’ mental effort as the result ofinteracting with instructional materials and environments (Swelleret al., 1998). The subjective category of the mental effort measure-ment was often used as the main indicator of learners’ overall cog-nitive load in earlier studies, because its higher reliability, validity,and sensitivity to learners’ small cognitive load changes whencompared to the other two categories (physiological, and task/per-formance-based) (Paas & van Merriënboer, 1994; Paas, van Merrië-nboer, & Adam, 1994). Paas and van Merriënboer (1994) proposeda 9-point symmetrical category scale to ask learners to report theirinvested mental effort. Later a similar 7-point symmetrical scalewas proposed the tested by other researchers (Kalyuga, Chandler,& Sweller, 1999; Marcus, Cooper, & Sweller, 1996). Recent studiessuggested that the measurements also need to include data sourcesto identify individual types of cognitive load to better inform fol-low-up design actions, which suggest that learners’ self-reportedmental effort was related to the intrinsic cognitive load; and self-reported task difficulty rating might indicate the germane cogni-tive load (Ciernak, Scheiter, & Gerjets, 2009; DeLeeuw & Mayer,2008).

2.7. Purposes of the study

Based on previous discussions on the lack of empirical supporton how learners’ motivational processing and cognitive processingmight be influenced by online GBLEs’, this study aimed to addressthe following questions.

1. What are learners’ motivational processing levels induced byonline GBLE based on the ARCS model of motivational design?

2. What are learners’ cognitive processing levels induced by onlineGBLE based on cognitive load theory?

3. Is there any empirical relationship between motivational pro-cessing and cognitive processing that might support the theoryof motivation, volition, and performance?

3. Methodology

3.1. Overview of the online instructional game

The ‘‘Trade Ruler” game developed by the Nobel Prize Founda-tion was selected as the target online GBLE for two reasons. First,the content of the instructional game (economic theory) is novelto the participants (undergraduate students majoring in Educa-tion), and second, the interaction between learners and the gameis enriched by its multimedia components and consistent cognitiveactivities.

Based on the Heckscher–Ohlin Theory, this online GBLE was de-signed to teach general public about why countries need to tradegoods and services with each other. The theory won the SverigesRiksbank Prize in Economic Sciences in Memory of Alfred Nobelin 1977 for its contribution to the field of international trade. Thegame is accessible online with web browsers. When players accessthe entry page (http://nobelprize.org/educational_games/econom-ics/trade/) they can review the overview of the instructional gameconsisting of the introduction and rules of the game, systemrequirement for playing the game, history of Nobel Prize, and theHeckscher–Ohlin Theory. Players are directed to the theory pageof the game, Why Trade, at the end of the overview page. Playerscan easily start the game using a link consistently listed on thetop-right corner of the page.

Once the game starts, an ambient sound plays to simulate theoceanic climate of the island. The player is the ‘‘trade ruler” ofthe island of his/her choice. The ruler has two tasks. First, the ruleris responsible of managing the island’s production on its labor-intensive (jeans) and capital-intensive (cell phones) products.Some islands are better for manufacturing labor-intensive prod-ucts while others might be advantageous in making capital-con-centrated goods. The second task of the ruler then is to decidewhat to trade with its trading partner, to maximize the islanders’welfare. Based on the rules of Heckscher–Ohlin Theory the gameprovides immediate feedback to the ruler’s trading decisions andit completes one play cycle. Each player has three cycles to accu-mulate as many points as possible. See Appendix A for the screen-shots accompanied by each step.

3.2. Participants and setting

The study recruited undergraduate students from a subject poolof a public Midwestern University in the United States. All partic-ipants were majoring in Education and novice about the topic ofthe GBLE. They accessed the target online GBLE in a laboratory set-ting with minimal interruption. No time limit was imposed for par-ticipants to finish the game. All participants were instructed toread the intended economic theory on the entry page then proceedto the game. After completing the game participants were redi-rected to an online survey program to respond to the motivational

Page 5: Evaluating learners’ motivational and cognitive processing in an online game-based learning environment

Table 2ARCS levels by components.

698 W.-H. Huang / Computers in Human Behavior 27 (2011) 694–704

processing and cognitive load survey. At the end 144 sets of re-sponses were valid for data analysis.

ARCS components Mean score

Attention 5.68Relevance 5.51Confidence 6.20Satisfaction 5.28

3.3. Instrument and data analysis

For motivational processing, this study used a validated IMMSscale (Huang et al., 2006) consisting of 20 items. Minor modificationswere incorporated to accommodate the online GBLE setting. The ori-ginal grammatical structure and tone of each survey item was pur-posefully maintained to align with the research questions. Uponthe analysis of the scale’s reliability, scores of all ARCS componentswere calculated to indicate the level of motivational processing.For learners’ cognitive processing, this study asked participants toself-report the mental effort investment level and the difficulty levelassociated with the learning task on a 9-point symmetrical Likertscale. See Table 1 for the all employed items and reported ratings.

To identify the potential relationship between learners’ motiva-tional processing and cognitive processing suggested by the theoryof MVP, the study conducted a canonical correlation analysis toinvestigate preliminary associations between two sets of multiplevariables (Newton & Rudestam, 1998, p. 278). According to the re-search question, the first set of variables was derived from themotivational processing components (attention, relevance, confi-dence, satisfaction) and the second set of variables was from learn-ers’ self-reported mental efforts (intrinsic cognitive load) anddifficulty ratings (germane cognitive load).

4. Results

4.1. Scale reliability

The overall reliability of the scale on standardized Cronbach’sAlpha is .91 (n = 144 on 20 items), which indicated a good reliabil-ity of the scale. On the 9-point Likert Scale the highest item mean is

Table 1Reported levels by items.

Items for motivational processing

AttentionThere was something interesting at the beginning of the game that got my attention

Absolutely disagree (1) � Absolutely agree (9)The interface design of the game is eye-catchingThe quality of the writing in the game helped to hold my attentionI enjoyed the game so much that I would like to know more about his topicThe way the information is arranged in the game helped keep my attentionThe game has things that stimulated my curiosityI really enjoyed learning with the gameThe wording of feedback after the exercises, or of other comments in the game, helpThe variety of reading passages, exercises, illustrations, etc., helped keep my attentioI could relate the content of the game to things I have seen, done or thought about iIt was a pleasure to work on such a well-designed game

RelevanceIt is clear to me how the content of the game is related to things I already knowThere were examples that showed me how the game could be important to some peThe content of the game is relevant to my interests

ConfidenceThe game was more difficult to understand than I would like for it to beThe game had so much information that it was hard to pick out and remember the iThe game is so abstract that it was hard to keep my attention on itThe exercises in the game were too difficultI could not really understand quite a bit of the material in the game

SatisfactionIt felt good to successfully complete the game

Items for cognitive processingHow much mental effort did you invest to learn the content from the game?

Very very low mental effort (1) � Very very high mental effort (9)How difficult was it for you to learn the content from the game?

Very very easy (1) � Very very difficult (9)

6.62 while the lowest is 4.17. See Table 1 for the tabulated resultsby items.

4.2. Levels of motivational processing

Based on the scale validated in the previous study, the attentionsubscale has 11 items; confidence subscale has five items; and therelevance subscale has three items. Each subscale generated anaveraged score based on 144 participants’ responses on includeditems. The confidence scale has the highest mean of 6.20 whilethe relevance scale has the lowest mean at 5.51. All average scores,nevertheless, were above the mid-point of the 9-point scale. SeeTable 2 for the averaged ARC levels. To identify if the difference be-tween subscales is significant, paired-sample t-test was employedat p = .05. The results reported that the confidence level was signif-icantly different from the rest of the motivational processing com-ponents; the attention component was different from theconfidence and the satisfaction components; the relevance compo-nent was only different from the confidence component; and thesatisfaction component was different from the attention and confi-dence components. This finding suggested a need to further exam-ine the role of relevance during motivational processing and itsrelationship with outcome processing. Because the relevance com-ponent, in contrast with the attention and confidence componentsthat are intrinsic in nature (Malone & Lepper, 1987), might be partof extrinsic motives (Ryan & Deci, 2000), it is possible that the rel-evance component needs to be extracted from the motivational

Reported level 9-point scale

. 5.78

5.925.634.415.825.775.53

ed me feel rewarded for my effort 5.69n on the game 5.73n my own life 4.89

6.09

5.88ople in the learning setting 6.14

4.17

4.97mportant points 6.24

6.546.626.00

5.18

5.07

4.50

Page 6: Evaluating learners’ motivational and cognitive processing in an online game-based learning environment

Table 3Paired t-test results.

Pair t df Sig. (p = .05) Cohen’s d

A–R 1.61 143 .11 .13A–C �3.91 143 .00* .32A–S 2.53 143 .01* .21R–C �5.03 143 .00* .42R–S 1.27 143 .21 .11C–S 4.80 143 .00* .40

* Significant at p = .05.

Table 4Tests of canonical dimensions.

Dimension Canonical corr. F df 1 df 2 p

1 .35 2.97 8 276 .003*2 .19 1.75 3 139 .159

Fig. 1. ARCS levels.

W.-H. Huang / Computers in Human Behavior 27 (2011) 694–704 699

processing stage in the theory of MVP. Table 3 shows the results ofpaired-sample t-test.

4.3. Perceived cognitive load

At the end of the study participants self-reported to two itemsmeasuring the level of intrinsic cognitive load (mental effort invest-ment) and germane cognitive load (difficulty rating). The result indi-cated that participants perceived a higher level of mental effortinvestment than the task difficulty level. A paired t-test further con-firmed that both scores are significantly different from each other(t = 3.77, p < .01). See Table 1 for the item content and scores. Thisfinding confirmed the novelty of the subject matter for our partici-pants as supported by the high intrinsic load. The germane load thatindicates the depth of the learning, however, only suggested a rathershallow learning experience for our participants.

4.4. Canonical correlation between motivational processing andcognitive processing

Tests of dimensionality for the canonical correlation analysis, asshown in Table 4, indicated that the canonical correlation betweenmotivational processing and cognitive processing is significant withone dimension (p < .05) (Dimension 1). Dimension 1 reported acanonical correlation of .35 between two sets of variables. In termsof original variables’ importance in predicting the identified canon-ical correlation, the attention, confidence, and mental effort invest-ment components were found to positively contribute to thecanonical correlation. While identifying the underlying constructsof canonical variates, structure correlation analysis indicated thatthe attention, confidence, and mental effort investment might beessential for their individual variate. Table 5 shows the standardizedcanonical coefficients, structure correlation coefficients, and theredundancy index. The finding of canonical correlation mainly im-plied an underlying relationship between motivational processing

Table 5Standardized canonical coefficients.

Dimension 1*

Canonical coefficient

Motivational processing componentsAttention 1.08Relevance �.42Confidence .40Satisfaction �.48Cognitive processing componentsMental effort investment .92Difficulty level �.65

and cognitive processing in the GBLE of this study. Future researchwith a larger sample size might be able to identify significant corre-lations between the two constructs in GBLEs (Fig. 1).

5. Discussion

5.1. Motivational processing in GBLE

Overall the result of motivational processing in Trade Ruler wasbeyond average. The study identified that learners perceived a sig-nificantly higher level of confidence during the learning process.Learners’ responses toward attention and relevance components,however, seemed to be similar, which were both lower than theconfidence level perceived by learners.

The design of the Trade Ruler game might have helped learnersbelieve that they can accomplish the learning task easily as indi-cated by their confidence levels. For example, the game uses com-mon terminologies to convey complex concepts. An ‘‘island”simplifies the concept of countries involved in the internationaltrading process. Players needed to choose their own island whileeach island is differentiated by its abilities to produce ‘‘jeans”and ‘‘cell phones”, which uses simple items to represent complexconcepts of international trading. The jeans stand for labor-inten-sive manufacturing capacity while the cell phone symbolizes cap-ital-intensive industries. Other game features could also sustainplayers’ confidence level. First, players receive immediate and con-structive feedback from the game on their manufacturing and trad-ing decisions either from the news reporter or the reaction ofcitizens. Second, players have three opportunities to trade withtheir selected trading partners. They can learn from their previoussuccess or failures to inform follow-up decisions.

Participant’s attention level was reported the second highestamong ARC components for motivational processing. But it is onlystatistically different from the confidence component. The targetGBLE has provided several features to support learners’ attentionlevel. In the beginning, the game uses oceanic ambience audioand brief animation to open the entry page. Later players get to

Structure correlation Redundancy index

.05.78.22.58�.03

.05.77�.44

Page 7: Evaluating learners’ motivational and cognitive processing in an online game-based learning environment

700 W.-H. Huang / Computers in Human Behavior 27 (2011) 694–704

design his or her characters in the game by customizing the char-acters’ hair, facial features, clothing, and names. Finally, the gameuses vivid color contrast for all visual representations and its pagelayout remains consistent throughout the game, which preventsthe game from overloading players’ visual modality in processinginformation. In this study learners reported a moderately high le-vel of attention to initiate their exploratory activities. The attentionlevel represents learners’ curiosity induced by the GBLE. Too muchof it in the early stage of the motivational process could distractlearners from the intended learning task (Keller, 2008). Consider-ing learners’ significantly higher confidence level later in the moti-vational processing, this finding implies that the attention stimulilevel might be appropriate in the GBLE.

Although participants reported a moderate to high levels on theconfidence and attention components, they did not perceive a sig-nificantly high level of relevance from interacting with the game.The novelty of the game content might contribute to the finding.By reviewing the items on relevance component, it is clear thatalthough learners were moderately interested in the value of thegame content for other people, they did not see the immediate im-pact of understanding international trade theory in their own lives.The self-determination theory (Ryan & Deci, 2000) also could par-tially explain the finding. The theory considers relevance as part ofextrinsic motivation to drive learner behaviors and it often de-mands a high level of contextual support to help learners internal-ize the behavior. In other words, the design of GBLE in this studymight need to strengthen the connection between the subject mat-ter and learners’ experiences and needs.

The perceived satisfaction level reported from participants wasthe lowest when compared to the ARC components. The t-test sug-gested that it is only significantly different from the attention andconfidence components. Satisfaction, in the context of MVP theory,is the result of learners’ cognitive evaluation on the discrepancybetween invested efforts and perceived outcome. A substantiallearning outcome coupled with a small effort investment, for in-stance, can increase the satisfaction level. In this study learners re-ported a moderate level of satisfaction that is higher than the mid-point of the scale, which implies that learners might consider thelearning outcome a fair result of their invested efforts. This finding,however, presents a sustainability issue. Given learners were onlymoderately satisfied with the learning experiences, they might notcome back to the same GBLE for future learning tasks.

5.2. Cognitive processing and learners’ motivation in online GBLE

As indicated in the integrative theory of motivation, volition,and performance (MVP) (Keller, 2008), a full motivational learningcycle should begin with the motivational processing that helpslearners identify the performance goals and end with an outcomeprocessing with learners’ satisfaction level towards the learningprocess. The motivational processing’s goal is to synthesize learn-ers’ interests, motives, values, and expectancies thus confirms theirintention to participate in the learning process further. In the con-text of ARCS model of motivational design, the motivational pro-cessing refers to the attention, relevance, and confidencecomponents respectively. The satisfaction component, based onthe theory of MVP, becomes the result of the outcome processingat the end of the motivational learning cycle. Between the motiva-tional processing (ARC) and outcome processing (S) are series ofinterrelated learning activities that demand learners’ cognitiveinformation processing capacity. These cognitive processing activ-ities, therefore, could impact learners’ perceived satisfaction levels.Learners with a high level of intention to pursue the performancegoal could be either encouraged or frustrated at the end of thelearning process due to experienced cognitive processing activities.In this study learners started out with an effective motivational

processing that contributed to continuously explore the learningtask. At the end of the learning process, however, they reported arelatively low level of satisfaction as the result of the outcome pro-cessing. This finding can be explained by theories reviewed previ-ously. Considering the MVP theory (Keller, 2008), although thestages between motivational processing and outcome processingserve numerous purposes, all of them require cognitive processingcapacity. For instance, the volitional control processing stagewould need learners’ mental effort to convert intentions into ac-tions (Gollwitzer, 1999). Clearly learners’ cognitive capacities werein high demand in the online GBLE. Since researchers on cognitiveload have concluded that an overloaded cognitive capacity can de-motivate learners (Sweller et al., 1998), this paper argues that thetarget online GBLE might overload learners’ cognitive capacity thuslead to a fairly unsatisfactory learning experience.

5.3. Relationship between motivational processing and cognitiveprocessing

The canonical relationship between motivational processingand cognitive processing was found significant in the study. Acanonical coefficient of .35 in Dimension 1, considered by many,is sufficient for further empirical investigation (Garson, 2008). Ren-cher (2002) argued that the redundancy index calculation is notappropriate for multivariate analysis, which might contribute tothe low redundancy indexes found in the study. Furthermore, thestructure correlation analysis showed promising results on how ori-ginal variables of motivational processing and cognitive processingcould impact the canonical correlation between the two constructswith exceptions of satisfaction and task difficulty variables. Basedon the theory of MVP, learners’ motivational processing and cogni-tive processing contribute to the final perceived satisfaction level,which suggests satisfaction’s independent role in forming thecanonical variate from either side. With regards to the perceivedtask difficulty, the finding, to a large extent, concurs with previouscognitive load studies. Because the germane cognitive load (mea-sured by self-reported task difficulty) is part of the overall cognitiveload (measured by mental effort investment) thus it does not di-rectly contribute to the canonical variate. On the other hand, theattention and confidence components of motivational processingand mental effort investment of cognitive processing demonstrateda strong association with their individual canonical variates (i.e., thecanonical structure loading >.30) (Garson, 2008). This finding sug-gests their potential impact in managing the canonical correlationsince they could substantially affect the composition of individualvariates that enable the canonical correlation. In summary the find-ings preliminarily support the relationship suggested by the theoryof MVP that learners’ motivational processing could impact theircognitive information processing effort. Moreover, the findingsempirically validate a previous conceptual framework in calculatingthe relationship between learners’ motivational involvement andmental effort investment in the context of cognitive load (Paas,Tuovinen, van Merriënboer, & Darabi, 2005). The results of thisstudy, however, did not conclude the direction of the effect betweenmotivational and cognitive processing. Likewise the roles of bothprocessing in supporting learners’ perceived satisfactory learningexperiences are beyond the scope of this study.

6. Conclusion and future research

Grounded in the theory of MVP, the study empirically con-firmed the underlying relationship between learners’ motivationalprocessing and cognitive processing in an online GBLE. The find-ings further present new research directions to investigate thedynamics between learners’ motivational processing and their

Page 8: Evaluating learners’ motivational and cognitive processing in an online game-based learning environment

W.-H. Huang / Computers in Human Behavior 27 (2011) 694–704 701

cognitive activities. Future research should focus on the utilizationof game characteristics in online GBLEs to better manage learners’motivational level, volitional control, and cognitive activities.

Appendix A

1. The starting page of the game.

2. The welcome page of the game that states the goal. The bot-tom of the page also shows the dashboard for the player. Atthe top the players controls the volume of the game’s audiocomponent.

3. On this page the player can select his or her island to rule.Note that each island has different allocation of labor andcapital resources that impact its production capabilities. Inthis case the player selects the Pink Island.

4. After selecting the island, the player can customize his or heravatar in the game and name it.

5. On these pages the player gets to choose the trading partnerbased on its labor and capital resources. In this case theplayer chose the Yellow Island that has less labor resourcebut more capital capacity than the Pink Island.

Page 9: Evaluating learners’ motivational and cognitive processing in an online game-based learning environment

702 W.-H. Huang / Computers in Human Behavior 27 (2011) 694–704

6. On this page the player receives the specific of his or hergame tasks.

7. This page presents the trade ruler’s command center. All deci-sion-making and feedback are delivered via the big monitoron the right side of the room. The player needs to click onthe production, trade, and council buttons to decide what toproduce, what to trade, and receive feedback from the game.

8. The player clicks the production button to decide what tomanufacture. On the page the player can adjust the produc-tion levels of jeans and cell phones in response to the island’slabor and capital resources. Note that the value of goods alsochanges based on the adjustment. The player should seek forthe highest goods value as the result of this activity.

9. On this page the game provides immediate feedback to yourproduction adjustment decision.

10. After receiving the feedback, the player clicks the trade but-ton to decide what to trade. On this page the player can ad-just the amount of jeans and cell phones to be sold to thetrading partner. In this case the ruler decided to trade sixpairs of jeans for six cell phones with the partner (YellowIsland).

11. On this page the trading partner provides feedback to thetrade adjustment. In this case the Yellow Island liked thedeal.

Page 10: Evaluating learners’ motivational and cognitive processing in an online game-based learning environment

W.-H. Huang / Computers in Human Behavior 27 (2011) 694–704 703

12. At the end of the 1st play cycle, the player can review theoutcome of the trade decision. Note that in the result arethe citizens of the Pink Island were neither happy nor upsetabout the trade outcome.

13. The player clicks on the council button to resume the 2ndplay cycle.

14. At the end of three cycles, the player receives feedback fromthe game by analyzing the player’s production and tradingdecisions against the theory.

15. Finally, the player gets to be compared to other players whohave played the game. Here shows an award stand with theplayers.

References

Ames, C. (1992). Classroom: goals, structures, and student motivation. Journal ofEducational Psychology, 84, 261–271.

Anderman, E. M., & Maehr, M. L. (1994). Motivation and schooling in the middlegrades, review of educational research. Journal of Educational Psychology, 64,287–309.

Ang, C. S., Zaphiris, P., & Mahmood, S. (2007). A model of cognitive loads inmassively multiplayer online role playing games. Interacting with Computers,19(2), 167–179.

Apt, C. C. (1970). Serious games: The art and science of games that simulate life inindustry, Government and Education. New York: Viking.

Armstrong, A. M. (1989). Persistence and the causal perception of failure: Modifyingcognitive attributions. Journal of Educational Psychology, 70, 154–166.

Arnone, M. P. (2003). Using instructional design strategies to foster curiosity. (ERICClearinghouse of Information and Technology, Syracuse, NY. No. ED 479842).

Astleitner, H., & Wiesner, C. (2004). An integrated model of multimedia learning andmotivation. Journal of Educational Multimedia and Hypermedia, 13(1), 3–21.

Avedon, E., & Sutton-Smith, B. (1971). The study of games. New York: Wiley.Ayres, P., & Paas, F. (2007). Making instructional animations more effective: A

cognitive load approach. Applied Cognitive Psychology, 21, 695–700.Ayres, P., Kalyuga, S., Marcus, N., & Sweller, J. (2005). The conditions under which

instructional animation may be effective. Paper presented at an InternationalWorkshop and Mini-conference, Open University of the Netherlands: Heerlen,The Netherlands.

Baird, J. R., & White, R. T. (1982). Promoting self-control of learning. InstructionalScience, 11, 227–247.

Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman.Berliner, D. C., & Gage, N. L. (1998). Educational psychology. Boston, NY: Houghton

Mifflin Company.Brünken, R., Plass, J., & Leutner, D. (2003). Direct measurement of cognitive load in

multimedia learning. Educational Psychologist, 38, 53–61.Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of

instruction. Cognition and Instruction, 8, 293–332.Chang, M., & Lehman, J. D. (2002). Learning foreign language through an interactive

multimedia program: An experimental study on the effects of the relevancecomponent of the ARCS model. CALICO Journal, 20, 81–98.

ChanLin, Lih.-Juan. (2009). Applying motivational analysis in a Web-based course.Innovations in Education and training International, 46(1), 91–103.

Cheng, Y. C., & Yeh, H. T. (2009). From concepts of motivation to its application ininstructional design: Reconsidering motivation from an instructional designperspective. British Journal of Educational Technology, 40, 597–605.

Chyung, S. Y. (2001). Systematic and systemic approaches to reducing attrition ratesin online higher education. American Journal of Distance Education, 15, 36–49.

Ciernak, G., Scheiter, K., & Gerjets, P. (2009). Explaining the split-attention effect: Isthe reduction of extraneous cognitive load accompanied by an increase ingermane cognitive load? Computers in Human Behavior, 25(2), 315–324.

de Crook, M. B. M., van Merriënboer, J. J. G., & Paas, F. G. W. C. (1998). High versuslow contextual interference in simulation-based training of troubleshootingskills: Effects on transfer performance and invested mental effort. Computers inHuman Behavior, 14, 249–267.

Deimann, M., & Keller, J. M. (2006). Volitional aspects of multimedia learning.Journal of Educational Multimedia and Hypermedia, 15, 137–158.

Dempsey, J. V., & Johnson, B. (1998). The development of ARCS gaming scale. Journalof Instructional Psychology, 25, 215–221.

Page 11: Evaluating learners’ motivational and cognitive processing in an online game-based learning environment

704 W.-H. Huang / Computers in Human Behavior 27 (2011) 694–704

DeLeeuw, L. E., & Mayer, R. E. (2008). A comparison of three measures ofcognitive load: Evidence for separable measures of intrinsic, extraneous, andgermane load. Journal of Educational Psychology, 1, 223–234.

Dickey, M. D. (2007). Game design and learning: A conjectural analysis of howmassively multiple online role-playing games (MMORPGs) foster intrinsicmotivation. Educational Technology Research and Development, 55(3), 253–273.

Driscoll, M. P. (2000). Introduction to theories of learning and instruction. In M. P.Driscoll (Ed.), Psychology of learning for instruction (2nd ed., pp. 3–28). Boston,MA: Allyn and Bacon.

Garson, G. David (2008). Canonical correlation, from Statnotes: Topics in MultivariateAnalysis. Available from http://faculty.chass.ncsu.edu/garson/pa765/statnote.htm. Retrieved 09/29/2009.

Gee, J. P. (2003). What video games have to teach us about learning and literacy. NewYork: Palgrave.

Gollwitzer, P. M. (1999). Implementation intentions. American Psychologists, 54(7),493–503.

Gredler, M. (1994). Designing and evaluating games and simulations. Houston: Gulf.House, J. D. (2003). Instructional activities and interest in science learning for

adolescent students in Japan and the United States: Findings from the thirdinternational mathematics and science study (TIMSS). International Journal ofInstructional Media, 30, 429–443.

Huang, W., & Johnson, J. (2002). Motivational level of a computer-based simulation:A formative evaluation of the US Army Recruiting Simulation (USAREC). Paperpresented at the Annual Convention of the Association for EducationalCommunication and Technology, Dallas, TX.

Huang, W., & Aragon, S. (2009). An integrated evaluation approach for E-learningsystems in career and technical education. In W. Victor (Ed.), Handbook ofresearch on E-learning applications for career and technical education: Technologiesfor vocational training. Hershey PA, USA: IGI Global.

Huang, W., & Johnson, T. (2008). Instructional game design using cognitive loadtheory. In R. Ferdig (Ed.), Handbook of research on effective electronic gaming ineducation (pp. 1143–1165). Hershey PA, USA: Information Science Reference.

Huang, W., Huang, W. Y., Diefes-Dux, H., & Imbrie, P. K. (2006). A preliminaryvalidation of attention, relevance, confidence, and satisfaction model-basedinstructional material motivational survey in a computer-based tutorial setting.British Journal of Educational Technology, 37, 243–259.

Jacobson, T. E., & Xu, L. (2002). Motivating students in credit-based informationliteracy courses: Theories and practice. Libraries and the Academy, 2, 423–441.

Johnson, T. E., & Huang, W. D. (2008). Complex skills development for today’sworkforce. In D. Ifenthaler, J. M. Spector, & P. Pirnay-Dummer (Eds.),Understanding models for learning and instruction: Essays in Honor of Norbert M.Seel (pp. 305–325). New York City NY, USA: Springer.

Kalyuga, S. (2007). Enhancing instructional efficiency of interactive E-learningenvironments: A cognitive load perspective. Educational Psychology Review, 19,387–399.

Kalyuga, S. (2009). Instructional design for the development of transferableknowledge and skills: A cognitive load perspective. Computers in HumanBehavior, 25, 332–338.

Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention andredundancy in multimedia instruction. Applied Cognitive Psychology, 13, 351–371.

Khalil, M. K., Paas, F., Johnson, T. E., & Payer, A. F. (2005a). Interactive and dynamicvisualizations in teaching and learning of anatomy: A cognitive loadperspectives. The Anatomical Record Part B: New Anatomist, 286B, 8–14.

Khalil, M. K., Paas, F., Johnson, T. E., & Payer, A. F. (2005b). Design of interactive anddynamic anatomical visualizations: The implication of cognitive load theory.The Anatomical Record Part B: New Anatomist, 286B, 15–20.

Keller, J. M. (1983). Motivational design of instruction. In C. M. Reigeluth (Ed.),Instructional design theories and models: An overview of their current status(pp. 386–434). Hillsdale, NJ: Lawrence Erlbaum Associates.

Keller, J. M. (1987a). Strategies for stimulating the motivation to learn. Performanceand Instruction, 26, 1–7.

Keller, J. M. (1987b). The systematic process of motivational design. Performanceand Instruction, 26(9/10), 1–8.

Keller, J. M. (1993). Motivation by design. Florida: Unpublished manuscript, FloridaState University.

Keller, J. M. (2008). An integrative theory of motivation, volition, and performance.Technology, Instruction, Cognition, and Learning, 6, 79–104.

Klein, J. D., & Freitag, E. (1991). Enhancing motivation using an instructional game.Journal of Instructional Psychology, 18, 111–117.

Lee, M. J. (1990). Effects of different loci of instructional control on students’ metacognition and cognition: Learner vs. program control. Paper presented at theAnnual Convention of the Association for Educational Communication andTechnology, (ERIC Document Reproduction Service No. ED 323938).

Malone, T. W., & Lepper, M. R. (1987). Making learning fun. A taxonomy of intrinsicmotivations for learning. In R. E. Snow & M. J. Farr (Eds.), Aptitude, Learning, andInstruction. Volume 3: Cognitive and Affective Process Analyses (pp. 223–253).Hillsdale, NJ: Lawrence Erlbaum.

Marcus, N., Cooper, M., & Sweller, J. (1996). Understanding instructions. Journal ofEducational Psychology, 88, 49–63.

Mayer, R. E. (2001). Multimedia learning. New York: Cambridge University Press.Mayer, R. (2003). Learning and instruction. Upper Saddle River, NJ: Merrill Prentice

Hall.Mayer, R., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia

learning. Educational Psychologist, 38, 43–52.Means, T. B. (1997). Enhancing relevance: Embedded ARCS strategies vs. purpose.

Educational Technology Research and Development, 45, 5–17.

Nelson, B., & Erlandson, B. (2008). Managing cognitive load in educational multi-user virtual environments: Reflection on design practice. Educational TechnologyResearch and Development, 56, 619–641.

Newton, R. R., & Rudestam, K. E. (1998). Your statistical consultant, answers to yourdata analysis questions. Thousand Oaks, CA: Sage Publications.

Paas, F. G. W. C., & van Merriënboer, J. J. G. (1994). Instructional control of cognitiveload in the training of complex cognitive tasks. Educational Psychologist, 6,351–371.

Paas, F. G. W. C., van Merriënboer, J. G. J., & Adam, J. J. (1994). Measurement ofcognitive load in instructional research. Perceptual and Motor Skills, 79, 419–430.

Paas, F., Tuovinen, J. E., Tabbers, H., & van Gerven, P. W. M. (2003). Cognitive loadmeasurement as a means to advance cognitive load theory. EducationalPsychologist, 38, 63–71.

Paas, F., Tuovinen, J. E., van Merriënboer, J. G. J., & Darabi, A. A. (2005). Amotivational perspective on the relation between mental effort andperformance: Optimizing learner involvement in instruction. EducationalTechnology Research & Development, 53(3), 25–34.

Palmgreen, P. (1984). Uses and gratifications: A theoretical perspective. In R. N.Bostrom (Ed.). Communication yearbook (Vol. 8, pp. 61–72). Beverly Hills, CA:Sage Publications.

Pannese, L., & Carlesi, M. (2007). Games and learning come together to maximiseeffectiveness: The challenge of bridging the gap. British Journal of EducationalTechnology, 38(3), 438–454.

Papastergiou, M. (2008). Are computer science and information technology stillmasculine fields? High school students’ perceptions and career choices.Computers and Education, 51(2), 594–608.

Porter, L. W., & Lawler, E. E. III, (1968). Managerial attitudes and performance.Homewood, IL: Richard D. Irwin.

Prensky, M. (2001). Digital game-based learning. New York: McGraw-Hill.Rencher, A. C. (2002). Methods of multivariate analysis. New York: Wiley.Rieber, L. P. (1996). Seriously considering play: Designing interactive learning

environments based on the blending of microworlds, simulations, and games.Educational Technology Research & Development, 44(2), 43–58.

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation ofintrinsic motivation, social development, and well-being. AmericanPsychologists, 55(1), 68–78.

Sachs, J. (2001). A path model for adult learner feedback. Educational Psychology, 21,267–275.

Sankaran, S. R., & Bui, T. (2001). Impact of learning strategies and motivation onperformance: A study in web-based instruction. Journal of InstructionalPsychology, 28, 191–198.

Schunk, D. H. (1990). Goal setting and self-efficacy during self-regulated learning.Educational Psychologist, 25, 71–86.

Small, R. V., & Ferreira, S. M. (1994). Information location and use, motivation, andlearning patterns when using print or multimedia information resources.Journal of Educational Multimedia and Hypermedia, 3, 251–273.

Small, R. V., & Gluck, M. (1994). The relationship of motivational conditions toeffective instructional attributes: A magnitude scaling approach. EducationalTechnology, 34(8), 33–40.

Small, R. V. (2000). Motivation in instructional design. Teacher Librarian, 27, 29–31.Song, S. H., & Keller, J. M. (2001). Effectiveness of motivationally adaptive computer-

assisted instruction on the dynamic aspects of motivation. EducationalTechnology Research and Development, 49, 5–22.

Spitzer, D. R. (1996). Motivation: the neglected factor in instructional design.Educational Technology, 36, 45–49.

Steers, R. M., & Porter, L. M. (1983). Motivation and work behavior. New York:McGraw-Hill.

Suits, B. (1978). The Grasshopper: Games, life, and Utopia. Ontario, CA: University ofToronto Press.

Sweller van Merriënboer , J. J. G., & Paas, F. G. W. C. . Cognitive architecture andinstructional design. Educational Psychology Review, 10(3), 251–296.

van Gerven, P. W. M., Paas, F., van Merriënboer, J. J. G., & Schmidt, H. G. (2006).Modality and variability as factors in training the elderly. Applied CognitivePsychology, 20, 311–320.

van Merriënboer, J. J. G., Clark, R. E., & De Croock, M. B. M. (2002). Blueprints forcomplex learning: The 4C/ID-model. Educational Technology Research andDevelopment, 50, 39–64.

Vroom, V. (1964). The motivation to work. New York: Wiley.Warschauer, M. (2007). The paradoxical future of digital learning. Learning Inquiry,

1, 41–49.Weiner, B. (1985). An attributional theory of achievement motivation. Psychology

Review, 92, 548–573.Wongwiwatthananukit, S., & Popovick, N. G. (2000). Applying the ARCS model of

motivational design to pharmaceutical education. American Journal ofPharmaceutical Education, 64(2), 188–196.

Zimmerman, B. J., & Martinez-Pons, M. (1988). Construct validation of a strategymodel of student self-regulated learning. Journal of Educational Psychology, 80,284–290.

Zimmerman, B. J. (1989). Models for self-regulated learning and academicachievement. In B. J. Zimmerman & D. H. Schunk (Eds.), Self-regulated learningand academic achievement: Theory, research, and practice (pp. 1–25). New York:Springer.

Zumbach, J., & Mohraz, M. (2008). Cognitive load in hypermedia readingcomprehension: Influence of text type and linearity. Computers in HumanBehavior, 24, 875–887.