Educational Research Review 2 (2007) 1–12

Neuroimaging of cognitive load in instructional multimedia

Robert R. Whelan ∗Centre for Educational Development and Technology, University of the South Pacific, Suva, Fiji

Received 26 April 2006; received in revised form 28 November 2006; accepted 29 November 2006

Abstract

This paper reviews research literature on cognitive load measurement in learning and neuroimaging, and describes a mappingbetween the main elements of cognitive load theory and findings in functional neuroanatomy. It is argued that these findings maylead to the improved measurement of cognitive load using neuroimaging. The paper describes how current measures of cognitiveload cannot accurately show the distinction between different types of cognitive load in different learning conditions, and existingapproaches to cognitive load assessment are limited in terms of their precision and methodology. A literature review discussesthe conceptual framework of Sweller’s [Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design.Learning and Instruction 4, 295–312; Sweller, J. (1999). Instructional design in technical areas. Camberwell, Australia: ACERPress] cognitive load theory, and describes various approaches to load measurement and their limitations. The paper then describeshow the core components of cognitive load – intrinsic, extraneous, and germane load – may be observable using neuroimagingtechniques, and argues for the exploration of new links between education research and neuroscience.© 2006 Elsevier Ltd. All rights reserved.

Keywords: Cognitive load; Neuroimaging; fMRI; Multimedia learning

1. Introduction: cognitive load theory and measurement

A growing body of research in the field of cognitive load measurement in instruction aims to provide improvedapproaches to the design and evaluation of instructional materials (Brunken, Plass, & Leutner, 2003; Kalyuga &Sweller, 2005; Paas, 1993; Paas, Tuovinen, Tabbers, & van Gerven, 2003). This paper reviews key contributions andobstacles in this area, and describes a mapping between the main components of cognitive load theory on one hand,and functional neuroanatomical features of the brain on the other, that may offer a more effective approach to cognitiveload measurement.

The aim of the paper is to lay the theoretical groundwork for a new approach to the measurement of cognitive loadin instruction, an approach that it more accurate, reliable and less invasive than existing approaches. The methodologyused in the preparation of this paper involved broad literature review in the fields of cognitive load measurementand functional magnetic imaging of high-level cognitive processes. Research into the literature on the imaging ofcognitive processes was based on reviews of the key journals in the field, including Brain & Cognition, CognitiveBrain Research, Human Brain Mapping, the Journal of Cognitive Neuroscience, Memory & Cognition, NeuroImage,Neuron, Neuroscience, and others, over a three-year period. The author has also conducted wide ranging practical andexperimental work on the measurement of cognitive load in multimedia learning, some of which is reported below.

∗ Tel.: +679 323 2041; fax: +679 323 1539.E-mail address: robert.whelan@usp.ac.fj.

1747-938X/$ – see front matter © 2006 Elsevier Ltd. All rights reserved.doi:10.1016/j.edurev.2006.11.001

2 R.R. Whelan / Educational Research Review 2 (2007) 1–12

2. Cognitive load theory

John Sweller outlined the theoretical framework behind cognitive load by drawing distinctions between intrinsic,germane, and extraneous cognitive load in the design of instructional materials (1994, 1999). According to his theory,intrinsic load is defined as fundamental to understanding a concept, and it is a function of the inherent complexityor element interactivity of a concept or schema. Extraneous load, by contrast, is defined in terms of the relationshipbetween design elements and their presentation, and is evident in poorly designed instruction. Germane load, on theother hand, relates to the degree of learner effort involved in the construction and automation of schemas, and it isassociated with factors such as motivation and interest on the part of the student. In this respect, intrinsic and extraneousload most closely reflect objective instructional design factors, and they stand apart from germane load, which wouldtypically be defined in terms of a learner’s subjective experience.

Sweller argued however that extraneous and germane load can be manipulated by the instructional designer,whereas intrinsic load is an immutable feature of a concept (although recent work by Ayres, 2006; Gerjets,Scheiter, & Catrambone, 2004 indicates that this may not be the case). Sweller also described the role of schemaacquisition as central to learning and skill development. Schemas are the basic unit of analysis and evaluationin instructional design. Schema acquisition and automation not only reflect the emergence of expertise, but alsoallow a learner to reduce the load on working memory since schemas can be grouped analogously to ‘chunks’ ofinformation, and can become managed as individual units in working memory rather than as more complex col-lections of individual units. According to Sweller then, learning is the storage of automated schemas in long-termmemory.

Sweller (1994) uses the concept of element interactivity in schemas to convey the sense of intrinsic load. Elementinteractivity is the interaction between the intrinsic building blocks in instructional material, such that a learner candraw meaning from instructional material. Designers can thus reduce cognitive load and better facilitate learningby controlling extraneous cognitive load and optimizing the element interactivity underlying intrinsic cognitive loadwherever possible. Germane load can also be optimized for learners with motivating, engaging presentation andinstructional contexts.

Cognitive load theory is not without its explanatory limits. Learners in today’s world are adept at highly complexmultiple task performance that blurs the lines between intrinsic, germane, and extraneous cognitive load in practice.Cognitive load theory does not translate readily into a ‘full-spectrum’ interpretative device. For example, recent studiesshow (Whelan, 2006) that when learning materials are too easy, collecting reliable measures of cognitive load is muchmore difficult. When learners are less challenged, their reports of the load reveal an under-load effect, where the greaterrange of variance in the dependent measures makes it less easy to reliably address questions about the predictive validityof load measurement instruments for design evaluation.

This relates to what Sweller (1999) describes as the “low cognitive load effect,” and in the research of Kalyuga,Ayres, Chandler, and Sweller (2002), to the expertise reversal effect. While Kalyuga’s research is invaluable in its ownright, the main point is that instructional material has to be challenging at a certain minimum threshold in order tofall under the remit of the cognitive load paradigm. Below this threshold, the cognitive load is too light to supportmeaningful assessment of load or to offer significant input for improved learning design.

A more specific critique of cognitive load theory is that there can appear to be an overlap between the different formsof postulated load in instructional material: what seems like intrinsic load may at times be extraneous load, or viceversa. For example, the interpretation of complex mathematical notations may be hindered by a poorly chosen font,while ambiguous or unclear images may obscure the intended message of an illustration. While it is true that intrinsicand extraneous load can be difficult to separate out in certain conditions, and debate exists about the operationalizationof germane load in particular (Kalyuga, in correspondence), it is nevertheless these three theoretical constructs that canmost usefully frame the design problem about the difficulty of (multimedia) learning materials in a way that contributesto their improved design, as a growing body of research demonstrates.

3. Measuring cognitive load

Notwithstanding certain considerations about the scope and consistency of cognitive load theory, it follows fromthis theoretical framework that the quantification or measurement of cognitive load during the instructional designprocess is of fundamental importance for the success of the learning materials.

R.R. Whelan / Educational Research Review 2 (2007) 1–12 3

A growing body of research is available about different approaches for measuring cognitive load. Paas (1993)and Paas et al. (2003) have reported extensively on the use of approaches such as expert opinion and task analysis,introspective rating scales, performance-related data, and psycho-physiological data such as heart rate, pupil dilation,and galvanic skin response. However, current measures of cognitive load do not accurately show the distinctionbetween different types of cognitive load in different learning conditions, and existing approaches to cognitive loadassessment are limited in terms of their precision and methodology. The following section discusses the most widelyused approaches to cognitive load measurement.

3.1. Dual task paradigm

In the dual task load measurement paradigm, a learner’s performance on a second task is assumed to be a directrepresentation of the degree of cognitive load imposed by the primary task. For example, the secondary task is usuallya timed response to a visual or auditory cue, where typical responses range from 300 ms to over a second, dependingon the nature of the primary task. In their recent research on cognitive load measurement using this technique, Brunkenet al. (2003, p. 12) explain that, “The secondary task approach can be used in two different ways. In one approach,a secondary task is added to a primary task with the intention of inducing memory load. The dependent variable ofinterest is the performance in the primary task. A second approach is to use a secondary task for the measurement ofthe memory load induced by a primary task. Here, the performance in the secondary task is the variable of interest. Ifdifferent variants of a primary task induce different amounts of memory load, then the performance in the secondarytask will vary accordingly”. The secondary task approach offers advantages over other approaches, including its abilityto give immediate real-time indications of cognitive load with a high degree of sensitivity, as well as its effectivenessin a within-subject design that makes the measurement of cognitive load independent of individual differences thatwould corrupt a between-subjects design.

However, the secondary task technique has certain limitations, which Meshkati and Loewenthal (1988) point out.One is the risk of the secondary task’s intrusion into the primary task, which can lead to changes of strategy bythe subject that can distort the performance and structure of the primary task. The secondary task yields maximalinterference when the response modalities are the same. For instance, complex tasks have high structural complexitythat limits the sharing of attention between task inputs. Indeed, the nature of the primary task can impact the utility andefficiency of the secondary task. Thus, the choice of the secondary task’s modality is a crucial determinant in researchinvolving this method. Another concern is that the secondary task risks overloading the subject, and distorting primarytask performance. Meshkati and Loewenthal argue that secondary tasks should not interfere with the primary task, theyought to be easy to learn, self-pacing, constant, and compatible with the primary task. Thus, an efficient secondarytask might be a visual or auditory monitoring task, where subjects must respond as quickly as possible to a particularsignal. A less efficient secondary task might be to solve a complex mathematical problem.

A further consideration is that secondary task performance will vary between individuals because the task invokesarousal, and different personalities will respond to this in different ways. For example, ‘Type A’ personalities performdifferently under high load conditions than Type B (Damos & Bloem, 1985), as do subjects with different decisionstyles (Meshkati & Loewenthal, 1988). Motivation and training on the secondary task can also become problematicalvariables, especially in the context of individual differences and cognitive load perception. For this reason, within-subjects research designs are preferred because the measurement of cognitive load is made independently of individualdifferences that would otherwise corrupt a between-subjects design.

3.2. Physiological measures of cognitive load

Physiological methods for measuring workload assume that task load levels show themselves in the body’s metabolicprocesses, from brain waves to pulse rate, pupil diameter, breathing rate, respiration content, auditory canal temperature,voice pattern, endocrine, and galvanic skin response. Researchers frequently use one, or a combination of thesemeasures to assess load. In one experiment conducted by Paas (1993), for example, involving electrocardiogram (ECG)measurements of learners’ responses to varying conditions of cognitive load, correlations were indeed significant, buttoo low to conclude that heart-rate variability is a useful measure of cognitive load. With regard to the sensitivity ofthe heart rate measure, Paas (1993) found no significant difference between the ECG readings in the instruction andtransfer conditions. Paas concludes that although further work on heart rate as a measure of cognitive load may prove

4 R.R. Whelan / Educational Research Review 2 (2007) 1–12

worthwhile, it is not appropriate in the current context, and may prove more complex to administer and yield lesssatisfactory results. Sensitivity to workload levels is an ongoing concern when using physiological measures, and thesemeasures are prone to contamination from environmental, emotional and physical stresses unrelated to the primarytask (Brunken et al., 2003). These measures are also intrusive and can be an encumbrance during task performance.

3.3. Subjective workload measurement

Although easiest to implement and analyze, subjective methods of cognitive workload measurement are not withoutcontroversy. Subjective methods typically involve asking the participant, in very structured ways, how he or she wouldrate the difficulty of the learning material, or the amount of workload. However, the setting, situation and contextcan often affect the rater’s score as much as the actual task difficulty. “Unless the subjective measures are properlystructured, they may serve only as gross indicators of stress level and have little diagnostic value, i.e., they may notindicate the source or type of workload involved” (Meshkati & Loewenthal, 1988, p. 255).

Here, again, individual differences play a significant role. For example, Meshkati and Loewenthal cite research thatsubjects who score highly on intelligence tests will actually rate a problem as more difficult than subjects who scoreless highly, even though the problems are the same (Borg, Bratfish, & Dorinc, 1971). The approach can also havelimited sensitivity to changes in workload levels; perceived workload may go up as well as down, while performanceremains the same, particularly when tasks present only a low workload. There is also a lack of consistency betweenperformance ratings and subjective ratings of workload, difficulty and effort. Performance measures also tend to betask-specific, they can vary within tasks, and they cannot easily be replicated across unrelated tasks.

For example, Whelan (2006) compared two approaches to the measurement of cognitive load – self-report ques-tionnaires, and the dual task methodology – and concluded that each measure produces a unique response profilethat in most treatment conditions only reflects one dimension of the overall cognitive load in a given instructionaltreatment. The measurement instruments did not produce uniform results, indeed, the instruments tended to capturehighly selective components of cognitive load. For example, the secondary task methodology showed its strength inassessing extraneous load, as reflected in its sensitivity to modality effects, whereas the self-report task complexity index(Braarud, 2001) tended to reflect the degree of intrinsic load, or element interactivity, but it performed erratically in themeasurement of extraneous load. Similarly, the Paas cognitive load questionnaire (Paas, 1993) was sensitive to varia-tions in intrinsic load, and a differential sensitivity to high and low extraneous load conditions was not evident. A thirdself-report questionnaire, Tsang and Valazquez’ (1996) workload profile index, was sensitive to intrinsic load effectsunder low extraneous load conditions, but sensitive to extraneous load effects under high intrinsic load conditions.The conclusion was reached that none of the approaches used are entirely comprehensive in their diagnostic utilityand predictive power for all types of load. Thus, this paper argues that the need to accurately and comprehensivelymeasure cognitive load in the instructional design process encourages the exploration of alternative approaches to themeasurement of cognitive load.

4. Neuroimaging cognitive load

In the last decade, fMRI has evolved into an indispensable tool making possible detailed analysis of cognitivedynamics. Recent advances in this scanning technology have allowed researchers to identify and study specific regionsof the brain that mediate attentional control, working memory and cognitive workload. fMRI is the natural heirto longstanding research in cognition using electro-encephalography (EEG), but with greater resolution, acuity andexplanatory power. While EEG technology is used to measure and record coarse electrical activity in the brain, fMRIcan be used more extensively, to record the brain’s hemodynamic responses and activity at highly detailed, accuratespatial and functional levels (Ogawa, Lee, Nayak, & Glynn, 1990). Neuroscientists have been using this technology tounderstand how the brain functions during learning, and these findings can make a valuable and unique contributionto educational psychologists interested in instructional design.

This paper draws specifically from over a decade of fMRI research into the brain functions underlying the ‘executivecontrol’ and coordination of cognition, including speech comprehension and image recognition, during learning. Themain focus is on the frontal lobes of the brain, where specific control mechanisms (such as the dorsolateral prefrontalcortex, see Fig. 1) are located. Other regions of interest, in the left hemisphere of the brain, roughly behind the ear, areBroca’s and Wernicke’s areas, responsible for speech production and comprehension, respectively. Another important

R.R. Whelan / Educational Research Review 2 (2007) 1–12 5

Fig. 1. The brain viewed from the right side, showing the regions of interest underlying cognitive load: the dorsolateral prefrontal cortex (DLPFC)associated with intrinsic load, the parietal cortex and Wernicke’s area associated with extraneous load, and the superior frontal and intraparietalsulcus associated with germane load. (Image modified from Bain, 2003.)

structure, located in the back half of the brain, is the parietal lobe, which is involved in the integration of sensoryexperience and the processing of visuospatial information. Regions of the brain are closely integrated however, andthey inter-operate and ‘communicate’ through the activation of neural pathways with electrochemical charges thatconsume oxygen carried by hemoglobin in the blood.

The basic principle behind functional magnetic resonance imaging is that active areas of the brain (e.g., duringlearning) will consume more oxygen. This oxygen consumption by neurons (which leaves a tiny magnetic signature)is detected by pulses rebounding in powerful magnetic fields generated by the MRI scanner. Volumetric images of themagnetic fields are then compiled from ‘blood oxygenation level dependent’ (BOLD) contrasts, and from this, a(nanimated, functional) map of the neural correlates of cognitive tasks can be deduced.

Research by educational psychologists into the measurement of cognitive load in instructional design has not yettaken advantage of the new possibilities presented by this technology, although research that links discoveries inneuropsychology with pedagogy offers great promise to educators and learners. Atherton and Bart (2001) propose athree-phase methodological framework for applying neuroimaging-based research to educational practice: discoveryphase (exemplified by specific localization studies), functional analysis phase (including analysis of the dynamicproperties of activations), and pedagogical evaluation phase (which would apply specific mappings from activationpatterns onto learning performance). Atherton and Bart argue that “fMRI has great potential for improving educationalpractice by providing finer grained models of learning and cognition” (p. 12).

This paper argues that cognitive load theory has a basis in functional neuroanatomy, and fMRI techniques will allowus to accurately observe the properties of certain brain functions related to different types of cognitive load. The resultswould allow us to verify key assumptions of cognitive load theory, with broad implications for the measurement andapplication of cognitive load theory and instructional design.

The potential benefits of this line of research include: (1) exploration of a neural basis to intrinsic, extraneousand germane load factors of cognitive load theory; (2) a new approach to the measurement of cognitive load that is

6 R.R. Whelan / Educational Research Review 2 (2007) 1–12

sensitive enough to detect features of cognitive workload that existing methods do not yet capture, and (3) advance inthe debate over the mechanisms underlying cognitive control and working memory. Moreover, these benefits are set ina framework of learning and multimedia instruction, as opposed to neuroscience per se. The remainder of this paperdraws from the research literature to describe the mapping between intrinsic, extraneous and germane cognitive loadand the underlying functional neuroanatomy.

4.1. Neuroimaging intrinsic cognitive load

Intrinsic load is defined in terms of the integral complexity of an idea, concept, or set of concepts that gives it sense.Intrinsic load reflects the absolute difficulty of the material to be learned, or the task to be carried out (Sweller, 1994,1999). Intrinsic load represents the “natural” difficulty of the material and it cannot be altered without also alteringwhat is understood. Recent research by neuroscientists into the mechanisms underlying cognition, though sometimesemploying somewhat different terminology from research by educational psychologists, is adding to a growing bodyof knowledge about how the brain responds to different levels of intrinsic cognitive load.

For example, in neuroimaging studies of working memory, Newman, Just, and Carpenter (2002) analyzed thecollaboration, synchronization, and timing of different neural networks activated by verbal problems designed toinduce high intrinsic cognitive load either at the beginning, or toward the end of a problem-solving task. In the firstkind of problem, high intrinsic load is induced early in the sentence (e.g., “The first month after April is the monthbefore my favorite month”), whereas in the second type of problem, the load is induced later on (e.g., “The day beforemy favorite day is the first day after Monday”).

Newman et al. (2002) used a 1.5-T scanner with 14 participants who underwent 20 trials of each of the two cognitiveload conditions. (Note that what may, to educational psychologists, appear to be relatively small sample sizes, andthe use of right-handed participants, are conventions in neuroimaging research that are due to the methodologicalrequirements of the experimental paradigm.) The regions of interest (ROIs) in the brain were the operculum andtriangularis in the inferior frontal lobe, the intraparietal sulcus and inferior region of the parietal lobe, the frontal eyefield, and the middle frontal gyrus, also known as the dorsolateral prefrontal cortex (DLPFC). The main hypothesis wasthat the two types of problem task would activate the ROIs in a different time course or sequence. The mean number ofactivated voxels (representing the volume of neural activation) was calculated within each ROI for each condition, anda mean time course of the activated voxels in each ROI was calculated for each condition. ANOVAs was conductedwith the effects of condition, ROI and hemisphere as within-subject variables, which showed that the effect of cognitiveload significantly affected the overall amount of activation. The functional connectivity analysis indicated the presenceof two attentional control and working memory networks that were significantly modulated by task difficulty: the leftDLPFC, frontal operculum, left intraparietal sulcus, left inferior parietal lobe and the right intraparietal sulcus formedone “buffering and management” network, while the left intraparietal sulcus and the left opercularis formed the other“calculation” network. Newman et al. concluded that the right DLPFC played a “strategic planning role” (p. 820) inthe problem task, while the left DLPFC was involved in the cognitive control processes necessary to provide top-downsupport” (p. 820). The DLPFC structure was implicated in the management of the buffering, retrieval, and computationof verbal information in the high load condition, while the frontal opercularis was observed to be involved in “goalmanagement.”

In another example of fMRI research involving variable intrinsic task load difficulty, the research team of Banichet al. (2000) used fMRI data from the prefrontal regions of the brain to study the “attentional set” that drives theselection and maintenance of particular task-relevant information during cognition. As the experimental task, the teamused variants of the Stroop task where participants are asked to apply a set of rules to distinguish between a correctand a false condition. Specifically, participants identify the color that a word is printed in while ignoring the worditself, a task that can cause interference if the word “Red” is written in yellow ink for instance. In this case (dubbedan “incongruent” condition), the need for attentional selection – and thus, the cognitive load – is higher than in caseswhere the word’s meaning has nothing to do with the color it is printed in (the “neutral” condition). Another version ofthe Stroop task is designed to observe spatial cognition and involves pictures of objects that might have incongruent orneutral colors, e.g., a yellow-colored cherry. The complex element interactivity of the rules of these tasks create highintrinsic load for the participants.

In their study, Banich et al. (2000) used a 1.5-T scanner and asked 10 right-handed participants to attend first tothe color of the presented item and, in the second condition, to the nature of the item. When participants attended to

R.R. Whelan / Educational Research Review 2 (2007) 1–12 7

color, they had to monitor a series of items (either words or objects) and determine whether any of the items was everpresented in a particular color (i.e., purple), a task which imposed additional cognitive load. Unpaired t-tests were usedto generate statistical probability maps of the on and off phases of voxels that activated in specific regions of interestin the brain.

Banich et al. (2000) confirmed the findings from a number of studies showing that the prefrontal cortex plays amajor role in imposing the “attentional set.” The incongruent trials that had higher demand for cognitive resources werepositively correlated with one frontal lobe structure in particular, the dorsolateral prefrontal cortex (DLPFC), whichwas activated “when there was a need to override automatic or intrinsic attentional biases” (p. 7). Thus, “prefrontalactivation is most likely to occur when that attentional set is difficult to impose” (p. 7). Indeed, Banich et al. arguethat since the right hemisphere DLPFC was activated in both the verbal and spatial conditions during the Stroop task,it may be directly involved in control processes for working memory and cognitive load management. In any case,this frontal lobe structure appears to play a key role in selective attention and in influencing attentional processing andmaintaining the attentional set in cognitive tasks where intrinsic load is varied.

Indeed, a broad review of fMRI literature shows that numerous researchers have put forward corroborating theoriesabout the mechanism underlying the control of intrinsic cognitive load. For example, Braver, Barch and Cohen’s (2003)hypothesis for the management of executive processes is that the DLPFC maintains context-related information thatprovides a mechanism of cognitive load control and interference attenuation by “serving as a top-down bias on thelocal competitive interactions that occur during processing.” They suggest that “the PFC performs both mnemonic andinhibitory functions in the service of control, and that each is preferentially observable under different task situations”(p. 1). Braver and others have utilized different experimental paradigms to show that the DLPFC plays a key rolein reducing interference effects or “crosstalk,” and maintaining stable attentional processes under high intrinsic loadconditions (Barch, Sabb, Braver, & Noll, 2000; Braver et al., 1997; Braver & Speer, 2003; Carpenter, Just, Keller, Eddy,& Thulborn, 1999; D’Esposito, Postle, Ballard, & Lease, 1999; Dobbins, Foley, Schacter, & Wagner, 2002; Miller &Cohen, 2001; Veltman, Rombouts, & Dolanc, 2003).

Based on this review of research in neuroimaging, we propose that intrinsic cognitive load can be observed throughfMRI as the maintenance and manipulation demands placed on the prefrontal cortex, in particular on the DLPFC.

4.2. Neuroimaging extraneous cognitive load

Extraneous cognitive load in learning materials shows itself as the load found in discontiguous or incongruousinformation sources in instructional designs. Extraneous load takes a variety of forms, according to Mayer (1997,2001). For example, the spatial contiguity principle emphasizes that corresponding words and pictures should bepresented near rather than far from each other; the temporal contiguity principle describes how corresponding wordsand pictures should be presented simultaneously rather than successively; the coherence principle states that learningis improved when extraneous words, pictures, and sounds are excluded; and finally, the modality principle underscoresthe importance of distributing information across modalities rather than overloading one modality with too muchinformation (for example, designing audio–visual presentation as opposed to visual-only). This paper argues that theimpact of extraneity on cognition during learning will show itself at the neurological level in a range of ways, butprincipally at the architecture level as opposed to the functional level discussed in the mapping of intrinsic load to theDLPFC. Neuroimaging research literature indicates that extraneous load shows itself as disruption in the activation ofthe sensory modality-specific mechanisms that underlie attentional modulation.

Our review showed that several lines of research in neuroscience are exploring the impact of extraneous load oncognition framed in terms of attentional modulation. For example, Meredith (2001) describes a range of perceptualand behavioral effects related to the phenomena of multisensory convergence in the brain, from object identificationto spatial orientation to language use. Multisensory convergence involves the processing of information by neuronalpathways that are specialized as bi-, tri-, or even multi-modal in their processing capacity. These mechanisms accountin part for the temporal and spatial contiguity principles and the modality principle described in Mayer’s (1997, 2001)multimedia learning research.

“The temporal relationship of different stimuli is critical for the elicitation of response enhancement and stimulicombinations that occur within 100 ms of each other have the highest likelihood of evoking response amplification”(Meredith, 2001, p. 35). Likewise for the spatial contiguity principle, “stimulus combinations that are aligned in spacewill fall within the excitatory receptive field areas and will generate response enhancement” (p. 35). Regarding the

8 R.R. Whelan / Educational Research Review 2 (2007) 1–12

modality effect, Meredith writes that “for selective attention processes, the convergence of excitatory inputs from onemodality with inhibitory inputs from another presents a vigorous model” of attentional modulation (p. 35).

Meredith notes that methodologies for investigating this property are still wanting, but describes various functionalarchitectures and general principles for multisensory convergence. For example, excitatory-excitatory convergencecan lead to amplification of sensory processing signals in neural pathways that respond most strongly to bi-modalinputs. The researchers used t-tests to show a significantly greater response from these specialist pathways to bi-modal rather than uni-modal stimuli. By contrast, excitatory-inhibitory convergence can lead to “active inhibition,”that is, modality-specific processing of multi-modal inputs that preferentially suppresses non-relevant stimuli duringcross-modal stimulation.

This finding concurs with research by Just et al. (2001), who studied the multiple-modality processing constraintsinherent in the brain’s architecture, or how the brain is affected by extraneous load. Their main research question was:how does the brain manage concurrently performed conflicting tasks? They gave their research participants two tasksto perform at the same time: auditory sentence comprehension and mental rotation of visually-depicted 3-D objects.FMRI data was collected for 18 right-handed participants while carrying out the tasks. A key finding was that theamount of brain activation in the most involved areas of the cortex, the temporal and parietal lobes, was substantiallyless in the dual task condition than in the sum of the two single tasks. This applied to the pre-frontal, association, andsensory areas of the brain. Just et al. believe that this phenomenon is evidence of an inbuilt metabolic, neurotransmitter,or neuromodulating constraint on the possible amount of activation in the brain. There is, therefore, a limit on howmuch attention is available for multiple-task performance. Another finding was that participants performed the dualtasks with the same high degrees of accuracy as in the single task, but with lower levels of brain activation.

Thus, in order to adapt to the extraneous load, the dual task brought about the brain’s use of a strategy of faster butaccurate encoding of less information per unit of time. The researchers suggest that “the constraint on co-processingmay apply not to the number of tasks that can be performed simultaneously, but to the amount of computation performedper unit of time in each task” (p. 425).

In a study that addressed a similar research question about the modality specificity of neural architecture, Michael,Keller, Carpenter, and Just (2001) used fMRI to observe language processing in both the visual and auditory modalities.A guiding question was the degree to which high-level cognition involves the sharing or the dissociation of processingresources for a particular modality. The study involved eight right-handed participants who read or listened to sentencesthat were of two levels of complexity and then responded to a true–false question about each one. For these four condi-tions, three primary (18 in total) regions of interest (ROI’s) in the brain were observed in terms of voxel activation witha 3-T scanner (especially the DLPFC, Broca’s area in the left inferior frontal gyrus; and Wernicke’s area in the middletemporal lobe). In addition, the extent of lateralisation was also monitored. The dependent measure was the percentchange in the brain’s signal intensity in the ROI compared to the baseline control condition (fixation on an asterisk). Foreach ROI, an ANOVA was used to compare the data by modality and sentence complexity as within-subjects factors.

Michael et al. (2001) found that many of the voxels in the temporal ROI were modality and condition specific, andthe “centroid of activation” was differently located for the auditory than for the visual modality. In the frontal ROI,however, “the percentage of activated voxels in a given modality or condition is lower, hovering around 50%, a classichalf full, half empty situation in terms of modality specificity. As sentence complexity increased, so did the degree ofneural activation, although this was modulated by the stimulus modality, and the auditory channel consistently evokedthe largest activation. All of these results indicate modality effects at higher levels of processing” (p. 249). A keyfinding was that “the functional impact of sentence structure is independent of the input modality” and that the “neuralsystems subserving listening and reading both have the capability to dynamically recruit additional tissue as the taskdemand increases” (p. 251), or as the extraneous load is added.

Based on this body of research, this paper proposes that extraneous cognitive load can also be observed throughfMRI in terms of architectural constraints in the brain that modulate attention across sensory modalities.

4.3. Neuroimaging germane cognitive load

Germane load is a reflection of the effort involved in the creation and automation of concepts or schemas, and itis often defined in terms of motivational factors that make the content meaningful and relevant for learners. Germaneload is a generalized feature of an individual learner’s experience, and is associated with motivation and overall interestin the topic of study.

R.R. Whelan / Educational Research Review 2 (2007) 1–12 9

Such generic, high-level emotions would seem to be a property of deep brain function, molded by language useand community, and not an object available to direct analysis with neuroimaging. Indeed, some debate exists in theeducational psychology community about the application and operationalization of germane load, and the literatureshows that the bulk of research in the field of cognitive load measurement focuses on intrinsic and extraneous load.

Nevertheless, neuroimaging research has operationalized representations of motivation and related states in thebrain in a variety of ways. Research often involves monetary rewards offered for successful decision-making undercircumstances of high or low risk, and focuses on a range of specific brain pathways such as the thalamic, striatal, andorbitofrontal brain regions, the medial hypothalamic/preoptic area, the midbrain/pons, the midline and intralaminarthalamic nuclei, and the medial prefrontal and anterior cingulate areas (Ernst et al., 2004; Kirsch et al., 2003; Sewards& Sewards, 2003).

Interestingly, recent fMRI research into motivation shows close links with research into intrinsic cognitive load.For example, Taylor et al. (2004) argued that working memory intersects with functions that help determine “valuefor the organism” (p. 1045). Taylor et al. used a trial-related design focusing on the right superior frontal sulcus andbilateral intraparietal sulcus of the brain. They rewarded their participants with either a high or low financial reward,depending on the risk associated with the task. Participants showed greater sensitivity in the high reward condition,and, interestingly, the researchers found an interaction between reward and retrieval from working memory in the rightdorsolateral prefrontal cortex, the same area implicated in intrinsic load management. Taylor et al. found that “whensubjects perform a simple working memory task, financial incentives motivate performance and interact with someof the same neural networks that process various stages of working memory. Areas of overlap and interaction mayintegrate information about value, or they may represent a general effect of motivation increasing neural effort” (p.1054).

5. Can cognitive load be observed?

This paper has sought to inter-weave findings from hitherto largely incommensurate bodies of research in the fieldsof neuroimaging and educational psychology to construct a case for the observation and measurement of cognitive loadin a comprehensive and accurate way using fMRI. It is postulated that changes in intrinsic load manifest themselvesin the functional pathways of the brain that mediate attentional maintenance and manipulation, especially the DLPFC,while extraneous load can be observed in terms of macro-level processing constraints in the brain involving modality-specific neural structures in the posterior parietal cortex and Wernicke’s Area. Germane load can be observed in thebrain structures underlying motivation.

This argument is based on a body of research that is not often explored by educational psychologists working inthe behavioral paradigm. For example, Banich et al. (2000) showed how mechanisms in the brain responsible forattentional control, particularly the dorsolateral prefrontal cortex, play a “strategic role” in managing the attentionalset and mitigating intrinsic load. Meredith (2001) offered an insight into the mechanisms underlying extraneous load,describing the multivalence of neuronal pathways, their dynamic adaptivity and ability to respond to multi-modalstimuli. Michael et al. (2001) found that a “centroid” of activation was located differently for auditory than for visualtasks, and that the sharing of modality-specific resources suggests a utility in functional dissociation and overlap thatcan adapt to extraneous task difficulty.

The next step would be to conduct an exploratory fMRI study focusing on cognitive load representation andmeasurement in the brain that, in the framework proposed by Atherton and Bart (2001) would establish a more solidbasis for a research program, allowing the evidence to advance from a functional analysis phase to a more applied,pedagogical evaluation phase.

6. An exploratory research design

The initial objective of fMRI-based cognitive load measurement would be to use event-related fMRI to investigate thedifferential responses of the pre-frontal cortex (dorsolateral and frontopolar) as compared to the vision and languageprocessing regions of the brain (the posterior parietal cortex, and Wernicke’s Area), under conditions of varyingcognitive load induced with an adapted n-back task (Germane load effects may be better observed with a modifiedform of this design).

10 R.R. Whelan / Educational Research Review 2 (2007) 1–12

The research questions would be: (1) Do the prefrontal cortex and the occipital-parietal lobes show differentialactivation patterns under intrinsic and extraneous load? (2) Can neuroimaging provide evidence of a distinct neuralbasis of intrinsic and extraneous cognitive load control? (3) How does neuroimaging compare to other methods ofcognitive load assessment?

An experiment would consist of four types of task-load conditions: low intrinsic/low extraneous, high intrinsic/lowextraneous, and low intrinsic/high extraneous, and high intrinsic/high extraneous load. The n-back paradigm (withadaptations) would be used to induce high and low intrinsic load in verbal and spatial modalities, and interferenceeffects in the form of spatial and temporal discontiguities to induce high or low extraneous load. The n-back taskrequires participants to encode a stimulus, maintain and rehearse it, track its order, update it, compare it and respondaccording to a specific task goal. In an n-back task, items in a sequential list (they may be letters, shapes, patterns, orlocations) are evaluated in terms of their relationship to a target item presented n items previously in the sequence.Thus, in a typical n-back task, subjects are asked to press a button whenever a letter X appears onscreen only after two(2-back) or 3 (3-back) occurrences of any item in the list other than X.

The anticipated outcome is a significant differential effect of cognitive load on the two groups of ROIs. High intrinsicload should correlate with significantly increased activation of the DLPFC, while high extraneous load should correlatewith increased activation in the early-processing ROIs relative to the baseline. A time-course analysis is anticipated toshow a differential synchronized activation between ROIs under different load conditions. A functional connectivityanalysis would be used to focus on extraneous load, and would be expected to show which mechanisms are workingto process information in the different load conditions.

Insofar as the comparison between alternative approaches to cognitive load measurement is concerned, the reso-lution and accuracy of activation patterns during imaging should go beyond what the secondary task and subjectivequestionnaires reveal: when compared, fMRI is expected to show its advantage in cognitive load measurement withits ability to accurately detect finer-grain differences in different types of load. Nevertheless, subjective measures inparticular may always be used as a supplement to provide deeper understanding and alternative perspectives.

7. Conclusions

In response to the need to explore new avenues for the accurate, comprehensive, and reliable measurement of cogni-tive load, this paper has argued that intrinsic, extraneous, and germane cognitive load can be measured more effectivelythan with current techniques, using fMRI neuroimaging. A literature review described a conceptual framework andexperiment designs in neuroscience that may inform behavioral research on cognitive load. The advantages of usingneuroimaging to study the mechanisms underlying cognitive load were discussed, and a research design was sketchedout. It must be mentioned that, due to the nature of the fMRI paradigm, it can be applied only to distinct and isolatedlearning tasks as current technology does not yet allow analysis over the longer periods of time that would be requiredto engage in complex learning activities.

Cognitive load theory, as noted, has certain explanatory limitations, and attempts to measure cognitive load duringlearning using fMRI must acknowledge such limits. Specifically, low cognitive load effects and expertise reversaleffects should be designed out of any experimental approach, while the learning materials used should be carefullydesigned to avoid any potential ‘overlap’ between intrinsic and extraneous load types.

It should be underlined also that fMRI is a nascent technology and is not without its limitations. One critique ofthe approach, for example, is that different areas of the brain may have different types of hemodynamic response,which would require more sophisticated analytic techniques than the general linear model that is usually applied tofMRI data. FMRI data is also often interpreted overly-simplistically in terms of the localisation of brain functions,and it does not readily reflect the dynamic, distributed nature of the brain’s activities. There are other problems relatedto the temporal acuity of fMRI data, the small sample sizes, the indirect nature of the BOLD signal, and the limitedstatistical techniques commonly used to interpret fMRI data. Debate and counterarguments to these concerns thrivein the field however, and the unique nature of the insights offered by fMRI research cannot be ignored by educationalpsychologists.

Thus, as this paper has argued, certain components of cognitive load during learning, especially intrinsic andextraneous load, may be observable using neuroimaging techniques in a way that would inform both educationalpsychology and fMRI research, and continued advances in fMRI technology may allow for the operationalization andobservation of germane load as the technology matures.

R.R. Whelan / Educational Research Review 2 (2007) 1–12 11

References

Atherton, M., & Bart, W. M. (2001, April). Education and fMRI: Promise and cautions. In: Paper presented at the annual meeting of the AmericanEducational Research Association, Seattle, WA.

Ayres, P. (2006). Impact of reducing intrinsic cognitive load on learning in a mathematical domain. Applied Cognitive Psychology, 20, 287–298.

Bain, R. (2003). Image of cortical input and output (modified). Downloaded from the World Wide Web in May 2003, from http://www.music.sc.edu/fs/bain/vc/musc726a/MUSC%20726%20Lecture/more%20ear-brain/CorticalInput&OutputPathway.jpg.

Banich, M., Milham, M., Atchley, R. A., Cohen, N. A., Webb, A., Wszalek, T., et al. (2000). Prefrontal regions play a predominant role in imposingan attentional ‘set’: Evidence from fMRI. Cognitive Brain Research, 10, 1–9.

Barch, D. M., Sabb, F. W., Braver, T. S., & Noll, D. C. (2000). The anterior cingulate cortex and response competition: Evidence from an fMRIstudy of overt verb generation. Journal of Cognitive Neuroscience, 12, 298–305.

Borg, G., Bratfish, O., & Dorinc, S. (1971). On the problem of perceived difficulty. Scandinavian Journal of Psychology, 12, 249–260.

Braarud, P. O. (2001). Subjective task complexity and subjective workload: Criterion validity for complex team tasks. International Journal ofCognitive Ergonomics, 5(3), 261–273.

Braver T. S., Barch, D. M., & Cohen, J. D. (2003). Mechanisms of cognitive control: Active memory, inhibition, and the prefrontal cortex. Availableonline February 14, 2003, from http://www.iac.wustl.edu/∼ccpweb/pdfs/Mechanisms.pdf (Manuscript).

Braver, T. S., Cohen, J., Nystrom, L. E., Jonides, J., Smith, E. E., & Noll, D. C. (1997). A parametric study of prefrontal cortex involvement inhuman working memory. NeuroImage, 5(1), 49–62.

Braver, T. S., & Speer, N. K. (2003). Maintenance and manipulation in working memory: Evidence against functional dissociation within prefrontalcortex. Available online February 14, 2003, from http://www.iac.wustl.edu/∼ccpweb/publications.html (Manuscript).

Brunken, R., Plass, J. L., & Leutner, D. (2003). Direct measurement of cognitive load in multimedia learning. Educational Psychologist, 38,53–61.

Carpenter, P. A., Just, M. A., Keller, T. A., Eddy, W. F., & Thulborn, K. R. (1999). Time course of fMRI-activation in language and spatial networksduring sentence comprehension. NeuroImage, 10, 216–224.

Damos, D. L., & Bloem, K. A. (1985). Type A behavior pattern, multiple-task performance and subjective estimation of mental workload. Bulletinof the Psychomonaic Society, 23, 53–56.

D’Esposito, M., Postle, B. R., Ballard, D., & Lease, J. (1999). Maintenance versus manipulation of information held in working memory: An fMRIstudy. Brain & Cognition, 41, 66–86.

Dobbins, I. G., Foley, H., Schacter, D. L., & Wagner, A. D. (2002). Executive control during episodic retrieval: Multiple prefrontal processes subservesource memory. Neuron, 35(5), 989–996.

Ernst, M., Nelson, E., McClure, E., Monk, C., Munson, S., Eshel, N., et al. (2004). Choice selection and reward anticipation: An fMRI study.Neuropsychologia, 42(12), 1585–1597.

Gerjets, P., Scheiter, K., & Catrambone, R. (2004). Designing instructional examples to reduce intrinsic cognitive load: Molar versus modularpresentation of solution procedures. Instructional Science, 32, 33–58.

Just, M. A., Carpenter, P. A., Keller, T. A., Emery, L., Zajac, H., & Thulborn, K. R. (2001). Interdependence of nonoverlapping cortical systems indual cognitive tasks. NeuroImage, 14, 417–426.

Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2002). The expertise reversal effect. Educational Psychologist, 38(1), 23–31.Kalyuga, S., & Sweller, J. (2005). Rapid dynamic assessment of expertise to improve the efficiency of adaptive E-learning. Educational Technology

Research & Development, 3(53), 83–93.Kirsch, P., Schienle, A., Stark, R., Sammer, G., Blecker, C., Walter, B., et al. (2003). Anticipation of reward in a nonaversive differential conditioning

paradigm and the brain reward system: An event-related fMRI study. NeuroImage, 20(2), 1086–1095.Mayer, R. E. (1997). Multimedia learning: Are we asking the right questions? Educational Psychologist, 32(1), 1–19.Mayer, R. E. (2001). Multimedia learning. Cambridge: Cambridge UniversityMeredith, M. A. (2001). On the neuronal basis for multisensory convergence: A brief overview. Cognitive Brain Research, 14(1), 31–

40.Meshkati, N., & Loewenthal, A. (1988). An eclectic and critical review of four primary mental workload assessment methods: A guide for developing

a comprehensive model. In P. A. Hancock, & N. Meshkati (Eds.), Human mental workload (pp. 251–267). Elsevier.Michael, E. B., Keller, T. A., Carpenter, P. A., & Just, M. A. (2001). An fMRI investigation of sentence comprehension by eye and by ear: Modality

fingerprints on cognitive processes. Human Brain Mapping, 13, 239–252.Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24, 167–202.Newman, S. D., Just, M. A., & Carpenter, P. A. (2002). The synchronization of the human cortical working memory network. NeuroImage, 15(4),

810–822.Ogawa, S., Lee, T.-M., Nayak, A. S., & Glynn, P. (1990). Oxygenation-sensitive contrast in magnetic resonance image of rodent brain at high

magnetic fields. Magnetic Resonance Medicine, 14, 68–78.Paas, F. (1993). Instructional control of cognitive load in the training of complex cognitive tasks. Thesis. The Hague: University of Twente,

Koninklijke Library.Paas, F., Tuovinen, J., Tabbers, H., & van Gerven, P. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational

Psychologist, 38(1), 63–71.Sewards, T., & Sewards, M. (2003). Representations of motivational drives in mesial cortex, medial thalamus, hypothalamus and midbrain. Brain

Research Bulletin, 61(1), 25–49.

12 R.R. Whelan / Educational Research Review 2 (2007) 1–12

Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4, 295–312.Sweller, J. (1999). Instructional design in technical areas. Camberwell, Australia: ACER Press.Taylor, S., Welsh, R., Wager, T., Phan, K., Fitzgerald, K. D., & Gehring, W. J. (2004). A functional neuroimaging study of motivation and executive

function. NeuroImage, 21(3), 1045–1054.Tsang, P., & Velazquez, V. (1996). Diagnosticity and multidimensional subjective workload ratings. Ergonomics, 30(3), 358–381.Veltman, D. J., Rombouts, S. A. R. B., & Dolanc, R. J. (2003). Maintenance versus manipulation in verbal working memory revisited: An fMRI

study. NeuroImage, 18(2), 247–256.Whelan, R. (2006). Measuring cognitive load. Dissertation. New York University.