A Comparison between theories

Permission from author required to link to the IDKB. If citing this work in a publication or scholarly work, it should be referenced as: 

Dabbagh, N. (date retrieved). The Instructional Design Knowledge Base. Retrieved month, day, year from Nada Dabbagh's Homepage, George Mason University, Instructional Technology Program. Website: http://cehdclass.gmu.edu/ndabbagh/Resources/IDKB/index.htm Instructional design is an ill-structured problem-solving process that is defined by the context of the problem, the knowledge and skills of the instructional designer, and the quality of available resources. The purpose of the Instructional Design Knowledge Base (IDKB) is threefold:

1. To support faculty teaching courses in Instructional Design; 

2. To support students learning Instructional Design; 3. To support practitioners in the field of Instructional

Design.

The IDKB has two components. Click on one of the images to the right or on the links below to enter the area that best fits your purpose.

Doing an Instructional Design Project , and  Just Exploring Resources

Check out the new pedagogical models website! (uploaded July  

2009)

Check out CATS, an in-progress instrument for evaluating the cognitive affordances of learning technologies CATS: Cognitive Affordances of Technologies Scale (uploaded February 2011)

Feedback is welcome!

Dr. Nada Dabbagh's Homepage | IT Program Home | Graduate School of Education | George Mason University

©2002 - 2014

Basic PrinciplesObjectivism/Behaviorism Cognitivism/Pragmatism Constructivism/Interpretivism

Learning happens when a correct response is demonstrated following the presentation of a specific environmental stimulus

Learning can be detected by observing an organism over a period of time

Emphasis is on observable and measurable behaviors

Uses a "black box" metaphor - the learner is a black box, what happens inside is unknown

Emphasis is on relationships between environmental variables and behavior

Instruction utilizes consequences and reinforcement of learned behaviors

Believes behavior is guided by purpose

Cues are antecedents to behavior and set the conditions for its occurence

Learning is a change of knowledge state

Knowledge acquisition is described as a mental activity that entails internal coding and structuring by the learner

Learner is viewed as an active participant in the learning process

Emphasis is on the building blocks of knowledge (e.g. identifying prerequisite relationships of content)

Emphasis on structuring, organizing and sequencing information to facilitate optimal processing 

Focus is on how learners remember, retrieve, and store information in memory

Examines the mental structure and processes related to learning 

Learning is viewed as an active process that occurs within the learner and which can be influenced by the learner

The outcome of learning is not only dependent on what the teacher presents but also on what the learner does to process this information.

Learners build personal interpretation of the world based on experiences and interactions

Knowledge is embedded in the context in which it is used (authentic tasks in meaningful realistic settings)

Create novel and situation-specific understandings by "assembling" knowledge from diverse sources appropriate to the problem at hand (flexible use of knowledge) 

Believes that there are many ways (multiple perspectives) of structuring the world and its entities

Believes that meaning is imposed by the individual rather than existing in the world independently

Embedded Theories 

Objectivism/Behaviorism Cognitivism/Pragmatism Constructivism/InterpretivismPavlov's Classical Conditioning

Skinner's Operant Conditioning

Stimulus-Response Theory

Thorndike's Laws and Connectionism

Information Processing

Component Display Theory

Dual Coding Theory 

Elaboration Theory

Gestalt Theory

Mental Models

Schema Theory

Subsumption Theory

Cognitive Flexibility Theory

Generative Learning Theory

Knowledge as Tools

Situated Cognition 

Social-Cultural Learning

Pavlov's Classical Conditioning

We use the term "classical conditioning" to describe one type of associative learning in which there is no contingency between response and reinforcer. This situation resembles most closely the archetypal experiment from I. Pavlov in the 1920s, where he trained dogs to associate a tone with a food-reward. In such experiments, the subject initially shows weak or no response to a conditioned stimulus (CS, e.g. a tone), but a measurable unconditioned response (UR, e.g. saliva production) to a

Information Processing

Information processing: a) students are actively processing, storing and retrieving information, and b) teaching involves helping learners to develop information processing skills and apply them systematically to mastering the curriculum. Cognitive structures relate to structure of the subject matter. Information processing emphasizes cognitive structures built by the learner. Two types of memory exist.

Episodic memory - The recall of events

Cognitive Flexibility Theory

Cognitive flexibility theory focuses on the nature of learning in complex and ill-structured domains. Spiro &Jehng (1990, p. 165) state: "By cognitive flexibility, we mean the ability to spontaneously restructure one's knowledge, in many ways, in adaptive response to radically changing situational demands...This is a function of both the way knowledge is represented (e.g., along multiple rather single conceptual dimensions) and the processes that operate on those mental representations (e.g., processes of schema assembly rather than intact schema retrieval)." The theory is largely

unconditioned stimulus (US, e.g. food). In the course of the training, the CS is repeatedly presented together with the US; eventually the subject forms an association between the US and the CS. In a subsequent test-phase, the subject will show the conditioned response (CR, e.g. saliva production) to the CS alone, if such an association has been established and memorized.

Classical conditioning. (n.d.). Retrieved September, 7, 2002, from University of Wuerzburg, Biozentrum Department of Genetics and Neurobiology Web site: http://www.biozentrum.uni-wuerzburg.de/genetics/behavior/learning/classical.html

 

which is in detail and sequence

Semantic memory - Intentional learning which involves encoding, storage, and retrieval of information

Information process theory of learning. (n.d.). Retrieved September 7, 2002, from The University of Missouri in Columbia, The College of Education Web site: http://tiger.coe.missouri.edu/~t377/IPTheorists.html

  Component Display Theory

Component Display Theory (CDT) classifies learning along two dimensions: content (facts, concepts, procedures, and principles) and performance (remembering, using, generalities). The theory specifies four primary presentation forms: rules (expository presentation of a generality), examples (expository presentation of instances), recall (inquisitory generality) and practice (inquisitory instance). Secondary presentation forms include: prerequisites, objectives, helps, mnemonics, and feedback. The theory specifies that instruction is more effective to the extent that it contains all necessary primary and secondary forms. Thus, a complete lesson would consist of objective

concerned with transfer of knowledge and skills beyond their initial learning situation. For this reason, emphasis is placed upon the presentation of information from multiple perspectives and use of many case studies that present diverse examples. The theory also asserts that effective learning is context-dependent, so instruction needs to be very specific. In addition, the theory stresses the importance of constructed knowledge; learners must be given an opportunity to develop their own representations of information in order to properly learn.

Kearsley, G. (n.d.). Cognitive flexibility theory. Retrieved September 7, 2002, from Explorations in Learning & Instruction: The Theory Into Practice Database Web site: http://home.sprynet.com/~gkearsley/tip/spiro.html

Spiro, R. J. &Jehng, J. (1990). Cognitive flexibility and hypertext: Theory and technology for the non-linear and multidimensional traversal of complex subject matter. D. Nix & R. Spiro (Eds.), Cognition, Education, and Multimedia. Hillsdale, NJ: Erlbaum.

 

Generative Learning Theory

Skinner's Operant Conditioning

We use the term operant conditioning to describe one type of associative learning in which there is a contingency between the response and the presentation of the reinforcer. This situation resembles most closely the classic experiments from Skinner, where he trained rats and pigeons to press a lever in order to obtain a food reward. In such experiments, the subject is able to generate certain motor-output, (the response R, e.g. running around, cleaning, resting, pressing the lever). The experimenter chooses a suited output (e.g. pressing the lever) to pair it with an unconditioned stimulus (US, e.g. a food reward). Often a discriminative stimulus (SD, e.g. a light) is present, when the R-US contingency is true. After a training period, the subject will show the conditioned response (CS, e.g. touching the trigger) even in absence of the US, if the R-US association has been memorized.

Operant conditioning. (n.d.). Retrieved September, 7, 2002, from University of Wuerzburg, Biozentrum Department of Genetics and Neurobiology Web site: http://www.biozentrum.uni-wuerzburg.de/genetics/behavior/learning/operant.html

followed by some combination of rules, examples, recall, practice, feedback, helps and mnemonics appropriate to the subject matter and learning task. Indeed, the theory suggests that for a given objective and learner, there is a unique combination of presentation forms that results in the most effective learning experience.

Kearsley, G. (n.d.). Component display theory (M.D. Merrill). Retrieved September 7, 2002, from Explorations in Learning & Instruction: The Theory Into Practice Database Web site: http://home.sprynet.com/~gkearsley/tip/merrill.html

 

Dual Coding Theory

The dual coding theory attempts to give equal weight to verbal and nonverbal processing. The theory assumes that there

Generative learning is the active process of saying, "Oh. That's like ..." It's the process of constructing links between new and old knowledge, or a personal understanding how new ideas fit into an individual's web of known concepts. "The essence of the generative learning model is that the mind, or the brain, is not a passive consumer of information. Instead, it actively constructs its own interpretations of information and draws inferences from them" (Wittrock, p348). Learning involves mental activity - thinking. For example, with respect to reading a textbook or paper, without active construction of relations between parts of a text, or between the text and personal knowledge, the student will pass over the words and wonder what has been read.

Furey, D. (2002). Generative learning. Retrieved September 8, 2002, from Stem-Net, Information Technology for Learning Resources Course Web site: http://www.stemnet.nf.ca/~dfurey/metacog/generate.html

 

 Knowledge as Tools

This viewpoint uses the analogy that in some ways conceptual knowledge is similar to a set of tools. Brown, Collins&Duguid (1989) credit this image to Richard Burton, who explored the idea in a symposium for the Secretary of Education of Kentucky, and D. N. Perkins who wrote about it in his book, Knowledge as Design (1986). Both knowledge

Stimulus-Response Theory

All complex forms of behavior, including reasoning, habit, and emotional reactions are composed of simple stimulus-response events which can be see and measured. We can trace a child's attitude back to a specific stimulus. Once we've identified a stimulus that produced certain response, we can predict the individual's behavior. Furthermore, if we can control the stimulus, we can control the individual's behavior.

There are two kinds of responses:

1. Elicited response - the response occur in the presence of a stimulus

2. Emitted response - the movement was emitted by the organism - not as a response to a stimulus.

 

 

Thorndike’s Laws and

are two cognitive subsystems, one specialized for the representation and processing of nonverbal objects/events (i.e., imagery), and the other specialized for dealing with language. Dual Coding theory identified three types of processing:

1. Representational - the direct activation of verbal or nonverbal representations

2. Referential - the activation of the verbal system by the nonverbal system or vice-versa

3. Associative processing - the activation of representations within the same verbal or nonverbal system.

A given task may require any or all of the three kinds of processing.

Kearsley, G. (n.d.). Dual coding theory (A. Paivio). Retrieved September 7, 2002, from Explorations in Learning & Instruction: The Theory Into Practice Database Web site: http://home.sprynet.com/~gkearsley/tip/paivio.html

Elaboration Theory

According to elaboration theory, instruction should be organized in increasing order of complexity for optimal learning. For example,

and tools are only completely understood through use. Using them can change the way the user sees the world and results in the user adopting the culture in which they are used. It is possible to own a tool without actually knowing how to use it. This is also true of knowledge. For example, a student may memorize an algorithm and use it in math class, but may not be able to apply it when presented with a real world situation.Frequent use is also important in learning how to use tools and knowledge. Someone who owns a tool and uses it frequently develops a good understanding of the tool and the world in which they use it (Brown, Collins &Duguid (1989). This understanding of the world and the tool is dynamic based on the user's interactions. Similarly, learning is a lifelong process.No tool can be used or understood properly without considering the culture in which it is used. Consider the violin as a musician's tool. Placed in an orchestra, the vile is played one way, but put the same instrument in a bluegrass band as a fiddle and it is played in an entirely different way. Similarly, conceptual tools reflect the cumulative knowledge of the culture in which they are used and the experience of the individuals who use them. "Activity, concept and culture are interdependent. No one can be totally understood without the other two" (Brown, Collins &Duguid 1989, p. 33).Brown, J. S., Collins, A. &Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, January-February, 32-42

Situated Cognition

Situated cognition is a research approach, spanning many disciplines and objectives, that relates social, behavioral/psychological, and neural perspectives of knowledge and action (Clancey, 1997, p. 343). Lave (1991, p. 84) clarifies: "‘Situated’…does not imply that something is concrete and particular, or that it is not generalizable, or not imaginary. It implies that a given social practice is multiply interconnected with other aspects of ongoing social processes in activity systems at many levels of particularity

Connectionism

Thorndike was a professor at Teachers College, Columbia (1904-40), where he worked on educational psychology and the psychology of animal learning. As a result of studying animal intelligence, he formulated his famous "law of effect", which states that a given behavior is learned by trial-and-error, and is more likely to occur if its consequences are satisfying. His works include Psychology of Learning (1914) and The Measurement of Intelligence (1926).

Thorndike, Edward L(ee) (1874-1949). (n.d.). Retrieved September 7, 2002, from San Francisco State University, Journalism Department Web site: http://userwww.sfsu.edu/~rsauzier/Thorndike.html

 

when teaching a procedural task, the simplest version of the task is presented first; subsequent lessons present additional versions until the full range of tasks are taught. In each lesson, the learner should be reminded of all versions taught so far (summary/synthesis). A key idea of elaboration theory is that the learner needs to develop a meaningful context into which subsequent ideas and skills can be assimilated. Elaboration theory proposes seven major strategy components:

1. An elaborative sequence2. Learning prerequisite sequences3. Summary4. Synthesis5. Analogies6. Cognitive strategies7. Learner control.

Kearsley, G. (n.d.). Elaboration Theory. Retrieved September 7, 2002, from Explorations in Learning & Instruction: The Theory Into Practice Database Web site: http://home.sprynet.com/~gkearsley/tip/reigelut.html

 

 Gestalt Theory

and generality." Thus, situated cognition should not be characterized as only allowing for concrete learning in localized situations. Instead, situated cognition emphasizes the web of social and activity systems within which authentic practice takes shape.

Clancey, W. J. (1997). Situated cognition: On human knowledge and computer representations. Cambridge: Cambridge University Press.

Lave, J. (1991). Situated learning in communities of practice. In L. B. Resnick, J. M. Levine, & S.D. Teasley (Eds). Perspectives on socially shared cognition (pp. 63-82). Washington, DC: American Pscyhological Association.

Wilson, B. & Myers, K. (1999). Situated cognition in theoretical and practical context [Electronic version]. For inclusion in Jonassen, D. & Land, S. (Eds.). Theoretical foundations of learning environments. Mahwah NJ: Erlbaum. Retrieved September 8, 2002, fromhttp://ceo.cudenver.edu/~brent_wilson/SitCog.html

 Social-Cultural Learning

Vygotsky was a leader in social-cultural learning theory. According to Lefrancois, (1994), his complex theory is actually comprised of three identifiable themes: culture, the central role of language and the "zone of proximal development" (Galant). Cultures are created through the use of tools and symbols, which

Gestalt theory emphasizes higher-order cognitive processes in the midst of behaviorism. The focus of Gestalt theory is the idea of "grouping", i.e., characteristics of stimuli cause us to structure or interpret a visual field or problem in a certain way. The primary factors that determine grouping are:

1. Proximity - elements tend to be grouped together according to their nearness

2. Similarity - items similar in some respect tend to be grouped together

3. Closure - items are grouped together if they tend to complete some entity

4. Simplicity - items will be organized into simple figures according to symmetry, regularity, and smoothness.

These factors are called the laws of organization and were explained in the context of perception and problem-solving.

Kearsley, G. (n.d.). Gestalt Theory (Wertheimer). Retrieved September 7, 2002, from Explorations in Learning & Instruction: The Theory Into Practice Database Web site: http://home.sprynet.com/~gkearsley/tip/wertheim.html

separates humans from animals. Vygotsky thought that the development of intelligence was a function of the internalization of the tools found in one's culture (Driscoll, 2000). As cultures grow, new tools emerge. Thus both a historical perspective and a cultural perspective are important in understanding how the human mind functions. In fact, the two perspectives are almost one in the same.

The second of Vygotsky's themes, the central role of language, suggests that language is actually possible because of the tools and symbols belonging to cultures (Galant). The learning of language takes place through social processes. A person goes through three stages of speech: social special (external speech), egocentric speech (when children talk think out loud, regardless of someone listening) and inner speech (soundless speech, which allows us to direct our thinking and behavior).

The third aspect of Vygotsky's theory is the zone of proximal development, which is the gap between a child's potential development and his/her actual development. On a spectrum, if what the child can do unassisted is at one end and what the child cannot do yet is on the other, the zone of proximal development, or what the child can do with assistance, is in between (Driscoll, 2000).

For a list of additional resources on socio-

 Mental Models

Mental models are representations of reality that people use to understand specific phenomena. Norman (in Gentner& Stevens, 1983) describes them as follows: "In interacting with the environment, with others, and with the artifacts of technology, people form internal, mental models of themselves and of the things with which they are interacting. These models provide predictive and explanatory power for understanding the interaction." Some of the characteristics of mental models are:

1. They are incomplete and constantly evolving

2. They are usually not accurate representations of a phenomenon, they typically contain errors and contradictions

3. They are parsimonious and provide simplified explanations of complex phenomena

4. They often contain measures of uncertainty about their validity that allow them to used even if incorrect

5. They can be represented by sets of condition-action rules

Gentner, D. & Stevens, A.(1983). Mental

cultural theory, visit http://carbon.cudenver.edu/~mryder/itc_data/soc_cult.html.

Driscoll, M. (2000). Psychology of Learning for Instruction. Needham Heights, MA: Allyn & Bacon.

Galant, M. (n.d.). Vygotsky's cultural/cognitive theory of development. Retrieved August 28, 2002, from Cortland College, Educational Psychology Web site: http://facultyweb.cortland.edu/~ANDERSMD/VYG/VYG.HTML

Lefrancois, G. R. (1994). Psychology for Teaching. Belmont, CA: Wadsworth Publishing Company.

 

Models. Hillsdale, NJ: Erlbaum.

Kearsley, G. (n.d.). Mental models. Retrieved September 7, 2002, from Explorations in Learning & Instruction: The Theory Into Practice Database Web site: http://home.sprynet.com/~gkearsley/tip/models.html

 Schema Theory

Bartlett (1932, 1958) is credited with first proposing the concept of schema (plural: schemata). He arrived at the concept from studies of memory he conducted in which subjects recalled details of stories that were not actually there. He suggested that memory takes the form of schema which provide a mental framework for understanding and remembering information.

Bartlett, F. C. (1932). Remembering: An experimental and social study. Cambridge: Cambridge University Press.

Bartlett, F. C. (1958). Thinking. New York: Basic Books. 

Kearsley, G. (n.d.). Schema. Retrieved September 7, 2002, from Explorations in Learning & Instruction: The Theory Into

Practice Database Web site: http://home.sprynet.com/~gkearsley/tip/schema.html

  Subsumption Theory

Ausubel's theory is concerned with how individuals learn large amounts of meaningful material from verbal/textual presentations in a school setting (in contrast to theories developed in the context of laboratory experiments). According to Ausubel, learning is based upon the kinds of superordinate, representational, and combinatorial processes that occur during the reception of information. A primary process in learning is subsumption in which new material is related to relevant ideas in the existing cognitive structure on a substantive, non-verbatim basis. Cognitive structures represent the residue of all learning experiences; forgetting occurs because certain details get integrated and lose their individual identity.

Kearsley, G. (n.d.). Subsumption theory. Retrieved September 7, 2002, from Explorations in Learning & Instruction: The Theory Into Practice Database Web site: http://home.sprynet.com/~gkearsley/tip/ausubel.html

Principal Theorists (TOP)Objectivism/Behaviorism Cognitivism/Pragmatism Constructivism/Interpretivism

BanduraPavlovSkinnerThorndike 

AndersonAusubelGardnerGagnéMerrillNormanNovakReigeluthRummelhart  

Bransford and the CTGVBrunerDeweyGrabingerLave & WengerPapertPiagetSpiro and colleaguesVygotsky

Goals of Instruction (TOP)Objectivism Cognitivism/Pragmatism Constructivism/Interpretivism

Communicate or transfer behaviors representing knowledge and skills to the learner (does not consider mental processing)

Instruction is to elicit the desired response from the learner who is presented with a target stimulus

Learner must know how to execute the proper response as well as the conditions under which the response is made

Learner acquires skills of discrimination

Communicate or transfer knowledge in the most efficient, effective manner (mind-independent, can be mapped onto learners)

Focus of instruction is to create learning or change by encouraging the learner to use appropriate learning strategies

Learning results when information is stored in memory in an organized, meaningful way. 

Teachers/designers are responsible for

Build personal interpretations of the world based on individual experiences and interactions (constantly open to change, cannot achieve a predetermined, "correct" meaning, knowledge emerges in relevant contexts)

Learning is an active process of constructing rather than acquiring knowledge

Instruction is a process of supporting knowledge construction rather than communicating knowledge

(recalling facts), generalization (defining and illustrating concepts), association (applying explanations), and chaining (automatically performing a specified procedure).

 

assisting learners in organizing information in an optimal way so that it can be readily assimilated 

Do not structure learning for the task, but engage learner in the actual use of the tools in real world situations 

Learning activities should be authentic and should center around the ?problematic? or ?puzzlement? as perceived by the learner

The focus is on the process not the product

Role of teacher is a mentor not a ?teller?

Encourage reflective thinking, higher-order learning skills

Encourage testing viability of ideas and seeking alternative views

Instructional Models (TOP)Objectivism Cognitivism/Pragmatism Constructivism/Interpretivism

Computer-Based InstructionContract LearningIndividualized InstructionProgrammed InstructionInformation Processing ModelSystems Approach

Collins & Stevens Inquiry Teaching ModelKeller's ARCS Model of MotivationMerrill?s Component Display Model  

Action LearningAnchored Instruction Authentic LearningCase-Based LearningCognitive ApprenticeshipCognitive Flexibility HypertextCollaborative LearningCommunities of PracticeComputer-Supported Intentional Learning Environments (CSILEs)Discovery LearningDistributed LearningEpistemic GamesGenerative learningGoal-Based Scenarios (GBSs)Inquiry-Based LearningMicroworlds/SimulationsMOOs and MUDsProblem-Based Learning (PBL)REALsReciprocal TeachingSituated LearningWebQuest(s)

Computer-Based InstructionComputer-assisted instruction was first used in education and training during the 1950s. Early

Collins & Stevens Inquiry Teaching ModelCollins' Cognitive Theory of Inquiry Teaching is a prescriptive model, primarily Socratic in nature, meaning that it relies upon a dialectic process of

Action LearningRegRevansis considered the architect of action learning. Inglis (1994) defined AL as "a process which brings people together to

work was done by IBM and such people as Gordon Pask, and O.M. Moore, but CAI grew rapidly in the 1960s when federal funding for research and development in education and industrial laboratories was implemented. The U.S. Government wanted to determine the possible effectiveness of computer-assisted instruction, so they developed two competing companies, (Control Data Corporation and Mitre Corporation) who came up with the PLATO and TICCIT projects. Despite money and research, by the mid seventies it was apparent that CAI was not going to be the success that people had believed. Some of the reasons are: CAI had been oversold and could not deliver, lack of support from certain sectors, technical problems in implementation, lack of quality software, and high cost. Computer-assisted instruction was very much drill-and-practice - controlled by the program developer rather than the learner. Little branching of instruction was implemented although TICCIT did allow the learner to determine the

discussion, questions and answers that occurs between the learner and instructor. The process is guided in order to reach the predetermined objectives, which are described in this theory as teacher goals and subgoals. Ultimately, the learners will discover "how to learn".Teacher goals and subgoals is one of three main portions of Collins' theory. The second is the strategies used to realize said goals and subgoals and the third is the control structure for selecting and pursuing the different goals and subgoals.Brooks , C. E. (n.d.). Cognitive theory of inquiry teaching. Retrieved September 10, 2002, from University of Arkansas, Educational Technology Department Web site: http://comp.uark.edu/~brooks/cognitive.html   Keller's ARCS Model of MotivationJohn M. Keller proposed four conditions that must be met for a learner to be motivated to learn. Attention, relevance, confidence, and satisfaction (ARCS) are the conditions that, when integrated, motivate someone to learn. Moreover, Keller suggests that the ARCS conditions occur as a sequential process (Driscoll, 1993, p. 312). The conditions should be sustained to keep the learner interested in the topic. Once a learner’s attention is lost, motivation is lost, and learning does not occur. Shneiderman (1998, p. 25) states that "memorable educational experiences are enriching, joyful, and transformational." Motivation theory argues that relevant phenomena fulfill personal needs or goals, which enhances effort and performance (Means, Jonassen, & Dwyer, 1997). How then can one ensure that the ARCS model remain active? The key is

find solutions to problems and, in doing so, develops both the individuals and the organization" (p. 3). According to Spence (1998), Revans loosely defined action learning as the process of learning "from and with peers while tackling real problems (O'Neil and Marsick 1994)" (p. 1). However, Revans (1980) also says that it is not just project work, job rotation, case studies or business games. According to Inglish (1994), action learning differs from these other methodologies in the following ways.

Learning is centered around the need to find a solution to a real problem.

Learning is voluntary and learner driven.

Individual development is as important as finding the solution to the problem.

Action learning is a highly visible, social process, which may lead to organizational change.

Action learning takes time. As originally envisioned, an action learning program would take 4-9 months, excluding implementation. (Spence, 1998, p.1)

Problem, set, client, set advisor, and process are the basic elements of action learning. The following is a brief explanation of each element (Spence, 1998).

Problem - The problem should be non-technical in nature and deal

sequence of instruction or to skip certain topics. Contract LearningContract learning involves the use of contingency contracts, which define the terminal behavior the student is to achieve and conditions for achievement and consequences for completion or non-completion of the assigned task(s). The contingency contract is mutually agreed upon by teacher and student after negotiations. Contract learning is often used in open educational systems in which students from various grade levels share in learning activities.Contract learning can also be useful in a college setting. According to Knowles, (1991, p. 39) "Contract learning is, in essence, an alternative way of structuring a learning experience: It replaces a content plan with a process plan." For more information on the concept of contract learning and adult learning theory, visit http://www.msu.edu/user/coddejos/contract.htm.Codde, J. R. (1996). Using learning contracts in the college

to vary the conditions to engage the learner. Because each component of Keller’s ARCS model builds upon the next model, the instructor should keep the four components in mind when designing instruction.Driscoll, M. (1993). Psychology of learning for instruction. Needham Heights, MA: Allyn & BaconFernández, J. (1999). Attribution theory and Keller’s ARCS model of motivation. Retrieved September 10, 2002, from George Mason University, Graduate School of Education, Instructional Design & Development Immersion Program Web site:http://chd.gse.gmu.edu/immersion/knowledgebase/strategies/cognitivism/keller_ARCS.htmMeans, T., Jonassen, D., Dwyer, F. (1997). Enhancing relevance: Embedded ARCS strategies vs. purpose. Educational Technology Research and Development, 45, 5-17.Shneiderman, B. (1998). Relate—Create—Donate: A teaching/learning philosophy for the cyber-generation. Computers & Education, 31, 25-39.

Merrill's Component Display ModelCDT specifies how to design instruction for any cognitive domain. CDT provides the basis for the lesson design in the TICCIT computer based learning system (Merrill, 1980). It also was the basis for the Instructional Quality Profile, a quality control tool for instructional materials (Merrill, Reigeluth& Faust, 1979). Fo example, if we were designing a complete lesson on equilateral triangles according to CDT, it would have the following minimum components:Objective - Define an equilateral triangle (Remember-Use) Generality - Definition (attributes, relationships) 

with either strategic or tactical issues. The outcome of the problem solutions must matter to the participants. Participants within a set may work on the same problem or different problems.

Set - A set of four to six action learners that solve the problem together. Set members should be competent and committed and come have a range of expertise.

Client - The owner of the problem. May be a set member or sponsoring organization.

Set Advisor - The group facilitator. The set advisor explains the action learning process and builds appropriate interpersonal skills. This person also maintains open communication with the client. As the action learning progresses, set members may take on some of these responsibilities.

Process - Observation of the problem, reflection and hypothesis forming and action.

Action learning has been applied in many areas of adult education such as nursing education and human resource development graduate programs.

Inglis, S. (1994). Making the Most of Action Learning. Aldershot, England: Gower.O'Neil, J., and Marsick, V. J. (1994).

classroom. Retrieved August 19, 2002, from Michigan State University, Educational Technology Certificate Program Web site: http://www.msu.edu/user/coddejos/contract.htmDriscoll, M. (2000). Psychology of learning for instruction. Needham Heights, MA: Allyn & Bacon. Individualized InstructionSimilar to programmed learning and teaching machines individualized instruction began in the early 1900s, and was revived in the 1960s. The Keller Plan, Individually Prescribed Instruction, Program for Learning in Accordance with Needs, and Individually Guided Education are all examples of individualized instruction in the U.S. (Saettler, 1990). Programmed InstructionAfter experimental use of programmed instruction in the 1920s and 1930s, B. F. Skinner and J.G. Holland first used programmed instruction in behavioral psychology courses at Harvard in the late 1950s. Use of programmed instruction appeared in elementary and

Instance - Examples (attributes present, representations) Generality Practice - State definition Instance Practice - Classify (attributes present) Feedback - Correct generalities/instances Elaborations - Helps, prerequisites, contextIf the generality was presented by an explanation or illustration, followed by practice examples, this would be an expository strategy (EG, Eeg). On the other hand, if the students were required to discover the generality on the basis of practice examples, this would be an inquisitory strategy (IG, Ieg).Merrill, M.D. (1980). Learner control in computer based learning. Computers and Education, 4, 77-95.Kearsley, G. (n.d.). Component display theory (M.D. Merrill). Retrieved September 7, 2002, from Explorations in Learning & Instruction: The Theory Into Practice Database Web site: http://home.sprynet.com/~gkearsley/tip/merrill.html

Becoming critically reflective through action reflection learning TM. New Directions for Adult and Continuing Education no. 63, 17-29. (EJ 494 200).Revans, R (1980). Action learning: New techniques for management. London: Blond & Briggs.Spence, J. (1998). Practice application brief: Action learning for individual and organizational development [Electronic version]. (Developed with funding from the Office of Educational Research and Improvement, National Library of Education, U.S. Department of Education, under Contract No. RR93002001). Retrieved August 24, 2002, from http://ericacve.org/docs/pab00009.htm   Anchored InstructionAnchored instruction is a major paradigm for technology-based learning that has been developed by Cognition & Technology Group at Vanderbilt (CTGV) under the leadership of John Bransford. While many people have contributed to the theory and research of anchored instruction, Bransford is the principal spokesperson and hence the theory is attributed to him.The initial focus of the work was on the development of interactive videodisc tools that encouraged students and teachers to

secondary schools around the same time. Much of the programmed instruction in American schools was used with individuals or small groups of students and was more often used in junior high schools than senior or elementary schools (Saettler, 1990). Early use of programmed instruction tended to concentrate on the development of hardware rather than course content. Concerned developers moved away from hardware development to programs based on analysis of learning and instruction based on learning theory. Despite these changes, programmed learning died out in the later part of the 1960s because it did not appear to live up to its original claims (Saettler, 1990).Information Processing ModelThis model represents information processing as a computer model. Information processing easily relates to computer input-process-output. Processing information involves subroutines or procedures. Subroutines are performed in a hierarchical manner to complete tasks. Flow of control can be

pose and solve complex, realistic problems. The video materials serve as "anchors" (macro-contexts) for all subsequent learning an d instruction. As explained by CTGV (1993, p52): "The design of these anchors was quite different from the design of videos that were typically used in education...our goal was to create interesting, realistic contexts that encouraged the active construct ion of knowledge by l earners. Our anchors were stories rather than lectures and were designed to be explored by students and teachers. " The use of interactive videodisc technology makes it possible for students to easily explore the content.Anchored instruction is closely related to the situated learning framework (see CTGV, 1990, 1993) and also to the Cognitive Flexibility theory in its emphasis on the use of technology-based learning.CTGV (1990). Anchored instruction and its relationship to situated cognition. Educational Researcher, 19(6), 2-10.CTGV (1993). Anchored instruction and situated cognition revisted. Educational Technology, 33(3), 52- 70.Kearsley, G. (n.d.). Anchored Instruction (John Bransford& the CTGV). Retrieved September 7, 2002, from Explorations in Learning & Instruction: The Theory Into Practice Database Web site: http://tip.psychology.org/anchor.html

diagrammed. Logic Theorist was a computer program by Newell, Shaw and Simon (1955-60) used to simulate the human process of solving theorems in symbolic language. At the same time MIT, had a pattern recognition program.There have been many computer models for human information processing. Two types of information processing models are those dealing with simulation, or step by step, and those that are dealing with artificial intelligence and are task driven. Logic Theorist emulated six human characteristics of problem-solving behavior.Information process theory of learning. (n.d.). Retrieved September 10, 2002, from University of Missouri in Columbia, The College of Education Web site: http://tiger.coe.missouri.edu/~t377/IPTheorists.html System ApproachThe systems approach developed out of the 1950s and 1960s focus on language laboratories, teaching machines, programmed instruction, multimedia

  Authentic LearningAuthentic learning refers to the idea that learners should be presented to problems that are realistic situations and found in everyday applications of knowledge (Smith & Ragan, 1999). Authentic learning is the type of learning promoted by anchored instruction, in which instruction is "anchored" in a realistic problem situation (Cognition and Technology Group, 1990).Young (1993), recommends the following test of "authenticity." Learning situations should include some of the characteristics of real-life problem solving, including ill-structured complex goals. There should also be an opportunity to distinguish between relevant and irrelevant information. Finding and defining problems as well as solving them should be a generative process. Finally, students should engage in collaborative activities in which they draw upon their beliefs and values.For more information on Authentic Learning, visit Tiffany Mara's (University of Michigan, Ann Arbor) Web site, http://www-personal.umich.edu/~tmarra/authenticity/authen.html.Cognition and Technology Group. (1990). Anchored instruction and its relationship to situated cognition. Educational Researcher, 19(8), 2-10.Smith, P. and Ragan, T. (1999). Instructional

presentations and the use of the computer in instruction. Most systems approaches are similar to computer flow charts with steps that the designer moves through during the development of instruction. Rooted in the military and business world, the systems approach involved setting goals and objectives, analyzing resources, devising a plan of action and continuous evaluation/modification of the program. (Saettler, 1990)

design (2nd ed.). New York: John Wiley & Sons, Inc.Young, M. (1993). Instructional design for situated learning. Educational Technology Research & Development, 41(1), 43-58.   Case-Based LearningCase-based learning using case studies to present learners with a realistic situation and require them to respond as the person who must solve a problem (Smith & Ragan, 1999). In order to solve problems, learners select and manipulate several principles. According to Hudspeth and Knirk (1989),A complete case describes an entire situation and includes background information, the actions and reactions of persons involved, the solution, and the possible consequences of the actions taken. Case materials should have enough background information and detail to that they are readable and believable (p. 31).Case-based learning is appropriate for learning to problem solve when there is no one correct solution, particularly with more complex ill-structured problems (Smith & Ragan, 1999). Case studies can be written so that learners use more cognitive strategies as they proceed through increasing levels of instruction. Cases were traditionally used in professional education to teach decision making skills, such as the

Harvard Business School case approach. Use of case-based studies has also become widespread in the field of medical education.For an example of investigative case-based learning in biology, visit http://www.bioquest.org/case99.html.Hudspeth, D., &Knirk, F.G. (1991). Case study materials: Strategies for design and use. Performance Improvement Quarterly, 2(4), 2.Smith, P. and Ragan, T. (1999). Instructional design (2nd ed.). New York: John Wiley & Sons, Inc.   

 Cognitive ApprenticeshipThe focus of this learning-through-guided-experience is on cognitive and metacognitive skills, rather than on the physical skills and processes of traditional apprenticeships. Applying apprenticeship methods to largely cognitive skills requires the externalization of processes that are usually carried out internally. Observing the processes by which an expert listener or reader thinks and practices these skills can teach students to learn on their own more skillfully (Collins, Brown, Newman, 1989, p. 457-548). This method includes:

1. Modeling - involves an expert's

carrying out a task so that student can observe and build a conceptual model of the processes that are required to accomplish the task. For example, a teacher might model the reading process by reading aloud in one voice, while verbalizing her thought processes (summarize what she just read, what she thinks might happen next) in another voice.

2. Coaching - consists of observing students while they carry out a task and offering hints, feedback, modeling, reminders, etc.

3. Articulation - includes any method of getting students to articulate their knowledge, reasoning, or problem-solving processes.

4. Reflection - enables students to compare their own problem-solving processes with those of an expert or another student.

5. Exploration - involves pushing students into a mode of problem solving on their own. Forcing them to do exploration is critical, if they are to learn how to frame questions or problems that are interesting and that they can solve (Collins, Brown, Newman, 1989, 481-482).

    Cognitive Flexibility Hypertext

Cognitive Flexibility Hypertext (CFH) is a hypermedia learning environment that provides users with several nonlinear paths of traversing content through the use of cases, themes, and multiple perspectives. A CFH supports exploration of an ill-structured (ill-defined) knowledge domain through multiple representations of the content, promoting flexible knowledge acquisition to enhance transfer to real-world contexts (Jonassen, Dyer, Peters, Robinson, Harvey, King and Loughner, 1997). The WWW is an ideal medium for designing CFH due to its hyperlinking feature and access to widespread resources that add richness to content.   Collaborative LearningCollaborative learning, also called cooperative learning, is heavily emphasized in most constructivist approaches (Roblyer, Edwards, &Havriluk, 1996). Actually, students working in groups to solve problems achieves many goals that supporters of both constructivism and directed instruction consider to be important. The CTGV finds that collaborative learning is the best way to promote generative learning.Perkins (1991) finds that collaborative learning demonstrates the notion of distributive intelligence, which states that accomplishment is not a function of one

person, but rather a group in which each contributes to the achievement of desired goals. Cooperative learning is an ideal way for students to learn the skills that extend beyond the classroom of sharing responsibility and working together toward common goals. According to Driscoll (2000), collaboration also provides students with a way to understand point of view outside their own. Advances in technology over the past several years have made computer-supported collaborative learning possible. Web-based technologies can make thinking more visible through virtual access to knowledge experts.Driscoll, M. (2000). Psychology of learning for instruction. Needham Heights, MA: Allyn & Bacon.Perkins, D. (1991). Technology meet constructivism: Do they make a marriage? Educational Technology, 31(5), 18-23.Roblyer, M.D., Edwards, J. &Havriluk, M.A. (1996) Learning Theories and Integration Models (Chapter 3). In Roblyer, Edwards, &Havriluk, Integrating educational technology into teaching. Prentice Hall.    Communities of PracticeCommunities of Practice include learners and instructors who interact with one

another and other experts via virtual spaces, to build a reciprocal interchange of ideas, data, and opinions. Transformative styles of communication are characteristic, where the contributor, participator and the lurker or receiver, are "changed" as they share in the goal of learning and knowledge generation and application (Wilson & Cole, 1996).    Computer-Supported Intentional Learning Environments (CSILEs)Essentially, CSILE is a type of computer conference in which learners create communal databases entirely through person-to-group rather than person-to-person communication. CSILE is a collective knowledge building effort that requires students to do planning, goal setting and problem solving. CSILEs are based on a specific environment developed at the Ontario Institute for Studies in Education.    Discovery LearningDiscovery learning has various definitions. At one end of the spectrum we find discovery learning in its simplest form. The tools and information needed to solve a problem or learn a concept are provided

and the learner "makes sense" of them. Another definition is discovery learning as experimentation with some extrinsic intervention -- clues, coaching, and a framework to help learners get to a reasonable conclusion. At the other end of the continuum is the expository teaching model of discovery learning where the learner "discovers" what the teacher decides he is to discover using a process prescribed by the teacher.Bardin, D. (1999). Discovery learning. Retrieved September 10, 2002, from San Diego State University, Encyclopedia of Educational Technology Web site: http://coe.sdsu.edu/eet/Articles/discoverylearn/start.htm   Distributed LearningDistributed Learning is when learning is distributed across space, time, and various media. When telecommunications media is utilized, distributed learning refers to off-site learning environments where learners complete courses and programs at home or work by communicating with faculty and other students through e-mail, electronic forums, videoconferences, and other forms computer-mediated communication and Internet and Web-based technologies. Distributed learning environments "result in a diffuse sense of cognition - where what is

"known" lies in the interaction between individuals and artifacts, such as computers and other technological devices (Dabbagh&Bannan-Ritland, in preparation).Dabbagh, N. &Bannan-Ritland, B. (in preparation - under contract). Online learning and course management systems: Concepts, strategies, and application. Upper Saddle River, NJ: Prentice Hall, Inc. Expected Publication date, January 2003. Epistemic GamesEpistemic games are a formalized structure learning communities use to create knowledge. Conventions are set that represent defined cultural patterns or forms. Working together to generate these forms is called participating in epistemic games. The game involves creating rules or conventions to be followed in generating a given epistemic form. The products of working together are called epistemic forms. "Completed" forms contain new knowledge and adhere to defined structures accepted by the community (Collins & Ferguson, 1993; Morrison & Collins, in press).   Generative learningGenerative learning is a learning process in which learners are given an overall problem and are asked to generate sub-problems, subgoals, and strategies in order to achieve the larger task (Duffy &Jonassen, 1992).

Generative learning strategies can be divided into four major stages: (1) recalling information from long-term memory; (2) integrating new knowledge with prior knowledge; (3) relating prior knowledge to new concepts and ideas in a meaningful way; and (4) connecting new materials to information or ideas already in the learner's mind (Generative Learning, 2000). Using this strategy, a learner relates new ideas to prior knowledge in order to provide meaning to the new material (Ryder, 1998).   Goal-Based Scenarios (GBSs)GBSs offer learners the opportunity to role-play from a certain "character's" perspective or point of view. Their "goal" is for the learner to accomplish a mission or task associated with their role in the scenario. In order to achieve this goal, the learner needs to acquire particular skills and knowledge. This is where and when learning takes place. A GBS, therefore, serves both, to motivate learners, and to give them the opportunity to "learn by doing." "As long as a goal is of inherent interest to learners, and the skills needed to accomplish those goals are the targeted learning outcomes, [we] have a match and a workable GBS (Naidu, 2001, p.2)."A designer of a GBS tends to look at it from the top-down. What drives the design of a GBS is the set of target skills the designer

wishes the student to gain in the GBS. A student, on the other hand, tends to look at a GBS from the bottom-up. What drives a student is the context and structure of the activities the GBS offers.   Inquiry-based LearningInquiry-based learning is an approach to instruction that engages students in investigations to satisfy curiosities. Curiosities are satisfied when individuals construct mental frameworks that adequately explain their experiences (Haury, 1993). The learner's involvement in the learning content fosters skills and attitudes that permit the learner to seek resolutions to questions and issues while constructing new and meaningful knowledge (Inquiry-based Learning: Explanation, 2001, April).   Microworlds/SimulationsIn microworlds, students test 'What do you think will happen if…?' questions in "…constrained problem spaces that resemble existing problems in the real world (Jonassen, 1996, p.237)." Learners generate hypotheses as they use their knowledge and skill to guess what will happen, try out those guesses, and reformulate them based on the results of their actions within the microworld. Microworlds provide the learner

with the observation and manipulation tools necessary to explore and test. The key idea behind microworlds is creating an environment in which students explore the ideas being learned (Jonassen, 1996).

Simulations are similar to microworlds in that they are experiential and model reality. Simulations "…range from models that mirror the simplified essence of reality to elaborate synthetic environments with immersion interfaces that place students inside alternate virtual worlds (Dede, 1996, p.14)." Microworlds differ from simulations in that microworlds are structured to match the user's cognitive level so that it is appropriate to the users needs and level of experience (Rieber, 1992).   MOOs and MUDsMOO is an acronym for MUD, Object-Oriented. MUD stands for Multi-user Dimension, or Dungeon, reflecting its origin as a form of the Dungeons and Dragons game developed for multi-users on the Internet. "In Web-based learning, simulated role portrayal can be facilitated through Multi-User Dialogue (MUD) environments, in which instructors create a multi-user space with a central theme, characters and artifacts (Khan, 2001, p.81, in Walker, 1997)."

Most MUDs still retain this game-like atmosphere, with players earning levels often by shooting and killing other players. MOOs, however, developed as more social spaces, lending themselves readily to use as a virtual classroom, or as spaces for conferences and meetings.Walker, J. (1997, revised 2001). Workshop on synchronous communication in the language arts classroom. Retrieved September 10, 2002, from Georgia Southern University, Department of Writing and Linguistics Web site:http://www2.gasou.edu/facstaff/jwalker/tutorials/cte.html   Problem-Based Learning (PBL)PBL engages the learner in a problem-solving activity. In this process, instruction begins with a problem to be solved rather than content to be mastered (Hsiao, 1996). Students are introduced to a real-world problem and are encouraged to dive into it, construct their own understanding of the situation, and eventually find a solution (Grabowski, Koszalka, &Mccarth, 1998). Major goals of PBL are to help students develop collaborative learning skills, reasoning skills, and self-directed learning strategies (Hsiao, 1996).Five Strategies for Using PBL:

1. The Problem as a Guide - The problem is presented in order to

gain attention prior to presenting the lesson.

2. The Problem as an Integrator or Test - The problem is presented after readings are completed and/or discussed -- these are used to check for understanding.

3. The Problem as an Example - The problem is integrated into the material in order to illustrate a particular principle, concept or procedure.

4. The Problem as a Vehicle for Process - The problem is used to promote critical thinking whereby the analysis of how to solve it becomes a lesson in itself.

5. The Problem as a Stimulus for Authentic Activity - The problem is used to develop skills necessary to solve it and other problems -- skills can include physical skills, recall of prior knowledge, and metacognitive skills related to the problem solving process. A form of authentic assessment of the skills and activity necessary in the content domain (Duffy & Cunningham, 1996, p.190).

    

REALsRich Environments for Active Learning. Based on constructivist ideas, REALs involve students in constantly shaping and reshaping knowledge constructed through their learning experiences. REALs may be implemented through cooperative learning, generative learning, student centered learning, and problem based learning (Schott).Grabinger and Dunlap (1995) used the term to summarize the literature on constructivist learning theory and its five instructional design implications. Learning is active knowledge construction by learners, learners gaining knowledge in realistic contexts and the social negotiation of learning. Thus, learning environments should be characterized by five themes (Bostock, 1998, par. 14):

1. Student responsibility and initiative2. Generative learning strategies3. Authentic learning contexts4. Authentic assessment5. Cooperative support

Bostock, S. (1998). Constructivism in mass higher education: A case study. Learning Technology. Retrieved August 24, 2002, from Keele University, Learning Technology Web site: http://www.keele.ac.uk/depts/cs/Stephen_Bostock/docs/sin98pa6.htmGrabinger, S.R. and Dunlap, J.C. (1995) Rich environments for active learning: a

definition. ALT-J, Journal of the Association for Learning Technology, 3 (2) 5-34.Schott, M. (1999). Rich environments for active learning. Retrieved August 24, 2002, from San Diego State University, Encyclopedia of Educational Technology Web site: http://coe.sdsu.edu/eet/Articles/reals/start.htm   Reciprocal TeachingPalincsar (1986) describes the concept of reciprocal teaching: Reciprocal teaching refers to an instructional activity that takes place in the form of a dialogue between teachers and students regarding segments of text. The dialogue is structured by the use of four strategies: summarizing, question generating, clarifying, and predicting. The teacher and students take turns assuming the role of teacher in leading this dialogue. Purpose: The purpose of reciprocal teaching is to facilitate a group effort between teacher and students as well as among students in the task of bringing meaning to the text.Reciprocal teaching. (2002). Retrieved September 10, 2002, from North Central Regional Educational Laboratory, Pathways to School Improvement Web site: http://www.ncrel.org/sdrs/areas/issues/students/atrisk/at6lk38.htm 

  Situated LearningLave argues that learning as it normally occurs is a function of the activity, context and culture in which it occurs (i.e., it is situated). This contrasts with most classroom learning activities which involve knowledge which is abstract and out of context. Social interaction is a critical component of situated learning -- learners become involved in a "community of practice" which embodies certain beliefs and behaviors to be acquired. As the beginner or newcomer moves from the periphery of this community to its center, they become more active and engaged within the culture and hence assume the role of expert or old-timer. Furthermore, situated learning is usually unintentional rather than deliberate. These ideas are what Lave & Wenger (1991) call the process of "legitimate peripheral participation."Other researchers have further developed the theory of situated learning. Brown, Collins &Duguid (1989) emphasize the idea of cognitive apprenticeship: "Cognitive apprenticeship supports learning in a domain by enabling students to acquire, develop and use cognitive tools in authentic domain activity. Learning, both outside and inside school, advances through collaborative social interaction and the social construction of knowledge." Brown et al. also emphasize the need for a new

epistemology for learning -- one that emphasizes active perception over concepts and representation. Suchman (1988) explores the situated learning framework in the context of artificial intelligence.Situated learning has antecedents in the work of Gibson (theory of affordances) and Vygotsky (social learning). In addition, the theory of Schoenfeld on mathematical problem solving embodies some of the critical elements of situated learning framework.Brown, J. S., Collins, A. &Duguid, S. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32-42.Kearsley, G. (n.d.). Situated learning (J. Lave). Retrieved September 10, 2002, from Explorations in Learning & Instruction: The Theory Into Practice Database Web site: http://tip.psychology.org/lave.htmlLave, J., & Wenger, E. (1991). Situated learning: Legitimate periperal participation. Cambridge, UK: Cambridge University Press.Suchman, L. (1988). Plans and situated actions: The problem of human/machine communication. Cambridge, UK: Cambridge University Press.   WebQuest(s)A WebQuest is "an inquiry-oriented activity

in which some or all of the information that students interact with comes from resources on the Internet" (Goldstein, 1997, para. 1). There are two types of WebQuests:Short term WebQuest - Lasts one to three periods or days and its goal is basic knowledge acquisition. A good short term WebQuest will also include some type of subject integration.Long term WebQuest - Takes between one week and on month to complete. A well planned long term WebQuest involves "extending and refining knowledge" (Goldstein, 1997, Types of WebQuests).Goldstein, B. (1997). So - What's WebQuest???. Retrieved September 10, 2002, from http://webeducator.net/lab37/webquest/wqfaq.html#definition

Implications for Instructional Design (TOP)Objectivism/Behaviorism Cognitivism/Pragmatism Constructivism/Interpretivism

Behavioral objectives Cognitive objectives Authentic assessment methods

Dick & Carey instructional design model

Performance-based assessment

Systems models

Events of Instruction

Learning strategies

Learning taxonomies (Gagné's intellectual skills)

Prerequisite skills

Task analysis

Learning through exploration

Problem-oriented activities

"Rich" environments

Visual formats and mental models

Behavioral objectives

Behaviorists place "an emphasis on producing observable and measurable outcomes in students" (Ertmer& Newby, 1993, p. 56). They believe that learning occurs when learners show the correct response to a certain stimulus (Smith & Ragan, 1999). The current instructional design application of behavioral objectives is reminiscent of these behaviorist views. However, most current instructional designers writing objectives based on action do not share the behaviorists' disinterest in the cognitive processes that also take place. Rather, they write objectives with an attempt to extract "best evidence" of the cognitive processes that cannot be directly observed.

Ertmer, P. and Newby, T. (1993). Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective. Performance Improvement Quarterly, 6 (4), 50-72.

Cognitive objectives

Cognitive psychology has influenced the types of goals and objectives that are developed as a result of task analysis (Smith & Ragan, 1999). In addition to observable performance, attention is now given to the underlying "understanding" of a performance. For example, an objective might specify that a learner should be able to explain the reasoning behind his/her performance. Bloom's taxonomy addresses the cognitive domain.

Smith, P. and Rag, T. (1999). Instructional design (2nd ed.). New York: John Wiley & Sons, Inc.

 

Authentic assessment methods

As a result of the change in goals and methods of education, constructivist learning environments tend to use more qualitative assessment strategies rather than quantitative ones (Roblyer, Edwards &Havriluk, 1996). For example, one of the more popular ways to assess students in a constructivist learning environment is through portfolios. Portfolios consist of samples of students' work and products developed. A portfolio might also include teacher narration describing students' work habits, strengths and weaknesses. Performance-based assessments and checklists of criteria used to judge students' performance might also be included (Linn, 1994).

According to Wiggins (1990), "Assessment is authentic when we directly examine student performance on worthy intellectual tasks" (par. 1). Assessments should be built on

Smith, P. and Ragan, T. (1999). Instructional design (2nd ed.). New York: John Wiley & Sons, Inc.

 

Dick and Carey instructional design model

Walter Dick and Lou Carey (1996) advocate a systems approach model for designing instruction in the fourth edition of their text, The Systematic Design of Instruction. Their work is based on the behaviorist view that there is a predictable link between a stimulus and the response it produces in a learner (Colaric, n.d.). It is the designer's responsibility to determine the sub-skills a student must master in order for the behavior to be learned and choose the stimulus and strategy for instruction in order to assemble the sub-skills. The basic steps in the Dick and Carey instructional design model are as follows:

1. Determine instructional goal2. Analyze the instructional goal3. Analyze learners and contexts4. Write performance objectives5. Develop assessment instruments

Learning strategies

Cognitivists conduct learner analyses to determine a learner's predisposition to learning and decide how to design instruction so that is can be assimilated according to the learner's existing mental structures (Ertmer& Newby, 1993). Learner characteristics are considered when a designer plans what instructional techniques, called learning strategies, to use in the instruction (Smith & Ragan, 1999). Strategies that focus on structuring, organizing, and sequencing information for optimal processing are based on cognitivism. For example, outlining, summarizing, synthesizing, and advance organizers.

Ertmer, P. and Newby, T. (1993). Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective. Performance Improvement Quarterly, 6 (4), 50-72.

Smith, P. and Ragan, T. (1999). Instructional design (2nd ed.). New York: John Wiley & Sons, Inc.

 

intellectual challenges such as problem-solving, experimental research, discussion, and writing. Furthermore, the goal of assessment is primarily to support the needs of the learner. The best tests should teach students the type of work that matters most.

Linn, R. (1994). Performance assessment: Policy promises and technical measurement standards. Educational Researcher, 23(9), 4-14.

Roblyer, M.D., Edwards, J. &Havriluk, M.A. (1996). Learning theories and integration models (Chapter 3). In Roblyer, Edwards &Havriluk, Integrating educational technology into teaching. Prentice Hall.

Wiggins, G. (1990). The case for authentic assessment. [Electronic Version]. Washington DC: ERIC Clearinghouse on Tests Measurement and Evaluation. (ERIC Identifier ED328611). Retrieved September 3, 2002, from:http://www.ed.gov/databases/ERIC_Digests/ed328611.html

  

Learning through exploration

While constructivists differ among themselves about how much guidance a

6. Develop instructional strategy7. Develop and select instructional

materials8. Design and conduct formative

evaluation9. Revise instruction10. Summative evaluation

For a visual of the Dick and Carey design model, visit http://www.student.seas.gwu.edu/~tlooms/ISD/GIFS/dc_design.gif.

Colaric, S. (n.d.). Systems approach model for designing instruction. Retrieved September 2, 2002, from Pennsylvania State University, Personal Web page: http://www.personal.psu.edu/staff/s/m/smc258/KB/DickCarey.html

Dick, W. and Carey, L. (1996). The systematic design of instruction (4th ed.). New York: Harper Collins Publishing.

 

Performance-based assessment

The idea of assessing students based on observable performance started with

Learning taxonomies

Robert Gagne, among others who developed taxonomies, made one of the first attempts to classify learning behaviors and supply specific measures for determining different levels of learning. Gagne developed a taxonomy for intellectual skills, one of his five learned capabilities. Closely related to the development of taxonomies are instructional objectives and instructional systems design.

Kearsley, G. (n.d.). Taxonomies. Retrieved September 7, 2002, from Explorations in Learning & Instruction: The Theory Into Practice Database Web site: http://tip.psychology.org/taxonomy.html

 

 

Prerequisite skills

A prerequisite is something a person must know or be able to do before they are able to learn something else (Smith & Ragan, 1999). To determine prerequisite information, an analysis must be done

teacher should provide, all agree that there should be some flexibility in achieving desired goals (Roblyer, Edwards &Havriluk, 1996). Most constructivist approaches emphasize exploration over "getting the right answer." A few of the radical constructivists believe that students should have total freedom and infinite time when it comes to learning through exploration. However, Perkins (1991) states, "Education given over entirely to WIG (without any given) instruction would prove grossly inefficient and ineffective, failing to pass on in straightforward ways the achievements of the past" (p. 20).

Perkins, D. (1991). Technology meets constructivism: Do they make a marriage? Educational Technology, 31 (5), 18-23).

Roblyer, M.D., Edwards, J. &Havriluk, M.A. (1996). Learning theories and integration models (Chapter 3). In Roblyer, Edwards &Havriluk, Integrating educational technology into teaching. Prentice Hall.

 

 

behaviorism. According to Ertmer& Newby (1993), "Behaviorism equates learning with changes in either the form or frequency of observable performance" (p. 55). Once a student can display the proper response following the presentation of a certain environmental stimulus, learning has been achieved. Traditional behaviorist assessment makes no evaluation of the knowledge structure or mental processes leading to a student's response.

Criterion-referenced assessment, which measures what a student can do as compared to behavior described in specific learning objectives (Smith & Ragan, 1999), is based on behaviorist principles. Such assessment is used to determine a student's individual competency in skills defined as goals for instruction, as opposed to rank them with other learners.

Ertmer, P. and Newby, T. (1993). Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective. Performance Improvement Quarterly, 6 (4), 50-72.

Smith, P. and Ragan, T. (1999). Instructional design (2nd ed.). New York: John Wiley & Sons, Inc.

from the learner's (novice's) perspective, rather than the expert's perspective. An expert tends to overlook some of the things they needed to know in order to achieve the learning goal. Determining prerequisite skills does not specify instructional strategies. An analysis for prerequisites can be used for a top-down, problem-based environment as well as a bottom-up structured instructional strategy.

Smith, P. and Ragan, T. (1999). Instructional design (2nd ed.). New York: John Wiley & Sons, Inc.

 

 

Task analysis

With the shift to cognitivism, analysis of relevant concepts goes beyond behavioral observations of job performance. "Content analysis has outgrown the mere listing of statements the learner will be able to recite. It has advanced way beyond the old conventions of S-R tables" (Tiemann and Markle, 1984, p. 26).

Problem-oriented activities

Students solving problems, whether in a specific content area or in an interdisciplinary approach, is the focus of most constructivist models (Roblyer, Edwards &Havriluk, 1996). For example, one problem might require students to use only math skills, while another might require math, science and language arts skills. According to Jungck (1991) constructivist methods often integrate problem posing, problem solving and "persuasion of peers" (p. 155). Furthermore, problems can be presented with specific goals, as "what if" questions or as open-ended questions. Problem solving in a constructivist learning environments is usually more complex and demands more time and use of varied skills than problem solving with directed instruction.

Jungck, J. (1991). Constructivism, computer exploratoriums, and collaborative learning: Construction scientific knowledge. Teaching Education, 3 (2), 151-170.

Roblyer, M.D., Edwards, J. &Havriluk, M.A. (1996). Learning theories and integration models (Chapter 3). In Roblyer, Edwards &Havriluk, Integrating Educational

 

Systems models

Behaviorist theories contributed to the development of "more efficient methods of creating directed instruction" (Roblyer, Edwards &Havriluk, 1996, p. 62). Systems models take information from learning theories and turn them into step-by-step procedures for planning instruction. Systems models were developed in response to problems teachers were having in satisfying the needs of large numbers of students. According to Saettler (1990) these models were initially embraced more by military and industrial trainers than by K-12 classroom teachers. While systems approaches are heavily used in the design and development of self-contained tutorials, teachers can also use the same approach to plan their own directed instruction with technology. Systems models can help teachers evaluate the effectiveness of their own teaching as well as the usefulness of computer-based resources. Most instructional design models and methods are rooted in systems models.

Roblyer, M.D., Edwards, J. &Havriluk, M.A. (1996). Learning theories and integration models (Chapter 3). In Roblyer, Edwards &Havriluk, Integrating educational technology into teaching. Prentice Hall.

During task analysis, goal statements are transformed into a format that can be used to guide the rest of the instructional design process (Smith & Ragan, 1999). To complete a learning task analysis:

1. Write a learning goal.2. Determine the types of learning of

the goal.3. Conduct an information-processing

analysis of that goal.4. Conduct a prerequisite analysis

and determine the type of learning of the prerequisites.

5. Write learning objectives for the learning goal and each of the prerequisite (p. 63).

Upon completion of a task analysis, the designer has a list of goals describing what learners should know or be able to do upon completion of instruction, as well as the prerequisite skills an knowledge needed to achieve those goals.

Smith, P. and Ragan, T. (1999). Instructional design (2nd ed.). New York: John Wiley & Sons, Inc.

Tiemann, P. and Markle, S. (1984). On

Technology into Teaching. Prentice Hall.

  

"Rich" environments

Most constructivist approaches advocate what Perkins (1991) terms "richer learning environments" (p. 19) as opposed to the "minimalist" classroom environment, which depends on the teacher, textbooks and prepared materials (Roblyer, Edwards &Havriluk, 1996). According to Perkins, most constructivist models use any combination of the following five basic resources.

Information banks - allow access to required information (e.g., books, encyclopedias)

Symbols pads - supports learners' short-term memory (e.g., notebooks, laptops)

Construction kits - facilitates manipulation and building (e.g., Legos, Tinkertoys, Logo)

Phenomenaria - allows exploration (e.g., terrarium, computer simulation)

Task managers - gives help and feedback when tasks are completed

Saettler, P. (1990). The evolution of American educational technology. Englewood, CO: Libraries Unlimited.

 Events of Instruction/Conditions of Learning

Events of Instruction

Preparation for Learning 1. Attending - gain learner's attention 2. Expectancy - inform learner of objectives 3. Retrieval - recall relevant information and/or skills to working memory or stimulate recall of prior learning.

Acquisition and Performance 4. Selective perception - remembering stimulus features, distinctive features. 5. Semantic encoding - provide learning guidance 6. Retrieval and responding - elicit a performance7. Reinforcement - provide informative feedback

Retrieval and Transfer 8. Cueing retrieval - assess performance 9. Generalizing - applying learning to a new situation

Barba, R. (1997). Events of instruction (Robert Gagne). Retrieved September 10, 2002, from

getting expertise into an expert system. Performance and Instruction Journal, 23 (9), 25-29.

(e.g., teachers, electronic tutors)

Perkins, D. (1991). Technology meets constructivism: Do they make a marriage? Educational Technology, 31 (5), 18-23.

Roblyer, M.D., Edwards, J. &Havriluk, M.A. (1996). Learning theories and integration models (Chapter 3). In Roblyer, Edwards &Havriluk, Integrating educational technology into teaching. Prentice Hall.

 

Visual formats and mental models

Cognition and Technology Group at Vanderbilt (CTGV) is a strong advocate of helping students build good "mental models" of problems to be solved (Roblyer, Edwards &Havriluk, 1996). To promote the use of these mental models, teachers should present problems in visual rather than written formats. CTGV researchers (1990) say, "Visual formats allow students to develop their own pattern recognition skills," and they are "dynamic, rich , and spatial" (p. 3). The use of visual formats may be particularly important for low-achieving students who have difficulty reading.

Cognition and Technology Groups at

San José University, College of Education Web site: http://www.sjsu.edu/depts/it/edit186/gagne.html

 

Vanderbilt (1990). Anchored instruction and its relationship to situated cognition. Educational Researcher, 19 (6), 2-10.

Roblyer, M.D., Edwards, J. &Havriluk, M.A. (1996). Learning theories and integration models (Chapter 3). In Roblyer, Edwards &Havriluk, Integrating educational technology into teaching. Prentice Hall.

Formative EvaluationCoined by Michael Scriven (1967), the term "formative evaluation" is used for the type of evaluation that occurs during the developmental stage of the instructional design process (Seels& Glasgow, 1990). The instructional designer evaluates materials during the process of instructional development to determine where there are weaknesses in the instruction so that revisions can be made (Smith & Ragan, 1999). There are several variations of formative evaluation.

Explore the models below to learn more.

Dick & Carey Flagg

Clinical (one-to-one evaluation)

Small group evaluation

Field trial

Needs assessment

Pre-production formative evaluation

Production formative evaluation

Implementation formative evaluation

Seels& Glasgow Smith & Ragan

Internal review

Tutorial and small-group tryouts

Operational tryout

Design reviews

Expert reviews

Learner validation

Ongoing evaluation

Tessmer Evaluation for Technology-based Instruction

Expert review

One-on-one evaluation

Small group evaluation

Field test

Rapid prototype

Alpha testing

Beta testing

Summative Evaluation

Instructional designers may be involved in evaluating the effectiveness of instructional materials after the materials have been implemented into the instructional contexts for which they were designed (Smith & Ragan, 1999). However, Seels& Glasgow (1990) find that "summative evaluation is seldom carried out by the designers responsible for developing the original instruction and is not an integral part of our system model" (p. 199). Nevertheless, it is important to be familiar with the process. Like formative evaluation, there are several variations of summative evaluation.

Explore the models below to learn more.

Dick & Carey Smith & Ragan Kirkpatrick

Expert judgementField trial

Determine goals of evaluationSelect indicators of successSelect orientation of evaluationSelect design of evaluationDesign or select evaluation measuresCollect dataAnalyze dataReport results

Level 1 - Reaction

Level 2 - Learning

Level 3 - Transfer

Level 4 - Results

Note: In Kirkpatrick's four-level model, evaluation always begins with Level 1 and each successive level builds on information gathered from the previous level.

Expert Judgement

According to Dick and Carey (1996), the expert judgement phase of summative evaluation is used to find out if either current or candidate instruction can meet an organization's identified instructional needs. The following

Determine goals of evaluation

Smith and Ragan (1999) suggest to determine goals of evaluation as the first step in a goal-based summative evaluation. The most important part of this stage is determining questions that should be answered as a result of the evaluation. The client organization and/or funding agencies

Level 1 - Reaction

Level 1 of Kirkpatrick's model of evaluation measures the the participants' reaction to the instructional program (Winfrey, 1999). After completing the program students are asked to evaluate the training. This is usually done using a questionnaires, or what are sometimes referred to as "happy sheets" or

activities are part of the expert judgement phase of summative evaluation when reviewing candidate instruction:

1. evaluating the congruence between the organization's instructional needs and candidate instruction

2. evaluating the completeness and accuracy of candidate instruction

3. evaluating the instructional strategy contained in the candidate instruction

4. evaluating the utility of the instruction

5. determining current users' satisfaction with the instruction (p. 323)

The expert judgement phase has already been accomplished if the instruction was tailored to the identified needs of the organization, systematically design and developer and been through formative evaluation. However, the instruction must be subjected to expert judgement if the

and other stakeholders should identify the questions. These questions will guide the remainder of the summative evaluation. Questions might include:

Does implementation of the instruction solve the problem identified in the needs assessment?

Do the learners achieve the goals of the instruction?

What are the costs of the instruction? What is the "return on investment" of the instruction? (p. 355)

Both the client and evaluator should agree on the questions before moving on to subsequent steps of summative evaluation.

Smith, P. and Ragan, T. (1999). Instructional design (2nd ed.). New York: John Wiley & Sons, Inc.

 

 

 Select indicators of success

In the select indicators of success phase of summative evaluation, Smith and Ragan

"smile sheets" (Kruse, 2002). This level of evaluation differs from surveys used in formative evaluation in that the questionnaires are distributed to the entire student population. A typical Level 1 questionnaire might ask questions about the relevance of objectives, interest level, interactivity, ease of navigation and perceived transferability to the workplace. Most organizations conduct at least a Level 1 evaluation because it is the easiest and cheapest to administer.

Carliner, S. (2002). Summary of the Kirkpatrick model. Retrieved October 11, 2002, from VNU Business Media, An Overview of Online Learning http://www.vnulearning.com/wp/kirkpatrick.htm

Kruse, K. (2002). Evaluating e-Learning: Introduction to the Kirkpatrick model. Retrieved October 11, 2002, from the eLearning Guru.com Web site: http://www.e-learningguru.com/articles/art2_8.htm

 

 

Level 2 - Learning

organization is unfamiliar with the instruction and its developmental history (Dick & Carey, 1996). Usually expert judgementis used to select from the available instructional options in order to choose one or two that are most promising for a field trial.

Dick, W. and Carey, L. (1996). The Systematic design of instruction, 4th ed. New York: Harper Collins Publishing.

 

Field Trial

The purpose of the field trial phase of summative evaluation is to determine the effectiveness of instruction with the target group in the intended setting (Dick & Carey, 1996). There are two parts to the field trial phase: outcomes analysis and management analysis. The outcomes analysis reviews the impact of the instruction on the learner, the job and the organization. Management analysis assesses "instructor and supervisor attitudes related to learner

(1999) recommend that the evaluator and his clients "determine where to look for evidence of the impact of the instructional program (p. 355). The following questions can be used to help target the program's impact: If the program is successful, what will we observe

In instructional materials and learners' activities?

In teachers'/trainers' knowledge, practice, attitudes?

In learners' understanding, processes, skills, attitudes? (p. 356)

Smith, P. and Ragan, T. (1999). Instructional design (2nd ed.). New York: John Wiley & Sons, Inc.

 

Select orientation of evaluation

Once questions for summative evaluation are identified, the evaluator and clients must agree on the most appropriate orientation for answering the questions (Smith & Ragan, 1999). The following issues are addressed by decisions on orientation.

Level 2 of Kirkpatrick's model of evaluation measures how much participants learned (Carliner, 2002). This is often accomplished with a criterion-referenced test, with the criteria being the objectives for the course. This type of evaluation ensures quality through conformance to course requirements. "Assessing at this level moves the evaluation beyond learner satisfaction and attempts to assess the extent students have advanced in skills, knowledge, or attitude" (Winfrey, 1999, Level 2 Evaluation - Learning, para. 1). Several methods may be used to evaluate learning including formal and informal testing, team assessment and self-assessment. Usually pre- and posttests are administered in order to determine the amount of learning that occurs.

Carliner, S. (2002). Summary of the kirkpatrick model. Retrieved October 11, 2002, from VNU Business Media, An Overview of Online Learning http://www.vnulearning.com/WP/kirkpatrick.htm

Winfrey, E. C. (1999). Kirkpatrick's four levels of evaluation. Retrieved October 11, 2002, from San Diego State University, Encyclopedia of Educational Technology Web site: http://coe.sdsu.edu/eet/Articles/k4levels/

performance, implementation feasibility, and costs" (p. 323).

Dick, W. and Carey, L. (1996). The Systematic design of instruction, 4th ed. New York: Harper Collins Publishing.

 

 

 

Will the approach be more goal-based or goal-free?

If one perspective predominates, will there be aspects of the other orientation?

Are quantitative or qualitative data appropriate as evidence to answer the questions?

Will a more experimental or naturalistic approach be used? (p. 356)

Smith, P. and Ragan, T. (1999). Instructional design (2nd ed.). New York: John Wiley & Sons, Inc.

 

Select design of evaluation

According to Smith and Ragan (1999), "Evaluation designs describe what data will be collected, when the data should be collected, and under what conditions data should be collected in order to answer the evaluation questions" (p. 356). Instructional designers should actually begin developing this plan during needs assessment when the learning goals and reasons for identifying them can clearly be recalled. Three issues to consider when designing evaluation are internal validity

index.htm

 

 

 Level 3 - Transfer

Level 3 of Kirkpatrick's model of evaluation attempts to answer the question, "Are the newly acquired skills, knowledge, or attitude being used in the everyday environment of the learner?" (Winfrey, 1999, Level 3 Evaluation - Transfer, para. 1). Typically a Level 3 evaluation assesses the amount of learned material students actually use in their work environment 6 weeks to 6 months (or longer) after completing a course (Carliner, 2002). This type of evaluation may be conducted in the form of tests, observations, surveys, and interviews with co-workers and supervisors.

Carliner, S. (2002). Summary of the kirkpatrick model. Retrieved October 11, 2002, from VNU Business Media, An Overview of Online Learning http://www.vnulearning.com/WP/kirkpatrick.htm

Winfrey, E. C. (1999). Kirkpatrick's four levels of evaluation. Retrieved October 11, 2002,

(many things cause changes in learners' performance/attitude other than the instructional program), external validity (ability to generalize results of evaluation to learners or contexts not part of the evaluation) and control (designer determines the limits of what can be done with internal and external validity). Using comparison groups and randomization can help a designer deal with the issues associated with evaluation.

Smith, P. and Ragan, T. (1999). Instructional design (2nd ed.). New York: John Wiley & Sons, Inc.

 Design or select evaluation measures

When designing summative evaluation, most evaluators plan for several different measures of the effectiveness of the instructional program (Smith & Ragan, 1999). An evaluator most often plans for measurement in the categories of payoff, learning, attitude, implementation and cost. Multiple measures can also be used within any one of these categories.

Smith, P. and Ragan, T. (1999). Instructional design (2nd ed.). New York: John Wiley & Sons, Inc.

from San Diego State University, Encyclopedia of Educational Technology Web site: http://coe.sdsu.edu/eet/Articles/k4levels/index.htm

 

 

 

 Level 4 - Results

Level 4 of Kirkpatrick's model of evaluation measures the impact of training from a business perspective. Success of the training may be evaluated in terms of increased production, improved quality, decreased costs, reduced frequency of accidents, increased sales, and even higher profits or return on investment (Winfrey, 1999). According to Kruse, "The only scientific way to isolate training as a variable would be to isolate a representative control group within the larger student population, and then rollout the training program, complete the evaluation, and compare against a business evaluation of the non-trained group" (Level Four: Business Results, para. 1). Unfortunately, Level 4 evaluation is rarely completed because of difficulty in obtaining appropriate business data and ability to

 Collect data

After the evaluator selects or develops appropriate measurement instruments, the next phase of summative evaluation is to plan for the collection of data (Smith & Ragan, 1999). Within the data collection plan should be scheduling of data collection periods. These are determined by the evaluation design and the types of payoff and implementation measures. It is the responsibility of the evaluator to ensure that all data collection policies are strictly followed.

Smith, P. and Ragan, T. (1999). Instructional design (2nd ed.). New York: John Wiley & Sons, Inc.

Analyze data

Smith and Ragan (1999) recommend analyzing data in a fashion that will be easy for the decision makers "to see how the instructional program affected the problem presented in the needs assessment" (p. 360). Descriptive statistics (e.g. means, range, frequency) or inferential statistics (differences between two instructional programs or from pretest to posttest within the same program) may

isolate training as a unique variable.

Kruse, K. (2002). Evaluating e-Learning: Introduction to the Kirkpatrick model. Retrieved October 11, 2002, from the eLearning Guru.com Web site: http://www.e-learningguru.com/articles/art2_8.htm

Winfrey, E. C. (1999). Kirkpatrick's four levels of evaluation. Retrieved October 11, 2002, from San Diego State University, Encyclopedia of Educational Technology Web site: http://coe.sdsu.edu/eet/Articles/k4levels/index.htm

 

be required.

Smith, P. and Ragan, T. (1999). Instructional design (2nd ed.). New York: John Wiley & Sons, Inc.

Report results

In Smith and Ragan's (1999) summative evaluation model, report results is the final phase. According to Morris (1978), the following sections should be included in a summative evaluation report.

1. Summary2. Background3. Description of evaluation study4. Results5. Discussion6. Conclusions and recommendations

Smith, P. and Ragan, T. (1999). Instructional design (2nd ed.). New York: John Wiley & Sons, Inc.

Morris, L. L. (1978). Program evaluation kit. Beverly Hills, CA: Sage Publications.

Scrieven, M. (1967). The methodology of evaluation (AERA Monograph series on curriculum evaluation, No. 1). Chicago IL: Rand McNally.

Seels, B. and Glasgow, Z. (1990). Exercises in instructional design. Columbus, Ohio: Merrill Publishing Company.

Smith, P. and Ragan, T. (1999). Instructional design (2nd ed.). New York: John Wiley & Sons, Inc.