revising instructional materials
TRANSCRIPT
Revising Instructional
MaterialsLaSheena Nation
Objectives Describe various methods for summarizing data
obtained from formative evaluation Summarize date obtained from evaluation studies Given summarized formative evaluation data,
identify weaknesses in instructional materials and instructor-0led instruction
Given formative evaluation data for a set of instructional materials, identify problems in the materials, and suggest revisions for the materials
Background During the revision process, designers must keep
a system perspective on their work and remain open to possibilities that revisions may be warranted in any stage of the design process.
Two basic types of revision should be considered with your materials:o Changes made to the content or substance of the
materials to make them more accurate or effective as a learning tool
o Changes related to the procedures employed in using your materials
Concepts There are many different ways in which data
collected in a formative evaluation may be summarized to point areas of learner difficulties and possible revisions. The methods that will be described are merely suggestions, you may find other techniques that may work better for your data.
Data Analysis for One-to-One Trials
The designer has five types of basic information available1. Learner characteristics and entry level behavior 2. Direct response to instruction3. Learning time4. Posttest performance5. Response to an attitude questionnaire
Data Analysis for One-to-One Trials
There is a three step process in collecting data1. Step 1: Describe the learners and indicate their
performance based off of entry level measurements.2. Step 2: Compile all comments and suggestions about
the instructor that resulted from going through with each individual learner.
3. Step 3: Obtain the individuals item performance and them combine it with item scores for each objective and for a total score.
Data Analysis for One-to-One Trials
Revising the instruction with information collected from what's available and through the 3 step process.
There are three main sources of information/evaluation for changes to be made 1. Learner suggestions2. Performance of the learner3. Your own insights
Data Analysis for One-to-One Trials
The goal is to determine if the rubric or test items are fault by basing them on the learners performance. If a flaw is found to exists then changes must be made to make them clear and consistent with the overall objective and the intent of the instruction. If there is no flaw found, but the learners sis not perform well, then consideration of changing the instructor will be reviewed.
Data Analysis for Small-Group and Field Trials
The small group formative evaluation provides the designer with a somewhat different data summary situation. The data from eight to twenty learners are of greater collective interest than individual interest; therefore showing the overall problems and reactions this representative group of learners have.
Data Analysis for Small-Group and Field Trials
The available data usually includes the following:1. Item performance on the pretest2. Posttest3. Responses to an attitude questionnaire4. Learning and test time 5. Comments made directly in materials
Data Analysis for Small-Group and Field Trials
The fundamental unit of analysis for all the assessments is the individual assessment item. Performance on each item must be scored either correct or incorrect. If an item has multiple parts, then each part should be scored and reported separately so that the information is not lost. The individual item information is required for three reasons.
Data Analysis for Small-Group and Field Trials
What are the three reason individual item information is required?1. To determine weather there are particular problems
with the item or weather it is effectively measuring the performance described in the objective.
2. To identify the nature of difficulties learners are having with the instruction.
3. To indicate learner performance on an objective, and eventually on an entire test.
Data Analysis for Small-Group and Field Trials
Graphing Learners Performance can be done In two ways1. Display data through carious graphing techniques2. Summarizing formative evaluations data involves the
instructional an analysis chart
Data Analysis for Small-Group and Field Trials
Ways data can be used: o For attitude questionnaireo For obtaining comments /reacting to materialso To discuss alternative approaches during wither small-
group or field-trials evaluations
Sequencing for Examining Data
The suggested sequence to use data are as follows:o Instructional analysis and entry level skillso Objectives, pretest, and posttesto Instructional strategyo Learning timeo Instructional procedures
Revision Process As you begin the revision process it is suggested
that you summarize you data based upon the following:o In the left column, list the components of the
instructional strategy.o Add additional columns for summarizing
• Problems identifies• Proposed changes• Evidence gathered for identifying problem
Revision Process of Selected Materials and Instructior-Led Instruction
Working with selected materials avails little opportunity to revise materials directly, especially if they are copyrighted or commercially produced.o Consider the following adaptions for the future
1. Omit potions of the instruction2. Include other available materials3. Develop supplementary instruction
Instructional Design Process
Learn More
