Designing and Conducting Formative Evaluations

Christopher Walker

BackgroundThis presentation will present

information on formative evaluations, their use the design and conduct of them

ObjectiveDiscussion and clarity on the following:• Describe the purposes for and various stages of formative evaluation of

instructor-developed materials, instructor-selected materials, and instructor-presented instruction

• Describe the instruments used in a formative evaluation

• Describe the instruments used in a formative evaluation

• Develop an appropriate formative evaluation plan and construct instruments for a set of instructional materials or an instructor presentation

• Collect data according to a formative evaluation plan for a given set of instructional materials or instructor presentation

Dick and Carey’s Model

What is Formative Evaluation?

• Formative evaluation is applied to the initial development of instruction

• It has three stages

• It was originally used as a process to improve instruction after the first draft of instruction was developed

Formative Evaluation Designs• The instructional strategy is the framework for the design. The components will

help you create questions that will allow you to evaluate instruction better

• The evaluation design needs to yield information about the location of and the reasons for any problems. This prevents the evaluation that is too narrow or too broad

What questions should be asked about each component of the materials?• Are the materials appropriate for the type of learning outcome?

• Do the materials include adequate instruction on the subordinate skills, and are these skills sequenced and clustered logically?

• Are the materials clear and readily understood by representative members of the target group?

• What is the motivational value of the materials?

What questions should be asked about each component of the materials? continued

• Can the materials be managed efficiently in the manner they are mediated?

• Do the learners find the materials relevant to their needs and interest?

• Are they confident as they work through the materials?

• Are they satisfied with what they have learned?

Learning and Learner Specialist in Formative Evaluation • SMEs may be able to comment on the accuracy and currency of the instruction

• Learner specialist may be able to provide insights into the appropriateness of the material for the eventual performance content

• Learning specialist may be able to critique your instructions related to what is known about enhancing that particular type of learning

• Always remember that it is imperative to have all instruction reviewed by a specialist

One-to-one evaluation with learners• The purpose of one-to-one evaluation is to identify and remove the mist obvious

errors in the instruction and to obtain initial performance indications and reactions to the content by learners in several areas

• Clarity

• Impact

• Feasibality

One-to-one evaluation with learners

Small group evaluation• Small group evaluation has two main purposes.

• 1. to determine the effectiveness of changes made by one-to-one evaluation and identify any remaining problems associated with learners

• 2. small group evaluation is used to determine how learners will use the instruction without the interaction of the instructor

Small group evaluation

Field Trial• Field trial is the third stage in the formative evaluation

• Its used to determine whether the changes in the instruction after the small group stage was effective or more changes need to be made

• It determines if the instruction can be used in the context in which it is intended

Field trial evaluation

Formative evaluation in the performance context• The purpose of the performance context formative evaluation is to make three

fundamental determinations

• 1. what has been the impact on the organization

• 2. what suggestions do the learners and others think can improve the instruction

• 3. the learners how do they feel about using their new skills in new settings and have they been successful in using them

Formative evaluation of selected materials• The instructor that conducts a field trial us able to observe the progress and

attitudes of learners using a set of adopted or adapted materials

• Materials of formative evaluation concludes that the instructor choose existing materials and resources.

One-to-one

• Participation by SMEs

• A. You should provide the expert with the following

• B. You should be looking for verification of the following

• C. The number of SMEs you should approach for assistance varies with the complexity of the information and skills covered in your materials

Small group evaluation

• Identify a group of learners that typifies your target population

• Arrange for a group to participate

• During the pretest, instruction, and posttest make notes about suggestions for instructors

• Administer the pretest of required entry skills

Examples

LifelongLearnerSummary

As a change agent, formative evaluation is to be done regularly. This ensures that the information or data that is obtained is current

cwalker6541@myasu.alasu.eduChristopher Walker