chapter 10 ppt eval & testing 4e formatted 01.10 kg edits

© 2013 Springer Publishing Company, LLC.

Chapter 10Assembling and Administering TestsOermann & GabersonEvaluation and Testing in Nursing Education4th edition


Effects on Reliability and Validity

♦ Test appearance and administration procedures can affect reliability and validity– Design flaws and clerical errors may ↑

measurement error and ↓ reliability of scores– Test administration problems may ↓ reliability

of scores and ↓ validity of inferences from those scores

2


Test Design Rules

♦ Allow enough time– Designing and assembling the test are not simply

clerical tasks– Teacher should make all decisions about the

appearance of the test– Allow enough time for this phase to avoid errors

that could affect scores

3


Test Design Rules

♦ Arrange test items logically– Group items of the same format together– Within each item format, arrange items according

to order in which content was taught– Within each content area, arrange items in order

of difficulty (easy to difficult)

4


Test Design Rules (cont’d)

♦ Write directions– General directions• Time allowed• How to record answers• Whether students may ask questions• Whether students may write on the test booklet

5


Test Design Rules

♦ Write directions (cont’d)– Item-specific directions• Multiple-choice: correct or best response• Matching: basis for the match • Completion: whether spelling counts• Computation: degree of precision• Essay: length of answer; whether organization, spelling,

grammar will count

6


Test Design Rules

♦ Use a cover page– For general directions– Can be numbered to maintain test security– Protects test contents from view during

distribution• Allows all students to have equal testing time

7


Test Design Rules

♦ Avoid crowding– Helps students to read efficiently and avoid errors

in recording answers– Ample space between and within items– Indent certain elements– Avoid printing multiple-choice options in tandem

8


Poor Spacing

Which method of anesthesia involves injection of an agent into a nerve bundle that supplies the operative site?

a. General; b. Local; c. Regional; d. Spinal; e. Topical

9


Better Spacing

Which method of anesthesia involves injection of an agent into a nerve bundle that supplies the operative site?a.Generalb.Localc.Regionald.Spinale.Topical

10


Test Design Rules

♦ Keep related material together– Multiple-choice stem and responses on same page– Both columns of matching exercise side by side on

one page, with the related directions– Stimulus and related items for context-dependent

and interpretive exercises on same page, if possible

11


Test Design Rules

♦ Facilitate scoring– Hand scoring• Separate answer sheet• Answers in test booklet—arrange items to

permit easy scoring– e.g., columns of Ts and Fs to left of true-false items for

students to circle

12


Poorly Arranged for Hand Scoring

1. A stethoscope is required to perform auscultation. T F

13


Arranged for Easy Scoring

T F 1. A stethoscope is required to perform auscultation.

14


Poorly Arranged for Hand Scoring

1. List 3 responsibilities of the circulating nurse during induction of general anesthesia. ________, ________, _________

15


Arranged for Easy Scoring

1-3. List 3 responsibilities of the circulating nurse during induction of general anesthesia.

1. _____________________________2. _____________________________3. _____________________________

16


Test Design Rules

♦ Random pattern of correct answers to objectively scored items – Teachers may inadvertently favor certain keyed response

positions.– Teachers may arrange test items so that correct answers

form a pattern for easy scoring.– Students who detect a pattern of correct answers may use

this information to obtain higher scores than their knowledge would merit.

– Use response positions with approximately equal frequency.

17


Test Design Rules

♦ Arrange response alternatives in logical order– Alphabetical or chronological order, or order

of size or degree– Reduces reading time – Tends to randomly distribute the correct

answer position

18


Test Design Rules

♦ Number items continuously throughout test– Helps students find items they may have skipped– Helps to prevent student errors when recording

answers on separate answer sheet

19


Test Design Rules

♦ Proofread– Assure that test is free of spelling, structural,

and typographical mistakes that contribute to measurement error

– Spell-checking programs not sufficient– May ask another teacher to proofread

20


Test Design Rules

♦ Prepare and verify an answer key– Facilitates efficient scoring – Verifies test item accuracy– Teacher should:• prepare ideal responses to essay items.• identify intended responses to completion items.• decide point values of required answer elements if

scoring essays analytically.

21


Reproducing the Test

♦ Assure legibility– Poor-quality copies may affect student

performance– Uppercase and lowercase lettering (not all capitals)– Letter-quality original– Copies with crisp, dark print; no artifacts

22



♦ Print on one side of page– Easier to score if students record their answers

on the test itself– Printing on both sides of each page could cause

student-response errors

23



♦ Reproduce enough test copies– More test copies than the number of students – Allows extra copies for proctors or to replace

defective copies distributed inadvertently to students

24



♦ Maintain test security– Protect test materials from unauthorized access – Secure the test during preparation, duplication,

storage, administration, scoring– Preparing alternate forms of the test by

scrambling the order of items or options not recommended without evidence of true alternative-forms equivalence

25


Test Administration: Environment

♦ Face-to-face– Well-ventilated room– Limited distractions– Signage indicating testing in progress

♦ Online– Students have computer capabilities and Internet

access for allotted exam time

26


Test Administration: Distribution

♦ Distribute answer sheet before test booklet♦ Students keep booklet closed until told

to begin♦ Students check booklet for completeness♦ Proctor carefully for test security

27


Test Administration: Distribution

♦ Proctor reads general directions aloud while students read silently– Helps English language learners and students with

learning disabilities♦ Proctor answers any questions about the test

procedures♦ Students begin the test all at once

28


Test Administration: Questions During Testing♦ Student questions are distracting to others♦ Proctors may give inadvertent cues to correct answer♦ May decide not to allow questions– Students record questions on comment sheet to be

reviewed later

♦ If allowing questions:– Student raises hand, proctor goes to student’s seat and

answers quietly and briefly

29


Test Administration: Cheating

♦ Activity with intent to gain a higher test score than student is likely to earn by achievement – Acquiring test materials in advance– Sharing test materials with others– Using a substitute to take the test– Using unauthorized materials during the test – Exchanging information with others during the test,

including use of hand-held devices– Copying or retaining test materials to share later

30



♦ Controlling cheating—face-to-face testing– Teacher’s responsibility– Establish standardized testing procedures, such as• Personal belongings left outside of testing room• No sunglasses, earbuds, caps with brims or bills

– Most effective through proctoring• Ideally minimum of two proctors• Devote full attention to proctoring

31



♦ Responding to observed cheating– Be aware of and apply program policy on

academic integrity– If certain, collect test materials and remove

student from room quietly– If uncertain, observe student closely, make eye

contact, verify observation with other proctor

32



♦ Establish an appropriate penalty for cheating on a test– Example: assign highest possible failing score for that test

♦ Discovery of test security breach in advance of scheduled test date – Attempt to obtain verifiable evidence that some students

have seen the test– If so, prepare another test or another way of assessing

student learning

33


Test Administration:Collecting Test Materials♦ Advise students of procedure during general

instructions♦ Keep distractions to minimum♦ Verify that students turn in all test materials♦ If large group, designate one proctor to collect

materials

34


Test Administration:Collecting Test Materials♦ Enforce time limit– Don’t give students extra time unless they have

been granted that specific accommodation for a disability

– Announce time left near the end of the test period and encourage students to finish quickly

– At end of time limit, collect materials from remaining students

35

chapter 10 ppt eval & testing 4e formatted 01.10 kg edits

Documents