item writing handbook - ardms

ITEM WRITING

HANDBOOK

2019

Version 6.0

https://sonic.ardms.org/teams-and-projects/brc/Stock%20Images/GettyImages-538334630.jpeg

About Inteleos Inteleos is an internationally recognized non-profit global medical certification organization that

manages the American Registry for Diagnostic Medical Sonography® (ARMDS®), Alliance for

Physician Certification & Advancement™ (APCA™) and the Point-of-Care Ultrasound (POCUS)

Certification Academy™.

About ARDMS The American Registry for Diagnostic Medical Sonography®(ARDMS®), incorporated in 1975, is

an independent, nonprofit organization that administers examinations and awards credentials in

areas of ultrasound. Our credentials have been awarded to over 90,000 medical professionals

worldwide and are recognized as the international standard in sonography credentialing.

ARDMS administers examinations and awards credentials for the following specialties:

• Registered Diagnostic Medical Sonographer® (RDMS®)

• Registered Diagnostic Cardiac Sonographer® (RDCS®)

• Registered Vascular Technologist® (RVT®)

• Registered Musculoskeletal™ Sonographer (RMSKS™)

These credentialing programs are accredited by the American National Standards Institute (ANSI)

to the International Organization for Standardization (ISO) 17024 Standard for organizations that

certify personnel.

MISSION AND VISION ARDMS Vision: The American Registry for Diagnostic Medical Sonography (ARDMS) creates the

global standards of professional excellence in sonography.

ARDMS Mission: The American Registry for Diagnostic Medical Sonography (ARDMS) empowers

sonographers to provide exceptional patient care through rigorous assessments and continual

learning.

©2019 American Registry for Diagnostic Medical

Sonography, Inc.

All rights reserved.

2019 Version 6.0

Table of Contents Introduction .............................................................................................................................................. 1

Purpose of this Handbook .................................................................................................................. 1

How to Use this Handbook ................................................................................................................. 1

Background Information.......................................................................................................................... 2

The role of testing in professional credentialing ............................................................................... 2

Why sonography professionals take ARDMS examinations ............................................................. 2

Security of Test Content .......................................................................................................................... 3

Considerations when writing a test item ........................................................................................... 4

Coming up with an idea ................................................................................................................. 4

Thinking about thinking ................................................................................................................. 5

Cognitive demand versus item difficulty ....................................................................................... 7

Putting it all together ...................................................................................................................... 7

Characteristics of High-Quality Items ..................................................................................................... 8

Fairness in Testing ................................................................................................................................... 8

The Item Development Process .............................................................................................................. 9

Item Writing Guidelines ........................................................................................................................ 10

General submission guidelines ....................................................................................................... 10

Guidelines for writing the stem ....................................................................................................... 11

Guidelines for writing the answer options ...................................................................................... 14

Submitting Items ................................................................................................................................... 17

References ............................................................................................................................................ 19

Appendix A – Glossary of Terms .......................................................................................................... 20

Appendix B – Item Writing Checklist ................................................................................................... 22

Appendix C – Guidelines for Submitting Images and Digital Video Clips .......................................... 23

Appendix D – Taxonomy of Media-Enhanced Items ........................................................................... 24

Appendix E – Sample Stem Starters ................................................................................................... 34

Appendix F – Instructions for Submitting Items in the RIW Portal .................................................... 35

1 ARDMS Item Writing Handbook

Introduction

Purpose of this Handbook

Creating exam questions – or items – that are relevant and reflective of current practice can be a

difficult task. Item writing requires not only subject matter expertise but also training and practice

in writing test items to appropriately elicit knowledge from others. The purpose of the American

Registry for Diagnostic Medical Sonography (ARDMS) Handbook, Version 6.0 (the Handbook) is to

help both novice and experienced item writers in creating quality items.

Features of this Handbook include:

❖ A glossary (Appendix A) defining terminology used in the item writing process (terms

defined in the glossary are in italics the first time they are used in the Handbook)

❖ Presentation of item writing guidelines and best practices

❖ Examples of item writing Do’s and Don’ts

❖ Information on how to submit items and related images, digital video clips, and graphics

How to Use this Handbook We recommend that Subject Matter Experts (SMEs) new to item writing read through the

Handbook in its entirety. Following this, the Handbook should be retained as a handy reference

when questions regarding the item writing process arise. To aid in using the Handbook as a

reference, important information is highlighted in call-out boxes and illustrations.

One section of the Handbook that should be frequently referred to when writing test items is

Appendix B, which contains item writing guidelines that should be carefully followed. These

guidelines are based upon best practice and testing research. They are discussed in more detail

in the body of this Handbook.

ARDMS is the leading credentialing body for the sonography

community. Our success is due to our volunteers who share our

mission of quality care and patient safety. Since 1975, our exam

questions have been written by credentialed sonography

professionals active in the field. The knowledge and experience of

our dedicated volunteers are vital to making certain that our exams

are relevant and reflective of current clinical practices.

THANK YOU for your commitment to excellence in your profession.


Background Information

The role of testing in professional credentialing

For many professions, examinations are used as the means by which professional competence is

judged. The measure used to make this judgment is reflected in the following question:

“Does the candidate meet or exceed minimum competence?”

When writing test items, the item writer should focus on important job elements and create items

that allow all candidates to demonstrate the knowledge and skills required to practice the

sonography profession competently. Each item should examine:

✓ Knowledge all professionals should know to practice their field safely and competently

✓ Ability to apply what they know in their work environment AND/OR

✓ Various techniques or maneuvers professionals use in their work environment

The content of each ARDMS examination is intended

to address subject matter delineated in the content

outline of the examination. The content outline is

derived from a practice analysis of tasks performed

on the job. Thus, each examination obtains its

pertinence to current practice because each item

written for the examination is directly linked to a task

on the content outline.

Content outlines for ARDMS examinations are

available on the ARDMS website and in the Resource Center of the volunteer portal for item

writers to reference.

Why sonography professionals take ARDMS examinations

ARDMS credentials help patients, healthcare professionals, and healthcare employers identify

sonography professionals who meet or exceed the most rigorous sonographer criteria in the

industry. Additionally, ARDMS credentials help sonography professionals:

Practice analysis

Content outline

Exam items


✓ Demonstrate their commitment to the sonography profession

and its performance standards;

✓ Remain current on new trends in ultrasound and ultrasound

technology by requiring professionals to complete annual

training classes; and

✓ Establish their professional credentials by recognizing their

individual professional achievements.

Security of Test Content ARDMS is committed to preserving the integrity of items and credentialing procedures by ensuring

the security of examination materials. All item writers must read, understand, and adhere to a set

of terms which govern their participation in the ARDMS exam development process. These terms

were outlined in the Statement of Service (SOS) that all exam development volunteers must sign.

Your signature will be kept on file in the Inteleos Volunteer Engagement office. ARDMS requires

that you sign the SOS on an annual basis.

The Mechanics of Item Writing

Anatomy of a test item The primary item type used across all ARDMS examinations is the single-select multiple-choice

item (SSMC).

❖ All items submitted must be your own original work and not published

elsewhere.

❖ All items, and any associated media (e.g., images or videos) will become

the intellectual property of Inteleos and may not be used elsewhere.

❖ Items should be submitted through the method agreed upon with

Inteleos staff.

❖ Items should NOT be transmitted via e-mail.

❖ NO copies of items should be retained by item writers.

Security – Your Role as an Item Writer

https://sonic.ardms.org/teams-and-projects/brc/Stock%20Images/GettyImages-459013149.jpeg


A test item is like a puzzle piece.

SSMCs consist of three parts:

❖ The stem asks a question relating to a specific area of the content outline.

❖ The key is the correct answer to the question asked in the stem.

❖ Three distractors are plausible but incorrect answers to the question asked in the stem.

SSMCs may be media-enhanced, and Appendix C contains guidelines for submitting images.

ARDMS examinations may also use other media-enhanced item types in addition to SSMCs.

Appendix D provides a taxonomy of these other media-enhanced item types and item writing

guidelines specific to each of these are included in Appendix D.

Considerations when writing a test item

Think about a test item as a piece of a puzzle with the entire

examination providing a picture of each candidate’s competency.

Just like each puzzle piece contributes a small piece of

information about what the picture on the puzzle looks like, so

should each test item contribute a small piece of information

about the candidate’s knowledge and skills. A poorly written test item means that it is not

contributing as it should to providing a picture of candidate competence.

Coming up with an idea

Before starting item writing projects, the Exam Program Manager conducts a gap analysis of the

examination’s item bank to determine content areas in which more items are needed. Item

writers then may be directed to write items to these specific areas.

EXAMPLE OF A SINGLE-SELECT MULTIPLE-CHOICE ITEM

Axial resolution can be determined by evaluating which quality control test?

A. A flat interface

B. A round hole in a plastic block

C. Closely-spaced parallel membranes

D. Increasingly closely spaced fine wire

➔ STEM

➔ KEY

DISTRACTORS


Any writing exercise – including writing test items – begins with coming up with an idea. What do

you want to write about? Common errors seen in the workplace are good places to start when

trying to think of ideas for test items related to a particular task from the content outline. Test

items may also be developed in relation to a specific image or video.

Thinking about thinking

Performing a job task – or answering a test item – requires thinking. Sometimes this thinking only

involves recalling information. More frequently, however, it requires a higher-level thinking skill,

such as correlating data from different sources.

The type or level of thinking involved in successfully completing a task is termed

cognitive demand. The cognitive demand required by any particular job task or

test item is determined by the level of thinking typically needed to perform the

task or respond to the item. The actual level required is based upon the

individual. Individuals who routinely perform a job task may simply recall how they

have done the task in the past. For example, an experienced sonographer

reviewing an ultrasound may use pattern recognition skills to identify normal structures, whereas

a novice sonographer may require a higher-level thought process. Similarly, if a test taker has

been exposed to the specific content of a test item, through formal training or study, their

response may be based upon simple recall, even if the test item was intended to assess a higher-

level thinking skill.

Despite the subjectivity that may be involved in assigning cognitive demand to test items, test

items should be written to target a specific cognitive level. These cognitive levels can be thought

of as hierarchical, with the simplest level of thinking on the lowest level of the hierarchy and

higher levels on successive levels.

Inteleos classifies items to three cognitive levels:

Evaluation/Synthesis

Application/Analysis

Knowledge/Comprehension


Each individual test item should be written to an appropriate cognitive level. The verb contained

in the task statement from the content outline that the item is being written to address provides

an indication of the appropriate level. Items should be written at or below the cognitive level

indicated by the verb in the task statement.

Examinations should include items across all three levels to fully measure the breadth of

knowledge and skills sonography professionals use to perform their work tasks. An exam displays

the best fidelity to practice when it includes items from across the full range of cognitive levels.

Level 1: Knowledge/Comprehension

Description:

Recall a fact, information or a procedure

Sample action verbs

Define, identify, recognize, recall

Sample item

According to ALARA (As Low as Reasonably

Achievable) guidelines, which parameter

should be minimized?

A. Frequency

B. *Transmit power

C. Time gain compensation

D. Overall gain

Level 2: Application/Analysis

Description:

Engage mental processes beyond habitual

response using information or conceptual

knowledge; thinking which requires two or

more steps; use reasoning

Sample action verbs

Apply, use, measure, obtain, compare,

calculate

Sample item

What is the Nyquist limit if a pulsed-wave

Doppler instrument has a pulse repetition

frequency of 16 kHz?

A. 5 kHz

B. *8 kHz

C. 10 kHz

D. 32 kHz

Level 3: Evaluation/Synthesis

Description:

Make judgments based on criteria and

standards; put parts together to make a

whole; use complex reasoning

Sample action verbs

Analyze, assess, evaluate, synthesize,

correlate

Sample item

Which structure is indicated by the arrow on

this image?

A. Esophagus

B. Common carotid artery

C. Longus colli muscle

D. *Internal jugular vein

*indicates the correct response


Cognitive demand versus item difficulty

Cognitive demand and item difficulty are often used interchangeably. However, while these

concepts are related, they are not the same thing. Items that require a greater cognitive demand

frequently are more difficult, but low cognitive demand recall items may also be difficult –

particularly if they address content that is more trivial or obscure.

Putting it all together

Now you have all of the ingredients for writing test items:

Voilà! The genesis

of a test item

Next address the appropriate

cognitive level

First, focus on the

content to be tested

Cognitive demand

•The thinking skills used by a typical test taker to arrive at a correct response

Item difficulty

•A statistical property of a test item

Example:

•When was ultrasound first used for medical purposes?

•Recall question but may be difficult because it deals with a trivial fact


But Item writing is an art as well as a science. This Handbook is intended to provide ARDMS item

writers with information that will assist you in honing the art of item writing. Once a core item is

drafted, you can use the formatting guidelines presented to refine your item so that it assesses

what it is meant to assess.

Characteristics of High-Quality Items The critical question addressed in any ARDMS examination is, “Is the individual competent

enough to practice sonography?” Test items should be written so they elicit information needed to

answer this question for each candidate. How the item is written should not interfere with

gathering this information. To avoid interference, items should be free of grammatical errors and

consistent in format and style.

Test-wise candidates, even if they are unfamiliar with the test content, can sometimes use clues

in the item to determine the correct answer. Following item writing guidelines can assist in

preventing cueing of correct responses.

Appendix B provides an item development checklist that provides a quick reference guide to use

as you are writing your test items and to serve as a quality check before submitting them. The

item writing guidelines used to create the checklist will be discussed in more detail in the body of

this Handbook. But first, fairness as a testing concept requires special attention.

Fairness in Testing Fairness in a testing context means that test items do not

disadvantage any candidates by unnecessarily including

content that would be more familiar to certain groups (e.g.,

geographic, socioeconomic, ethnic) than to others.

The introduction of bias into an examination raises exam

validity concerns because it means that the exam is

assessing something other than what it is intended to

assess. Bias can also have legal and ethical implications. For

these reasons, added attention should be given to fairness

and the potential for bias during the item writing process.


Types of item review

The Item Development Process Writing a test item is an important step – but it is just that, a first step. Once you have completed

and submitted a test item, it undergoes a series of reviews. The goal of the review process is to

make certain that test content is developed according to best practice and that high-quality test

items are used to evaluate candidate competence.

Ed

ito

ria

l A review to determine whether the item is grammatically correct, meets style guidelines, and free of typographical and spelling errors

Inh

ou

se

SM

E A review by an Exam Clinical Specialist, an Inteleos staff member who holds ARDMS credentials; the review provides an initial check that the item is clinically relevant and reflective of current practice, and meets fairness criteria.

Co

mm

itte

e A review by the Assessment Committee specific to the examination; the review provides a check that the item is appropriate for the examination, the key is correctly identified, and the item is fair and unbiased.

Sta

tisti

ca

l Statistical properties (item difficulty and item discrimination) of all test items are evaluated at various times. Each item undergoes pretesting prior to being used as a scored item.

FAIRNESS GUIDELINES

✓ Unless required to address a job-specific or job-related content:

O References to gender, ethnicity, nationality, religious affiliation, etc. should

be avoided.

O Items should be gender-neutral.

O Controversial material that might invoke an emotional response should be

avoided.

✓ Any unnecessary referrals to words or content that might be less familiar to some

candidate groups (e.g., geographic, socioeconomic, ethnic) should be avoided.

✓ Opinion-based items should be avoided.


Item Writing Guidelines

General submission guidelines

Guideline 1: Submit items that are your original work.

Do not plagiarize. Copying exact language from textbooks and/or journal articles constitutes

plagiarism.

Guideline 2: Provide at least one reference for each item submitted.

All items written for Inteleos are required to have at least one reference. Items without a

reference will not be accepted.

❖ Use sources that are commonly accepted in the subject area.

❖ Recently-published articles and pending research are generally not appropriate

references, since they are likely to contain material that is not yet commonly accepted or

known.

When submitting an item reference, please include the title, author, publication date, and page(s)

of the textbook or article used to develop the item.

Guideline 3: Use correct grammar, punctuation and spelling; do not use personal

pronouns.

Grammatical errors often help candidates identify the correct answer without comprehending the

subject matter.

Guideline 4: Use vocabulary and language appropriate for the subject matter and

the candidate population.

Abbreviations and acronyms are a particular issue of concern. Spell out the term or phrase the

first time it is used and place the abbreviation or acronym after it in parentheses. Thereafter, the

abbreviation may be used.

Guideline 5: Write items to assess important material.

Items should be based on the appropriate examination content outline and should not be focused

on trivial information, experimental techniques or unresolved industry debates. Items should

address well-recognized standards.

Write items asking candidates to recall specific measurements or numeric values only if there is a

published and professionally accepted standard and if the ability to remember the measurement

or value is important and relevant to professional practice.

AVOID GIVING THESE CLUES TO THE RIGHT ANSWER


Poor Item:

What are the normal measurements of a uterus?

A. 6 cm long, 5 cm wide, and 4 cm thick

B. *8 cm long, 5 cm wide, and 4 cm thick

C. 8 cm long, 6 cm wide, and 2 cm thick

D. 10 cm long, 8 cm wide, and 5 cm thick

Better Item:

A uterine measurement of 8 cm long, 5 cm wide, and 4 cm thick, corresponds with which

patient profile?

A. *Nulliparous

B. Postpartum

C. Pre-pubertal

D. Postmenopausal

Guidelines for writing the stem

Guideline 6: Write the stem in the form of a question.

Inteleos uses only the “question and answer” format in its examinations. Do not submit items that

are formatted as incomplete or fill-in-the-blank sentences.

Appendix E provides a listing of potential ‘stem starters.’ Please note that these are not intended

to be a comprehensive list of questions that can be used within items. Rather, this Appendix

contains examples that can serve as inspiration for you as you begin to write.

Poor Item:

A small crater in a carotid artery plaque is called:

A. A thrombus

B. *An ulceration

C. A fibrous plaque

D. A calcific plaque

Better Item:

Which carotid plaque is associated with cerebral emboli?

A. Hyperechoic

B. *Ulcerated

C. Fibrous

D. Calcific




Guideline 7: Focus on a single concept, fact, or problem in each item.

A well-written stem allows the candidate to formulate a tentative answer even before reading the

potential responses. This is because only a single concept is being tested.

Avoid adding unnecessary information to the stem, as it can unintentionally confuse candidates

and increase reading time.

Poor Item:

Heart attacks are the number one cause of death in the United States, and coronary artery

disease is preventable to some degree. Which of the following conditions is a risk factor for

premature artery disease?

A. Anemia

B. Cancer

C. Gall bladder disease

D. *High cholesterol

Better Item:

Which risk factor is associated with premature coronary artery disease?

A. Increased arterial elasticity

B. Decreased arterial pressure

C. Lower total cholesterol

D. *Elevated low-density lipoproteins

Guideline 8: Avoid using a negatively-worded stem.

Positively-worded stems are preferred to prevent confusion and misreading of the item.

Negatively-worded stems may be used in the event that the item contains important content, and

a positively-worded stem with three plausible distractors cannot be identified. However,

negatively-worded stems should very much be the exception.

In the event a negatively-worded stem must be used, the negative word in the stem, such as NOT

or EXCEPT, should be capitalized.



Poor Item:

Which Doppler finding is not associated with severe aortic insufficiency (AI)?

A. Shortened AI pressure half-time (PHT)

B. Increased maximum aortic velocity

C. Diastolic flow reversal ascending aorta

D. *Decreased left ventricular outflow tract (LVOT)/aortic velocity ratio

Better Item:

Which echocardiographic finding is decreased in the presence of severe aortic insufficiency?

A. Left ventricular outflow tract (LVOT)/aortic velocity ratio

B. Proximal isovelocity surface area (PISA) radius

C. *Pressure half-time (PHT)

D. Jet area

Guideline 9: Reference any images included in the stem within the text of the stem.

Briefly refer to the existence of an image somewhere in the stem when creating items that include

an image in the stem. Doing so will indicate to the candidate to expect to see an image.

Sample reference language:

Which condition does the sonographic finding in this image most likely represent?

Which finding is most consistent with the palpable mass identified in this image?

Guidelines for submitting images and digital video clips and using them within items are found in

Appendix C.

Guideline 10: Place the majority of the wording in the stem.

The central idea or theme of the item should be placed in the stem and not in the answer options.

Placing as much information as possible in the stem avoids unnecessary repetition of material in

the responses.



Poor Item:

What happens when a patient has severe proximal arterial occlusive disease?

A. The Doppler waveforms will not be distinguished from venous waveforms.

B. The Doppler waveforms will demonstrate slight flow reversal in the diastolic component.

C. *The Doppler waveforms will appear markedly dampened, making interpretation difficult

for distal segments.

D. The Doppler waveforms will eventually return to a relatively normal waveform pattern once

the disease process has been stabilized.

Better Item:

Which Doppler waveforms are consistent with severe proximal arterial occlusive disease?

A. Triphasic

B. Diastolic flow reversal

C. *Monophasic

D. High resistance

Guidelines for writing the answer options

Guideline 11: Use plausible distractors.

Distractors should be credible and representative of misconceptions found in a clinical setting.

One useful strategy in generating distractors is to brainstorm all of the potential misconceptions

an entry-level professional may have. Also think of common errors made in professional practice.

Poor Item: Which condition is characterized by shortness of breath when the patient is in the recumbent position?

A. Asthma B. Fainting C. *Orthopnea D. Snoring

Better Item: Which condition is characterized by shortness of breath when the patient is in the recumbent position?

A. Apnea

B. Dyspnea

C. Hyperpnea

D. *Orthopnea




Guideline 12: Avoid use of the phrases “all of the above” or “none of the above” as

answer options.

“All of the above” and “None of the above” are convenient for item writers when you can’t think of

another plausible distractor. But they can be obvious giveaways for candidates, especially when

the phrases are used only occasionally.

“All of the above” facilitates guessing if a candidate thinks two answers are correct. “None of the

above” may mean that a candidate knew none of the answer options were correct, but it doesn’t

mean that he or she actually knew the correct answer.

Guideline 13: Keep all answer options homogeneous in format and content.

Answer options should be similar with regard to their length and the grammar used within them.

Poor Item:

What is a function of blood platelets?

A. Fighting disease

B. Transmitting oxygen to the body

C. *To aid in blood clotting

D. Carrying food to cells

Better Item:

Which component of blood aids in clotting?

A. Serum

B. *Platelets

C. Monocytes

D. Plasma

Poor Item:

Which condition is associated with anencephaly?

A. *Polyhydramnios

B. Abruptio placenta

C. Decreased maternal serum alpha-fetoprotein (MSAFP)

D. Two-vessel umbilical cord

Better Item:

Which condition is associated with anencephaly?

A. *Polyhydramnios

B. Abruptio placentae

C. Chorioangioma

D. Two-vessel cord

Grammar is not parallel. The three

distractors begin with an -ing verb

while the key does not.

Option length is not equal.



Items should also be homogenous in terms of content. Make sure that the item is testing a

single topic or concept and can be assigned to a single task on the content outline. Item stems

that ask candidates to select statements that are true or correct frequently do not contain

answer options which are homogeneous because these items do not contain a central idea in the

stem.

Poor Item:

Which statement is most accurate concerning a urinoma?

A. Irregular in shape

B. Orients superior and medial to the adjacent kidney

C. Caused by blunt trauma

D. *Located between Gerota fascia and the renal capsule

Better Item:

How does a urinoma typically present on ultrasound?

A. *A thin-walled anechoic collection

B. A mass filling the intrarenal space

C. Fluid collection superiorly and medially to the kidney

D. Encapsulation external to the kidney structures

Guideline 14: Avoid using specific determiners in the answer options.

Determiners such as “all”, “never”, “always”, or “often” encourage candidates to begin to

eliminate answer options without fully considering them. This improves the chances of guessing

the correct response.

There is not one single

theme to the item. Answer

options deal with shape,

location and etiology.


1. GRAMMATICAL INCONSISTENCIES IN THE ANWER OPTIONS (Guideline 13).

2. SPECIFIC DETERMINERS INCLUDING ALWAYS AND NEVER IN THE ANSWER OPTIONS

(Guideline 14).

3. PAIRS OR TRIPLETS OF ANSWER OPTIONS (Guideline 15).

4. ANSWER OPTIONS THAT CONTAIN WORDING IDENTICAL OR SIMILAR TO WORDS IN

THE STEM.

5. CONSPICUOUS CORRECT OPTION.

6. BLATANTLY RIDICULOUS ANSWER OPTIONS.

AVOID GIVING THESE CLUES TO THE RIGHT ANSWER


Poor Item:

Which description is most accurate regarding pancreatic pseudocysts?

A. Covered by a smooth internal wall

B. Are usually unilocular

C. *May mimic a solid mass

D. Have regular contours

Better Item:

Which description is most accurate regarding pancreatic pseudocysts?

A. Covered by a smooth internal wall

B. Contain anechoic contents

C. *May mimic a solid mass

D. Have regular contours

Guideline 15: Avoid a single pair or triplets in answer options.

Groupings of two or three of the answer options which are similar in wording or structure can

allow candidates to eliminate answer options and aid in guessing.

Poor Item:

What is the correct caliper placement when measuring an abdominal aortic aneurysm?

A. *Outer wall to outer wall

B. Patent lumen

C. Inner wall to outer wall

D. Inner wall to inner wall

Better Item:

What is the correct caliper placement when measuring an abdominal aortic aneurysm?

A. *Outer wall to outer wall

B. Inner wall to inner wall

C. Outer anterior lumen to inner posterior lumen

D. Inner anterior lumen to outer posterior lumen

Submitting Items Now that you understand the guidelines for writing items, you’re ready to start submitting items.

Writers are usually asked to submit 10 items per quarter.

All new writers will be assigned an item writing mentor. The writer will submit all items to the

mentor for approval. Mentors will review items and provide feedback.


This item contains two

pairs which is acceptable.



Instructions on how to use ARDMS’s item writing portal to submit items are found in Appendix F.


References Haladyna, T.M. (2004). Developing and validating multiple-choice test items (3rd ed.). New York:

Lawrence Erlbaum Associates, Inc.

Haladyna, T.M., Downing, S.M., & Rodriguez, M.C. (2002). A review of multiple-choice item-writing

guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309-334.

Parshall, C. G. & Brunner, B. (2017). Content development and review. In Davis-Becker, S. &

Buckendahl, C.G. (Eds.), Testing in the Professions: Credentialing Policies and Practice (pp. 85-

104). New York: Routledge.


Appendix A – Glossary of Terms Assessment

Committee A volunteer group responsible for developing an examination

Bias

A situation that occurs when an aspect of an examination means that

certain candidate groups are systematically advantaged or

disadvantaged

Candidate

An individual who is under consideration to receive a credential.

Passing an examination is typically a component of the process to

receive the credential.

Cognitive Demand An estimate of the likely mental behavior required of a typical test taker

in responding correctly to a test item (Haladyna & Rodriguez, 2013)

Content Outline

A document developed through a practice analysis that summarizes the

tasks that comprise competent practice and the related knowledge,

skills, and abilities that candidates are expected to master

Distractor An incorrect (but plausible) answer option in multiple-choice test items

Exam Clinical

Specialist

An Inteleos staff member who has one or more ARDMS credentials in

sonography and is responsible for providing inhouse subject matter

expertise in the development of ARDMS examinations

Exam Program

Manager

An Inteleos staff member who is responsible for overseeing the test

development process for one or more ARDMS examinations

Gap Analysis

An activity undertaken to provide a quantitative review of an

examination item bank to determine the content breakdown of items

within the bank in order to identify content areas that require more

items to be written. A gap analysis is performed to direct item writing

efforts.

Hotspot A type of image-containing item on a computer-based test. Test takers

respond to the item by clicking directly on the image.

Item

The basic unit of measurement on an examination, commonly referred

to as a question. They are referred to as items rather than questions

because they may be phrased as questions (in the case of multiple-

choice items) or tasks to perform (in the case of hotspots or other item

types).

Item Bank

A collection of items written for an examination. The collection is usually

maintained through the use of an electronic content management

system that stores not only the item itself, but also information such as

the area of the content outline the item is classified to, revision and

usage history, and statistical data on item performance.

Item Difficulty

The proportion of test takers who answered an item correctly. It is

usually measured by a statistic called the p value, which is determined

by dividing the number of test takers responding correctly by the total

number of test takers administered the item.


Item Discrimination

A measure of how well an item is able to distinguish between higher

and lower ability test takers. It indicates the extent to which success on

a particular test item corresponds to success on the overall test. It is

frequently measured by a statistic called the point-biserial correlation

(ptbis).

Item Types The variety of formats in which test items may be written

Item Writer An expert in a particular field who composes an item for an ARDMS

examination

Key The correct response to an item

Practice Analysis

A research process for determining critical tasks performed within a job

role or practice. Also called Job Task Analysis (JTA), a practice analysis

is performed to determine the content outline for credentialing

examinations.

Pretesting The process of field testing a test item before it is used as an item that

counts toward test takers’ scores on an examination

Response Options All of the answer choices within a test item. These choices include the

distractors and the key.

Statement of Service

(SOS)

An agreement between Inteleos and an ARDMS volunteer that outlines

the terms and conditions by which voluntary work is performed on

behalf of Inteleos

Stem

The part of a test item that describes the task the test taker is to

perform to respond to the item. A stem can take a variety of formats,

depending on the item type, including a question or directions.

Subject Matter

Expert (SME)

An individual with experience and expertise in a particular field.

Individuals who hold ARDMS credentials and who provide input into the

test development process for ARDMS examinations are referred to

SMEs.

Test-wise

Ability of a test taker who is skilled in detecting inadvertent clues to the

right answer to an item, even when the test taker would otherwise not

know the answer


Appendix B – Item Writing Checklist Does the submitted item:

☐ 1. Contain your original work?

☐ 2. Include a reference?

☐ 3. Use correct grammar, punctuation and spelling?

☐ 4. Use appropriate language and vocabulary?

☐ 5. Assess important material?

☐ 6. Meet fairness guidelines?

☐ 7. Have a stem that is in the question format?

☐ 8. Deal with a single concept, fact or problem?

☐ 9. Use positive wording?

☐ 10. Contain a reference to the image if the stem contains an image?

☐ 11. Have a majority of the wording in the stem?

☐ 12. Contain plausible distractors?

☐ 13. Not include “all of the above” or “none of the above” in the answer options?

☐ 14. Use answer options that are homogeneous in terms of format and content?

☐ 15. Not use specific determiners in the answer options?

☐ 16. Not contain answer options in a single pair or a triplet?

Refer to the body of this Handbook to view a detailed explanation of each guideline.


Appendix C – Guidelines for Submitting

Images and Digital Video Clips All submitted material (items, images, digital video clips, etc.) become the property of ARDMS.

Please do not submit anything you have published, intend to publish or that is or will be used in

any other venue.

1. Remove all identifying information (patient name, hospital name, etc.) to comply with

relevant laws and regulations, including HIPAA. Please obtain appropriate permissions and

authorizations.

2. Please include only clips and frames that contain information relevant to the test item.

3. All images must be digital. No hardcopy images will be accepted, nor will images obtained

using cellphones.

a. JPG (compressed) file format is acceptable – use highest PPI (resolution) available.

Other acceptable file formats for images include PNG, BMP, and DICOM.

b. The image should be at least 1024 x 768 pixels (width x height).

c. Acceptable video formats include AVI, MP4, and WMV.

4. If the image is color, use RGB (Red, Green, Blue). If the image is black and white, use

grayscale.

5. To name the file, use the item ID (provided by the item authoring software) followed by the

name of the diagnosis. An exhibit classification is required. You must save the item to

generate the ID, for example, “52341 ventricular septal defect.”


Appendix D – Taxonomy of Media-Enhanced

Items

Classification Name

A Single-select multiple choice with single still image

B Single-select multiple choice with single video

C Single-select multiple choice with side-by-side images

D Single-select multiple choice with multiple vertically-presented

media

E Single-select multiple choice with still images as answer

options

F Single-select multiple choice with videos as answer options

G Non-discrete single-select hotspot


Item Type A: Single-select multiple choice (SSMC) with single still image

Description:

This is a four-option multiple choice item with a still image in the stem. Image may be

sonographic or another type of image, such as a chart or graph. All answer options are text-based.

Item template:

Sample stem: Which structure is demonstrated in this image?

A. Answer option A

B. Answer option B

C. Answer option C

D. Answer option D

Potential tasks to be assessed by the item

1. Identify

2. Recognize

3. Assess/evaluate

Technical specifications

1. The maximum recommended size of the image is 440x330 pixels to prevent candidates from

having to scroll to navigate through the item.

2. Arrow(s) may be used to mark a particular structure or location on the image. Arrows should be

white. Note: Arrows to illustrate calipers for fetal measurements on the Fetal Echocardiography

examination may be yellow.

3. Labeling on the image should not cue the correct response.

Item writing guidelines specific to the item type

1. The image should be referred to in the item stem.

2. The image must be the focal point of the item and necessary for responding to the item.

3. Any arrow(s) should be referred to in the stem. For example, “Which structure is indicated by

the arrow on this image?”

Static Image


Item Type B: Single-select multiple choice (SSMC) with single video

Description:

This is a four-option multiple choice item with a video in the stem. All answer options are

text-based.

Item template:

Sample stem: Which abnormality is indicated by this video?

A. Answer option A

B. Answer option B

C. Answer option C

D. Answer option D


1. Identify

2. Recognize

3. Assess/evaluate


1. The maximum recommended size of the video is 440X330 pixels to prevent candidates

from having to scroll to navigate through the item.

2. The maximum recommended length of the video is 3 seconds.

3. Labeling on the video should not cue the correct response.


1. The video should be referred to in the item stem.

2. The video must be the focal point of the item and necessary for responding to the item.

3. Any arrow(s) should be referred to in the stem. For example, “Which structure is

indicated by the arrow on this video?”

Video


Item Type C: Single-select multiple choice (SSMC) with side-by-side images

Description:

This is a four-option multiple choice item with a set of two side-by-side images in the stem.

All answer options are text-based.

Item template:

Sample stem: Which finding is indicated by the arrows shown on the images?

A. Answer option A

B. Answer option B

C. Answer option C

D. Answer option D


1. Identify

2. Recognize

3. Compare

4. Assess/evaluate

5. Integrate/synthesize


1. The maximum size of each image is 440X330 pixels.

2. The images should be separated by a single space.

3. Arrow(s) may be used to mark a particular structure or location on the image(s). Arrows

should be white in color. Note: Arrows to illustrate calipers for fetal measurement on the

Fetal Echocardiography examination may be yellow.

4. Labeling on the image(s) or video(s) should not cue the correct response.


1. The images should be referred to in the item stem.

2. The images should be the focal point of the item and necessary for responding to the

item.

Image Image


3. Each image should contribute to the information needed by the candidate to respond to

the item. Their use should not be redundant.


Item Type D: Single-select multiple choice (SSMC) with multiple vertically-presented media

Description:

This is a four-option multiple choice item. The stem includes vertically-presented media that

might include still images, videos, charts, or other graphics. Note that Item Type C is

preferred for stems with two still images to prevent scrolling.

Item template:

Sample stem: Based upon the image and video, which diagnosis is the most likely?

A. Answer option A

B. Answer option B

C. Answer option C

D. Answer option D

Potential tasks to be assessed by items

1. Assess/evaluate

2. Integrate/synthesize


1. The maximum recommended size of each media is 440X330 pixels.

2. If a video is used, the maximum recommended length of the video is 3 seconds.

3. An item should include no more than one video.

4. Arrow(s) may be used to mark a particular structure or location on the image(s) or

video(s). Arrow(s) should be white in color. Note: Arrows to illustrate calipers for fetal

measurements on the Fetal Echocardiography examination may be yellow.

Image/Video/Graphic

Image/Video/Graphic


5. Labeling on the image(s) or video(s) should not cue the correct response.

6. Scrolling should be kept to a minimum.


1. The image(s) and/or video should be referred to in the item stem.

2. The image(s) and/or video must be the focal point of the item and necessary for

responding to the item.

3. Each image, graphic, and/or video should contribute to the information needed by the

candidate to respond to the item. Their use should not be redundant.


Item Type E: Single-select multiple choice (SSMC) with still images as answer options

Description:

This is a four-option multiple choice item. Each answer option is a still image.

Item template:

Sample stem: Which image demonstrates a volvulus?

A.

B.

C.

D.


1. Identify

2. Recognize

3. Assess/evaluate


1. The maximum recommended size of each image is 440X330 pixels.


video(s). Arrows should be white in color. Note: Arrows to illustrate calipers for fetal


3. Labeling on the images should not cue the correct response.



1. Use image options in place of text-based answer options to provide a more authentic

assessment.

2. Use of this item type may be appropriate when a text-based answer option would include

a ‘normal’ or ‘no abnormality’ as the key or a plausible distractor.

Image

Image

Image

Image


Item Type F: Single-select multiple choice (SSMC) with videos as answer options

Description:

This is a four-option multiple choice item. Each answer option is a video.

Item template:

Sample stem: Which video clip demonstrates the inferior vena cava?

A.

B.

C.

D.


1. Identify

2. Recognize

3. Assess/evaluate


1. The maximum recommended length of the video is 3 seconds.


video(s). Arrows should be white in color. Note: Arrows to illustrate calipers for fetal


3. Labeling on the images should not cue the correct response.



1. Use video options in place of text-based answer options to provide a more authentic

assessment.

2. Use of this item type may be appropriate when a text-based answer option would include

a ‘normal’ or ‘no abnormality’ as the key or a plausible distractor.

Video

Video

Video

Video


Item Type G: Non-discrete single-select hotspot

Description:

This is an item which includes a single image in the stem. Candidates respond to the item by

clicking directly on the image. The item is referred to as a non-discrete hotspot because the

regions of the image available for selection are not visualized.

The stem includes standardized instructional language providing directions on how candidates

should respond to the item.

Item template:

Sample stem:

Identify the gallbladder.

Using your mouse, place the cursor on the appropriate region of the image and then left-click the

mouse button to indicate your selection.


1. Identify

2. Recognize

3. Interpret


1. The maximum recommended size of the image is 440X330 pixels.

2. Labelling on the images should not cue the correct response.

3. Candidates should not be required to scroll.


1. Use hotspot images in place of text-based answer options to provide a more authentic

assessment and direct measurement of the content.

Still Image


Appendix E – Sample Stem Starters Use the stem starters when you’re at a loss for words or don’t know how to verbalize the point

you’d like to make. Note that the fill-in-the-blanks indicate places in which words should be

inserted to complete the item. Fill-in-the-blanks should not be used within a completed item.

Which sonographic feature best

characterizes ______?

Which finding is (characteristic of, most

consistent, most often presents) with

______?

What is most likely represented by the finding

demonstrated in this image?

Which method is most (consistent, accurate,

appropriate) with ______?

Which (finding, abnormality) is most

consistent with _____?

Which diagnosis is most consistent with the

findings demonstrated on this image of a

______ in a patient with a history of ______?

Which sonographic finding is (most

commonly/most strongly) associated with

______?

Which description is associated with ______?

Which scanning technique would

(increase/reduce) the ______?

Which vessel would be (enlarged, occluded,

obliterated) in the presence of ______?

Which post-transplant complication

characterized by ______?

Which sonographic (view, plane, window)

best demonstrates ______?

Which finding requires further follow-up?

Which patient position is best suited for

evaluation of ______?

Which sonographic view is best for assessing

______?

Which technique is optimal for evaluation of

______?

Which adjustment is most likely to improve

visualization of ______?

Which sonographic finding is most likely to be

demonstrated in a ______ patient referred

for ______, with a history of ______?

Which associated abnormality presents with

______?

Which (region/area) would require focused

evaluation of ______?

What is the most likely diagnosis based on

this image?

Which measurement of the ______ should

prompt further evaluation of ______?

What is the characteristic feature of a

(normal/abnormal)______?

What is the most accurate sonographic

measurement for ______?

What is the optimum time frame to

obtain/perform ______?

What is a potential (advantage/

disadvantage) of ______?

Which condition may be (misdiagnosed/

mistaken for ______)?

Which symptom would be considered an

indication for ______?

Which symptoms are commonly associated

with _____?

What is the characteristic (shape,

appearance) of a ________?

Which region within the Doppler waveform

represents _________?

What is a major risk factor for ________in

patients with _________?

Which common sonographic finding is

associated with _________?

Visualization of which anatomy confirms

______?


Appendix F – Instructions for Submitting

Items in the Remote Item Writing (RIW) Portal PRO! Is the Remoted Item Writing Portal that is used for volunteers to submit exam items. It is a

website that can be accessed from any internet-connected computer.

• Web address (URL): https://ardms-prod-riw.epic-exam.com/ (Bookmark this site for easy

access.)

• Username is your ARDMS #

• Your password will be assigned and emailed to you after you complete item writing training

and sign the Statement of Service agreement.

Item Writing

Username = ARDMS or

APCA ID#

https://ardms-prod-riw.epic-exam.com/


Navigating the Item Writing Portal

The front page features an overview of all projects assigned.

Navigating the Item Writing Portal

Assigned projects are shown. If you are writing for more than one project or exam, they will be shown underneath the Project Name header.


Creating an Item

To access the item writing screen, select “Create Item”.

Creating an Item

Select the “Content” icon to select which area of the content outline this item will address.


Creating an ItemIn the “Select Content” screen, use the scroll bar on the right to view all available content areas from which to choose the topic for this new item. Once a selection has been made, click the “Select” button. NOTE: Depending upon the project needs, all content areas may not display. Additionally, as content areas become full, they will disappear from this list.

Creating an ItemEnter the stem remembering to phrase it in the form of a question with a question mark (?) at the end.


Inserting an Image from Item Writer’s LibraryTo upload an image from the writer’s library, select the image button.

Inserting an Image from Item Writer’s Library

Select “Browse” and find the image/video.



The computer’s picture folder will open and a selection can be made.


Once the image is selected, the “General” tab will display along with the image. Toggle the description categories and “tag” the image by selecting no more than four tags.



Note that exhibit tagging is required.


Select the “Details” tab and complete the information. If your item text refers to an arrow on the image, please let us know in the comments where the arrow needs to be placed by our in-house image editor. You may place other image-editing notes here.



Select the “Reference” tab and choose the appropriate reference. See below if the reference is not listed. Use the scroll bar to view existing resources.


After selecting the reference, add the beginning page number for your exact topic. After you have completed this, click “Save Exhibit.” It will return to the create item screen and your image will be shown.



If the reference is not listed, select “Reference Note.” Add the reference and include the title, editor/author, edition, publisher, publication date, and page number(s). Then click “Save Exhibit.”


A note about images…..

ALL PATIENT AND FACILTY INFORMATION SHOULD BE REMOVED BEFORE

UPLOADING AN IMAGE.

Please use the best image quality that you can. You may include more than one

image per question.

We ask that you use non-copywritten images that you have obtained from scanning

patients. (Not from a previously published work.) If you do not scan patients, you

may use images from your colleagues, provided you have their permission.

Individual file sizes should be kept to no more than 10 MB per image and 20 MB

per video.


Inserting an Image from Inteleos Image Bank

To search and select an existing image, select the folder option.


This window will open for you to choose from the “Exhibit Classifications” to select the image required. Use the scroll bar to view other classifications.



Select the appropriate image. Edit the criteria if needed, then click “Select Exhibit.”

Adding the Response Options to an Item

Next, add the responseoptions, keeping in mindto include the correct answer, or key. Then, indicate the key by clicking the small, round radio button to the left of the key.

NOTE: You may also add images to the answer choices.


Referencing the Item

The final part ofsubmitting an item isadding the reference. Having valid and currentreferences for our itemsis one way we maintainthe credibility of ourexams.

Reference the over-arching concept of the item. For example, if your item addresses intracranial hemorrhages, include a reference that addresses them.

Referencing the Item

Use this scroll bar to view the existing resources in the library. Once youhave found your desiredreference, click on it.

You will then need to add the beginning page number for your exact topic for your item here.

Add the ending page number for your exact topic for your item here.

When completed, “Add Reference.”


Referencing the ItemIf you do not see yourreference and edition inthe library, please addthe reference in theReference Notes section.

Add your referencewhere the arrow indicates. Include title ofwork, editor/author,edition, publisher,publication date, andpage(s). Then click “Add Reference Note.”

Submit your reference tothe Project Manager and it will be added to the reference library.

Save/Submit/Send to Mentor

Click Save to save your item in system for further editing (Status will be“Incomplete”)

If you have been paired with a Mentor, click Send to Mentor. The item will belocked for editing until the Mentor adds his/her feedback (within 5-7 days).

Click Submit – ONLY when all of the required fields have been completed. Once yousubmit an item, you cannot go back and edit it. (Status will be “Pending”)

The Save/Submit/Send to Mentor buttons are at the bottom ofthe page.


FAQs

What about the other fields in PRO!—like Rationale andKeywords?We do not require you to use some of the additional features ofPRO!. You may skip them.

Where can I get some additional guidance on writing items?Ask your Mentor, email your project manager or ItemDevelopment Coordinator for additional help and guidance on writing items. The project manager information is shown on the home screen and an email can be initiated from there.

You can also view three short videos on our YouTube channel.https://www.youtube.com/playlist?list=PLydRf_PGVd9K66qkUxAeMZS1Jf7pLf1d0

I am credentialed in multiple specialties. Can I be a part ofmultiple writing projects?Yes! View all your projects on the Projects Overview page, the firstscreen you see when you log in.


Do your exam items face the FACTs?

Fair

Test items should not advantage or disadvantage any candidate group. Make

sure that:

• The language used is able to be understood by all candidates.

• The content is not derogatory or prejudicial to any candidate groups.

Appropriate

Test items should be appropriate for the test purpose. Make sure that:

• The content of the item is within the scope of practice.

• The item can be classified to a specific task on the content outline.

Current

Test items should be reflective of current practice. Make sure that:

• The content of the item reflects current practice guidelines.

• The item is referenced to an up-to-date well-recognized source.

Technically accurate

Test items should be correct and valid. Make sure that:

• The content of the item is accurate for all appropriate practice situations.

• The key is correctly identified and all other answer options are plausible,

but incorrect.

©2019 American Registry for Diagnostic Medical Sonography, Inc.

All rights reserved.

2019 Version 6.1

WHO TO CONTACT

Questions concerning this Handbook, item writing deadlines, item writing and

mentoring assignments, the PRO! Remote Item Writing Portal (RIW), or the use of

references and/or media should be directed to:

[email protected]

Questions concerning potential conflicts of interest, or about other volunteer

opportunities should be directed to:

[email protected]

item writing handbook - ardms

Documents