ExamSoft Assessment Conference

#ASSESS2015

COLLABORATIVE QUESTION BANKS:

Sharing Content in ExamSoft

I N D I A N A U N I V E R S I T Y 3

Today’s Participants

Rachel Mahan (Organizer) – IUSMJennifer Schwartz, MD – IUSMWendy Senour – MU-COMJessie Belnap – PSTCC

Representatives from three different institutions

I N D I A N A U N I V E R S I T Y 4

Discussion Objectives

After this discussion, you will be equipped to:• Justify the use of ExamSoft to create

collaborative content• Create and maintain collaborative

question banks• Manage and address faculty concerns

You’ll leave knowing how to collaborate using ExamSoft.

I N D I A N A U N I V E R S I T Y 5

Sharing content in ExamSoft allows you to…

• Track question stats across multiple exams and per exam

• Facilitate creation of team-taught/multiple-instructor exams

• Peer review questions• Create an approved question

bank

I N D I A N A U N I V E R S I T Y 6

Aggregate or Exam-Level Stats

I N D I A N A U N I V E R S I T Y 7

Common Faculty Concerns

• Intellectual property rights• Other people viewing their work• Different skill- and comfort-

levels with technology• Exam security

I N D I A N A U N I V E R S I T Y 8

Indiana University School of Medicine

• 2nd largest medical school in US

• 1400+ M.D. and M.D./Ph.D. students in 2014-2015

• >50% of IN doctors trained at IUSM

• 9 campuses across the state

Sources: IUSM Fact Sheet for 2014-2015 http://medicine.iu.edu/files/9514/1522/2357/Fact_Sheet_2014-15.pdf

http://medicine.iu.edu/ume/

9I N D I A N A U N I V E R S I T Y

Dr. A’s Question

Bank Folder

Cardiology Review Folder – Originals

Neurology Review Folder – Originals

Oncology Review Folder – Originals

Statewide Question Bank

Dr. A’s folder 1

Dr. A’s folder 2

Dr. B’s folder 1 Dr. C’s

folder 1

Cardiology Review Folder – Copies

Neurology Review Folder – Copies

Oncology Review Folder – Copies

Dr. B’s folder 2

Dr. B’s folder 3

Dr. B’s Question

Bank Folder

Dr. C’s Question

Bank Folder

43

6

8

9

MoveDuplicate

Move

Duplicate

Move

10I N D I A N A U N I V E R S I T Y

Dr. A’s Question

Bank Folder

Dr. A’s folder 1

Dr. A’s folder 2

Dr. B’s folder 1 Dr. C’s

folder 1Dr. B’s

folder 2

Dr. B’s folder 3

Dr. B’s Question

Bank Folder

Dr. C’s Question

Bank Folder

Move 3 4

Duplicate

Cardiology Review Folder – Originals

Neurology Review Folder – Originals

Oncology Review Folder – Originals

Move 6

Cardiology Review Folder – Copies

Neurology Review Folder – Copies

Oncology Review Folder – Copies

Duplicate 8

Move 9 Statewide Question Bank

Jessie Belnap, MSN, RNLead Instructor, 1st Level

COGNITIVE LEVEL: NURS1150

NURS1160

NURS1170

NURS2150

NURS2160

NURS2170

Knowledge/Comprehension

40%-50%

25%-35%

25%-35%

10%-15%

5%-10% 0

Application 45%-55%

50%-65%

50%-65%

50%-70%

55%-75%

50%-70%

Analysis 0-5% 5%-10% 5%-10% 10%-20%

10%-20%

15%-25%

Synthesis 0 0 0 0-5% 0-5% 5-10%TEST ITEM FORMAT Alternative format 2%-4% 5%-6% 5%-6% 7%-8% 9%-10% 10%-

15%Dosage Question 5 5 5 5 5Cultural 1-2 1-2 1-2 1-3 1-3 3-5Delegation 3-5 3-5 5-10

About MU-COM

• First osteopathic medical school in Indiana (est. August of 2013)

• Initially iPad exclusive but have recently branched into utilizing computer labs for essay and rubric assessments

• Multiple choice examinations in biomedical courses and lecture component of clinical courses in first two years of undergraduate medical education

• A core group of approximately 30 faculty have access to ExamSoft.

• Over 130 guest lecturers have contributed questions, but do not have access to ExamSoft.

• Anatomy and Clinical Courses Closed group of faculty review questions and have input on

all questions during review and exam administration.

• Scientific Foundations of Medicine (SFoM) Open group of faculty peer-review together in conference

room directly in ExamSoft. Review can take 8-10 hours for one exam.

• Systems Based Courses (e.g. Cardiovascular, Renal, and Pulmonary) Course Coordinator (CC) reviews all questions. Internal

faculty review only review questions they wrote.

• Non-content review conducted by Coordinator of Examinations.

Pre-exam Peer Review

• Anatomy and Clinical Courses Course Coordinator (CC) and internal faculty meet as a group

to review exam and question stats and make group decision on modifications.

• SFoM and Systems Based Courses CC reviews exam statistics. Internal faculty review questions directly in ExamSoft.

• External lecturers are involved in varying degrees.• Post exam changes to questions are always documented

using standardized codes in Internal Comments.• CC’s present exam statistics during Post-course Wrap-up

to faculty.

Post-exam Review