test taking skills - s3.amazonaws.com introductory comments ... (> 2.5 mm) with a right...

Test-Taking Skills Donald J. Sefcik, DO, MS, MBA, FACOFP

9/1/2015

1

Test-Taking Skills

Written

Donald Sefcik, DO, MBA, FACOFP

Disclosure

Why are Standardized Tests so hard?

Faculty-generated tests Standardized tests

9/1/2015

2

Introductory Comments

• Do you know someone that:

– Studied LESS and scored HIGHER than you?

What’s the impact of study hours COMLEX Level 1 exam

Hours Studied <100 101-150 151-200 201-250 251-300 301-350 351-400 >400

Mean Score 552.75 533.50 536.17 530.43 545.95 549.85 523.00 536.25

538.2 538.8

MSUCOM Research; AACOM 2012

Hours <100 101-150 151-200 201-250 251-300 301-350 351-400 >400

Maximum 599 539 581 657 778 717 679 692

Minimum 480 528 428 428 415 382 397 367

Range 119 11 153 229 338 311 330 325

What the C1 mean doesn’t tell us

128 326

MSUCOM Research; AACOM 2012

9/1/2015

3

Higher Scorers

_

X

???

Overall View

“Have to…” “Get to…”

Highest Scorers

Lowest Scorers

Study Techniques / Resources

Efficient Effective

Highest Scorers

Lowest Scorers

9/1/2015

4

Majority of Effort

Preparation Exam Day

Highest Scorers

Lowest Scorers

Skill Development

Test-

Wiseness

Test-

Taking

Highest Scorers

Lowest Scorers

Maximizing your SCORE

Test-Wiseness Test-Taking Skills

• Reading Accuracy

• Reading Fluency

• Guessing

• Changing Answers

• Test Blueprint

• Long-term Memory

• Transfer

• Self-Regulation

TEST

ANXIETY

9/1/2015

5


EXAM DAY


• Reading Fluency

• Guessing


Example

Infarction of the right ventricle occurs when the proximal

segment of the right coronary artery is occluded. Which of the

following is the most sensitive electrocardiographic (EKG)

finding of a right ventricular infarction?

A. ST segment elevation in I, aVL, V5 and V6

B. ST segment depression in V1, V2 and V3

C. ST segment elevation in V1, V2 and V3 with

ST segment depression in aVF

D. ST segment elevation in V4R with an upright T wave

E. ST segment elevation in V1 (> 2.5 mm) with a

right bundle-branch block

Example












STEM

OPTIONS

9/1/2015

6

Example












Interrogatory

Distractors

Impact of Stem (Vignette)

Examinees with

Higher Scores

Examinees with

Lower Scores

No Vignette 99% 90%

Short Vignette 98% 82%

Long Vignette 98% 66%

http://www.nbme.org/pdf/itemwriting_2003/2003iwgwhole.pdf

Examinees with

Higher Scores

Examinees with

Lower Scores

No Vignette 99% 90%



A patient with hyponatremia and highly concentrated urine is noted

on CXR to have a centrally-located, suspicious-appearing, solitary

pulmonary nodule. Which of the following is the most likely explanation?

A. small cell carcinoma

B. squamous cell carcinoma

C. adenocarcinoma

D. large cell carcinoma

E. mesothelioma

EXAMPLE

9/1/2015

7

Pulmonary MassesApical

TB

Peripheral

Large Cell Carcinoma

AdenocarcinomaCentral

Small Cell Carcinoma

Squamous Cell Carcinoma

Pleural-Based

Mesothelioma

A patient with hyponatremia and highly concentrated urine is noted

on CXR to have a centrally-located, suspicious-appearing, solitary

pulmonary nodule. Which of the following is the most likely explanation?

A. small cell carcinoma (central)

B. squamous cell carcinoma (central)

C. adenocarcinoma (peripheral)

D. large cell carcinoma (peripheral)

E. mesothelioma (pleural-based)

EXAMPLE Reading Accuracy

Exam Day

Performance

• Selecting Answers

Know for Sure (Record & Confirm Choice)

Familiar but Not Sure (Maximize Probability)

No Idea (Test-Taking Skills; Guessing Better)


• Reducing Test Anxiety (Exam Site Issues)

9/1/2015

8

• Focus on the Stem– Find the BEST answer

– If you know if after reading the stem, “go for it”

• Read ALL options before recording answer– Avoid partly correct responses

• Go with instinctive (GUT) feeling– You picked it for a reason

• Once decide incorrect– Do not consider it again

• Avoid “reading into” the question– The stem sets the task

During the Examination

Question

What probable etiologies are suggested when drugges foss

in extremis?

A. Sloorgs thanceed with the droxemia

B. Potcoms fotted secondary to an oral deficiency

C. The dagts ptopted and the frodds roanzed

D. The dhiths buazzed secondary to gastrointestinal crooleets

E. Taggoks were removed from the juklets

Question

Patients suffering from strugleum frequently undertrosl the gruifreit

because:

A. all patients with strugleum have krodncurz.

B. strugleum always presents with qugruestful.

C. there are never cases of strugleum without feskabolose.

D. patients with strugleum commonly have gizzlehorst.

E. no fuegrools results in strugleum.

9/1/2015

9

Question

Along the eastern coast of Maine, the cases of huestoolduds are

most often secondary to:

A. poor nutrition.

B. the moisture in the frocklesters.

C. the abundance of greezles.

D. a relative abundance of kleasternezers.

E. an oversupply of fhoulsteens that occur directly as a result of

kjuoost, djurts and bhespernets maldoorneys.

Question

The drug of choice for patients experiencing

zhiglerastasis secondary to kweatlostosis is:

A. dragrootmycine.

B. brusterocilline.

C. noopkweetosporine.

D. zhiglerpurocine.

E. pneustodregcycline.

9/1/2015

10

When GUESSING is the ONLY alternative

General AlternativeIf four of the five options are very specific and one is

very general…

Often the more general option is correct response



Longest AlternativeOften the exam writer wants to be sure you have

enough information to recognize the correct

response.


Chase 1964



Longest Alternative

Question

Along the eastern coast of Maine, the cases of huestoolduds are

most often secondary to:

A. poor nutrition.

B. the moisture in the frocklesters.

C. the abundance of greezles.

D. a relative abundance of kleasternezers.

E. an oversupply of fhoulsteens that occur directly as a result of

kjuoost, djurts and bhespernets maldoorneys.

9/1/2015

11


Grammar Agreement The stem and responses should be grammatically

correct…

Singular stem should not be completed by plural

option


Broad & Whitney 1972



Grammar Agreement

Question

What probable etiologies are suggested when drugges foss

in extremis?

A. Sloorgs thanceed with the droxemia

B. Potcoms fotted secondary to an oral deficiency

C. The dagts ptopted and the frodds roanzed

D. The dhiths buazzed secondary to gastrointestinal crooleets

E. Taggoks were removed from the juklets


Specific Determiners The implication of absoluteness (always, never,

must, none, only…) is often incorrect.

Words that permit exception (seldom, usually, often,

perhaps, etc.) are more commonly associated with

the correct response.


Milman 1965

9/1/2015

12



Specific Determiners

Question

Patients suffering from strugleum frequently undertrosl the gruifreit

because:

A. all patients with strugleum have krodncurz.

B. strugleum always presents with qugruestful.

C. there are never cases of strugleum without feskabolose.

D. patients with strugleum commonly have gizzlehorst.

E. no fuegrools results in strugleum.


Clang Association Similarity between a word in the stem and correct

response.


Diamond & Evans 1972



Clang Association

Question

The drug of choice for patients experiencing

zhiglerastasis secondary to kweatlostosis is:

A. dragrootmycine.

B. brusterocilline.

C. noopkweetosporine.

D. zhiglerpurocine.

E. pneustodregcycline.

9/1/2015

13


Deductive Approach (Convergence)A. W & X

B. Y & Z

C. X & Y

D. V & Z

E. X & Z


Smith 1982


Deductive Approach (Convergence)A. W & X V = 1

B. Y & Z W = 1

C. X & Y X = 3

D. V & Z Y = 2

E. X & Z Z = 3


Smith 1982

A patient with a history of angina and syncope is noted to have

a systolic murmur. Which one of the following sets of findings

distinguishes chronic mitral regurgitation from other etiologies?

A. Laterally displaced PMI, normal S2, murmur intensity decreases with Valsalva

B. Normal PMI, split S2, murmur is loudest during midsystole

C. Laterally displaced PMI, normal S2, murmur is loudest during late systole

D. Sustained PMI (heave), normal S2, murmur radiates to apex

E. Laterally displaced PMI, split S2, murmur intensity decreases with Valsalva

Deductive Approach

9/1/2015

14

A patient with a history of angina and syncope is noted to have

a systolic murmur. Which one of the following sets of findings

distinguishes chronic mitral regurgitation from other etiologies?

A. Laterally displaced PMI, normal S2, murmur intensity decreases with Valsalva

B. Normal PMI, split S2, murmur is loudest during midsystole

C. Laterally displaced PMI, normal S2, murmur is loudest during late systole

D. Sustained PMI (heave), normal S2, murmur radiates to apex

E. Laterally displaced PMI, split S2, murmur intensity decreases with Valsalva

Deductive Approach

Guessing Strategies

• Changing Answers….

– 1/5 = What is percent chance right?

– If first guess = 20% (1/5)

– Second guess = 25% (1/4)

– BUT - If right and change it, now 100% wrong!

9/1/2015

15

Research indicates:

Optimal amount of stress (maximizes performance)

Yerkes-Dodson Law 1908

High

Performance Test Anxiety

Low

Low Stress High

Test Anxiety

Can Cause

• Psychological Changes– GI Symptoms, Insomnia, Headaches

• Emotional Changes– Irritability, Moodiness, Frustration, Anger

• Cognitive Changes – Memory Blocks, Attention Span Limitations

Test Anxiety

Caused by

• Lack of Preparation– Cramming

– Procrastination

– Poor Study Habits

• Recurring Thoughts

– During the exam

– BEFORE the exam

Test Anxiety

9/1/2015

16

Ways to Reduce

• Plan Studying & Improve Study Skills

• Understand the Examination – Test Blueprint (Table of Specifications)

– Pilot Items

• Build Confidence– Study effectively & efficiently

Test Anxiety



• Test Blueprint


• Transfer

• Self-Regulation

Plan

Activity (a verb not a noun)

• Select Topics (AOBFP Exam Blueprint)

• When to Study (Metacognition & Self-Regulation)

• How to Study (Long-Term Memory & Transfer)

• Ways to Reduce Test Anxiety (Pilot Items)

9/1/2015

17

PLANNING

for

Standardized

TestsScoring

Content

Skills

Timed

Guessing

Items

Standardized Tests - PLANNING

Test-Wise Test-Takers: First Step in Planning

Blueprint – Why Important?

Exam Content %

Addiction Medicine 3

Adolescent Medicine 4

Behavioral Sciences 12

General Medicine 48

Geriatrics 5

Surgery 13

Obstetrics / Gynecology 4

Pediatrics 4

Sports Medicine 3

Women’s Issues 4

http://www.aobfp.org/cert-req/cert-app.html

9/1/2015

18

Blueprint – Distribution of Items

Exam Content %



Behavioral Sciences 12 Weighting

General Medicine 48 Weighting

Geriatrics 5

Surgery 13 Weighting


Pediatrics 4

Sports Medicine 3

Women’s Issues 4


Breakdown by Discipline

General Medicine 48%

Allergy / Immunology / Rheumatology 5

Cardiology 5

Dermatology 5

Endocrinology 5

Gastroenterology 5

Hematology 4

Nephrology / Urology 4

Neurology 5

OPP 5

Pulmonology 5



Behavioral Sciences 12%

Preventive Medicine 5

Psychiatry 4

Medical Jurisprudence 3


9/1/2015

19


Surgery 13%

EENT 5

General Surgery 3

Orthopedics 5


Blueprint – Guides Study Order

Exam Content % Interest




General Medicine 48

Geriatrics 5

Surgery 13


Pediatrics 4

Sports Medicine 3

Women’s Issues 4


Blueprint – Be Mindful of Priorities

Exam Content %




General Medicine 48 DS #10

Geriatrics 5

Surgery 13

Obstetrics / Gynecology 4 DS #1

Pediatrics 4

Sports Medicine 3

Women’s Issues 4

INVERSE of INTEREST

9/1/2015

20

Ways to Enhance

MEMORY

Information

Storage

Retrieval

Recognition Recall

Information IN

Storage

Study Habits

Learning Style

Organization

Retrieval

Associations

Understanding

Cues

Information OUT

Retrieval

Recognition Recall

Less Active

Factual

Cues Present

Repetition

Cramming

More Active

Problem Solving

Cue Absent

Bridges

Strategies

Ways to Enhance the Process

RememberingTransfer

30% 70%

9/1/2015

21

PLANNING

for

Standardized

TestsScoring

Content

Skills

Timed

Guessing

Items


Pilot Items

1. Can’t identify them

2. Don’t count

3. Capitalize on presence

Preparation

PRACTICE

• Study Techniques (Clinical Vignettes)

Study Aids (Which Resources to Use)

Study Behaviors (Reading; Q&A Activities)

Preparation

Improving Performance

Content Context(Information) (Abilities and Attitude)

Study Aids Study Behaviors

9/1/2015

22

• Total-Time Hypothesis

Reading next day (Distributive Trials)• Reduces time required to “relearn it”

• Increase memory

Time Management

Ebbinghaus 1880; Medina 2008

• Escape Syndrome

– Sitting around thinking about/talking about

“how bad it’s going to be”…

Time Management

9/1/2015

23

• Understand & Associate (Meaning)

– Meaningless Facts memorized

• Day 5 = Recall ~ 40%

• Day 30 Recall < 10%

– Principles Understood with Meaning

• Day 5 = Recall > 95%

• Day 30 Recall > 90%

Self-Directed Learning Methods

Maddox 1964; Medina 2008


• Develop a Strategy

– Choose strategy to meet your needs

Reading

• 50% of time should be spent reflecting

• Additional Suggestions

– Paraphrasing (increase understanding)

– Retrieval (create notes from memory)

McWhorter 1998

• Self-Evaluation

– Be sure “Closing the Gap”

• Question-and-Answer Resources

–Maximize potential


9/1/2015

24

Impact of Stem (Vignette)

Examinees with

Higher Scores

Examinees with

Lower Scores

No Vignette 99% 90%



http://www.nbme.org/pdf/itemwriting_2003/2003iwgwhole.pdf

Examinees with

Higher Scores

Examinees with

Lower Scores

No Vignette 99% 90%



Q&A resources

• Fluency

– Practice reading faster

– 10 minutes = 10 items; 60 minutes = 60 items

– Do not look up any answers until finished!

– Better if global not by topic / content

Q&A resources

• Fluency

– What else do we know about reading?

– When the answer is not readily apparent…

– Lower scorers: stem-options-stem-options

– Higher scorers: stem-stem-stem…

9/1/2015

25

Q&A resources

• Accuracy

– Feedback is critical to your success!

– Read explanations before you score yourself

– When you calculate your score, break it down

Q&A resources

• Accuracy

– Break your score into component parts

– 1. A*

– 2. C^

– 3. D#

A* = I know this

C^ = I have some familiarity with this

D# = I have no idea

Q&A resources

• Accuracy

– Feedback

– Overall score = 47/60 (78%)

– BUT…

* = 11/20 (55%)

^ = 17/20 (85%)# = 19/20 (95%)

This is a problem!

9/1/2015

26

Q&A resources

• Accuracy

– Anticipation is critical to your success!

– For every practice item manipulate the stem to

make each distractor become a correct answer.

Example

A 64yo male with thoracic complaints….

Historical and Physical features

EKG, Labs, X-rays, other…

A. Myocardial Infarction

B. Pericarditis

C. Pulmonary Embolism

D. Dissecting Thoracic Aortic Aneurysm

E. Somatic Dysfunction

Example

A 64yo male with chest tightness….

Hx: HTN and Chol; PE: Carotid Bruits

ST Segment elevation (II, III & aVF); CK-MB


B. Pericarditis




9/1/2015

27

Example

A 64yo male with acute pleuritic pain….

Hx: Virchow’s Triad; PE: Sinus Tachycardia

EKG: S1Q3 3; CXR: hemidiaphragm elev


B. Pericarditis




Why are Standardized Tests so hard?

Faculty-generated tests Standardized tests




• Reading Fluency

• Guessing


• Test Blueprint


• Transfer

• Self-Regulation

TEST

ANXIETY

PLANNING

for

Standardized

TestsScoring

Content

Skills

Timed

Guessing

Items


Test-Wise Test-Takers: First Step in Planning

References 1. West DC et al. Critical Thinking in Graduate Medical Education.

JAMA. 2000;284(9):1105-1110

2. McGaghie WC, et al. What is the Impact of Commercial Test Preparation Courses on Medical Examination Performance? Teaching and Learning in Medicine. 2004;16(2):202-211.

3. Werner LS, Bull BS. The Effect of Three Commercial Coaching Courses on Step One USMLE Performance. Medical Education. 2003;37:527-531

4. Sefcik DJ, Prerost FJ, Arbet SE. Personality Types and Performance on Aptitude and Achievement Tests: Implications for Osteopathic Educators Journal of the American Osteopathic Association. 2009;109(6):296-301

5. Millman J, Bishop CH. An Analysis of Test-WisenessEducational and Psychological Measurement. 1965;25(3):707-726

6. Evans W. Test Wiseness: An Examination of Cue-Using Strategies. Journal of Experimental Education. 1984;52(3):141-144

7. Dolly JP, Williams KS. Using Test-Taking Strategies to Maximize Multiple-Choice Test Scores. Educational and Psychological Measurement. 1986;46(3):619-625

8. Norman GR et al. Factors Underlying Performance on Written Tests of Knowledge. Medical Education. 1987;21:297-304

9. Wenghofer E et al. Doctor Scores on National Qualifying Examinations Predict Quality of Care in Future Practice. Medical Education 2009;43:1166-1173.

10. Baeten M, Kyndt E, Struyven K, Dochy F. Using Student-Centered Learning Environments to Stimulate Deep Approaches to Learning: Factors Encouraging or Discouraging Their Effectiveness. Educational Research Review 2010;5:243-260.

11. Skochelak SE. A Decade of Reports Calling for Change in Medical Education: What Do They Say? Academic Medicine 2010;85(9):S26-S33.

12. Prince M. Does Active Learning Work? A Review of the Research. Journal of Engineering Education. 2004;7:223-231.

13. Winne PH, Nesbit JC. The Psychology of Academic Achievement. Annual Review of Psychology 2010;61:653-678.

14. Laatsch L. Evaluation and Treatment of Students With Difficulties Passing the Step Examinations. Academic Medicine. 2009;84(5):677-683.

15. Jairam D, Kiewra KA. An Investigation of the SOAR Study Method. Journal of Advanced Academics 2009;20(4):602-629.

16. Michael J. Where's the Evidence that Active Learning Works? Advances in Physiological Education. 2006;30:159-167.

17. Light, Greg, Roy Cox, and Susanna Calkins. Learning and Teaching in Higher Education 2nd ed. London: SAGE, 2009.

18. Medina, J., Brain Rules. Seattle, WA: Pear Press, 2008.

19. Sefcik D, Bice G, Prerost F. How to Study for Standardized Tests. Jones & Bartlett Learning, 2013.

test taking skills - s3.amazonaws.com introductory comments ... (> 2.5 mm) with a right...

Documents