creating tests that measure critical thinking in nursing education
TRANSCRIPT
![Page 1: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/1.jpg)
Creating Tests that Measure Critical
Thinking in Nursing An ExamSoft Client Webinar
![Page 2: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/2.jpg)
Crea%ng tests that measure
cri%cal thinking in nursing: Test item wri%ng that is both art and science
Ainslie t. Nibert, PHD, RN, FAAN February 17, 2015
![Page 3: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/3.jpg)
Resources for Developing Cri2cal Thinking Test Items and Alternate Format Items:
Na2onal Council Website
• www.nscbn.org – NCLEX Test Plans
• 2013 RN • 2014 PN
– Candidate FAQ – Alternate item formats FAQ – Exam Development FAQ
3
Source: https://www.ncsbn.org/2324.htm
![Page 4: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/4.jpg)
Rela%onship between Tes%ng & the Curriculum
4
Internal and External
Curriculum Evaluation
Outcome Predictors
![Page 5: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/5.jpg)
5
q Standardized Exams
q NCLEX-RN/PN®
External Evaluation Compare Individual Group with Population at Large
![Page 6: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/6.jpg)
6
q Writing Critical Thinking Test Items
q Item Analysis Software & Blueprinting
q Test Item Banking & Exam Delivery
Internal Evaluation Evaluation of course objectives (faculty designed or outsourced)
![Page 7: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/7.jpg)
7
Five Guidelines to Developing Effective Critical Thinking Exams
q Assemble the “basics.”
q Write critical thinking test items. q Pay attention to housekeeping duties.
q Develop a test blueprint. q Scientifically analyze all exams.
![Page 8: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/8.jpg)
8
Sound Instruction
Educator’s
Golden
Triangle Instruction
Evaluation Objectives
Outcomes
![Page 9: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/9.jpg)
9
Philosophy Outcomes Objectives
Instructional Methods
Evaluation Tools
![Page 10: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/10.jpg)
10
Definition
Critical Thinking
The process of analyzing and understanding how and why we reached a certain conclusion.
![Page 11: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/11.jpg)
11
Bloom‘s Taxonomy: Benjamin Bloom, 1956 (revised)
Terminology changes "The graphic is a representation of the NEW verbage associated with the long familiar Bloom's Taxonomy. Note the change from Nouns to Verbs [e.g., Application to Applying] to describe the different levels of the taxonomy. Note that the top two levels are essentially exchanged from the Old to the New version." (Schultz, 2005) (Evaluation moved from the top to Evaluating in the second from the top, Synthesis moved from second on top to the top as Creating.) Source: http://www.odu.edu/educ/llschult/blooms_taxonomy.htm
![Page 12: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/12.jpg)
Post-Exam Item Analysis: An important aspect of item writing
Helps to determine the quality of a test
12
![Page 13: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/13.jpg)
13
Consistency of Scores
![Page 14: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/14.jpg)
Reliability Tools
14
q Kuder-Richardson Formula 20 (KR20)—EXAM
Ø Range from –1 to + 1
q Point Biserial Correlation Coefficient (PBCC)—TEST ITEMS
Ø Range from – 1 to + 1
![Page 15: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/15.jpg)
15
q Item difficulty 30% - 90%
q Item Discrimination Ratio 25% and Above
q PBCC 0.20 and Above
q KR20 0.70 and Above
Standards of Acceptance
![Page 16: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/16.jpg)
Standards of Acceptance Nursing
Nursing-PBCC 0.15 and Above
Nursing-KR20 0.60 - 0.65 and Above
16
![Page 17: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/17.jpg)
17
3-Step Method for Item Analysis
1. Review Difficulty Level
2. Review Discrimination Data
q Item Discrimination Ratio (IDR)
q Point Biserial Correlation Coefficient (PBCC)
3. Review Effectiveness of Alternatives
q Response Frequencies
q Non-distracters Source: Morrison, Nibert, Flick, J. (2006). Critical thinking and test item writing (2nd ed.).Houston, TX: Health Education Systems, Inc.
![Page 18: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/18.jpg)
18
![Page 19: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/19.jpg)
19
![Page 20: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/20.jpg)
20
Content Validity
Does the test measure what it claims to measure?
![Page 21: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/21.jpg)
21
Content Validity
q Test Blueprint Ø Rational/Logical Pre-Assessment Tool
q Database Programs
Ø Storage of item discrimination data
![Page 22: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/22.jpg)
22
Test Blueprints
q Faculty Generated
q Electronically Generated
![Page 23: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/23.jpg)
An electronic blueprint for each exam in each course
23
![Page 24: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/24.jpg)
24
![Page 25: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/25.jpg)
NCLEX-‐RN® Client Needs Percentages of Items 2011 vs. 2014
25
Source: https://www.ncsbn.org/4701.htm
![Page 26: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/26.jpg)
NCLEX-‐RN® Client Needs Percentages of Items 2011 vs. 2014
Increases vs. Decreases
26
![Page 27: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/27.jpg)
27
Cri%cal Thinking Test Items
q Contain Rationale
q Written at the Application Level or Above
q Require Multilogical Thinking to Answer
q Ask for High Level of Discrimination
Source: Morrison, Nibert, & Flick (2006)
![Page 28: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/28.jpg)
28
Test Item Rationales
Test Review
q Learning Experience
q Faculty Friendly
q Legally Defensible
![Page 29: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/29.jpg)
29
Written at the Application Level and Above
q Prepare students for NCLEX®
q Promote thinking about clinical problems
q Cause teaching methods to become creative
![Page 30: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/30.jpg)
30
Require Multilogical Thinking to Answer
Definition
Thinking that requires knowledge of more than one fact to logically and systematically apply concepts to a clinical problem
![Page 31: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/31.jpg)
Highly Discriminating
• Questions should provide viable alternatives that require a high degree of discrimination to answer.
31
![Page 32: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/32.jpg)
Cri%cally-‐thinking Ques%ons
Which interven2on is most important? Which interven2on, plan, assessment data is/are most cri2cal to developing a plan of care?
Which interven2on should be done first?
What ac2on should the nurse take first? Which interven2on, plan, nursing ac2on has the highest priority?
What response is best? 32
![Page 33: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/33.jpg)
NCLEX® AlternaOve Test Item Formats
• Mul2ple-‐response items • Fill-‐in-‐the-‐blank items • Hot spot • Chart/exhibit format • Ordered Response items (Ranking) • Audio item format • Graphic Op2ons (graphics imbedded as answer op2ons)
• Any item formats, including standard mul2ple-‐choice items, may include mul2media, charts, tables or graphic images.
33
![Page 34: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/34.jpg)
Latest NCLEX® Test Item Format ConsideraOons
Units of Measure • Interna2onal Systems of Units (SI) • Metric • Imperial Measurement
Generic vs. Trade Names for Medica2ons • Generic names only in most cases • References to general classifica2ons of medica2ons
34
![Page 35: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/35.jpg)
35
Housekeeping Tips
![Page 36: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/36.jpg)
36
Item Writing Rules
q Get rid of names
q Get rid of ‘multiple’ multiples
q Use non-sexist writing style
q Develop parsimonious writing style
Ø Cross out “of the following”
Ø Delete scenarios
q Write items independent of each other
![Page 37: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/37.jpg)
37
… and More Rules
q Use a question format when possible
q Make distracters plausible and homogeneous
q Equal in length
q No opposites
![Page 38: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/38.jpg)
38
… and More Rules
q Eliminate “all of the above” and “none of the above”
q Rewrite any “all except” questions
q Ensure that alternatives do not overlap
q Vary correct answer
![Page 39: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/39.jpg)
39
… and the MOST IMPORTANT Rule
Develop written testing policies
Ø A Role of the Testing Committee
Ø Guidelines: Writing style & Format
Ø Exam Administration Procedures
![Page 40: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/40.jpg)
40
![Page 41: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/41.jpg)
41
Tes%ng CommiNee Responsibili%es
Recommenda2ons typically made to the Curriculum and Student Affairs Commi]ees to coordinate policy crea2on & enforcement:
◦ Wri2ng style ◦ Format ◦ Pilot items ◦ Grades/scores ◦ Review and analysis ◦ Item revision ◦ Track students ◦ Accredita2on prep
![Page 42: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/42.jpg)
Standardized Tes%ng: Vigilance with Test Security
1. Encourage moral behavior (Academic honesty program at your school with clear language placed in handbooks)
2. Discourage chea2ng a. Before tes2ng
1. Minimize access to exams and viewing of exam content 2. Use highest levels of security available in Blackboard for unit tests and all
security features available in standardized tes2ng plaborm -‐ protect logins & access codes; ac2ve dashboarding
3. Train proctors for live proctoring ac2vi2es
b. During tes2ng 1. Establish secure environment 2. No devia2ons to test procedures or breakdown of environmental security
allowed. Ex: leaving room equates to the test being over for that student regardless of reason
3. Vigilant proctoring
3. Detect chea2ng with Data Forensics and take ac2ons as needed
42
![Page 43: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/43.jpg)
43
![Page 44: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/44.jpg)
Item Writing Tools for Success …
Knowledge
Test Blueprint
Testing Software
![Page 45: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/45.jpg)
45
Faculty
![Page 46: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/46.jpg)
Have QuesOons? Need More Info?
Thanks for your 2me & a]en2on today!
46
866-429-8889
![Page 47: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/47.jpg)
References American Psychological Associa2on. (2004) Code of Fair Tes2ng Prac2ces in Educa2on.
Washington, DC: Joint Commi]ee on Tes2ng Prac2ces. h]p://www.apa.org/science/programs/tes2ng/fair-‐code.aspx
Morrison, S., Nibert, A., & Flick, J. (2006). Cri$cal thinking and test item wri$ng (2nd ed.). Houston, TX: Health Educa2on Systems, Inc.
Morrison, S. (2004). Improving NCLEX-‐RN pass rates through internal and external curriculum evalua2on. In M. Oermann & K. Heinrich (Eds.), Annual review of nursing educa2on (Vol. 3). New York: Springer
Na2onal Council of State Boards of Nursing. (2013) 2013 NCLEX-‐RN test plan. Chicago, IL: Na2onal Council of State Boards of Nursing. h]ps://www.ncsbn.org/3795.htm
Nibert, A. (2010) Benchmarking for student progression throughout a nursing program: Implica$ons for students, faculty, and administrators. In Capu2, L. (Ed.), Teaching nursing: The art and science, 2nd ed. (Vol. 3). (pp.45-‐64). Chicago: College of DuPage Press.
47
![Page 48: Creating Tests that Measure Critical Thinking in Nursing Education](https://reader030.vdocuments.net/reader030/viewer/2022032616/55a620101a28abfc648b4764/html5/thumbnails/48.jpg)
Click to edit Master title style
Click to edit Master subtitle style For More Information:
Call: 1.866.429.8889
Email: [email protected]
Visit: learn.examsoft.com