outcome assess allied dental accreditation handoutjune …...patient assessment, diagnosis and...

ADEA Allied Accredita-on Workshop June 11, 2013

William Hendricson Un. Of Texas HSC Dental School

2:00 – 4:00 pm Session June 11, 2013 1

Helping CODA Accredita2on Site Visitors Do Their Job

Bill Hendricson MS, MA, MEd Assistant Dean, Educa-on & Faculty Development UTHSCSA Dental School [email protected] (210) 567-‐0436

ADEA Allied Dental Accredita.on Workshop June 11, 2013 Portland, Oregon

2:00 – 4:00 pm Agenda •  Clarify language issues related to “competency” •  How to report your program’s competencies and the methods you use to demonstrate students’ a[ainment of these competencies

•  Example: CODA standard – school competency linkage table •  Example: competency assessment map

•  New DH Standard 2-‐19: Communica-on with Diverse Popula-on and Cultural Competence

•  New DH Standard 2-‐22: Ethics & Professionalism

What is a “Competency”?



2:00 – 4:00 pm Session June 11, 2013 2

Language When describing methods used to evaluate students’ a[ainment of competency, use: •  Skill assessment •  Progress assessment •  Progression exam •  Graded clinical evalua-on •  Clinical progress evalua-on •  Competency cer-fica-on

Schools/Programs Write Your Own “Gradua2on Competencies”

CODA DH Educa2onal Outcome Standard

Your Program’s Competencies

2-‐26 Graduates must be competent in problem solving strategies related to comprehensive pa-ent care and management of pa-ents.

4. Use cri-cal thinking skills and comprehensive problem-‐solving to iden-fy oral health care strategies that promote pa-ent health and wellness.

2:00 – 4:00 pm Agenda •  Clarify language issues related to “competency” •  Repor2ng your program’s competencies & methods to demonstrate students’ aQainment of these competencies






2:00 – 4:00 pm Session June 11, 2013 3

Competency Based Planning Process

4 Ques2ons

What do entry-‐level DDS need to KNOW & DO?

Competencies

What learning experiences help students obtain the competencies?

Curriculum

What EVIDENCE will we accept as proof of competency?

Outcomes

2

4 3

Reference Standards -‐ CODA 1

Hendricson, Kleffner. Curricular & Instruc-onal Implica-ons of Competency-‐Based Dental Educa-on. J Dent Educ 1998; 62:183-‐196.

Harden, Crosby, Davis. AMEE Guide 14: Outcome-‐based educa-on. Part 1: Introduc-on to outcome-‐based educa-on. Med Teacher. 1999; 21(1): 7-‐14

CODA 2013 S2pulates that Entry Level Dental Hygienists Must Be Cer2fied as Competent in 11

Components of Pa2ent Care

Professional Responsibility, Thinking Process, Ethical Values, Communica2on, Oral Health Planning, Professional Development (N = 5)

Pa2ent Care & Clinical Skills (N = 6)

Std CODA – S2pulated Competencies Pgs 21-‐25 Assessment

2-‐16

Provide dental hygiene care for child, adolescent, adult & geriatric pa2ent.

Assess treatment needs of pa2ents with special needs.

2-‐17 Provide the dental hygiene process of care

2-‐18

Provide dental hygiene care for all classifica2ons of periodontal disease including moderate to severe periodontal disease.

2-‐19

Demonstrate competence in interpersonal & communica2on skills to interact with diverse popula2ons & health care team.

2-‐20 Plan, implement & evaluate community oral health programs

2-‐21 Provide life support for medical emergencies in DH prac2ce.

2-‐22 Apply principles of ethical reasoning & decision making, professional responsibility for academic environment, research, pa2ent care & prac2ce mgmt.

2-‐23 Apply legal & regulatory concepts to provision of oral health care services.

2-‐24 Apply self-‐assessment skills for life-‐long learning.

2-‐25 Evaluate current scien2fic literature.

2-‐26 Solve problems in comprehensive pa2ent care & pa2ent mgmt.



2:00 – 4:00 pm Session June 11, 2013 4

Write Your Competencies to Parallel CODA Standards CODA S2pulated Competencies # Your Parallel Competency

2-‐16

Provide dental hygiene care for child, adolescent, adult & geriatric pa-ent.

Assess Tx needs of pa-ents with special needs.

2-‐17 Provide dental hygiene process of care

2-‐18 Provide dental hygiene care for all classifica-ons of periodontal disease

2-‐19 Demonstrate competence in interpersonal & communica-on skills to interact with diverse popula-ons & health care team.

2-‐20 Plan community oral health programs

2-‐21 Provide life support for medical emergencies i

2-‐22 Apply principles of ethical reasoning & decision making & professional responsibility

2-‐23 Apply legal & regulatory concepts

2-‐24 Apply self-‐assessment skills

2-‐25 Evaluate current scien-fic literature.

2-‐26 Solve problems in comprehensive pa-ent care

Example for CODA 2-‐20 for One Program CODA # CODA Standard # Graduates of the program will be

demonstrate competency to:

2-‐20 Plan, implement & evaluate community oral health programs

3.1 3.2 3.3

•  provide oral health services in a variety of community & ins-tu-onal segngs.

•  Assess needs & resources, plan, implement & evaluate community oral health service learning programs.

•  Implement Sealant Projects, Give Kids A Smile & school-‐based educa-onal events for underserved popula-ons in an IPE team format.

See Distributed Example from UT HSC DS



2:00 – 4:00 pm Session June 11, 2013 5


4 Ques2ons


Competencies


Curriculum


Outcomes

2

4 3




Schools Write “Gradua2on Competencies” To Address

CODA Educ Standards

Expose Explore Engage Evidence of Competence

Experience – Evaluation Continuum

Experience


4 Ques2ons


Competencies


Curriculum


Outcomes

2

4 3




Schools Write “Gradua2on Competencies” To Address

CODA Educ Standards



2:00 – 4:00 pm Session June 11, 2013 6

Assessment Principles in Competency-‐Based Educa2on

Grussing PG. Curricular design: competency perspec-ve. Am J Pharm Educ 1987; 51: 414-‐419.

Hall GE, Jones HL. Competency-‐based educa-on: a process for the improvement of educa-on. Englewood Cliffs, NJ: Pren-ce-‐Hall, Inc. 1976.

Newble D. Assessing clinical competence. Med Educ 1992; 26: 504 – 511.

Purposes of Assessment?


Miller’s Pyramid of Competency

Training Pathway

Knows Comprehends “Fundamentals”

Knows How Can apply knowledge

Shows How “Safe” prac2ce

Does Pa2ent care

Miller GE. Acad Med. 1990; 65: 563-‐567

Knows suture materials & instruments used to close incisions

Explains how to place sutures to close an incision; general to specific

Closes an incision on a model; narrates technique & answers ques-ons

Safely and efficiently closes incisions on pa-ents without assistance



2:00 – 4:00 pm Session June 11, 2013 7

Miller’s Pyramid of Competency

Assessment Pathway

Knows Comprehends “Fundamentals”

Knows How Can apply knowledge

Shows How “Safe” prac-ce

Does Pa-ent care

Miller GE. Acad Med. 1990; 65: 563-567

Assess comprehension: MCQ, Short answer essay, MTF, Matching

Assess knowledge applica2on Extended essays, Case-‐based MCQ, Oral exams

Assess use in controlled situa2ons OSCE, lab projects, simula-ons

Assess work performance Assess process, product & outcomes with pormolio, test pa-ents and longitudinal observa-on

Competence Authen2c

High Fidelity

Create Assessment Pathway Map for Each Standard

Help CODA Site Visitors Do Their Job!

EXAMPLE – Competency Assessment Map Patient Assessment, Diagnosis and Treatment Planning

CODA Standards: 2-23a (Patient Assessment & Diagnosis); 2-23b (Comprehensive Treatment Planning)

Dental School Competency Domains: 5 & 6 5: Graduates must be competent in patient assessment and diagnosis (educational outcomes on PDEC pg. 9) 6: Graduates must be competent in comprehensive treatment planning and assessment of treatment outcomes (educational

outcomes on PDEC pg. 10)

DS 1 DS 2 DS 3 DS 4 • Assessments in Physical

Evaluation 1 • Case-based exams in systemic and

oral pathology • Patient assessment, Dx & TxP

progression assessment after TxP block at start of DS3

• DS4 Portfolio: Present and defend cases in GPG Wed conferences

• Assessments in Dental Radiography

• Case-based exams and OSCE in Clinical Medicine

• Case presentations (2): complex patient and implant patient

• DX / TxP planning GCE (complex case)

• DX / TxP planning GCE (patient with medical co-morbidity or special needs)

• Oral Medicine OSCE in Physical Evaluation 2

• DEC GCE for DS 3

• PA / Dx /TXP component on Mock WREB

• Radiographic Interpretation Exams (2)

• DS3 CATs case presentation • Monthly progress assessment for competency domains 5 & 6

• Patient assessment skill exams (2) • Perio: Assessment, Dx and TxP GCE

• DS4 Case conference • DS4 CATs case presentation

• Patient assessment case presentations (4) and AxiUm entries (4) throughout DS2

• 2 full mouth radiographic surveys • Oral Medicine GCE

• PA / Dx / TxP OSCE at end of DS2 • Oral Medicine rotation (case-based examination)

• Outcomes of Care GCE

• Remediation & re-take for students who do not pass PA / Dx / TxP OSCE

• Monthly progress assessment for competency domains 5 & 6

GCE = Graded Clinical Examination CATs = Critically Appraised Topic Summaries DEC = Dental Emergency Clinic OSCE = Objective Structured Clinical Exam (station exam)

Knowss Knows & Shows Hows Shows How & Doess Does

UTHSC Dental



2:00 – 4:00 pm Session June 11, 2013 8



•  New DH Standard 2-‐19: Communica2on with Diverse Popula2on and Cultural Competence


NEW 2-‐19 Graduates must be competent in interpersonal & communica2on skills to effec2vely interact with diverse popula2on groups & other members of health care team.

Intent: Dental hygienists should be able to effec-vely communicate with individuals, groups & other health care providers. The ability to communicate verbally and in wri[en form is basic to the safe & effec-ve provision of oral health services for diverse popula-ons. Dental Hygienists should recognize the cultural influences impac-ng the delivery of health services to individuals and communi-es

Examples of evidence to demonstrate compliance may include: •  student projects demonstra-ng ability to communicate effec-vely with a variety of individuals, groups & health care providers.

•  examples of individual & community-‐based oral health projects implemented by students during previous academic year

•  evalua2on mechanisms designed to monitor knowledge and performance

Expose Explore Engage Evidence of Competence

Demonstrating Compliance with CODA 2-19



2:00 – 4:00 pm Session June 11, 2013 9

2-‐19: Assessing Students’ Competency for Mul2-‐Cultural Health Care •  Qualita-ve assessment •  Quan-ta-ve assessment •  Observed performance / professional judgment •  Self-‐Assessment

Gregorczyk & Bailit. Assessing the Cultural Competency of Dental Students and Residents. J Dent Ed. 2008; 72(10): 1122-‐1127.

Pilcher, Charles & Lancaster. Development & Assessment of a Cultural Competency Curriculum. J Dent Ed. 2008; 72(9): 1020-‐1028.

Gozu, et al. Self-‐administered Instruments to Measure Cultural Competence of Health Professionals: A Systema-c Review. Teach Learn Med. 2007; 19(2): 180-‐190.

Describe the pa-ent popula-on that students encounter in dental clinics & extramural sites. Provide pa-ent demographic data.

How does your program ensure that all students have experiences func-oning in a mul-cultural work environment?

Describe how student interpersonal skills & communica-on skills are assessed in a mul-cultural work environment.

Qualita2ve Assessment •  Case presenta-ons – mul-cultural care •  Pormolios – CSL projects & diverse pa-ent care •  Cultural or language dilemma analysis essays •  Reflec-on journals •  Assessment rubrics for above •  Debriefing seminars

Quan2ta2ve Assessment for MCC Buyer Beware: Many Instruments; Few Validated

•  Providers Guide to Quality & Culture Quiz HRSA; 23 items; assess CC knowledge

•  Cultural Competence of Health Prac22oners’ Assessment

HRSA / Na-onal Center for Cultural Competence Assess cross-‐cultural communica-on & decisions

•  Home-‐grown CC tests – be careful! Nunn, et al. Mul--‐cultural Competency Instrumenta-on: Review and Analysis of Reliability. J Counsel Dev.2006;84: 471-‐482.



2:00 – 4:00 pm Session June 11, 2013 10

Assessment of Observed Performance Probably Best Pay-‐Off!

•  Cultural Competency / Diversity OSCEs Standardized / Simulated Pa-ents (SPs)

•  Mul2-‐Cultural Triple Jump Exercises

Checklists & Evalua2on Sugges2ons:

Cultural Competency in Medical Educa3on: A Guidebook for Schools.

HRSA. 2004 www.HRSA.Gov

hQp://www.hrsa.gov/culturalcompetence/index.html

An OSCE is a sta2on exam

Examinees rotate from sta-on to sta-on at -med

intervals

Perform tasks at each sta-on



2:00 – 4:00 pm Session June 11, 2013 11

Cultural Competency OSCE Resources Aeder L, Altshuler L, Kachur E. et al. The “Culture OSCE” – Introducing A Forma-ve Assessment into a Postgraduate Program. Educa3on for Health. 2007. 20(1): h[p://www.educa-onforhealth.net Robins L, White C, Alexander G, Gruppen L, et al. Assessing Medical Students’ Awareness of and Sensi-vity to Diverse Health Beliefs Using a Standardized Pa-ent Sta-on. Acad Med. 2001. 76(1): 76-‐80.

Pa2ent Encounter Observed

WriQen Report Findings & Plan

Verbal Exam Comprehension & Reflec-on

Triple Jump Smith R. The triple-‐jump exam as an assessment tool in the medical curriculum at the University of Hawaii. Acad Med. 1993 May;68(5):366–372.

Self-‐Assessment Inventories Buyer Beware: Many Instruments; Few Validated

•  Example: Clinical Cultural Competency Ques-onnaire Echeverri. Am J Pharm Ed. 2010; 74(10) 181-‐187.

•  Accuracy of self-‐appraisal / report?

•  Untested psychometric proper-es

•  May be useful as learning ac2vity Gozu, et al. Self-‐administered Instruments to Measure Cultural Competence of Health Professionals: A Systema-c Review. Teach Learn Med. 2007; 19(2): 180-‐190.



2:00 – 4:00 pm Session June 11, 2013 12

Competency in Health Care Team Par2cipa2on / IPE

OSLO Objec2ve Structured

Leadership Observa2on

SPP Rota2on Student “Chiefs” Peer Teachers

Team Self-‐Assess Team RRR

“Lessons” Case Present Team Planning

Project

IPE Opportuni2es – Best Bet If Feasible IPCSL Clinical Teams IPE rota-ons at protected popula-on ctrs







2:00 – 4:00 pm Session June 11, 2013 13

NEW CODA 2-‐22: Graduates must be competent in the applica2on of the principles of ethical reasoning, ethical decision making & professional responsibility as they pertain to the academic environment, research, pa2ent care & prac2ce management.

Intent: Dental hygienists should understand & prac2ce ethical behavior consistent with the professional code of ethics throughout their educa2onal experiences. Examples of evidence to demonstrate compliance may include: • documents which ar-culate expected behavior of students such as policy manuals, college catalog, etc. • evalua-on of student experiences which promotes ethics, ethical reasoning and professionalism • evalua-on strategies to monitor knowledge & performance of ethical behavior If you don’t know where you’re going, you’ll end up someplace else.” Yogi Berra


4 Ques2ons


Competencies


Curriculum


Outcomes

2

4 3




2-‐22: Apply principles of ethical reasoning, decision-‐making & professional responsibility

“IDEALS” Model 6 Ques2ons for Effec2ve Thinking •  Iden2fy problem. What’s the real ques-on we’re facing? •  Define context. What circumstances frame this problem? •  Enumerate choices. What are our most plausible op-ons? •  Analyze op2ons. What is our best course of ac-on, all things

considered? •  List reasons explicitly. Why are we making this choice? •  Self-‐assess & correct. Let’s review. What was missed?

Facione. Thinking & Reasoning in Human Decision Making. Millbrae, CA: California Academic Press, 2007.



2:00 – 4:00 pm Session June 11, 2013 14

Evidence of Competency AQainment? 1.  Ethical dilemma simula-ons / essays requiring

applica-on of an analysis process such as IDEALS 2.  Reflec-on journals / logbooks with reflec-ve comments 3.  Ethical case presenta-ons based on 1 & 2 4.  Scoring rubrics exist for 1, 2 & 3 5.  Scenario-‐based wri[en tests (I don’t favor) 6.  Ethical dilemma sta-ons during OSCEs 7.  RRR – Research, wRite, Report (oten a team project) 8.   Longitudinal observa2on (session, daily, monthly by

clinical faculty with rou-ne feedback & documented incident reports) – Specific criteria on eval forms

9.   Program has documented, distributed protocol for determining when student has “crossed the line”

Written Tests

Context Free MCQ

Case-Based MCQ

Brief focused Essay

Extended Essay

Feedback & Reflection

Self Assess

Reflective Writing

Learn Plan

360O Eval

Inquiry CATs Critically Appraised Topics

RRR Research, wRite, Report

Technology Simulation

Virtual Reality Computer-based

Models Lab Simulators

Station Exams OSCE Triple Jump Exercise

Work Samples Case Present

Portfolio of cases & products

Chart Audit

With Oral Exam (CPR)

Observation Based

Assessment

Single Task/Product Rate with Checklists

Test Cases

Global Ratings Overall performance

across time

Assessment Toolkit Albino (JDE; 2008) Kramer (JDE; 2009)

Competency Porrolio

•  Exemplary (best) work vs. representa-ve work? •  Linked to CODA standards •  What types of evidence & how much? •  Self assessment & reflec2on exercises •  Student generated learning plan / objec2ves •  High % of assessment – must have grading “teeth” •  Final review panel & criteria

outcome assess allied dental accreditation handoutjune …...patient assessment, diagnosis and...

Documents