ارزشیابی صلاحیت بالینی با استفاده از objective structured clinical...
TRANSCRIPT
ارزشیابیارزشیابی
صالحیت بالینیصالحیت بالینی
با استفاده ازبا استفاده از
Objective Structured
Clinical Examination (OSCE)
ACGME Core Competencies Medical knowledge Patient care Interpersonal and communication skills Professionalism Practice-based learning and
improvement Systems-based practice
Non-core
Core
OSCE is suggested the first best method for evaluation of: Interviewing Counseling & educating pt’s & families Performing procedures Physical examination Preventive health services Creation of therapeutic relationship with patients Listening skills Respectful & altruistic behaviors Sensitive to cultural, age, gender & disability
issues
Measurement Measurement propertiesproperties
ValidityValidity is the is the hallmark of OSCEhallmark of OSCE (Stillman, et al, Newbel, et al).(Stillman, et al, Newbel, et al).
Van der Vleuten, C. The assessment of professional competence: developments,research and practical implications Advances in Health Science Education 1996, Vol 1: 41-67
Validity
Reliability
Educational impact
Acceptability
Feasibility
ValidityValidity
Validity: does the test measure what it is supposed to?
Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice. 2nd ed. Upper Saddle River, NJ: Prentice-Hall; 2000:86-89.
Content validityContent validity
Is there something missing?
Does the test “covers” the content domain?
Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice. 2nd ed. Upper Saddle River, NJ: Prentice-Hall; 2000:86-89.
OSCE : blueprintOSCE : blueprint
Historytaking
Interpersonal communication
Physical Exam
Procedure
CVS Chest pain Disch drugs Cardiac BP
RS Haemoptysis
Smoking Resp Peak flow
GIS Abdo pain Gastroscopy Abdo PR
Repro Amenorrhoea
Abnormal smear Cx smear
NS Headache Eyes Ophthalmosc
MS Backache Hip
Generic Pre-op assess
Consent for post mortem
IV canulationBlood trans
Construct Construct validityvalidity
Known groups methodConvergence divergence evaluating
The ability of a test to measure a construct or abstract concept
Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice. 2nd ed. Upper Saddle River, NJ: Prentice-Hall; 2000:86-89.
Validation of Blueprint Face validity Content validity
Content relevance Content coverage
را وظایفی شما واقعا همان Blueprintآیا منعکس می کند
که در محیط های بالینی واقعی از یک پزشک انتظار می رود؟
OSCE راهنمائي جهت باال بردن روائي
مشكالت ارائه شده توسط بيمار يا بيمار نما مهم -1.و مرتبط با برنامه آموزشي باشد
ايستگاه هاي مورد ارزيابي قبال آموزش داده شده -2.باشند
بررسي و مرور ايستگاه ها توسط متخصصين -3
ايستگاه 10تعداد مناسب ايستگاه ها )كمتر از -4نباشد(
ReliabilityReliability
Reliability: does the test measure consistently between people and over time?
Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice. 2nd ed. Upper Saddle River, NJ: Prentice-Hall; 2000:86-89.
Test Sample
Test Sample
Content specificity
Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice. 2nd ed. Upper Saddle River, NJ: Prentice-Hall; 2000:86-89.
Is a mater of samplingAcross content
Movaffaghi Z, Dastani M
Reliability as a function of testing time
Testing
TimeMCQ PMP OSCE
Mini
CEX
1 hour0.62 0.36 0.54 0.73
2 hours0.76 0.53 0.69 0.84
3 hours 0.93 0.69 0.82 0.92
4 hours 0.93 0.82 0.90 0.96
Day 11. Well Baby Care H/Ps 2. Vaginal bleed H/Ps 3. Breast Examination P/Ps 4. Level of Consciousness P/Ps 5. Shortness of Breath P/Ps 6. Shortness of Breath H/Ps 7. Diarrhea H/Ps 8. Intermittent Claudication P/Ps 9. Extracellular Fluid volume H-
P/Ps 10. Back Injury P/Ps 11. Left Lower Quadrant Pain P/Ps 12. Pregnancy H/Ps 13. Heart Murmur (pediatric) S/Ps 14. Anemia H/Ps
15. Splenomegaly P/Ps
Day 21. Knee Examination P/Ps 2. Chest Pain H/Ps 3. Left Flank Pain H/Ps 4. Spinal Cord Injury P/Ps 5. Thyroid P/Ps 6. Back Pain Lab/Ps 7. Lightheadedness Ps 8. Anorexic Adolescent I 9. Post-operative Fever H/P 10. Contraception I 11. Acute Respiratory Distress
Ps 12. Bad News. I 13. Shortness of Breath Ps 14. Oncology (Hypercalcemia) Ps 15. Mental Status H
H - history, P - physical examination, I - interview, Ps - problem-solving, S - mechanical simulation, Lab - laboratory procedure.
OSCEعواملي كه سبب كاهش پايائي :شوند مي
- كم بودن تعداد و يا زمان ايستگاه ها1
- چك ليست ها يا آيتم هائي كه افتراق كمي دارند2
- بيماران غير قابل اعتماد يا بيماراني كه براي همه 3آزمون شوندگان بطور يكسان نقش ايفا نكنند.
- عدم يكنواختي نظر ممتحنين نسبت به پاسخ و عيني 4نبودن مالك نمره دهي
The Components of The OSCE
The coordinating committee
Designing
1. Blueprinting2. Station development3. Case Review and Validation4. Grading
The coordinator
Examinee
Marking scheme of the checklists
The examiners
Examination site
Instruction to Patients /SPs Time keeper
Material & Logestics
The Timeline
3-6 months prior Coordinator appointed Committee formulated
2-3 months prior Content finalized Stations developed Standardized patients recruited
The Timeline
1 month prior Stations finalized Artifacts obtained Standardized patients trained
7 days prior walk-through Standardized patients confirmed Faculty confirmed
The OSCE Format
Built upon a blueprint
Stations )interactive/non-interactive( require the candidate to demonstrate both clinical knowledge and specific communication or psycho-motor skill)s(
The OSCE Format
Validation of stations essential to ensure they are truly representative
Examination of results to ensure reliability, generalizability, and dependability of results
Design Step 1 – Select Design Step 1 – Select CompetenciesCompetencies
Breakdown ACGME’s six global competencies Breakdown ACGME’s six global competencies into specific competenciesinto specific competencies
Technical skills, case management skills, etc.Technical skills, case management skills, etc.
Avoid too broadAvoid too broad
Avoid too specificAvoid too specific
Appropriate for the resident’s training levelAppropriate for the resident’s training level
Example Competency Example Competency StatementsStatements
The Global Communication Skills CompetencyThe Global Communication Skills Competency
Communicate accurately, efficiently and supportively with the Communicate accurately, efficiently and supportively with the patient, patient’s family and the health care teampatient, patient’s family and the health care team
Specific Communication Skill CompetencySpecific Communication Skill Competency
Build a Relationship (Rapport Building)
Component Skills used to Build a Relationship
Establish initial rapportEstablish initial rapport Support Support
Empathy Partnership
Reflection Legitimation
Respect Apology
Design Step 2 – Select Design Step 2 – Select SamplingSampling
Define clinically relevant problems and assignments which:
• Resemble the actual professional tasks
• Allows measurement of those observable types of skills, activities, attitudes or products.
Design Step 3 – Select Design Step 3 – Select RatersRaters
Raters are qualified individuals who have an Raters are qualified individuals who have an
opportunity to opportunity to observeobserve and draw conclusions and draw conclusions
about residents – they about residents – they judgejudge the presence and the presence and
the quality of the competency to be assessedthe quality of the competency to be assessed
How many ratersHow many raters
AttendingsAttendings
Other professional staffOther professional staff
Patients/SPsPatients/SPs
SelfSelf
Design Step 4 – Select Design Step 4 – Select Rating CriteriaRating Criteria
Develop the measurement tools (Checklists)Develop the measurement tools (Checklists)
Define minimal acceptable level of performance Define minimal acceptable level of performance
Angoffs MethodAngoffs Method
نكته در طراحي ايستگاه هاي نكته در طراحي ايستگاه هاي 77OSCEOSCE
- نوشتن و نصب دستورات واضح براي داوطلبان بر روي در ورودي هر - نوشتن و نصب دستورات واضح براي داوطلبان بر روي در ورودي هر 11ايستگاه (آنچه را كه دستيار بايد انجام دهد)ايستگاه (آنچه را كه دستيار بايد انجام دهد)
- نوشتن دستورات و توضيحات واضح براي آزمون گيرندگان در مورد هر - نوشتن دستورات و توضيحات واضح براي آزمون گيرندگان در مورد هر 22ايستگاهايستگاه
- تهيه فهرست از تجهيزاتي كه براي هر ايستگاه الزم است.- تهيه فهرست از تجهيزاتي كه براي هر ايستگاه الزم است.33
تدوین دستورالعمل مشروح برای بیمار یا بیمار نما با توضیح جزئیاتتدوین دستورالعمل مشروح برای بیمار یا بیمار نما با توضیح جزئیات- - 44
- تهيه سناريو براي بيمار نما در صورتي كه قرار است نقشي را ايفا - تهيه سناريو براي بيمار نما در صورتي كه قرار است نقشي را ايفا 55نمايد.نمايد.
- تنظيم برنامه هر ايستگاه: شامل تمام جوانب مهم مهارتي مورد آزمون- تنظيم برنامه هر ايستگاه: شامل تمام جوانب مهم مهارتي مورد آزمون66
- تعيين مدت زمان هرايستگاه- تعيين مدت زمان هرايستگاه77
هزینه های برآورد شده برای هزینه های برآورد شده برایOSCEOSCE دالر دالر 2828--10001000 از از به ازای هر دانشجو متغیر است.به ازای هر دانشجو متغیر است.
.مهمترین بخش هزینه ها، هزینه های پرسنلی است.مهمترین بخش هزینه ها، هزینه های پرسنلی است
50%50% هزینه ها به مربوط به مرحله طراحی آزمون هزینه ها به مربوط به مرحله طراحی آزمون می باشد. می باشد.
دالر برآورد شده است دالر برآورد شده است 77--5050هزینه بیمار نما ساعتی هزینه بیمار نما ساعتی )بسته به میزان تعامل آنها با آزمون شوندگان(.)بسته به میزان تعامل آنها با آزمون شوندگان(.
OSCEOSCE هزینه هایهزینه های
خدا قوت