European Hematology Examand

Progress testDr. Tomás Navarro

Chair Curriculum-Exam CommitteeInstitut Català d’Oncologia-Germans Trias i Pujol Hospital

Josep Carreras Leukaemia Research Institute

Reasons for the project

§ A ‘logical’ next step toward the harmonization of hematology education and training

§ Provide an extra quality stamp to hematologists§ Facilitate mobility§ Contribute to improve patient care throughout Europe

Composition Curriculum-Exam CommitteeChair:Tomás Navarro, SpainVice chairMarielle Wondergem, The NetherlandsMembers:- Antonio Almeida, Portugal- Gunnar Birgegård, Sweden- Janaki Brolin, Sweden- Carlos Fernández de Larrea, Spain- Mahesh Prahladan, UK- Alicia Rovó, Switzerland

EHA office:- Nathalie van Havre- Wietske Hollegien- Judith Lewis-Stride- Manon Saris- Enrica Orsini

Target group

§ Hematologists with an interest in self-assessment and education. Independent of their career phase

§ Main target group are hematologists who just finalized their specialty training

§ Having completed the Online Curriculum Passport

Benchmarking:§ National Hematology Societies § European Medical Specialty Societies

§ European Society Anaesthesiology § European Society of Cardiology § European Society/Board of Radiology§ European Board of Urology§ European Respiratory Society

Exam development

European knowledge test

Multiple choice questionnaire§ Allows a large sample size of test items§ Testing knowledge in several areas with high reproducibility§ One single test covering the full European Curriculum§ Can be used in a web-based format§ Automatic objective correcting and scoring

Exam development

European knowledge test

§ Multiple choice questionnaire in English§ official language of EHA§ recognized scientific language worldwide§ Wording in different languages can cause different

interpretations § Mobility requires good level of English

Exam development

EvaluationDiscuss thresholds for high cost (JAK2 inhibition) treatment in PV

SynthesisSuggest a treatment algorithm for myelofibrosis

AnalysisCompare recent risk score models in ET

ApplicationDiscuss how symptom profile influences therapy choice in PV

UnderstandingHow does JAK2V617F mutation cause uninhibited erythropoiesis?

RecallWhich mutation is most common in PV?

The taxonomy ladder

What did we decide?

§ The European Hematology Curriculum (version 3) is our basis and must be reflected in the examination

§ All taxonomic levels cannot be tested but we should aim at the highest level possible

§ The context should be as ”real” as possible§ Must include multiple samples to cover the Curriculum as reliably as

possible

Ergo:§ Well written MCQs that test understanding and its application

Multiple choice question writing

Guidelines for the development of Questions & answers for the European Exam in Hematology

Writing questions§ First: what do you want to test?

(Find common clinical misunderstandings, knowledge gaps in residents)

§ Find a clinical scenario, preferably authentic, reflecting normal clinical pratice.

§ Check the evidence base for the correct answer§ Find 4 distractors if you can, but don’t use unrealistic ones, rather

use 3§ Consider the stem for a 2-step question

Each question consists of three parts: ‒ The stem ‒ The question ‒ The answer options

The Stem

a). A clinical scenario (improves validity, allows higher taxonomic level)

§ Clinically plausible (authentic if possible)

§ Short§ Enough info to anticipate the

right answer

b). Stem without clinical scenario

§ Fact control§ Used when clinical stem is

difficult/impossible to match with the question

§ Low taxonomic level, should be rather few

The Question

§ Direct, unambiguous§ Preferably address single issue§ Knowledgeable student can anticipate correct answer before seeing

options§ Should ask for the best answer: most likely/most appropriate/most

predictive/most reliable/most important§ Avoid negative questions, never use double negatives

The answer options

§ Plausible and realistic§ Clearly related to the stem and question§ One option must be clearly the best§ Generally relate to the same modality§ 5 options if you can find 4 plausible distractors, otherwise 3

distractors

The Exam Format

Test Format

§ Preferably questions with a “real scenario” stem§ Low number of “fact questions”§ Number of questions: 100§ Exam duration: 2.5 hours

Exam question development

§ Recruitment of question writers: 20 European hematologists SWGs as an important source

§ Preparation of guidelines and workshops for question writers (question development training)

§ In pairs, these writers work on their questions and review each other’s questions

The Question Review Group

§ The Question review group consists of 7 hematologists.§ This committee is responsible for the final round of review of all the

questions.§ This group selects the final 100 questions to be included in the

exam, representing all Curriculum sections.§ The review ensures all sections of the curriculum are addressed in

balance, that the questions are relevant and, well written, according to the Question Writing Guidelines.

Quality Assurance

§ To guarantee independent judgment of the quality of the exam and the passing score definition, EHA collaborates with CITO, an internationally recognized professional research and knowledge institute in the field of educational measurement and testing.

§ CITO’s Psychometric Research- and Knowledge Center performs the psychometric analysis of the results.

Candidate ExperienceAt the end of the exam, the candidates are asked to complete a short evaluation survey. Results of 2020 survey:

Exam sessions

2017: 1st exam at the EHA Congress - 64 candidates

2018, 2nd exam at the EHA Congress & parallel session in Bern* - 80 candidates in total

2019, 3rd exam at the EHA Congress & 5 parallel sessions - 149 candidates (incl 87 at main session)

2020, although there was no main session, still 127 participants in 12 parallel sessions!*As of 2019, European Exam is the official knowledge exam for Swiss hematologists

2021 Exam§ Collaborating with National Societies (NS) for organizing

parallel sessions§ Hold the exam on Thursday, June 3, one week before

the Congress§ The Italian Society of Hematology is also very welcome

to host a session!

Progress test

Progress test

What is the goal?

§ To see what level you have achieved in different areas of hematology compared to your European colleagues

§ To pinpoint your own “knowledge gaps” and be provided with suggestions to improve them

§ Monitor your progress through the years§ To prepare for the exam

Progress test

§ The Progress test consists of almost 100 multiple choice questions and is modelled on previous European Hematology Exams.

§ On completion, you receive immediate feedback (on your correct & incorrect answers), including references for more in-depth reading and studying.

§ 3 weeks after the end of the testing period, you will receive comprehensive results per curriculum section, together with feedback on your overall performance in relation to the passing grade.

Progress test

Format: feedback for participant§ Achieved level of competence in

every section of the European Hematology Curriculum: “expectation for end-of-training” level: below or at/above

§ Indicates progress

Progress test

§ The progress test is available twice per year

§ The next progress test will be available in March 2021

Thank you!Gràcies

Gracias

Merci

Danke

Obrigado Grazie mille!!

Tak Dzieki

ευχαριστίες

Mila esker

Bedank

teşekkür ederim