Statistical interpretation of studies among doctors and
medical studentsAZEVEDO André; CARDOSO Ana; FIGUEIREDO Carlos; GONÇALVES Francisco; GONÇALVES Raquel; MOITA João; MONTEIRO Renata; NOGUEIRA Miguel; PINTO Sérgio; RODRIGUES Daniel; RODRIGUES Marta; VIEIRA Joana; TEIXEIRA Ana
Advisors:Ricardo Filipe Sousa SantosAlfredo Mendes Castro
May 2012
Introdução à Medicina II
2
Introduction
Aims
Methods
Results
Discussion
Conclusion
References
Acknowledgments
Medicine came to be as man sought cures for the diseases and remedies for the injuries that affected
human kind.
Fig.1
Evolutionpast prese
nt
3
Introduction
Aims
Methods
Results
Discussion
Conclusion
References
Acknowledgments
Primitive man looked on disease as a curse cast on him by an evil spirit; his treatment consisted of driving the demon that possessed him[1].
Fig.2
[1] Claridge, J.; Fabian, T. History and Development of Evidence-based Medicine. World Journal of Surgery; 2005; 29, 547-553
Evolutionpast prese
nt
4
Introduction
Aims
Methods
Results
Discussion
Conclusion
References
Acknowledgments
Observation of disease, its symptoms and treatments, were first found in written documents.
Fig.3
Evolutionpast prese
nt
5
Introduction
Aims
Methods
Results
Discussion
Conclusion
References
Acknowledgments
However,
were the main foundation for medical decision making[2].
Expert opinion Experience Authoritarian judgment
[2] Dahm, P.; Sur, R.: History of Evidence-based medicine. Indian Journal of Urology; 2011; 27; 487-489
Evolutionpast prese
nt
6
Introduction
Aims
Methods
Results
Discussion
Conclusion
References
Acknowledgments
Progressively, Medicine became to analytically study and observe the phenomenon of disease.
Textbooks and journals began to become more prominent[1].
Fig.4 Fig.5
[1] Claridge, J.; Fabian, T. History and Development of Evidence-based Medicine. World Journal of Surgery; 2005; 29, 547-553
Evolutionpast prese
nt
7
Introduction
Aims
Methods
Results
Discussion
Conclusion
References
Acknowledgments
The use of scientific methodology and statistical analysis in biomedical research combined as the roots of Evidence-Based Medicine.
Biostatistics is an applied science for biology, medical and health sciences and it plays an increasingly important role
in the production of medical knowledge and in the clinical practice [3].
Informatics explosion with online journals and large databases[1].
[1] Claridge, J.; Fabian, T. History and Development of Evidence-based Medicine. World Journal of Surgery; 2005; 29, 547-553[3] Sahai H, Ojeda MM. Teaching Biostatistics to Medical Students and Professionals: Problems and Solutions. Int J of Math Educ in Sci % Technol. 1999; 30(2):187–196.
Fig.6 Database scheme.
Evolutionpast prese
nt
8
Introduction
Aims
Methods
Results
Discussion
Conclusion
References
Acknowledgments
doctors must have a minimum grasp of biostatistic concepts to fully understand biomedical literature.
doctors need to stay up to date with the literature that drives clinical practice.
Informatics explosion with online journals and large databases[3]
[3] Sahai H, Ojeda MM. Teaching Biostatistics to Medical Students and Professionals: Problems and Solutions. Int J of Math Educ in Sci % Technol. 1999; 30(2):187–196.
9
Introduction
Aims
Methods
Results
Discussion
Conclusion
References
Acknowledgments
Accounting for this problems, many researchers have asked the question of whether or not
medical do possess said knowledge.
10
Introduction
Aims
Methods
Results
Discussion
Conclusion
References
Acknowledgments
Lack of knowledge to adequately interpret and evaluate results in medical research articles.
United States of America[5]
United Kingdom
Israel[4]
There was a seeming uniformity to the conclusions exposed on the reviewed literature.
[4] Cahan A, Gilon D, Manor O, Paltiel O. Probabilistic reasoning and clinical decision-making: do doctors overestimate diagnostic probabilities? QJM : monthly journal of the Association of Physicians. 2003;96(10):763-9. Epub 2003/09/23.[5] Windish DM, Huot SJ, Green ML. Medicine residents' understanding of the biostatistics and results in the medical literature. JAMA : the journal of the American Medical Association. 2007;298(9):1010-22. Epub 2007/09/06.
11
Introduction
Aims
Methods
Results
Discussion
Conclusion
References
Acknowledgments
Will the case in study reflect the same results?
Portugal
Given that no other literature was found regarding statistical knowledge of Portuguese physicians (either in general or in the Oporto region), our work was designed to analyze this parameter.
12
Introduction
Aims
Methods
Results
Discussion
Conclusion
References
Acknowledgments
Do doctors and medical students from Portugal have the necessary knowledge to adequately
interpret biostatistical information in scientific articles?
13
Aims
Introduction
Methods
Results
Discussion
Conclusion
References
Acknowledgments
To assess physicians and medical students’
knowledge of biostatistics
To compare participants confidence in evaluating
statistical information with their actual knowledge of
statistical concepts
To evaluate participants attitude towards
biostatistics
To assess the existence of correlations between
academic qualifications and participants biostatistical
knowledge
14
STUDY DESIGN
Methods
Introduction
Aims
Results
Discussion
Conclusion
References
Acknowledgments
Type of Study
Cross sectional study
Target population
Physicians and medical
students from a Portuguese University Hospital
Unit of analysis
The individual
15
Methods
Introduction
Aims
Results
Discussion
Conclusion
References
Acknowledgments
STUDY PARTICIPANTS
4 classes of the 5th year of MIMED
(“Mestrado Integrado em
Medicina” at FMUP) were randomly
selected out of a total of 8, using a random number
generator.
7 departments out of 43 at HSJ
(“Hospital de São João”) were
randomly selected. Only 5 colaborated.
16
Methods
Introduction
Aims
Results
Discussion
Conclusion
References
Acknowledgments
STUDY PARTICIPANTS - INCLUSION AND EXCLUSION CRITERIA
Students: - Participants must belong to Hospital's students
community;- Participants must be studying at the Hospital at the
moment - Participant must be matriculated at the 5th year;- Participants mustn't have participated in the pilot survey.
Physicians:- Participants must integrate the medical population of the
Hospital.
Inclusion criteria
17
Methods
Introduction
Aims
Results
Discussion
Conclusion
References
Acknowledgments
Students:
- Individuals studying at the hospital under mobility projects:
- The students studying abroad under mobility projects – http://erasmusfmup.blogspot.com
STUDY PARTICIPANTS - INCLUSION AND EXCLUSION CRITERIA
Exclusion criteria
18
Methods
Introduction
Aims
Results
Discussion
Conclusion
References
Acknowledgments
Pilot surveyDevelopme
nt of the questionna
ire
1 2
Modifications based on pilot survey’s results
3
Meeting with head
of departments and 5 th
year students
4
Questionnaire delivery
5
19
Personal contact with the departments and students
Data collection
Beginning of data insertion
Running of syntax
1st contact with the departments and students
(via e-mail)
Questionnaire delivery
CALENDAR
Methods
Introduction
Aims
Results
Discussion
Conclusion
References
Acknowledgments
20
Methods
Introduction
Aims
Results
Discussion
Conclusion
References
Acknowledgments
1st: Questions regarding attitude towards statistics.
2 nd: Confidence questions about interpreting and assessing statistical
concepts.
3 rd: Biostatistical knowledge test that assesses understanding of: • statistical methods; • study design;• interpretation of study results.
4th: Questions about the academic and professional formation of the
interrogated person.
DATA COLLECTION METHODS:QUESTIONNAIRE
21
Methods
Introduction
Aims
Results
Discussion
Conclusion
References
Acknowledgments
1st: 4 questions 2nd: 3 questions
DATA COLLECTION METHODS:QUESTIONNAIRE
22
Methods
Introduction
Aims
Results
Discussion
Conclusion
References
Acknowledgments
3rd: 9 questions 4th: 7 questions
DATA COLLECTION METHODS:QUESTIONNAIRE
23
Methods
Introduction
Aims
Results
Discussion
Conclusion
References
Acknowledgments
Attitudes will be rated on a 5-point Likert Scale;
Confidence questions will be rated using a 5-point scale;
Questions that aim to access participants knowledge of statistical methods will be classified as correct or incorrect. Missing values will be counted as incorrect responses.
VARIABLES DESCRIPTION
24
STATISTICAL ANALYSIS
Methods
Introduction
Aims
Results
Discussion
Conclusion
References
Acknowledgments
Data (from the questionnaire) was processed using Statistical Package for Social Sciences (SPSS)
Several paramaters were analysed for the general population and in separate for students and physicians
Attitudes and confidence were quantified in a 1 to 5 scale and 4 and 5 were considered “Agree” and “Confident”,
respectively. Analysis used frequency tables
25
STATISTICAL ANALYSIS
Methods
Introduction
Aims
Results
Discussion
Conclusion
References
Acknowledgments
The percentage of each group that answered correctly at each question was calculated
The mean of correct answers were compared between different demographic groups, using T-test (results were
considered statistically relevant for p < 0,05)
Tables were constructed in order to present the results of the analysis
26
Results
Introduction
Aims
Methods
Discussion
Conclusion
References
Acknowledgments
Table 1. Percentages of Correct Answers for the Knowledge-Based Questions
Doctors (n=35) Students (n=52)
Mean correct (%) Mean Correct (%)Interpreting Results in a Kaplan-
Meyer Analysis 17,1 5,8
Identifying a continuous variable 60 78,8
Identifying an ordinal variable 40 51,9
Identifying a nominative variable 51,4 69,2
Determining which test is most specific 42,9 67,3
Interpreting the meaning of P<0.05 2,9 9,6
Understanding the purpose of double-blindness 74,3 92,3
Interpreting standard deviation 31,4 46,2
Understanding the definion of bias 71,4 84,6
Recognizing a case and control study 48,6 28,8
Interpreting CI 95% and statistical significance 14,3 11,5
27
Results
Introduction
Aims
Methods
Discussion
Conclusion
References
Acknowledgments
Table 2. Characteristics of the Participants (percentage).
Student (n=52)
Doctor (n=35)
% %Total 59,8 40,2
Academic Degree
Student 100 ///Licenciate - 71,4
Master - 14,3Doctorate - 8,6
Is a specialised physician
54,3
Years since completing Licenciature
Less then one 77 0One to three 78,8 25,7Four to ten 86,5 25,7
Eleven to twenty 13,5 11,4More then twenty
one 0 31,4
Trained in Biostatistics 61,5 54,3
Trained in EBM (Evidence Based
Medicine) 46,2 31,4
Reads Scientific Literature frequently 50 85,7
Has had oportunity to work in research
projects 30,8 65,7
28
Results
Introduction
Aims
Methods
Discussion
Conclusion
References
Acknowledgments
Table 3. Atittude and confidence results by General, Physicians and Students' characteristics (percentage)
Doctors (n=35) %*
Students (n=52) %*
1 -"I have a desire to learn more about biostatistics." 85,7 50
2- "I can understand almost all statistical terms encountered in scientific literature." 17,1 34,6
3 -"Because it is easy to lie with statistics, I don't trust it." 17,1 1,9
4 -"I normally use statistics to form opinions or make clinical decisions." 57,1 55,8
5 -Interpreting a studies results with a statistical method 17,1 38,5
6 -Evaluating whether the correct statistical procedure was used in a certain study 2,9 11,5
7- Interpreting the value of P for a given result
25,7 59,6
29
Results
Introduction
Aims
Methods
Discussion
Conclusion
References
Acknowledgments
Table 4. Knowledge Scores by General, Physicians and Students' characteristics
Students (n=52) Doctors (n=35)
Characteristic Mean correct (%)
P value
*Mean
correct (%)P
value *
Has had previous training in biostatistics 52,8 0,133 45,5 0,573Has had previous training in evidence
based medicine 48,7 0,645 40,5 0,493
Regularly reads medical journals 49,3 0,875 45,2 0,207Has previous participated in scientific
research work 54,5 45,5 0,462
The amount of years passed since medical school graduation is... … <10 47,0
0,112 … >10 35,3Is a specialist 42,6 0,678
Total 49,7 41,3
30
Doctors (n=35) %*
Students (n=52) %*
1 -"I have a desire to learn more about biostatistics." 85,7 50
2- "I can understand almost all statistical terms encountered in scientific literature." 17,1 34,6
3 -"Because it is easy to lie with statistics, I don't trust it." 17,1 1,9
4 -"I normally use statistics to form opinions or make clinical decisions." 57,1 55,8
5 -Interpreting a studies results with a statistical method 17,1 38,5
6 -Evaluating whether the correct statistical procedure was used in a certain study 2,9 11,5
7- Interpreting the value of P for a given result
25,7 59,6
Discussion
Introduction
Aims
Methods
Results
Conclusion
References
Acknowledgments
Table 3. Atittude and confidence results by General, Physicians and Students' characteristics (percentage)
ATITTUDDE/ CONFIDENCE
31
Student (n=52) Doctor (n=35)
% %Total 59,8 40,2
Academic Degree
Student 100 ///Licenciate - 71,4
Master - 14,3Doctorate - 8,6
Is a specialised physician
54,3
Years since completing Licenciature
Less then one 77 0One to three 78,8 25,7Four to ten 86,5 25,7
Eleven to twenty 13,5 11,4
More then twenty one 0 31,4
Trained in Biostatistics 61,5 54,3
Trained in EBM (Evidence Based
Medicine) 46,2 31,4
Reads Scientific Literature frequently 50 85,7
Has had oportunity to work in research
projects 30,8 65,7
Discussion
Introduction
Aims
Methods
Results
Conclusion
References
Acknowledgments
Table 2. Characteristics of the Participants (percentage)
DEMOGRAPHICS
32
Doctors (n=35) Students (n=52)
Mean correct (%) Mean Correct (%)
Interpreting Results in a Kaplan-Meyer Analysis 17,1 5,8
Identifying a continuous variable 60 78,8
Identifying an ordinal variable 40 51,9
Identifying a nominative variable 51,4 69,2
Determining which test is most specific 42,9 67,3
Interpreting the meaning of P<0.05 2,9 9,6
Understanding the purpose of double-blindness 74,3 92,3
Interpreting standard deviation 31,4 46,2
Understanding the definion of bias 71,4 84,6
Recognizing a case and control study 48,6 28,8
Interpreting CI 95% and statistical significance 14,3 11,5
Discussion
Introduction
Aims
Methods
Results
Conclusion
References
Acknowledgments
Table 1. Percentages of Correct Answers for the Knowledge-Based Questions
TEST
33
Discussion
Introduction
Aims
Methods
Results
Conclusion
References
Acknowledgments
Limitation Explanation
Low answer rate (87 out of 208)
It caused serious problems in results’ interpretation because it was not possible to extract large
amounts of information observable by statistical data.
Difficulties in contact with the heads of department
Caused problems in collecting surveys of physicians.
Existence of some forced conclusions
With some lack of information, it becomes possible to have
conclusions that are not wrong but are less observable as expected.
STUDY LIMITATIONS
34
Most of physicians and students did not know how to interpret the P-value (93,1%), in spite of revealing some confidence in this matter(46%).
Though almost 2/3 of the inquired had reading habits, half of them lacked confidence in understanding the results of a statistical method used in a research.
64,4% of the general population showed interest in learning more about biostatistics.
Conclusion
Introduction
Aims
Methods
Results
Discussion
References
Acknowledgments
35
1. Claridge, J.; Fabian, T. History and Development of Evidence-based Medicine. World Journal of Surgery; 2005; 29, 547-553
2. Dahm, P.; Sur, R.: History of Evidence-based medicine. Indian Journal of Urology; 2011; 27; 487-489
3. Sahai H, Ojeda MM. Teaching Biostatistics to Medical Students and Professionals: Problems and Solutions. Int J of Math Educ in Sci % Technol. 1999; 30(2):187–196.
4. Cahan A, Gilon D, Manor O, Paltiel O. Probabilistic reasoning and clinical decision-making: do doctors overestimate diagnostic probabilities? QJM : monthly journal of the Association of Physicians. 2003;96(10):763-9. Epub 2003/09/23.
5. Windish DM, Huot SJ, Green ML. Medicine residents' understanding of the biostatistics and results in the medical literature. JAMA : the journal of the American Medical Association. 2007;298(9):1010-22. Epub 2007/09/06.
References
Introduction
Aims
Methods
Results
Discussion
Conclusion
Acknowledgments
36
Acknowledgments
Introduction
Aims
Methods
Results
Discussion
Conclusion
References
We gratefully thank our advisors Ricardo Santos and Alfredo Castro for their support that helped us completing the task we have been assigned. We also direct our acknowledgements to the head of departments and students that cooperated with us. And finally, a special gratitude to Prof. Altamiro Pereira for his constructive criticism.