how to evaluate an article

8/6/2019 How to Evaluate an Article

1/38

Dr. Noha A. DashashMBBS, DPHC, ABFM, SBFMConsultant Family Physician,Director of PHC, Jeddah GovernorateMember of the National EBM CommitteeSupervisor, Jeddah EBM Working Group

Dr. Noha A. DashashMBBS, DPHC, ABFM, SBFMConsultant Family Physician,Director of PHC, Jeddah GovernorateMember of the National EBM CommitteeSupervisor, Jeddah EBM Working Group


2/38

29 May 2011 EBM Jeddah Working Group 2

In The Name Of AllahThe Most Merciful

The Most Compassionate


3/38


Components of an articleComponents of an article

Title

Abstract Introduction-background

Methods (Materials & Methods)

Results Discussion- Conclusion


4/38


IMRADIMRAD

Introduction

why

Methodology how

Results and

What they found Discussion

What they THINK the results mean


5/38


The science of trashing aThe science of trashing a

paperpaper Does it answer your question?

Title

Abstract

Introduction

AIM, Objectives

Study Design: Is it suitable for answering your question?


6/38


The science of trashing aThe science of trashing a

paperpaper

METHODS

If methodology is not sound

trash the paper before reading the

results even if statistically

significant


7/38


Critical AppraisalCritical Appraisal

critique

1. A critical review or commentary,

especially one dealing with worksof art or literature.

2. A critical discussion of a

specified topic.3. The art of criticism.

The Free dictionary, by Farlex


8/38



critique

review, oranalyze

a serious examination andjudgment of something;

"constructive criticism is alwaysappreciated"

The Free dictionary, by Farlex


9/38



Appraisal: the classification of

someone or something with

respect to its worth


10/38



Critical appraisal: an appraisal

based on careful analytical

evaluation


11/38


(critically appraise) evaluation for

validity and usefulness


12/38



13/38


Measures of StudyMeasures of StudyReliabilityReliability

B ias, I nternal validity,

C onfounding,

E xternal validity,

P ower


14/38


BiasBias

A systematic tendency to produce anoutcome that differs from the

underlying truth

factorarisingfrom Design

Conductofa study

skewsthe data in one particulardirection,eitherawayfromortowardsthetrue

population valuethatis beingestimatedwiththestudy


15/38


BiasBias

Selection bias

example, we select sicker patients to

receive active treatment and fitter

patients to receive inactive treatment

How can it be overcome? Randomization CONCEALMENT


16/38


BiasBias

Observer bias

example, we know that the patient

had active treatment so wesubconsciously encourage her to rateher quality of life as higher than itreally is.

How can it be overcome? Blinding


17/38


BiasBias

Participant bias

example, in a study of aspirin versus

no treatment, people allocated to notreatment take aspirin anyway

Contamination


18/38


BiasBias

Withdrawal bias ordrop out bias

when we lose people to follow up,

those that remain for analysis at theend of the study may not be

representative of the group originally

included at the start of the study.


19/38


BiasBias

Recall bias

example, mothers of children withleukemia may remember living nearhigh voltage power cables becausethey fear a link between power linesand cancer, while mothers of childrenwithout leukemia are likely to forget

whether they lived near a power line,because they regard it as a trivial fact.


20/38


BiasBias

Instrument or measurement bias

E.g. blood pressure measurement


21/38


BiasBias Publication bias (in Systematic reviews/MA)

results from researchers and journalsbeing biased towards publishing onlypositive results

If negative or equivocal results remainhidden, then clinicians can get amisleadingly excessive view of howeffective a treatment really is (or of how

harmful an exposure might be).

authors of good systematic reviewsmake a point of tracking downunpublished research.


22/38


WorkWork--up Biasup Bias (Verification Bias)(Verification Bias)

In Studies of Diagnostic Tests:

Subjects with a negative test are sometimes not

referred to the 'gold standard' assessment.

In some cases, the investigators may refer a small

random sample of subjects with a negative test to

the 'gold standard' assessment in order to

determine if they really do not have the disease.

'gold standard' results are not available on anentire population,

work-up bias,

equations for sensitivity and specificity give biased

results.


23/38


Internal validity ofInternal validity ofmeasurementmeasurement

how well a study was run (researchdesign, operational definitions used, howvariables were measured, what

was/wasn't measured

Wasitreallythetreatmentthatcausedthe difference between thesubjectsin

thecontrol andexperimental groups?"In descriptive studies (correlational, etc.)

internalvalidityrefersonlytotheaccuracy/qualityofthestudy


24/38


results of study are due to the unusual typeof people who were in the study.

Or, it might only work because of the unusualplace you did the study in

(e.g. you did your educational study in a collegetown with lots of high-achieving educationally-oriented kids).

Or, study done in a peculiar time. For instance, smoking cessation study done the

week after the General Surgeon issues the well-publicized results of the latest smoking and cancerstudies,

you might get different results than if you had doneit the week before.


25/38


Internal validity ofInternal validity ofmeasurementmeasurement

1. How good (VALIDITY of) ourassessment tools are for measuring

the things we want to measure in thestudy (outcomes)

Quality of life: asking people how are

you on a scale of 1 to 10?

or

Using a validated Questionnaire(e.g.QALY)


26/38


Internal validity ofInternal validity ofmeasurementmeasurement Cont.

Glucose control assessment

monthly urine dipstick test?

Or

glycosylated hemoglobin


27/38



Smoking

Asking people how much do you smoke?

Or

asking people to keep their empty

cigarette packets for measuring tobacco

exposure.

Whichhasmoreinternalvalidity


28/38



2. Proxy or intermediate endpoints For example a study may suggest that a treatment

reduces heart disease, when the only outcome

assessed was blood cholesterol.

The internal validity of cholesterol measurements for

measuring heart disease is poor


29/38


ConfoundingConfounding

is an exposure that the

researcherdid notmeasure that

independently affectstheoutcomes of the study andwhich

is associated with the exposure


30/38


ConfoundingConfounding

Example: study of the relation between lung

cancer and coffee drinking

association is found between the two.

cause coffee drinkers are more likely to

smoke

real risk factor for lung cancer

smoking. In this study, the left-out variable ofsmoking

confounds the relationship between coffee

and lung cancer


31/38


External validityExternal validity

External validity is related to

generalizing


32/38


Threats to External ValidityThreats to External Validity


33/38


Threats to External ValidityThreats to External Validity

if you did your smoking cessation study the

week after the Surgeon General issues the

well-publicized results of the latest smoking

and cancer studies

type of people who were in the study


34/38


PowerPower

the ability of the study to detect

an effect if in truth there is an

effect (ie were we to perform an ideal studyof the entire population). We can

increase the power of the study

by:


35/38


We can increase the power ofWe can increase the power ofthe study by:the study by:

(a) increasing the number of

outcome events we observe

by increasing the sample size

or duration of the study


36/38


We can increase the power ofWe can increase the power ofthe study by:the study by:

(b) improving the precision of our

measurements

by increasing the sample size

or duration of the study


37/38


In a NutshellIn a Nutshell


38/38

THANK YOUTHANK YOU

how to evaluate an article

Documents