CRITICAL READING

ST HELIER VTS 2008RCGP Curriculum Core Statement Domain 3

AS

What is expected….

• Hierarchy of evidence• Statistical terms• Data Presentation• Seminal trials

What is expected….

• Mean, median, mode• Normal distribution curve, std deviation• ARR, RRR• NNT, NNH• P value, confidence intervals• Sensitivity, specificity• PPV, NPV

Critical Reading

• Critical Appraisal– Look at the specific objectives, methods, and

results of the study

• Critical reflection– Judge and discuss the implications for the world

outside the study

Critical Appraisal

• Title, author, institute, journal

• Ethics

• References

• Conflicts of interests

Critical Appraisal

• Introduction– Background– Aims– Relevance– Originality

• Methods– Design– Outcome measures– Subjects

Design

• Time Frame• Prospective• Cross-Sectional• Retrospective

• Observational / Interventional• Controlled / Uncontrolled• Randomised / Non-randomised

Bias

• Selection Bias– Typical sample?– Exclusions (numbers and reasons) clear?– Same conditions for both groups

• Measurement Bias– Observer error– Validity– Reliability

Bias

• Validity – closeness to true measure– Internal – Can I believe the results?– External – If I can, do they apply to the real world?

• Reliability – repeatability of the measurements

Implications

• Generalisation• Consequences

• PPeople and ethics• RResources• AAudit protocols and quality• TTime and training• SSafety and society

Critical Appraisal

• Presentation• Experimental Design• Numbers• Information Bias• Selection Bias

Summarising A Paper

• General aim – one sentence why the study was done

• Objective – what was the specific aim of the study?

• Design• Setting – where was the study done?

Summarising a Paper

• Population• Target• Sampling frame• Study population

• Methods• Intervention• Baseline• Outcome

• Results• Author’s conclusions – main points

Does treatment work?

• Consider the DCCT Trial

• Incidence of Neuropathy is 9.6% in control group (CER)

• Intervention group (6yrs very tight control) is 2.8% (EER)

Does treatment work?

• Relative risk reduction is:

9.6%-2.8% 9.6% equals 71%

• Absolute risk reduction is:

9.6% - 2.8% equals 6.8%

RRR

• Same result if 96% got neuropathy in the control group and 28% in intervention group

• Fails to show baseline risk or size of effect• Tiny effect can look impressive

NNT

• Inverse of ARR• 100/6.8%=14.7, i.e. treat 15 patients for 6

years to prevent one extra neuropathy

• Practical result• Easier to compare treatments• Beware intention to treat figures

Screening

• Sensitivity %age with +ve test result• High if few missed

• Specificity %age with true negative result• High if few false alarms

• Predictive value proportion of +ve results who have condition

• Depends on prevalence

Odds Ratios

• Ratio of events in intervention group v control group

• Forest plot• 1 means no effect• CI including 1 – no effect• Further away from 1, more likely a true effect

PPV and NPV

• Positive Predictive Value

• Ability of test to pick out disease

– Eg 160 positive results from a diseased population of 200:

160/200 = 80% PPV

• Negative Predictive Value

• Ability of test to pick out those who do not have disease– Eg 70 negative results

from a healthy population of 100:70/100 = 70% PPV

Qualititive studies

• Studying ideas and concepts• Narrative based medicine• Understanding patient• Turns anecdotes into data/observations

• Passive observation of behaviour• Participation Groups• One to One interviews• Focus Groups• Document Studies

Systematic Reviews

• Does the review examine an important clinical question

• Was there a substantial effort to search all relevant literature

• Was methodological quality assessed and trials weighted accordingly

• How sensitive are the results to the way the review has been done

• Have the results been interpreted with common sense and due regard to the broader issues

RCT

• Expensive• Time-consuming• Small numbers (usually)• Subjects and settings limit generalisation of

results• Random allocation cannot overcome bias –

Intention to treat, drop-out rates etc

Audit

• Systematic survey with a purpose that requires repeating

• More than a survey as part of process of change– Uncontrolled observational study that may

become an uncontrolled prospective interventional study

ASCOT

Summary of all end points

The area of the blue square is proportional to the amount of statistical information

Amlodipine perindopril better Atenolol thiazide better0.50 0.70 1.00 1.45

Primary

Non-fatal MI (incl silent) + fatal CHD

SecondaryNon-fatal MI (exc. Silent) +fatal CHDTotal coronary end pointTotal CV event and proceduresAll-cause mortalityCardiovascular mortalityFatal and non-fatal strokeFatal and non-fatal heart failure

Tertiary Silent MIUnstable anginaChronic stable anginaPeripheral arterial diseaseLife-threatening arrhythmiasNew-onset diabetes mellitusNew-onset renal impairment

Post hoc Primary end point + coronary revasc procsCV death + MI + stroke

2.00

Unadjusted Hazard ratio (95% CI)

0.90 (0.79-1.02)

0.87 (0.76-1.00)0.87 (0.79-0.96)0.84 (0.78-0.90)0.89 (0.81-0.99)0.76 (0.65-0.90)0.77 (0.66-0.89)0.84 (0.66-1.05)

1.27 (0.80-2.00)0.68 (0.51-0.92)0.98 (0.81-1.19)0.65 (0.52-0.81)1.07 (0.62-1.85)0.70 (0.63-.078)0.85 (0.75-0.97)

0.86 (0.77-0.96)0.84 (0.76-0.92)

Adverse events leading to treatment discontinuation

Adverse eventAmlodipine perindopril

(%)

Atenolol thiazide(%)

Total 2358 (24.5) 2402 (25.0)

Serious 162 (1.7) 254 (2.6)*

* p<0.0001