The Role of External Quality Assurance in Improving Patient Safety

September 2018

Tony Badrick

Pathology services: the critical infrastructureof modern healthcare

SA power outage: How did it happen?

IT failure sees operations postponed amid backlog of 10,000 blood tests

Leeds still working to recover from pathology IT crash

Pathology IT crash in Leeds drags on

Leeds pathology IT crash into third week without resolution

Essential service

Headlines

FOX NEWS Health

Mistakes at US lab force hundreds of Zikatests to be repeated

Published February 17, 2017

Reuters

Colon cancer screening test suspended due to manufacturing defect

GPs asked not to request pathology testing following NHS hack16 May 2017 By Sofia LindGPs in East London have been told not to send any routine samples through to pathology at Barts Health NHS Trust, after it was badly affected by Friday's NHS cyber attack.

WALL STREET JOURNALTheranos Slashes Staff, Voids More Test ResultsEmbattled blood-testing company laid off about 150 employees Friday amid rising legal costs from shareholder lawsuits and civil and criminal probes

Error and blame

Maufacturing model

Approaches to improving patient safety

EQA• Characterise test bias and imprecision across multiple methods

• Identify interfering substances and quantify their effects across multiple methods

• Provide clinical laboratories with reliable information for replacing unsatisfactory methods

• Identify clinical laboratories that are at risk for poor performance

• Satisfy accreditation and regulatory requirements

• Assessment of method robustness to clinically relevant interference

• Assessment of individual laboratory performance

• Audit of wider aspects of analytical performance and educational activities.

Chem Path Workshop2018

Position of EQA in the LaboratoryLaboratory Quality Management System

eg ISO 15189

Quality Policy

Quality Assurance

Quality Control

Internal Quality Control

External QualityAssurance

Other aspects of good laboratory practice

All measures taken to assure quality

•Sturgeon C. Best Pract & Res Clin Endo & Metab 2013; 27: 802-822

Combining Patient and Control QC• Patient-based QC should not replace control sample QC, but should supplement it.

• How to best combine patient-based QC and control sample QC?

−Run patient QC in parallel with routine control QC.−Use patient QC as a “trigger” for control QC testing.−Do both.

Curtis Parvin: AACC Short Course, Monday July 28, 2014

13

14

15

EQA driving Improvement

Improvements in performance in medical diagnostics tests documented by inter-laboratory comparison programsTholen DW (2002) Accreditation and Quality Assurance Vol 7, Issue 4:146-152

A survey of clinical genetic laboratories in the United States indicated that increased participation in PT correlated directly with fewer PT failures and fewer incorrect patient test reportsHudson KL, Murphy JA, Kaufman DJ et al (2006) Oversight of US genetic testing laboratories Nat Biotechnol. 24(9):1083–1090.

There is evidence that EQA failure rates decrease withincreased experience performing EQA.

Chem Path Workshop2018

Chem Path Workshop2018

Chem Path Workshop2018

Key Requirements of an EQA• Specimen distribution frequency and reporting requirements

• Specimen material distributed

• Definition of target values

• Assessment of overall performance

• Assessment of individual method performance

Key Requirements of an EQA• Assessment of method robustness to clinically relevant interference

• Assessment of individual laboratory performance

• Communication with participating laboratories

• Audit of wider aspects of analytical performance and educational activities

Kalra clin biochem 37;1052-62: 2004

Reference limits and flaggingabnormal results

Date of Birth

15%

48%

5%

32%

0% 20% 40% 60%

dd/mm/yy

dd/mm/yyyy

dd/month/yy

dd/month/yyyy 01 April 1980

01 April 80

01/04/1980

01/04/80

Flagging abnormal results

42%

12%

2%

5%

5%

12%

2%

5%

2%

5%

9%

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

Bold Result, space, Bold H/L

Bold result, no space, Bold H/L

Bold result, * before result

Bold result, space * after result

Bold result, space, H/L (not bold)

Bold result

H/L before result, space, no bold

Bold result, no space, Bold +/-

Bold result, no space, Bold H/L, Underlined

H/L not bold

* before analyte name, Bold result

Corrected Calcium

6%31%

3%3%3%

17%3%

9%3%3%3%3%3%

6%6%

0% 5% 10% 15% 20% 25% 30% 35%

Ca (Corr)

Adj. Ca.

CorrectedCa

Calcium (Alb. Corr.)

Calcium (corr. Alb.)

CorCalcium

Corr. Calcium

Ca Alb Corr

11%

28%

6%

6%

6%

6%

6%

17%

17%

0% 5% 10% 15% 20% 25% 30%

TP, Alb, Est Glob, Bili T, Bili C, AST, ALT, GGT, ALP

TP, Alb, Bili T, Bili C, ALP, AST, ALT, GGT

TP, Alb, Glob, Bili T, Bili C, ALP, GGT, ALT, AST

TP, Alb, Glob, Bili T, Bili C, Bili unconj, ALP, ALT, AST, GGT,Amylase, Lipase, LDH

Bili T, Bili C, ALT, AST, LDH, ALKP, GGT, TP, Alb, Glob

Bili, ALP, ALT, GGT, TP, Alb, Glob

Bili T, Bili C, TP, Alb, GGT, ALT, AST, ALP

Bili T, ALP, GGT, ALT, Alb, TP

TP, Alb, Glob, Bili, Bili C, ALP, ALT, GGT

Variations on the LFT

4%

4%

13%

13%

13%

4%

4%

4%

4%

4%

4%

4%

4%

4%

4%

4%

4%

0% 2% 4% 6% 8% 10% 12% 14%

Chol, Trig, LDL-Chol, HDL-Chol, Total/HDL Ratio

Chol, Trig, HDL Chol, nonHDL chol, LDL Chol,…

Chol, Trig, HDL Chol, LDL Chol

Chol, Trig, LDL Chol, HDL Chol, Chol/HDL Ratio

Chol, Trig, HDL-C, LDL-C, Ratio,

LDL, Chol/HDL

Chol, Trig, LDL Chol (calc), HDL Chol

Chol, HDL, Trig, LDL, Chol/HDL Ratio, CR Index

Chol, Trig, HDL Chol, LDL (calc)

HDL Chol, Chol, Trig, LDL Chol, Non-HDL Chol

Chol, Trig, HDL Chol, LDL Chol (calc), Non-HDL Chol,…

LDL Chol, Non HDL Chol, Trig

Chol HDL, Chol LDL, Trig

T Chol , LDL Chol, HDL Chol, Trig

Chol, Trig, HDL, Chol/HDL Ratio, LDL, Non HDLC

Chol, Trig, HDLC, LDL Chol, TC/HDLC, LDL/HDL

Chol, Trig, HDL Chol, Chol/HDL, Non HDL-Chol, LDL Chol

Variations on the lipids

PITUS- AustraliaA national project for the Standardisation of Pathology Units and Terminology (PUTS) was initiated by the RCPA in July 2011 with support from Government. Around 80 pathologists, scientists, informaticians, and other clinicians worked in 8 working groups to establish guidelines for the use of terminology and standardised units covering each of the pathology disciplines.

Informatics EQA• EQA is usually an artificial process−No normal request−Sample added −Report not generated

• Should test the entire cycle−Request−Result−Report

IT EQA

IEQA Result

43

Chem Path Workshop2018

Chem Path Workshop2018

Interpretative Comments•Commenting on Results of EQA

•Often occurs as part of an normal EQA−Anatomical Pathology−Transfusion−Haematology

•Model for all Reported Comments?

Interpretative CommentsThe ideal interpretive comment:

(i) describes the abnormalities in the technical data,

(ii) interprets that information including the clinical implications such as for diagnosis, and

(iii) provides knowledge for follow-up including further testing or specialist referral.

EQA for Referring Doctors

“Measurement is the first step that leads to control and eventually to improvement. If you can’t measure something, you can’t understand it. If you can’t understand it, you can’t control it. If you can’t control it, you can’t improve it.”

• Clinical handover is an area of high patient risk that is recognized by ACSQHC. The RCPAQAP KIMMS group ran a survey to investigate the current state of play with regards to finding, communicating and recording High Risk (HR) results in Australian pathology laboratories. A survey was distributed to 71 Australian laboratories. All of these representative laboratories participate in RCPAQAP programs – Clinical Pathology, Haematology or Microbiology.

• The survey asked about the demographics of the laboratory, what their critical results process is, how this is monitored and finally, the actual number and type of critical results obtained on a typical day.

• Definition: A HR result is one deemed to be of high enough risk to patient health that is should be immediately notified by phone. They are also known as critical results.

High Risk Results

Notification timeOnly 119 of the 333 results had the time taken to notify the referrer recorded. As this is likely to be the major KPI for notification of HR results, this is a poor outcome. It is unknown whether this information is not kept or is too difficult to extract from laboratory’s LIS.

DiscussionAs expected, most HR results are from inpatients and emergency departments, however a significant number come from non-hospital situations (43%). A third of respondents still find communication of HR results a problem, and a third do not see this issue as a “a clinical transfer”. There is no common practice of when a pathologist should be called into an escalation procedure, and in many cases they are not involved. More than a quarter of laboratories rely on staff to recognize a HR result i.e. with no input from the LIS system.

ConclusionThere are many different practices in the management of HR results in Australia. Best practice has been outlined in the document Consensus Statement for the Management of Laboratory Results that Pose High Risk to Patients and Require Timely Communication. The results of this survey will be discussed in conjunction with the above statement at the next KIMMS workshop.

High Risk Results

The most frequently monitored and highest risk lists have 6 monitors in common (KIMMS).

Their contributions to incident detection frequencies and total risk are as follows:

(i) Haemolyzed samples; they contribute more than a fifth to total error frequency and more than a quarter to total risk

(ii) Retracted reports contribute, as above, 3% to total frequency, 11% to total risk

(iii) Specimens not collected are second in frequency (18%) and sixth in risk (5%)

(iv) Registration errors add 8% to total incident frequency and 9% to total risk

(v) Contributing similar percentages to total error frequency and total risk is mismatched or discrepant labeling: 5% of incident frequency and 6% of risk.

(vi) Incorrect fill (improper sample volume) is eighth in frequency (accounting for 4% of total incidents) and seventh in risk (contributing a similar 4% to total risk)

What’s to be done next? Clinical laboratory stewardship, concentration on fewer monitors, and integration of risk

calculation to meet the challenge of economic value.

1. Pursue clinical laboratory stewardship:

2. Concentrate on fewer monitors:

3. Calculate risk:

4. Meet the challenge of demonstrating economic value:

5. Value:

6. Diagnostic completeness:

7. Negative values:

8. The negative values vary from one domain to another:

9. The bottom line:.Badrick KIMMS58

Pursue clinical laboratory stewardship:

The imperative here is to extend quality measurement to steps in the total testing process that laboratorians do not control, antecedent to test ordering and subsequent to result reporting.

Clinical laboratory stewardship’s collaboration with clinician test orderers and clinical consumers of result report information is essential not only to develop relevant, measurable indices at both ends of the total testing process, but also to act on the evidence that they generate.

.Badrick KIMMS59

Concentrate on fewer monitors:

The imperatives here are simultaneously to distribute appropriately a suite of a limited number of monitors over the entire sequence of process steps and to recognize the particular importance of specimen quality.

.Badrick KIMMS60

Calculate risk:

The imperative here is to connect process defects with potential bad outcomes in a more explicit way.

• FMEA introduces calculated risks of patient harm-outcomes by defining them as measured error-event frequencies multiplied by error-harm estimates, and projections of detection difficulties.

• Harm estimates often have limited exportability from one medical setting to another however, KIMMS has accumulated experience that has focused attention and prevention on two kinds of process-events:

a) events whose high frequencies contribute substantially to total risk and

b) those events whose frequencies are low, but whose high-harm estimates gain them a place in the “top ten” of clinical laboratory testing risks.

.Badrick KIMMS61

Chem Path Workshop2018

Peer Group

Capability <4

Peer Group

Capability 4-6

Peer Group

Capability > 6

Our

Laboratory

Capability

>6

Better than peer-

group cell

Better than peer-

group cell

As good as peer-

group cell

Our

Laboratory

Capability

4-6

Better than peer-

group cell

As good as peer-

group cell

Worse than peer-

group cell

Our

Laboratory

Capability

<4

As good as peer-

group cell

Worse than peer-

group cell

Worse than peer-

group cell

Value:Epner defines value as the remainder when harms and missed opportunities from testing are subtracted from its benefits. He proposes, instead of a calculation of laboratory testing’s quality, a calculation of testing’s value, defined as test results’ variable worth, as a commodity, to the 5 sorts of consumers.

Epner proposes three more global measures of laboratory testing’s positive economic value: − broadening test-process turnaround time to time–to-diagnosis, − broadening result report accuracy to diagnostic accuracy,− and adding a new intermediate outcome—diagnostic completeness.

.Badrick KIMMS65

Diagnostic completeness:

The new index is composed of a set of standards determined from consumers’ standpoints. These consumer-set standards define the economic demands of consumers of laboratory information in the five domains: (i) care teams of clinicians, (ii) administrators of health systems in which clinicians labour, (iii) patients to whom health systems provide services, (iv) governmental regulators and insurers paying for patient services delivered by systems, and (v) researchers who develop diagnostic and management techniques for the other four types of consumers.

.Badrick KIMMS66

The bottom line:

Pursuit of economic value shapes the present and will drive the future of clinical laboratory testing, as efforts to increase performance quality, measured by QIs, contributed to shaping the period 1990-2010.

In this current and future economic context, however, rather than competing with economic value, the triple strategy of:

(i) clinical laboratory stewardship,

(ii) concentration on a suite of relatively few quality indicators, and

(iii) integration of risk calculations contributes to economic value

.Badrick KIMMS67

Questions

Done