Ola H. Elgaddar

MD, PhD, MBA, CPHQ

Lecturer of Chemical Pathology

Medical Research Institute

Alexandria University

Ola.elgaddar@alexu.edu.eg

Establishing the performance of a new

diagnostic tool such as an internally

developed or modified method.

Confirmation, through the provision of

objective evidence, that the requirements for

a specific intended use or application have

been fulfilled’ (doing correct test)......

ISO 9001:2005

A process to determine performance

characteristics before a test system is utilized

for patient testing.

Confirmation, through the provision of

objective evidence, that specified

requirements have been fulfilled’ (doing test

correctly)……ISO 9001:2005

Our laboratory performed Glucose

verification on the new closed chemistry

analyzer XYZ.

Precision study was performed following the

CLSI published protocol and the resulted

imprecision % was compared to Westgard

allowable imprecision and it was found to be

higher, accordingly, the lab decided to reject

the new method

It is impossible to discuss quality in medical

laboratories, without having analytical quality

specifications (analytical goals, or analytical

performance goals) set first.

These goals must be “fitting for purpose”,

which is attaining adequate patient care.

Drawing up specification

documents for new analytical

methodology or equipment

Assist the organizers of

EQAS to design and deliver

appropriate monitoring of

performance

To help the IVD industry in

designing products

To encourage laboratories to

decide which particular

examinations are less than

satisfactory and require

expenditure of scarce resources

on improvement

Fraser: The 1999

Stockholm Consensus

Conference on quality

specifications

DOI 10.1515/cclm-

2014-0914

The main aim of the organizers of the

conference was to provide a forum for

reaching consensus on the setting of global

analytical quality specifications in laboratory

medicine.

The following hierarchy of models should be

applied to set analytical quality specifications:

1. Evaluation of the effect of analytical performance

on clinical outcomes in specific clinical settings.

2. Evaluation of the effect of analytical performance

on clinical decisions in general:

a. Data based on components of biological

variation,

b. Data based on analysis of clinicians’ opinions.

3. Published professional recommendations:

a. From national and international expert bodies,

b. From expert local groups or individuals.

4. Performance goals set by:

a. Regulatory bodies,

b. Organizers of EQA schemes.

5. Goals based on the current state of the art:

a. As demonstrated by data from EQA or

Proficiency Testing schemes,

b. As found in current publications on

methodology.

Where available, and when appropriate for

the intended purpose, models higher in the

hierarchy are to be preferred to those at

lower levels.

The hierarchy was mainly used by EQAS

providers, but most of them use the state of art only

(Level 5)

Many professional bodies recommendations

(Level 3) are actually based on biological variations

(Level 2a), rather than on subjective opinions.

Most equipments and reagents manufactures do

not include clinically-based analytical quality

specifications for their products

Most regulatory authorities generally do not

require that analytical systems meet a priori set

specified quality requirements.

The primary aim was to revisit the

“Consensus Agreement” from the Stockholm

Conference investigating to what extent the

advocated hierarchy is still valid or if it

should be changed.

Sandberg et al.: Defining analytical performance specifications.

DOI 10.1515/cclm-2015-0067

Model 1: Based on the effect of analytical

performance on clinical outcomes:

1. Direct outcome studies – investigating the

impact of analytical performance of the test on

clinical outcomes

2. Indirect outcome studies – investigating the

impact of analytical performance of the test on

clinical classifications or decisions

Advantage: Most relevant to patients and society Disadvantages: Useful only when the link between the test and clinical decision making is strong

Analytical specifications derived in direct or indirect outcome studies will often be influenced by the current measurement quality and results may vary according to the actual test method used, the investigated population and healthcare settings

Model 2: Based on components of biological

variation of the measurand

This attempts to minimize the ratio of “analytical

noise” to the biological signal.

Advantage:

Can be applied to most measurands for which

population-based or subject-specific biological

variation data can be established.

Limitations:

There is a need to carefully assess the relevance

and validity of the published biological variation

data

Model 3: Based on state-of-the-art

This relates to the highest level of analytical

performance technically achievable.

If the best laboratories can only achieve a certain

quality and better quality is needed (according to

models 1 or 2), then improvements are required in

the technology.

If most laboratories can achieve a certain quality,

then laboratories not meeting this level may need to

change their practice.

Advantage:

State-of-the-art performance data are readily

available.

Limitations:

There may be no relationship between what is

technically achievable and what is needed to

minimize the ratio of ‘ analytical noise ’ to the

biological signal or needed to obtain an improved

clinical outcome.

- Not all tests will have the same model. - Even each test might have more than one model,

based on the “intended use of the test” Used in a POCT context, or in a lab? Screening versus confirmation tests? Used for monitoring, diagnosis or something else?

- Even the data obtained from EQAS has different sources, because the EQA providers use different specification!

For each measurand:

- Quality specification

- Intended uses

- Number of EQA materials

- Calculation models for each EQA

A simple model is needed!

(Nordin G, 15Th EFLM postgraduate course, Zagreb 2015)

On January 1,2015 an EFLM Task Force on

Performance Specifications in Laboratory Medicine

(TF-PS) was created to coordinate the activities of

the 5 Task & Finish Groups (TFG) established as

outcome of the 1st Strategic Conference held in

Milan at the end of 2014.

The TFG established under the TF-PS are:

1. TFG-DM “Allocation of laboratory tests to

different models for performance specifications”

2. TFG-PSEQA “Performance specifications for

EQAS”

3. TFG-TE “Total error”

4. TFG-PSEP “Performance specifications for the

extra-analytical phases”

5. TFG-BVD “Biological variation database”

-As Westgard says, the data base is only “hosted” by westgard website, but the calculations belong to Dr. Carmen Ricos.

-The database is regularly updated - There are some concerns about the most widely used data in our medical labs were: 27 analytes were driven from 10+ publications 129 analytes were driven from 2 - 9 publications 202 analytes were driven from only one publication Biological variation database: structure and criteria used for generation and update. Perich et al CCLM 2014

-Till the moment, we do not have settled quality

goals.

- Not one size fits all

- Even the regulatory authorities (Like FDA), do not

require a set of quality specifications

- Westgard website did not calculate the Biological

variation data base, Ricos did!