who in vitro diagnostic prequalification and post …...who in vitro diagnostic prequalification and...

WHO in vitro diagnostic prequalification and post-market surveillance for new applicants: a detailed overview

Helena Ardura, Mercedes Perez and Anita SandsWHO PQ Team – Diagnostics AssessmentEssential Medicines and Health Products

1Copenhagen, Denmark 18-21 September 2017

The importance of IVDs


• Every year, hundreds of millions of US dollars’ worth of IVDs are purchased for distribution in resource-limited countries.

• IVDs play a critical role in ensuring blood safety, surveillance, diagnosis and treatment initiation and monitoring.

• IVDs save lives, reduce suffering and improve health, but only when they are of good quality, safe, effective, available, affordable, acceptable and properly used.

IVDs QA activities within WHO


• WHO has been assessing IVDs performance and operational characteristics since 1988 :

• HIV assays since 1988

• Hepatitis B assays since 2000

• Hepatitis C assays since 2000

• Syphilis assays since 2001

• Chagas assays since 2002

• Malaria assays since 2002

• CD4 technologies ad-hoc in 1996 & 2003

Trends in IVDs


• Globalised industry sectors with outsourced production

• Rapid emergence of new technologies • Increasing expectations on quality, safety and

performance• Increasing workload for regulators• Easy to operate tests/methods facilitate near

patient testing, hard-to-reach populations, non-lab environments

WHO's response: PQ in a regulatory framework


Pre-market Post-

market

Performance Quality Safety

Pre-2008

2008 - 2014

Post-2014

Changes

Performance Quality Safety

Reliance


• The aim of PQDx is to promote and facilitate access to safe, appropriate and affordable IVDs of good quality

• Focus is placed on IVDs for priority diseases and their suitability for use in resource-limited settings

PQDx: aim, scope and impactHIV

Malaria

Hepatitis C

Hepatitis B

HPV

G6PD

Cholera*

*As of January 2018


• The findings of PQDx generate independent technical information on safety, quality and performance of IVDs, principally used by other UN agencies, WHO Member States and other interested organizations.

• The PQDx status, in conjunction with other procurement criteria, is used by UN agencies, WHO Member States and other interested organizations to guide their procurement of IVDs.

PQDx: aim, scope and impactHIV

Malaria

Hepatitis C

Hepatitis B

HPV

G6PD

Cholera*

*As of January 2018

PQDx components

PQDx undertakes a comprehensive assessment of

individual IVDs through a standardized procedure aimed at

determining if the product meets WHO prequalification

requirements.

• The prequalification assessment process includes

three components:

Review of a product dossier

Performance evaluation

Manufacturing site(s)

inspection

Reference documents

PQDx is aligned with best international practice for IVDs

• ISO (and EN) standards

• Global Harmonization Task Force/International Medical

Device Regulators Forum (GHTF/IMDRF) guidance

• CLSI guidance

• Requirements of mature national regulatory authorities

including: FDA, EU, TGA, Health Canada, Japanese

Ministry of Health, Labour and Welfare

What does PQ do differently to regulators?

Requirements are based on the same set of standards – PQ is aligned with internationally accepted practice

BUT

PQ review is of aspects of particular relevance for resource-limited settings

WHO prequalification: Full assessment

Dossier review Site inspection

Dossier incomplete

Laboratory evaluation

Prequalification decision

Dossier complete

Dossier screening

Priority product

Yes

No

Pre-submission form

New

pathway

Maintenance of PQ status(PMS, changes, commitments, annual report, re-inspection)

Resources optimization: full Vs abridged assessment

Intention: harmonization initiatives, leveraging existing evidence and avoiding duplication of efforts:

• Comparison of requirements with existing mechanisms

• Benefit of existing evidence but independent decisions

abridged PQ assessment

Determining eligibility for abridged assessment

• Submission and review of WHO pre-submission form

• Evidence of prior regulatory review of the product or a

similar product manufactured by the same manufacturer

• Products will not be eligible for abridged assessment

based on anticipated stringent regulatory review

• Contact WHO to discuss before submitting

pre-submission form

Prequalification of IVDs Programme

Pre-submission form Pre-submission form

Dossier review Site inspection Laboratory evaluation

Dossier incompleteDossier incomplete


Dossier completeDossier complete

Dossier screeningDossier screening

Priority productPriority product

YesYes

NoNo

Pre-submission formPre-submission form

Abridged site

inspection



Yes Yes

Full PQ assessment

Full PQ assessment

NoNo

YesYes

Priority productPriority product

NoNo

Decision on Abridged PQ assessment

Decision on Abridged PQ assessment

Full prequalification assessment Abridged prequalification assessment

PQDx components

The prequalification assessment process includes 3 components:

Review of a product dossier


Manufacturing site(s) inspection

Purpose of the product dossier

• The dossier contains a subset of the technical

documentation held by the manufacturer • to demonstrate that the IVD to which it applies conforms to the

“Essential Principles of Safety and Performance of Medical Devices” as defined by GHTF

• The dossier reflects the status of the IVD at a

particular moment in time

Purpose of the dossier

• It should also provide sufficient information to inform the

PQ Inspection team regarding:• Sites responsible for design and manufacture to enable planning of

inspection/s

• Information regarding the maturity of the manufacturer’s QMS

• It should provide sufficient information to determine the

regulatory version submitted to PQ and to ensure the

data in the dossier is relevant to this version

• It should demonstrate that the manufacturer has

considered the safety and performance in WHO Member

States.

PQDx components


Manufacturing site(s) inspection

Requirements for Manufacturing site inspection

� Fully implemented quality management system (design &

development, manufacturing including quality control,

storage, distribution)

� Risk management to meet ISO 14971:2007

� Robustness of the Product

� Products undergoing prequalification have to be in routine

manufacturing

� Sufficient capacity to ensure reliable delivery

PQDx requirements prior to inspection

� Transfer from R&D to production completed

� Established and evaluated suppliers

� Validated processes (acceptance ranges determined, in-

process controls established)

� Trained personnel (requirements determined, training plan,

records)

� Established "out-of-specification" process

� Batch manufacturing records established (include all

manufacturing information, full traceability of material and

equipment)

Quality Management System

Fully implemented quality management system (design &

development, manufacturing including quality control,

storage, distribution)

• Meets ISO 13485:2003 requirements

• Competence of personnel

• Work environment

• Quality control processes, quality control plan

established

• Storage conditions, temperature and humidity,

components and kit, real time data required

Risk management

ISO 14971:2007

• Risk Management Plan – Policy

• Risk analysis

• Risk evaluation

• Risk control

• Evaluation of residual risk

• Risk management report

• Production and post production

information feedback

• Risk management file

• Evaluation of readiness for inspection (Stage 1)

• Desktop review of QMS documentation

• Stage 1 inspection (1 inspector day to inspect QMS

status, facility, staff competencies, critical suppliers incl.

outsourced activities, internal audit and management

commitment / review)

• Onsite Inspection (Stage 2)

• Follow up onsite inspection

• Re-Inspection (risk based, after 3-5 years)

Inspection


PQDx components




• Independent verification of the performance of IVDs submitted for prequalification assessment.

• Assays are challenged with a focus on their use in resource-limited settings and in the context of WHO guidelines (SRA review has different priorities based on local populations and product use)

• The dataset obtained complements the verification and validation data submitted by the manufacturer in the product dossier

• Currently takes place in a WHO Collaborating Centre (CC) and/or a site otherwise designated by WHO



Dossier review Site inspection

Dossier incomplete



Dossier complete

Dossier screening

Priority product

Yes

No

Pre-submission form

New

pathway

Performance evaluation pathways


Option 1: Performance evaluation coordinated by WHO at an

earlier stage of the prequalification process:

• The performance evaluation will be scheduled by

WHO as soon as the product is designated as

meeting WHO prioritization criteria.

Performance evaluation pathways


• Option 2: Performance evaluation commissioned by the manufacturer and carried out at a Prequalification Evaluating Laboratory listed by WHO

• The manufacturer selects a laboratory from the list of WHO Prequalification Evaluating Laboratories

• The manufacturer will bear the cost of the evaluation and be responsible for coordinating it directly with the laboratory

• The selected evaluating laboratory will inform PQ as soon as an evaluation has been commissioned by a manufacturer



Preparation

• Acceptance of Pre-submission form and prioritization

• Identification of WHO Collaborating Centres/designated site

• Protocol is sent to the designated site(s) and the manufacturer

• Ethical approval by the WHO Collaborating Centre

• Delivery of kits to the WHO Collaborating Centre

Testing

• WHO verify the Instructions for Use

• Manufacturer demonstration at the WHO Collaborating Centre/designated site

• Testing using approved protocol and manufacturer

• Draft laboratory evaluation report sent to WHO

Report

• WHO review the submitted draft report and submit to the manufacturer

• Manufacturer submit comments to WHO within 30 days

• WHO send the final report to the manufacturer

Evaluation of molecular technologies (NAT): collaboration with different partners


• HIV NAT qualitative (EID) and quantitative (VL) assays: Collaboration with ILB, CDC Atlanta:• In 2014, WHO and PEPFAR decided to align QA

mechanisms for procurement eligibility.

• Goal of alignment: to reduce duplication of effort for each organization and for the manufacturers, leverage each others resources for laboratory evaluation, and to create one list of "approved" products.

• Other partners involved in HPV and HCV evaluations


Performance evaluations on molecular technologies are divided into:

– Analytical performance studies: precision, LOD, carry-over, linearity,

subtype detection.

– Clinical performance studies: sensitivity, specificity, misclassification rate,

bias.

– Invalid rates

• Panels may be composed of retrospectively or prospectively collected

specimens

– Adhere to manufacturer’s instructions and claims

• Analytical and clinical performance studies may be conducted by different

sites

Molecular technologies


Performance evaluations on molecular technologies are divided into:

– Analytical performance studies: precision

– Clinical performance studies: classification at clinically relevant levels.

– Invalid rates

• Panels are prospectively collected as they require fresh whole blood

– Adhere to manufacturer’s instructions and claims

CD4 Technologies


• Serology evaluations are performed on archived panels from different geographical regions.

• Performance characteristics include:

• sensitivity and specificity

• limit of detection

• inter-reader variability (RDTs)

• seroconversion sensitivity

• antigen detection if applicable

• genotype detection

• lot-to-lot variation

Serology evaluations: HIV, HIV/TP, HBsAg, HCV

Operational Characteristics


Operational characteristics are assessed with a focus on the IVDs’ use in resource-limited settings. A standard appraisal sheet is used by technicians.

• Time to first result

• Infrastructure requirements (footprint of instrument, environmental requirements, electrical needs etc.)

• Waste disposal and biosafety

• Specimen and reagent storage and transport

• Materials required but not provided

• Level of skill required

• Calibration and maintenance needs

• Data management

• Instructions for use

• Etc.


36


Prequalification: decision


• Final prequalification outcome depends on: • Results of dossier assessment and acceptance of action plan

• Results of inspection(s) and acceptance of action plan

• No level 5 nonconformities outstanding for either dossier or for inspection

• Meeting the acceptance criteria for the laboratory evaluation

• WHO PQDx Public Report is posted on WHO website and product is added to the list of WHO prequalified products

• Product is then eligible for WHO and UN procurement

Post - Prequalification Activities


Maintenance of Prequalification Status


Manufacturers are required to notify WHO of planned changes made to any PQed product.

• Clear guidance available as to what changes must be reported

• Not all changes will be charged an assessment fee

• Early notification to PQ team advised

Changes to prequalified products


Every year, manufacturers are

required to report to WHO:

• Sales data

• Number of complaints

• Number of field safety

corrective actions

Annual reporting


• Manufacturers are required to report:• Any serious adverse event to the relevant

NRA within their respective timelines, and to WHO within 10 days.

• Any moderate adverse event or any change in the trend of mild adverse events to the relevant NRA within their respective timelines, and to WHO within 30 days.

• All complaints (both administrative and technical including serious, moderate and mild adverse events) to the relevant NRA, and to WHO as a periodic summary report, annually.

Post-Market Surveillance


• Any non/critical dossier or inspection “non-

conformities” must be resolved within the

agreed timelines

• Implementation will be reviewed at the next

inspection

Commitments to Prequalification


• Ongoing compliance with the requirements of

WHO PQ, including ISO 13485

• Timing will depend on

• Number and nature of non-conformities identified in

initial inspections/dossier

• Number of complaints

• Number and nature of changes

• Production figures

Re-inspection

Guidance for manufacturers

• WHO has placed an

increased emphasis

on the creation of

better guidance.

Guidance for manufacturers applying to PQDx

� Sample dossiers

� Technical Guidance Series

� Technical Specification Series

http://www.who.int/diagnostics_laboratory/g

uidance/en/

How to ensure your dossier meets PQ expectations


Sample dossiers

• Fictitious IVDs

• Provide examples of formatting and reporting details required

• Also examples of how to complete an Essential Principles checklist

• Good examples of risk assessment

Technical Guidance Series


• The Technical Guidance Series (WHO PQDx TGS) is produced for manufacturers interested in WHO prequalification of their IVD.

• The series is intended to help manufacturers in meeting prequalification requirements by making process more transparent and improving quality of submissions

Goal: to continue to create TGS in response to needs identified primarily in the dossier assessment phase of PQ.

Technical Guidance Series (TGS)


Provide detailed guidance on a specific aspect related to IVD performance.

• Currently published or in draft:• TGS 1 Standards applicable to the WHO Prequalification of

in vitro diagnostics• TGS 2 Establishing stability of an in vitro diagnostic for WHO

Prequalification• TGS 3 Principles of performance studies• TGS 4 Test method validation of in vitro diagnostic medical

devices• TGS 5 Panels for quality assurance and quality control of in

vitro diagnostic medical devices• TGS 6 Instructions for use

Technical Specifications Series (TSS)


• Summarize minimum performance requirements for WHO prequalification, to establish:

• appropriate performance evaluation and re-evaluation criteria,

• appropriate reference methods and reference materials.

• Specific requirements tailored to types of infections, conditions, etc.

• Clarify requirements for:• Manufacturers

• Assessors

Technical Specifications Series (TSS)


• Currently published or in preparation• TSS1: HIV RDTs (published)

• TSS2: G6PD RDTs (published)

• TSS3: Malaria RDTs (published)

• TSS4: HPV NATs (published)

• Planned• HIV NAT – quantitative & qualitative

• HCV NAT - quantitative & qualitative

• HBsAg, HBc RDTs and EIAs

• HCV RDTs and EIAs

• POC CD4

• HIV/Syphilis RDTs

• QC for RDTs

• Accessories

Q&A


WHO normative guidance on PMS

� Stakeholders’ roles/responsibilities

– Manufacturers

– End-users

– National regulatory authorities

– National reference laboratories

� Template forms

– IVD complaint report

– Manufacturer investigation report

– Field safety corrective action report

WHO post-market surveillance of IVDs

Reactive PMS

Evaluation of EQA/QC dataEvaluation of EQA/QC data

Pre-distribution Pre-distribution

Possible issuance of Field Safety Notice

Post-distribution Post-distribution

ComplaintComplaint

Possible Field Safety Corrective Action

Possible Field Safety Corrective Action

Lot verification testingLot verification testing

Proactive PMS

Any class of IVDAny class of IVD

Examples of complaints related to IVDs Serious – death or serious injury of patient, user or other person, false negative results

An individual is diagnosed as HIV+ after transfusion of a

blood product. The blood donation was screened as

negative by an HIV-1/2 RDT.

A HIV+ individual reporting for ART initiation was re-tested

to confirm their HIV diagnosis. The re-testing results are

HIV negative.

A healthcare worker operating a small instrument-based

test at POC couldn't remove a cartridge and inserted a

knife into the instrument and was electrocuted, resulting in

death.

Examples of complaints related to IVDs


Moderate - Anomalies that lead to a higher than expected invalid/error rate, unreturnable or inconclusive results, false positive results

The invalid rate for an RDT exceeds 5%, for any reason.

Higher than usual background which may or may not

obscure reading window and prevent reading of results.

Greater than expected discrepant rate between assay 1 and

assay 2, suspected reduced specificity of assay 2.

Mild - Deficiency found prior to use, expected and foreseeable side effects

The packaging of a single use device is labelled with the

caution 'do not use if the packaging is opened or damaged'.

Prior to use, obvious damage is observed and the device is

not used.

A component labelled as lyophilized is found to be fluid, this

is discovered by the user prior to use. Entire test kit must be

discarded

Desiccant has changed colour/density. Device is discarded

and a new device used.


WHO receipt of IVD complaints

• Complaints from manufacturer

– Any serious event must be reported

to WHO within 10 days

– Any moderate event or change in

trend of mild events must be

reported to WHO within 30 days

– All complaints must be reported

annually in summary form

• Complaints from end-users

– As soon as you become aware

Complaint notified using WHO IVD complaint form Complaint notified using WHO IVD complaint form

WHO reviews information contained in complaint form

WHO reviews information contained in complaint form

WHO forwards complaint form

to manufacturer, if received from

end-user

WHO forwards complaint form

to manufacturer, if received from

end-user

WHO opens complaint file, assigns reference number (COMP-xxxx)

WHO opens complaint file, assigns reference number (COMP-xxxx)

WHO’s expected response from manufacturer

• Root cause analysis

– How/why did this happen

• Analysis regarding related areas

– Is this same issue impacting/occurring elsewhere

• Correction with completion dates

– Fix now

• Corrective action, if applicable, with planned completion

dates

– To prevent recurrence

WHO review of manufacturer investigation

• WHO reviews investigation

report and supporting

documentary evidence within

7 days of receipt

• May request for clarification by

official letter

• Aim is to evaluate if complaint

handling procedure has been

implemented properly

Manufacturer submits initial investigation report

Manufacturer submits initial investigation report

WHO reviews investigation report and prepares letter with any requests

for clarification

WHO reviews investigation report and prepares letter with any requests

for clarification

WHO may request

additional information from

end-user

WHO may request

additional information from

end-user

WHO reviews follow-up investigation report to see if queries have been

answered

WHO reviews follow-up investigation report to see if queries have been

answered

Final investigation report is received at end of enquiry or when FSCA is

complete

Final investigation report is received at end of enquiry or when FSCA is

complete

WHO review of manufacturer investigation cont’d

• For each complaint, the following activities may be required

– Testing and inspection of retained samples from affected lots;

• WHO will review the testing panel used in the investigation, i.e. specific

investigative panel or usual QC lot release panel

• WHO will review acceptance criteria for lot testing conducted;

– Review of stability of product (all components) at different

temperatures or humidity (claimed shelf life, in-use stability,

shipping stability);

– Complete risk assessment report;

– Update risk management file.

• Risk management in the intended use setting should guide

the investigation

Root cause analysis

• RCA using methodology such as fishbone (Ishikawa)

diagram to determine possible causes for an observed

effect (i.e. the problem)

• 6 Ms

– Methods

– Machines (equipment)

– (Man) power

– Materials

– Measurement

– Mother nature

Analysis of related areas

• Why is this important?

• If the root cause is related

to the quality system, then

other products are likely to

be affected

• Case study

– High rate of invalids,

related to certain lots of

nitrocellulose

Correction (fix now)

• If deemed necessary, a correction

may be conducted before the root

cause is determined

• For example, certain lots are

conclusively known to be affected

and will give incorrect results,

decision to recall these lots is a

correction

Corrective action (prevent reoccurrence)

• Actions taken to prevent reoccurrence of problem

• Examples

– Change in standard operating procedure for manufacturing

– Change in composition of QC panel

• WHO will determine:

– If suggested corrective/preventive actions are acceptable;

– If suggested actions for customers are acceptable;

– If risk management update is acceptable.

WHO complaint handling since 2015 (n=67)

• In order of frequency

– False negative

results

– ↑ invalid rate

– False positive

results

– Defective reagent

Type of complaint

False negative

False negative andfalse positiveFalse positive

Falsification

Invalid rate

Mislabelled

Unreturnable result

Defective reagent

Weak reactivity

Software

Erroneous results

Reactive PMS: Field Safety Corrective Action

• A FSCA is triggered when there is an unacceptable increase in risk vs. benefit associated with use of the IVD

• FSCAs may include

• Changes to labelling/IFU

• Recalls/destruction

• Exchange (swap-out)

• End-users will be informed using a Field Safety Notice sent by the manufacturer, via their in-country distributor

Manufacturer submits draft FSN to WHO

Manufacturer submits draft FSN to WHO

WHO reviews in draft FSN and gives go-ahead for dissemination

WHO reviews in draft FSN and gives go-ahead for dissemination

WHO posts FSN on WHO website

for WHO prequalified IVDs

WHO posts FSN on WHO website

for WHO prequalified IVDs

Manufacturer submits FSCA report with subjective measure of FSCA

effectiveness

Manufacturer submits FSCA report with subjective measure of FSCA

effectiveness

Manufacturer sends FSN to all affected customers

Manufacturer sends FSN to all affected customers

Proactive post-market surveillance of IVDs

• Evaluation of EQAS and QC

testing results

– Across sites using the same

assay, same/different lot

• Lot verification testing

– Independent of the

manufacturer, NOT a

substitute for their lot release

testing

– Ensures that only lots

continue to meet established

criteria

– Uses a risk-based approach

Evaluation of EQA/QC dataEvaluation of EQA/QC data

Pre-distribution Pre-distribution

Possible issuance of Field Safety Notice

Post-distribution Post-distribution

Possible Field Safety Corrective ActionPossible Field Safety Corrective Action

Lot verification testingLot verification testing

Proactive PMS

Proactive PMS of IVDs: evaluation of EQA/QC data

• Review data from external

quality assessment schemes

(proficiency testing) and end-

user QC

• Greatest value when there are

many users are of the same

assay

• Useful to report differences in

performance such as this

example of a shift (lot to lot)

Proactive PMS of IVDs: lot testing

• Lot verification by suitably qualified laboratory using SOPs

so that each lot testing event is consistent

• Through physical inspection of packaging, labelling and

instructions for use

– Looking for breaches of packaging that might affect stability

Proactive PMS of IVDs: results of lot testing

• Lot testing panel characterized by an agreed reference

standard

– Positive and negative clinical specimens, diluted specimens to

test analytical sensitivity (LoD)

– Record invalid rates and other anomalies as these are

significant, e.g. incomplete clearing, high background, faint lines

• Lot acceptance criteria must be in place (pass/fail) for both

inspection and testing

Contact us

Contact us by email

[email protected]

By emailing [email protected]

http://www.who.int/diagnostics_laboratory/evaluations/en/

Sign up to our mailing list

Check our website

who in vitro diagnostic prequalification and post …...who in vitro diagnostic prequalification and...

Documents