of diagnostic tests in the aquatic manual - home: oieoie.int/eng/a_aahrwf2015/presentations/3.4....

Validation of diagnostic tests in the Aquatic Manual

AAHL FISH DISEASES LABORATORY

Moody NJG and Crane MStJ

3rd OIE Global Conference on Aquatic Animal Health “Riding the wave to the future”

Why validate methods?

OIE validation guidelines

AFDL – AbHV and WSSV

Conclusions and Recommendations

Nick Moody | Validation of diagnostic tests in the Aquatic Manual

Why validate methods?



One of the roles of the OIE• To safeguard world trade by publishing health standards for international trade

in animals and animal products

• For OIE‐listed aquatic animal diseases, these health standards are published in:• OIE Aquatic Animal Health Code (referred to as the Aquatic Code) (1) • OIE Manual of Diagnostic Tests for Aquatic Animals (referred to as the Aquatic Manual) (2).

• The Aquatic Code describes the standards required to improve aquatic animal health and welfare and develop animal health measures to facilitate safe international trade in aquatic animals and animal products. They are used by competent authorities to provide early detection, reporting and control measures for OIE‐listed diseases

• The implementation of the standards described in the Aquatic Code are supported by general and disease‐specific technical information contained in the Aquatic Manual


The OIE and the WTO• The OIE is formally recognised by the World Trade Organisation (WTO) as the

organisation responsible for animal health standards regarding the application of the Agreement on the Application of Sanitary and Phytosanitary Measures (SPS Agreement)

• The SPS Agreement requires WTO members to provide scientific justification for health requirements for the import and export of animals and animal products.

• While still permitting countries to implement their own standards to provide an acceptable level of protection, the SPS Agreement also specifies that the measures implemented should not be unnecessary, arbitrary, scientifically unjustifiable, or disguised restrictions on international trade

• The Aquatic Code and Aquatic Manual, as international standards, are recognised by the WTO as fundamental references for aquatic animal health management and are key components of the WTO framework for international trade.


What is validation?• Validation is the process that determines the fitness of an assay, which has been

properly developed, optimised and standardised, for a specific and defined diagnostic purpose (OIE)

• Important that this validation is documented

• Moving towards molecular assays for surveillance as well as diagnosis

Why validate methods?• Resolutions of the recent Third Global Conference of OIE Reference Centres

(Incheon, South Korea, October 2014);• Apply OIE standards and guidelines for the validation of diagnostic tests so as to ensure

confidence and reliability of diagnostic tests in the improvement of disease control programmes

• Achieve or maintain accreditation to the ISO 17025 or equivalent quality management system in diagnostic laboratories.

• Assay validation is a core requirement of a laboratory quality management system based on ISO 17025

• Section 5.4.5.1: “Provision of objective evidence.….”

• OIE Reference Centres should therefore be using and promoting assays validated according to the OIE validation guidelines

• However, the process to attain assay validation is not a simple task


OIE validation guidelines


OIE Validation Guidelines• Principles and methods of validation of diagnostic assays for infectious diseases


Same chapter in both manuals because principles and methods are the same

• Chapter 1.1.5 – Terrestrial Manual

• Chapter 1.1.2 – Aquatic Manualhttp://www.oie.int/fileadmin/Home/eng/Health_standards/aahm/current/1.1.02_VALIDATION.pdf

Assay development and validation pathway


http://www.oie.int/fileadmin/Home/eng/Health_standards/tahm/1.01.05_VALIDATION.pdf

• Assay validation criteria are in bold shadowed boxes

OIE Validation Guidelines• Aquatic Manual [http://www.oie.int/en/international‐standard‐setting/aquatic‐manual/access‐online]

• Chapter 1.1.2: Principles and methods of validation of diagnostic assays for infectious diseases

• Terrestrial Manual [http://www.oie.int/en/international‐standard‐setting/terrestrial‐manual/access‐online/]

• Chapter 1.1.5: Principles and methods of validation of diagnostic assays for infectious diseases

• Guideline 3.6: OIE Validation Guidelines• Guideline 3.6.1: Development and optimisation of antibody detection assays• Guideline 3.6.2: Development and optimisation of antigen detection assays• Guideline 3.6.3: Development and optimisation of nucleic acid detection assays• Guideline 3.6.5: Statistical approaches to validation• Guideline 3.6.6: Selection and use of reference samples and panels• Guideline 3.6.7: Principles and methods for the validation of diagnostic tests for infectious

diseases applicable to wildlife

• Guideline 3.6.8: Comparability of assays after minor changes in a validated method (under study)


Two examples from AFDL

AbHV (enzootic)WSSV (exotic)

for surveillance



Abalone herpesvirus ‐ AbHV• AFDL involvement was to develop detection capability and understand the

disease (awareness of validation requirements but not planned)

• Late 2005: clinical disease in abalone in Victoria: VIC‐1• Mid 2008: clinical disease in abalone in Tasmania: TAS‐1

• >2000 surveillance samples from wild abalone sent to AFDL – all negative

• 2009–2010: clinical disease in abalone in Tasmania: TAS‐2• Not all lesion‐positive samples were positive by current qPCR (2 more assays available)• Multiple assays and confirmation by sequencing• >1000 surveillance samples from wild abalone sent to AFDL – all negative

• 2010–2011: clinical disease in abalone in Tasmania: TAS‐3 and TAS‐4• >1000 surveillance samples from wild abalone sent to AFDL – all negative

• 2011: clinical disease in abalone in Tasmania and NSW: TAS‐5

• Assays developed for diagnostic testing, now being used for surveillance


AbHV

VIC‐1 TAS‐1 TAS‐2 TAIWAN TAS‐3 TAS‐4 TAS‐5

DATE 2005 200820092011

200320102011

2011 2011

Location Victoria Tasmania Tasmania Taiwan Tasmania Tasmania Tasmania

Real‐time PCR Detection details

ORF49 + + ‐ + ‐ + +

ORF66 + + + + + + +

ORF77 + + + + + + +

Conventional PCR Detection Details

AB1213 + + + + + + +

AB1617 + + + + + + +

From 2009 onwards – all samples tested with 3 real‐time assays• Charged commercial rates for testing

• For translocation and management plans Australia governments and industry wanted to know DSe

• Funded research project to determine this


AbHV• 1400 archival samples*:

• Tested by 3 real‐time assays (ORF49, ORF66 and ORF77 qPCRs)• Data subjected to Bayesian (Latent Class Model) analysis• For two assays in parallel for surveillance testing for all known variants

DSe of 86% and DSp of 98%

• As part of a different project using quantified plasmid positive controls:• ASe ORF49 qPCR: 20‐200 copies• ASe ORF66 qPCR: 2‐20 copies• ASe ORF77 qPCR: 2‐20 copies

• Ongoing work over the past 9 years, with significant external funding• Over 7000 samples tested, technology transfer, proficiency testing

• To be published so the information is available in the public domain and added to Chapter 2.4.1 “Infection with abalone herpesvirus” of the Aquatic Manual

*Caraguel CGB, Ellard KA, Crane MStJ, Williams LM, Mohr PG, Hoad J and Moody NJG (in preparation) Evaluation of the diagnostic accuracy of qPCR primer sets to demonstrate freedom from AbHV in apparently healthy populations of Australian abalone.


AFDL WSSV Validation• Confirmatory testing of imported commodity shrimp

• CSIRO WSSV qPCR used (only WSSV nPCR in the Aquatic Manual)

• Limited positive control material so gradual accumulation over time• 300 known‐negative samples (Queensland shrimp farms)• 100 known‐positive samples (positive test in two laboratories)

• 100/100 positives test‐positive by CSIRO WSSV qPCR• 79/100 positives test‐positive by OIE WSSV nPCR• 300/300 negatives test‐negative by both assays

• Demonstrated the CSIRO WSSV qPCR was more sensitive than the OIE WSSV nPCR

• Appropriate for detection of WSSV in sub‐clinically affected material


AFDL WSSV Validation• OIE WSSV qPCR described in the Aquatic Manual so equivalence testing of

the CSIRO WSSV qPCR and OIE WSSV qPCR• Tested submitted samples with both assays and at NEAT and 1/10 dilutions


General observations• Validation is important, not just for accreditation• Provides us with confidence in our test results • Provides stakeholders with confidence in our test results• Increased ability to withstand legal scrutiny of our test results

• Difficult to plan how to do it (usually reactive testing)• Difficult to get adequate numbers of known‐positive, sub‐clinically affected

material

• Ongoing process

• Difficult to get the information in to the public domain

Conclusions and Recommendations



Conclusions• Provision of adequately validated methods in the Aquatic Manual is critical

as these methods are used to support standards and recommendations in the Aquatic Code which in turn form the basis of Member countries’ scientifically‐sound decisions relating to aquatic animal health for the WTO Agreement on the Application of Sanitary and Phytosanitary Measures.

• However, validation is also an onerous, complicated, time‐consuming and costly exercise, particularly with regard to molecular assays targeting pathogens of aquatic animals.

• Absence of alternative, secondary, confirmatory assays (serology, virus isolation)• Reliance on herd‐testing and lethal‐sampling (no option to re‐sample an individual) • Difficulties obtaining adequate numbers of known‐positive animals• Difficulties obtaining adequate numbers of known‐negative animals


Recommendations

• A way forward would be for the OIE, through the Aquatic Animal Health Standards Commission (Aquatic Animals Commission), to facilitate:

• greater international collaboration with regard to sharing of known‐positive and known‐negative material for assay validation

• improved sharing of assay validation data generated by different laboratories, that is not currently in the public domain, to contribute to the assay validation dossiers

• easier access for publication of validation information

• guidance on improved and transparent feedback on OIE test methods

• improved use of Table 5.1 to provide greater clarity on the levels of validation of assays, depending on the purpose of the testing


Table 5.1

*Bioassay does not apply to sentinel animals

Method

SurveillanceLevel of

Validation

DiagnosisLevel of

ValidationLarvae Juveniles Adults Presumptive Confirmatory

Gross signs d c c N/A c d Stage 1

Bioassay d d d N/A* d c Stage 1

Histopathology d c cNot

validateda a Stage 4

Transmission EM d d dNot

validatedd c Stage 1

In situhybridisation

d c cNot

validatedd a Stage 2

qPCR d a aFully

validateda a

Fully validated

PCR and sequencing

d c c Stage 1 d a Stage 4

• Proposed Table 5.1 when validation of the AbHV qPCR assays is completed

A request for assistance

OIE Reference Laboratory for Yellowhead disease



Validation of new YHV1 assays

• Pre 2012: Six YHV genotypes (YHV1–6)

• 2012: YHV7 ‐ Australia• Detected by AFDL in broodstock prawns, unpublished (in preparation)

• 2014: YHV8 ‐ China• Detected by AFDL in imported commodity prawns from China, unpublished

• 2012 YHV9 ‐ China • Detected by a Chinese laboratory and the sequence published on Genbank

• 2014 YHV10 ‐ China • Detected by AFDL in imported commodity prawns from China, unpublished

• Specificity issues identified with OIE assays


Validation of new YHV1 assays

• Two new candidate assays, specific for YHV1• RT‐qPCR (real‐time TaqMan probe)• RT‐nPCR (conventional PCR)

• Seeking additional YHV1 positive material for assay validation• Clinical material• Nucleic acid

• Materials Transfer Agreements etc., etc.

• Provision of Proficiency Panels• Inter‐laboratory repeatability testing• Capability assessment


Acknowledgements

• AAHL Fish Diseases Laboratory

• National and international colleagues

• Fisheries Research and Development Corporation

• Australian Government Department of Agriculture

• OIE

CSIRO ANIMAL, HEALTH AND FOOD SCIENCES

Thank youNick MoodyTeam Leader – Aquatic Disease DiagnosisAAHL Fish Diseases Laboratory

t +61 3 5227 5749e [email protected] www.csiro.au

of diagnostic tests in the aquatic manual - home: oieoie.int/eng/a_aahrwf2015/presentations/3.4....

Documents