validation studies of real time pcr protocols for ... validation… · validation studies of real...

Validation studies of Real Time Validation studies of Real Time PCR protocols for PCR protocols for

Salmonella and Salmonella and ListeriaListeriaSalmonella and Salmonella and ListeriaListeriamonocytogenesmonocytogenes

Monica Gianfranceschi, David Rodriguiez-Lazaro,

Elisabetta Delibato, Antonietta Gattuso, Damiano Comin

and Dario De Medici,

Components of a microbiological criterion (Codex)

• Micro-organism of concern

• Analytical method

• Sampling plan

• Microbiological limits

• The food-stuff

Task 7.3 and Task

7.5

Task 7.4 and Task

2

• The food-stuff

• The point of the food chain

where the limit applies

• Actions to be taken when

unsatisfactory

• Results

Task 7.4 and Task

7.5

Background

Testing against criteria

Food Business Operators (Reg 2073/2005)For validation and verification of procedures

based on HACCP and GHP

For batch acceptability testing during the storage life

4

For batch acceptability testing during the storage life

(PO)

Competent Authorities (Reg 882/2004 onofficial controls)To verify that food is in compliance with the Community

criteria during the entire shelf life (FSO)

Classical Cultural Methods

• No expensive infrastructure • Cheap in consumables• Cheap equipments

5

• Laborious to perform• Large volumes of materials• Time consuming

• rapid methods replaced “alternative methods”.

• Rapid =

– a shorter time to detection

– better flow through

– automation

Alternative method or rapid methods

6

– automation

Based on “Golden standard”:

a “rapid method” can be defined as any method or system that

reduces the time taken to obtain a microbiological test result

(Feng, 1996; Fung, 1994)

Rapid (Alternative) methods

Advantages for Food Business Operators:• Rapid corrective measures and actions • Possible to verify the efficency of the GHP and HACCP

Results available more rapidly than by

using classical cultural methods

7

Advantages for Competent Authorities:• More rapid management of emergency situation and crisis• Evaluate microbiological hazard prevalence/concentration

in a specific commodity (large monitoring), in order to evaluate the risk for the population

The use of alternative analytical methods is

acceptable when the methods are:

• validated against the reference method

Alternative method in Regulation EC 2073/2005 on microbiological criteria

8

• if a proprietary method, certified by a third

party in accordance with the protocol set out

in EN/ISO standard 16140 or other

internationally accepted protocols, is used.

Proprietary Methods

Validation bodies

• AFNOR (Association Français de

Normalisation, France)

• NordVal (part of the Nordic Committee

on Food Analysis, Norway)

• MicroVal (European Validation and

Certification Organisation, Europe)

9

Certification Organisation, Europe)

• AOAC (Association of Official Analytical

Chemist, USA)

Validation certificates of AFNOR, NordVal and MicroVal are based on ISO 16140

• AOAC ~ ISO 16140

ISO for molecular methods

Standard reference Title Status

EN ISO 20837:2006Microbiology of food and animal feeding stuffs - Polymerase chain reaction (PCR) for the detection of food-

borne pathogens - Requirements for sample preparation for qualitative detection published


borne pathogens - Requirements for amplification and detection for qualitative methods published

EN ISO 22118:2011Microbiology of food and animal feeding stuffs - Polymerase chain reaction (PCR) for the detection and

quantification of food-borne pathogens - Performance characteristics published

EN ISO 22119:2011Microbiology of food and animal feeding stuffs - Real-time polymerase chain reaction (PCR) for the detection

of food-borne pathogens - General requirements and definitions published


publishedEN ISO 22174:2005Microbiology of food and animal feeding stuffs - Polymerase chain reaction (PCR) for the detection of food-

borne pathogens - General requirements and definitions published

CEN ISO/TS 13136:2012

Microbiology of food and animal feed - Real-time polymerase chain reaction (PCR)-based method for the

detection of food-borne pathogens - Horizontal method for the detection of Shiga toxin-producing

Escherichia coli (STEC) and the determination of O157, O111, O26, O103 and O145 serogroups

published

ISO/TS 15216-1Microbiology of food and animal feeding stuffs- Horizontal method for detection of hepatitis A virus and

norovirus in food using real-time RT-PCR- Part 1: Method for quantitative determinationpublished

ISO/TS 15216 -2Microbiology of food and animal feeding stuffs- Horizontal method for detection of hepatitis A virus and

norovirus in food using real-time RT-PCR- Part 2: Method for qualitative determinationpublished

ISO/PDTS WI 00275221

“Microbiology of food, feeding stuffs and environmental samples — Polymerase chain reaction (PCR) for the

detection of food-borne pathogens — Detection of botulinum type A, B, E and F neurotoxin producing

clostridia”

draft

“General requirements relating to molecular methods for detection and quantification of

microorganisms” - Draft standard of PCR detection of enteropathogenic Yersinia spp.in discussion

Microbiology of food and animal feeding stuffs - Detection of Vibrio parahaemolyticus in seafoods: Part 1 -

Quantitative determination of total, thermostable direct haemolysin (TDH) and thermostable direct-related

haemolysin (TRH) positive Vibrio parahaemolyticus using nucleic acid hybridisation

in discussion

Revision of EN ISO 16140

PG 1: Terminology

PG 2: Proprietary methods

PG 3: Intermediate validation

PG 4: Method verification

11

PG 5: In house method validation

PG 6: Technical requirements for the

establishment/revision of standardised

methods

Revision of EN ISO 16140

PG 1: Terminology

PG 2: Proprietary methods

PG 3: Intermediate validation

PG 4: Method verification

12

PG 5: In house method validation

PG 6: Technical requirements for the

establishment/revision of standardised

methods

Intermediate validation (non proprietary methods)

First Draft text EN ISO 16140-1 - Terminology of Method Validation”

Intermediate validation

A validation study including an inter-laboratory study with a limited

number of participating laboratories

NOTE

13

NOTE

This type of validation study is restricted to non-proprietary methods. It is

applicable when the method of analysis deals with a complex

methodology or with an analyte for which only a restricted number of

laboratories have the competence and the infrastructure to participate in

a validation study.

Since the validation process has high costs, the

laboratories involved should be preferably reduced

VALIDATION STRATEGYPRE-VALIDATION EVALUATION

Objective: Define the best conditions for alternative tests compatible with ISO standards for detection of Listeria monocytogenes and Salmonella spp.

Prerequisites: Prerequisites:

• Rapid protocol (next-day results)

• As easy and simple as possible (easy to be implemented in a routine lab)

• Compatible with ISO standard (i.e. the ISO standard must be continued

once the results are obtained by the alternative methods)

• Not too expensive (similar price as standard)

Approach:

• Use the same enrichment broths (Half-Fraser for L. monocytogenes,

BPW for Salmonella spp.)

• Evaluate several strategies for reduction of time for final

Validation Strategy:Pre-Validation Evaluation for Listeria and Salmonella

results:

� Reduce the time of incubation

� Use of easy and simple bacterial DNA extraction protocols

� Use real-time PCR assays for detection

Parameters to evaluate:

� Size of sample (25 g or 50 g)

� Dilution of the sample (1:3, 1:5 or 1:10)

Validation Strategy Pre-ValidationEvaluation for Listeria and Salmonella

� Dilution of the sample (1:3, 1:5 or 1:10)

� Incubation time (i.e. 5, 8-10 or 18-24 hours of (pre)-enrichment)

� Bacterial DNA extraction protocol (boiling, resin protocol, or commercial

column-based kit)

Listeria monoctyogenes1-10 ufc

10-100 ufc

(100-1000 ufc)

25 g

50 g

ISO

Dilutions: 1:3

1:5

Listeria

monoctyogenes

VALIDATION STRATEGYEVALUATION APPROACH

50 g1:5

1:10

Incubation:

8 h

18 h

24 hDNA extraction:

Boiling

Chelex

Column

monoctyogenes

DilutionIncubation

timesISO

Baseline

QIAGEN

Baseline

CHELEX

Baseline

BOILING

1 cfu /

sample1:3

8 h 3/3 1/3 0/3 0/3

18 h 3/3 1/3 0/3 0/3

24 h 3/3 3/3 1/3 0/3

VALIDATION STRATEGYPRE-VALIDATION EVALUATION RESULTS

1:5

8 h 3/3 0/3 0/3 0/3

18 h 3/3 2/3 0/3 0/3

24 h 3/3 2/3 1/3 0/3

1:10

8 h 3/3 0/3 0/3 0/3

18 h 3/3 0/3 1/3 0/3

24 h 3/3 3/3 0/3 0/3✔

1:10 Half Fraser

(ISO)

Incubation 24 h

Validation StrategyFinal protocol: Listeria

monocytogenes in Cheese

Listeria monoctyogenes

DNA extraction: QIAGEN column

Real-time PCRRodríguez Lázaro et al (2004) Appl. Environ. Microbiol.

ISS

Strains

1-10

10-

100

3.4 cfu/g

0.34 cfu/g

5 hours1°

detection

18 hours

3 a.m. 2°

detection

3 hours 3°

detectionAfter 24 hours

ISO method Real-Time PCR

CFU

L. monocytogenes/samplesDilution Rate*

pre-

enrichment

(HFB)

hours

Numbers of

replicates

Positive

samples

Positive

samples

hly system

(FAM)

Ct±SD

IAC system

(HEX)

Ct±SD

1-10 CFU

(0.34 CFU/g)

25g + 50mL

5h 5 2 0 - 32.92±1.25

8h 5 5 0 - 33.00±0.48

24h 5 5 5 35.91±0.67 32.61±1.46

25g + 100mL

5h 5 2 0 - 32.55±0.79

8h 5 5 0 - 32.12±0.90

24h 5 5 5 29.59±0.66 32.89±1.10

25g + 225mL

5h 5 2 0 - 32.86±1.10

8h 5 5 0 - 33.51±0.5825g + 225mL 8h 5 5 0 - 33.51±0.58

24h 5 5 5 29.16±0.91 32,17±0.64

10-100 CFU

(3.4 CFU/g)

25g + 50mL

5h 5 2 0 - 33.28±0.84

8h 5 5 0 - 33.63±0.71

24h 5 5 5 31.26±0.31 32,70±1.83

25g + 100mL

5h 5 2 0 - 33.30±0.42

8h 5 5 0 - 33.32±0.70

24h 5 5 5 26.61±0.37 32,46±1.43

25g + 225mL

5h 5 2 0 - 33.10±0.63

8h 5 5 0 - 33.62±1.03

24h 5 5 5 25.38±0.16 32.86±1.40

Reduction of volume for expensive media order to reduce cost for analysis

No differences were found when we use 100 or 225 ml of Half Frazer with

25 g of sample after 24 hours of pre-enrichment in Real Time PCR

Real Time PCR positive after 24 hours pre-emrichment in HF

After 5 and 8 hours of pre-enrichment in HF all the samples

were negative

23

Salmonella spp.

1-10 ufc

10-100 ufc

100-1000 ufc

25 g

50 g


Incubation:

6 h

10 h

DNA extraction:

Boiling

Chelex

Column

10 h

24 h

DilutionIncubation

timesISO

Baseline

QIAGEN

Baseline

CHELEX

Baseline

BOILING

1 cfu /

sample1:10

6 h 3/3 1/3 1/3 0/3

10 h 3/3 3/3 3/3 3/3

24 h 3/3 3/3 3/3 3/3 ✔


DilutionIncubation

timesISO

Baseline

QIAGEN

Baseline

CHELEX

Baseline

BOILING

1 cfu /

sample1:10

6 h 3/3 1/3 1/3 0/3

10 h 3/3 3/3 3/3 3/3

24 h 3/3 3/3 3/3 3/3

Salmonella spp.(it is possible also use 50 g)

Dilution 1:10 Buffered Peptone Water

(ISO)

VALIDATED STRATEGY FOR SALMONELLA

Incubation 24 h (it is possible at 10 h, but better

Ct values after 24 h)

DNA extraction: Chelex

Real-time PCRJosefsen MH, et. al (2007). AEM 73:3040-3048.

Malorny B, et al (2004) AEM. 70:7046-7052.

Preparation of spiked samples for Ring Trial

SalmonellaListeria monocytogenes

27

Lenticules Lyophilized strains

Protocols

28

Ring trialLaboratories Salmonella

Listeria

monocytogenesCountry

Accredited

17025

Università di Bologna X X Italy

National Veterinary Institute X X Norway Yes

Centro National de Tecnologia y

Seguridad AlimentariaX X Spain Yes

National Food Chain Safety Office X X Hungary Yes

Istituto Superiore di Sanità X X Italy Yes

University of Zagreb X X Croatia Yes

Istituto Tecnologico Agrario de Castilla

y LeonX X Spain

y LeonX X Spain

University of Copenhagen X X Denmark

Agence nationale de sécurité sanitaire

de l’alimentation, de l’environnement et

du travail

X France Yes

IZS Venezie X X Italy Yes

IZS Lazio e Toscana X X Italy Yes

IZS Lombardia e Emilia Romagna X X Italy Yes

IZS Meridione X Italy Yes

IZS Abruzzo e Molise X Italy Yes

NRL

Ring Trial Listeria monocytogenes

Worst-case scenario

Soft Cheese

� High level of natural competitive micro-flora

Participants: 12 laboratories (all labs are very expert in the detection of

Listeria monocytogenes using ISO 11290-1 or/and Real Time PCR

methods

30

� High level of natural competitive micro-flora

� High level of amplification inhibitors (high level of fat and proteins)

The samples were spiked using a lyophilized standard containing:

� an ATCC strain of Listeria monocytogenes difficult to isolate (ATCC

19111)

� a strains of Listeria innocua ( with a concentration one log less

concentrate than Listeria monocytogenes)

Platform

31

6 3 2 1

12

Ring Trial Listeria monocytogenes Results

Laborato

ry

Sample A Sample B Sample C Sample D Sample E Sample F Sample G Sample H

ISO qPCR ISO qPCR ISO qPCR ISO qPCR ISO qPCR ISO qPCR ISO qPCR ISO qPCR

1 — — �� — — — �� — �� — ��

2 ��

3 �� — — ��

4 ��

5 �� — �� — ��

6 �� — — ��

7 — �� — �� — �� — �� — �� — �� — �� — ��

8 �� — — — — — ��

9 �� — �� — — �� — ��

10 �� — �� — �� — ��

11 �� — — — — ��

Low Concentration

32

Laborato

ry

Sample A Sample B Sample C Sample D Sample E Sample F Sample G Sample H

ISO qPCR ISO qPCR ISO qPCR ISO qPCR ISO qPCR ISO qPCR ISO qPCR ISO qPCR

1 — �� — �� — �� — �� — �� — �� — ��

2 ��

3 ��

4 ��

5 �� — ��

6 ��

7 — �� — �� — �� — �� — �� — �� — �� — ��

8 — �� — — — ��

9 — �� — ��

10 �� — �� — �� — �� — ��

11 ��

Medium Concentration

Contamination

level

Diagnost

ic

Specifici

ty

Diagnost

ic

Sensitivt

y

Positive

predictiv

e value

Negative

predictive

value

Accordance

(%)

Concordance

(%)

Concordanc

e odd ratio

(COR)

ISO

LOW

— 70.45 — —

72.11

(58.42 ,

86.02)

56.60

(47.63 ,

76.02)

1.98

(1.0 , 4.78)

MEDIUM

— 72.73 — —

83.79

(69.53 ,

96.09)

57.80

(46.79 ,

84.81)

3.77

(1.3 , 13.8)

Contamination levelDiagnostic

Specificity

Diagnostic

Sensitivty

Positive

predictive

value

Negative

predictive

value

Accordance

(%)

Concordance

(%)

Concordance

odd ratio

(COR)

ISO

LOW — 70.45 — —72.11

(58.42 , 86.02)56.60

(47.63 , 76.02)1.98

(1.0 , 4.78)

MEDIUM — 72.73 — —83.79

(69.53 , 96.09)57.80

(46.79 , 84.81)3.77

(1.3 , 13.8)

LOW+MEDIUM — 71.59 100.00 —78.52

(65.98 , 90.7)57.32

(47.4 , 79.9)2.72

(1.17 , 6.87)

NONE 100.0 — — 87.12 100.00 100.00 1.00

Statistical analysis and ISO 16140 evaluation parameters

obtained for the qPCR-based method from the trial data

96.09) 84.81)(1.3 , 13.8)

LOW+MEDIUM

— 71.59 100.00 —

78.52

(65.98 , 90.7)

57.32

(47.4 , 79.9)

2.72

(1.17 , 6.87)

NONE 100.0 — — 87.12 100.00 100.00 1.00

qPCR

LOW

— 87.50 — —

76.6

(64.62 ,

87.72)

76.10

(65.78 ,

87.18)

1.03

(0-90 , 1.16)

MEDIUM

— 97.73 — —

96.10

(88.3, 100.00)

95.50

(87.03 ,

100.00)

1.20

(1.04 , 1.17)

LOW+MEDIUM

— 92.61 100.00 —

86.45

(79.19 ,

93.63)

86.24

(79.74 ,

93.44)

1.02

(0.98 , 1.07)

NONE 100.0 — — 64.72 100.00 100.00 1.00

qPCR

LOW — 87.50 — —76.6

(64.62 , 87.72)76.10

(65.78 , 87.18)1.03

(0-90 , 1.16)

MEDIUM — 97.73 — —96.10

(88.3, 100.00)95.50

(87.03 , 100.00)1.20

(1.04 , 1.17)

LOW+MEDIUM — 92.61 100.00 —86.45

(79.19 , 93.63)86.24

(79.74 , 93.44)1.02

(0.98 , 1.07)

NONE 100.0 — — 64.72 100.00 100.00 1.00

Relative accuracy

(%)

Relative specificity

(%)

Relative sensitivity

(%)

False negative ratio

(%)

Falso positive ratio

(%)

82.75

(78.01 , 87.48)

96.70

(93.00 , 100.00)

97.62

(96.00 , 100.00)

1.18 0.0

Relative accuracy Relative specificity Relative sensitivity False negative ratio Falso positive ratio

L. monocytogenes detection by the qPCR method was affected in a

lower extend in that worst scenario, being therefore proved to be

the more reliable than the reference method. At top of that, the

qPCR method is cost effective (3 € vs 15 €) and time saving (27

hours vs 7 days).

CONCLUSIONS

Relative accuracy

(%)


(%)


(%)


(%)


(%)

82.75

(78.01 , 87.48)

96.70

(93.00 , 100.00)

97.62

(96.00 , 100.00)

1.18 0.0

Ring Trial Salmonella

Pork loin was aseptically removed in a slaughtering; the meat was stored

for ageing for three days at 4°C.

325 samples of 25-g were prepared used ageing pork loin.

Each sample was used for artificially contaminated..

The contamination of the pork samples were performed using commercial

lenticules.

Partecipants: 13 laboratories

35

lenticules.

The contamination was performed using three different blinded

contamination levels of lenticules:

L0 = negative control

L1 = lenticules contaminated with 10-20 CFU (0.4-0.8 CFU/g of pork

meat) of Salmonella Typhimurium NCTC 12023 (HPA, Salisbury, UK)

L2 = lenticules contaminated with 20-160 CFU (0.8- 6.4 CFU/g) of

Salmonella Typhimurium NCTC 12023 (Oxoid, Milan, Italy)

Platform

36

6 3 2 1

13

1

Contamination

level

Diagnost

ic

Specificit

y

Diagnost

ic

Sensitivt

y

Positive

predictiv

e value

Negative

predictive

value

Accordance

(%)

Concordance

(%)

Concordanc

e odd ratio

(COR)

LOW — 100.0 — — 100.0 100.0 100.0

Contamination

level

Diagnostic

Specificity

Diagnostic

Sensitivty

Positive

predictive

value

Negative

predictive

value

Accordance

(%)

Concordance

(%)

Concordanc

e odd ratio

(COR)

ISO

LOW — 100.0 — — 100.0 100.0 100.0

MEDIUM — 100.0 — — 100.0 100.0 100.0

LOW+MEDIUM — 100.0 100.0 — 100.0 100.0 100.0

NONE 100.0 — — 100.00 100.0 100.0 100.0

qPCR

LOW — — — 100.0 100.0 100.0

MEDIUM — 100.0 — — 100.0 100.0 100.0

LOW+MEDIUM — 100.0 100.0 — 100.0 100.0 100.0

NONE 100.0 — — 100.00 100.0 100.0 100.0

ISO

LOW — 100.0 — — 100.0 100.0 100.0

MEDIUM — 100.0 — — 100.0 100.0 100.0

LOW+MEDIUM — 100.0 100.0 — 100.0 100.0 100.0

NONE 100.0 — — 100.00 100.0 100.0 100.0

qPCR

LOW — — — 100.0 100.0 100.0

MEDIUM — 100.0 — — 100.0 100.0 100.0

LOW+MEDIUM — 100.0 100.0 — 100.0 100.0 100.0

NONE 100.0 — — 100.00 100.0 100.0 100.0Relative accuracy

(%)


(%)


(%)


(%)


(%)

100.0 100.0 100.0 0.0 0.0

Salmonella IAC

Results of Salmonella Ring Trial

38

A lot of IAC negative in positive samples, probably the

method does not work well in duplex using Roche (the

lab has 100% of concordance with the real samples)

6 negative IAC in negative samples (presence

of inhibitors) 1 in lab 3 and in lab 8, 2 in lab 7

and 9

University of BolognaGerardo Manfreda

Alessandra De Cesare

Frederique

National Veterinary InstituteTaran Skjeral

GroJohannessen

CNTAJavier Perez

Maria Jose Saiz

Hungarian Food Safety Office

AnsesLegall Francoise

Marianne Chemmay

Berthand Lombard

Nathalie Gnanou-Besse

IZS VenezieDamiano Comin

Maria Grimaldi

Antonia Ricci

Antonia Lettini

IZS Lombardia e Emilia RomagnaMarina Nadia Losio

Barbara Bertasi

Acknowledgements

39

Zsuzsanna Sréterné Lancz

Mészáros László

University of ZagrebLidija Kozačinski

Danijela Horvatek Tomić

ITACyLDavid Rodriguez Lazaro

Marta Hernanez-Perez

University of CopenhagenJonh Olsel

Dziuginta Jakociune

Barbara Bertasi

Enrico Pavoni

IZS Lazio e ToscanaStefano Bilei

Paola De Santis

Sarah Lovari

IZS MezzogiornoFederico Capuano

Yolande Proroga

Istituto Superiore di SanitàMonica Virginia Gianfranceschi

Elisabetta Delibato

Antonietta Gattuso

Michele Sonnessa

Validation of Sampling scheme of

Campylobacter in poultry (Antonio Valero,

Damiano Comin & Alessandra De Cesare)

PO= less than 2 % positive with a contamination > 10.000 cfu of

Campylobacter/g of skin

Validation protocol:Validation protocol:

30 different lots

3 samples for lots

All the samples was sampled after chilling (according with EU

regulation 2073)

2 slaughterhouses:

1. 90 carcasses from 30 batches of a medium slaughterhouse

2. 90 carcasses from 30 batches of a large slaughterhouse.


Campylobacter in poultry (Antonio Valero,

Damiano Comin & Alessandra De Cesare)

The analysis were performed using 35 grams of skin (10 g for quantitative

detection, and 25 for qualitative)

EFSA suggested to use the skin of the neck and half breast (we use the

skin of the entire breast, because half breast was insufficient.

The validation were performed in three slaughterhouses:

A. 90 carcasses from 30 batches of heavy broiler of a large

slaughterhouse.

B. 90 carcasses from 30 batches of light broiler of a large

slaughterhouse.

C. 90 carcasses from 30 batches of a medium slaughterhouse

In A e B were analyzed 36 batches (20% more)


Campylobacter in poultry (moving window

approach?)


Campylobacter in poultry (Damiano

Comin IZS Venezie)

A. 58 samples out 108 samples were positive (53.7) 23 lots out 36 were positives (63.9%); all the positive samples werecontaminated with less of <104 ufc/g of Campylobacter

B. 75 samples out 108 were positives (69.44), 28 lots out 36 wereB. 75 samples out 108 were positives (69.44), 28 lots out 36 werepositive 8 (77.8%); 3 samples out 108 samples (2.7 %) or 3 lotsout 36 lots (9.3%) were contaminated with more than<104 ufc/g (one lot with 18.000, one lot with 100.000 and one lotwith 100.000 and 10.000)

C. 51 samples out 90 were positive (56.6%), 16 lot out 30 werepositives (53.3%); one out 90 samples (1.1%) or one lot out 30 lots (3.3%) was contaminated with more than of 104 ufc/g ofCampylobacter (15.000 CFU/g)

validation studies of real time pcr protocols for ... validation… · validation studies of real...

Documents