![Page 1: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/1.jpg)
1
AOAC – NMKL - NordVal
International Symposium 21/22 Mai 2015 in Stockholm
Harmonization of the NordVal International
Validation Protocol with ISO 16140:2015
for the validation of alternative methods.
Sven Qvist
Chairman
NordVal International
www.nmkl.org
![Page 2: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/2.jpg)
2
Recognised international validation organisations have participated in the elaboration of the new ISO/FDIS 16140-2:2015.
AFNOR Certification
AOAC
MicroVal
NordVal International
![Page 3: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/3.jpg)
ISO 16140 method validation
program consists of 6 parts 1. Vocabulary
2. Protocol for validation of alternative methods
3. Protocol for verification of reference and
alternative methods in a single laboratory
4. Protocol for in-house method validation
5. Protocol for interlaboratory validation
6. Protocol for validation of confirmation methods
3
![Page 4: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/4.jpg)
4
Objective of an international protocol for validation of alternative methods
Guidance for applicants, kit producers, expert laboratories, steering groups and technical committees on the procedure of validation of alternative microbiological methods
Obtain international recognition of validations carried out according to ISO 16140:2015
![Page 5: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/5.jpg)
5
Present legal basis for the use of alternative microbiological methods.
EC 2073/2005 states that the use of alternative methods are acceptabel when validated against an authorised reference method and certified in accordance with ISO 16140:2003 or other internationally similar protocol.
![Page 6: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/6.jpg)
6
Importance of validation
• To ensure reliable analytical results
• To ensure a high level of food safety
• Demonstrate equivalence with ref. method
• Demonstrate fit for purpose
• To be in accordance with EU regulations
• To be in compliance with the manufacturer's
claims
![Page 7: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/7.jpg)
7
Validation protocol contain sections for both qualitative and quantitative methods - both divided into:
comparison study of the alternative method
against a reference method
(performed by an expert laboratory)
collaborative study of the alternative method.
Valid results from at least 10 laboratories
(inclusion of 12-15 labs is advisable).
![Page 8: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/8.jpg)
NordVal International criteria for
approval of expert laboratories
• Economical independent of the applicant.
• Accredited to perform the ref. method.
• Be familiar with the principle of the
method to be validated.
•A competent person shall be in charge.
•A detailed protocol shall be approved.
8
![Page 9: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/9.jpg)
9
Method Comparison Study - Qualitative methods
For food: at least 5 relevant food matrices
For other categories: a relevant number of matrices.
For each matrix 60 samples are analysed
Inclusivity, exclusivity, sensitivity, accuracy,
specificity, limit of detection and agreement
between methods are studied
![Page 10: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/10.jpg)
Method comparison study on
inclusivity and exclusivity
Inclusivity: Select 50 pure cultures of the
target microorganisms relevant to the
alternative method and the food matrices
(for Salmonella select 100 pure cultures)
Exclusivity: Select 30 pure cultures of
non-target microorganisms known to
cause interference with the target
microorganism
10
![Page 11: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/11.jpg)
11
Method comparison study for accuracy, sensitivity and specificity Meat matrices in the present NordVal protocol
comprises one main category and 5 submatrices:
1.1 Raw red meat
1.2 Raw white meat
1.3 Raw smoked and salted products
1.4 Heat treated products
1.5 Fermented products
![Page 12: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/12.jpg)
12
Method Comparison Study – Matrices for meat
New ISO 16140:2015 validation protocol – 4
categories
1. Raw meat and ready-to- cook meat products
2. Ready-to-eat and ready-to-reheat meat products
3. Raw poultry and ready-to-cook poultry products
4. Ready-to-eat and ready-to-reheat poultry products
![Page 13: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/13.jpg)
13
Method Comparison Study – Matrices for meat ISO 16140:2015 validation protocol - Category 2:
2.1 Cooked meat products
2.2 Fermented or dried meat products
2.3 Raw cured (smoked) products (aw > 0,92)
2.4 Raw cured (smoked) products (aw < 0,92)
![Page 14: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/14.jpg)
ISO Interpretation of results
between ref. and alt. method
Reference
Method
Alternative
Method
+
+
-
-
+
-
-
+
-
+
Confirmed
Alt. Method
Interpretatio
n
Not needed
PA
Not needed
NA
Not needed
ND
+
PD
-
NA 14
![Page 15: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/15.jpg)
15
Method Comparison Study - Qualitative methods - Screening
Responses Reference method
positive (+/ )
Reference method
Negative (-/)
Alternative
method
Positive
+/+ positive
agreement
(PA)
-/+ positive
deviation
(PD)
Alternative
method
Negative
+/- negative
deviation (ND)
-/- negative
agreement (NA)
![Page 16: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/16.jpg)
16
Method Comparison Study - Qualitative methods - Confirmation
Responses Reference method
positive (+/ )
Reference method
Negative (-/)
Alternative
method
Positive
+/+ positive
agreement
(PA)
-/+ true positive
(TP)
Alternative
method
Negative
+/- false negative
(FN)
-/- negative
agreement (NA) +
false positives (FP)
![Page 17: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/17.jpg)
17
Estimation of accuracy, sensitivity, specificity and agreement - Confirmation Matrix PA
NA
ND
PD
Sum Relative Accuracy
AC (%)
Relative
Sensitivity SE (%)
Relative
Specificity SP
(%)
N (PA+NA+FP) x 100
N
PA + TP x 100
PA+FN
NA x 100
NA+FP
Food cat 1
Food cat 2
Food cat 3
Food cat 4
Food cat 5
Other
![Page 18: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/18.jpg)
18
NordVal International acceptance
criteria for qualitative methods
The overall sensitivity should 95%
The agreement between method, kappa
should indicate very good agreement,
i.e. κ > 0.80.
![Page 19: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/19.jpg)
19
Relative level of detection study of the alternative method against the reference method- ISO 16140:2015
Level 1 Level 2 Level 3
Concentration negative low higher
No of replicates 5 20 5
No of positives 0 25-75% 100%
RLOD =
LOD - alt.
LOD - ref.
![Page 20: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/20.jpg)
Method collaborative study
of a qualitative method
Determine the variability of the results obtained by the alternative method in different laboratories using identical samples
The expert and 10 participating laboratories are using both the alternative and the reference method for the study (10 valid results required)
One food matrix, 8 replicates and three different levels of contamination are analysed in the study
20
![Page 21: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/21.jpg)
Organisations and locations of
the participating laboratories
The 10 collaborators shall come from a minimum of 5 different organisations (advisable with at least 12 collaborators)
A maximum of 3 data sets can be produced by one organisation with laboratories in different locations
T
21
![Page 22: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/22.jpg)
22
Performance characteristics for
the study of quantitative methods
Inclusivity
Exclusivity
Reliability
Repeatability
Reproducibility
Uncertainty
![Page 23: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/23.jpg)
23
Comparison Study-Quantitative methods
For each matrix, calculate the mean and the SD of
the results obtained by the reference method and
its confidence level 2 ∙ SD
Calculate the mean and the SD of the results
obtained by the alternative method.
Plot the results into a designed diagram.
![Page 24: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/24.jpg)
24
Quantitative Methods
Milk powder
0
1
2
3
4
5
6
0 1 2 3 4 5 6
levels
log
cfu
/g
reference method
alternative method
![Page 25: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/25.jpg)
25
Acceptance criteria for
quantitative methods
If the mean of the alternative method is included in the confidence interval of the reference method (i.e. refmean ± 2 SD), there is no significant deviation between the methods.
Repeatability of less than 0.4 log cfu/g illustrates satisfactory precision.
![Page 26: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/26.jpg)
26
NordVal International Steering Group
Denmark: Sven Qvist (chairman)
Finland: Annika Pihjalasaari
Iceland: Franklin Georgsson,
Norway: Kjell Hauge
Sweden: Charlotta Engdahl Axelsson
Secretary: Hilde Skår Norli, NMKL
![Page 27: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/27.jpg)
NordVal International Organisation
• NMKL national committees appoint the steering
group members - one representative from each
of the Nordic countries.
• NordVal International appoints experts in ad
hoc technical committees.
27
![Page 28: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/28.jpg)
NordVal International policy
• The persons involved in the NordVal
International validation programme shall be
independent and cannot have special
interests in the production and distribution
of test kits, special reagents or instruments.
• The members must have the required
competence in microbiology and validation.
28
![Page 29: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/29.jpg)
29
![Page 30: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/30.jpg)
30
![Page 31: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/31.jpg)
31
![Page 32: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/32.jpg)
32
![Page 33: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/33.jpg)
33
![Page 34: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/34.jpg)
Future status of validated and
certified alternative methods
• Capacity, rapidity, cost effectiveness
and reliability have resulted in
increased and dominating role as
preferred methods in the food industry.
• Acceptance by authorities as official
standard reference methods should be
considered as a possibility in future
legislation. 34
![Page 35: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/35.jpg)
NordVal International policy
• The persons involved in the NordVal
International validation programme shall be
independent and cannot have special
interests in the production and distribution
of test kits, special reagents or instruments.
• The members must have the required
competence in microbiology and validation.
35
![Page 36: AOAC NMKL - NordVal International · PDF fileAOAC – NMKL - NordVal International Symposium ... AOAC MicroVal NordVal ... EC 2073/2005 states that the use of](https://reader030.vdocuments.net/reader030/viewer/2022020715/5aa27e4a7f8b9a1f6d8d4d48/html5/thumbnails/36.jpg)
Increased scope for items to be
covered by ISO 16140-2:2015
• Present validation protocol: Food, animal feed,
environmental samples and samples from the
primary production stage.
• Future items for consideration: Water, virus,
confirmation methods, microtoxins, parasites.
36