Methodologies

Participants testing for germline and somatic variants used different

approaches (Figure 3); the most commonly used platform for germline testing

was the Illumina MiSeq, whereas the Ion Torrent PGM was more frequently

use for somatic testing.

The majority of participants (80%) used versions of the Burrows-Wheeler

Aligners (Li & Durbin) or Bowtie (Langmead & Salzberg) for read alignment

and the most frequently used variant callers were those within the GATK

package plus United Genotyper, although a range of programmes were used

(see Figure 4).

Variant calling accuracy

When testing the same gene the majority of laboratories consistently detect the

same variants. In 2015, 74% of submissions achieved >80% concordant

variant result calls and this increased in 2016 to 89% of submissions (Figure

5). The reporting of somatic variants is more variable when present at a low

level. If a laboratory did not report the variant then it was not possible to

determine if this was due to a failure to call the variants or the bioinformatics

pipeline thresholds did not allow calling at the low level. This is being

addressed in the 2017 Somatic EQA.

Figure 5 – Percentage of correct germline variant calls out of the total number of calls

Quality of NGS data

Submitted data was assessed using a panel of quality metrics to provide

laboratories with a comparison of the standard of their own data with other

laboratories providing the same NGS tests. Each laboratory received a report

detailing the quality of their data and a visual display of where they were

positioned in respect to other participants. Figure 6 shows an example of a

quality report for FASTQ files where the laboratory individual value lies out-with

the consensus range.

Figure 6 – Benchmarking FASTQ

file dataRange of values from all submissions

using the same platform

Individual laboratory value

Benchmarking the quality of diagnostic

Next Generation Sequencing

Next Generation Sequencing (NGS) is being introduced into clinical genetics

laboratories worldwide and internal validation of results by alternative methods

is not possible due to the large amount of data generated by NGS. Therefore

the European Molecular Genetics Quality Network (EMQN) and the UK

National External Quality Assessment Service (UK NEQAS) for Molecular

Genetics have developed a joint External Quality Assessment (EQA) scheme

for NGS, with the aims to assess and improve quality, enable laboratories to

benchmark their NGS point mutation service against others using the same

and different approaches, work towards consistency of reporting clinical

results generated by NGS and contribute towards defining best practice.

To develop an EQA scheme for clinical laboratories using NGS for germline

and somatic testing:

• Which is generic (independent of genes tested, diseases, and platforms

used) so applicable to all laboratories performing NGS.

• To provide a panel of quality indicators related to individual laboratory NGS

data to enable laboratories to benchmark their NGS service against others.

• To assess variant calling against an EQA consensus sequence to provide

an assessment of the data analysis pipeline.

The UK NEQAS and EMQN experience has demonstrated that disease-specific EQAs has improved the quality of results and

consistency in clinical reports. The NGS EQA results enable diagnostic laboratories to address the quality of their NGS testing.

These technical NGS EQAs play an important role in enabling laboratories to benchmark this technology, assess the accuracy of

data and facilitate high quality reporting for patient benefit. Further developments will incorporate in future EQAs copy number

variants and structural variant detection as well as the current single nucleotide variants and indels.

INTRODUCTION

AIMS

• Since 2013, four pilot EQAs were run. The two latest EQA runs (2015 and

2016) were divided into Germline and Somatic schemes, in order to address

the different challenges of NGS in these settings.

• Laboratories were provided with a genomic DNA (from a cell line) or DNA

sample extracted from formalin-fixed paraffin-embedded (FFPE) tissue (and

matched germline) for NGS using their usual approach.

• Participants were required to submit technical details, and reportable

variants detected.

• The sequence of the DNA provided was validated in three clinical

laboratories using different NGS platforms. A pipeline to produce an EQA

consensus genome has been developed reducing the need to rely on time-

consuming expensive prior validation. The Schemes have developed

sophisticated tools for the integration and benchmarking of NGS data.

• These NGS Germline and Somatic EQAs are technology and testing

context independent and seamlessly fit with wet laboratory and

bioinformatics processes.

• In 2016 participants were allowed to submit up to three sets of results,

including whole exome/genome sequencing for both the Germline and

Somatic EQAs.

Participating laboratories has increased from 24 in 2013, to 227 in 2016 for the

Germline EQA and Somatic testing laboratories increased from 22 in 2015 to

76 in 2016 (Figure 1). Laboratories were located in 45 different countries.

Genes tested

The most frequently tested genes are displayed in Figure 2. In all EQA runs

BRCA1 and BRCA2 were tested by the majority of laboratories.

There was a substantial increase in the number of genes tested from 126

genes analysed in 2013 to 19,031 in 2015. The mean panel size increased

from four genes tested in 2013 to eight in 2015. The introduction of whole

genome sequencing was observed in the 2016 EQA

METHODS

ACKNOWLEDGMENTS: The Schemes would like to thank all the participating laboratories, the validating laboratories and the NGS EQA Specialist Advisory Group. For further information please contact Dr Sandi Deans (Sandi.Deans@ed.ac.uk).

RESULTS & DISCUSSION

RESULTS & DISCUSSION/continued

S. Deans1, M.W. Gutowska-Ding2, F. Khawaja1, K. Brugger3, J.W. Ahn4, S. Abbs3, J. Coxhead5, P. Westwood6,

K. Thompson7, C. Boustred8 G.R. Taylor 4, E. Souche9, A.P.A. Stegmann10, B. Ylstra11 and S. Patton2

1UK NEQAS for Molecular Genetics, Edinburgh, United Kingdom, 2EMQN, Manchester, United Kingdom, 3Cambridge University Hospitals NHS Foundation Trust, Cambridge, UK,4Viapath Genetics Laboratories, London, UK, 5Institute of Genetic Medicine, Newcastle University, Newcastle, UK, 6NHS Greater Glasgow & Clyde, Glasgow, UK,

7The Wellcome Trust Centre for Human Genetics, Oxford, UK, 8Genomics England, London, UK, 9Center for Human Genetics, Leuven, Belgium, 10MUMC Clinical Genetics, Maastricht, Netherlands, 11VUV University Medical Center, Amsterdam, Netherlands

CONCLUSIONS

Figure 1 – Number of participants per EQA

year

Figure 2 – Most frequently tested genes

(the larger the text the more commonly

tested)

Figure 3 – Percentage of laboratories using

each NGS EQA platform for the 2016

Germline and Somatic EQAs

Figure 4 – Variant caller programmes

used for the 2016 EQAs