new software developed with south west water -...
TRANSCRIPT
Beyond the technology
beyond the technology
New software developed with South West Water
Analytical Quality Control
(AQC) data manipulation,
charting and regulatory
compliance reporting -
Keith Huxford – Project Implementation Manager, CSols Ltd.
www.csols.com Data Visualisation & Interpretation
21st November 2012 Chateau Impney Droitwich Spa, WR9 0BN
• About CSols
• Why does Quality Matter
• What is Analytical Control?
• Process Overview
• Previous Capabilities
• Project Scope & Milestones
• Final Product & Challenges
• AqcTools Software
• Summary
Contents
About CSols
a laboratory systems integrator
we combine expertise, products,
services to help labs perform optimally
over 2 decades in business
interfaced 1000’s of instruments
installed handheld Remote Sampler
and BactiAnalyst enterprise systems
worked with 25 different LIMS
& IT systems
interfaced 10 different
chromatography data handling
systems
all industries, but particular focus on
water/environmental and hospital
laboratories
Customers
Water & Environmental
Environment Agency
Northumbrian Water
Portsmouth Water
SEPA
Scottish Water
Severn Trent
South West Water
South East Water
Northern Ireland Water
Thames Water
SEPA
Veolia Water
Analytical Services
Alcontrol
ALS
Scientifics
LGC
Servaco
Chemtest
Eurofins
SGS
Intertek
Expro
Inspectorate
Products and Services
Handheld Applications
Remote Sampler
for paperless external sample collection
BactiAnalyst
for automation in microbiology labs
Portable Worksheet Manager
PC Applications
Seamless Instrument Integration
Analytical Quality Control Charting
LabCache Laboratory Database
Implementation, consultancy & custom applications
Why does quality matter?
Hubble space telescope!
CBs = chlorobiphenyls
OCPs = organochlorine
pesticides
New laboratories joined the
sample in 2003
Source: Wells, de Boer, Trends
in Analytical Chemistry, Vol.
25, No. 4, 2006.
• QA/QC improves
performance over time
Costs of wrong data release
direct costs of fixing issue and restoring loss of
confidence / accreditations
indirect costs caused by incorrect trend
analysis, result in:
incorrect water protection measures
incorrect environmental remediation
measures
Value of proper QA/QC
From measurements to measures
20% less underestimates
5 % less overestimates
%
Labs
%
Labs
30% below 20% above 10% below 15% above
Without QA/QC With QA/QC
Analysis step: Defining the Probabilities of Wrong Decisions
%
Labs
%
Labs
30% below 20% above 10% below 15% above
Without QA/QC With QA/QC
Value of proper QA/QC
From measurements to measures
Data ‘feels right’ but pesticide analysis
data is incorrectly believed to be below
limit value
0
10
20
30
40
50
60
70
80
90
100
1998 2000 2002 2004 2006 2008 2010
Limit value
Value of proper QA/QC
Incorrect data lead to this Water Framework
Directive ecological status map: it should
have been red!
Value of proper QA/QC
From measurements to measures
Bad QA/QC – data wrong by a factor of 2!
0
20
40
60
80
100
1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
Limit value
Costs of Infringements
From measurements to measures France : €28 million financial sanction and a
€117,882 daily penalty payment for not
meeting the requirements of the Nitrate
directive
Spain : to pay €624,150 per year for every
1% of inshore bathing waters that continue
to fail to meet the quality standards set in
the Bathing Water Directive (European Court
penalty decision Brussels, 25 November 2003)
Costs of Rejected Data
From measurements to measures
7000 inorganic results corresponds to 56M€
105000 organic results corresponds to 840 M€
What is AqcTools?
CSols software for improved monitoring
of instrument and analytical quality
performance
CSols developers understand the
industries and application – they are
not just LIMS configuration staff!
developed in collaboration with South
West Water, major water,
environmental and hospital
laboratories
What does AqcTools do?
improves monitoring of instrument and
analytical quality performance
readily view charts and exceedances
provides a single standard networkable
solution for all laboratory tests
allows for audited exceptions handling and
reporting
allow managers to view long term trends or
provide historical charts for quality audit
purposes
Routine Analysis of AQCs
the most effective way of monitoring an Instrument’s
performance is to regularly use Analytical Quality
Control (AQC) samples with independently verified or
known results:
once placed in an instrument / method AQCs are run at
regular intervals and can be used to flag discrete changes in
instrument performance leading to over or under reporting of
result values
whilst simple flagging against a target value is straightforward,
AqcTools let the user identify trends and use specific statistical
calculations and chart types
AqcTools designed for analysts
Analysts want to be able to perform AQC statistical
analysis and chart data :
as soon as each instrument run is complete, so that the
AQCs can be correctly evaluated before releasing all
results (for QA/QC to review)
on the instrument workstation so there is no need to export
and carry the AQC results to a separate workstation
easily without having to navigate functions not relevant to
the AQC analysis
record all data as well as any decisions taken during the
interpretation for auditing and for further analysis.
Configurable AqcTools options
can be standalone or linked to instruments
can be seamlessly integrated with LIMS /
LIS, Excel or other database systems
supports manual data entry eg
pH, taste, colour, suspended solids, micro etc.
can be used to import historical data
utilizes LabCache Oracle or Microsoft SQL
Server database
Analytical Quality Control
What is Analytical Quality Control?
Analytical
Methods
How can we be confident that the results we send to
customers are accurate?
Sample
Certificate of
Analysis
Analytical Methods
LIMS
Data transfer
Sample result verification
Randomly placed AQC
Standard
Batch of
Samples
Process Overview
LIMS
Sample result verification
Sample results sent to
customers
AQC results verification
All results
checked and
verified
Process Overview
Previous South West Water Capabilities
Surprisingly, many water testing laboratories are still
hand plotting AQC results.
• UKAS accreditation audit in 2010 highlighted critical
deficiencies in ability to monitor AQC performance
• From 2000 to 2010 the South West Water LIMS system
had very limited AQC charting functionality and
many charts were manually plotted in Excel
Chart of means - E coli on MLSB
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
01/1
1/20
10
02/1
1/20
10
03/1
1/20
10
04/1
1/20
10
05/1
1/20
10
06/1
1/20
10
07/1
1/20
10
08/1
1/20
10
09/1
1/20
10
10/1
1/20
10
11/1
1/20
10
12/1
1/20
10
13/1
1/20
10
14/1
1/20
10
15/1
1/20
10
16/1
1/20
10
17/1
1/20
10
18/1
1/20
10
19/1
1/20
10
20/1
1/20
10
21/1
1/20
10
22/1
1/20
10
23/1
1/20
10
24/1
1/20
10
25/1
1/20
10
26/1
1/20
10
27/1
1/20
10
28/1
1/20
10
29/1
1/20
10
30/1
1/20
10
Date analysed
Mea
n C
FU
Mean CFU Mean = 41 LAL = 18 LWL = 26 UWL = 57 UAL = 64
Essential functions of new charting software:
• Communicate with the LIMS system to retrieve live
AQC data
• Plot Shewhart charts displaying the Mean, 1 SD, 2
SD (warning limit) and 3SD (action limit)
Project Scope
• Ability to calculate relative standard deviation
and bias on any dataset at any time.
• Rule breaches are highlighted on the chart and
investigations associated with them
• Easy to recalculate and update chart limits at
any time with full audit facilities.
• Flexible high level and daily reporting functions
Project Milestones
June 2010 October 2010 November 2010 December 2010 January 2011
Laboratory awarded MCerts accreditation
UKAS audit identified problems with performance
reporting capabilities
No standard packages available that met requirements. Project scoped and sent to multiple solution providers
Only two providers proposed solutions, meetings held with
providers
CSOLS awarded contract to supply
software
April 2011 May – June
2011
July 2011 August 2011 October 2011
Requirements Agreed for Chemistry (V1.0)
Coding for Application Functionality
Conversion to work with Oracle
Requirements agreed for Microbiology (V2.0)
Onsite Testing
Project Design
Project Development
Nov – Dec 2011
Coding for Application Functionality
Jan 2012
Onsite Testing
End Jan 2012
V2.0 Released to SWW
Chart can be viewed for any
user specified data range
Raw data and
rolling
calculations can
be easily
accessed
Rule failures are
flagged on the chart
and fully investigated
Final AqcTools Product
Final Product
Chart limits can be updated
at any time using calculations
from any set of data. These
limits are also sent to LIMS
Several reports can be
incorporated to collate
data from all available
charts in an easy to
understand format
South West Water Challenges
Timescales: Only had one year to demonstrate to auditors
that they were able to effectively monitor AQC performance
Guidance: No consistent national AQC policy from UKAS.
Different Laboratories given different advice
Budgetary considerations
> Overcome by careful planning of time and resources, with the
project being given a high priority within the department
> Overcome by calling a meeting with all other water company
laboratories to discuss the differences in our policies with and aim to try
to find common ground and best practice
> Overcome by working with CSols to create a new commercial
product at reduced cost to SWW
> Overcome by creating a plan that would work for 80% of the
parameters and then taking each exception in turn to find a solution
Scope: Hundreds of Excel charts covering hundreds of
parameters, each with their own individual quirks
Benefits of using AqcTools software
easier and direct networkable viewing of AQC data
reduced analyst and QC effort needed to access
statistics and charting
reduced IT support costs, easier to support
transcription free analysis of data
optional seamless integration with LIMS or other
databases or Excel or Quality Analyst
saves time and money
satisfies the regulators
CSols AqcTools Software - Main Menu
Information display area
AqcTools Configurable Chart Window
AqcTools Configurable Chart Limits
Ual Upper Action Limit (usually Mean+3 x Standard Deviation)
Uwl Upper Warning Limit (usually Mean+2 x Standard Deviation)
Usd Upper Standard Deviation (Mean+Standard Deviation)
Mean Average
Lsd Lower Standard Deviation (Mean+-Standard Deviation)
Lwl Lower Warning Limit (usually Mean-2 x Standard Deviation)
Lal Lower Action Limit (usually Mean+3 x Standard Deviation)
AqcTools Chart Limits
Where the data has
been subject to
changes in the released
values of the control
limits it is denoted on
the charts as a shift in
the control lines
AqcTools Chart Types
By Batch Mean Chart + Individual Points
By Batch Individual Points
By Batch: Mean
By Day: Mean Chart + Individual Points
By Batch/#: Individual Points Chart
By Day/#: Individual Points Chart
View Bad Data / Exception References
Show Bad Results When unticked the
chart is displayed
without any stars
denoting excluded
results.
Ticked Unticked
Show Exception
References
When unticked the
chart is displayed
without tag labels which
denote the exception
type i.e. ‚r’ or ‚d’
Ticked Unticked
Spreadsheet view of extracted data
Extracted dataset properties
Dataset calculations
Summary
AqcTools provides a convenient, cost effective
and dedicated mechanism for analysts to
perform those AQC statistical manipulations and
exceptions management/reporting required by
regulatory authorities
when combined with Links for LIMS, analysts get
very timely feedback on those subtle changes
that are effecting result quality so that poor
quality results are not released for use
resulting in faster, easier and better
quality of reporting of results
Message for decision makers
From measurements to measures Data quality is far from trivial
But it avoids that you are at risk
Is it considered as a priority issue
within your area of responsibility ?
A Final Word
Whoever observes an error and does
not take measures to correct it,
commits a second one!
References
South West Water purchase CSols AqcTools for
monitoring analytical performance
http://www.csols.com/pdf/CSols-SWW-Aqctools-
Press%20Release.pdf
The England & Wales Environment Agency benefits
from a CSols handheld computer solution Heartlands
http://www.csols.com/downloads/CSols-EA-
Case%20Study.pdf
Any Questions?
CSols Ltd.
The Heath
Runcorn
Cheshire
WA7 4QX
UK
Tel +44 (0)1928 513535, Fax +44 (0)7006 061106
email: [email protected], web: www.csols.com
www.aqctools.co.uk
CSols Ltd is registered in England and Wales.
Company Registration Number 3918227. VAT Number 539 6933 95.