c validate fda-compliant ms excel spreadsheets · 2017-04-27 · spreadsheet •microsoft excel...

1

R A U L S O T O , M S C , C Q E

I V T A P R I L 2 0 1 7 S A N D I E G O

SPREADSHEET VALIDATION 101: CREATE AND VALIDATE FDA-COMPLIANT

MS EXCEL SPREADSHEETS

ABOUT THE AUTHOR

• 20+ years of experience in the medical devices, pharmaceutical, biotechnology, and consumer electronic industries

• MS Biotechnology, emphasis in Biomedical Engineering

• BS Mechanical Engineering

• ASQ Certified Quality Engineer (CQE)

• I have led validation / qualification efforts in multiple scenarios:

• High-speed, high-volume manufacturing and packaging equipment

• Laboratory : information systems and equipment / instruments

• Enterprise resource planning applications (i.e. SAP)

• IT network infrastructure

• Software applications

• Product improvements, material changes, vendor changes

• Contact information:

• Raul Soto [email protected]

2

TABLE OF CONTENTS

1. REGULATORY PERSPECTIVE

2. SPREADSHEET VALIDATION

3. BUILDING A COMPLIANT SPREADSHEET

4. HANDS-ON EXERCISES

SCOPE

SPREADSHEET

• Microsoft Excel file that may contain:

• mathematical calculations

• Statistical analysis

• Graphing

• Outside our scope:

• Macros or custom code

• Database lookups

4

5R E G U L A T I O N S G O V E R N I N G S P R E A D S H E E T S

PART 1: REGULATORY

PERSPECTIVE

RAUL SOTO

IVT APRIL 2017 SAN DIEGO

WHICH SPREADSHEETS NEED TO BE

VALIDATED?

• Companies have received observations from FDA for not

validating spreadsheets used for:

• In–process analytical calculations

• Finished Product analytical calculations

• Tracking and trending quality metrics

• Databases / list

6

FDA 483

Pharmaceuticals (Eli Lilly):

FDA WARNING LETTERS

http://www.contractormisconduct.org/ass/contractors/18/cases/305/221/f

da-warning-letter.pdf

Pharmaceuticals (Cardinal Health):

FDA WARNING LETTERS

http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2013/ucm

369409.htm

Pharmaceuticals (Aarti Drug Limited):

FDA WARNING LETTERS

http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2006/u

cm075766.htm

Biologics (National Genetics Institute):

FDA WARNING LETTERS

11

http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2011/ucm2

56498.htm

Medical Devices (Aubrey Inc.):

FDA WARNING LETTERS

http://www.ofnisystems.com/media/g3667d.pdf

Medical Devices (B. Braun Medical Ltd):

FDA WARNING LETTERS

http://www.ofnisystems.com/Resources/Warning_Letters/g4452d.pdf

Medical Devices (Eldon Biologicals A/S):

FDA WARNING LETTERS

http://www.ofnisystems.com/media/g1483d.pdf

Medical Devices (EP Medsystems):

MEDICAL DEVICES

• 21 CFR 820.70(i): Automated processes

• When computers or automated data processing systems are

used as part of production or the quality system, the

manufacturer shall validate computer software for its intended

use according to an established protocol.

• All software changes shall be validated before approval and

issuance.

• These validation activities and results shall be documented.

15

MEDICAL DEVICES

• Let’s take this apart:

• Applies to computer / data processing systems used for

production or quality

• The manufacturer is responsible for validation

• Validation must be for the intended use(s)

• Must have an established protocol

• Changes must be validated before implemented

• Document all validation activities and results

16

MEDICAL DEVICES

• Section 6.0

• “Many other commercial software

applications, such as word processors,

spreadsheets, databases, and

flowcharting software are used to

implement the quality system. All of

these applications are subject to the

requirement for software validation, but

the validation approach used for each

application can vary widely.”

17

MEDICAL DEVICES

• Section 6.1

• “Numerous commercial software

applications may be used as part of the

quality system (e.g., a spreadsheet or

statistical package used for quality system

calculations, a graphics package used for trend analysis, or a commercial database

used for recording device history records or

for complaint management). The extent of

validation evidence needed for such

software depends on the device

manufacturer's documented intended use of that software.”

18

PHARMACEUTICALS

• 21 CFR 211.165(e): Testing and Release for Distribution

• (e) The accuracy, sensitivity, specificity, and reproducibility

of test methods employed by the firm shall be established

and documented. Such validation and documentation

may be accomplished in accordance with 211.194(a)(2).

19

BIOLOGICS

• 21 CFR 606.140: Laboratory Controls

• Laboratory control procedures shall include:

• (a) The establishment of scientifically sound and appropriate

specifications, standards and test procedures to assure that

blood and blood components are safe, pure, potent and

effective.

• (b) Adequate provisions for monitoring the reliability,

accuracy, precision and performance of laboratory test

procedures and instruments.

• (c) Adequate identification and handling of all test samples so

that they are accurately related to the specific unit of product

being tested, or to its donor, or to the specific recipient, where

applicable.

20

ISO 13485DESIGN AND MANUFACTURE OF MEDICAL DEVICES

• Clause 7.5.2.1

• The organization shall establish documented procedures for

the validation of the application of computer software (and

changes to such software and/or its application) for

production and service provision that affect the ability of the

product to conform to specified requirements.

• Such software applications shall be validated prior to initial

use.

21

ISO 13485 : 2015DESIGN AND MANUFACTURE OF MEDICAL DEVICES

• Will explicitly mandate:

• That software used as part of the quality system must be

validated and subject to change control.

• That we must have procedures to govern this

• That validation must be documented and the documentation

maintained.

22

FDA LAB MANUAL OF QUALITY PROCEDURES

• http://www.fda.gov/ScienceResearch/FieldScience

/LaboratoryManual/ucm174286.htm

23

4.5.3 VALIDATION OF SPREADSHEETS

SUMMARY

• Lock all non-input cells

• Make spreadsheets read-only, with password protection

• Automatically reject data outside valid ranges

• Positive and negative testing: Test by entering expected and

extreme values, and nonsensical data

• Keep record of all cell formulas

• Document changes made to the spreadsheet

• Change control with version numbers

• Periodic re-validation spreadsheets, including

• verification of cell formulas

• manual verification of spreadsheet calculations25

26V A L I D A T I O N P L A N , E X E C U T I O N , R E P O R T I N G

PART 2: VALIDATING THE

SPREADSHEET

RAUL SOTO


VALIDATING YOUR SPREADSHEET

• validation, software. (NBS) Determination of

the correctness of the final program or

software produced from a development

project with respect to the user needs and

requirements. Validation is usually

accomplished by verifying each stage of the

software development life cycle. See:

verification, software.

• Source: FDA Glossary of Computer System Software

Development Terminology

http://www.fda.gov/iceci/inspections/inspectionguides

/ucm074875.htm

27

GOVERNANCE

• Spreadsheet Validation Procedure• Outlines HOW spreadsheets will be validated

• What document(s) need to be produced

• Roles and responsibilities, Authors and Approvers, etc.

• Templates (i.e. Spreadsheet template)

• Change Control Procedure

• May be a separate procedure, or part of the main

Spreadsheets Validation Procedure

• Validated Spreadsheet Inventory

• Training28

VALIDATED SPREADSHEETS INVENTORY

• You need to know which validated spreadsheets are being

used, and where.

• Inventory document should contain:

• Spreadsheet Name and number

• Current Version

• Area(s) where used

• Owner

• Location

• Protected network folder address

• eDMS

29

TYPICAL VALIDATION ELEMENTS

• Validation Plan

• Scope

• Requirements Specification

• Design Specification

• Traceability Matrix

• Testing

• Development

• Installation Qualification

• OQ / PQ -or-

• System / User Acceptance

• Validation Report

• Procedures & Governance

• Templates

30

You can create individual

documents for each deliverable

-OR-A single Validation Protocol that includes ALL validation

elements

VALIDATION PLAN SCOPE

• Spreadsheet name

• Description and Purpose

• What the spreadsheet will be used for

• Process(es) supported

• Where it will be used

• References

• Analytical methods, SOPs, etc.

32

REQUIREMENTS

• Describes, with detail, WHAT you want the spreadsheet to do.

• Requirements should be

• Numbered

• Specific

• Testable

• Traceable

• MS Excel version required

• Excel Add-Ins required, if any

33

REQUIREMENTS

• Input / output cells and formula cells requirements

• Formatting (colors, font size and type, etc.)

• Data validation (field type, size, etc.)

• Expected input data and ranges, per input cell

• Expected output result, per output cells

• Any unit conversions required

• Descriptions for every formula

• Source (USP, literature, SOP, etc.)

• Page formatting

• Titles

• Headers

• Footers 34

REQUIREMENTS

• Charts and Graphs

• Title and Description of each

• Required labels for each axis

• Security requirements

• Password protection

• Output requirements

• Color printer, PDF-creating software

• Specific folder in a specific shared drive

• Automatic eMail to specific people

• Watermarks

35

DESIGN SPECIFICATION

• Describes, with details, HOW the spreadsheet will

comply with the Requirements

• Address all requirements, individually or by groups

• Describes the actual spreadsheet as written

• Shows actual EXCEL version of formulas

• Describes algorithms used, if any

• Describe conditional cell formatting used

• Describe data validation used

• Describe outputs produced (i.e. graphs, tables, etc.)

• OS in which the spreadsheet was developed

• i.e. MS Windows 7 SP1, Mac OS X 10.10

36

TRACEABILITY MATRIX

• Traces every requirement to its

corresponding design

element, and to the test

script(s) where the

requirement is challenged and

tested.

37

DEVELOPMENT TESTING & DRY RUNS

• Recommended!

• Typically not documented formally

• Dev Testing helps discover bugs and fix them while we’re not

yet in formal validation.

• Dry Runs help discover issues with the validation test scripts,

and allows us to fix these before we move into formal

document change control.

• Use formula auditing!

38

INSTALLATION QUALIFICATION (IQ)

• IQ includes Installation and Configuration

• State all prerequisites, ensure they are met prior to

execution

• List all spreadsheet file(s) to be installed

• List network location and path where spreadsheet files will

be installed

• Development environment – does not have to be

validated

• Test environment – may need to be validated

• Production environment – must be validated

39

OPERATIONAL QUALIFICATION (OQ)

• State all prerequisites, ensure they are met prior to

execution

• Verify that all requirements documented in the

Requirements section are met

• Ensure all outputs have the correct formatting

• Provide printouts of all formulas, input values used, output

results.

• Positive v Negative testing !!!

• Security testing

• Performance testing

• Simulate output issues

40


• Ensure all non-input cells are locked

• Ensure conversion factors are correct

• Ensure spreadsheets are read-only, with password protection

• Ensure the spreadsheet automatically rejects data outside valid

ranges

• Test for rounding errors

• Ensure Excel formulas match the source formula

• Ensure formula input / output cells are correct

• Compare formula results vs manual calculations

• Ensure units and spec limits are present and correct

41


42

• use the INFO command to provide information about the execution

environment.

OQ: POSITIVE VS NEGATIVE TESTING

• Positive testing : test and verify every single formula using

values that should produce correct results

• Ensure the spreadsheet produces correct results for

various input data sets

• i.e. at least two data sets

• Data set 1: known to result in a PASS decision

• Data set 2: known to result in a FAIL decision

• Include test cases with borderline data

• Verify spreadsheet results against results from manual

calculations

43


• Negative testing :

• Use incorrect inputs to ensure the

spreadsheet is robust, able to

handle them without crashing or

producing false or incorrect

outputs, and that it is not possible to

bypass security

44


Negative testing examples:

• Required data entry : ensure input fields cannot be left blank

• Field type test: ensure data entered is of the correct type (i.e. dates, alphanumeric, etc.)

• Enter alpha or special (@%^#) characters on a numeric field

• Enter a decimal number in an integer field

• Enter negative numbers in a positive numbers field

• Enter nonsensical inputs (i.e. ASCII CRTL codes)

• http://academic.evergreen.edu/projects/biophysics/technotes

/program/ascii_ctrl.htm

• Field size test: ensure user can only enter the specified number of characters in a field

45


Negative testing examples:

• Date bounds test: test upper and lower bounds for dates.

• Numeric bounds test: for example, if the number in a field should be between 10 and 50, the system should not accept numbers

lower than 10, and higher than 50.

• Numeric limits test: make sure the spreadsheet doesn’t crash or produce incorrect output when entering a number outside of

Excel’s limits:

• Excel 2013 limits:

• https://support.office.com/en-za/article/Excel-specifications-and-

limits-ca36e2dc-1f09-4620-b726-67c00b05040f

46


• For a good discussion, with

examples, of Positive vs Negative

software testing, please refer to:

• http://www.softwaretestingclass.co

m/positive-and-negative-testing-

in-software-testing/

48

OQ SECURITY TESTING

• Security testing:

• Ensure it is not possible to:

• Unlock locked cells / worksheets

• Alter formulas, output results, output

graphs /charts/ tables

• Add or remove worksheets

49

OQ PERFORMANCE TESTING

• Output issues : ensure the spreadsheet behaves as expected

when you

• Disconnect printer

• Force-close the PDF-rendering application

• No data loss even if it crashes

• Performance :

• Can it handle multiple users simultaneously?

• Open multiple instances of the spreadsheet at the same

time

• Execute spreadsheet while various other applications are

running

50

OPERATIONAL QUALIFICATION

• Formula Auditing

• Excel’s Formula Auditing tools allows us to demonstrate that

the inputs come from the expected cells

51

TEST EXECUTION

• Ensure test scripts are executed in

the appropriate sequence.

• Document deviations when actual

results do not match expected

results

• Take screenshots and printouts as

documented evidence

52

TEST EXECUTION

• Include all executed test scripts

• Include all screenshots, printed

outputs, manual calculations

• Post-approvals

53

CHANGE CONTROL AFTER VALIDATION

• Update Validation documentation

• Additional or changed requirements

• Corresponding design elements changes

• Update Testing

• New or updated test scripts

• Test new functionality (i.e. new/updated formula)

• Regression testing: ensure areas not changed still work

correctly

• Update traceability matrix

• Upon release, increase the spreadsheet Version number

• REMOVE ACCESS to the old version of the spreadsheet

54

CHANGE CONTROL

• Spreadsheets used for GxP processes should be subjected to

formal version control and validation.

• The organization’s change management system should be

used to document whenever we:

• Create a new spreadsheet for regulated / validated uses

• Make changes to a validated spreadsheet

• Retire/decommission a validated spreadsheet

• Move a validated spreadsheet to a different version of

Excel (i.e. from Excel 2007 to Excel 2010) from the one used

in development and validation

• Move a validated spreadsheet to a different operating

system (i.e. from Windows to Linux or MacOS)

55

VALIDATION: HOW MUCH IS TOO MUCH?

•Depends on the level of risk

56

57D E S I G N A C O M P L I A N T S P R E A D S H E E T

PART 3: BUILDING THE SPREADSHEET

RAUL SOTO


FDA’S INTERNAL GUIDELINES

FOR SPREADSHEET DESIGN AND VALIDATION

• Spreadsheet Design and Validation for the Multi-User

Application for the Chemistry Laboratory: Part I

• www.spreadsheetvalidation.com/pdf/LIB_Design_Multi-

User2A.pdf

• Spreadsheet Design, Verification and Validation, Use and

Storage of Single-User Workbook Files in the US FFDA

Laboratories: Part II

• www.spreadsheetvalidation.com/pdf/LIB_Part_II_Single-

User.pdf

58

COMMON SPREADSHEET ISSUES FOUND BY

FDA IN INDUSTRY

• Spreadsheet files not maintained, some deleted

• Rounding errors

• Formulas don't match (Excel vs source, Excel vs manual

calculation)

• Conversion factors issues

• Spreadsheet doesn't show specs or limits

• Unclear documentation

• Units not labeled

• Units: Weights described as volumes, areas as ratios

• Security and data integrity issues

• Cells not locked

• Regressions that inverted X and Y axis

59

IMPORTANT !!!

• Locking / unlocking cells

• Protect spreadsheets /worksheets

• Cell formatting:

• Data types

• Look and feel : Font, Colors

• Validation criteria definitions for input cells

• Define formulas

• Use of logical statements in formulas

• Use of Page Set-up

60

CREATING THE SPREADSHEET

• Verify formulas against sources

• Use correct units of measurement

• Lock all non-input cells

• Disable column/row add/remove

• Unlock input cells

• Distinguish input, calculation (formula) vs output cells

• Cell background color

• Font : bold, italics, size, color

• Use conditional formatting on output cells

• Different cell background color and/or font color for results

that pass vs results that fail

61


• Worksheets

• Main

• Revision History

• Any data tables needed

62


• The following information (at minimum) should be included on

the main worksheet:

• Spreadsheet Name

• Version number

• Date of last modification

• Every printout should contain

this information, plus

• Date / time of use

• User

63


• Add a field where the user will enter his/her name every time

the spreadsheet is used.

• Use Page Setup to print the following information:

• Spreadsheet filename

• Correct date and time

• Page numbers and total pages

• Make the spreadsheet read-only

64

REVISION HISTORY

• Each spreadsheet should have a Change History worksheet

(a.k.a. Revision History)

• Every time the spreadsheet is updated, an entry should be

made in this Change History:

• New version number

• Date of change

• Change control number

• Name of person(s) who made the change

• Itemized description of ALL changes made (with DETAILS)

65

UNITS CONVERSION

• You can use the Excel CONVERT function to convert values

from one unit to another in your formulas

• Click HELP in Excel for a list of all units included.

66

SECURITY LAYERS

• Lock all cells except user input cells.

• HOME tab/ CELLS group/ Format / Protection

• Password-protect the worksheet

• File / permission / encrypt with password

• Install the validated spreadsheet in a protected folder on a

networked shared drive

• Access to the folder should only be given to authorized

personnel

• OR save the validated spreadsheet as a controlled document

in your EDMS

67

QUESTIONS

68

69

Thank You !

RAUL SOTO


c validate fda-compliant ms excel spreadsheets · 2017-04-27 · spreadsheet •microsoft excel...

Documents