an introduction to risk-based computer systems validation 10_poeta_pres.pdf · an introduction to...

20th Annual Validation Week Oct 28-30, 2014

genzymeA SANOFI COMPANY

An Introduction to Risk-Based Computer Systems Validation

Matthew PoetaPrincipal Technologist, IT Compliance


Outline• What is a Computer System?

• What is Validation?

• What is Risk Management?

• Implement Risk-Based Computer Systems Validation

• Questions

• Interactive Exercise

• Provide bonus material

Note: The views presented here are not necessarily those of any corporation.


What is a Computer System?A functional unit, consisting of one or more computers and associated peripheral input and output devices, and associated software, that:

• uses common storage for all or part of a program and also for all or part of the data necessary for the execution of the program;

• executes user-written or user-designated programs;

• performs user-designated data manipulation, including arithmetic operations and logic operations;

• can execute programs that modify themselves during their execution.

A computer system may be a stand-alone unit or may consist of several interconnected units.

(American National Standards Institute definition)


What is a Computer System?

A computer system “includes hardware, software, peripheral devices, personnel, and documentation; e.g., manuals and Standard Operating Procedures.” (FDA)

Examples of computer systems include:

• PI Data Historian System

• DeltaV manufacturing control system

• SCADA / PLC

• Honeywell Building Management System



Question: Is a lab instrument with a dedicated computer considered a Computer System?

It depends how you define your boundaries.

Boundaries could be set at any of the following:

Infrastructure (hardware and middleware, VMware, layered software, virus protection):

May just be a data center that contains OS’s, databases, antivirus, servers, performance monitoring, backup/archive/restore processes) that will serve as “home” for applications


What is a Computer System?Boundaries continued...

Application layer (including configuration):May be infrastructure dedicated to application as well as application

Controllers:May be infrastructure, application layer, and controllers (digital and analog I/O cards, buses, etc.)

Peripherals:May be infrastructure, application layer, controllers, and peripheral equipment (including pH meters, agitators, level sensors, etc.)


What is a Computer System?Environments…

Ideally, computer system would have three environments in place:

Development

• Used for developing or “playing”. Sometimes called a “sandbox”

Validation

• Perform Validation Testing

• Load system (configuration, etc.) into Prod. Environment

Production

• Used for Performance Qualification (PQ)

• Operations phase of system


What is a Computer System?Environments…

Points to consider:

• Test environments could be virtual spaces.

• They could be standalone exact copies of production spaces.

• Ideally all three environments would be in place but it depends on budget, space, and other constraints.

• It should at least be demonstrated that Validation Environment is an acceptable copy of Production Environment. Risk assessment can help justify why it’s ok to not have every peripheral.


What is a Computer System?Computer systems can be Local (e.g. Building Mgmt System)

Site 1 Site 2 Site 3 Site 4

Site 1

Computer systems can be Common (same system implemented at different sites

System Owner at the site who is ultimately responsible for availability, and support and maintenance, of a system and for the security of the data residing on the system. (GAMP)

Business Process Owner at the site who ensures that business needs are met by the system.

For local systems, one person may fulfill both roles.

System has defined boundaries and environments.

Each site would have its own System Owner / Business Process Owner and boundary / environments definitions


What is a Computer System?Computer systems can be Enterprise (e.g. SAP, TrackWise, ELMS)

Site 1 Site 2 Site 3 Site 4

Enterprise Site

One Enterprise level System Owner

Boundaries and environments defined at Enterprise level

Business Process Owner at each site



Computer Systems vary in complexity. They can be:

• Non-Configured application

• Configured application

• Custom code (bespoke)

Validation effort will increase from Non-Configured to Custom



Takeaway…

There are many different ways to define and implement a computer system gives its:

• Boundaries

• Environments

• Sites

• Complexity

How the boundaries and environments are defined, how the system is set up, and the complexity of the system should help to inform your specific validation process.



Regulations…

In the regulated biotech / pharmaceutical industry in the USA, The Code of Federal Regulations Title 21 – Food and Drugs, Part 11 – Electronic Records; Electronic Signatures(aka 21 CFR Part 11 or simply “Part 11”) is the driving regulation set forth by the United States Food and Drug Administration.



Regulations…

Part 11, III(C)(1) – Validation – “We suggest that your decision to validate computerized systems, and the extent of the validation, take into account the impact the systems have on your ability to meet predicate rule requirements. You should also consider the impact those systems have on the accuracy, reliability, integrity, availability, and authenticity of required records and signatures. Even if there is no predicate rule requirement to validate a system, in some instances it may still be important to validate the system.”



Regulations…

Predicate Rule (Definition): The underlying requirements set forth in the Act (Federal Food, Drug, and Cosmetic Act), Public Health Service Act, and other FDA regulations. (Part 11, §11.1)

Example: 21 CFR Parts 210 & 211, cGMP in Manufacturing, Process, Packing, or Holding of Drugs and Finished Pharmaceuticals



Regulations…Part 11, III(C)(1) – Validation – “We recommend that you base your (computer systems validation) approach on a justified and documented risk assessment and a determination of the potential of the system to affect product quality and safety, and record integrity.”

Additionally, EU Guidelines to GMP, Volume 4, Annex 11: Computerised Systems (aka Annex 11) states that Risk Management should be applied throughout the lifecycle of the computerized system. (1.)



Guidance…GAMP 5, Section 5.1 (Quality Risk Management –Overview) – “For a given organization, a framework for making risk management decisions should be defined to ensure consistency of application across systems and business functions. Terminology should be agreed upon, particularly regarding definitions and metrics for key risk factors. Such a framework is most effectively implemented when it is incorporated into the overall QMS, and is fully integrated with the system life cycle.”

GAMP: Good Automated Manufacturing Practice, published by International Society for Pharmaceutical Engineers (ISPE)


What is Validation?

Establishing documented evidence which provides a high degree of assurance that a specific process will consistently produce a product meeting its predetermined specifications and quality attributes. (FDA Guidelines on General Principles of Process Validation, published in ISPE Baseline® Guide, Vol. 1: Active Pharmaceutical Ingredients, Second Ed., 2007)

A documented program that provides a high degree of assurance that a specific process, method, or system will consistently produce a result meeting pre-determined acceptance criteria. (ICH Q7, published in ISPE Baseline® Guide, Vol. 6: Biopharmaceutical Manufacturing Facilities, 2004)


What is Validation?Confirmation, through the provision of objective evidence, that the requirements for a specific intended use or application have been fulfilled. (ISO/TC209 Working Groups at 2001-03-31 –Committee Draft ISO/CD 14644-6 “Cleanrooms and associated controlled environments” – Part 6: Terms and Definitions”)

Defined generically, a process to determine that a system or process is fit for the intended use. (GAMP® Good Practice Guide: Manufacturing Execution Systems – A Strategic and Program Management Approach, 2010)

Basically… validation means that a Process, Equipment, Instrument, or System has been verified to be fit for its intended use. It consistently and reliably performs its defined functionality as expected and meets its defined acceptance criteria.


What is Risk Management?• “Quality risk management is a systematic process for the

assessment, control, communication and review of risks to the quality of the drug (medicinal) product across the product lifecycle.” (ICH Q9, Section 4)

• “Application of quality risk management enables effort to be focused on critical aspects of a computerized system, in a controlled and justified manner, leading to specific benefits, such as:

• “scaling of life cycle activities and associated documentation according to system impact and risks

• “highlighting of areas where more detailed information is needed”

• GAMP 5, Appendix M3 (Science Based Quality Risk Management), Section 3 (Benefits)


Looking at Risk Management briefly:

• “It is neither always appropriate nor always necessary to use a formal risk management process…. The use of informal risk management processes (using empirical tools and/or internal procedures) can also be considered acceptable.” (ICH Q9, Section 1 (Introduction))

• “It is commonly understood that risk is defined as the combination of the probability of occurrence of harm and the severity of that harm.” (ICH Q9, Section 1)

• Harm – “Damage to health, including the damage that can occur from loss of product quality or availability.” (ICH Q9)

• For each identified risk scenario, score the Severity and Probability and calculate the Risk Classification. Depending on the formality of the Risk Assessment tool being used (e.g. FMEA), you may also score the likelihood that the fault will be detected before the harm occurs.

What is Risk Management?


Probability / Likelihood of Occurrence Low Medium High

Severity of Impact

High L2 L1 L1

Medium L3 L2 L1

Low L3 L3 L2

Risk Classification:

The risk is classified by taking into account the likelihood of the risk occurring and the severity of impact. The risk classifications used are:


Severity (definition): “A measure of the possible consequences of a hazard.” (ICH Q9, Section 7)

Probability (definition): Likelihood that something will go wrong… likelihood that the Harm will occur.


Detectability

Low Medium High

Risk Classification Level

L1 High High MedL2 High Med LowL3 Med Low Low

Risk Priority:By combining the risk classification with the detectability, the risk is assigned a priority level. The appropriate measures are documented in the assessment, to mitigate the adverse event that poses the risk. The risk priority of the fault conditions is used to select the appropriate risk mitigation, and focus the qualification effort. The Risk Priorities are described as:


Detectability (definition): “The ability to discover or determine the existence, presence, or fact of a hazard.” (ICH Q9, Section 7)

Hazard (definition): “The potential source of harm.” (ICH Q9, Section 7 refers to ISO / IEC Guide 51)


Failure Modes and Effects Analysis (FMEA)• RPN = Probability x Detectability X Severity

• RPN = Risk Priority Number


Low 0 – 20 No mitigation requiredMedium 21 – 40 Mitigation required unless justifiedHigh > 40 Risk unacceptable.

Mitigation required.Risk Benefit Analysis required if risk cannot be lowered.


Following the identification and scoring of risk scenarios, determine a mitigation strategy. Identify controls that could be or are already in place. These could include:• Modification of the design to remove or reduce the harm

• Placement of Engineering Controls to remove or reduce the harm

• Placement of Procedural Controls to inform the patient, operator, etc. of the risks and how to avoid them

• Scenarios have been determined to be LOW risk and justification is provided for not testing those requirements



Low risk justification example:• The software is Commercial Off-The-Shelf (COTS). Based

on the fact that it is so widely used and accepted, the group feels confident that specific functionality does not need to be explicitly tested.

• COTS – “Software defined by a market-driven need, commercially available, and whose fitness for use has been demonstrated by a broad spectrum of commercial users.” (IEEE)

• Specific functionality is tested implicitly. (e.g. the ability of a peristaltic pump to turn on / off is demonstrated during a CIP WFI rinse test)



What is Risk Management?“Why rank the Probability? It is a computer system running code that does the same thing over and over again.”

Alternate strategy… based on GxP Failure Impact and Business Failure Impact, determine required Mitigation and Testing

Requirement FailureMode

GxP Failure Consequence

Business Failure Consequence

MitigationStrategy

Testing Strategy

Req 1Req 2Req 3

Note: GxP = Good “x” Practice, where “x” = Clinical (GCP), Laboratory (GLP), Manufacturing (GMP), etc. “c” can be added at the front to make cGMP (i.e. Current Good Manufacturing Practice)


Takeaways…• There are different methods of risk assessment

and the proper method should be put in place depending on the application.

• Whatever the method, higher risks should be mitigated as such:• Modification of the design to remove or reduce the

harm

• Placement of Engineering Controls to remove or reduce the harm

• Placement of Procedural Controls to inform the patient, operator, etc. of the risks and how to avoid them



If higher risks cannot be effectively reduced through mitigation activities, then a Risk / Benefit Analysis may be appropriate.

Consider if the risk assessment should be:

• ad hoc (one time)

• Living (periodically reviewed for changes to risk rankings and mitigation effectiveness)



Risk-based Computer Systems Validation

Implement Risk-based CSV


Different models exist:


Waterfall (1)

(1) http://en.wikipedia.org/wiki/Waterfall_model

(2) http://www.pharmpro.com/articles/2008/03/gamp-standards-validation-automated-systems

V-Model (2)

Validation lifecycle should be:

• Scalable

• Traceable

• Defensible


Implementation of a system begins with definition of User Requirements.• System must be able to have at least six unique users.

• System must be Part 11 / Annex 11 compliant:

• Physical and logical security must be in place

• System must maintain audit trails which cannot be modified or deleted and must be viewable in human readable format

• System must meet requirements for Closed or Open Systems, as applicable

• System must meet requirements for hybrid signatures, digital signatures, and / or biometrics, as applicable

Note: Consult with CSV subject matter expects for guidance



User Requirements…• System must be compliant with organization password requirements:

• Minimum length

• Number of unique passwords before passwords can be reused

• Required characters (e.g. at least 1 capital letter, 1 lower case letter, 1 number, 1 special character, etc.)

• System must have the following procedures in place:

• Operation and Maintenance

• System admin

• Business continuity

• Disaster recovery

• Backup and restore



Functional Specification (or Functional Requirements Specification, FS or FRS) describes the desired functionality of the system.

Provide traceability throughout lifecycle…


URS Description FSReq 1 Part 11 compliant System requires unique user ID / password

combinationsReq 2 Complies with

corporate password policies

System requires passwords contain 6 chars, 1 special char, etc.

Req 3 Minimum 50 L/min pump flow

Minimum pump flow is 25 ± 3 L/min


Low 0 – 20 No mitigation requiredMed 21 – 40 Mitigation required unless

justifiedHigh > 40 Risk unacceptable.

Mitigation required.Risk Benefit Analysis required if risk cannot be lowered.

Implement Risk-based CSVAt the specification development stage, risk assessment performed to identify:• Risks to patient or product safety

• Required mitigation activities

• Required testing activities and prioritization of those activities

Probability / Likelihood of Occurrence Low Med High

Severity of Impact

High L2 L1 L1Med L3 L2 L1Low L3 L3 L2

Detectability

Low Med High

Risk ClassLevel

L1 High High MedL2 High Med LowL3 Med Low Low


Configuration Specifications record system configuration.• Computer make / model / peripherals (e.g. QC systems)

• Created user groups including permissions

• Interfaces



URS Description FS CSReq 1 Part 11 compliant System requires unique user ID /

password combinationsN/A

Req 2 Complies with corporate password policies


Password minimum length set to 6

Req 3 Minimum 50 L/min pump flow

Minimum pump flow is 25 ± 3 L/min N/A


Design Specifications show how the system and requirement functionality will be designed.• Hardware design spec

• Software design spec



URS Description FS CS (H)DSReq1

Part 11 compliant System requires unique user ID / password combinations

N/A N/A

Req2

Complies with corporate password policies


Password minimumlength set to 6

N/A

Req3

Minimum 50 L/min pump flow

Minimum pump flow is 25 ± 3 L/min

N/A Pump is 2 HP, 220V, 3 phase

Design Qualification is executed to ensure proper system design.


Installation Qualification – Verify and document that system has been set up correctly as designed in DS / CS.• Interfaces created

• User groups in place

• Infrastructure built as specified (e.g. server size, OS, etc.)



Implement Risk-based CSVOperational Qualification – demonstrate that discreet functions perform correctly, as described in the FS.• Data points are recorded to tags

• Backups occur as scheduled

• Functions can be called

• System security / audit trails are functional

URS Description FS CS (H)DS IQ / OQReq1

Part 11 compliant

System requires unique user ID / password combinations

N/A N/A Step 2.5, 3.4

Req2



Password minimum length set to 6

N/A Step 2.1, 3.6, 4.9


Implement Risk-based CSVPerformance Qualification – Demonstrate that system as a whole performs correctly, as described in URS• In Production Environment, system can be operated by cross

section of all user types with different permissions and roles

URS Description FS CS (H)DS IQ / OQ PQReq1

Part 11 compliant

System requiresunique user ID / password combinations

N/A N/A Step 2.5, 3.4 Att. 8, 9

Req2



Password minimumlength set to 6

N/A Step 2.1, 3.6, 4.9

Att. 5, 7


Implement Risk-based CSVValidation Plan – Outline steps with rationale provided for steps not performed and test environments.

URS

FS

CS

DS

Risk Assessment

IQ

OQ

PQ

Traceability TraceabilityDev

Val

Prod

Release



Following completion of PQ, handoff of system to system owner.

System owner ensures continuing operation including…• Maintenance

• Periodic Restore, BCP, DRP testing

• System periodic review (system still fit for purpose)

• Periodic risk review (including mitigation effectiveness, new risks?)

• Change Control, Service Level Agreements



What about commissioning?Commissioning may or may not be applicable depending on your system and how boundaries are defined

• If the system includes a skid containing controllers and field devices, FAT and SAT would be applicable

• If the system is virtual, then commissioning would probably not be applicable



Decommissioning…• System no longer in use

• Not supported by vendor

Points to consider…• Records retention

• System migration to virtual environment

• Future system access

• Quality Systems review (Change Control, Deviations, Investigation, CAPA, etc.)


References• The Code of Federal Regulations, Title 21 – Food and Drugs, Part 11

– Electronic Records; Electronic Signatures

• EU Guidelines to GMP, Volume 4, Annex 11: Computerised Systems

• ICH Q9

• GAMP 5


Acknowledgements• Ali Kara’a, IT Manager, Genzyme

• Xach Kibbie

• Russell Jacob, Genzyme

• Mark Pisaro, Consultant

• Steve Ostrove, Consultant

• Chris Potter, Consultant

• Gale Robinson, Consultant

• Institute of Validation Technology

• John Kirchner

• Allison Pulikowski


Questions


Interactive Exercise• Given a set of user and functional requirements

for a computer system, participants will perform a risk assessment for that system. Participants will propose ways to mitigate higher risks and begin to develop a test protocol based on the results of the risk assessment.


Bonus Materials• TBD…

an introduction to risk-based computer systems validation 10_poeta_pres.pdf · an introduction to...

Documents