develop a comprehensive master plan for computer validation in a
TRANSCRIPT
MVP Workshop – IVT April 2013
Master Validation Plan
Dr. Teri Stokes, GXP International,
Concord, MA – USA
www. GXPInternational.com Email: [email protected]
What is aVMP?
• Validation Master Plan (VMP):
Outlines the principles involved in the
qualification of a facility, defining the areas
and systems to be validated, and provides a
written program for achieving and maintaining
a qualified facility. It should include process
validation, facility and utility qualification and
validation, equipment qualification, cleaning
and computer validation. It drives a structured
approach to validation projects. (Wikipedia)
Slide 15
What is an MVP?
• Master Validation Plan (MVP):
Defines the areas with systems to be validated,
and provides a written program for achieving
and maintaining computerized systems to GXP
requirements. MVP should include OQ, IQ,
and PQ validation efforts for GXP compliance.
It drives a consistent, structured approach to all
computer validation projects.
• MVP has a qualifying analysis form for
identifying systems to be under its directives.
Slide 15
Validation Package Definitions
• Operational Qualification (OQ) – Documented evidence
that a system operates as intended throughout its expected
range as per functional design specifications. (Supplier Team)
• Installation Qualification (IQ) – Documented evidence that
all system components are installed to supplier instructions &
user requirements. (Infrastructure/IT Team)
• Performance Qualification (PQ) – Documented evidence
that a system operates as intended in the user’s work process.
(User Team – Start-up & Ongoing)
Slide 38
VMP, MVP, VP and VSR
• Validation Master Plan (VMP): to set a
standard approach for facility, products,
process, and systems qualification
– Master Validation Plan (MVP): to set a standard
approach for IQ/OQ/PQ computer validation for
systems in a designated GXP area
• Validation Plan (VP): to define the approach for
validation of a specific system for OQ, IQ, or PQ
• Validation Summary Report (VSR):
to describe results, issues, and status
Slide 5
Validation Package Model
Control of Software &
Platform Environment
Package Plan Verification/Validation Plan
Control of System & SW
Changes & Audit of
CSV Package, QMS
Package Summary Report Verification/Validation Summary Report
Standard CSV
package
SOPs, Training &
Materials for Human
Work with System
Prepared and maintained by a CSV Package Team for IQ, OQ, or PQ
Requirements &
Specifications for SW,
System & Services
Slide 24
1. System Control 2. Human Control 3. Testing Control
Validation Package Model
Control of Software &
Platform Environment
CSV Package Plan Verification/Validation Plan
Control of System & SW
Changes & Audit of
CSV Package, QMS
CSV Package Summary Report Verification/Validation Summary Report
Standard CSV
package
SOPs, Training &
Materials for Human
Work with System
Prepared and maintained by a CSV Package Team for IQ, OQ, or PQ
Requirements &
Spec.s for SW, System
& Services - SLA
Slide 30
1. System Control 2. Human Control 3. Testing Control
Test PlansStartup & Ongoing
Test Cases, Scripts,
Data & Result Logs
Test Summary
Reports
QMS = Quality Management System SLA = Service Level Agreement
Trace
Corporate Structure for GXP Compliance
MVP for a GXP
Group of systems
3. System level
CSV Packages
& procedures
1. Corporate policy and
procedures
2. Project/Group level
plans
Per system Users’ PQ
PackageLocal SOPs
•Dept.
•Work ProcessSystem SOPs
User SOPs
MVP for IT
platforms
Supplier’s
OQ Pkg.VP Pkg. Plan
VP Package Summary ReportValidation
Package Teams
Platform
IQ Pkg.
Global Regulations
& StandardsMHW, FDA, EU, ISO, IEEE...
CV Policy/VMP Company Management
CSV SOPsMVP for a
SW Project
IEEE Format for a Software V&V Plan
IEEE Std. 1012-1986
1. Purpose
State plan identifier. Describe system. State assumptions and scope for this plan. What’s included, excluded, and limited?
2. Referenced Documents
Identify key documents associated with plan activities.
3. Definitions
Define terms and acronyms needed to understand this Plan.
4. Verification & Validation (V&V) Overview
Team structure, master task schedule, assigned responsibilities, tools, techniques, and methodologies to be used for tasks.
Slide 25
IEEE Format for a Software V&V Plan
IEEE Std. 1012-1986
5. Life-Cycle Verification & Validation (V&V)
Management of V&V, concept phase, requirements phase, design phase, implementation phase, test phase, installation and checkout phase, operation and maintenance phase, retirement/decommission phase
6. Software Verification & Validation Reporting
Required reports, optional reports
7. Verification & Validation Administrative Procedures
Anomaly reporting and resolution, task iteration policy, deviation policy, control procedures, standards, practices, and conventions
Slide 26
1. Package Summary Report identifier - Unique ID traceable to associated Validation Plan.
2. Summary of all Life Cycle Tasks - Identifies CSV Package Team. Includes Task List from Validation Plan with updated status and roles responsible. References Audit Reports and/or Test or Package Summary Reports to cover Platform and Application Provider phases of Life Cycle.
3. Summary of Anomalies and Resolutions - Describes any deviations from Validation Plan and reasons why. Highlights any anomalies pending from the Test Summary Report.
CSV Package Summary Report
IEEE Std. 1012-1992 Adapted
Slide 55
CSV Package Summary Report
4. Assessment of overall software quality - Discusses
assumptions and limits to scope of CSV Package. Gives
highlights from Test Summary Report. Describes how Package
items are sufficient to assess system quality and user readiness
for Go-Live in a regulated environment.
5. Recommendations - Recommends a Go-Live release status
to the System Sponsor as Full Pass, Pass with Specified
Conditions, or System Fail.
6. Approvals - Names, titles, signatures, dates and meaning of
signatures.
7. Appendix - Table of Contents to list Package items.
IEEE Std. 1012-1992 Adapted
Slide 56
OECD GLP – Acceptance Testing
• “…there should be evidence that the
system was adequately tested for
conformance with acceptance criteria …
prior to being put into routine use.”
• “Formal acceptance testing requires the
conduct of tests following a pre-defined
plan and retention of all testing
procedures, test data, test results, a
formal summary of testing, and a record
of formal acceptance.”
Slide 27
IEEE Format for a Software Test Plan
Software Test Plan Outline - IEEE Std. 829-1983
1. Test plan identifier
2. Introduction
3. Test items
4. Features to be tested
5. Features not to be tested
6. Approach
7. Item pass/fail criteria
8. Suspension criteria & resumption requirements
Slide 28
IEEE Format for a Software Test Plan
Software Test Plan Outline - IEEE Std. 829-1983
9. Test deliverables
10. Testing tasks
11. Environmental needs
12. Responsibilities
13. Staffing and training needs
14. Schedule
15. Risks and contingencies
16. Approvals (IEEE Tel: 800-678-4333)
Slide 29
Test Plan Summary Report
1. Test Summary Report Identifier - Unique ID traceable to
associated Test Plan.
2. Summary - Describes items tested (application version), test
environment (platform system), and test documentation used (Test Cases,
Test Scripts, T.S. Envelope contents).
3. Variances - States any deviations from Test Plan or Test Scripts and
reason why.
4. Comprehensive Assessment - Discusses assumptions and limits
to scope of testing. Were scope of testing and results obtained sufficient to
assess system reliability? Discuss reasons for limits chosen.
IEEE Std. 829-1983 Adapted
Slide 37
5. Summary of Results - Gives table of testing results per Test Case.
Table of anomalies and their resolutions. List of outstanding issues and
risks (unresolved anomalies).
6. Evaluation - Pass/Fail conclusion based on test results and criteria
in the Test Plan.
7. Summary of Activities - Tester/Witness staffing, task list from
Test Plan with updated status.
8. Approvals - Names, titles, signatures, dates and meaning of
signatures.
9. Appendix - Table of Contents list for test documentation
Test Plan Summary Report
IEEE Std. 829-1983 Adapted
Slide 38
CSV Package Team Model – Any Package
Package Sponsor
Test Coordinator
Package Manager
(Pkg. QC)
CSV Package Team
Quality
AssuranceTeam Leader
Ad hoc
Members
Trains Team and
Audits CSV Pkg.
Funds and Approves
Validation Work
Team to Develop and
Maintain a CSV Pkg.
Slide 31 Reference Stokes Article – Part 4.
Software Supplier’s (SDLC) OQ Package
Code & Tools Mgt. Logs,
SDLC, Controlled SW
Devel. Platform, Tools IQ
OQ Verification Plan
SW Upgrade Plan, Bug
Tracking, Audit Logs,
QMS with SQC & QA
OQ Verification Summary Report
Standard SW supplier’s
CSV package
SW Engr. SOPs, Pgmr.
Trng, Product Manuals,
User Help Desk
Prepared by internal or external software supplier team
MRS, FRS, SDD, Code,
SDLC Docs., Support
Contract (SLA), (Escrow)
Slide 32
1. System Control 2. Human Control
Reference Stokes Article – Part 3.
Software Supplier’s (SDLC) OQ Package
Code & Tools Mgt. Logs,
SDLC, Controlled SW
Devel. Platform, Tools IQ
OQ Verification Plan
Test Cases, Scripts,
Data & Result Logs
Master Test Plan &
Sub Test Plans –
U,F,S,M,S,I,R*
SW Upgrade Plan, Bug
Tracking, Audit Logs,
QMS with SQC & QA
OQ Verification Summary Report
Standard SW supplier’s
CSV package
SW Engr. SOPs, Pgmr.
Trng, Product Manuals,
User Help Desk
Prepared by internal or external software supplier team
MRS, FRS, SDD, Code,
SDLC Docs., Support
Contract (SLA), (Escrow)
Slide 33
Test Summary
Report (per Plan)
*U,F,S,M,S,I,R = Unit, Function, Smoke, Module, System, Interface, Regression
1. System Control 2. Human Control 3. Testing Control
Trace
Platform System IQ Package
Data Ctr. Config. Mgt.
Logs, Ops. SOPs, System
Manuals & Security WI
IQ Validation Plan
Dis. Rec. Plan, Change
Control Log, Backup &
Archive Logs, Supplier
Records, IT Audit Log
IQ Validation Summary Report
Standard platform
CSV package
IT/IS CVs, Training
Records, Dept. SOPs,
WIs, Check Lists
Prepared and maintained by IS/IT Department Team
Slide 34
2. Human Control 1. System Control
Reference Stokes Article – Part 2.
SLA, SLA Monthly
Reports, Security
Logs, Help Desk Log
Platform URS
Platform System IQ Package
Data Ctr. Config. Mgt.
Logs, Ops. SOPs, System
Manuals & Security WI
IQ Validation Plan
Test Cases, Scripts,
Data & Result Logs
IQ Test Plan(s)
Startup & Ongoing
Dis. Rec. Plan, Change
Control Log, Backup &
Archive Logs, Supplier
Records, IT Audit Log
IQ Test Summary
Report(s)
IQ Validation Summary Report
Standard platform
CSV package
IT/IS CVs, Training
Records, Dept. SOPs,
WIs, Check Lists
Prepared and maintained by IS/IT Department Team
SLA, SLA Monthly
Reports, Security
Logs, Help Desk Log
Platform URS
Slide 35
3. Testing Control 1. System Control 2. Human Control
Trace
Application User’s PQ Package
Applic. Admin. SOP &
Applic. Config. Mgt. Logs
PQ Validation Plan
Change Control Log,
QA Audit Log, Supplier
Reports & BDG* Minutes
PQ Validation Summary Report
Standard user’s
CSV package
User Manuals, CVs,
Trng Records, Work
SOPs, WIs, Help Logs
Prepared and maintained by User Department(s) Team
Needs Analysis, RFP,
Contract, URS, User
Access Profiles, SLAs
Slide 36
1. System Control 2. Human Control
Reference Stokes Article – Part 1.
Application User’s PQ Package
Applic. Admin. SOP &
Applic. Config. Mgt. Logs
PQ Validation Plan
Test Cases, Scripts,
Data & Result Logs
PQ Test PlansStartup & Ongoing
Change Control Log,
QA Audit Log, Supplier
Reports & BDG* Minutes
PQ Validation Summary Report
Standard user’s
CSV package
User Manuals, CVs,
Trng Records, Work
SOPs, WIs, Help Logs
Prepared and maintained by User Department(s) Team
Needs Analysis, RFP,
Contract, URS, User
Access Profiles, SLAs
Slide 37
PQ Test Summary
Reports
* BDG = Business Decision Group
1. System Control 2. Human Control 3. Testing Control
Trace
Xcalibur CSV Package – PQ Testing
XCAL-VP01
XCAL-TC05
XCAL-TC04
XCAL-TC03
XCAL-TC02
XCAL-TC01
XCAL-TP01
XCAL-TPSR01
XTC1.TS1
XTC2.TS1-TS4
XTC3.TS1-TS5
XTC4.TS1
XTC5.TS1-TS3
XCAL-URS01XCAL-CMLOG
XCAL-SOP(s)
XCAL-Trning
Materials &
Records
XCAL-OGTP
XCAL-Supplier
Audit Certificate
XCAL-VP Pkg.
Audit Certificate
XCAL-VPSR01 CSV Package Summary Report
XCAL-System
Manuals &
Online HelpXCAL-Part 11
Gap Analysis
Mgt. Control
System Control Test Control User Control
GXP System Life Cycle Model
2.SYSTEM PLAN
What? URS
3.DESIGN
How? SDD
4.BUILD
Program or
Configure
5.TEST
Verify to SDD
& Release
6.COMMISSION
Accept &
Validate to URS
9.RETIRE
Decommission
& Replace
Test Fit
to Design
Test Fit to
Work Process
8.MAINTAIN
Fix & Modify
7.OPERATE
Use & Monitor
1.SYSTEM IDEA
Needs Analysis
Report & RFP
Application
Life Cycle
SDLC
RFP – Request
for proposal
URS - User
requirements
specification
SDD - Software
design
description
Platform System
1.
Configure
2.
Install
3.
Test
Test Fit to Install Specs.
PQ
OQ
IQ
System User
Software
Supplier
SDLC - Software
Development
Life Cycle
Ongoing Testing for Change Control
2.SYSTEM PLAN
What? URS
3.DESIGN
How? SDD
4.BUILD
Program or
Configure
5.TEST
Verify to SDD
& Release
6.COMMISSION
Accept &
Validate to URS
9.RETIRE
Decommission
& Replace
Test Fit
to Design
Test Fit to
Work Process
8.MAINTAIN
Fix & Modify
7.OPERATE
Use & Monitor
1.SYSTEM IDEA
Needs Analysis
Report & RFP
Application
Life Cycle
SDLC
Platform System
1.
Configure
2.
Install
3.
Test
Test Fit to Design & Work
SDLC - Software
Development
Life Cycle
Software
Supplier
System User
Change
Symbol
System Focus in “Operation” Phase
Fit ?User/
Owner
Software
Supplier
Patches &
Updates
Application
Life Cycle
Use & Monitor
Fix & Modify
Re-Accept
7. Operate
8. Maintain
6. Commission
Application
Support SLA
Platform
Support SLA
Help Desk
C.M. Log
Platform
Business Decision Group (BDG) - Prioritize change & resources
Applic.
9. Retire
1. System availability
2. Install & test updates
3. Backup & restore
4. Manage capacity
Part 11/GLP Package Model
Control of Instrument &
Computer Components-
Config. Mgt. Logs
Validation Plan for
Instrument Type A
Test Cases, Scripts,
Data & Result LogsControl of System
Changes & Audit of
CSV Pkge & Audit Trail
CSV Package Summary Report
Standard CSV
package
SOPs, Training &
Materials for Analyst
Work with System
Prepared and maintained by a CSV Package Team
Specifications for
Method, Instrument,
System & Services
System Control Human Control
Test Summary
Report(s)
Testing Control
Type A Test Plan(s)Startup & Ongoing
MVP – Lab X
Master Plan Package Model
Control of Software &
Platform Environment
Master Validation
Plan – IQ or PQ
Control of System
Changes & Audit of
Work Process, QMS
Validation Summary Report
SOPs, Training &
Materials for Human
Work with System
Prepared and maintained by a CSV Package Team
Work Process
Requirements for
System
1. System Control 2. Human Control 3. Testing Control
Master Test Plans
Startup & Ongoing
Test Cases, Scripts,
Data & Result Logs
Test Summary
Reports
QMS = Quality Management System SLA = Service Level Agreement
Trace
Mini VP Form
per System
Mini TP Form
per System
What is Common Sense?
• Working Definition:
Common sense is knowledge based on “native
good judgment” and personal experience. It is
that combination of “book” smart, “street” smart,
and “work” smart that a person can apply to
scoping computer validation projects.
Slide 15
Any Questions or Comments?
??
Common Sense Computer Validation
MVP Workshop – IVT April 2013
Thank You
MVP
Coffee and Case Study #1
Appendix 6.2-6.4 for IQ MVP Examples
MVP for a
Group of systems
3. System level
CSV Packages
& procedures
1. Corporate policy and
procedures
2. Project/Group level
plans
Per system Users’ CSV
PackageLocal SOPs
•Dept.
•Work ProcessSystem SOPs
User SOPs
MVP for IT
platforms
Supplier’s
CSV Pkg.Pkg. Plan
CSV Package Summary ReportCSV Package
Teams
Pltfm.
CSV
Pkg.
Global Regulations
& StandardsMHW, FDA, EU, ISO, IEEE...
CSV Policy Company Management
CSV SOPsMVP for a
Project
IQ MVP CASE
STUDY 6.2-6.4
MVP Case Study
6.2 Example of an MVP for IQ of Regulated Platform
Systems in a CRO (Pages 168 - 181)
6.3 Template Form for Mini VP Form to be used per
Platform System under MVP (Pages 182 – 188)
6.4 Example of a completed Mini VP form for a
Pharmacovigilance/Data Management (PVDM) platform
system under the MVP (Pages 189-196)
6.7 Example of an IQ Validation Summary Report for the
PVDM Platform System
Use Book Chapter 6 – The IT Infrastructure Role in ValidationComputer Validation: A Common Sense Guide by T. Stokes
MVP Case Study
1. What advantage does this MVP approach bring to the CRO?
2. Is there any disadvantage to working this way?
3. Did the Validation Summary Report give a full review of this
IQ validation effort?
4. Were all the Mini VP tasks completed?
5. What issues arose and how were they resolved?
6. Was the system recommended for release?
7. How do the Mini documents and Summary Report relate to
the IQ MVP?
8. Where might you want to apply an MVP method in your
own company?
Use Book Chapter 6 – The IT Infrastructure Role in ValidationComputer Validation: A Common Sense Guide by T. Stokes