validationof manufacturing execution systems · pdf fileraulsoto, msc, cqe ivt...

R A U L S O T O , M S C , C Q EI V T P H I L A D E L P H I A - A P R I L 2 0 1 6

VALIDATION OF MANUFACTURINGEXECUTION SYSTEMS (MES)

1M A N U F A C T U R I N G E X E C U T I O N S Y S T E M S

• The contents of this presentation represent the opinion of the speaker; and not necessarily that of his present or past employers.

2IVT APRIL - PHILADELPHIA (C) 2016 RAUL SOTO

ABOUT THE AUTHOR

• 20+ years of experience in the medical devices, pharmaceutical, biotechnology, and consumer electronic industries

• MS Biotechnology, emphasis in Biomedical Engineering• BS Mechanical Engineering• ASQ Certified Quality Engineer

• I have led validation / qualification efforts in many scenarios:• High-speed, high-volume manufacturing and packaging equipment• Enterprise resource planning applications (i.e. SAP)• IT network infrastructure• Quality Systems Software• Mobile applications• Laboratory : information systems and equipment / instruments• Product improvements, material changes, vendor changes

• Contact information:• Raul Soto [email protected]


4

Winner of Siemens’ 2016 Manufacturing

Star Award forCamStar MES

Upgrade project

http://camstar.industrysoftware.automation.siemens.com/user-groups/manufacturing-star-awards/#Johnson-Johnson

IVT APRIL - PHILADELPHIA (C) 2016 RAUL SOTO

OUTLINE

• Introduction

• What is an MES

• Benefits

• ANSI / ISA – 95

• eDHR

• Scope:

• What should be included in the Validation project

• SDLC Approach

• What is it

• Validation Deliverables

• Requirements, Design, Trace Matrix

• Testing Protocols and Strategies

• Interfaces

• Going Live

• Governance

• Hypercare


INTRODUCTION



WHAT IS AN MES

• Manufacturing Execution System• Takes in vast quantities of data coming from the PLCs

and SCADA/HMI systems• Converts that into useful information about production

ops• Scheduling• Materials handling• Quality samples


ERP MES CONTROLS


BENEFITS OF MES

• Provide complete history of manufactured lots/ batches/ units

• Electronic Device History Record / Batch Record

• Electronic approvals and audit trails

• Full traceability of product to raw materials and manufacturing process parameters at every stage

• Reduce use of paper


BENEFITS OF MES

• MES system is not just a piece of fancy technology

• The BUSINESS VALUE of MES comes from the integration of data from multiple information sources: production data, quality data, business data, controls data, etc.

• Materials consumed, equipment used, parametric data, exceptions, date / time stamps

• Enforces as-designed process

• Ensures the right procedures, operators, equipment materials are used

• Enforces right sequence of operations

• Can track equipment usage and communicate with eCMMS


ANSI / ISA – 95 CONTROL HIERARCHY LEVELS

• Level 0: Physical production process

• Level 1: Sensing/manipulation of production process

• Level 2: Monitoring, supervisory control and automated control of production process (i.e. SCADA HMI systems)

• Level 3: Workflow / recipe control to produce desired products, maintain records and optimize production process. Timeframe: shifts, hours, minutes, seconds (i.e. MES, LIMS)

• Level 4: Establish basic plant schedule – production, material use, delivery, shipping. Timeframe: month, weeks, days, shifts (i.e. ERP)


ANSI / ISA – 95 CONTROL HIERARCHY LEVELS

IVT December (c) 2016 Raul Soto 12

http://w

ww

.we

rum.c

om

/en/m

es/p

rod

ucts/p

as-x/isa

/isa.jsp

EDHR

• eDHR: Electronic Device History Records

• paperless, electronic systems within MES

• enforce production processes

• capture all information associated with as-built production records.

• provides the error-proofing and real-time visibility necessary to produce consistent product quality each and every time.


EDHR

• If product quality issues arise, eDHR can be used to contain suspect product, either in process or in the field, to take action to address the issue.

• The electronic records stored in eDHR yields information that can also help MD&D companies optimize production processes.



R A U L S O T O

I V T D E C E M B E R 2 0 1 6 A M S T E R D A M

SCOPE : WHAT SHOULD BE INCLUDED INTHE VALIDATION PROJECT

16


M A N U F A C T U R I N G E X E C U T I O N S Y S T E M S

SOFTWARE VALIDATION

• validation, software. (NBS) Determination of the correctness of the final program or software produced from a development project with respect to the user needs and requirements. Validation is usually accomplished by verifying each stage of the software development life cycle. See: verification, software.

• Source: FDA Glossary of Computer System Software Development Terminology http://www.fda.gov/iceci/inspections/inspectionguides/ucm074875.htm


SCOPE: SOFTWARE

• MES Application

• Databases / Data warehouse • (i.e. Oracle)

• Middleware • (i.e. Archestra)

• Interfaces • (more on this later!)

• ETL tools

• Reporting tools not part of the application • (i.e. Cognos or Business Objects)

• Reports created with these tools

• Other ancillary / support software


SCOPE: HARDWARE

• Servers (for multiple environments)

• Scanners

• Printers

• Bar code readers

• RFID systems

• others


SCOPE

• One or multiple sites?

• One or multiple manufacturing suites / lines?

• Are you also decommissioning a legacy MES system?

• Data migration and /or archival

• GAMP 5: COTS vs Configuration vs Customization

• Medical Devices : eDHR

• 21 CFR 820.184 - DEVICE HISTORY RECORD


SCOPE

• Define your project scope early on, document all changes

• Map your AS-IS business process flow (pre-MES), vs your TO-BE

• Include in scope any configuration and/or customizations required (i.e. reports, interfaces)

• Minimize custom-coding, maximize the use of configuration alternatives.

• Workflows


PROJECT PLANNING

• Workflows

• Involve key stakeholders and end users from the start

• Testing, validation, user training and documentation require time and resources. Make sure you factor that into your project scope from the start.


SDLC APPROACH

23


M A N U F A C T U R I N G E X E C U T I O N S Y S T E M S

WHAT IS AN SDLC

• SDLC : Systems Development Life Cycle

• Series of steps / phases that provide a model for development and lifecycle management of an application or piece of software.

• Software validation is NOT a one-shot deal.

• Validated software and its documentation require continuousupdates and improvements for the life of the system

• End-of-life has to be managed• Data migration or archival• Hardware decommissioning


TYPICAL SDLC VALIDATION DELIVERABLES


Validation Plan

Assessments

Requirements

Design

Development

SAT / FAT

IQ

OQ / System Testing

PQ / UAT

Traceability Matrix

Procedures

Validation Report

PURCHASING REQUIREMENTS

• NOT YET the same as your validation requirements

• Develop high level requirements to be used as part of the purchasing decision

• Vendor assessment should include compliance with these requirements

• Make sure you have a good idea of how much configuration and/or customization is needed to meet your main requirements• MINIMIZE the amount of customization / custom code

• These requirements will also be the basis for FAT / SAT

• SLA : Service Level Agreement


DEVELOPMENT / CONFIGURATION

• Development testing occurs before validation• COTS functionality + configuration

• Vendor or hired consultants

• Unit testing, test configuration, integration testing

• If custom coding is required, development testing should be more comprehensive

• Use bug tracking software (i.e. FogBugz, Redmine, Debbugs) to keep track of bug resolution

• Does not need to be formally documented, but IT HELPS

• Have Coding standards and follow them!


VALIDATION DELIVERABLES



TYPICAL LIST OF VALIDATION DELIVERABLES

• Assessments

• (Safety, 21CFR Part 11, Risk, Compliance,

Vendor etc.)

• Validation Plan

• Requirements

• Design (DS, FDS, TDS, DDS)

• Traceability Matrix

• Testing protocols and scripts

• Development *

• Integration *

• FAT / SAT *

• Installation

• OQ / PQ or System Testing / UAT

• Testing Reports

• Training

• Procedures

• Operational

• Administration

• Change Control

• Validation Report

IVT December (c) 2016 Raul Soto 29

* Do not fall strictly under validation

30

ASSESSMENTS

• Vendor Assessment

• Compliance Assessment (i.e. GxP, SOX, etc.)• 21 CFR Part 11 ER/ES (especially on Medical Devices)

• GAMP category

• Information Security

• Safety / EHS

• Risk Analysis / FMEA

• others


VALIDATION PLAN

• System Overview

• Scope of Validation

• Roles and Responsibilities

• Assessments Required

• Validation Deliverables, author and approvers

• Validation and Testing Strategy

• Acceptance Criteria

• How to deal with discrepancies, defects, changes in scope, changes in strategy


VALIDATION PLAN

• This is where you establish WHAT needs to be validated, and HOW it will be validated.

• Get everyone’s buy-in and commitment to the Validation Plan from the beginning

• Update as needed, following your organization’s formal change control process for documentation

• Define clear expectations for all roles

• Definitions of deliverables, author and approvers for each


VALIDATION REPORT

• Final scope

• List of Actual Deliverables• Document Numbers

• Date of Completion

• any additions or deviations from planned

• Summary of testing results per protocol; any additions or deviations

• Summary of defects and discrepancies, and resolution status

• Statement declaring the validated state of the system

• Release to Production


RESOURCES

• You will need resources from multiple disciplines (i.e. QA, IT, Engineering, Operations, technical writers, etc.)

• Plan which resources you need, when will you need them, and for how long

• This will help you budget for consultants, and borrow internal resources. “I need two validation resources, 25-30% of their time, during August and September” is a different conversation than “I need to borrow some of your people for the MES project”

• PM tools -such as MS Project- help


REQUIREMENTS

• Types:

• User Requirements

• Functional Requirements

• Regulatory Requirements

• Performance Requirements

• Security Requirements

• Classify by type and criticality

• Low criticality : nice to have

• Medium criticality: important but not critical

• High criticality: must have

• All regulatory requirements should be classified as HIGH criticality

36

• What you want your MES system to do


REQUIREMENTS

• Clear• Complete• Correct• Consistent• Concise• Prioritized

• Relevant• Feasible• Verifiable• Modifiable• Traceable• Unique

37

Requirements should be


REQUIREMENTS

Written definition of the MES software functions:

• Proposed Workflows and Use Cases

• How users will interact with the system

• Functions that the MES software will perform

• Expected system inputs and outputs

• Reports the system is expected to generate, contents

• Interfaces (user, external)

• Required alarms and checks

• Performance: data throughput, reliability, timing

• Coding standards


REQUIREMENTS

• Required response times

• Intended operating environment (hardware platform, software operating system, middleware, database software, etc.)

• All ranges, limits, defaults, specific values that the MES software requires

• Mirroring & Replication of main and transactional databases

• Backup, restore, disaster recovery

• User Roles that need to be defined in the system

• Any safety-related specifications

• Part 11: Electronic Records & Signatures requirements

• Dates (Manufacturing, Expiration): based on local time or UTC?39IVT APRIL - PHILADELPHIA (C) 2016 RAUL SOTO

REQUIREMENTS: UDI

Unique Device Identification

• US Initiative to assign a unique identifier to all medical devices

• Signed into law on 2007, mandatory since 2013 FDA Final Rule

• Label of a medical device must have a unique identifier

• Unique identifier should ID device through distribution and use

• Unique identifier must include lot / serial number

• Typical elements:• Graphics and symbols

• Regular and 2D barcodes

• Lot / batch / serial number, expiration date, date of manufacture



DESIGN

• Design Specifications describe the actual software solution as:

• Out-of-box / configured / custom-coded.

• Address all requirements, individually or in groups

• Design documentation may include one or more of the following:

42

Design Specification

Detailed Design Specification

Technical Design Specification

Database Design Specification

Software Design Specification

Hardware Design Specification

Architecture Design

Security Design

Interfaces


DESIGN

Design documentation should include, at least:

• System architecture

• Modules

• Screens

• Formulas, algorithms, and logic used

• Data structures, data flow diagrams

• Supporting software that is required for MES operation (i.e. Archestra)

• Hardware required

• Parameters that are measured or recorded


DESIGN

• Reports generated

• Definition of control and data variables, where used

• Messages: Errors, alarms, warnings

• Actual workflows and use cases, as configured (or coded)

• User roles, as configured

• Physical security

• Information security

• Actual reports as configured

• Interfaces, as configured


REQUIREMENTS TRACEABILITY MATRIX

• The RTM should enable us to trace:

• From individual Requirements to their specific Design Element(s) where the requirement is addressed

• From Design Elements to the specific Test Script(s) where they are being challenged.

45

• We should be able to trace back and forward

Requirement Design elements Testing


REQUIREMENTS TRACEABILITY MATRIX

• Requirements => start RTM draft

• Design documents => update RTM draft

• Test scripts => finalize RTM

• Final RTM should be approved before the system is released for production

• RTM should be updated as part of system change control every time requirements, design, or test scripts are changed, added, removed.


LIVING DOCUMENTS

• The following documents should be treated as “living documents” and maintained up to date throughout the life of the system

• Requirements • Design• Traceability Matrix• Risk Assessments (i.e. FMEA)


TESTING

48M A N U FA C T U R I N G E X E C U T I O N S Y S T E M S


TESTING

• Protocols, Test Scripts, Reports

• Pre- and post- approved

• Consistency in roles pre- and post- approving is important

• Avoid conflicts of interest: Approvers cannot execute, testers can’t approve

• Dependencies and order of testing must be clear, evident, obvious

• Enforce Good Documentation Practices (GDP)

• Enforce the use of standard templates

• Keep evidence (printouts, screenshots, labels, etc.)


TESTING ENVIRONMENTS

• Development / Sandbox • For DEV testing• Qualification not required

• Test / Staging• For your formal validation (OQ, System Testing, UAT)• Qualified• Functionally equivalent to your PROD environment

• Production Environment• Live system• Performance testing• Qualified


TESTING MODELS

• Installation / Operational / Performance Qualification (IQ / OQ / PQ)

• Installation Testing / System Testing / User Acceptance Testing


TESTING: IQ

• Installation Qualification

• Document / verify the correct installation and configuration of all software and hardware components, as per the Design Specification

• List actual software components and objects installed: name, version, location

• List actual hardware installed: name, model, quantity, S/N, location

• Turn-key test to ensure the system is ready for OQ / System testing


TESTING: OQ / PQ MODEL

Operational Qualification

• Ensure system can perform under controlled conditions, non-saleable product

• Document that system is installed and configured as per Design Specifications

• AND complies with requirements as per Requirements Specifications

• Testing will challenge all requirements, interfaces, reports, etc.

• Can test each subsystem individually

• Positive vs Negative testing

• Formal change control


TESTING: OQ / PQ MODEL

Performance Qualification / Production Qualification

• Test that system is able to function under normal manufacturing conditions

• End-to-End challenge of the complete process flow

• Normal mix of lots, products, operators, shifts, etc.

• Performance testing / load testing

• Can use saleable product


TESTING: SYSTEM TESTING / UAT MODEL

• System Testing

• Test and verify that the system as integrated is functionally complete

• Challenge compliance with functional and non-functional specifications

• Integration of MES with interfaces

• You can test individual functionalities, then end-end process workflows


SYSTEM TESTING

• Tests functional requirements, configuration, security, ER/ES, compliance• May include positive testing, negative testing, boundary testing, interface

testing

• Positive Testing:

• Ensures that system performs as intended, using normally expected inputs• Negative Testing:

• Ensures that system doesn’t accept invalid inputs• Boundary Testing:

• Challenges that performance is as expected when specific variables are set to their max / min values.

• Interface Testing:

• Tests that system components can pass data correctly to one another


TESTING: SYSTEM TESTING / UAT MODEL

• User Acceptance Testing

• Testing focused on user-related requirements

• Challenge that the MES system is capable of supporting your normal business process

• Test cases should address ease of use from the standpoint of operators, QC technicians, etc.


USER ACCEPTANCE TESTING

• Tests functional requirements, configuration, security, ER/ES, compliance• May include positive & negative testing, business process testing, end-to-end

testing, stress testing, performance testing

• Business Process Testing:

• Verifies that the system works as intended following business process flows

• End – to – End Testing:

• Verifies that the system is capable of supporting the intended process flows, from beginning to end; ensure data integrity and that the correct data passes between components and interfaces.

• Performance Testing:

• Verifies system stability, resource usage, and responsiveness under specific workloads.• Stress Testing:

• Tests system’s performance beyond the limits established in the specified requirements


TESTING EXECUTION

• Ensure test sets and scripts are executed in correct order

• GDP (Good Documentation Practices) are IMPORTANT

• Include evidence: screenshots, printouts, labels, etc.

• Changes to approved protocols / test scripts must follow formal documentation management (version up, re-approve, etc)

• Coordinate execution dates / times with owners of systems MES interfaces with

• Follow process for handling testing defects / deviations

• If you execute PQ / UAT with real product, QA must provide disposition of such product


TESTING TOOLS

• There are electronic testing tools that can be used in substitution of paper test scripts• Example: HP Quality Center, Valgenesys

• MUST validate these tools BEFORE you use them

• Pros: reduce GDP errors, standardize testing process, enforce use of correct templates, enforce approvers rules, manage test defects, can search for documents electronically

• Cons: administration, training, cost, less flexibility, maintenance, fixes / upgrades

• Tools become your official repository of validation documentation

• 21 CFR Part 11 full compliance may require additional wrap-around software solution (HPQC)


PROJECT CHANGE MANAGEMENT

• You must have a formal change control process while in the project, not just after go-live.

• Design freeze date – enforce it!

• Any design changes requested after this date => after Go Live

• Except for critical (regulatory, business) changes


CHANGE CONTROL

• All changes should be assessed, documented, pre-approved, tested, post-approved

• Assessment

• Nature and scope of the change

• Risk assessment

• Documentation that needs to be updated (Requirements, Design, RTM, FMEA, etc.)

• Testing

• Functional tests

• Regression tests


INTERFACES



INTERFACES

• Do NOT underestimate interfaces, especially to LIMS and your ERP.

• Define and document all your interfaces at the Design stage

• What data passes MES External system

• Test your interfaces configuration in Development testing

• Test your interfaces functionality explicitly in OQ / System Testing


SAMPLE INTERFACES DIAGRAM


POSSIBLE INTERFACES

• ERP (i.e. SAP, JD Edwards)

• Master production schedule

• Lot quantity and status

• Bill of materials

• LIMS (i.e. Labware, NuGenesis)• QC Audit data

• Manufacturing lines, SCADA / HMI (i.e. Wonderware, RSView)• Manufacturing orders

• Bill of materials

• Lot data

• Raw Materials Tracking software

• Data exchange of r.m. lots used

• Documentation eDMS (i.e. Documentum)• Change control and procedures data

• Product hold information

• Distribution Control system

• QC release data

• Lot data


GOING LIVE



GOVERNANCE

• Procedures

• System Administration

• Operation

• Change Control

• Maintenance

• Training

• Security and Access Control

• Configuration Management

• Backup / Restore / Disaster Recovery

• others


GOVERNANCE

• “Hypercare”

• SLA with vendors

• Change Control

• Support structure

• Periodic Review (and revalidation)


REFERENCES

• General Principles of Software Validation; Final Guidance for Industry and FDA Staff, Jan 11 2002

• http://www.fda.gov/downloads/RegulatoryInformation/Guidances/ucm126955.pdf

• ISPE GAMP5 Guide: Manufacturing Execution Systems

• http://ispe.org/gamp-good-practice-guide/manufacturing-execution-systems

• Understanding Manufacturing Execution Systems (MES)

• http://www.freedomcorp.com/solutions/qad/White%20Papers/MES%20White%20Paper.pdf

• Unique Device Identification (UDI)

• http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/UniqueDeviceIdentification/

• http://www.labelingnews.com/downloads/UDI_eBook.pdf

• Global Unique Device Identification Database (GUDID) Guidance for Industry and FDA Staff, June 27 2014

• http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM369248.pdf


REFERENCES

• US 21 CFR 820 Quality System Regulation

• http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?CFRPart=820

• US 21 CFR 11 Electronic Records and Signatures

• http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?CFRPart=11&showFR=1

• Annex 11 Computerised Systems. Jan 2011 updateEudraLex, Rules Governing Medicinal Products in the European UnionVol. 4 Good Manufacturing Practice, Medicinal Products for Human and Veterinary Use

• http://ec.europa.eu/health/files/eudralex/vol-4/annex11_01-2011_en.pdf

• ISO 13485 Medical devices -- Quality management systems -- Requirements for regulatory purposes

• http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=36786

• ANSI/ISA 95 and ANSI/ISA 88 Standards

• http://www.werum.com/en/mes/products/pas-x/isa/isa.jsp


QUESTIONS
