Download - Software validation do's and dont's may 2013
![Page 2: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/2.jpg)
© Copyright John Cachat
Housekeeping
Phones are muted
Use the question
block for questions Copy of presentation
available upon request
2
![Page 3: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/3.jpg)
© Copyright John Cachat
About John M. Cachat
• Driving Business Performance
– Helping companies align their business and technology
– Focus on people, process, and then the technology
– Subject matter expert on business process management
– On-going research into next generation of technology for enterprise
systems
• 28 years experience in enterprise systems
– USAF Research Project (1985)
– Founder of enterprise quality software company (1988)
– Chair of ASQ technical committee on computerizing quality (1992)
• Trusted advisor to global organizations, government agencies,
and professional groups
http://www.linkedin.com/in/johncachat
3
![Page 4: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/4.jpg)
© Copyright John Cachat
Today’s Discussion
• Discusses how to solve several software validation issues, including: – Requirements
– Defect Prevention
– Time and Effort
– Validation Coverage
– Software Life Cycle
– Plans & Procedures
– Software Validation After a Change
– Independence of Review
– Flexibility and Responsibility
4
![Page 5: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/5.jpg)
© Copyright John Cachat
What is Driving You?
• Business Excellence
– Faster Product Launch
– Lower Costs (CMMI view)
• Industry Requirements (i.e., FDA)
– Compliance
• Can you have both?
5
![Page 6: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/6.jpg)
© Copyright John Cachat
Software Requirements Specification
(SRS)
• A complete description of the behavior of a system to be developed.
• It includes a set of use cases that describe all the interactions the users will have with the software.
• Use cases are also known as functional requirements. In addition to use cases, the SRS also contains non-functional (or supplementary) requirements.
• Non-functional requirements are requirements which impose constraints on the design or implementation (such as performance engineering requirements, standards, or design constraints
6
![Page 7: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/7.jpg)
© Copyright John Cachat
Let’s Talk Business
• Do I have to validate all software?
– NO
• What software do I have to validate?
– Based on Risk
• To end users of your product
• To your company
7
![Page 8: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/8.jpg)
© Copyright John Cachat
Manufacturing vs. Software
• Same Concepts? Yes
– Prevention vs. Detection
– Cannot inspect quality into the result
– Total cost of quality (prevention, inspection, failure)
• Same Thing? Not really, software is
– Very easy to change, quickly
– One change can impact a lot
– Hard to inspect everything, literally everything
8
![Page 9: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/9.jpg)
© Copyright John Cachat
Build, Buy, Use – Impacts Everyone!
9
www.fda.gov/downloads/.../Guidances/ucm126955.pdf
![Page 10: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/10.jpg)
© Copyright John Cachat
Major Sections
• Section 1. Purpose
• Section 2. Scope
• Section 3. Context for Software Validation
• Section 4. Principles of Software Validation
• Section 5. Activities and Tasks
• Section 6. Validation Of Automated Process
Equipment And Quality System Software
10
![Page 11: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/11.jpg)
© Copyright John Cachat
Section 1. Purpose
• Provide general validation principles that the
FDA considers to be applicable to the
validation of medical device software or the
validation of software used to design,
develop, or manufacture medical devices.
11
Or pharmaceuticals, or cars, or airplanes,
or anything that if it does not work,
people may die, or get sick, or ……
![Page 12: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/12.jpg)
© Copyright John Cachat
Section 2. Scope
• The scope of this guidance is somewhat
broader than the scope of validation in the
strictest definition of that term.
• Planning, verification, testing, traceability,
configuration management, and many other
aspects of good software engineering
discussed in this guidance are important
activities that together help to support a final
conclusion that software is validated.
12
![Page 13: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/13.jpg)
© Copyright John Cachat
Section 2. Scope
• Based on the intended use and the safety risk
associated with the software to be
developed, the software developer should
determine the specific approach, the
combination of techniques to be used, and
the level of effort to be applied.
• Recommends an integration of software life
cycle management and risk management
activities.
13
![Page 14: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/14.jpg)
© Copyright John Cachat
Section 2. Scope
• Validate these:– Software used as a component of a medical device;
– Software that is itself a medical device
– Software used in the production of a device and
– Software used in implementation of the device manufacturer's quality system
• What about these?– Accounting
– Plant floor scheduling
– Microsoft desktop software
– Plant floor automation
14
![Page 15: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/15.jpg)
© Copyright John Cachat
Section 2. Scope
• THE LEAST BURDENSOME APPROACH
• Clarification
– Software validation process should not be
confused with any other validation requirements,
such as process or product validation
– Does not cover any specific safety or efficacy
issues with respect to product being
manufactured
15
![Page 16: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/16.jpg)
© Copyright John Cachat
Section 3.
Context for Software Validation
• 3.1. Definitions and Terminology
– 3.1.1 Requirements and Specifications
– 3.1.2 Verification and Validation
– 3.1.3 IQ/OQ/PQ
• 3.2. Software Development as Part of System Design
• 3.3. Software is Different from Hardware
• 3.4. Benefits of Software Validation
• 3.5 Design Review
16
![Page 17: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/17.jpg)
© Copyright John Cachat
3.1. Definitions and Terminology
17
http://www.fda.gov/iceci/inspections/inspectionguides/ucm074875.htm
![Page 18: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/18.jpg)
© Copyright John Cachat
3.1.1 Requirements and Specifications
• A requirement can be any need or expectation for a system or for its software.
• Requirements reflect the stated or implied needs of the customer, and may be market-based, contractual, or statutory, as well as an organization's internal requirements.
• There can be many different kinds of requirements (e.g., design, functional, implementation, interface, performance, or physical requirements).
• A specification is defined as “a document that states requirements.”
18
![Page 19: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/19.jpg)
© Copyright John Cachat
3.1.2 Verification and Validation
• Treats “verification” and “validation” as separate and distinct terms
• Software verification provides objective evidence that the design outputs of a particular phase of the software development life cycle meet all of the specified requirements for that phase.
• Software validation to be “confirmation by examination and provision of objective evidence that software specifications conform to user needs and intended uses, and that the particular requirements implemented through software can be consistently fulfilled.”
19
![Page 20: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/20.jpg)
© Copyright John Cachat
3.1.2 Verification and Validation
• Software verification - a developer cannot test forever & testing does not guarantee no defects
• Software validation - it is hard to know how much evidence is enough.
• Software validation is a matter of developing a “level of confidence”
• How much “confidence?” How much risk?
20
![Page 21: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/21.jpg)
© Copyright John Cachat
3.1.3 IQ/OQ/PQ
• Installation qualification (IQ) - the system has been built and installed correctly and that all required supporting services are available and connected correctly.
• Operational qualification (OQ) - demonstrate that the newly acquired software functions as expected; that all parts and components operate correctly
• Performance qualification (PQ) - demonstrate compliance with all requirements given in the User Requirements Specification document
21
![Page 22: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/22.jpg)
© Copyright John Cachat
3.2. Software Development as Part of System Design
• Software validation must be considered within the context of the overall design validation for the system
• End user rarely cares about the software
• The user's needs and intended uses from which the product is developed
– Correct blood pressure reading
– Anti-lock brakes work
– Airplane controls work
22
![Page 23: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/23.jpg)
© Copyright John Cachat
3.3. Software is Different from Hardware
• Seemingly insignificant changes in software code can create unexpected and very significant problems elsewhere in the software program The vast majority of software problems are traceable to errors made during the design and development process.
• One of the most significant features of software is branching, i.e., the ability to execute alternative series of commands, based on differing inputs.
• Software also has the speed and ease with which it can be changed concern regarding change management
• Software failures often occur without advanced warning (no noise, vibration, etc)
23
![Page 24: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/24.jpg)
© Copyright John Cachat
3.3. Software is Different from Hardware
• Because of its complexity,
the software development
process should be
controlled more tightly
than hardware.
24
![Page 25: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/25.jpg)
© Copyright John Cachat
3.4. Benefits of Software Validation
• This is the business part
– decreased failure rates
– fewer recalls and corrective actions
– less risk to patients and users, and
– reduced liability to manufacturers
25
This Car Runs on Code
It takes dozens of microprocessors running
100 million lines of code to get a premium
car out of the driveway.
As Much Software Code as an Airbus A380
![Page 26: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/26.jpg)
© Copyright John Cachat
3.5 Design Review
• Design reviews are documented,
comprehensive, and systematic
examinations of a design to evaluate the
adequacy of the design requirements,
to evaluate the capability of the design
to meet these requirements, and to
identify problems.
• FMEA - Failure mode effect analysis
• SET – Success every time
26
![Page 27: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/27.jpg)
© Copyright John Cachat
FMEA
27
![Page 28: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/28.jpg)
© Copyright John Cachat
Section 4.
Principles of Software Validation 4.1. Requirements
4.2. Defect Prevention
4.3. Time and Effort
4.4. Software Life Cycle
4.5. Plans
4.6. Procedures
4.7. Software Validation after a Change
4.8. Validation Coverage
4.9. Independence of Review
4.10. Flexibility and Responsibility
28
![Page 29: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/29.jpg)
© Copyright John Cachat
Section 4.
Software Validation Summary
• Software testing is a necessary activity.
• In most cases software testing by itself is not sufficient to establish confidence that the software is fit for its intended use
• Validation coverage should be based on the software's complexity and safety risk - not on firm size or resource constraints.
• The final conclusion that the software is validated should be based on evidence collected from planned efforts conducted throughout the software lifecycle
• Whenever software is changed, a validation analysis should be conducted not just for validation of the individual change, but also to determine the extent and impact of that change on the entire software system.
29
![Page 30: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/30.jpg)
© Copyright John Cachat
Section 5. Activities and Tasks
5.1. Software Life Cycle Activities
5.2. Typical Tasks Supporting Validation
5.2.1. Quality Planning
5.2.2. Requirements
5.2.3. Design
5.2.4. Construction or Coding
5.2.5. Testing by the Software Developer
5.2.6. User Site Testing
5.2.7. Maintenance and Software Changes
30
![Page 31: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/31.jpg)
© Copyright John Cachat
Software Life Cycle
• Quality Planning
• System Requirements Definition
• Detailed Software Requirements Specification
• Software Design Specification
• Construction or Coding
• Testing
• Installation
• Operation and Support
• Maintenance - Change Control, Revision Control
• Retirement
31
![Page 32: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/32.jpg)
© Copyright John Cachat
Quality Planning Tasks
• Risk Management Plan
• Configuration Management Plan
• Software Quality Assurance Plan (Software Verification & Validation Plan)
• Verification and Validation Tasks, and Acceptance Criteria
• Schedule and Resource Allocation
• Reporting Requirements
– Formal Design Review Requirements
– Other Technical Review Requirements
• Problem Reporting and Resolution Procedures
• Other Support Activities
32
![Page 33: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/33.jpg)
© Copyright John Cachat
5.2.2 Requirements• All software system inputs;
• All software system outputs;
• All functions that the software system will perform;
• All performance requirements that the software will meet, (e.g., data throughput, reliability, and timing);
• Definition of all external and user interfaces, as well as any internal software-to-system interfaces;
• How users will interact with the system;
• What constitutes an error and how errors should be handled;
• Required response times;
• The intended operating environment for the software, if this is a design constraint
• All ranges, limits, defaults, and specific values that the software will accept; and
• All safety related requirements, specifications, features, or functions that will be implemented in software.
33
![Page 34: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/34.jpg)
© Copyright John Cachat
Testing Coverage
• Statement Coverage
• Decision (Branch) Coverage
• Condition Coverage
• Multi-Condition Coverage
• Loop Coverage
• Path Coverage
• Data Flow Coverage
34
![Page 35: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/35.jpg)
© Copyright John Cachat
Section 6. Validation Of Automated Process Equipment
And
Quality System Software
• 6.1. How Much Validation Evidence Is Needed?
• 6.2. Defined User Requirements
• 6.3. Validation of Off-the-Shelf Software and
Automated Equipment
35
![Page 36: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/36.jpg)
© Copyright John Cachat
6.1. How Much Validation Evidence Is
Needed?• The level of validation effort should be
commensurate with the risk posed by the automated operation
• The extent of validation evidence needed for such software depends on the device manufacturer's documented intended use of that software
• COTS - consider auditing the vendor's design and development methodologies used in the construction of the software and assess the development and validation documentation generated for the COTS software
36
![Page 37: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/37.jpg)
© Copyright John Cachat
IEEE Technical Resources
• SRS - Software Requirements Specification IEEE 830
• SQAP - Software Quality Assurance Plan IEEE 730
• SCMP- Software Configuration Management Plan IEEE 828
• STD - Software Test Documentation IEEE 829
• SVVP - Software Validation & Verification Plan IEEE 1012
• SDD - Software Design Description IEEE 1016
• SPMP - Software Project Management Plan IEEE 1058
37
![Page 38: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/38.jpg)
© Copyright John Cachat
Software Best Practices
• Develop software iteratively
• Manage requirements
• Use component-based architectures
• Visually model software
• Verify and validate
• Software change control process
• Document, document, document
38
![Page 39: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/39.jpg)
© Copyright John Cachat
Summary
• You DO NOT have to
validate all software
• You validate based on risk
• Software is not the same as
hardware
• Plan for the entire software
life cycle
39
![Page 40: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/40.jpg)
© Copyright John Cachat
About Us
John Cachat
Contact
Proven expertise in business
information systems
Rapid Solution
Development™ process
ServicesAssess Current Status
Develop Short and Long
Term Plans
Develop Specific Solutions
to Your Problems
Assist in ROI Analysis
40
![Page 41: Software validation do's and dont's may 2013](https://reader033.vdocuments.net/reader033/viewer/2022051210/54bd67994a7959c74e8b45bb/html5/thumbnails/41.jpg)
© Copyright John Cachat
Software Validation: The Do’s and Don’ts
&
Contact:
John Cachat
Copy of Presentation
&
Request a Demo
Visit:
http://peproso.com/webinars
Future Webinars
41