Medical Device Software Verification, Validation, and Compliance

David A. Vogel

A R T E C H H O U S E B O S T O N | L O N D O N

a r t e c h h o u s e . c o m

Contents

Preface xvii The Author's Background and Perspective of Validation xvii

Acknowledgments xxi

Background 1

CHAPTER 1 The Evolution of Medical Device Software Validation and the Need for This Book 3 The Evolution of Validation in the Medical Device Industry 3 Building a Language to Discuss Validation 4

Terminology is the Foundation 5 Correct Versus Consistent Terminology 6 Terminology Need Not Be Entertaining 7

Risk Management and Validation of Medical Device Software 8 About This Book 8 Goals of This Book 9 Intended Audience 10 Are You Wasting Time? 12

References 12

\_П/А1 I L i \ Z.

Regulatory Background 13 The FDA: 1906 Through 1990 13 The FDA Today (2009) 16 How the FDA Assures Safety, Efficacy, and Security 17 Quality System Regulations and Design Controls 20 Understanding How Regulation Relates to Getting the Job Done 22 Medical Devices Sold Outside the United States 24

References 25

VIII Contents

CHAPTER 3

The FDA Software Validation Regulations and Why You Should Validate Software Anyway

Why the FDA Believes Software Should Be Validated Therac 25 Building Confidence The Validation Regulations Why You Should Validate Software Anyway

References

CHAPTER 4 Organizational Considerations for Software Validation

Regulatory Basis of Organizational Responsibility A Model for Quality Systems

Roles, Responsibilities and Goals for the Quality System The Structure of the Quality System Quality System Processes Quality System Procedures

Thinking Analytically About Responsibility Untangling Responsibilities, Approvals, and Signatures What Happened to the Author? The Meaning of Approval: What That Signature Means

So, What Could Go Wrong with a Design Control Quality System? What Happened? Designing Streamlined RR&A Requirements for the Quality System

Fixing the Problem: Designing a Value-Added Approval/Signature Process Regulatory Basis for Treating Approvals and Signatures Seriously

Reference

CHAPTER 5 The Software (Development) Life Cycle

What Is a Software Life Cycle? Software Validation and SDLCs: The Regulatory Basis Why Are Software Development Life Cycle Models Important? What Do Different Software Development Life Cycle Models Look Like?

Waterfall and Modified Waterfall Sashimi Modified Waterfall Model Spiral Model Extreme Programming: Agile Development Models

How Do You Know What Life Cycle Model to Choose? How Do Software Development Life Cycles Relate to the Quality System? The ANSI/AAMI/IEC 62304:2006 Standard An Organization for the Remainder of This Book

Reference

CHAPTER 6

Verification and Validation: What They Are, What They Are Not 75

What Validation is NOT 75 Validation and Its Relationship to Verification and Testing 76 Software Validation According to Regulatory Guidance 79 Can Other Definitions of Validation Be Used? 81 User Needs and Intended Uses 82 Software Verification According to Regulatory Guidance 82 How Design Controls, Verification, and Validation Are Related 84 Validation Commensurate with Complexity and Risk 85 Is All Validation Created Equal? 87

Reference 87

CHAPTER 7 The Life Cycle Approach to Software Validation 89

Validation and Life Cycles 90 Combined Development and Validation Waterfall Life Cycle Model 91 A Validation Life Cycle Model 93 The Generic or Activity Track Life Cycle Model 95 Life Cycles and Industry Standards 102 Final Thoughts on Selecting an Appropriate Life Cycle Model 103

References 103

CHAPTER 8 Supporting Activities that Span the Life Cycle: Risk Management 105

Introduction to Activities Spanning the Life Cycle 105 Risk Management 106 Risk in the Regulations and Guidance Documents 107 ISO 14971: Application of Risk Management to Medical Devices 108 AAMI's TIR32:2004: Medical Device Software Risk Management 110 Risk and the IEC 62304 Standard on Life Cycle Processes 111 IEC/TR 80002-1: Application of 14971 to Medical Device Software 112 The Risk Management Process 112 The Language of Risk Management 113 Risk Management Outputs 114

The Risk Management Plan 114 The Risk Management File 115

Risk Management Concepts and Definitions 115 Risk Management Activities 117

Risk Analysis 117 Qualitative Probability Analysis 122 Ignoring Probability 123 Qualitative Probabilities 123

Risk Evaluation 129 Risk Control 130

Overall Residual Risk Evaluation 134

Contents

Summary 140 References 141

CHAPTER 9 Other Supporting Activities: Planning, Reviews, Configuration Management, and Defect Management 143

Planning 143 Design and Development Planning 143 Why Planning Is Important 144 How Many Plans Are Required? 145 Plan Structure and Content 147 What Does a Plan Look Like? 148 Evolving the Plan 152

Configuration Management 153 Regulatory Background 153 Why Configuration Management? 154 What Goes into a Configuration Management Plan? 155

Defect (and Issue) Management 160 Regulatory Background 161 Why Defect Management Plans and Procedures Are Important 161 Relationship to Configuration (Change) Management 161 Planning for Defect Management 165

Reviews 167 Regulatory Background 167 Why the Focus on Reviews? 168 What Is Meant by a Review? 171 Who Should Be Participating in the Reviews? 172 How Reviews Are Conducted 173

Traceability 177 Why Traceability? 177 Regulatory Background 178 Traceability Beyond the Regulatory Guidance 182 Practical Considerations: How It Is Done 185 Trace Tools 185 Trace Mapping 188 Can Traceability Be Overdone? 189 References 189

Validation of Medical Device Software 191

CHAPTER 10 The Concept Phase Activities 193

The Concept Phase 193 Regulatory Background 194 Why a System Requirements Specification Is Needed 195 Validation Activities During the Concept Phase 196

Contents XI

Make or Buy? Should Off-the-Shelf (OTS) Software Be Part of the Device? 198 The System Requirements Specification 200 Who Is the Intended Audience? 200 What Information Belongs in an SyRS? 201

How Are System Requirements Gathered? 204 Further Reading 205

Select Bibliography 205

The Software Requirements Phase Activities 207

Introduction 208 Regulatory Background 208 Why Requirements Are So Important 210 The Role of Risk Management During Requirements Development 214 Who Should Write the Software Requirements? 215 The Great Debate: What Exactly Is a Requirement? 217 Anatomy of a Requirement 219 How Good Requirements Are Written 223 Summary 231

References 231

CHAPTER 12 The Design and Implementation Phase Activities 233

Introduction 233 Regulatory Background 234 Validation Tasks Related to Design Activities 236

The Software Design Specification (Alias the Software Design Description) 236 Evaluations and Design Reviews 239 Communication Links 239 Traceability Analysis 240 Risk Management 246

Validation Tasks Related to Implementation Activities 247 Coding Standards and Guidelines 248 Reuse of Preexisting Software Components 248 Documentation of Compiler Outputs 249 Static Analysis 250 References 251

CHAPTER 13 The Testing Phase Activities 253

Introduction 253 Regulatory Background 253 Why We Test Software 255 Defining Software Testing 256

Testing Versus Exercising 257 The Psychology of Testing 258

XII Contents

Levels of Testing 260 Unit-Level Testing 261 Unit-Level Testing and Path Coverage 263 McCabe Cyclomatic Complexity Metric and Path Coverage 263 Other Software Complexity Metrics and Unit Test Prioritization 267 Integration-Level Testing 267 Device Communications Testing 269 System-Level Software Testing 272 System-Level Verification Testing Versus Validation Testing 274

Testing Methods 275 Equivalence Class Testing 276 Boundary Value Testing 279 Calculations and Accuracy Testing 282 Error Guess Testing 286 Ad Hoc Testing 287 Captured Defect Testing 288 Other Test Methods 289

Test Designs, Test Cases, and Test Procedures 290 Managing Testing 295

The Importance of Randomness 295 Independence 296 Informal Testing 297 Formal Testing 298 Regression Testing 300

Automated Testing 302 Summary 303

References 304 Select Bibliography 304

CHAPTER 14 The Maintenance Phase Validation Activities 305

Introduction 305 A Model for Maintenance Activities 308

Software Release Activities: Version n 309 Collection of Post-Market Data 312

Process and Planning 313 Sources of Post-Market Data 313

Analysis 315 The Maintenance Software Development Life Cycle(s) 318

Software Development and Validation Activities 320 Software Release Activities: Version n + 1 321

References 321

Validation of Nondevice Software 323

(-ontents XIII

CHAPTER 15

Validating Automated Process Software: Background 325

Introduction 325 Regulatory Background 326 Nondevice Software Covered by These Regulations 330 Factors that Determine the Nondevice Software Validation Activities 332

Level of Control 332 Type of Software 334 Source of the Software 334 Other Factors That Influence Validation 335

Risk 336 Size and Complexity 336 Intended Use 336 Confidence in the Source of the Software 337 Intended Users 337 Industry Guidance 340

AAMI TIR36:2007: Validation of Software for Regulated Processes 341 GAMP 5: Good Automated Manufacturing Practice 341

Who Should Be Validating Nondevice Software? 342 Reference 343

CHAPTER 16 Planning Validation for Nondevice Software 345

Introduction 345 Choosing Validation Activities 346 Do-It-Yourself Validation or Validation for Nonsoftware Engineers 347 The Nondevice Software Validation Spectrum 349 Life Cycle Planning of Validation 350 The Nondevice Software Validation Toolbox 352

Product Selection 354 Supplier Selection 354 Known Issue Analysis 355 Safety in Numbers 355 Third-Party Validation 356 Output Verification 357 Backup, Recovery, and Contingency Planning 358 Security Measures 359 Training 360

The Validation Plan 360 Reference 361

CHAPTER 17 Intended Use and the Requirements for Fulfilling Intended Use 363

Introduction 363 Intended Use 364

Why It Is Necessary to State Intended Use 364 Intended Use and Validation of Nondevice Software 365

XIV Contents

Contents of a Statement of Intended Use 365 Determining Intended Use 366

Requirements for Fulfilling the Intended Use 369 Requirements for Custom-Developed Software 369 Requirements for Acquired Software 370 Information Content of Requirements 370 Example: Intended Use and Requirements for Validation of a Text Editor 372

CHAPTER 18 Risk Management and Configuration Management of Nondevice Software Activities that Span the Life Cycle 375

Risk Management 375 Applying the 14971 Risk Management Process to Nondevice Software 375 Harm 376 Risk, Severity, and Probability 378 Managing the Risk 382 Controlling the Process to Reduce Risk 383 Risk Acceptability 383 Detectability 387

Configuration Management for Nondevice Software 387 Why Configuration Management Is Important 388 Configuration Management Planning 389 Configuration Management Activities 391 References 392

CHAPTER 19 Nondevice Testing Activities to Support Validation 393

Why Test—Why Not To Test 393 Testing as a Risk Control Measure 395 Regulatory Realities 395 Testing Software That Is Acquired for Use 396 IQ, OQ, and PQ Testing 397 Validation of Part 11 Regulated Software 399 Summary 400

CHAPTER 20 Nondevice Software Maintenance and Retirement Activities 401

Maintenance Activities 401 Release Activities 402 Post-Release Monitoring 403 Risk Analysis and Risk Management 404 Security 405

Retirement of Software 406 About the Author 409

Index 411