network infrastructure validation conference @upra (2003)

11

COMPUTER COMPUTER VALIDATIONVALIDATION

TESTING FOR TESTING FOR IT AND IT AND

NETWORK NETWORK EQUIPMENTEQUIPMENT

Presented by:Raúl Soto, BSMEComputer Validation Team LeaderAstraZeneca Ltd – IPR Pharmaceuticals

[email protected][email protected]

22

Presentation OverviewPresentation Overview

1. 1. IntroductionIntroduction•• Why Validate Computer Systems?Why Validate Computer Systems?•• Computer Validation General PrinciplesComputer Validation General Principles

2. 2. Areas to be coveredAreas to be covered

•• Servers and Network Equipment & Systems ValidationServers and Network Equipment & Systems ValidationIQ and OQ testsIQ and OQ tests

•• Software ValidationSoftware ValidationGAMP software categoriesGAMP software categoriesSoftware Development testingSoftware Development testing

3. 3. Wrapping UpWrapping Up

4. Q & A4. Q & A

33

DisclaimerDisclaimer

This presentation and all This presentation and all

opinions therein opinions therein

are solely the responsibility are solely the responsibility

of the author; of the author;

and not that of the UPR and not that of the UPR

system system

or or AstraZenecaAstraZeneca PLC.PLC.

44

Why validate Computer Systems?Why validate Computer Systems?

55

Why validate Computer Systems?Why validate Computer Systems?

It’s the LAW:It’s the LAW:

•• Code of Federal Regulations (CFR)Code of Federal Regulations (CFR)

It makes good business sense:It makes good business sense:

•• Understanding of your processes / systemsUnderstanding of your processes / systems

•• Improved operational efficiencyImproved operational efficiency

•• Reduced risk of failureReduced risk of failure

•• Maintenance of quality standardsMaintenance of quality standards

66

Validation is a Regulatory RequirementValidation is a Regulatory RequirementFDA Requirements for Validation of Computerized SystemsFDA Requirements for Validation of Computerized Systems

Examples:Examples:

21 CFR 211.68 Automatic, mechanical, and electronic equipment21 CFR 211.68 Automatic, mechanical, and electronic equipment

21 CFR 820.70 Production and Process Controls21 CFR 820.70 Production and Process Controls

21 CFR 11.10 Controls for Closed Systems21 CFR 11.10 Controls for Closed Systems

77

Computer ValidationComputer ValidationGeneral PrinciplesGeneral Principles

What is a “Validation” ?What is a “Validation” ?

•• “Establishing “Establishing documented evidencedocumented evidence which provides a which provides a high high degree of assurance degree of assurance that a process, equipment, or system will that a process, equipment, or system will consistently fulfill its intended purpose, meeting its consistently fulfill its intended purpose, meeting its predeterminedpredetermined specifications and quality attributes.”specifications and quality attributes.”

-- FDA Guidelines on General FDA Guidelines on General Principles of Process ValidationPrinciples of Process Validation(1987)(1987)

88


Which Systems Require Validation?Which Systems Require Validation?

•• Systems that automate processes regulated by Systems that automate processes regulated by GxPsGxPs

•• Systems with an impact on product Systems with an impact on product qualityquality, , safetysafety, , identityidentity, , efficacyefficacy, or , or puritypurity..

•• Systems used to make quality decisionsSystems used to make quality decisions

•• Systems in scope for 21 CFR Part 11Systems in scope for 21 CFR Part 11

99


Validation vs. QualificationValidation vs. Qualification

•• We We validate validate systems that automate systems that automate GxPGxP--regulated processes.regulated processes.

Validation is the overall combination of plans, activities, Validation is the overall combination of plans, activities, documents, and approvalsdocuments, and approvals

Validation includes one or more qualificationsValidation includes one or more qualifications

•• QualificationsQualifications are test protocols that verify the system or are test protocols that verify the system or components of the systemcomponents of the system

Infrastructure entities, such as networks or data centers, are Infrastructure entities, such as networks or data centers, are qualified for use as part of your overall system validation.qualified for use as part of your overall system validation.

1010


Qualification ProtocolQualification Protocol

•• Qualification tests should be contained in a Qualification tests should be contained in a protocol protocol document. document.

•• This document should be approved by the Management of the areas This document should be approved by the Management of the areas impacted by the validation, and by Quality Assurance, BEFORE anyimpacted by the validation, and by Quality Assurance, BEFORE anyvalidation testing occurs.validation testing occurs.

•• The protocol should contain:The protocol should contain:

overview of the systemoverview of the systemdescription of the validation testing strategydescription of the validation testing strategydetailed description of all tests to be performeddetailed description of all tests to be performedprepre--established acceptance criteria for all testsestablished acceptance criteria for all tests

1111


Installation Qualification (IQ) ProtocolInstallation Qualification (IQ) Protocol

•• Documented verification that a system is installed according to Documented verification that a system is installed according to written and prewritten and pre--approved specifications.approved specifications.

Operation Qualification (OQ) ProtocolOperation Qualification (OQ) Protocol

•• Documented verification that a system operates according to Documented verification that a system operates according to written and prewritten and pre--approved specifications throughout all approved specifications throughout all specified operating ranges.specified operating ranges.

Performance Qualification (PQ) ProtocolPerformance Qualification (PQ) Protocol

•• Documented verification that a system is capable of Documented verification that a system is capable of performing or controlling the activities of the process it is performing or controlling the activities of the process it is required to perform or control, according to written and prerequired to perform or control, according to written and pre--approved specifications, while operating in its specified approved specifications, while operating in its specified operating environment.operating environment.

1212

Computer ValidationComputer Validation* Main Areas ** Main Areas *

Industrial Automation EquipmentIndustrial Automation Equipment

•• Machines or lines controlled by Programmable Logic Controllers Machines or lines controlled by Programmable Logic Controllers ((PLCsPLCs), ), ControlLogixControlLogix, , DeviceNetDeviceNet, PCs, or other computerized control , PCs, or other computerized control systemssystems

•• Computerized Vision or inspection systemsComputerized Vision or inspection systems

SoftwareSoftware

•• Operating Systems, firmware, or software packages (cannedOperating Systems, firmware, or software packages (canned--offoff--thethe--shelf, configurable, or customshelf, configurable, or custom--made)made)

Networks & Related EquipmentNetworks & Related Equipment

•• Servers, Routers, switches, cabling, workstationsServers, Routers, switches, cabling, workstations•• DNS service, Domain Controllers, DHCP systemsDNS service, Domain Controllers, DHCP systems•• WAN circuitsWAN circuits

1313

IT NETWORK IT NETWORK EQUIPMENT EQUIPMENT VALIDATIONVALIDATION

1414

IT & Network Equipment ValidationIT & Network Equipment Validation

What is a network (as far as the FDA is concerned) ?What is a network (as far as the FDA is concerned) ?

•• 1. (ISO) An arrangement of nodes and interconnecting 1. (ISO) An arrangement of nodes and interconnecting

branches. 2. A system [transmission channels and supporting branches. 2. A system [transmission channels and supporting

hardware and software] that connects several remotely located hardware and software] that connects several remotely located

computers via telecommunications.computers via telecommunications.(Source: FDA (Source: FDA -- Glossary of Computerized System and Software Development Glossary of Computerized System and Software Development

Terminology, Aug 1995)Terminology, Aug 1995)

•• Includes:Includes:

Supporting hardware (e.g. servers, workstations, transmission Supporting hardware (e.g. servers, workstations, transmission

channels)channels)

Supporting software (e.g. network operating system)Supporting software (e.g. network operating system)

Processes and procedures (e.g. change management)Processes and procedures (e.g. change management)

People (e.g. administrators, auditors)People (e.g. administrators, auditors)

1515

IT & Network Equipment ValidationIT & Network Equipment ValidationWhy must networks be validated ?Why must networks be validated ?

•• Our industry produces two critical outputs: Our industry produces two critical outputs: medical products medical products

and and datadata..

•• Medical products are supported and marketed based upon the Medical products are supported and marketed based upon the

quality and meaning of the underlying data.quality and meaning of the underlying data.

•• The integrity of this data must be assured and maintained.The integrity of this data must be assured and maintained.

•• The validation process provides the mechanism for assuring The validation process provides the mechanism for assuring

and maintaining data and process integrity.and maintaining data and process integrity.

1616


Why must networks be validated ?Why must networks be validated ?

•• Networks are systems that are actively involved in creating, Networks are systems that are actively involved in creating,

modifying, maintaining, archiving, retrieving, and transmitting modifying, maintaining, archiving, retrieving, and transmitting

data (electronic records and electronic signatures).data (electronic records and electronic signatures).

•• Successful network validation offers a “high degree of Successful network validation offers a “high degree of

assurance” that the system will perform its intended functions, assurance” that the system will perform its intended functions,

according to predetermined specifications.according to predetermined specifications.

•• It’s a regulatory requirement.It’s a regulatory requirement.

1717


IT & Network Equipment IQ and OQIT & Network Equipment IQ and OQ

•• IQ (Installation Qualification)IQ (Installation Qualification)

Document that the system has been Document that the system has been installedinstalled according to according to

predefined specifications & acceptance criteria.predefined specifications & acceptance criteria.

•• OQ (Operational Qualification)OQ (Operational Qualification)

Document that the system Document that the system performanceperformance meets predefined meets predefined

specifications & acceptance criteria.specifications & acceptance criteria.

1818

IT & Network Equipment ValidationIT & Network Equipment ValidationNetwork Validation Example :Network Validation Example :

•• GMP ServersGMP Servers

•• Office Computing Equipment (network printers and PCs)Office Computing Equipment (network printers and PCs)

•• IT Network Equipment (routers, switches, cabling, WAN IT Network Equipment (routers, switches, cabling, WAN

circuits)circuits)

•• DNS Service & Domain Controller ServersDNS Service & Domain Controller Servers

•• DHCP ServiceDHCP Service

1919


IQ for IT SystemsIQ for IT Systems

2020


IQ Recommended Forms:IQ Recommended Forms:

Cover formCover form

•• approval signatures, document number, etc.approval signatures, document number, etc.

System description and info System description and info [one for each device in IQ][one for each device in IQ]

•• what device is used forwhat device is used for

•• If it’s part of a larger systemIf it’s part of a larger system

•• what changes are being made to the system (if applicable)what changes are being made to the system (if applicable)

Documents, manuals, drawings Documents, manuals, drawings [one for each device in IQ][one for each device in IQ]

•• Include version no., locationInclude version no., location

Instruments used Instruments used

•• include copies of calibration certificationsinclude copies of calibration certifications

Software installed Software installed [one for each device in IQ][one for each device in IQ]

•• OS, antivirus, other applications, etc. OS, antivirus, other applications, etc.

•• Include name, brief description, license #, version #, path.Include name, brief description, license #, version #, path.

2121


IQ Recommended Forms:IQ Recommended Forms:

Electrical utilities Electrical utilities [one for each device in IQ][one for each device in IQ]

•• Actual Actual vsvs spec: device voltage (V), phase, current (A)spec: device voltage (V), phase, current (A)

•• panel, breaker locations; signature & panel, breaker locations; signature & licencelicence # of electrician # of electrician cerifyingcerifying

the installation meets National Electric Code (NEC)the installation meets National Electric Code (NEC)

Signatures log for IQSignatures log for IQ

•• Log the name, title, signature, and initials of Log the name, title, signature, and initials of everyone everyone whose signature whose signature

or initials appear in any of the IQ formsor initials appear in any of the IQ forms

Spare parts listSpare parts list

•• if applicableif applicable

Specific Equipment forms Specific Equipment forms [one for each device in IQ][one for each device in IQ]

•• server, PC, printer, router, etc.server, PC, printer, router, etc.

2222


IQ Specific IT Equipment Forms:IQ Specific IT Equipment Forms:

ServerServer

RouterRouter

SwitchSwitch

Network PCNetwork PC

Network PrinterNetwork Printer

Network CablingNetwork Cabling

WAN CircuitWAN Circuit

2323

IT & Network Equipment ValidationIT & Network Equipment ValidationIQ form example:IQ form example:

Performed By: _____________ Performed By: _____________ Verified By: ____________Verified By: ____________

Comments:Comments:

Complies with Acceptance Criteria: Complies with Acceptance Criteria: □□Y Y □□N N □□N/A If No or N/A, explain in CommentsN/A If No or N/A, explain in Comments

(Tests & info collected(Tests & info collected))

Equipment Number ________ Equipment Number ________ IQ Protocol _____________IQ Protocol _____________Page _ of _Page _ of _

IQ TEST TITLEIQ TEST TITLE

Location / Room: ____________________Tag No. _________

TCP/IP – Ethernet configured (Y/N) ____

CPU Type/Qty _____ CPU clock Speed ____OS _________ RAM ________

Manufacturer: ________ Model: __________ Tag/ Property # ______ Serial # _________

Server Name: _________ IP Address: _____ Domain ________

Expansion Boards (Qty, type, model) _______Serial, Parallel, USB Ports (qty each) _______

Disk Array Configuration _________HD Qty/Capacity ________

Input V _____ Input frequency _______UPS connected (Y/N) ___ Type ______

2424

IT & Network Equipment ValidationIT & Network Equipment ValidationIQ Attachments (recommended):IQ Attachments (recommended):

Network Topology & Racks DiagramsNetwork Topology & Racks Diagrams

License EvidenceLicense Evidence

•• for OS and all applications installed (SQL, antivirus, etc.)for OS and all applications installed (SQL, antivirus, etc.)

•• Include copy of document, or printout / screenshot of license nuInclude copy of document, or printout / screenshot of license numbermber

Physical Access ReportPhysical Access Report

•• list of everyone with physical access to computer rooms and list of everyone with physical access to computer rooms and

cabinets/closets, cabinets/closets,

•• Include name, account #, access level, access status (active/inaInclude name, account #, access level, access status (active/inactive)ctive)

Electrical Power Equipment Calibration Forms Electrical Power Equipment Calibration Forms

•• copies of calibration certifications for copies of calibration certifications for multimetersmultimeters, clamp meters, etc. used , clamp meters, etc. used

in IQin IQ

2525


IQ Attachments (recommended):IQ Attachments (recommended):

System Information Reports System Information Reports –– for Servers or PCsfor Servers or PCs

•• Start>Programs>Administrative Tools>Computer ManagementStart>Programs>Administrative Tools>Computer Management

•• Click over Click over System Tools>System InformationSystem Tools>System Information

•• Click Click SAVE TEXT REPORTSAVE TEXT REPORT icon, print it and attach it.icon, print it and attach it.

IP Address Configuration IP Address Configuration –– for Servers or PCsfor Servers or PCs

•• run run IPCONFIGIPCONFIG command, print resultscommand, print results

Electrical DrawingsElectrical Drawings

Deviations found during IQDeviations found during IQ

•• Log listing all deviations, with description, statusLog listing all deviations, with description, status

•• All individual deviation forms, properly completed and approved.All individual deviation forms, properly completed and approved.

2626

IT & Network Equipment ValidationIT & Network Equipment ValidationServer IQ Form Contents: Server IQ Form Contents:

(including DHCP and Domain Controller servers)(including DHCP and Domain Controller servers)

Server General Information:Server General Information:

•• Manufacturer, model, serial no., location, property tag numberManufacturer, model, serial no., location, property tag number

•• Server name, IP Address, domain, installed servicesServer name, IP Address, domain, installed services

•• Server DescriptionServer Description

•• Operating SystemOperating System

•• CPU type and quantity, clock speedCPU type and quantity, clock speed

•• RAM amount RAM amount

•• Removable storage drives: type, quantityRemovable storage drives: type, quantity

•• Hard drives: quantity, capacity; are they hotHard drives: quantity, capacity; are they hot--swappable?swappable?

•• Disk Array configuration: None, RAID 0, RAID 1, RAID 4, RAID 5, Disk Array configuration: None, RAID 0, RAID 1, RAID 4, RAID 5, otherother

•• Ports: quantity of USB, serial, parallel portsPorts: quantity of USB, serial, parallel ports

Expansion BoardsExpansion Boards

•• Quantity, type, modelQuantity, type, model

2727

IT & Network Equipment ValidationIT & Network Equipment ValidationServer IQ Form Contents: Server IQ Form Contents:

Communications Communications

•• Configured for TCP/IP (Configured for TCP/IP (y/ny/n))

•• Configured for Ethernet (Configured for Ethernet (y/ny/n))

Network interface cardsNetwork interface cards

•• quantity, speed; quantity, speed;

•• other adaptersother adapters

PowerPower

•• quantity of power suppliesquantity of power supplies

•• UPS (UPS (y/ny/n); if Y then Plant or Stand); if Y then Plant or Stand--alone UPS?alone UPS?

Input voltage (V) & frequency (Hz)Input voltage (V) & frequency (Hz)

Room EnvironmentRoom Environment

•• Operating Temperature (max/min), Operating Temperature (max/min),

•• Operating RH% (max/min)Operating RH% (max/min)

Room environmental conditions monitoring documentationRoom environmental conditions monitoring documentation

•• (copy of chart recordings)(copy of chart recordings)

2828


Network PC IQ Form Contents:Network PC IQ Form Contents:(1 PC per model)(1 PC per model)

Manufacturer, model, serial no., location, property tag numberManufacturer, model, serial no., location, property tag number

Notebook or DesktopNotebook or Desktop

Location (room)Location (room)

CPU, RAM, HD/RAID array, CPU, RAM, HD/RAID array,

COM ports, Parallel ports, USB ports, other portsCOM ports, Parallel ports, USB ports, other ports

OSOS

NICsNICs, TCP/IP or Ethernet, TCP/IP or Ethernet

IP Address, if staticIP Address, if static

Input voltage & frequencyInput voltage & frequency

UPS (UPS (y/ny/n); if Y then Plant or Stand); if Y then Plant or Stand--alone UPS?alone UPS?

Operating Temperature (max/min), Operating RH% (max/min)Operating Temperature (max/min), Operating RH% (max/min)

2929


Network Printer IQ Form Contents:Network Printer IQ Form Contents:

Manufacturer, model, serial numberManufacturer, model, serial number

Location, property tag numberLocation, property tag number

Network printer or standNetwork printer or stand--alonealone

Amount of RAMAmount of RAM

IP Address, if networkedIP Address, if networked




3030


Router IQ Form Contents:Router IQ Form Contents:

Manufacturer, model, serial numberManufacturer, model, serial number

IP AddressIP Address


# of network ports, if they are Ethernet or AUI# of network ports, if they are Ethernet or AUI

# of serial ports# of serial ports




3131


Switch IQ Form Contents:Switch IQ Form Contents:

Manufacturer, model, serial number Manufacturer, model, serial number


IP AddressIP Address

Supports 10 Supports 10 BaseTBaseT, 100 , 100 BaseTBaseT, Wireless ?, Wireless ?

Backbone: UTP, Fiber?Backbone: UTP, Fiber?

Switch configuration settingsSwitch configuration settings




3232


Network Cabling IQ Form Contents:Network Cabling IQ Form Contents:

Fiber or UTPFiber or UTP

UTP: CAT5 or above?UTP: CAT5 or above?

Cabling descriptionCabling description

Labeling scheme (closet / rack / port)Labeling scheme (closet / rack / port)

Labeling certificationLabeling certification

3333


WAN Circuit IQ Form Contents:WAN Circuit IQ Form Contents:(1 per circuit)(1 per circuit)

LocationLocation

Service providerService provider

From / toFrom / to

Fiber / microwave / otherFiber / microwave / other

Data / voice / bothData / voice / both

BandwithBandwith

Routers connectedRouters connected

Certified ? Certified ? –– copy of communications circuit certificationcopy of communications circuit certification

Circuit exclusive to company (closed system)?Circuit exclusive to company (closed system)?

3434


OQ/PQ for IT SystemsOQ/PQ for IT Systems

3535


Network OQ TestingNetwork OQ Testing

•• Standard OQ TestsStandard OQ Tests

•• IQ completion and approval verificationIQ completion and approval verification

•• Risk assessment Risk assessment –– safety department evaluationsafety department evaluation

•• Instrument calibration documentation & evidenceInstrument calibration documentation & evidence

•• ComponentComponent--specific OQ Testsspecific OQ Tests

•• ServersServers•• RoutersRouters•• SwitchesSwitches•• etc.etc.

3636

Server OQ TestingServer OQ Testing1.1. Server clock accuracyServer clock accuracy

2.2. Diagnostic testDiagnostic test

3.3. Startup & ShutdownStartup & Shutdown

4.4. Loss of power & UPS testLoss of power & UPS test

5.5. Server power supply redundancy testServer power supply redundancy test

6.6. Communications redundancy testCommunications redundancy test

7.7. Log files verificationLog files verification

8.8. Virus ProtectionVirus Protection

9.9. Backup & RestoreBackup & Restore

10.10. SecuritySecurity

3737

Server OQ TestingServer OQ Testing

•• Server Clock AccuracyServer Clock Accuracy

Verify time and date displayed are correct, and document this.Verify time and date displayed are correct, and document this.

Using a calibrated chronometer, measure a period of 24 hours. Using a calibrated chronometer, measure a period of 24 hours.

At the end of the 24At the end of the 24--hr period, verify that the time displayed in the hr period, verify that the time displayed in the

server corresponds to the time shown in the chronometer, ±2 secserver corresponds to the time shown in the chronometer, ±2 sec

3838


•• Hardware Manufacturer Diagnostic TestHardware Manufacturer Diagnostic Test

Execute the Diagnostic test provided by the server’s manufactureExecute the Diagnostic test provided by the server’s manufacturer; r;

print test results and attach them to Raw Data.print test results and attach them to Raw Data.

If Diagnostic test shows any error, document this, and:If Diagnostic test shows any error, document this, and:

•• Explain if this is acceptable; orExplain if this is acceptable; or

•• Correct problem, and repeat the diagnostic test.Correct problem, and repeat the diagnostic test.

For For Compaq Compaq ProliantProliant servers:servers:

•• Turn ON server, and press Turn ON server, and press F10 F10 as the server boots upas the server boots up

•• In In System SetupSystem Setup, select , select Diagnostic & UtilitiesDiagnostic & Utilities, then select , then select Quick Quick

Check DiagnosticCheck Diagnostic, and then , and then StartStart..

3939


•• Startup & ShutdownStartup & Shutdown

With server turned With server turned ONON, select , select Start Start --> Shutdown> Shutdown, the server should , the server should

shut down completely.shut down completely.

Turn ON serverTurn ON server

When the Login screen appears, log on using Administrator accounWhen the Login screen appears, log on using Administrator accountt

Once logged, go to Once logged, go to

Start Start --> Programs > Programs --> Administrative Tools > Administrative Tools --> Events Viewer> Events Viewer

In the Events Viewer select the In the Events Viewer select the Application Log Application Log and print it.and print it.

4040


•• Power Loss and UPS TestPower Loss and UPS Test

Ensure UPS is fully chargedEnsure UPS is fully charged

With the server turned ON, simulate a general power failure by With the server turned ON, simulate a general power failure by

unplugging the UPS from the power outlet.unplugging the UPS from the power outlet.

Ensure the UPS provides at least 15 minutes of power to the servEnsure the UPS provides at least 15 minutes of power to the server, er,

sufficient time to shut down the server properly, or to wait untsufficient time to shut down the server properly, or to wait until the il the

site’s emergency power comes onlinesite’s emergency power comes online

4141


•• Power Supply Redundancy TestPower Supply Redundancy Test

With the server ON, disconnect the power cord from one of its poWith the server ON, disconnect the power cord from one of its power wer

suppliessupplies

Ensure that the server stays ON, document any messages or Ensure that the server stays ON, document any messages or

warnings displayed.warnings displayed.

Reconnect the power cord, and repeat the test with the second poReconnect the power cord, and repeat the test with the second power wer

cord.cord.

4242

Server OQ TestingServer OQ Testing•• Communications Redundancy TestCommunications Redundancy Test

With the server With the server ONON, use , use Advanced Server Administrator Advanced Server Administrator tools to tools to

print the server’s Host Name and the IP Address print the server’s Host Name and the IP Address

Disconnect the Ethernet cable from the primary NIC card to simulDisconnect the Ethernet cable from the primary NIC card to simulate ate

a communications loss. Verify that the message “NIC Card Cable a communications loss. Verify that the message “NIC Card Cable

Unplugged” appears in the screen, and that the NIC is still workUnplugged” appears in the screen, and that the NIC is still working.ing.

Use a PC connected to the network to open a DOS windowUse a PC connected to the network to open a DOS window

Type Type NSLOOKUPNSLOOKUP and the server name. The network should respond and the server name. The network should respond

with the correct server name and IP address. If a “Request Time with the correct server name and IP address. If a “Request Time Out” Out”

message appears, the test has failed.message appears, the test has failed.

Type EXIT to close DOS windowType EXIT to close DOS window

Reconnect primary NIC Ethernet cable, the message “NIC Card CablReconnect primary NIC Ethernet cable, the message “NIC Card Cable e

Unplugged” should disappear.Unplugged” should disappear.

Repeat steps with the secondary NIC Card Ethernet cable.Repeat steps with the secondary NIC Card Ethernet cable.

4343


•• Log files verificationLog files verification

Go to the Go to the C:C:\\winntwinnt\\system32system32\\config config folder, and open itfolder, and open it

Verify the existence of the following log files:Verify the existence of the following log files:

•• SYSEVENT.EVT SYSEVENT.EVT -- System log fileSystem log file

•• APPEVENT.EVT APPEVENT.EVT -- Application log fileApplication log file

•• SECEVENT.EVT SECEVENT.EVT -- Security log fileSecurity log file

Go to Go to Start Start --> Programs > Programs --> Administrative Tools > Administrative Tools --> Event > Event

ViewerViewer

RightRight--click the click the Event ViewerEvent Viewer, select , select Properties Properties for each event log for each event log

file, and print a screenshot showing file, and print a screenshot showing Maximum Log Size Maximum Log Size and and Event Event

Log Wrapping Log Wrapping information.information.

Ensure that for all event log files:Ensure that for all event log files:

•• Maximum Log Size Maximum Log Size = 5120K= 5120K

•• Settings when maximum log size is reached Settings when maximum log size is reached = Overwrite = Overwrite

Events as NeededEvents as Needed

4444


•• Virus Protection VerificationVirus Protection Verification

Verify the antivirus icon appears in the bottom right corner of Verify the antivirus icon appears in the bottom right corner of the the

Windows screen.Windows screen.

Select this icon and ensure the antivirus software is activated.Select this icon and ensure the antivirus software is activated.

Perform virus scan of server boot sector and hard drives.Perform virus scan of server boot sector and hard drives.

Open virus scan report, print it and verify that boot sector andOpen virus scan report, print it and verify that boot sector and all all

hard drives are virus free. hard drives are virus free.

4545


•• Backup & RestoreBackup & Restore

Use a tool such as Use a tool such as VeritasVeritas Backup Exec Backup Exec or or Symantec Ghost Symantec Ghost to to

create a backup or an image of the server and all settingscreate a backup or an image of the server and all settings

Restore the backup or image (if possible, in a spare server of tRestore the backup or image (if possible, in a spare server of the he

same model), and verify the server’s functionality.same model), and verify the server’s functionality.

This description corresponds to the simple case of a single GMP This description corresponds to the simple case of a single GMP

server. For setups with server clusters, servers connected to a server. For setups with server clusters, servers connected to a mass mass

storage unit (e.g. EMC storage unit (e.g. EMC SymmetrixSymmetrix), or servers with an SQL database, ), or servers with an SQL database,

the backup test will be more complex (and beyond the scope of ththe backup test will be more complex (and beyond the scope of this is

conference)conference)

4646

Server OQ TestingServer OQ Testing•• SecuritySecurity

Physical Security Physical Security –– Access ControlAccess Control

•• Try to enter Computer Room using ID Card without access, card reTry to enter Computer Room using ID Card without access, card reader ader

should not grant accessshould not grant access

•• Repeat with authorized ID Card, card reader should grant accessRepeat with authorized ID Card, card reader should grant access

Logical Security Logical Security –– Password PolicyPassword Policy

•• Verify and document that the following controls are in place:Verify and document that the following controls are in place:

Password expiration period set (e.g. 90 days) as per SOPPassword expiration period set (e.g. 90 days) as per SOP

“Password Never Expires” option NOT active“Password Never Expires” option NOT active

Password length limit (e.g. 8 characters)Password length limit (e.g. 8 characters)

Blank passwords NOT allowedBlank passwords NOT allowed

Password Uniqueness enforcedPassword Uniqueness enforced

Account locked after 3 unsuccessful login attemptsAccount locked after 3 unsuccessful login attempts

Only Administrator can lock / unlock accountOnly Administrator can lock / unlock account

4747

Network ValidationNetwork Validation•• Office Computing Equipment OQOffice Computing Equipment OQ

PC Communications test: PC Communications test:

•• Login with an administrator accountLogin with an administrator account

•• PING the Domain Controller server IP AddressPING the Domain Controller server IP Address

•• Get return reply confirming that communication is OKGet return reply confirming that communication is OK

•• Do this for each PC Desktop / notebook modelDo this for each PC Desktop / notebook model

Network Printer Communications test:Network Printer Communications test:

•• Print test pagePrint test page

•• Ensure the page has the correct printer name, model, date and tiEnsure the page has the correct printer name, model, date and timeme

•• Do this for each network printer modelDo this for each network printer model

PC Security test:PC Security test:

•• Attempt to login with different combinations of correct & incorrAttempt to login with different combinations of correct & incorrect user ect user

name & password.name & password.

•• Attempt to cause an account lock, get IT to release the account Attempt to cause an account lock, get IT to release the account afterwardsafterwards

4848

Network ValidationNetwork Validation•• Office Computing Equipment OQOffice Computing Equipment OQ

Virus Protection verification:Virus Protection verification:

•• Refer to SERVER OQ TESTING for Virus Protection testRefer to SERVER OQ TESTING for Virus Protection test

StandStand--alone Printer Diagnostic Testalone Printer Diagnostic Test

•• Print a Configuration pagePrint a Configuration page

•• Ensure all diagnostics are okEnsure all diagnostics are ok

4949

Network ValidationNetwork Validation•• IT Network Equipment OQIT Network Equipment OQ

Physical Security: Physical Security:

•• Obtain printout of personnel authorized to enter the room where Obtain printout of personnel authorized to enter the room where

equipment is locatedequipment is located

•• If room has electronic access control system (e.g. using employeIf room has electronic access control system (e.g. using employee ID e ID

cards), test to ensure only authorized personnel can open door.cards), test to ensure only authorized personnel can open door.

System Security:System Security:

•• Connect physically to router or switch, using a laptopConnect physically to router or switch, using a laptop

•• Open telnet sessionOpen telnet session

•• Try to enter using incorrect and correct passwordsTry to enter using incorrect and correct passwords

5050


Diagnostic Tests: Diagnostic Tests:

•• Connect PC to router or switch, open telnet sessionConnect PC to router or switch, open telnet session

•• Turn off router or switchTurn off router or switch

•• Power upPower up

•• Verify that no startVerify that no start--up errors were generatedup errors were generated

UPS / Loss of Power Test:UPS / Loss of Power Test:

•• With UPS fully powered, disconnect main power and ensure the UPSWith UPS fully powered, disconnect main power and ensure the UPS

provides at least 15 minutes of powerprovides at least 15 minutes of power

Fault Tolerance / Power Supply Redundancy Test:Fault Tolerance / Power Supply Redundancy Test:

•• for routers or switches with dual power suppliesfor routers or switches with dual power supplies

•• Disconnect power cable from first power supplyDisconnect power cable from first power supply

•• Spare power supply should keep equipment runningSpare power supply should keep equipment running

•• ReRe--connect power cable from first power supplyconnect power cable from first power supply

•• repeat with power cable from second power supplyrepeat with power cable from second power supply

5151


Communications Circuits redundancy test: Communications Circuits redundancy test:

•• Use this when a WAN connects more than 1 site within the same neUse this when a WAN connects more than 1 site within the same network, twork,

and there are redundant connections (mw and fiber, or a 3and there are redundant connections (mw and fiber, or a 3rdrd site)site)

•• From one site, PING the destination site’s DHCP serverFrom one site, PING the destination site’s DHCP server


•• Disconnect the circuit connecting to the remote siteDisconnect the circuit connecting to the remote site

•• PING againPING again


•• Try for all connectionsTry for all connections

Switch Loss of Communication test:Switch Loss of Communication test:

•• Power LEDPower LED

•• RJ45 port status LED should show OKRJ45 port status LED should show OK

•• Unplug the communications cable from the switchUnplug the communications cable from the switch

•• RJ45 LED should show no communicationRJ45 LED should show no communication

•• Plug cable, LED should show OK statusPlug cable, LED should show OK status

5252


Network Stress test: Network Stress test:

•• The objective of a stress test is to challenge system performancThe objective of a stress test is to challenge system performance in a e in a

situation where system resources are under unusual or extreme desituation where system resources are under unusual or extreme demand mand

in terms of quantity, volume, etc.in terms of quantity, volume, etc.

•• Test should challenge, during a high traffic scenario :Test should challenge, during a high traffic scenario :

PC and printer connectivity to networkPC and printer connectivity to network

use of the Domain Controller and Domain Name servicesuse of the Domain Controller and Domain Name services

Use of the Dynamic Host Connectivity Protocol (DHCP)Use of the Dynamic Host Connectivity Protocol (DHCP)

5353


Network Stress test example: Network Stress test example:

•• Use traffic generator software to induce a high volume of networUse traffic generator software to induce a high volume of network traffic k traffic

(e.g. 3x measured peak) to the Domain Controller and DHCP Server(e.g. 3x measured peak) to the Domain Controller and DHCP Serverss

•• Use various PCs, each connected to a different network node; froUse various PCs, each connected to a different network node; from each PC m each PC

execute the following test:execute the following test:

Login and open a DOS sessionLogin and open a DOS session

PINGPING the IP address for each Domain Controller serverthe IP address for each Domain Controller server

NSLOOKUPNSLOOKUP, the screen should display the , the screen should display the Default Server NameDefault Server Name

and and Default Server IP AddressDefault Server IP Address, and the “, and the “>>” prompt” prompt

Use Use IPCONFIG/RELEASE IPCONFIG/RELEASE to release the dynamic IP addressto release the dynamic IP address

Use Use IPCONFIG/ALL IPCONFIG/ALL to display the IP address value, it should be to display the IP address value, it should be

0.0.0.00.0.0.0

Use Use IPCONFIG/RENEW IPCONFIG/RENEW to get a new dynamic IP Address. Screen to get a new dynamic IP Address. Screen

should display the new IP Address, subnet mask, and default gateshould display the new IP Address, subnet mask, and default gateway. way.

Select a networked printer and print a Test PageSelect a networked printer and print a Test Page

5454

•• DNS Service & Domain Controller Servers OQDNS Service & Domain Controller Servers OQ

All the regular Server OQ tests All the regular Server OQ tests

•• For each Domain Controller server, even if they’re all the same For each Domain Controller server, even if they’re all the same modelmodel

DNS Service Functionality TestDNS Service Functionality Test

DNS Service & Domain Controller Redundancy TestDNS Service & Domain Controller Redundancy Test

Network ValidationNetwork Validation

5555

Network ValidationNetwork Validation•• DNS Service & Domain Controller Servers OQDNS Service & Domain Controller Servers OQ

DNS Service Functionality TestDNS Service Functionality Test

•• Purpose: to verify and document that the DNS service works as spPurpose: to verify and document that the DNS service works as specifiedecified

•• Log in from a networked PCLog in from a networked PC

•• NSLOOKUPNSLOOKUP, the screen should display the , the screen should display the Domain ControllerDomain Controller server server

name and IP address, and the “name and IP address, and the “>>” prompt” prompt

•• At the prompt, type the DHCP server name, system should display:At the prompt, type the DHCP server name, system should display:

SERVER: the Domain Controller server name SERVER: the Domain Controller server name

ADDRESS: the DC server IP AddressADDRESS: the DC server IP Address

NAME: the DHCP server nameNAME: the DHCP server name

ADDRESS: the DHCP server IP AddressADDRESS: the DHCP server IP Address

•• Type EXIT and close DOS windowType EXIT and close DOS window

5656

Network ValidationNetwork Validation•• DNS Service & Domain Controller Servers OQDNS Service & Domain Controller Servers OQ

DNS Service & Domain Controller Redundancy TestDNS Service & Domain Controller Redundancy Test

•• Purpose is to verify and document the response, and challenge thPurpose is to verify and document the response, and challenge the e

redundancyredundancy, of the DC and DNS services upon loss of communication., of the DC and DNS services upon loss of communication.

•• Disconnect the plant WAN from the corporate networkDisconnect the plant WAN from the corporate network

•• Disconnect the local Domain Controller server from the network tDisconnect the local Domain Controller server from the network to o

simulate a communications losssimulate a communications loss

•• Log in from a networked PC, and open a DOS windowLog in from a networked PC, and open a DOS window

•• Type Type SETSET, a series of PC settings should be displayed, a series of PC settings should be displayed

•• Record the value of the LOGON SERVER parameter, it contains the Record the value of the LOGON SERVER parameter, it contains the name name

of the Domain Controller server assigned to this PCof the Domain Controller server assigned to this PC

•• NSLOOKUP NSLOOKUP should display the values for one of the should display the values for one of the alternatealternate Domain Domain

Controller servers in the network.Controller servers in the network.

•• Execute this test for each Domain Controller server in the netwoExecute this test for each Domain Controller server in the network.rk.

5757

Network ValidationNetwork Validation•• DHCP Servers OQDHCP Servers OQ

All the regular Server OQ tests All the regular Server OQ tests

•• For each DHCP server, even if they’re all the same modelFor each DHCP server, even if they’re all the same model

DHCP Servers Emergency RepairDHCP Servers Emergency Repair

•• Verify the DHCP Emergency Repair disks exist, document location,Verify the DHCP Emergency Repair disks exist, document location,

backups, person responsible, etc.backups, person responsible, etc.

DHCP Scopes Configuration testDHCP Scopes Configuration test

•• Verify the Address Pools for each siteVerify the Address Pools for each site

•• Verify the Scope Options for each siteVerify the Scope Options for each site

5858

Network ValidationNetwork Validation•• DHCP Servers OQDHCP Servers OQ

DHCP Functionality & Redundancy TestDHCP Functionality & Redundancy Test

•• Log in, get IP addressLog in, get IP address

•• verify it is within the correct Subnet ranges for the siteverify it is within the correct Subnet ranges for the site

•• Verify it is not within the Exclusion ranges for the siteVerify it is not within the Exclusion ranges for the site

•• Release IP addressRelease IP address

•• Renew connection (simulate loss of communication)Renew connection (simulate loss of communication)

•• Display IP address, verify it is from an alternate DHCP server (Display IP address, verify it is from an alternate DHCP server (redundancy)redundancy)

•• verify it is within the correct Subnet ranges for the alternate verify it is within the correct Subnet ranges for the alternate sitesite

•• Verify it is not within the Exclusion ranges for the alternate sVerify it is not within the Exclusion ranges for the alternate siteite

5959

Network ValidationNetwork Validation•• Other items which may require Computer Validation :Other items which may require Computer Validation :

IBM AS400 ComputersIBM AS400 Computers

Laboratory Equipment (e.g. LIMS, HPLC)Laboratory Equipment (e.g. LIMS, HPLC)

Remote Access ServicesRemote Access Services

SQL Databases w/ GMP dataSQL Databases w/ GMP data

EMC EMC SymmetrixSymmetrix Storage UnitsStorage Units

6060

SOFTWARE VALIDATIONSOFTWARE VALIDATION

Software Categories Software Categories -- SummarySummary

IQ:IQ: Record version (including service pack, if Record version (including service pack, if applicable). The OS will be challenged applicable). The OS will be challenged indirectly by the functional testing of the indirectly by the functional testing of the application.application.

Operating Operating SystemSystem

11

IQ:IQ: For nonFor non--configurable firmware, record configurable firmware, record version. Calibrate instruments as necessary. version. Calibrate instruments as necessary. Verify operation against user requirements.Verify operation against user requirements.

IQ: IQ: For configurable firmware, record version For configurable firmware, record version and configuration. Calibrate instruments as and configuration. Calibrate instruments as necessary. necessary. OQ: OQ: Verify operation against user Verify operation against user requirements.requirements.

Manage custom firmware as Category 5 Manage custom firmware as Category 5 software.software.

FirmwareFirmware22

Validation ApproachValidation ApproachSoftware Software TypeType

CategoryCategory

Source: GAMP 4 Guide, Appendix M4

Software Categories Software Categories -- SummarySummary

IQ: IQ: Record version and configuration of Record version and configuration of environment; environment; OQ: OQ: verify against user verify against user requirements.requirements.

Consider auditing the supplier for critical and Consider auditing the supplier for critical and complex applications.complex applications.

Standard Standard Software Software PackagesPackages

33

Audit supplier and Audit supplier and (IQ, OQ) (IQ, OQ) validate complete validate complete system.system.

Custom Custom (Bespoke) (Bespoke) SoftwareSoftware

55

IQ: IQ: Record version and configuration of Record version and configuration of environment; environment; OQ: OQ: verify against user verify against user requirements.requirements.

Normally audit the supplier for critical and Normally audit the supplier for critical and complex applications.complex applications.

Manage any custom programming as CatManage any custom programming as Cat--5.5.

Configurable Configurable Software Software PackagesPackages

44

Validation ApproachValidation ApproachSoftware Software TypeType

CategoryCategory

Source: GAMP 4 Guide, Appendix M4

6363

Software Development TestingSoftware Development Testing

Unit TestUnit Test•• Unit testing focuses on testing configured or customized code atUnit testing focuses on testing configured or customized code at

the individual transaction, module, or component level.the individual transaction, module, or component level.

e.g. Simulated input e.g. Simulated input --> [Module] > [Module] --> Output> Output

String TestString Test•• String testing focuses on testing strings of transactions, modulString testing focuses on testing strings of transactions, modules, es,

or components which are commonly used together. or components which are commonly used together. e.g. Simulated input e.g. Simulated input --> [Module 1] > [Module 1] --> Output 1> Output 1

Output 1 Output 1 --> [Module 2 ] > [Module 2 ] --> Output 2> Output 2

Integrated TestingIntegrated Testing•• Integrated testing focuses on testing integrated scenarios whichIntegrated testing focuses on testing integrated scenarios which

are intended to simulate entire processes performed by the are intended to simulate entire processes performed by the software.software.

6464

Wrapping up …Wrapping up …

•• What have we covered ?What have we covered ?

Overview of General Validation ConceptsOverview of General Validation Concepts

Computer Validation tests Servers and IT Network systemsComputer Validation tests Servers and IT Network systems

Software Validation testsSoftware Validation tests

We’re Done !We’re Done !

network infrastructure validation conference @upra (2003)

Technology

computer systems

validation process

overall system validation

processes systems

gxps systems

quality decisions systems

system description

computer validationtesting