u.s. department of energy audit report · 2012. 11. 18. · the preaudit meeting was held at...

As

QX t6,,(fi22/ -br t - 2/a/Ldit RepoitYMP-94-05 of 61

U.S. DEPARTMENT OF ENERGYOFFICE OF CIVILIAN RADIOACTIVE WASTE MANAGEMENT

OFFICE OF QUALITY ASSURANCE

AUDIT REPORT

OF

SCIENCE APPLICATIONS INTERNATIONAL CORPORATION

TECHNICAL AND MANAGEMENT SUPPORT SERVICES

LAS VEGAS, NEVADA

AND YUCCA MOUNTAIN SITE

AUDIT YMP-94-05

MAY 16 THROUGH 20, 1994

Pnmpand bYL 4 Date: 0(6-Richani L MaualinAudit Team LeaderYucca Mountain Quality Assurance Division

Apprved by: .k2g.b! ,Donald G. HoitoiDirectorOffice of Quality Assurance

Date:

9407200242 940712PDR WASTEWM-11 PDR

Audit ReportYMP-94-05 of 61

1.0 EXECUTIVE SUMMARY

As a result of Quality Assurance (QA) Audit YMP-94-05, the audit team determinedthat Science Applications International Corporation/Technical and ManagementSupport Services (SAIC/T&MSS) is satisfactorily implementing an effective QAprogram in accordance with the U.S. Department of Energy (DOE) Office of CivilianRadioactive Waste Management (OCRWM) Quality Assurance Requirements andDescription (QARD), DOE/RW-0333P, Revision 0, for the Civilian Radioactive WasteManagement Program and SAIC/T&MSS implementing procedures for QA ProgramElements 1.0, 2.0, 4.0, 5.0, 6.0, 7.0, 10.0, 12.0, 15.0, 16.0, 17.0, 18.0, and SupplementIml. No implementation of QA Program Element Appendix C could be identified dueto the lack of activity.

The audit team identified one deficiency during the audit that resulted in the issuanceof one Corrective Action Request (CAR), YM-94-040, which identified the fact thatthere was no objective evidence that real-time training was completed prior toperforming work. The audit team identified five deficiencies that were correctedduring the course of the audit related to the performance of verification activities.These deficiencies are described in Section 5.5.2 of this report. Additionally, therewere ten recommendations resulting from the audit which are detailed in Section 6.0of this report.

2.0 SCOPE

The audit was conducted to evaluate compliance to, and the effectiveness of, theSAICJT&MSS QA Program as described in the QARD and SAICIT&MSSimplementing quality documents.

The QA program elements/requirements evaluated during the audit in accordance withthe published audit plan are as follows:

OA PROGRAM ELEMENTS

1.0 Organization2.0 Quality Assurance Program4.0 Procurement Document Control5.0 Implementing Documents6.0 Document Control7.0 Control of Purchased Items and Services

10.0 Inspection12.0 Control of Measuring and Test Equipment15.0 Nonconformances16.0 Corrective Action17.0 Quality Assurance Records18.0 Audits


Supplement m, Scientific Investigation

Appendix C, Mined Geologic Disposal System, was not evaluated due to lack ofactivity.

The following QA program elements were not evaluated since they are not included inthe SAIC/T&MSS QA Program at this time:

3.0 Design Control8.0 Identification and Control of Items9.0 Control of Special Processes

11.0 Test Control13.0 Handling, Storage, and Shipping14.0 Inspection, Test and Operating Status

Supplement I, SoftwareSupplement II, Sample ControlSupplement IV, Field SurveyingAppendix A, High Level Radioactive Waste Form ProductionAppendix B, Transportation

TECHNICAL AREAS

The technical areas audited were Q-List work associated with the MeteorologyProgram and the Monitoring of Conditions in Population Centers Relative to WindPatterns. The radiological survey technical work has been classified as non-Q and wasnot evaluated during this audit.

3.0 AUDIT TEAM AND OBSERVERS

The following is a list of audit team members, their assigned areas of responsibility,and observers:

Individual QA Program Element/Requirement

Richard L. Maudlin, Audit Team Leader (ATL),Yucca Mountain Quality Assurance Division (YMQAD)

Stephen R. Maslar, ATL-in-Training, YMQADPatout Cotter, Auditor, YMQADKenneth 0. Gilkerson, Auditor, YMQAD

Thomas E. Rodgers, Auditor, YMQADFredrick Bearham, Auditor, Headquarters QA DivisionAllan Barr, Technical Specialist, U.S. Department

of Commerce, National Oceanic and AtmosphericAdministration

1.0 and 2.0

10.0, 15.0 and 18.012.0, Appendix C

and Supplement Iml4.0, 7.0 and 17.05.0, 6.0 and 16.0

Meteorological MonitoringProgram


Pauline P. Brooks, Observer, U.S. Nuclear RegulatoryCommission (NRC)

John G. Spraul, Observer, NRCRodney M. Weber, Observer, NRC/Center for

Nuclear Waste Regulatory Analyses

4.0 AUDIT MEETINGS

The preaudit meeting was held at SAICIT&MSS facilities in Las Vegas, Nevada, onMay 16, 1994. A daily debriefing and coordination meeting was held withSAIC/T&MSS management and staff, and daily audit team meetings were held todiscuss issues and potential deficiencies. The audit was concluded with a postauditmeeting held at the SAIC/T&MSS facilities in Las Vegas, Nevada, on May 20, 1994.Personnel contacted during the audit are listed in Attachment 1 of this report. This listincludes an indication of those who attended the preaudit and postaudit meetings.

5.0 SUMMARY OF AUDIT RESULTS

5.1 Poeram Effectiveness

The audit team concluded that, in general, the SAIC/T&MSS QA Program isadequate and is being effectively implemented for the scope of this audit.Individually, QA Program Elements 1.0, 2.0, 4.0, 5.0, 6.0, 7.0, 10.0, 12.0, 15.0,16.0, 17.0, 18.0, and Supplement III are satisfactorily being implemented. Noimplementation of QA Program Element Appendix C could be identified due tolack of activity.

5.2 Stop Woik or Immediate Corrective Actions or Additional Actions

There were no Stop Work Orders, immediate corrective actions or relatedadditional items resulting from this audit.

5.3 OA Prawm Audit Activities

Details of the QA program audit activities are provided in Attachment 2. A listof objective evidence reviewed during the audit is provided in Attachment 3.

5A Technical Activities

Details of the technical audit activities are provided in Attachment 2. A list ofobjective evidence reviewed during the audit is provided in Attachment 3.

Audit RepoitYMP-94-05 of 61

5.5 Summary of Deficiencies

One CAR was issued as a result of this audit. The audit team identified fivedeficiencies that were corrected during the course of the audit related to theperformance of verification activities.

A synopsis of the deficiency documented as a CAR is contained in Section5.5.1 of this report and an information copy is included in Attachment 4.

A synopsis of the deficiencies corrected during the audit is contained in Section5.5.2 of this report.

5.5.1 Cometive Action Requests (CAR)

CAR YM-94-040

Contrary to Standard Procedure (SP) 1.31, Revision 8, Section 3.0, noobjective evidence could be provided that five individuals who hadperformed a document review of TMSS/93-003 (OrganizationalDescription), had read and understood SP 1.2, Revision 7, prior toperforming the review.

5.5.2 Deficiencies Cornected During the Audit

Deficiencies that are considered isolated in nature and require onlyremedial action can be corrected during the audit. The following fivedeficiencies were identified and corrected during the audit.

1. SP 1.2, Revision 7, requires that a QA Policy be prepared thatestablishes internal and external interfaces associated with theT&MSS scope of work. Contrary to this requirement, noevidence existed to indicate that the QA Policy addressedexternal interfaces. This was resolved during the audit throughthe generation of Interim Change Notice (ICN) 1 to SP 1.2,Revision 7, which deleted the requirements for internal andexternal interfaces being described in the QA Policy.

2. SP 1.71, Revision 1, requires that, for each relevant OCRWMQARD section, the applicable procedures or documents thatapply the QA requirements to the activity be defined, Contraryto this requirement, three T&MSS Quality Assurance GradingReports (QAGRs) (QA-002, QA-003, QA-004) identifiedactivities related to procurement; however, procurement elements4 and 7 were indicated as "Not Applicable." Prior to the auditexit meeting, the three grading reports in question were revised


and approved indicating that QA Program Element 7.0 applied tothe activities.

3. Work Instruction (WI)-QA-002, Revision 1, requires that copiesof completed surveillance reports be transmitted to the ProjectManager. Contrary to this requirement, there was no objectiveevidence that one surveillance report (YMP-SR-93-004) had beentransmitted to the Project Manager. This was corrected duringthe audit by issuance of a memorandum formally transmitting thesurveillance report to-the Project Manager.

4. A Nonconformance Report (NCR) was not issued on WINDSPEED SENSORS when they were repaired without determiningthe "as-found" calibration status, so that a determination could bemade as to the validity of previously collected data. Thisviolated the QARD requirements in Section 12.2.3B.2 whichrelated to calibration status of equipment prior to repair orrecalibration. NCR 94-0016 was issued and dispositioned duringthe audit.

5. A performance audit noted an out-of-tolerance precipitationgauge which was replaced. This action was not documented onan NCR as required by procedures WI-RED-006, Revision 2,and WI-MET-001, Revision 4. NCR 94-0017 was issued duringthe audit.

None of the deficiencies have an impact on the quality of thework being performed.

5.5.3 Follow-up of Pieviously Identified CARs

There were no CARs issued to SAIC/T&MSS since FY 1992; therefore,no follow-up of previously identified CARs was performed.

6.0 RECOMMENDATIONS

The following recommendations resulted from the audit and are presented forconsideration by SAIC/T&MSS management.

1. The Organizational Description document (TMSS/93-003) should be evaluatedand revised to more truly reflect and differentiate those activities of theparticipant versus those of the support contractor.

-


2. Future Management Assessment Reports should include all evidence taken intoaccount that was used to formulate the final conclusions (i.e., adequacy andeffectiveness).

3. Although no violation exists with calibration procedures, calibration proceduresneed to provide more specific detail including the standard used to perform thecalibration.

4. Although no violation exists in the training area, the technical training needs tobe more structured in the following -areas:

* Training requirements* Instructor qualifications* Specifics of the training* Who performs the training and when

5. WI-RED-006, Revision 2, Paragraph 4.2.2b, should be clarified to require thatthe maintenance of performance check records be kept in a central history filerather than in the individual equipment folder (history file) since they are partof an overall systems evaluation.

6. The documentation notification for lost Measuring and Test Equipment(M&TE) (Paragraph 4.7.1 of WI-RED-006, Revision 2) should be updated toreflect the evaluation performed on previously generated data or equipmentusage and clearly state whether an NCR was generated in addition to theinformation the notification now contains.

7. SP 2.2, Revision 4, needs to be updated to require the verification of datagenerated be performed in accordance with Yucca Mountain AdministrativeProcedure (YAP)-S.Il.1Q in lieu of Administrative Procedure (AP)-5.9Q, sinceAP-5.9Q has recently been superseded.

8. The Scientific Investigation Implementation Package (SUP), CharacterizingWind Patterns Relating to Population Centers, should be reviewed and updatedto be consistent with current QARD requirements. The SUP was issued inMarch 1992 and the work will be starting in the near future. Affectedpersonnel need to be trained to SP 2.2 prior to implementation of the SLIP.

9. Although no violation exists, the T&MSS matrix for QARD Supplement IIIrequired by QARD Section 2.2.2C does not fully address where theserequirements are actually implemented for the Meteorological MonitoringProgram. SP 2.2 is the only implementing document listed in Supplement IIImatrix. This is not an implementation procedure for Study Plan 8.3.1.12.2.1.WI-MET-001, WI-MET-002, WI-MET-003, WI-MET-006, and WI-MET-009


are the implementing procedures for this activity. The SP 2.2 procedurereferenced in the matrix is applicable to the SUP, Characterizing Wind PatternsRelative to Population Centers, which has not yet been implemented. It isrecommended that the matrix be reviewed and updated in accordance withprogram requirements.

10. Since site selection for specialized instrumentation has just started, it isrecommended that modeling personnel with the Air Resources Laboratory maybe solicited to help the site selection using atmospheric models.

7.0 LIST OF ATTACHMENTS

Attachment 1: Personnel Contacted During the AuditAttachment 2: Audit DetailsAttachment 3: List of Objective Evidence Reviewed During the AuditAttachment 4: Information Copy of Corrective Action Request

Audit ReportYMP-94-O5 of 61

AITACMUENT 1

Personnel Contacted Durine The Audit

Name - Orianization/Tifle PnmauditMeetinh

ContactedDrinz Audit

PostauditMeetini!

C. Barron

A. BarrK. BeallF. BearhamR. BostianD. BreesP. BrooksR. Brown, Jr.P. ChadwickD. ChandlerJ. ClarkJ. Conway

P. CotterL. CroftG. DonaldsonJ. DunhamJ. EstellaP. FransioliK. GilkersonS. GomezH. GreeneJ. HarperM. Harris

J. HitzR. HelmsK. JohnsonS. JohnsonG. JonesJ. KingJ. LongeneckerM. MaloneS. MaslarR. MaudlinC. McEntee

SAIC/DocumentAdministration Specialist.

USDC/Tech. SpecialistSAICIFTS/StaffYMQAD/AuditorSAIC/Resource ManagerSAIC/IRG MeteorologistNRC/ObserverSAIC/Sr. Buyer/APMSAIC/Manager, TrainingSAIC/Deputy Project ManagerSAIC/REFPD QA LiaisonSAIC/Meteorological Field

Activities Tech. SpecialistYMQAD/AuditorSAIC/EFPD, ManagerSAIC/M&TE CustodianSAIC/REFPD/StaffSAIC/Staff AdvisorSAIC/ScientistYMQAD/AuditorSAIC/SecretarySAIC/QATSSSAIC/QA ManagerSAIC/Environmental and

Regional Programs/APMSAIC/SecretarySAIC/Sr. StaffSAIC/Deputy QA ManagerSAIC/Personnel Dept. ManagerSAICIEFPD/ScientistSAIC/Tech. DirectorIRG/ConsultantSAIC/QA EngineerYMQAD/ATL-in-TrainingYMQAD/ATLSAIC/Document Administration

Secretary

xx

xxxxxx

x xxxxxxxx

x

xxxx

xxx

xxxx

xxx

xx

xxxxxx

xxxxx

x x

xxx

x

xxxx

xxxxxx

xxx

xxx

xx

xxx

X x x


T. MoranT. PaneC. PrimeJ. PrinceG. ProwellR RidingR. RindermanP. RodgersT. RodgersK. RoesnerC. ShannonM. ShawR. SpenceJ. Statler

D. SorensenJ. SpraulT. Tait

R. TaylorM. VoegeleJ. WeaverR. WeberT. Wooderson

SAIC/Tech. SpecialistSAIC/Lead Document ServicesSAIC/Documentation Assis.SAICIHPERD ManagerSAIC/MeteorologistSAIC/Engineering ManagerSAIC/Lead Quality EngineerSAICIPM Deputy ManagerYMQAD/AuditorSAICIEFPDM&O/Records ClerkSAIC/Document ControlDOE/YMQAD/DirectorSAIClUser and Document

Services Division ManagerSAICIREFPD, ManagerNRC/ObserverSAIC/Information and Planning

Systems, ManagerSAIC/Staff FTSSAIC/Project ManagerSAIC/SCS/APMNRC/ObserverIRG/Executive Vice President

xxxxx.xxxx

X

xxxxxx

xxxxxx

xxx

xx

x

xxx

xxxxxx

xx

x

xxx

x

x

x

LEGEND:

APM = Assistant Project ManagerDept = DepartmentEFPD = Environmental Field Programs DepartmentFTS = Field Test SupportHPERD = Health Physics Environmental Radioactivity DivisionIRG = Integrated Resources GroupM&O = Management and Operating ContractorPM = Property ManagementQATSS = Quality Assurance Technical Support ServicesREFPD = Radiological/Environmental Field Programs DepartmentSr. = SeniorSCS = Site Characterization SupportTech. = TechnicalUSDC = U.S. Department of CommerceUSGS = U.S. Geological Survey


ATTACHMENT 2

AUDIT DETAILS

The following is a summary of the SAIC/T&MSS QA Program activities covered during theaudit. The list of objective evidence reviewed and specific procedures audited is provided inAttachment 3.

1.0 ORGANIZATION

The evaluation of QA Program Element 1.0 was based on selected requirements fromthe QARD and a review of the T&MSS implementing procedures as referenced by theRequirements Traceability Network (RTN) matrix. Compliance with T&MSSprocedures was based upon personnel interviews, review of the proceduralrequirements, evaluation of applicable documentation produced as a result ofprocedural implementation, interviews with SAIC/T&MSS personnel, and examinationof objective evidence to determine the degree of compliance with selectedrequirements from the OCRWM QARD and T&MSS procedure SP 1.2. The specificrequirements selected for evaluation of adequacy, compliance, and effectiveness arelisted below.

Requirements:

OCRWVM QARD, Revision 0, Section 1.0, Organization

* Each affected organization shall prepare controlled documents that describeinternal and external interfaces, organizational structures, requirements, andresponsibilities for its scope of work.

* Affected organizations shall identify the responsibilities and authorities of thoseorganizations and management positions responsible for achieving andmaintaining quality.

* Quality shall be achieved and maintained by those who have been assignedresponsibility for performing work. Achievement shall be verified by personsor organizations not directly responsible for the work.

SP 1.2, Pirpartion, Review and Approval of T&MSS QA Policy, OrganizationalDescniption and QA Program Overview

* Prepare a QA Policy that establishes the responsibilities and authorities, internaland external interfaces, associated with the T&MSS Scope of Work and QAProgram.

* Prepare this document as a formal document on form TMSS/215 (Exhibit 6).


* Prepare a review package for the individual document, include the properlyinitiated forms TMSS/095 and TMSS/098 (Exhibits 4 and 5) and the draftdocument and submit to reviewers.

* - - Assign a technical reviewer who is qualified to perform a review of thetechnical adequacy of the document and is not materially responsible for thecontent of the document.

* Upon resolution of mandatory comments, sign form TMSS/098.

Results:

The results of the review revealed that T&MSS has adequately addressed therequirements of the QARD for those selected requirements listed above. In the area ofimplementation, procedural compliance was found to be satisfactory except for oneisolated requirement in SP 1.2 which reflects that internal and external interfaces willbe addressed in the QA Policy (TMSS/93-001). No external interface descriptioncould be found in the policy. This condition was resolved during the audit and detailscan be found in Item 1, Section 5.5.2 of this report. Also, one recommendation wasmade related to the Organization Description document (TMSS/93-003) as follows.

The way the present Organizational Description document (TMSS/93-003) is written,there is a great deal of confusion regarding which responsibilities are for T&MSS, theParticipant and T&MSS, the Support Contractor. There is no clear definitioncontained in this document to differentiate between the organizational responsibilitiesof the Participant and the Support Contractor. To add to the confusion, Exhibit 2reflects an organizational chart for FTS. In the block denoting "Drilling Support andSample Management Department," Note I states: "Work may be performed under bothOCRWM and T&MSS QA Programs." It is known that the drilling support andsample management facility activities are solely OCRWM and T&MSS has noParticipant responsibilities at this time for these activities. It is recommended that theOrganizational Description document (TMSS/93-003) be evaluated and revised to moretruly reflect and differentiate those activities of the Participant versus those of theSupport Contractor. The recommendation is summarized as Item I in Section 6.0 ofthis report.

Based on the above, QA Program Element 1.0 was determined to be satisfactory.

2.0 OUAUTY ASSURANCE PROGRAM

The evaluation of QA Program Element 2.0 was based on selected requirements fromthe QARD and a review of the T&MSS implementing procedures as referenced by theRTN matrix. Compliance with T&MSS procedures was based upon personnelinterviews, review of the procedural requirements, evaluation of applicabledocumentation produced as a result of procedural implementation, interviews with


SAIC/T&MSS personnel and examination of objective evidence to determine thedegree of compliance with selected requirements from the OCRWM QARD andT&MSS procedures SP 1.31, SP 1.32, SP 1.60, SP 1.71, WI-QA-002, WI-QA-005,WI-QA-008 and WI-HR-001. The specific requirements selected for evaluation ofadequacy, compliance, and effectiveness are listed below.

Requirements:

OCRWM QARD, Revision 0, Section 2.0, QA Prgram

* Surveillances shall be conducted to evaluate the quality of selected worksubject to the QARD. Surveillances shall be documented in a report toappropriate management.

* Senior management of an affected organization shall perform or directperformance of management assessments by personnel outside the QAorganization. Management assessments shall be planned and performedannually.

* The need for readiness reviews shall be identified for major scheduled orplanned work to insure program objectives are met and that personnel havebeen trained and are qualified.

* Each affected organization shall establish a program for evaluation, selection,indoctrination, training, and qualification of personnel performing work subjectto the QARD.

SP 1.31, Initial Evaluation, Qualification, Indoctrination and Training of T&MSSPersonnel

* Employees may perform quality-affecting activities prior to ... however, themanager assigning such activities must ensure that the employee has beentrained to the document(s) governing those activities and that the training anddemonstrated proficiency...have been documented before quality-affectingactivities are performed.

* Monitor changes in documents including those that are a part of an employeesbaseline training.

* Complete Section II of the Instructor Qualification form. Retain form intraining files.

* As a minimum, training materials shall consist of identified instructionalobjectives and approved lessons plans.


SP 1.32, Management Assessment

* Initiate an assessment on an annual basis.

* Prepare a Management Assessment Report, including title and signature page,that documents completed assessment results.

SP 1.60, Readiness Review

No readiness reviews have been performed-since the last audit

SP 1.71, Gmded Application of QA Controls

* Review DOE work tasks assigned through Technical Letters of Direction orother procurement devices and assign the work task to the applicable APM.

* For each relevant OCRWM QARD section, define the applicable procedures ordocuments that apply the QA requirements to the activity.

* APM review the QAGR for technical accuracy and sign off on formTMSS/297. T&MSS QA Manager review the QAGR package for compliancewith QA program requirements and sign form TMSS/297. T&MSS ProjectManager review QAGR package for completeness and sign form TMSS/297.

WI-QA-002, Quality Assurance Surveillance

* Document deficiencies noted during the surveillance that require action on aQuality Finding Report (QFR)/Management Corrective Action Report (MCAR)or NCR.

* Complete the Surveillance Report, Surveillance Report Datasheet and anySurveillance Report Continuation Sheets.

* Approve the Surveillance Report and issue it to the T&MSS Project Manager,The responsible APM, and the responsible manager and any other appropriateparties.

WI-QA-005, Qualification of Audit Pexsonnel

This area was evaluated during the previous audit. No personnel changes haveoccurred in the way of new audit personnel.


WI-QA-008, Cerlification of Inspection Personnel

* Document the completion of training, examination, capability demonstration,and experience. Identify the method of qualification.

* Undergo visual acuity examination pursuant to the criteria provided inExhibit 5.

* Complete the candidate's Certification Record and forward copies to the LasVegas Local Records Center (LRC);

WI-HR-001, Verification of Education and Experience

* Obtain completed forms TMSS/176 and TMSS/177 from the staff member onthe first day of employment at T&MSS or as soon thereafter as possible.

* Issue a memorandum to T&MSS training staff members manager.

Results:

The results of the review revealed that T&MSS has adequately addressed therequirements of the QARD for those selected requirements listed above. In the area ofimplementation, procedural compliance was found to be satisfactory except for threeisolated instances. The conditions adverse to quality related to a surveillance reportnot being transmitted to the project manager, and grading reports not identifying QAprogram elements for procurement activities. These two conditions were resolvedduring the course of the audit and are described as Items 2 and 3 in Section 5.5.2 ofthis report. The third condition related to the lack of documentation that demonstratesindividuals performing quality-affecting activities had been trained to the proceduralrequirements prior to performing the activity. It should be noted that T&MSS employs"Real Time Training" which allows individuals to sign the applicable documentindicating that they have read and understand the procedure for performing the work.However, in this particular instance, the personnel signed this statement from threedays to over a month after the work had been completed (i.e., Document Review).Also, the block where this sign off for procedural review occurs serves a dual purpose.The signature also signifies that all major comments have been resolved. The signofffor reading and understanding the procedure is by default and not by intent. Thiscreates a bad precedence in that the block could be signed for one reason, withouthaving fulfilled the reading and understanding of the procedure portion. Thiscondition was documented on CAR YM-94-040 and is described in Section 5.5.1 ofthis report. In addition, there was one recommendation made related to the level ofdetail provided in the Management Assessment Report as follows:

The Management Assessment that was performed during September 15 through 30,1993, did not in all instances provide sufficient detail as to what the results of the


activity were based on. An example was the evaluation of QA Training. The reportreflected that QA Training was adequate and effective; however, the discussion in thereport indicated that the results of the evaluation were based on verbal discussionswith line management. In an interview with the personnel who performed theassessment, they indicated that the statement of adequacy and effectiveness was basedon reviews of the T&MSS Trend Report, audits and previous historical assessments ofthis area. The 1993 Management Assessment Report, in the area of training, did notdocument all of the evidence utilized by the assessment team to arrive at the statedconclusions in the report. It is recommended that future Management AssessmentReports include all evidence taken into account that was used to formulate the finalconclusions (i.e., adequacy and effectiveness). This recommendation is summarized asItem 2 in Section 6.0 of this report.


4.0 PROCUREMENT DOCUMENT CONTROL7.0 CONTROL OF PURCHASED ITEMS AND SERVICES

The evaluation of QA Program Elements 4.0 and 7.0 was based on selectedrequirements from the QARD and a review of the T&MSS implementing proceduresas referenced by the RTN matrix. Compliance with T&MSS procedures was basedupon personnel interviews, review of the procedural requirements, evaluation ofapplicable documentation produced as a result of procedural implementation,-interviews with SAICT&MSS personnel, and examination of objective evidence todetermine the degree of compliance with selected requirements from the OCRWMQARD and T&MSS procedures SP 1.25, SP 1.28, SP 1.69, SP 1.72, WI-QA-003, andWI-QA-007. The specific requirements selected for evaluation of adequacy,compliance, and effectiveness are listed below.

OCRWM QARD, Revision 0, Section 4.0, Prcurement Document Control

Requirements:

* Procurement documents issued by each affected organization shall include thefollowing provisions, as applicable to the item or service being procured:

- A statement of the scope of work to be performed by the supplier- Technical requirements- QA program requirements- Right of access to supplier facilities- Provisions for establishing hold points- Documentation required to be submitted- Purchaser requirements for supplier to report nonconformances- Identification of any spare and replacement parts or assemblies


* Procurement document reviews shall be performed and documented prior toissuance of the procurement documents to the supplier.

* A review of the procurement documents and any changes thereto shall be madeto verify that documents include appropriate provisions to ensure that items orservices will meet the governing requirements.

* Reviews shall ensure that all applicable technical and QA programrequirements are included.

* Reviews shall be performed by personnel who have access to pertinentinformation and who have an adequate understanding of the requirements andscope of the procurement.

* Procurement document reviewers shall include representatives from thetechnical and QA organizations.

* Procurement documents shall be approved.

* Changes to the scope of work, technical requirements, QA programrequirements, right of access, documentation requirements, nonconformances,hold points, and lists of spare and replacement parts delineated in procurementdocuments, shall be subject to the same degree of control as used in thepreparation of the original documents.

OCRWM QARD, Revision 0, Section 7.0, Control of Punrhased Items and Services

Requirements:

* Procurements shall be planned and documented to ensure a systematic approachto the procurement process. Procurement planning shall:

- Identify procurement methods and organizational responsibilities.

- Identify what is to be accomplished, who is to accomplish it, how it isto be accomplished, and when it is to be accomplished.

- Identify and document the sequence of actions and milestones needed toeffectively complete the procurement.

- Provide for the integration of activities.

- Be accomplished as early as possible, and no later than at the start ofthose procurement activities which are required to be controlled.


- Be performed relative to the level of importance, complexity, andquantity of the item or service being procured and the supplier's qualityperformance.

- Include the involvement of the QA organization.

* Supplier selection shall be based on an evaluation, performed before thecontract is awarded, of the supplier's capability to provide items or services inaccordance with procurement document requirements.

* The organizational responsibilities for source evaluation and selection shall beidentified and shall include the QA organization.

* Measures for evaluating and selecting procurement sources shall include one ormore of the following elements:

- Evaluation of the supplier's history- Evaluation of supplier's current quality assurance records- Evaluation of the supplier's technical and quality capability

* The results of procurement source evaluation and selection shall bedocumented.

* The proposal/bid evaluation process shall include a determination of the extentof conformance to the procurement document requirements.

* Before the contract is awarded, the purchaser shall resolve, or obtaincommitments to resolve, unacceptable quality conditions identified during theproposal/bid evaluation.

* Supplier QA programs shall be evaluated either before or after contractplacement, and any deficiencies that would affect quality shall be correctedbefore starting work subject to QARD requirements.

* Supplier QA programs shall be accepted by the purchaser before the supplierstarts work subject to QARD requirements.

* The purchaser of items and services shall establish measures to interface withthe supplier and to verify supplier's performance.

* The extent of purchaser verifications shall be a function of the relativeimportance, complexity, and quantity of items or services being procured, andthe supplier's quality performance.


* Purchaser verifications shall be conducted as early as practical and shall notrelieve the supplier of the responsibility for the verification of qualityachievement.

* - Supplier generated documents shall be controlled, processed, and accepted inaccordance with the requirements established in the procurement documents.

* Measures shall be implemented to ensure that the submittal of these documents-is accomplished in accordance with the procurement document requirements.

* Methods for accepting supplier furnished items or services are appropriate tothe items or services being procured.

* The supplier shall verify that furnished items or services comply with thepurchaser's procurement requirements before offering the items or services foracceptance.

* The supplier shall provide the purchaser objective evidence that items orservices conform to procurement documents.

* Where design specifies the use of commercial-grade items, prescribedrequirements are an acceptable alternative to other requirements of this section.

SP 128, Procurement of Quality Affecting Hardwane and Services

Requirements:

* Determine whether the hardware or service to be procured is quality-affecting,quality-affecting commercial-grade, or non-quality affecting.

* Prepare a Purchase Requisition (PR) and applicable supporting documentation.

* Document the determination of applicable quality requirements for quality-affecting hardware and services.

* Prepare a procurement quality specification based on applicable requirements.

* - Review the PR package to ensure that technical requirements are adequate.

* If potential suppliers identified are not on the Qualified Suppliers List (QSL),evaluate and approve the potential suppliers.

* Initiate a Purchase Order (PO) based upon the requirements of the PR and thebid analysis.


* Review the PO to ensure that technical requirements are consistent with thePR.

* Review the PO to ensure that the PR is consistent with QA requirements.

* Perform acceptance of hardware or service(s).

* Upon receipt of documentation for PO closure, compile a record package ofdocumentation required to close the PO.

* The procurement staff shall submit a quality-affecting record package to theLas Vegas LRC.

SP 1.69, Obtaining Services Through the Technical Field Work Request Process

Requirements:

* Direct the use of the Technical Field Work Request (TFWR) system forobtaining quality-affecting services equipment from another participant.

* Assemble a TFWR package.

* Review the TFWR package to ensure that the technical and qualityrequirements are adequate.

* Perform acceptance of the service.

* Submit the QA records package classified as "lifetime."

SP 1.25, Acceptance of Hariiare and Services

Requirements:

* Perform and document the required reviews and verification of technical datasubmitted.

* Perform and document acceptance of services as required by the AcceptanceReport (AR) and the procurement documents.

* Verify that all inspections and reviews required by the AR and the procurementdocuments are complete and accurate.

* Provide a copy of the AR and any support documentation of the requester forwork orders or to the procurement organization for POs.


* Prepare and submit QA records packages as "lifetime" records.

WI-QA-003, Supplier Evaluation

Requirements:

* Determine whether the supplier evaluation is to be accomplished by facilityaudit or quality records reviews.

* Plan and perform supplier evaluation audit.

* Determine appropriate elements that are to be evaluated based on therequirements of the procurement documents.

* Document the results of the supplier audit.

* Document the results of the quality records review.

* Review the supplier's history in furnishing identical or similar products thatperform satisfactorily in actual use.

* Document the results of the history review.

* Re-evaluate each supplier shown on the QSL annually.

* Document the results of the annual re-evaluation.

* Suppliers previously audited shall be scheduled for reaudit three years after thedate of the original audit.

* Document the results of the audit on the Supplier Evaluation Report.

* Reschedule the supplier for its next triennial audit three years after the date ofthis triennial audit.

* Records generated shall be classified as "lifetime."

WI-QA-007, Maintenance of Qualified Supplier Uist

Requirements:

* A new QSL is issued to coincide with each calendar year quarter.

* Verify that the annual re-evaluations, if due, have been performed for eachsupplier.


* For each supplier, initiate a new QSL page.

* Initiate new QSL Index.

* Obtain T&MSS QA Manager's signature.

* Distribute QSL to appropriate users.

* Prepare and submit QA record packages as 'lifetime" records.

SP 1.72, Upgrade of Items Procured as Non-quality Affecting

There has been no implementation of this procedure.

Results:

The results of the review revealed that T&MSS has adequately addressed therequirements of the QARD for those selected requirements listed above. In the area ofimplementation, procedural compliance was found to be satisfactory based on a reviewof selected procurements and associated forms: PRs, POs, Standard QA Clauses,Checklists for the Preparation and Review of Quality Affecting ProcurementDocuments, Acceptance Reports, Technical Field Work Request Review Checklists,supplier qualifications and associated Supplier Evaluation Reports, Supplier EvaluationChecklist Cover Sheets, Supplier Evaluation Checklists, Supplier Evaluation Checklist- Calibration Services, QA Audit Report Checklists, Audit Reports, QSLs, QSLIndexes, and QSL Cover Pages.

Based on the above, QA Program Elements 4.0 and 7.0 were determined to besatisfactory.

5.0 IMPLEMENTING DOCUMENTS

The evaluation of QA Program Element 5.0 was based on selected requirements fromthe QARD and a review of the T&MSS implementing procedures as referenced by theRTN matrix. Compliance with T&MSS procedures was based upon personnelinterviews, review of the procedural requirements, evaluation of applicabledocumentation produced as a result of procedural implementation, interviews withSAIC/T&MSS personnel, and examination of objective evidence to determine thedegree of compliance with selected requirements from the OCRWM QARD andT&MSS procedures SP 1.1 and SP 2.3. The specific requirements selected forevaluation of adequacy, compliance, and effectiveness are listed below.


Requirements:

OCRWM QARD, Revision 0, Section 5.0, Implementing Documents

* Work shall be performed in accordance with controlled implementingdocuments. Documents shall contain:

- Responsibilities of the organization affected by the document- Technical and regulatory requirements- Description of work performed- Acceptance criteria- Prerequisites and environmental conditions- Verification points/hold points- Lifetime records required

* Documents are to be reviewed, approved, and controlled.

* Individuals are to comply with approved implementing documents.

SP 1.1/SP 2.3, Prepamtion, Review and Appoval of T&MSS Procedures

* Custodian prepares and processes the procedure in accordance with the stepsoutlined on form T&MSSI302.

* Responsible APM assigns a technical reviewer who is qualified to perform areview of the technical adequacy of the procedure and is not materiallyresponsible for the content of the procedure.

* Responsible APM documents the competence of the technical reviewer.

* Technical reviewers review the procedure in accordance with the steps outlinedon form T&MSS/3 1.

* Custodian prepares and processes the procedure in accordance the stepsoutlined on form T&MSS/303.

Results:

The results of the review revealed that T&MSS has adequately addressed therequirements of the QARD for those selected requirements listed above. In the areaof implementation, procedural compliance was found to be satisfactory based on areview of procedure preparation, procedure revision, ICN issuance, referenced orincluded form updating, and procedure cancellation.



6.0 DOCUMENT CONTROL

The evaluation of QA Program Element 6.0 was based on selected requirements fromthe QARD and a review of the T&MSS implementing procedures as referenced by theRTN matrix. Compliance with T&MSS procedures was based upon personnelinterviews, review of the procedural requirements, evaluation of applicabledocumentation produced as a result of procedural implementation, interviews withSAIC/T&MSS personnel, and examination of objective evidence to determine thedegree of compliance with selected requirements from the OCRWM QARD andT&MSS procedures SP 1.34 and WI-QA-014. The specific requirements selected forevaluation of adequacy, compliance, and effectiveness are listed below.

Requirements:

OCRWM QARD, Revision 0, Section 6.0, Document Control

* Documents that specify requirements or prescribe work are to be controlled.

* Document preparation and maintenance shall be assigned to the appropriateorganization.

* Technical documents shall be reviewed for:

- Adequacy- Correctness- Completeness

* Organizational position responsible for approving release of the document shallbe identified.

* Documents and changes thereto shall be controlled.

* Documents that control work shall be at work locations and used.

* Changes to documents shall be controlled in the same manner as the originaldocument.

SP 1.34, T&MSS Document Control

* Document Control Center (DCC) reviews the document distribution package toensure inclusion of all necessary documentation for compliance with governingprocedures, and for correct completion of the Controlled Document IssuanceAuthorization (CDIA).


* DCC assembles controlled documents and stamps with the controlled copystamp.

* DCC transmits the document copies, accompanied by the DocumentTransmittal/Acknowledgement Record (DTAR), to the distribution list ofDocument Holders (DHs) identified on the CDIA and to other DHs asidentified by T&MSS management.

* DHs return the signed DTAR to the DCC on or before the due date indicated.

* DCC issues a Reminder Notice, form TMSS/021 to the DH if the DTAR is notreceived within five working days of the due date indicated on the DTAR.

* DCC issues a Decontrol Notice, form TMSS/033, if the Reminder Notice hasnot been received within five working days of the due date indicated on theReminder Notice.

* DCC removes the decontrolled DH from the distribution list for the document.

* DCC forwards the DTAR to the DH of the cancelled document.

* DH returns the signed DTAR to the DCC on or before the due date.

WI-QA-014, T&MSS Quality Assumance Reviews of Quality Documents

* Performs document review in accordance with the controlling procedure usingSP 1.1, SP 1.2, or SP 1.35 QA Document Review Checklist, form TMSS/245.WI-QA-014, Revision 0, dated August 31, 1992, was subsequent to the issue ofthe SUP and meteorological study and not used for the review. WI-QA-014was invoked for the QA review of SP 1.34.

* Records comments on Document Review Comment form TMSS/095 andcomplete form T&MSS/098, as appropriate. If there are no comments, signand date form TMSS/098.

* Enters the following document review information into the Document ReviewLog:

- Designated QA reviewer- Review completion date- Timing and status indicator- Review status code: A = Approved

C = Approved with commentsD = DisapprovedS = Signed


Results:

The results of the review revealed that T&MSS has adequately addressed therequirements of the QARD for those selected requirements listed above. In the area ofimplementation, procedural compliance was found to be satisfactory based upon areview of document distribution packages which includes the following: stampedcontrolled documents, listings of dates of distribution for selected procedures, listingsof dates the DTARs were returned-by the DHs for these selected procedures, listingsof dates of issue of DTARs for selected cancelled procedures, dates these DTARs werereturned to DCC, record packages for selected procedures reviewed by the T&MSSQA organization, and review of the QA Document Review Log for selected attributes.


10.0 INSPECTION

The evaluation of QA Program Element 10.0 was based on selected requirements fromthe QARD and a review of the T&MSS implementing procedures as referenced by theRTN matrix. Compliance with T&MSS procedures was based upon personnelinterviews, review of the procedural requirements, evaluation of applicabledocumentation produced as a result of procedural implementation, interviews withSAIC/T&MSS personnel, and examination of objective evidence to determine thedegree of compliance with selected requirements from the OCRWM QARD andT&MSS procedure SP 1.25. The specific requirements selected for evaluation ofadequacy, compliance, and effectiveness are listed below.

Requirements:

OCRWM QARD, Revision 0, Section 10.0, Inspection

* Identification of characteristics to be inspected.

* Identification of inspection methods to be employed.

* Identification of inspector qualification level.

* Identification of acceptance criteria.

* Recording of inspection results.

* Final inspections include a review of the results and resolution ofnonconformances.

Audit ReportYMP-94-O5 of 61

* Inspection documentation identifies: items, date of inspection, name ofinspector, type of observation, acceptance criteria, results of inspection andreference to nonconformances.

* Education and experience requirements for inspection personnel qualification isdocumented.

* Initial documentation of personnel qualification including annual updates.

SP 1.25, Acceptance of Hardware and Services

* Receipt inspection instructions identify criteria, reference documents, andinspections to be performed.

* Receipt inspection documentation identifies results of inspection, inspectorqualification level, item identification and nonconformances.

* Receipt inspection includes checking for shipping damage, packagingrequirements and hardware received was hardware ordered or sent forcalibration.

Results:

The results of the review revealed that T&MSS has adequately addressed therequirements of the QARD for those selected requirements listed above. In the area ofimplementation, procedural compliance was found to be satisfactory based upon areview of the receipt inspection records packages associated with selected POs. Therecords included in the package were the acceptance report, based upon a review ofthe Receipt Inspection (RI) matrix and RI instructions.


12.0 CONTROL OF MEASURING AND TEST EOUIPMENT

The evaluation of QA Program Element 12.0 was based on selected requirements fromthe QARD and a review of the T&MSS implementing procedures as referenced by theRTN matrix. Compliance with T&MSS procedures was based upon personnelinterviews, review of the procedural requirements, evaluation of applicabledocumentation produced as a result of procedural implementation, interviews withSAIC/T&MSS personnel, and examination of objective evidence to determine thedegree of compliance with selected requirements from the OCRWM QARD andT&MSS procedures WI-RED-006 and WI-MET-001. The specific requirementsselected for evaluation of adequacy, compliance, and effectiveness are listed below.


Requirements:

OCRWM QARD, Revision 0, Section 12.0, Control of Measuxing and Test Equipment

* - Requirements are established to ensure M&TE is properly controlled, calibratedand maintained.

* Calibration standards shall have a greater accuracy than the required accuracyof the M&TE being calibrated.

* The method and interval of calibration for each device shall be defined, basedon the type of equipment, stability characteristics, required accuracy, intendeduse, and other conditions affecting measurement control.

* Calibrated measuring and test equipment shall be labeled, tagged, or otherwisesuitably marked or documented to indicate due date or interval of nextcalibration.

* Calibrated measuring and test equipment shall be uniquely identified to providetraceability to its calibration data.

* M&TE calibration documentation shall include the following information:

A. Identification of the M&TE calibrated.

B. Traceability to the calibration standard used for calibration.

C. Calibration data.

D. Identification of the individual performing the calibration.

E. Identification of the date of calibration and the recalibration due date orinterval, as appropriate.

F. Results of the calibration and statement of acceptability.

G. Reference to any actions taken in connection with out of calibration ornonconforming M&TE including evaluation results, as appropriate.

H. Identification of the implementing document (including revision level)used in performing the calibration.

*M&TE shall be properly handled and stored to maintain accuracy.


WI-RED-006, Contnrl of Measuring and Test Equipment

If calibration standards with a greater accuracy than required of the M&TEbeing calibrated do not exist...calibration standards with equal accuracy may beused...; the basis for calibration acceptance is documented and authorized bythe REFPD Division Manager.

* Calibrations of M&TE and Operating Equipment (OE) may be performed byeither qualified suppliers or REFPD personnel using approved T&MSSprocedures.

* REFPD personnel are qualified to used this procedure through completion oftraining in accordance with SP 1.31.

* Selection of M&TE shall be controlled to assure that such hardware are ofproper type, range, tolerance and accuracy to accomplish the desired function.

* Procure M&TE/OE and associated services in accordance with SPs 1.27, 1.28or 1.29.

* If the as-found condition was out of calibration (out-of-tolerance), perform thefollowing actions:

A. If the equipment is quality-affecting, issue an NCR.

B. Perform a documented evaluation that determines the validity andacceptability of measurements performed since the last calibration.Inspections or tests shall be repeated as necessary on hardwaredetermined to be suspect.

* M&TEIOE is considered out of calibration and shall not be used untilcalibrated if any of the following conditions exist:

A. The calibration due date or interval has passed without calibration.

B. The device produces results known or suspected to be in error.

* Enter the following information on the label:

A. Annotate "vendor" in the "BY" area and initial.

B. Date calibrated.

C. Due date for next calibration, if the device will not be placed in storage(Omit this step if the device will be placed in storage).


D. T&MSS Property Identification (ID) number of the device.

Provide for datasheet(s) to record data during calibration. Completeddatasheet(s) shall be the certificate of calibration. Therefore, the followinginformation shall be included as a minimum:

1. Identification of equipment calibrated--manufacturer, model and serialnumbers, and identification numbers as applicable.

2. Identification of calibration standards--manufacturer, model, serialnumbers, and calibration date.

3. Accuracy of the principal calibrating instrument(s).

4. Calibration procedure and revision number.

5. Calibration data reflecting "as-found" (prior to calibration) condition, ifapplicable; and as-left (after calibration) condition.

6. Specific readings of environmental conditions affecting the calibration,such as temperature, humidity, vibration, as appropriate.

7. A statement at the bottom of the last datasheet recording a successfulcalibration, which indicates the results and acceptability of thecalibration.

8. Space for the signature of the individual performing the calibration anddate of calibration.

9. Space for the Responsible Manager's (RMs) (or designee's) signature orinitials and date documenting the review of calibration data.

* Establish a controlled storage area in which to store M&TE and calibrationstandards so that accuracy is maintained.

* Sign and date the M&TE and Standards Usage Log to document check-outand check-in of M&TE. Record how and where M&TE was used and anydamage sustained.

* Notify the M&TE/OE Custodian if any M&TE is removed from service. Ifout-of-tolerance M&TE/OE has been used for quality-affecting work, initiate anNCR.

* Implement a Recall System by issuing notices to RMs at least two months priorto the calibration due date of M&TE only.


* Ensure that calibrations are performed in accordance with the establishedfrequency to maintain the required accuracy.

* Issue a second recall notice to the RM at least one month before the calibrationdue date.

* If the active M&TE is not calibrated by the end of the due month, issue anotice to the RM requiring deactivation and placement of an Out of Service tagon the M&TE.

* When a missing M&TE/OE has been designated as lost, submit a memo to theM&TE/OE Custodian and the RM detailing efforts made to locate the missingM&TE/OE.

* If lost, M&TE/OE was used for quality-affecting work since last calibration,issue an NCR to ensure that affected data is evaluated for impact by thepotentially nonconforming condition.

* (Technician[s]) perform the calibration in accordance with the applicableREFPD WI. If found out-of-tolerance during the calibration and there is animpact on quality-affecting data, prepare an NCR.

* Review the history files of all M&TE/OE to determine if the calibrationfrequency should be adjusted to a longer or shorter recall cycle (based on typeof equipment, stability characteristics, required accuracy, intended use, andother conditions affecting measurement).

* Recommend, in writing, to the REFPD Manager to extend or shorten thecalibration interval if appropriate. (REFPD Manager) Accept or reject therecommendation and notify M&TE/OE Custodian.

* Submit the approved calibration interval adjustment documentation as a record.

* Establish a technical justification for storing calibrated M&TE/OE withoutactivating the prescribed calibration period.

* Record for the REFPD Manager the maximum length of time the M&TE canbe stored and still receive the full calibration period upon activation.

* Update the M&TEIOE history file (the calibration due date for stored items islisted as the maximum shelf life plus the established period betweencalibrations).


Provide a storage area with controlled access, access log, a storage Datasheetcontaining the DOE/T&MSS property number, a brief item description, itemserial number, date of calibration, date placed in storage, and signature(s) ofpersonnel placing or removing the item in or from storage.

* M&TE/OE removed from storage and placed in service cannot be returned tostorage without recalibration.

WI-MET-001, Test, Checks, and Audits of Meteowlogical Equipment

* Calibrations performed by the EFPD are to be documented on form TMSS/259,which also becomes the calibration certificate.

* The acceptable tolerance limits are established in Exhibit 4.

* Technical procedures for the equipment in use are shown below:

EQUIPMENT CALIBRATION PERFORMANCE CHECK& AUDIT

Wind Direction 4.1* 4.10*Wind Sensors 4.2* 4.11*Temperature 4.3* 4.12*Delta Temperature 4.4* 4.13*Precipitation 4.5* 4.14*Barometric Pressure 4.6* 4.15*Humidity/Dew Point 4.7* 4.16*

* Paragraph numbers

* Wind Direction and Wind Speed sensors are to be sent to qualified outsidevendors for calibration. The results are to be documented on calibrationcertificates provided by the calibrating agent.

* Calibration of Temperature Sensors

- Place the M&TE Standard and the system sensor in an environment thatis near zero degrees Celsius.

- Record the standard and sensor readings after they stabilize.

- Repeat steps for two different temperature environments (between 0degrees and 40 degrees Celsius).


- Calculate the differences between the results from the standard and thesensor being calibrated. The acceptable tolerance for the difference is+/- 0.5 degrees Celsius.

Performance checks and audits are to be performed on active monitoringsystems once during each calendar quarter.

Results

The results of the review revealed that T&MSS has adequately addressed therequirements of the QARD for those selected requirements listed above. In the area ofimplementation, procedural compliance was found to be satisfactory based uponexamination of site facilities and equipment, calibration standards, calibration historyfiles, and reviews of technical procedures for calibration and control of M&TE.Several recommendations were made relative to enhancing and clarifying the program.These include a recommendation that T&MSS procedure WI-RED-006, Revision 2,Paragraph 4.2.2b, be clarified to address that performance audit and performance checkdocuments may be kept in a central history file rather than in each individualM&TE/OE history file folder; the calibration protocols described in T&MSS procedureWI-MET-001, Revision 4, be clarified to clearly reflect the calibration standards to beused in the calibration activity; and, that the documentation notifying the manager thatM&TE has been lost (Paragraph 4.7.1 of WI-RED-006, Revision 2) include theevaluation of impact to data or equipment (e.g., that an NCR was or was not issued) inaddition to depicting the steps taken to locate the missing M&TE/OE. Thisrecommendation is summarized as Items 3, 5, and 6 in Section 6.0 of this report. Onecondition adverse to quality concerning the calibration of a Wind Speed sensor wasidentified and corrected during the audit and is discussed as Item 4 in Section 5.5.2 ofthis report.


15.0 NONCONFORMANCES

The evaluation of QA Program Element 15.0 was based on selected requirements fromthe QARD and a review of the T&MSS implementing procedures as referenced by theRTN matrix. Compliance with T&MSS procedures was based upon personnelinterviews, review of the procedural requirements, evaluation of applicabledocumentation produced as a result of procedural implementation, interviews withSAIC/T&MSS personnel, and examination of objective evidence to determine thedegree of compliance with selected requirements from the OCRWM QARD andT&MSS procedures SP 1.23, WI-QA-006 and YAP-15.1Q. The specific requirementsselected for evaluation of adequacy, compliance, and effectiveness are listed below.


Requirements:

OCRWM QARD, Revision 0, Section 15.0, Nonconformances

* -NCRs clearly identify the nonconforming condition.

* NCRs document recommended dispositions and need for corrective action.

* -Dispositions are evaluated and approved.

* Responsibilities and authority for reviewing, evaluating, and approving thedisposition, and closing of NCRs are specified.

* Nonconforming -items are controlled pending approval of disposition.

* Technical justification of the acceptability of NCRs dispositioned, repaired oruse-as-is is documented.

* Specifying documents are revised to reflect as-built conditions resulting fromuse-as-is NCRs.

YAP-15.1Q, Control of Nonconformances

* NCRs are initiated for identified nonconformances and describe the conditionand requirements violated.

* NCRs identify the type of disposition (e.g., record of repairs).

* Justification is documented for all dispositions.

* The QA organization documents their concurrence with the disposition andreviews for reportability.

* The QA organization determines the need for corrective action.

* The performing organization completes the disposition and signs and dates theNCR to signify completion.

* The QA organization documents acceptance of final review and transmits NCRto NCR Coordinator.

* The NCR log is maintained up-to-date.

* Revised NCR are clearly identified with a delta symbol.


* Reworked instruments are recalibrated.

WI-QA-006, Trend Analysis and SP 1.23 Nonconfonnance Reporting

* Trend codes are assigned to each NCR.

* Trend analyses are performed on NCRs and plotted. Plotted data is reviewedto identify trends.

* Quality trend reports are prepared. -

Results:

The results of the review revealed that T&MSS has adequately addressed therequirements of the QARD for those selected requirements listed above. In the area ofimplementation, procedural compliance was found to be satisfactory based upon areview of selected NCR packages. These NCRs were the first processed in accordancewith YAP-15.1Q.


16.0 CORRECTIVE ACTION

The evaluation of QA Program Element 16.0 was based on selected requirements fromthe QARD and a review of the T&MSS implementing procedures as referenced by theRTN matrix. Compliance with T&MSS procedures was based upon personnelinterviews, review of the procedural requirements, evaluation of applicabledocumentation produced as a result of procedural implementation, interviews withSAIC/T&MSS personnel, and examination of objective evidence to determine thedegree of compliance with selected requirements from the OCRWM QARD andT&MSS procedures SP 1.37 and WI-QA-006. The specific requirements selected forevaluation of adequacy, compliance, and effectiveness are listed below.

Requirements:

OCRWM QARD, Revision 0, Section 16.0, Conective Action

* Conditions adverse to quality are to be identified.

* Conditions adverse to quality need to be evaluated to distinguish significantconditions.

* Conditions are to be documented and reported to appropriate management.


* Appropriate/responsible manager is to investigate condition and documentresults.

* Corrective actions are to be documented and evaluated for adequacy by the QAorganization.

* Significant conditions are to be evaluated for stop work.

* Quality organization shall follow-up to verify implementation of correctiveaction taken.

* Trend Analysis Reports (TARs) shall be periodically prepared and evaluated forpotential adverse quality trends.

SP 1.37, Deficiency Repouting System

* Conditions adverse to quality are evaluated to determine if a potential StopWork Order condition exists.

* Form TMSS/057 is used to document QFR/MCAR conditions.

* QFR/MCAR conditions are evaluated to determine whether the adversecondition constitutes a significant condition adverse to quality. Block 11 ofForm TMSS/057, Exhibit 6, contains examples of MCAR conditions.

* TMSS/057 forms have Block 2 completed.

* A preliminary trending code is assigned on Block 13, of Form TMSS1057.

* QA manager signs and' dates Block 14 of Form TMSS/057

* A status log exists for tracking QFR/MCAR conditions and information enteredincludes:

- Unique Number- Responsible Organization- -Date of Issue- Date of Response and Closure- Whether a Stop Work condition was identified

* Distribution of the QFR/MCAR to:

- T&MSS Project Manager- QA Manager


- Applicable APMs- Other interested parties

* Responses are provided and evaluated by the specified due date. Request forextensions are processed if appropriate.

* Responses to QFR/MCAR conditions follow format of Exhibit 7.

* Responses fully address the adverse condition.

* Revision to QFRs/MCARs are properly made.

* Corrective action implementation is verified prior to close-out of theQFR/MCAR -

* The QA record package for each QFR/MCAR is submitted within 10 workingdays of the closure date on the QFR/MCAR. Records are to include:

- TMSS/057 completed form- Memorandum(a) generated by the procedure- QFR/MCAR responses and relevant correspondence- Additional correspondence and close out notification

WI-QA-006, Trend Analysis

* A trend analysis code is assigned to each QFR.

* TARs are prepared on a quarterly basis (every three months).

* The following documents are used to compile the report:

- NCRs- QFRs- MCARs- DOE CARs- NCR Findings

* Trend information is plotted on charts.

* Current information is compared to previously issued TARs.

* TARs contain the following, as appropriate:

- Summary- Positive/negative trends


- Trend analysis charts- Results of any investigations for potential adverse trends- Identification of Deficiency Reports as a result of investigations

* - QA Manager directs investigations to determine if actual adverse trends exist.Actions are documented via surveillances, NCRs, QFRs, MCARs, etc.

* Trend reports are distributed to:

- T&MSS PM- T&MSS APM- YMQAD Director

* A QA records package is submitted to the LRC containing the TAR and theissuing memorandum.

Results:

The results of the review of selected Trend Analysis Reports revealed that T&MSS hasadequately addressed the requirements of the QARD for those selected requirementslisted above. In the area of implementation, procedural compliance was found to besatisfactory.


17.0 OUALITY ASSURANCE RECORDS

The evaluation of QA Program Element 17.0 was based on selected requirements fromthe QARD and a review of the T&MSS implementing procedures as referenced by theRTN matrix. Compliance with T&MSS procedures was based upon personnelinterviews, review of the procedural requirements, evaluation of applicabledocumentation produced as a result of procedural implementation, interviews withSAIC/T&MSS personnel, and examination of objective evidence to determine thedegree of compliance with selected requirements from the OCRWM QARD andT&MSS procedure SP 1.36. The specific requirements selected for evaluation ofadequacy, compliance, and effectiveness are listed below.

Requirements:

OCRWM QARD, Revision 0, Section 17.0, Quality Assurance Records

* QA records shall be classified as lifetime or nonpermanent.

* Implementing documents shall identify those documents that will be come QArecords and identify the organization responsible for submitting the QA records.


* Individuals creating QA records shall ensure that the QA records are legible,accurate, and complete.

Individuals handling QA records shall protect them from damage or loss.

* Records shall be considered QA records when stamped, initialed, or signed anddated as complete.

* -QA records may be originals or copies.

* Corrections shall include the initials or signature of the person authorized tomake the correction and the date the correction was made.

* Corrections to QA records shall be approved by the originating organization.

* Organizations originating the QA records shall develop implementingdocuments that identify means for replacement, restoration, or substitution oflost or damaged QA records.

SP 1.36, T&MSS Records Management

* Ensure that procurement documents, implementing procedures, or otherdocuments directing the conduct of Yucca Mountain Site CharacterizationOffice (YMSCO) quality-related activities identify the records or recordspackages to be generated, supplied, or maintained.

* Turn over YMSCO records to the responsible department or transmit them toRecords Administration.

* Transmit records to the LRC or forward to a record source.

* Clearly mark "Privileged" on records that have been identified as such.

* Limit access to privileged records.

* Protect records or documents destined to become records or records packages.

* Submit all records and record packages to T&MSS Records Administrationpersonnel for internal review.

* Maintain in-process records so that they are retrievable.

* Transmit T&MSS generated records and record packages to the LRC.


Results:

The results of the review revealed that T&MSS has adequately addressed therequirements of the QARD for those selected requirements listed above. In the area ofimplementation, procedural compliance was found to be satisfactory based upon areview of selected QA Records Packages associated with the various procurementsevaluated under QA Program Elements 4.0 and 7.0.

Based on above, QA Program Element 17.0 was determined to be satisfactory.

18.0 AUDHM

The evaluation of QA Program Element 18.0 was based on selected requirements fromthe QARD and a review of the T&MSS implementing procedures as referenced by theRTN matrix. Compliance with T&MSS procedures was based upon personnelinterviews, review of the procedural requirements, evaluation of applicabledocumentation produced as a result of procedural implementation, interviews withSAIC/T&MSS personnel, and examination of objective evidence to determine thedegree of compliance with selected requirements from the OCRWM QARD andT&MSS procedures WI-QA-001 and WI-QA-005. The specific requirements selectedfor evaluation of adequacy, compliance, and effectiveness are listed below.

Requirements:

OCRWM QARD, Revision 0, Section 18.0, Audits

* Internal audits are scheduled to provide coverage, consistency and coordinationwith ongoing work at intervals consistent with the schedule for accomplishingthe work.

* The audit schedule is developed annually and revised to reflect changingconditions.

* Checklist identifies specific requirements to be evaluated per element.

* Objective evidence is examined to determine if QA program elements are beingimplemented effectively.

* Adverse conditions are identified and documented.

* Audit reports are issued to management.

* Audit reports identify documents reviewed; persons interviewed; results(summary of checklist contents), and a statement of the QA programeffectiveness.


* Adverse conditions are adequately described to enable corrective action to betaken by the audited organization.

* Follow-up action is taken to verify corrective action is accomplished.

* Lead auditors maintain their proficiency by regular and active participation inthe audit process.

WI-QA-001, QA Audits

* The internal audit schedule was determined using the audit schedule checklist.

* The audit schedule is issued to OCRWM Director of QA, YMQAD Director,T&MSS Project Manager and applicable APM

* Audit checklists are prepared and used for each element to be evaluated.

* Pre and Post audit conferences are conducted by the Lead Auditor.

* Audit reports are issued to YMQAD Director, T&MSS Project Manager, APMand QA Manager.

WI-QA-005, Qualification of Audit Personnel

* Lead Auditor qualification is documented on TMSS/149, Record of Initial LeadAuditor Qualification.

* Lead Auditor annual evaluation form TMSS/299 is maintained current.

Results:

The results of the review revealed that T&MSS has adequately addressed therequirements of the QARD for those selected requirements listed above. In the area ofimplementation, procedural compliance was found to be satisfactory based on a reviewof selected internal audit reports. These audits represent all the internal auditsperformed since the last QATSS audit of T&MSS. One deficiency was noted.Review of objective evidence disclosed that a performance audit identified aninstrument out-of-tolerance which was replaced. This condition was not reported onan NCR. This deficiency was corrected during the audit, as noted in Item 5 in Section5.5.2 of this report.



SUPPLEMENT m SCIENTIIC VESTIGATION

The evaluation of Supplement Ill was based on selected requirements from the QARDand a review of the T&MSS implementing procedures as referenced by the RTNmatrix. Compliance with T&MSS procedures was based upon personnel interviews,review of the procedural requirements, evaluation of applicable documentationproduced as a result of procedural implementation, interviews with SAIC/T&MSSpersonnel, and examination of objective evidence to determine the degree ofcompliance with selected requirements from the OCRWM QARD and T&MSSprocedures SP 2.2, WI-MET-002, WI-MET-003, and WI-MET-010. The specificrequirements selected for evaluation of adequacy and compliance are listed below.

Requirements:

OCRWM QARD, Revision 0, Supplement I, Scientific Investigations

* Scientific investigations shall be performed using scientific notebooks, technicalimplementing documents (procedures), or a combination of both.

* Technical implementing documents used to perform scientific investigationsshall meet the requirements of Section 5.0 and 6.0.

* Data shall be identified to provide traceability, indicate usability, and documentvalidation status. Identification and traceability shall be maintained throughoutthe lifetime of the data.

* Data collected under the QARD shall be subject to validation, including:

- A review of associated documentation to establish technical adequacy,suitability for intended usage, and the satisfaction of QA anddocumentation requirements.

- The results of the data review shall be documented.

The reviewer shall be independent from the collector.


* Validation of existing data shall require documenting and subject to review, theknown level of confidence associated with the data to establish the adequacyand suitability of the data for the intended use.

* - Data reduction and transfer shall be controlled to permit independentreproducibility by another qualified individual.

SP 2.2, Scientific Investigation Control

* Scientific investigations shall be performed in accordance with approvedimplementation packages, using scientific notebooks, technical implementingdocuments (procedures), or a combination of both.

* Data collected shall be uniquely identified throughout the lifetime of the datato provide traceability, indicate its usability, and document its validation status.The data shall be validated and qualified to establish the adequacy andsuitability for the intended use.

* If the scientific investigation is part of a Study Plan or a field test, exit thisprocedure and proceed in accordance with the applicable YMSCO procedure.

* Compile a scientific investigation implementation package. Ensure that theimplementation package contains the following:

- Identification of QAGR

- Scientific Investigation Approval Memorandum

- Scope of Work

- Work Breakdown Structure (WBS) Element Reference

- Schedule

- Specification for use of scientific notebooks, technical implementingdocuments (procedures), or a combination of both

* The RM reviews the compiled data and associated documentation, orcoordinates the review by a qualified reviewer, to establish technical adequacy,suitability for intended use, and compliance with QA and documentationrequirements.

* Ensures independence of the reviewer from the data collector.


* The RM controls the data interpretation and analysis in accordance withapplicable guidelines in Exhibit 8.

* If existing data are in the package, validate data by reviewing and documentingthe known level of confidence associated with the data in accordance with AP-5.9Q.

* If deviations or deficiencies are identified document the deficiency in-accordance with SP 1.37.

WI-MET-002, Routine Openations and Maintenance of Meteorological Equipment

* This procedure applies to the meteorological monitoring program operated bythe EFPD (e.g. implements the Study Plan).

* Do Performance Checks and Acceptance Tests of equipment installed followingWI-MET-001.

* Data shall be collected from the sites at least five working days after the firstday of each calendar month.

* Procedures for the retrieval of data from the magnetic storage media into dataprocessing computers are given in WI-MET-003.

WI-MET-003, Meteorological Monitoring: Data Processing Instnzctions

* This procedure describes the process in acquiring, archiving, tracking,checking and reformatting meteorological data.

* Flowchart for Data Processing (Exhibit 4):

- Site technician downloads site data to PC- Data Clerk archives data to WORM- Data Clerk Copy download data files to network directory- Notify Database Manager of Transfer- Data Clerk Input file to database- Data Analyst edit database- Database Manager archive edited data to WORM- Select parameters for annual reports- Data Analyst generate stability parameter and wind roses- Notify report writer

WI-MET-010, Special Meteorological Field Measurements

No implementation of this procedure has occurred since the last audit.


Results:

The results of the review revealed that T&MSS has adequately addressed therequirements of the QARD for those selected requirements listed above. In the area ofimplementation, procedural compliance was found to be satisfactory based uponexamination of site facilities and operating equipment, data and reports, and reviewsof Study Plans, SlIP, technical procedures, and records. Several recommendationswere made relative to enhancing and clarifying the program. These include arecommendation that T&MSS procedure SP 2.2, Revision 4, Paragraphs 4.6.3 and4.6.4, be revised to reflect the "verification-of existing data" be performed inaccordance with YAP-S3I.1Q, rather than AP-5.9Q which has recently beensuperseded; that the SlIP for Characterizing Wind Patterns Relative to PopulationCenters, dated March 1992, be reviewed for currency and updated as required toreflect current QARD and technical requirements prior to implementation; thatmeteorological personnel responsible for revising the SIIP and implementing the SIIPbe trained to SP 2.2 (latest revision) prior to implementing the SUIP CharacterizingWind Patterns Relative to Population Centers (currently none of these person havecurrent training in this procedure); and, that the T&MSS Matrix-for QARDSupplement IlI required by QARD Section 2.2.2C fully addresses where theserequirements are actually implemented for the Meteorological Monitoring Program(e.g., WI-MET-001, WI-MET-002, WI-MET 003, WI-MET-006, and WI-MET-009).These recommendations are summarized as Items 7, 8, and 9 in Section 6.0 of thisreport. No deficiencies were identified during the evaluation of this element.

Based on the above, QARD Supplement HI was determined to be satisfactory.

TECHNICAL EVALUATION:

The evaluation of the Meteorological Monitoring Program was based on interviewswith SAIC/T&MSS personnel and examination of objective evidence. The areasevaluated to verify the technical adequacy of implementation are listed below:

* The background and experience level of the people in the MeteorologicalMonitoring Program.

* - Type of technical training provided.

* Interface the Meteorological Monitoring Program with other agencies in themeteorological community.

* Criteria used in site selection.

* Need for additional sites for data collection to be added.


* Monitoring and determining that the data collected complies with regulatoryguidance.

* Technicians visit each monitoring site at least once every nine days.

* The site visits document physical conditions of the site area and sensors, propertime keeping and correct on-site data recording and reasonableness of thecurrent results.

* Invalid data is removed from the database.

* Storage of data.

* The data collected is sent to the National Climatic Data Center (NCDC) forarchival.

* The operating characteristics of the instrumentation in use comply with theregulatory specification criteria.

* The calibration of sensors and on-site monitoring equipment are traceable to arecognized standard.

* The type of routine preventative maintenance performed.

* Establishment of response times for repair/replacement of sensors.

* Selection of the dispersion model.

* Site selection for the tethered balloon and sodar.

* Short-term measurements made in the Crater Flat area.

* Conducting of tractor investigations.

* Corrective actions, if any, are taken when it is determined that a sensor hasgone bad or is out-of-tolerance.

* Definition of the standard's calibration period.

* Shaft rotation check performance.

* Completion of isothermal comparison checks.

* The wind direction sensor is placed according to procedures and that differencein sensor and thethersonde is within acceptable tolerances: +/- 15 degree.


The frequency used, trainer alignment, and antenna angle used, are beingdocumented on the TMSS/168 form.

Results:

The education level of the people were very impressive. Several held advance degreesand appeared to be highly qualified to perform the work required. Technical trainingis being completed and documented, meeting the needs of the project; however, thereappears to be a need to specify the training in more detail and to formalize thedocumentation of the training provided.

The selection of the dispersion model has not been made. The failure to select amodel has impact on the type of data and the frequencies of data collection. Selectionof a model should be made as soon as possible. The site selection for specializedinstrumentation has just started; however, the procedures used appear adequate. It wassuggested that the modeling personnel with the Air Resources Laboratory may be ableto help with site selection using existing Atmospheric Models. This recommendationis referenced as Item 10 in Section 6.0 of this report.

The implementation of the Meteorological Monitoring Program with respect to theobjectives listed in the Study Plan 8.3.1.12.2.1, Meteorological Data Collection at theYucca Mountain Site, and the Scientific Investigation Package for Characterizing WindPatterns Relative to Population Centers were evaluated. The overall conclusion ofthis evaluation finds that the Meteorological Monitoring Program as presentlyimplemented, is technically satisfactory. Also, the work that has been started onCharacterizing Wind Flow Patterns in Relation to Population Centers appears to bemoving in the right direction, but is still in the early stages. The MeteorologicalMonitoring Program is collecting quality-affecting data for input into a yet to bedetermined transport and diffusion model.


ATFACHMENT 3

list of Obiective Evidence Reviewed Duine the Audit

OA PROGRAM ELEMENT 1.0

Procedures:

OCRWM QARD, DOE/RW-0333P, Revision 0, Section 1.0,"Organization"

SP 1.2, Revision 7, "Preparation, Review, and Approval of T&MSS QA Policy,Organizational Description and QA Program Overview"

Objective Evidence:

TMSS/93-003, Organizational Description, Revision 0TMSS/93-001, QA Policy, Revision 0TMSS/098 forms for the review of documents TMSS/93-001 and TMSS 93-003TMSS/O95 forms for the review and comments performed on TMSS/93-003


Procedures:

OCRWM QARD, DOEIRW-0333P, Revision 0, Section 2.0, "QA Program"SP 1.31, Revision 8, "Initial Evaluation, Qualification, Indoctrination, and Training of

T&MSS Personnel"SP 1.32, Revision 2, "Management Assessment"SP 1.60, Revision 2, "Readiness Review"SP 1.71, Revision 1, "Graded Application of QA Controls"WI-QA-002, Revision 1, "Quality Assurance Surveillance"WI-QA-005, Revision 2, "Qualification of Audit Personnel"WI-QA-008, Revision 1, "Certification of Inspection Personnel"WI-HR-001, Revision 1, "Verification of Education and Experience"

Objective Evidence:

SP 1.31:

Training Files for: George Donaldson Pete RoesnerGary Jones Robert CampbellJoe Conway Tim MoranPaul Fransioli Larry Croft


Training Files for: (Continued):

Grover Prowell Teri Lyn PaneGary Mansur Eileen MelanderDoug Chandler Mike HarrisTerry Tait Mike MaloneJim Harper Kent Johnson

Instructor Qualification Records for: J. Karasik, N. Temple and D. Hanson

Training Plans for Lesson Plan No. IND 94003C, Revision 5 and Lesson Plan No. AP 94001,Revision 0.

SP 1.32:

1993 Management Assessment Report, dated March 2, 1994

SP 1.71:

QA Grading Reports: QA-001, QA-002, QA-003, QA-004, QA-005, QA-006, and UDSD-94-001.

WI-QA-002:

Surveillance Reports: SR-93-004, SR-93-005, and SR-93-006

WI-QA-008:

Form TMSS/144, Certification Record for R R. Rinderman, dated 10/26/93

WI-HR-001:

TMSS/176 and TMSS/177 forms (verification of education and previous employment) for thefollowing as applicable:

V. BestD. HansonM. StoverS. GomezE. JorgensenM. Shaw


OA PROGRAM ELEMENT 4.0 AND OA PROGRAM ELEMENT 7.0

Procedures:

Compliance with the following procedures was reviewed:

QARD DOEIRW-0333P, Revision 0, Section 4.0, "Procurement DocumentControl" and Section 7.0, "Contiol of Purchased Items and Services"

SP 1.25, Revision 8, ICN 1, "Acceptance of Hardware and Services"SP 1.28, Revision 8, ICN 1, "Procurement of Quality Affecting Hardware and

Services"SP 1.69, Revision 2, "Obtaining Services Through the Technical Field Work Request

Process"SP 1.72, Revision 0, "Upgrade of Items Procured as Non-quality Affecting"WI-QA-003, Revision 1, "Supplier Evaluation"WI-QA-007, Revision 1, "Maintenance of Qualified Supplier List"

Objective Evidence:

Procurement Document Control:

PRs (SAIC form 1-932-023), POs (SAIC form 9-932-018), Standard QA Clauses (formTMSS/293), Checklists for Preparation and Review of Quality Affecting ProcurementDocuments (form TMSS/008), and Acceptance Reports (form TMSS/038) were reviewed forthe following procurements:

Westinghouse Engineering Services:

PR6110561, dated 05/02/94PR6110534, dated 03/29/94PR6110510, dated 03/07/94PR6077412, dated 12/20/93PR6077471, dated 02/14/94P039-950178-01-40, dated 05/1194

P039-950236-40, dated 05/11/94P039-950208-40, dated 04/25/94P039-950138-40, dated 03/18/94P039-940950-40, dated 01/18/94P039-950178-40, dated 04/01/94

Teledyne Geotech:

PR5892459, dated 03/03/93

Climatronics Corporation:

P039-930307-94, dated 03/05/93

PR6077404, dated 01/18/94PR5836818, dated 08/07/92PR5892481, dated 09/10/92

P039-930307-94, dated 03/05/94P039-930152-94, dated 08/07/92P039-930152-01-94, dated 09/10/92


PR5892409, dated 09/17/92 P039-930152-02-94, dated 09/17/92PR5892443, dated 10/01/92 P039-930152-03-94, dated 10/01/92P039-930152-04-94, dated 10/05/92

Ringard Metrology:

PR5947507, dated 01/13/93 P039-930328-94, dated 01/13/93PR5948169, dated 02/25/93 P039-940007-94, dated 02/25/93

Waters Manufacturing:

PR6077330, dated 11/16/93 P039-940856-40, dated 11/16/93

Met One Instruments:

PR6035157, dated 07/30/93 P039-940098-94, dated 07/30/93

Technical Field Work Request Review Checklists (form TMSS/286) and Acceptance Reports(form TMSS/038) were reviewed for the following procurements:

Work Order WO-007/Work Request WR No. 93372, dated 10/28/93Work Order WO-008/Work Request WR No. 94062, dated 03/07/94Work Order WO-009/Work Request WR No. 94061, dated 03/07/94

Supplier Evaluation:

Supplier Evaluation Reports (form TMSS/016), Supplier Evaluation Checklist Cover Sheet(Form TMSS/017), Supplier Evaluation Checklist (form TMSS/018), Supplier EvaluationChecklist - Calibration Services (form TMSS/019), and QA Audit Report Checklists (formTMSS/145) were reviewed for the following supplier qualifications, as applicable:

Met One Northwest (Audit A94-03S, dated 05/10/94)Westinghouse Engineering Services (Audit A94-01S, dated 12/10/93)SAIC Environmental Services Division (Audit A94-02S, dated 04/05/94)Climatronics Corporation (Audit A92-06S, dated 12/29/92)

The following revisions to the T&MSS QSL were reviewed:

94-02, RO94-01, RO93-04, RD93-02, RO93-01, RO93-01, RI

- -



Procedures:


QARD DOE/RW-0333P, Revision 0, Section 5.0, "Implementing Documents"SP-1.1, Revisions 9 and 10, "Preparation, Review, and Approval

of T&MSS Procedures"SP 2.3, Revision 9, "Preparation Review and Approval of T&MSS Procedures"

Objective Evidence:

QARD DOEIRW-0333P, Revision 0, Section 5.0, "Implementing Documents"SP 1.1, Revision 10, "Preparation, Review, and Approval of T&MSS Procedures"SP 1.32, Revision 2, "Management Assessments"SP 2.3, Revision 9, "Preparation Review and Approval of T&MSS Procedures"Study Plan 8.3.1.12.2.1, Meteorological Data Collection Study PlanSIP 1.2.3.6.1.2, Characterizing Wind PatternsLetter From R. Spence approving TMSS/93-O01, 002, and 003

The documents selected for the audit sample were:

SP 1.32, Revision 2 - The document review was conducted June 17 through 25, 1993,in accordance with SP 1.1, Revision 9, which required only the new or revisedprocedure and form TMSS/098 be retained as records; therefore, an indepth audit ofthe review process was not possible.

SIlP 1.2.3.6.2 for Characteristic Wind Patterns was reviewed in accordance with SP2.3 which has now been superseded by SP 1.1.

Study Plan 8.3.1.12.2.1 which was reviewed in accordance with YMSCO AP-1.1OQ,Revision 4, and T&MSS SP 2.3. The audit did not include a review of the AP-1.1OQprocess and was confined to the SP 2.3 process.


Procedures:


QARD DOE/RW-0333P, Revision 0, Section 6.0, "Document Control"SP-1.34, Revision 9, "T&MSS Document Control"WI-QA-014, Revision 2, "T&MSS Quality Assurance Reviews of Quality Documents"

-


Objective Evidence:

SP-1.34:

Document Distribution Packages for the following procedures:

WI-QA-006, Revision 6, "Work Instruction for Trending"WI-QA-006, Revision 1, "Work Inistruction for Trending"SP 1.34, Revision 9, "T&MSS Document Control"SP 1.1, Revision 10, ICN 2, "Preparation, Review, and Approval of T&MSS

Procedures"SP 1.32, Revision 2, "Record Review Package'8.3.1.12.2.1, Revision 1, Met. Data Collection Study Plan1.2.3.6.1.2, Revision 1,-SIIP for Characterizing Wind PatternsLetter From R. Spence approving TMSS/93-001, 002, and 003WI-QA-014, Revision 2, "T&MSS Quality Assurance Review of Quality Documents"


Procedures:


QARD DOE/RW-0333P, Revision 0, Section 10.0, "Inspection"SP 1.25, Revision 8, ICN 1, "Acceptance of Hardware and Services"

Objective Evidence:

SP 1.25:

Acceptance Reports, TMSS/038, associated with the following POs:

39-950208-40 39-940007-94 39-940098-94 39-930328-9439-940356-40 39-950045-40 39-950178-40


Procedures:

Compliance with the following requirements documents and procedures was reviewed:

QARD RWIDOE 0333P, Revision 0, Section 12.0, "Control of Measuring and TestEquipment"

WI-RED-006, Revision 2, "Control of Measuring and Test Equipment"WI-MET-001, Revision 4, "Test, Checks, and Audits of Meteorological Equipment"

Audit ReportYMP-94-0S of 61

Objective Evidence:

WI-RED-006:

M&TE/OE-History Files containing: Certificates of Calibration, calibration data sheets,required accuracy, calibration intervals, storage durations, and NCRs for the following devices(which were also physically examined in the field):

ID 05107 Barometric Pressure SensorID 05118 RH SensorID 05117 Temperature SensorID 16423 Wind Direction SensorID 2718 Perineometer

ID 05032 Precipitation Gage-ID 05025 Temperature SensorID 05047 Wind Speed SensorID 05083 Wind Speed Sensor

Note: The above equipment was located at 40 Mile Wash during the audit.

ID 05156ID 15648ID 00985ID 03408ID 05038~ ^nele

Barometric Pressure SensorWind Speed SensorWind Direction SensorTemperature SensorPyranometer

lam:_ e___ -- ____

IDIDIDIDID

03176 Wind Speed Sensor03174 Wind Speed Sensor03409 Temperature Sensor03405 Temperature Sensor00744 Wind Direction Sensor

ID V015Z wind Speed sensor

Note: The above equipment was located at Site No. 1-60 Meter Tower during audit.

ID 04022ID 20584TVPhno

Graduated CylinderMultimeter

ID 21115 Torque WatchID 03374 Barometer

wl u Z.Z ±.lgluai incrmometer Lf

VNote: The above M&TE Standards was located at Bldg 4226 Met Lab during audit.

M&TE and Standards Usage Log was reviewed for the following:

ID 20584ID 20643ID 22023

MultimeterMass WeightsDigital Thermometer

ID 04022 Graduated CylinderID 21115 Torque Watch

M&TE Standards Examined for Calibration Labels and Unique Identifiers as follows:

ID 04022ID 20584ID 22023ID 20643

Graduated CylinderMultimeterDigital ThermometerMass Weights

ID 21115 Torque WatchID 03374 BarometerID 20218 Barometer


Recall Notices (30/60 days) on memo from George Donaldson to:

Larry Croft, dated 4/25/94, for the following equipment:

ID 20646 BP Sensor-Cal, due 6/30/94ID 20647 BP Sensor-Cal, due 6/30/94ID 03233 Precip Gage-Cal, due 4/30/94ID 23704 RH Sensor-Cal, due 6/30/94ID 23705 RH Sensor-Cal, due 6/30/94

Reviewed memo, dated 4/1/94, to Distribution (including managers) from GeorgeDonaldson identifying that the calibration of ID 16358 Barometer and ID 03374Barometer expired on 3/31/94 and to be "tagged out of service."

Reviewed justification letters for M&TE/OE storage durations as follows:

Letter GJ: sg: M94-010 approved by D. Sorensen 5/2/94-reference DigitalVolt-ohm meters, Oscilloscopes, and Analog V/O Meters

Memo from Sorensen to Donaldson, dated 11/22/93, on "S-weights"Memo, dated 10/12/92, from P. Fransioli to Sorensen reference storage of Wind

Sensors, approved by Sorensen on 10/14/92.

"Long Term Storage Cabinet" and controlled access logs in Building 4226

NCR(s) examined during review of calibrated equipment:

T&MSS-94-0001T&MSS-94-0015T&MSS 94-0016 (generated during course of audit)

Quality M&TE/OE List, dated 4/25/94

Reynolds Electrical and Engineering Company, Inc. (REECo) WO-006 forREECo calibration of S Weights ID 20643

Technical Justification Letter for Using Standards of Equal Accuracy (Memo byGeorge Donaldson, dated 7/18/92)

WI-ME7T-OO1:

Following Standards Used by EFPD for Performing Calibrations and AssociatedCertificates of Calibration were reviewed:

ID 04022 Graduated Cylinder ID 21115 Torque WatchID 20584 Multimeter ID 03374 Barometer


ID 22023 Digital ThermometerID 20643 Mass Weights

ID 20218 Barometer

Following Field equipment Calibrated by EFPD and associated data sheets werereviewed:

ID 03409 Temperature Sensor calibrated by Paul Fransioli on 4/30/94ID 03405 Temperature Sensor calibrated by Paul Fransioli on 4/30/94ID 05107 Barometric Pressure Sensor calibrated by M. Jenkins on 8/19/93

Reviewed the following NCRs relative to out-of-tolerance calibration:

T&MSS-94-0001T&MSS-94-001-5T&MSS-94-0016

Reviewed PO 39-930307-94 relative to calibrations by MET ONE

Reviewed WO-006 for REECo calibration of S Weights ID 20643


Procedures:


YAP-15.1Q, Revision 0, ICN 1, "Control of Nonconformances"WI-QA-006, Revision 1, "Trend Analysis"SP 1.23, Revision 6, "Nonconformance Reporting"

Objective Evidence:

YAP-15.1Q and SP 1.23:

NCR Nos: T&MSS-94-000NT&MSS-94-0004T&MSS-94-0007T&MSS-94-0010T&MSS-94-0013

T&MSS-94-0002T&MSS-94-0005T&MSS-94-0008T&MSS-94-0011T&MSS-94-0014

T&MSS-94-0003T&MSS-94-0006, RIT&MSS-94-0009T&MSS-94-0012T&MSS-94-0015

WI-QA-006:

Quality Trend Analysis Reports for first, second, third and fourth quarters of 1993



Procedures:


QARD DOEIRW-0333P, Revision 0, Section 16.0, "Corrective Actidn"SP 1.37, Revision 6, "Deficiency Reporting System"WI-QA-006, Revision 0, ICN 1, "Trend Analysis"

Objective Evidence:

QFR 94-001-012, TMSS QFRsQFR Log, dated 5/16/94QFRs 93-010 through 93-017. Copies of records from LRC for 1993QFRs 93-012 and 93-016. Written extension request, 1993QFR 92-016. Evidence of QFR cancelled during 1993TAR - Trend Analysis Reports for fourth quarterly report from 1993TAR - Trend Analysis Report for third quarterly report from 1992


Procedures:

Compliance with the following procedure was reviewed:

QARD DOE/RW-0333P, Revision 0, Section 17.0, "QA Records"SP 1.36, Revision 10, ICN 1, "T&MSS Records Management"

Objective Evidence:

Records Packages:

NNA.940502.0091 NNA.940502.0080 NNA.940502.0086NNA.940502.0136 NNA.940502.0092 NNA.931115.0055NNA.940323.0308 NNA.940325.0036 NNA.940210.0030NNA.931102.0070 NNA.930406.0016 NNA.930219.0049NNA.930203.0103 NNA.930121.0005 NNA.940104.0044NNA.940428.0061 NNA.930119.0064

Audit A94-03S, Met One NorthwestRingard Metrology, 39-940007-94Westinghouse, 39-950178-40Quality Supplier List, 94-02, Revision 0



Procedures:

Compliance with the following procedure was reviewed:

QARD DOEIRW-0333P, Revision 0, Section 18.0, "Audits"WI-QA-001, Revision 2, ICN 1, "QA Audits"WI-QA-005, Revision 2, "Qualification of Audit Personnel"

Objective Evidence:

WI-QA-001:Audit Reports A93-06,-A93-06, and A94-01

WI-QA-005:Record of Initial Auditor Qualification, TMSS/149 and Annual Evaluation of LeadAuditor Qualification, TMSS/299

The above records were reviewed for R. R. Rinderman, K. B. Johnson, M. Malone,and J. W. Estella from the last audit performed.

SUPPLEMENT m

Procedures:

Compliance with the following requirements documents and procedures was reviewed:

QARD RW/DOE-0333P, Revision 0, Supplement III Scientific InvestigationSP 2.2, Revision 4, "Scientific Investigation Control"WI-MET-002, Revision 4, "Routine Operations and Maintenance of Meteorological

Equipment"WI-MET-003, Revision 5, "Meteorological Monitoring: Data Processing Instructions"WI-MET-010, Revision 0, "Special Meteorological Field Measurements"

Objective Evidence:

SP 2.2:

Scientific Investigation Planning Package, Characterizing Wind Patterns Relative toPopulation Centers, dated March 1992


WI-4ET-002WI-MET-003:

Study Plan 8.3.1.12.2.1, Revision 1, "Meteorological Data Collection at the YuccaMountain Site"

Meteorological Data Report for DOE (covering time period December 1992-January1993)

Report on Performance Audit of April 1994 (re: memo cover page LDC:PF:M94-01 1)

Technical Assessment for Qualification of Existing Environmental Monitoring Data(SCP:8.3.1.12) identified as attachment to memo LDC:GHP:M93-014

TECHNICAL

Procedures:

Study Plan 8.3.1.12.2.1, Revision 1, 6/24/93, Scientific Investigation ImplementationPackage for Characterizing Wind Patterns Relative to Population Centers, WBS1.2.3.6.1.2

Objective Evidence:

Personnel Resumes for Paul Fransioli, Grover Prowell, and Gary JonesPerformance checks (TMSS/285) for sensors on towers Nevada Test Site (NTS) 60 and

40 Mile Wash for January through February 1994DOE Data Report, January through December 1992NCRs, December 1993 through May 1994Calibration and Sensor History Records for all sensors on towers NTS 60 and 40 Mile

WashIndependent audit reports conducted by SAIC, San Diego, February and April of 1994Form TMSS/168, Sodar Checks and OperationsSite Checklists (TMSS/1 10) for towers NTS 60 and 40 Mile Wash, March Through

April 1994Meteorological Equipment Calibration (IMSS/259), November 1993 to January 1994


ATTACHMENT 4

Information Copy

of

Comnctive Action Request


* _ am I a

awlA# a W..%

OFFICE OF CIYIUAN fI. A RC TAW 6. Draft y-94-dORADIOACTIVE WASTE MANAGEMENT PAGE . OF -A

U.S. DEPARTMENT OF ENERGY CA

WASHINGTON, D.C.

-CORRECTIVE ACTION REOUEST1 Controlling Document initial avaivatiazi, vualiticatiof, 2 Related Report No.

S? 1.32, Pevision I I -- a t e Audit Report T)-54-OS3 Responsible Organization 4 Discussed With -

SAICITMS I ; Chadwick/K. Johnson/J. Larper6 Requirement:

5? 1.314 Section 3.0 states in part: lzmployees may perform quality-affecting activities prior tocompleting full position qualiflcation;... Eovever, the manager assigning such activities Mustinsure that the employee has been trained to the document sI foverning those activities., have beendocumented before quality-affecting activities are performed.

6 Adverse Condition:Contrary to the above, there is no objective evidence to reflect that five individuals, whoparticipated in the review and corment of TYtS/93-003 (Organizational Description), had completedtrairing to SP 1.2, Revision 6, i to performing the activity.

Does a Significant Condition I IDoes a stop work condition exist? 3 Response Due Date:Adverse to Ouality exist? Yes_ NcL Yes, _ No_._; IK Yes. Attach copy of SWO 20 Working daysIf Yes, Circle One: A B C D E I KtYes. Circle One: A 8 C from Issuance.

" Required Actions: E Remedial Ej Extent of Deficiency 0 Preclude Recurrence [0] Root Cause Determination

IZ Fiecommended Actions:Revise the 'Real Time Training requirement to reflect that the specific procedure to perform the taskwas read 12 to performing the activity.

7 Initiator t4 1wUbPicbard L. Maudlin 5/13/34 QADD ) V'Date5 2 $4

15 Response Accepted 16 Response Accepted

CAR Date OADD Date17 Amended Response Accepted 1B Amended Response Accepted

OAR Date OADD Date19 Corrective Actions Verified 20 Cosure Approved by:

OAR Date OADD Date

AEV. VZ4