company confidential registration management committee (rmc) 1 aqms accreditation programs anab...
TRANSCRIPT
Company Confidential
Registration Management Committee (RMC)
1
AQMS Accreditation Programs ANAB Findings
Atlanta, GAJuly 22, 2010
Steve HolladayANAB, Accreditation Assessor
Auditor WorkshopAtlanta, GA
July 22-23, 2010
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010
Objectives
• Provide an overview of the NCR’s identified from the witness audits.
• Present in process approach.
• Discuss steps that could have been taken to prevent the NCR’s.
• Rules of engagement.
2
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010
Goal
• Provide knowledge/information to the AS auditor pool with the goal to improve auditor/CB performance that will reduce repeat NCR’s which will, in turn, add value to the assessment process and the industry as a whole.
3
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 4
• 120 NCR’s in 54 Witnessed Audits (WA)– 2.22 Average per WA
• 50 NCR’s in approx 40 Office Assessments (OA)– 1.25 Average per OA
Overall
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 5
• 3 NCR
• DETAIL:– Assigning Auditors who either did not have the
proper NACE code or AS91XX qualifications.
– Client not made aware of the OP assessor.
• LEARNINGS: – Verify auditors are qualified for the full
dynamics of the audit.
– Communicate clearly with the client.
Pre-Audit Assignment
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 6
• 11 NCR’s
• DETAIL: 19011, 6.4.1.c– (4) The audit plan did not ID the organizations
functional units and processes to be audited.
• LEARNINGS:– The audit plan ensure the client will have the
proper resources available to the audit.
– Clearly demonstrates what processes are intended to be audited.
Pre-Audit Planning
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 7
• 11 NCR’s
• DETAIL: 17021. 9.3.2.1– (2) Audit Planning not effective in assuring the
AQMS is assessed to the min requirements for surveillance audits.
• LEARNINGS:– Ensure the auditors understand where the full
auditing requirements are for surv audits if not clearly ID in the audit plan.
Pre-Audit Planning
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 8
• 11 NCR’s
• DETAIL: MD 5 and AS9104– (5) Insufficient Auditor Days in the planning
without clear justification.
• LEARNINGS:– If deviation…..JUSTIFY.
– If the audit is being witnessed, provide to ANAB in the pre-audit information.
Pre-Audit Planning
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 9
• 5 NCR’s, several OFI’s.
• DETAIL:– Information missing or not correct in EQM.
– Client Profile form not complete or accurate. (scope, head count, regulatory, ITAR)
• LEARNINGS:– Ensure the person entering information is
knowledgeable about the information.
– Verify with Client and lead auditor before submitting.
Pre-Audit Preparation
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 10
• Total of 24 NCR’s
• DETAIL:– Sampling part of an element and making a
conclusion on the whole requirement.
• LEARNINGS:– Audit plan and report accuracy is critical.
On-site Audit Depth of auditing
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 11
• DETAIL: – Auditing to the scope of certification is not
adequate.
• LEARNINGS:– Ensuring the scope of certification, scope of
audit and audit plan is consistent.
– Validate the exclusions. Is scope consistent with the QM.
– Ensure the address and scope on the Certificate is correct.
On-site Audit Depth of auditing
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 12
• DETAIL: – Not following audit trails when objective
evidence suggest otherwise.
• LEARNINGS:– A plan is a plan. Follow the trail to its natural
conclusion if potential findings are evident.
– Keep head on a swivel. Don’t ignore clear findings.
On-site Audit Depth of auditing
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 13
• DETAIL: – Not fully verifying the effectiveness of the
actions taken on nonconformities identified during the previous audit.
• LEARNINGS:– Ensure the evidence is more than a record
review.
On-site Audit Depth of auditing
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 14
• 4 NCR’s
• DETAIL:– Design Applicability
• LEARNINGS– More of an issue with AS9110.
On-site Audit Interpretation
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 15
• DETAIL:– Not fully auditing Outsourcing when
applicable.
• LEARNINGS– Standard clearly requires “control of such
outsourced processes shall be identified” and therefore subject to audit. Not just a document verification.
On-site Audit Interpretation
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 16
• DETAIL:– Changes to the capability listing of the client.
• LEARNINGS– Similar to the scoping discussion. Verify any
changes to the capability listing as this could affect the scope of certification or introduce new processes and technology.
On-site Audit Interpretation
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 17
• 11 NCR’s
• DETAIL: – Calling the Clients CA/PA system effective with
ineffective/inadequate correction, root cause or corrective action.
• LEARNINGS:– Deep dive the CA/PA system. Validate the
information. If it doesn’t make since….make the call.
On-site Audit Decisions
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 18
• DETAIL:– Soft Grading 8 NCR’s
– Clearly stating findings during the audit and not raising the NCR or improper categorization.
– Accepting informal correction during the audit.
• LEARNINGS– Learn the definitions and follow them. Keep track
of the “verbal” findings identified during the audit. Accepting correction to verbal findings without reporting is consulting.
– As an auditor it is not your job to justify an NCR down.
On-site Audit Decisions
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 19
• DETAIL:– Continuing the audit when the audit objectives
are clearly unattainable.
• LEARNINGS:– Only the client can make the decision to
continue an audit.
– There should be a clear conclusion that the objectives are unattainable and reported to the client with options.» At the time this is realized not at the end!!
On-site Audit Decisions
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 20
• DETAIL: – Closing meeting does not address all of the
requirements.
• LEARNINGS– Clearly address the CA follow up and impacts
on the existing cert.
– Ensure the CA are presented to the correct requirements.
On-Site Audit Closing
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 21
• 22 NCR’s
• DETAIL:– Accuracy of the Audit Report and Checklist to
the observed audit.
• LEARNINGS– Anecdotal evidence should be documented as
such. Same for deductive evidence.
On-site/Off-site Audit Reporting
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 22
• DETAIL: – Report not in conformance with
AS9014/AS91XX
• LEARNINGS– Justify/explain all NA.
– Clear evidence to support the conclusion.
– Clear evidence to support the SCOPE.
– Include detailed notes
On Site/Off-Site Audit Reporting
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 23
• 3 NCR’s
• DETAIL:– Report did not contain mandatory items.
• LEARNINGS– Ensure the Checklist are complete or
references the location of the information.
– Explain and differences between the information left on site to the published report.
Post Audit Reporting
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 24
• 8 NCR’s
• DETAIL: Improper closing of NCR’s and Improper Certification Decision.
• LEARNINGS:– Clear evidence of Correction, RC and CA.
– Ensure persons closing NCR are AEA.
– Ensure the NCR’s are closed with appropriate evidence PRIOR to the cert decision.
Post Audit NCR Closure
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010
Office Audit NCR’s
• OASIS data base admin/accuracy
• Verification of Client OASIS data admin
• Justification of auditor days.
• Improper qualification of AS cert decision maker.
• Improper decision made with certification information.
25
Registration Management Committee (RMC)
Atlanta, GAJuly 22-23, 2010 26
Questions