as/en/jisq 9100:2016 evaluation guidance material _9100_2016.pdf · © 2017 iaqg all rights...

62
© 2017 IAQG All rights reserved AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 1 Revision: 2.1 AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL Prepared by the IAQG 9101 Writing Team

Upload: ngonga

Post on 06-Mar-2018

293 views

Category:

Documents


10 download

TRANSCRIPT

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 1 Revision:2.1

AS/EN/JISQ9100:2016EVALUATIONGUIDANCEMATERIAL

PreparedbytheIAQG9101WritingTeam

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 2 Revision:2.1

ContentsPage# Section Title

3 1.0 Introduction4 2.0 Processauditingapproach4 2.1 Generalapproach5 2.2 Specificapproaches8

3.0

Auditconsiderations8 Clause4-ContextOfTheOrganization12 Clause5–Leadership15 Clause6–Planning18 Clause7–Support24 Clause8–Operation48 Clause9-PerformanceEvaluation54 Clause10-Improvement58 4.0 Changecontrol

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 3 Revision:2.1

1.0 IntroductionThisdocumentprovidesgeneralguidanceforusebyauditteamswhenexecutingtheauditprocessdescribedbythe9101:2016standard,whenconducting9100:2016audits.

Thisguidanceisnotintendedtoaddto,subtractfrom,orinanywaymodifythestatedrequirements,buttoprovideexamplesandthoughtstimulationi.e.“thingstoconsider”,whenaskingquestionsandidentifyingobjectiveevidence.

NOTES:• Theguidancecontainedwithinthisdocumentcanalsobeusefulwhenpreparingandplanninganauditduringthecertificationcycle.

• Acceptablemeansofcompliancearenotlimitedtothoseitemslistedinthisdocument.

• Anyissuesidentifiedduringauditsaretobedocumentedagainst9100:2016requirements.

• 9100:2016clause0.3describestheprocessapproachdeveloping,implementing,andimprovingtheeffectivenessofaQualityManagementSystem

• 9101:2016clause0.2describestheprocessapproachwhenevaluatinganorganizationsQMS

• FurtherISOguidanceisalsoavailableasfollows:o Processapproachformanagementsystems-ISO/TC176/SC2/N544R3o ApplicationofISO9001:2015managementsystem-ISOTechnicalSpecificationISO/DTS9002

• Thisdocumentdoesnotprovideguidancefortheadditionalrequirementsdefinedby9110and9120.

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 4 Revision:2.1

2.0 ProcessAuditingApproach2.1 GeneralApproachWhenevaluatinganorganization'sQMS,therearebasicquestionsthatshouldbeaskedofeveryprocess,forexample:• Istheprocessappropriatelydetermined?• Istheprocessidentifiedandappropriatelydefined,includingforexampleprocessinputs,outputs,resourcesandcontrols?• AreProcessOwnerandProcessUserresponsibilitiesassigned?• Istheprocessimplementedandmaintained?• Istheprocesseffectiveinachievingthedesiredresults?Otherquestionscouldinclude:• Whatistheprocess?• Whatisittryingtoachieve?• Whoisthecustomeroftheprocess?• Doestheprocessaddressapplicablecustomerspecificrequirements?• AreProcessUsercompetenciesidentified?• Istheprocessoperating,asdefined?• Whatisthedesiredlevelofperformance?• Haveprocessperformancetargetsbeenestablished,includingthosespecifiedbythecustomer?• Haveprocessmeasures(e.g.KPIs)beenestablishedandaremechanismsinplacetocollateandanalyzethedata?• Whatisthecurrentlevelofperformance?• Isprocessperformanceregularlyreviewedbytopmanagement?• Whereperformanceisnotbeingachieved,areimprovementplansinplace?

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 5 Revision:2.1

2.2 SpecificApproaches

Thereareanumberofspecificauditapproachesthatcanbebuilt intotheauditplantoprovidefocusonkeyelementsoftheQMS, includingforexample:

2.2.1 CustomerFocus

The audit team should determine that customer satisfaction is being evaluated and appropriate actions are takenby the organization basedonavailable performance information (e.g., nonconformity data, corrective action requests, results of satisfaction surveys, complaints regardingproduct quality, OTD, service provision, responsiveness to customer and internal requests) provided by the organization's customers (e.g.,scorecards,reportcards).

2.2.2 OrganizationalLeadership

TheInterviewwithtopmanagementshouldbeusedtoevaluate:• Establishmentandcontinuedrelevanceoftheorganization'squalitypolicyandobjectives;• Establishmentofperformancemeasuresalignedtoqualityobjectives;• QMSdevelopment,implementation,andcontinualimprovement;• Topmanagementcommitment;• QMSperformanceandeffectiveness;• Performancetocustomerexpectations(e.g.,supplierrating,scorecard,auditresults);and• Actionstakentoaddressissuesthatarenotmeetingcustomerperformanceexpectations.

2.2.3 QMSPerformanceandEffectiveness

TheauditofQMSperformanceandeffectivenessshouldincludeareviewofthefollowing:• Processingofcustomercomplaints,customerfeedbackdata(e.g.,periodicperformancereportsreceivedfromcustomers),andotherrelevant

customerdata(e.g.,resultsofcustomersurveys);

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 6 Revision:2.1

• ResultsandactionsfrominternalandexternalauditsoftheQMS,includingtheirassociatedrecords;• Stakeholderfeedback(e.g.,feedbackfromregulatoryauthoritiesorotherinterestedparties);• Processingofprocess/productnonconformities,includingreviewofassociatedcorrectiveactionsandevaluationontheeffectivenessofactions

taken;• Processingofpreventiveactions,includingevaluationontheeffectivenessofactionstaken;• Managementreviewconduct,includingassociatedrecords(e.g.,processinputs/outputs,actionstaken);• Internalperformancemonitoring,measurement,reporting,andreviewsagainststakeholderandinternalperformanceobjectivesandtargets,

includingcontinualimprovementactivitiesandassociatedrecords;• Organization’scurrentperformanceagainsttargets,includingcustomerspecifictargetsandassociatedrecordsofapplicableactionstakenwhere

targetsarenotbeingmet;and• Thestatusandeffectivenessoftheorganization'sprocessperformanceimprovementactivitiesandtheiroutcomesrelatedtoproductquality.

2.2.4 ProcessManagement

TheauditteamshouldconductQMSauditsusingamethodthatfocusesonprocessperformanceandeffectiveness;thisensuresthatpriorityisgiventothefollowing:• Reviewingtheorganization'sprocesses,theirsequenceandinteractions,theidentificationoffunctionsandassignmentofresponsibilities,and

performanceagainstrequirementsanddefinedmeasures,withfocusonprocessesthatdirectlyimpactthecustomer;• Reviewingtheprocessforvalidationandapprovalofprocessesandprocesschanges;• Reviewingtheavailabilityofresourcesandinformationrequiredtooperateandsupportassociatedactivities,includingappropriatetrainingand

competencyofpersonnel;• Reviewingtheprocess-basedmanagementtechniques,includingtheexaminationofprocessmeasures(e.g.,quality,tacttime,cycletime,output

effectiveness,controllimits,processcapabilitydetermination);• Reviewingplansinplacetoensureperformanceobjectives/targetsaremonitored,measured,andanalyzedinordertorealizetheplanned

activitiesandachievetheplannedresults(e.g.,verifyperformanceinformationavailability,percentageofnonconformingparts/products,percentageOTD);

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 7 Revision:2.1

• Reviewingapplicableactiontakenwhenobjectives/targetsarenotmettopromotecontinualimprovement;and• Pursuingaudittrailsaddressingcustomerconcernsorrequestsforcorrectiveactions,performanceagainstobjectives,andrelevantprocess

controls.NOTE:Theaudit team should audit processes to sufficientdepth anddetail to evaluate if theorganization's processes are capableofmeetingplannedresultsandperformancelevels,includingapplicablecustomerspecifictargets.

2.2.5 ContinualImprovement

Theauditteamshouldevaluatetheorganization'sinterrelatedprocessesandactivitiesforcontinualimprovementoftheQMS,itsprocesses,theirconformity,andeffectivenessinorderto:

• Ensurefocusonissuesthatareimportanttotheorganization,theircustomers,andregulatoryauthorities;and

• Determinetheeffectivenessofanorganization'sapproachtocontinuallyimprovingprocessperformance.

NOTE:The organization should be able to demonstrate that they have a structured approach to achieve continual improvement of the QMS and itsprocesses.

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 8 Revision:2.1

3.AuditConsiderationsThissectionlooksattherequirementsof9100:2016byclausenumberandprovidesexamplesoftypicalevidencethatcanbesoughtandconsideredbytheauditteamduringexecutionoftheauditplan.

Clause# Considerations4. CONTEXTOFTHEORGANIZATION

4.1

UnderstandingtheOrganizationanditsContextThingstoconsider:• Policystatement(s)regardingtheorganizationspurposeandstrategicdirection• Individualstrategyortacticalplandocumentswrittentounderpintheorganizationspoliciesandprovidearoadmaptoachieveits

futuregoals• Recordsofmeetingswherecontextisroutinelydiscussedandmonitored,e.g.aspartofthestructuredmanagementreviewprocess

orwithineachoftherespectivefunctionoftheorganization(Purchase,HR,Engineering,Sales,Financeetc.)• StructuredriskassessmentofExternalandInternalissues.• UseofPEST(Political,Economic,Social,Technological),PESTLE(Political,Economic,Social,Technological,Legal,Environmental)and

SWOT(Strengths,Weaknesses,Opportunities,Threats)analysistools• Documentedinformationthatdescribesthecontextoftheorganizationthatcouldbeincludedaspartofasinglesourceof

documentedinformation(sometimesreferredtoasaQualityManual)SourcesofinformationrelatingtoExternalandInternalissuescouldinclude:External:• Reportsrelatingtocompetitiveenvironment,newtechnologies,newmarkets,customerexpectations,supplierintelligence,

economicconditions,politicalconsiderations,investmentopportunities,socialfactorsetc.• Identificationoffactorsrelatingtochanginglegislationandregulations,includingenvironmentalimpact• Feedbackrelatingtoproduct/serviceperformanceandlessonslearned• RegisterofidentifiedexternalrisksandtheirtreatmentInternal:• Structureoftheorganization,identificationofrolesandresponsibilitiesandarrangementsforgovernance

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 9 Revision:2.1

• Reports’showinghowwelltheorganizationisperforming• Statementsrelatingtotheorganizationsmission,visionandcorevalues• Emphasisplaceduponbusinessethicsandorganizationalcodesofconduct• Feedbackobtainedfromemployees,e.g.employeesurveyresults• Informationsystemsandthemechanismforcapturingandsharingknowledgeandlessonslearned• Organizationalcapabilitystudies,identificationofload/capacityandresourcerequirementstoachievedemand• Registerofidentifiedinternalrisksandtheirtreatment

4.2

UnderstandingtheNeedsandExpectationsofInterestedPartiesThingstoconsider:• Useofbrainstormingtechniquestoidentifyrelevantexternalandinternalinterestedparties,e.g.customers,partners,endusers,externalproviders,owners,shareholders,employees,tradeunions,governmentagencies,regulatoryauthorities,localcommunity

• Useofsurveys,networking,face-to-facemeetings,associationmembership,attendingconferences,lobbying,participationinbenchmarking,etc.,inordertogainstakeholderinformationandtheirrequirements

• Stakeholdermappingtodetermineimportanceusingthefour-quadrantmethod(power/influenceversusinterest/importancegrid).• UseofRACI(Responsible,Accountable,Consulted,Informed)methodtodeterminelevelofstakeholderengagement• FlowdownandcaptureofrequirementsrelevanttotheQMSdefinedincontracts,orders,statementsofwork,termsofbusinessetc.• Recordsofmeetingswhereinterestedpartiesandtheirrequirementsareroutinelydiscussedandmonitored,e.g.aspartofthestructuredmanagementreviewprocess,orwithineachoftherespectivefunctionoftheorganization(Purchase,HR,Engineering,Sales,Financeetc.)

NOTE:TheorganizationdeterminestherelevantinterestedpartiesandtheirrequirementsrelatingtotheactivitiesoftheorganizationandtheimpactontheirQMS

4.3

DeterminingtheScopeoftheQualityManagementSystemThingstoconsider:• Organizationevaluationof:

o Externalandinternalissues(see4.1)o Requirementsofrelevantinterestedparties(see4.2)o Itsproductsandservices

• ConsiderationofboundariesandapplicabilityoftheQMScaninclude:

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 10 Revision:2.1

o Rangeofproductsandserviceso Differentsitesandactivitieso Externalprovisionofprocesses,productsandserviceso Commonsupportprovidedbycentralizedfunctionso InfrastructureoftheQMSsuchasdistribution,accessibility,differentlayers(process,procedure,instructions,sitespecific

requirements)• DocumentwrittenandmaintainedbytheorganizationthatdescribesthescopeoftheQMS,including:

o Astatementofthetypesofproductsandservicescoveredo Assessmentoftheapplicabilityofthe9100requirementso Justificationforany9100requirement(s)thatis/aredeterminedas‘notapplicable’

• Alignmentbetweenthedocumentedscopeoftheorganization’sQMSandtheiragreedscopeofcertification4.4 QualityManagementSystemanditsProcesses

4.4.1

Thingstoconsider:• IdentificationoftheprocessesneededfortheQMS,includingtheirsequenceandinteraction,e.g.processframework,process

model,processgroupings,processflowdiagram,processmapping,valuestreammapping,Turtlediagrams,SIPOC(Supplier,Input,Process,Output,Customer)charts,processcards

• Informationneededtoensureeffectiveoperationandcontroloftheprocesses,e.g.definedprocessrequirements(shall),goodpractice(should),definedroles,requiredcompetencies,associatedtraining,guidancematerial

• Expectedinputsandoutputsfromeachoftheidentifiedprocesses,togetherwithassignmentofresponsibilitiesandauthoritiese.g.ProcessOwner,ProcessChampion,LeadProcessUser,ProcessUser.

• Criteriaandmethodtoensureeffectiveoperationandcontroloftheidentifiedprocesses,e.g.processmonitoringindicators,processperformanceindicators,targetsetting,datacollection,performancetrends,internal/externalauditresults

• Arrangementsforgoverningtheprocessese.g.processownerreviews,processcouncils,flightdeck/dashboards,risksandopportunitiesrelatingtotheprocess,resourceneeds,usertraining/competency,continualimprovementinitiatives,frequencyofreviews,agenda,minutes,actions

• Organizationapproachtowardscontinualimprovementandactiontakenwhenprocessperformanceisnotmeetingintendedresults• Captureofcustomer,statutoryandregulatoryrequirementsandthemethodusedtobuildtheseintotheorganizationsQMS,e.g.

requirementscapture,gapanalysis,requirementsembeddedintotheprocessdefinition,assignedcontractassuranceinstructions,formallinkstoinformation,useofspecifieddocumentation

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 11 Revision:2.1

4.4.2

Thingstoconsider:• Documentationcreatedandmaintainedbytheorganizationtosupporttheoperationoftheprocesses,e.g.manuals,expositions,

handbooks,documentedprocedures,workinstructions,guidancematerial,datacards,physicalsamples,ITsystems(includingintranetandinternet),universal/bespokesoftware,templates,forms

• Documentationidentifiedandretainedbytheorganizationtoshowthatprocessesarecarrieditasplanned,e.g.physicalhardcopyrecords,electronicmedia(dataservers,harddrives,compactdiscs,flashdrivesetc.)

• Specificdocumentationcreatedandmaintainedbytheorganizationthatincludesadescriptionofrelevantinterestedparties(see4.2),scopeoftheQMSincludingboundariesandapplicability(see4.3),descriptionoftheprocessesneededfortheQMStogetherwiththeirsequence,interactionandapplicationandassignmentofresponsibilitiesfortheprocesses.

NOTE:TheorganizationcancompilethisspecifiedinformationtogetherinasinglerepositoryandmayrefertoitasaQualityManual.

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 12 Revision:2.1

Clause# Considerations5. LEADERSHIP5.1 LeadershipandCommitment

5.1.1

GeneralThingstoconsider:• TopmanagementcommitmenttowardstheQMStodemonstratethattheyhaveapresenceintheorganization,providedirection,

leadbyexample,makedecisionsandhave:o TakenaccountabilityfortheeffectivenessoftheQMSe.g.establishedmeasures,system/processperformancemonitoring,

managementreview,realizationofplannedactivities,achievementofplannedresultsandtakingactionwhenprocessperformanceisnotmeetingintendedresults

o Establishedandmaintainthequalitypolicyandobjectivesalignedtothestrategicdirectione.g.contextoftheorganization,external/issues(see4.1)

o Integratedqualityrequirementsintotheorganizationsbusinessprocessese.g.systemarchitecture,businessmodel,processmodel,organizationfootprint,functionalalignment(Engineering,Purchasing,IT,Finance,HRetc.)

o Promotedtheprocessapproachandriskbasedthinkinge.g.processmodeling,processmapping,inputs,outputs,activities,interactions,interfaces,resources,controls,riskmanagement(identification,severity,ownership,treatmentetc.)

o Supportedprocessownersintheirprocessmanagementactivitiese.g.deployment,governance,processevaluation,processimprovement

o Enabledtheresources(includingpeople)requiredforaneffectiveQMSe.g.resourceplanning,workload,priorities,constraints,balance,organizationflexibility,businessbenefits,organizationgrowth

o CommunicatedtheimportanceofconformitytotheQMSandeffectivequalitymanagemente.g.meetings,briefs,e-mail,intranet,campaigns,roadshows,focusedtraining,voiceoftheregulator/customer,consequenceofnonconformity

o Createdanenvironmentforcontinualimprovemente.g.§ Proactive-product/service/processimprovementinitiatives,improvementprojects,wastereduction(lean),process

re-engineering,costreductionetc.§ Reactive-actingonprocessperformanceresults,auditfindings,complaints,escapesetc.

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 13 Revision:2.1

o Supportedotherrelevantmanagementrolese.g.organizationhierarchy,trust,empowerment,responsibledelegation,coaching,sharingknowledge,removingbarriers,routetoescalation

5.1.2

CustomerFocusThingstoconsider:• Topmanagementcommitmenttowardscustomerfocustodemonstratethat:

o Externalrequirementsaredeterminedunderstoodandmete.g.contracts,legislation,benchmarking,surveys,customersatisfaction,marketintelligence,futuretrends,customerexpectations

o Risksandopportunities(see6.1)e.g.competition,capability,resourcing,barrierstomarket,investment,businesscontinuity,innovation,futuretrends,newtechnology,newproducts/services,buildingoncurrentstrengthsaredeterminedandaddressed

o Focusonenhancingcustomersatisfaction(see9.1.2)ismaintainede.g.buildingrelationships,conductingsurveys,customerfeedback,customercommunication,customerperformance,complaintprofile,evaluationofrepeatbusinessandidentifyingopportunitiesforstrengtheningtheorganizationreputationandmarketpresence

o Customerperception(determinedbythecustomer)andcustomersatisfaction(measuredbytheorganization)arealignedo Productandserviceconformityandon-timedeliveryperformanceismeasurede.g.definingperformancecriteria,flowdown

acrosstheorganization,settingtargets,datacapture,datareporting,managementreviewo Actionistakenwhenproductandserviceconformityandon-timedeliveryperformanceisnotachievede.g.ownership,

containment,rootcause,correctiveaction,continualimprovement

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 14 Revision:2.1

5.2 Policy

5.2.1

EstablishingtheQualityPolicyThingstoconsider:• Topmanagementprovisionforestablishingthequalitypolicyincluding:

o Appropriatenessandalignmenttothecontextoftheorganization(see4.1),e.g.purpose,strategicdirection,mission,vision,ethicalprinciples,businessreputation,corevalues,business/functionalpolicies,codesofconductetc.

o Establishmentofaframeworkforsettingqualityobjectives(see6.2),e.g.businessplanning,corethemes,strategicenablers,keyperformanceindicators(KPIs),milestones,resources

o Commitmenttosatisfyapplicableinternal/externalrequirementse.g.industry,business,customer,statutory,regulatoryandotherinterestedparties(see4.2)

o CommitmenttowardscontinuousimprovementoftheQMSe.g.businessperformance,processmaturity,processeffectiveness,customerexpectations,investment,organizationalgrowth,scopeapplication

• Topmanagementprovisionforimplementingandmaintainingthequalitypolicy(see5.2.2)

5.2.2

CommunicatingtheQualityPolicyThingstoconsider:• Arrangementsforcommunicatingthequalitypolicywithinandoutsideoftheorganizationincluding:

o Availabilityacrosstheorganizationandmethodsusedtodeploye.g.hardcopiesdisplayedinprominentplaces,electroniccopiesviaintranet/internetsites,identificationanddistributiontorelevantinterestedparties

o Awareness,understandingandapplicationacrosstheorganization(see7.3and7.4)e.g.communicationmethods(meetings,briefs,e-mail,websites),respondingtofeedback,providinginterpretation,translationintootherlanguages(asrequired),timingandfrequency

• Maintainingdocumentedinformationtodemonstratethatthequalitypolicyiscontrolled,reviewedperiodicallyforcontinuingsuitability,updatedasrequiredandsubjecttofurthercommunicationaccordinglye.g.ownership,review,validity,applicability,distribution,feedback

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 15 Revision:2.1

5.3

OrganizationalRoles,Responsibilities,andAuthoritiesThingstoconsider:• Topmanagementprovisionforassigningrelevantrolesandresponsibilities(i.e.tasksallocatedtoarole)andauthorities(i.e.

permissionsallocatedwithintherole).• Assignmentofrelevantroles,responsibilitiesandauthoritiesacrosstheorganizatione.g.topmanagement,management

representative,functionalleaders,headsofdepartments,processowners,leadprocessusers,endusersetc.relatingto:o ConformanceoftheQMStothe9100standard(see4.3)o Deliveryofprocessoutputresults(see4.4.1)o ReportingofQMSperformanceandimprovementopportunities(see9.3)o Promotingcustomerfocus(see5.1.2)o MaintainingtheintegrityoftheQMSwhenchangesoccur(see6.3)

• Arrangementstodemonstratethatrelevantroles,responsibilitiesandauthoritiesarecommunicatedandunderstood,e.g.organizationchart,resourceallocation,roleprofiles,accountabilitystatements,jobdescriptions,termsofreference,training,competence,qualification,performancereview

• Appointmentofaspecificmanagementrepresentativewhohas:o Responsibilityandauthorityforoversightoftheabovee.g.assignmentofspecificduties,definedresponsibilities,clear

accountabilitieso Organizationalfreedomandunrestrictedaccesstotopmanagementtoresolvequalitymanagementissues,includingliaison

withexternalparties(asappropriate)e.g.organizationalstructure,hierarchy,reportinglines,conflictsofinterest,independence,recognizedauthority

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 16 Revision:2.1

Clause# Considerations6. PLANNING6.1 ActionstoAddressRisksandOpportunities

6.1.1

Thingstoconsider:• OrganizationdeterminationoftherisksandopportunitieswhenplanningfortheQMSarisingfrom:

o Externalandinternalissues(see4.1)o Requirementsofrelevantinterestedparties(see4.2)

• Organizationarrangementsforaddressingtheidentifiedrisksandopportunities,includingthosethathaveanimpacton:o TheQMSachievingitsintendedresultse.g.conformitytorequirements,realizingobjectives,meetingperformancetargetso Enhancingthedesirableeffectse.g.developingopportunities,creatingnewpossibilities,exploringnewmarkets,expanding

thecustomerbase,organizationgrowtho Preventingorreducingundesiredeffectse.g.proactiveriskmanagement,focusingonriskreduction,takingpreventive

measureso Achievingimprovemente.g.meetingtargets,removingnon-valueaddedactivity(waste),positiveperformancetrends

6.1.2

Thingstoconsider:• Organizationarrangmentsfor:

o Planningactionstoaddresstheidentifiedrisksandopportunitiestoensurethatappropriateprocesscontrolsareinplacee.g.ownership,documentedinformation,instructions,methods,verificationactivity,processmonitoring,performancemeasuresandtrends,training,competency

o IntegratingandimplementingactionsintotheQMSe.g.captureoflessonslearned,sharinggoodpractice,processreviews,processupdates,changecontrol,communication

o Evaluatingtheeffectivenessoftheactionstakenincluding;managementreview,effectonproductandserviceconformity,achievingplannedactivitiesandplannedresults,monitoringoftrends,effectoncustomersatisfactionetc.

• Organizationapproachtomanagingrisksandopportunitiestakingintoaccount:o Levelofactivityproportionatetothesizeandcomplexityoftheorganizationandtheimpactontheconformityofproducts

andserviceso Applicationofriskmanagementtechniquesincluding;riskmanagementplan,riskidentification,likelihood/impactof

occurrence,severityofoutcome(e.g.high,medium,low),riskownership,risktreatment,residualrisk,continualmonitoring

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 17 Revision:2.1

etc.o Takingadvantageofnewopportunitiesbybuildingoncurrentstrengths,anticipatingfuturetrends,introducingnew

technology,developingnewproductsorservices,openingnewmarkets,attractingnewcustomersetc.NOTE:Therearemanytoolsandmethodoligiesthatanorganizationcanadopttohelpmanagerisksandidentifyopportunitiesincluding;learningfromthepast(LessonsLearned),PEST(Political,Economic,Social,Technological),PESTLE(Political,Economic,Social,Technological,Legal,Environmental)SWOT(Strengths,Weaknesses,Opportunities,Threats),FMEA(FailureModesandEffectsAnalysis,brainstormingtechniques,BCM(BusinessContinuityManagement),benchmarkingetc.

6.2 QualityObjectivesandPlanningtoAchieveThem

6.2.1

Thingstoconsider:• Organizationarrangementsforsettingqualityobjectivesatvariouslevelsacrosstheorganizationincluding:

o Objectivesestablishedwithintherelevantfunctions(Engineering,Purchasing,Finance,HumanResources,Quality,ITetc.),linkedtotherespectivefunctionalstrategy,e.g.functionaldirection,processimprovement,milestonesetc.

o Objectivesestablishedwithinrelevantprocessesthatcouldinclude;directiondefinedbytheprocessownerrelatingtotheaimsoftheprocess,e.g.whattheprocessistryingtoachieve,resourcesneeded,timescalesetc.

o Objectivesestablishedandsuitablycascadedatvariouslevelsacrosstheorganizationstructuree.g.topmanagement,middlemanagement,supervisors,departments,projects,groups,individuals

• Maintainingdocumentedinformationtodemonstratethatthequalityobjectivesare:o Consistentwiththequalitypolicy(see5.2.1)toensurethattheyunderpinthestrategicdirectionoftheorganization,

supporttheneedsandexpectationsofinterestedpartiesandenhancecustomersatisfactiono Specificandmeasurableinordertogivecleardirectionastowhatisrequiredandtheexpectedoutcome,oftenreferredto

asSMART(Specific,Measurable,Attainable,RealisticandTime-bound)objectiveso Alignedtoapplicablerequirementse.g.thosedefinedbytheorganization,customerorregulatorandrelevanttothe

conformityofproductsandserviceso Monitoredusingsuitablemeanstoensurethattheobjectivesarebeingmete.g.BusinessPlanDeployment(BPD)charts,

dashboards,matrices,reports,progresscharts,trafficlightcharts,managementreviewo Communicatedattherelevantlevelswithintheorganizationtoensurethatteamsandindividualsareawareoftheir

importanceandcontributiono Updatedaccordinglytodemonstrateprogressandtotakeaccountofchangingcircumstancesthatcouldresultinnew,

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 18 Revision:2.1

expanded,amended,cancelledobjectivesetc.

6.2.2

Thingstoconsider:• Organizationplanningtoachieveitsestablishedqualityobjectivesincludingthedeterminationof:

o Whatwillbedonee.g.clearlyestablishedobjectivesacrosstheorganizationatvariouslevels(functional,process,department,team,individual)thataremeasurable-see6.2.1.

o Resourcesrequiredtodelivertheobjectivese.g.numberandcompetencyofpeople,adequateinfrastructure,suitableworkingenvironment,organizationalknowledge,investment,budgets,externalprovision

o Responsibilityforachievingobjectivesatthevariouslevelswithintheorganizatione.g.topmanagement,functionalleadership,departmentleadership,processowners,teams,individuals

o Timescalesforachievingobjectivese.g.steppedachievement(weekly,monthly,quarterly),specificmilestones(defineddatesorperiods),annualachievement

o Methodsusedtoevaluatetheresultse.g.periodiccomparisonofperformanceagainstestablishedtargetsorexpectationsduringmanagementreview,functionalreviews,processcouncils,departmentalreviews,personaldevelopmentreviews(appraisals),teammeetings

6.3

PlanningofChangesThingstoconsider:• TheapproachtakenbytheorganizationtoensurethatchangestotheQMSareplannedandimplementedinacontrolledmanner

takingintoaccount:o Thereasonforthechange;e.g.contextoftheorganization(see4.1),needsofinterestedparties(see4.2),customer

feedback,complaintsanalysis,auditresults,performancetrends,riskreductionorrealizinganopportunity(see6.1),continualimprovement,organizationgrowth,launchofnewproducts/services,organizationrestructuringetc.

o Assessingthepurposeofthechange(s)andpotentialimpactusingariskbasedthinkingapproachtoensuretheintegrityoftheQMSismaintainedi.e.focusonpriorities,avoiddisruption,ensurebusinesscontinuity,maintainproduct/servicereliability,protectthecustomer,maintaincapability,continuetomeetinternal/externalrequirementsetc.

o Theresourcesrequiredtoenablethechangesuchaspeople,knowledgeacquisition,infrastructure,environment,budget,trials/tests,ongoingmonitoring,structuredreviewsetc.

o Determinationofresponsibilityandauthorityforthechangee.g.processowner,leadprocessusers,endusersetc.,includingthenecessarycommunication,trainingandongoingreviewtoensurethechangeiseffective(i.e.theplannedactivitiescontinuetoberealizedandplannedresultsarebeingachieved)

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 19 Revision:2.1

Clause# Considerations7. SUPPORT7.1 Resources

7.1.1

GeneralThingstoconsider:• Organizationdeterminationoftheresourcesneededfortheestablishment,implementation,andcontinualimprovementoftheQMS

takingintoaccount:o Resourceplanning,includingload/capacitybalance,makeversusbuyanalysiso Utilizationofexistinginternalresourcese.g.people,facilities,materials,equipment,finance,capability,capacity,informationo Utilizationofresourcesprovidedbyexternalproviderse.g.processes,products,services

7.1.2 PeopleThingstoconsider:• OrganizationapproachforprovidingthenecessarypeopletoensureeffectiveimplementationandcontroloftheQMSe.g.

organizationalknowledge,capability,skillset,competency,experience,workforceagility,recruitment(temporary,permanent),termsofemployment

7.1.3

InfrastructureThingstoconsider:• Determinationoftherequiredinfrastructuretoenablethestrategicandoperationalneedsoftheorganizationincluding:

o Buildingse.g.manufacturing/assemblyplants,testfacilities,laboratories,servicecenters,officeso Utilitiese.g.electricity,gas,water,compressedairo Equipmente.g.machinetools,jigs,fixtures,tooling,workstations,ITequipment,softwareprograms,assetcareo Transportatione.g.materials/equipmenthandling(internal/external),packaging/protection,hazardoussubstanceso Informationandcommunicationtechnologye.g.ITinfrastructure(servers,backupsystems,businesscontinuity,accessibility,

security,networks)• Considerationoffacilitiesmanagemente.g.organizationalfootprint,realestateplanning,sustainability(energyusage,emissions),

statutoryinspections,restrictedareas,hardservices(infrastructuremaintenanceanddisposal),softservices(cleaning,catering,security)

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 20 Revision:2.1

7.1.4

EnvironmentfortheOperationofProcessesThingstoconsider:• Determinationoftherequiredenvironmenttoenablethestrategicandoperationalneedsoftheorganizationincluding:

o Socialenvironmente.g.ethics,equality,diversity,behaviors,policies,codesofconduct,o Psychologicalenvironmente.g.workload,workinghours,timeconstraints,conflictingdemands,flexibleworking,healthand

wellbeing,confidentialreportinglineso Physicalenvironmente.g.space,ergonomics,workplaceorganization(5S),visualmanagement(layout,zoning,signingetc.)

cleanliness,restrictedareas,electrostaticdischarge,lighting,temperature,humidity,ventilation,noise,FODprevention,wastereduction(Lean)

7.1.5 MonitoringandMeasuringResources

7.1.5.1

GeneralThingstoconsider:• Determinationoftheresourcesrequiredtoenablevalidandreliableresultswhenmonitoring(checking,inspectingobservingor

witnessingactivity)andmeasurement(determinationofphysicalcharacteristics)isappliedtoverifyorvalidateprocess,activity,productandserviceconformity

• Monitoringandmeasuringcanvarydependinguponthecomplexityandneedsoftheorganization,requirementsoftheoutputandexpectationsofthecustomer

• Theselectionandapplicationofappropriateresourcesafterconsideringthecharacteristics,featuresandattributesoftheactivitiesbeingundertaken

• Monitoringandmeasurementresourcescanincludesuitablepeople,facilities,instruments,gages,equipment,devicesetc.,thatarebothinternalorexternaltotheorganization

• Maintenanceofthemonitoringandmeasurementresourcestoensurecontinuedadequacyincludingforexample:o Peopletrainingandcompetencyprovisiono Facilitymaintenanceregimeso Assetcareprogramso Calibrationarrangements

• Documentedinformationtodemonstratethattheresourcesutilisedcontinuetobeappropriateandadequateincludingforexample:plans,schedules,databases,systems,registers,certificates,traceabilitytorecognizedstandards(whererequired)

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 21 Revision:2.1

7.1.5.2 MeasurementTraceabilityThingstoconsider:• Organizationapproachinunderstandingandapplyingthenecessaryarrangementsinordertovalidatemeasurementresultsfor

example:o Identificationandlocationofmeasurementequipment(organization,personal,customerowned)o Identificationofcalibrationequipment(gagenumber,serialnumber,referencenumber)o Methodsofcalibration(procedures,instructions,guides)o Measurementstandardsandacceptancecriteria(organization,national,international)o Frequencyofcalibrationandrecall(timeperiod,advancenotice,duedates,overduenotices)o Suitableenvironmentalconditions(cleanliness,temperature,humidity)o Calibrationstatus(marking,tagging,labeling,colorcoding)o Protectionandpreservation(storageconditions,preventionofadjustment,protectionfromdamage)o Measurementcapability(intervalanalysis,gageRepeatability&Reproducibility(R&R),measurementuncertainty,

measurementsystemanalysis)• Arrangmentsformaintainingdocumentedinformationincludingforexample:

o Equipmentregister(type,identification,location,method,frequency,acceptancecriteria)o Recordsofcalibration(identity,description,date,condition,results,authority,interval,nextduedate,adjustments,limitations

ofuse)• Actiontakenwhenmeasuringequipmentisfoundtobeunfitforitsintendedpurposeanditsimpactonpreviousmeasurementresults

(errorreports,investigation,previousequipmentuseage,riskassessment,reporting,productrecall)7.1.6 OrganizationalKnowledge

Thingstoconsider:• Organizationalapproachtowardsthedeterminationandcaptureofknowledgeincludingforexample:

o Planningforknowledgecapture(knowledgemanagementplan,presentandfutureneeds,sustainability,knowledgesources,approach,scope,ownership,timeliness,knowledgedistribution)

o Knowledgegainedfromexperience(personaldevelopment,successionplanning,subjectmatterexpertise,mentoringprograms,coaching,peerreviews,organizationinsight,businessexposure)

o Knowledgeacquisition(intellectualproperty,patents,lessonslearnedovertime,explicitandtacitknowledge,professionalmembership,industryassociations,benchmarking,conferences,communitiesofpractice,interestedparties,journals,

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 22 Revision:2.1

academia)• Maintainandshareorganizationalknowledge(knowledgemanagementsystem,electronicmedia,intranet,database,repository,

libraries,communitiesofpractice,newcomermentoring,subjectmatterexperts,masterclasses,continualimprovement,sharinggoodpractice,QMSupdates)

7.2 CompetenceThingstoconsider:• Organisationdeterminationofcompetencyrelatingtoperson(s)androle(s)suchas:

o Definedcompetencyrequirements,includinganyspecifiedinduction,training,assessment,evaluationandqualification,roleprofiles,jobdescriptions,accountabilitystatements,skillsmatrices,competencypacks,performancecriteria,developmentframeworks

o Understandingresourceneedsandrelatedcompetency(trainingneedsanalysis,gapanalysis)o Provisioningthenecessarycompetencythroughreassignmentordevelopmentofexistingperson(s),oracquiringcompetent

resourceexternallyo Evidenceofabilitybasedoneducation,training,skills,knowledge,experience,professionalmembershipetc.o Periodicassessmentandevaluationofcompetencytoensurecontinuedadequacyandeffectiveness

• Evidenceofcompetency,includingforexample;trainingcertificates,recordsofachievement,qualificationstatements,IndividualDevelopmentPlans(IDP),ContinuedProfessionalDevelopment(CPD),BodyofKnowledge(BoK),Onthejobtraining(OtJ),logbooks,workplaceassessments

7.3 AwarenessThingstoconsider:• Demonstrationofperson(s)awarenessof:

o Thequalitypolicy(see5.2)andrelevantqualityobjectives(see6.2)througheffectivetwowaycommunicatione.g.prominence,briefings,visualdisplay,intranet,newsletters,employeeengagement,accessandavailability

o TheircontributiontotheeffectivenessoftheQMS,andthebenefitsofimprovedperformancee.g.BusinessPlanDeployment(BPD),flowdownofobjectives,appraisals,performanceratings,Quality/Cost/Delivery(QCD)performancemonitors

o Relevantdocumentedinformation(includingchanges),theircontributiontoproduct/serviceconformityandtheimplicationofnotconformingwithrequirementse.g.accesstotheQMS,navigationofrequirements,notificationofchanges,‘what’snew’notices,nonconformancereporting,understandingofculpabilityandconsequences

o Theircontributiontoproductsafetye.g.individualaccountabilities,compliancetoprocess,attentiontodetail,safety

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 23 Revision:2.1

awarenesstrainingincludingproductendusageandpotentialimpactrelatingtoproductissues,noticeboards,safetyalerts,posters

o Theimportanceofethicalbehaviore.g.codeofconduct,internalmanagement/employeeworkingrelationships,fairtreatment,employeeworkrecognition,confidentialreportingmechanisms,protectinganonymity,noblameculture,awarenesscampaigns,noticeboards,posters,trainingprograms

7.4CommunicationThingstoconsider:• Useofacommunicationstrategy,policy,plan• Organisationapproachtowardsinternalandexternalcommunicationtakingintoaccount:

o Whatistobecommunicatede.g.policy,objectives,QMSrequirements,processes,organizationperformance,customersatisfaction,changesetc.

o Whentocommunicatee.g.frequency,importance,significance,scheduled,adhoc,shiftcoverageo Whotocommunicatewithe.g.relevantinterestedparties:

§ External(customers,regulators,stakeholders,agencies,localcommunity,investors,externalproviders,media)§ Internal(employees,employeerepresentatives,contractors)

o Howcommunicationoccurse.g.meetings,briefings,notices,e-mails,telephone,text,intranet,internet,directives,visualmanagement,campaigns,socialmedia,alerts,bulletins,WebEx’s,webinars,pressrelease,newsletters

o Whoundertakesthecommunication:managers,supervisors,teamleaders,teammembers,employeerepresentatives,corporatecommunication,publicrelations,marketing,externalproviders

• Methodstoenable‘twoway’communicationasappropriateinordertoverifyunderstandingandcapturefeedback7.5 DocumentedInformation7.5.1 General

Thingstoconsider:• ArrangementsforensuringthattheQMSincludesdocumentedinformationappropriatetotheorganization(size,products,services,

processes,complexity,competencyetc.)requiredby:o 9100standardo Organizationsownrequirements

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 24 Revision:2.1

7.5.2 CreatingandUpdatingThingstoconsider:• Arrangementsforcreatingandupdatingdocumentedinformationincluding:

o Identificationanddescriptione.g.subjecttitle,documentidentification,date,author,referencenumber,versionnumber,changehistory,retentioncategory,retentionperiod,securityclassification,exportcontrol,intellectualpropertyclassification,copyright

o Formate.g.language,softwareversion,layout,pagination,style,fonts,resolution,hyperlinksandmediae.g.paper,magnetic,electronicoropticalcomputerdisc,photograph,mastersample,ITsystem,portablemedia,microfiche,X-rayfilm

o Reviewandapprovale.g.reviewperiods,permissions,authorizedpersons,reviewmethods(interestedparties,draftrelease,ballot,buyoff,finalrelease)

7.5.3 ControlofDocumentedInformation7.5.3.1 Thingstoconsider:

• Arrangementsforcontrollingdocumentedinformationrequiredby:o 9100standardo Organizationsownrequirements

• Availabilitye.g.documentaccessibility(hardcopy,electronicmedia),readilyavailableatthepointofuse• Suitabilitye.g.format,mediasuitabletotheenvironment,easeofunderstanding,language,interpretation• Protectione.g.documentauthentication,documentmarkings(official,secret,restricted,confidential,private,sensitive,classified,

unclassified),accesscontrols(individual,rolespecific),physicalsecurity(masterdocuments,serverrooms,libraries)ITsecurity(UserID,password,servers,download,backup,encryption,‘readonly’,‘read/write’),protectionfromcorruptionandunintendedalterations

7.5.3.2 Thingstoconsider:• Documentcontrolarrangementsincluding:

o Distribution(internal/external)e.g.infrastructure,networks,portals,documentregisters,pointsofcontact,transmissiono Accessandretrievale.g.location,availability,timeliness,remoteaccesso Storageandpreservatione.g.physicalenvironment,protectionfromfireandflood,filingarrangements,physical/ITsecurity,

sustainability,redundanthardware/softwaretoaccesslegacymedia,archivearrangementso Changecontrole.g.documentreview,changenotices,versioncontrol,changehistory,re-issue,communication,distributiono Retentione.g.organization/legal/contractualretentionperiods,storage,preservation,backup,retentionofknowledge,

disposal

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 25 Revision:2.1

o Obsolescencee.g.withdrawal,replacement,legacyarchiveandsuitableidentification(“forinformationonly”,“nottobeusedafter….”,“uncontrolledcopy”,“forreferencepurposesonly”etc.

o Electronicdataprotectione.g.securitypolicy,systemaccessprofiles,passwordrules,storageandback-uppolicyincludingprotectionfromloss,unauthorizedchanges,unintendedalteration,corruption,physicaldamage

• Identificationandcontrolofexternaldocumentedinformatione.g.ownership,customerproperty,whereabouts,accessability,pointsofcontact,changenotices,distribution,return,disposal,documentmarkings,licensing,copyright,linkstoexternalrepositories

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 26 Revision:2.1

Clause# Considerations8. OPERATION8.1 OperationPlanningandControl

Thingstoconsider:• Organizationapproach,includingrelevantfunctionengagement(e.g.planning,engineering,operations,purchasing,quality)toplan,

implementandcontroloperationalactivity,takingintoaccount:o Actionsarisingfromorganizationrisksandopportunities,qualityobjectivesandchangestotheQMS(seeclause6)o Provisionfordeterminingrequirementsrelatingtotheorganizationandinterestedparties(see8.2.2)o Establishmentofprocesscriteriae.g.inputs,outputs,controls,resources,sequence,interaction,measures,responsibilities(see

4.4)o Establishmentofcriteriaforacceptanceofproductsandservicese.g.measurement,monitoring,inspection,verification,

validation,test,statisticaltechniqueso Determinationofinternal/externalresourcestoensureconformityandon-timedeliverye.g.people,facilities,materials,

equipment,finance,information,outsourcedprocesses,productsandservices,includingtheuseofsystemssuchasEnterpriseResourcePlanning(ERP),MaterialPlanningSystem(MPS),MaterialResourcePlanning(MRP),load/capacitymodels,salesorderreview,forecastingtechniques,processsimulation

o Controloftheprocessesinaccordancewithcriteriatodemonstratethatintendedoutputsaremetandimprovementsidentifiede.g.processachievement,expectedoutcome,opportunitiesforimprovement

o Othermethodsdeterminedbytheorganizationsuchasanintegratedphasedprocessapproache.g.AdvancedProductQualityPlanning(APQP),ProductionPartApprovalProcess(PPAP)

• Determinationandretentionofdocumentedinformationtoensureprocesscontrole.g.operationalprocessdefinition,productionplanningandcontrol,operationalinstructions,productionsystem,productdatamanagement,qualityplans

• Determinationandretentionofdocumentedinformationtodemonstrateconformitye.g.processoutputrecords,verification/validationresults,scheduleadherence,performancemeasures,managementreviewoutput

• Determinationoftheprocessesandcontrolsneededto:o Managecriticalitems(itemsthathaveasignificanteffectontheprovisionanduseoftheproductsandservices–see91003.2)

e.g.partsclassification,criticalpartplans,sourceandmethodchange,traceability,servicelifemanagemento Managekeycharacteristics(anattributeorfeaturewhosevariationhasasignificanteffectonproductfit,form,function,

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 27 Revision:2.1

performance,servicelife,orproducibility–see91003.3)e.g.componentproving,processcontrolplan,measurementandevaluationtechniques,reviewanddisposition

• Determinationoftheprocessandresourcesneededtomanageuseandmaintenanceoftheproductorservicee.g.servicenetwork,technicalservices,technicalpublications,problemresolutionsystem,feedbackmechanisms,productsupport,aftermarketsupport

• Determinationandcontroloftheproductsandservicestobeobtainedfromexternalproviderse.g.resourceplanning,load/capacitybalance,makeversusbuyanalysis,outsourcingpolicy,outsourcingprocesses,sourcechange(includingpermanentortemporaryworktransfertoanexternalprovider,orfromoneexternalprovidertoanother)

• Establishmentofcontrolsneededtopreventdeliveryofnonconformingproductsandservicese.g.processcontrol,verification/validationactivity,componentproving,riskmanagement,customereyes,delegatedproductreleaseverification

• Appropriateplanningandmanagementofproductandserviceprovisione.g.projectplanning,projectmanagement,programmanagementtakingintoaccount:projectmanagementplans,qualityplans,workbreakdownstructure,sequencing,phases,deliverables,dependencies,constraints,risks,resources,achievementofmilestones

• Controlofplannedandunintendedchanges(e.g.torequirements,processes,sources,methods,schedule,volumes)andconsideringcommunication,impactandmitigationofrisks,pre-postchangeevaluationetc.

8.1.1

OperationalRiskManagementThingstoconsider:• Organizationarrangementsformanagingriskwithinoperations(planningandcontrol,projectmanagement,requirementsforproducts

andservices,designanddevelopment,externalprovision,productionandserviceprovision),takingintoaccount:o Assignmentofresponsibilitiese.g.riskowners,facilitators,champions,coordinators,managers,actionownerso Riskcriteriae.g.criteriarelatingtothemagnitudeofrisk,probability/likelihood(veryhigh,high,medium,low,verylow),

impact/consequence(critical,significant,marginal,negligible)o Riskactivity:

§ Identificatione.g.scope,riskidentity,“if/then”statements,referencenumbers§ Assessmente.g.risktreatment,riskanalysis,applicationofcriteria,risklevels,scoring,ranking,prioritization,keyrisks,

riskassessmentmatrix,riskregister§ Communicatione.g.stakeholderengagement,monitoringandmeasurement,riskaggregation,riskstatus,risktransfer,

riskreports,riskmanagementsystems§ Managementofactionse.g.planning,identity,ownership,timescales,review,riskanalysisupdate§ Riskacceptancee.g.residualrisk,AsLowAsReasonablyPractical(ALARP),rationalrisk,proceedatrisk,contingency

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 28 Revision:2.1

plans,readacrossofriskoutcomestoothersimilarprojects,activities,situations8.1.2 ConfigurationManagement

Thingstoconsider:• Organizationapproachtothemanagementofconfiguration,takingintoaccount:

o Productidentityandtraceabilitye.g.BillofMaterial(BoM),GeneralArrangement(GA),productbreakdownstructure,illustratedpartscatalog,partnumbering,partlists,‘asproduced’condition,configuration(IT)system,standardparts(catalog,offtheshelf),productlogbook,ProductLifecycleManagement(PLM),ProductDataManagement(PDM)

o Identifiedchangese.g.alterationrequests,noticeofchange,amendments,deviations,waivers,partrevisionchanges,partnumberchanges,changecategories,servicebulletins,modificationbulletins,airworthinessdirectives,engineeringcommunicationnotices

o Documentedinformationisconsistentwithproduct/serviceattributese.g.regulations,definitions,drawings,specifications,standards,models,componentproving,productioninstructionsandrecords(manufacturing,assembly,test,repair),productchangerecords,authorizationofchange,typecertificates

8.1.3

ProductSafetyThingstoconsider:• Arrangementsforplanning,implementingandcontrollingtheprocessesneededtoensureproductsafety(abilityforaproductto

performtoitsintendedpurposewithoutcausingunacceptableriskofharmtopersonsordamagetoproperty–see91003.4)includingforexample:

o Assessmentsofhazardsandmanagementofriske.g.riskassessment,DesignFailureModesandEffectsAnalysis(DFMEA),ProcessFailureModesandEffectsAnalysis(PFMEA),safetyanalysis,FailureModesEffectsandCriticalityAnalyses(FMECA),faulttreeanalysis

o ManagementofSafetycriticalitemse.g.monitoringcontrolplan,criticalpartplans,inspectionandserviceintervals,componentlifing,cycliclife,lifemanagementplans

o Analysis,reportingandcommunicationofoccurredeventse.g.datacollectionandanalysis,internalescalationprocess,mandatoryreporting,safetyalertreports,in-servicereliability,operatingperformanceandtrends,lessonslearned,sharedindustryexperience,productsafetyreviews,airworthinessdirectives

o Trainingandcommunicatione.g.productsafetypolicy,promotingasafetyculture,productintegritytraining,productsafetytraining,awarenesscampaigns,safetynotices,safetyalerts,individualreportingmechanisms

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 29 Revision:2.1

8.1.4

PreventionofCounterfeitPartsThingstoconsider:• Arrangementsforplanning,implementingandcontrollingtheprocessesneededtopreventcounterfeitofsuspectcounterfeitparts

(unauthorizedcopy,imitation,substituteormodifiedpartwhichisknowlinglymisrepresented–see91003.1)includingforexample:o Planninge.g.policy,counterfeitpartsplan,governancearrangements,riskassessment,detectionstrategies,counterfeitsources,

reporting,training,engagementwithexternalproviders,communicationo Functionaltraining(prevention,mitigation,detection,dispositionandreporting):o Procuremente.g.trustedsourceselectiono Inspectione.g.preventionofcounterfeititems(visual/test)o Engineeringe.g.Obsolescencemanagemento Awarenesse.g.campaigns,posters,alertso Obsolescencemonitoringe.g.designdecisionsandpartselectionappropriateforservicelifeofproducto Partacquisitione.g.OriginalEquipmentManufacturers(OEM),authorizeddistributors,otherapprovedsourceso AssuringtraceabilitytoOEM,authorizedmanufacturer,approvedexternalprovidero Verificationandtestmethodologiese.g.partmarkings,visualfeatures,inspectionofattributes,functionaltestandvalidation,

packagingo Preventingre-entryintothesupplychain(see8.7.1)e.g.labeling,marking,segregation,quarantine,containment,disposition,

lessonslearned,annotationofdocumentedinformation,electronicsystem(MRP)updateso Reportinge.g.occurrences,events,externalintelligencegathering(databases,notifications,reporting),readacrosstoother

products/services,escalation,riskstointerestedparties,externalcommunication8.2 RequirementsforProductsandServices

8.2.1

CustomerCommunicationThingstoconsider:• Organizationapproachtowardscommunicatingwiththecustomerinrelationto:

o Product/serviceinformatione.g.promotionalmaterial,telephone,journals,conferences,seminars,exhibitions,tradeshows,catalogues,brochures,marketingcampaigns,socialmedia(websites,messageboards)

o Handlingenquiries,contractsororderse.g.engagement,support,pointsofcontact,accessibility,helpline,callcenters,internet‘contactus’,customerledger

o Handlingchangestoenquires,contractsororderse.g.changenotices,amendments,ledgerupdates

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 30 Revision:2.1

o Collatingcustomerfeedback(see9.1.2)e.g.satisfactionsurveys,questionnaires,meetings,socialmedia,forums,events,complaintprofile,reportcards

o Handlingcustomerpropertye.g.registers,identification,location,preservation,maintenance,storageo Contingencyactions(ifrequired)e.g.backupplans,alternativearrangements,proactiveengagement,considerationofrisk

8.2.2 DeterminingtheRequirementsforProductsandServicesThingstoconsider:• Determinationofrequirementsincluding:

o Definedrequirementstakingintoaccount:§ Statutoryandregulatoryrequirementse.g.Airworthinessregulations,ITAR/EAR,REACH,CS-25international/national

legislation,certificationassurance(CE,ULetc.)§ Organizationrequirementse.g.product/serviceknowledge,understandingofcustomerstatedandimpliedneeds,lessons

learned,derivedrequirementsfrompreviousprojects,conceptualthinking,operationalparameters(producibility,safety,reliability,obsolescence,disposaletc.)

§ Specialrequirementse.g.requirementsidentifiedashighrisk,suchasproductorprocesscomplexity,pastexperience(lessonslearned),product/processmaturity,boundariesofperformance,technicalorprocesscapability,identificationofcriticalitems(see91003.5)

o Organizationabilitytomeetrequirementse.g.internal/externalresources(people,facilities,materials,equipment,finance,information),capability,capacity,expertise,product/servicevalidation,operationalrisk(newtechnology,shortleadtimes,resourceconstraints,interdependencies)

8.2.3 ReviewoftheRequirementsforProductsandServices8.2.3.1 Thingstoconsider:

• Organizationreviewofrequirementscoordinatedwithapplicablefunctions(e.g.sales,planning,engineering,operations,purchasing,finance,legal,quality),beforecommittingtosupplyproductsandservicesincluding:

o Requirementsspecifiedbythecustomere.g.withinenquiries,contracts,purchaseorders,statementsofwork,memosofunderstanding,lettersofintent,termsandconditions

o Requirementsimpliedbythecustomere.g.useofpreviouscontractreviewactivity,customerpreferences,customerfeedback,organization/industryknowledge,lessonslearned,marketplace/competitorintelligence,inserviceperformancedata,previousexperience,readacrossfromothersimilarproductsorservices

o Requirementsfordeliveryandpostdeliverye.g.logistics,shipping,transport,installation,commissioning,training,handover,

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 31 Revision:2.1

customeracceptance,warranties,repairs,customersupporto Organization(e.g.policies,processes,procedures,standards,specifications,rules,codesofpractice)and,statutory/regulatory

requirements(e.g.airworthinessregulations,ITAR/EAR,REACH,CS-25international/nationallegislation,certificationassurance(CE,ULetc.)

• Resolutionofcontractororderrequirementsthathavebeensubjecttochangee.g.identificationofthedifferences,communicatingwiththecustomer,clarificationandunderstanding,impactassessment,mutuallyacceptableoutcome

• Confirmationofrequirementswhentheyarenotdocumentede.g.clarification,understanding,acknowledgementandsubsequentagreementintheformofretaineddocumentedinformation

8.2.3.2 Thingstoconsider:• Retentionofdocumentedinformationtocapturetheresultsofthereviewincludinganynewrequirementsorchangese.g.recordof

contractreview,includingforexamplecustomer,reference,date,persons,resources,conventional/specialrequirements,risksoutcome,changes

8.2.4 ChangestoRequirementsforProductsandServicesThingstoconsider:• Amendeddocumentedinformationandcommunicationofchangedrequirementse.g.updatedcontractreviewrecords,amended

orders/contracts,memos,changenotices,quailityplans,meetingminutes,togtherwithcommunicationtorelevantinterestedparties(personswithinoroutsidetheorganizationthatmaybeimpactedbythechange)

8.3 DesignandDevelopmentofProductsandServices8.3.1 General

Thingstoconsider:• Organizationarrangementsforestablishing,implementingandmaintainingadesignanddevelopmentprocessincludingplanning,

inputs,controls,outputsandchanges(see8.3.2to8.3.6)8.3.2 DesignandDevelopmentPlanning

Thingstoconsider:• Organizationapproachtowardsplanningdesignactivityincluding:

o Scopeofthedesigne.g.requirementscapture(see8.2.2)rationale,assumptions,objectives,complexity,size,detail,timescales,criticality,constraints,risks,producibility,maintainability

o Stagesofthedesignprocess,distinctactivitiesandreviewe.g.workbreakdownstructure,workpackages(tasks,resources,responsibilities,content,inputs/outputs),conceptdesign,preliminarydesign,detaildesign,reviewgates(preliminarydesign

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 32 Revision:2.1

review,detaildesignreview,criticaldesignreview)o Verificationandvalidationactivitye.g.checks,trials,tests,simulations,demonstrationsrequiredtoensurerequirementsaremeto Responsibilitiesandauthoritiese.g.roleprofiles,accountabilitystatements,delegationofauthority,levelsofapproval,register

ofauthorityandapprovals,authorizedsignatorieso Internal/externalresourcese.g.knowledgeacquisition,people,competency,investment,funding,facilities,equipment,

innovation,technology,interestedparties(customers,externalproviders,researchestablishments),information(principles,standards,rules,codes)

o Organizationalinterfaces(persons/functions)e.g.sales,projectmanagement,production,procurement,quality,finance,customers,endusers

o Subsequentprovisionandapplicatione.g.forwardthinking,readacross,re-useofnewtechnologies,product/servicederivatives,sustainability,recognizingobsolescence

o Levelsofcontrolrequiredorimpliedbyinterestedparties(customers,regulators,endusersetc.)e.g.customeracceptance,safetychecks,riskmanagement,verification/validationactivity,productcertification

o Requireddocumentedinformatione.g.designplan,designreviews,designoutputs(specifications,schemes,drawings,models,data,reports),controlplans,certificates

8.3.3 DesignandDevelopmentInputsThingstoconsider:• Determinationofinputrequirementstogetherwiththeretentionofdocumentedinformationincluding:

o Functionalandperformancerequirementse.g.customerneeds,operatingcharacteristics,performanceparameters,safety,usability,reliability,maintainability

o Informationtransfere.g.readacrossfromothersimilardesigns,lessonslearned,performancedata,in-servicedata,customerfeedback,externalproviderfeedback,bestpractice,benchmarking

o Statutoryandregulatoryrequirementse.g.legislation,regulations,directiveso Standardsorcodesofpracticee.g.policies,standards,specifications,rulesandaids,protocols,guidance,industrycodeso Consequencesoffailuree.g.riskprofile,DFMEA,safetyanalysis,FMECA,faulttreeanalysis,lessonslearnedo Consequenceofobsolescencee.g.sourceofsupply,sustainability,prohibitedmaterialsandsubstances,conflictminerals,

specialitymaterials,proprietaryparts,exposureofcounterfeito Adequacyofinputse.g.clear,complete,unambiguous,understandable,transmittableo Conflictinginputsareresolvede.g.functionalconsensus,communicatingwithinterestedparties,contract/orderamendment,

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 33 Revision:2.1

interestedpartyagreement,mutuallyacceptableoutcome8.3.4 DesignandDevelopmentControls

Thingstoconsider:• Determinationofcontrolstogetherwiththeretentionofdocumentedinformationincluding:

o Definitionofoutcomese.g.designintent,functionalandperformancerequirements,customer/enduserexpectationso Designreviewe.g.functionalrepresentation(customer,engineering,production,quality,projectmanagementetc.),review

gates(preliminarydesignreview,detaildesignreview,criticaldesignreview),commercial/technicalconsiderations,authorizedprogressiontonextstage(proceedtonextgate)

o Verificationactivitye.g.modeling,simulations,alternativecalculations,comparisonwithotherprovendesigns,experiments,tests,specialistoverview(stress,performanceetc.)

o Validationactivitye.g.functionaltesting,performancetesting,trials,prototypes,demonstrations,simulationso Managementofactionsarisingfromdesignreview,verificationorvalidatione.g.actionregisters,ownership,timescales,

escalation,changestoriskprofile8.3.4.1 Thingstoconsider:

• Planningandcontrollingverificationandvalidationtestsincluding:o Testrequirementse.g.productidentity,objectives,schedules,plans,specifications,resources(persons,equipment,

infrastructure),parameters,conditions,acceptancecriteria(pass,failetc.)o Testmethodse.g.instructions,procedures,schedule,checklists,logbooks,diaries,datacaptureo Configuratione.g.productconfiguration(item,assembly,module,product)andtestconfiguration(facilities,equipment,

conditions,software,controlledmonitoringandmeasuringdevices–see7.1.5)o Retaineddocumentationtodemonstratethatthe:

§ Testplanandtestprocedureshavebeenobservedandcriteriahasbeenmet§ Designmeetsthespecifiedrequirementsforallidentifiedoperationalconditionse.g.reports,calculations,testresults,

data,reviews,outputs8.3.5 DesignandDevelopmentOutputs

Thingstoconsider:• Outputsmeetinputrequirementse.g.checklists,designreviewrecords,authorizationtoproceed,customeracceptance,product

certification• Outputsareadequateforproductandserviceprovisione.g.standards,specifications,schemes,drawings,models,partlists,materials,

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 34 Revision:2.1

methods,manufacturinginstructions,technicalpackages,tooling,machineprograms,preservation,handling,packaging,specialisttraining,userinstructions,servicemanuals,repairschemes,externalprovision,togetherwith:

o Referencetomonitoringandmeasuringequipmente.g.inspectionequipment,gages,instruments,environmento Acceptancecriteriae.g.product/servicespecification,limits,tolerances,qualityacceptancestandardso Product/servicecharacteristicse.g.keycharacteristics,customercriticalfeatures,interfacefeatures,inspections,service

intervals,operatingcharacteristics,o Criticalitems(see91003.2)e.g.identification,keycharacteristics,specialhandling,serviceintervals,componentlifing,cycliclife,

lifemanagementplans,sourceandmethodchange,traceability• Outputsareapprovedpriortoreleasee.g.scopeofauthorization,authorizedpersons,levelsofauthorization,methodofauthorization

anddocumentedinformationisretained8.3.6 DesignandDevelopmentChanges

Thingstoconsider:• Organizationapproachfortheidentification,reviewandcontrolofchangesincluding:

o Implementationofaprocesstonotifythecustomerwhenchangesaffectthecustomerrequiremente.g.customercommunication,notificationsofchange,requestsfordeviation,contractamendments

o Configurationcontrole.g.alterationrequests,noticeofchange,amendments,deviations,waivers,concessions,partrevisionchanges,partnumberchanges,changecategories,servicebulletins,modificationbulletins,airworthinessdirectives,engineeringcommunicationnotice,productchangeboards

• Retaineddocumentedinformationthatincludeschangehistory,evaluationofchangeresults,authorizationofchangeandactionstakeninrelationtosubsequentactivitiesthatareimpactedbythechange

8.4 ControlofExternallyProvidedProcesses,ProductsandServices8.4.1 General

Thingstoconsider:• Organizationarrangementsforensuringexternalprovidedprocesses,productandservicesconformtorequirementswhen:

o Incorporatedintotheorganizationsownproductsandservicese.g.boughtoutfinished/completeproducts/serviceso Provideddirectlytotheorganizationscustomere.g.directship,directdeliveryo Aprocessorpartofaprocessisprovidede.g.outsourcing,offload,operationalprocessing(conventional/specialprocessesetc.),

supportprovidedon-siteattheorganizationpremises(ITsupport,facilitiesmanagementetc.)

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 35 Revision:2.1

• Takingintoaccount:o Retainingresponsibilityforexternalprovisionfromallsources,includingthosedefinedbythecustomere.g.organizationtakes

fullresponsibility,irrespectiveofanysourcesidentifiedbythecustomero Theuseofapprovedexternalprovidersincludingthosedesignatedbythecustomere.g.

§ Sourcingfromanapprovedsupplierlistwheretheorganizationsapprovalscopematchestherequirements§ Sourcinginaccordancewithcustomerinstructions(communications,contracts,memorandaetc.)

o Theidentificationandmanagementofrisks(see8.1.1)associatedwith:§ Externalprovisione.g.assignmentofresponsibilities,riskcriteria,riskidentification,riskassessment,communication,

managementofactions,riskacceptance§ Selectionanduseofexternalprovidersconsideringsuchthingsas:structure(ownership,parentcompany,subsidiaries,

locationetc.),marketintelligence,customerinput,financialstability,capability,termsandconditions,ratings,approvals(customer,regulator,3rdpartyetc.)

o Theapplicationofappropriatecontrolstoensurethatexternalproviderscontroltheirdirectandsub-tierexternalproviderse.g.approvals,contractconditions,flowdownrequirements,purchaseorderterms,mandatoryinstructions,nondisclosureagreements,statutory/regulatorydirectives,delegations

• Applyingcriteriafortheevaluation,selection,monitoringandre-evaluationofexternalprovidersincluding:o Commoditystrategye.g.products,services,categories,interestedpartyrequirementso Sourcingstrategye.g.policies,scope,goals,objectives,requirementscapture,riskso Selectioncriteriae.g.profile,portfolio,location,pre-requisites,capability,marketintelligence,strategicalignment,financial

stability,ethicalperformance,externalapprovals,externallyavailabledataandinformation(reports,disclosures,satisfactionindexes,performanceratings,reputation,otherpartyfeedbacketc.),

o Approvalmethodologye.g.approvalrequests(new,extendedetc.),scopeofapproval,externalapprovals/certifications(customer,regulator,3rdpartyetc.),authorizedpersons,2ndpartyauditactivity,approvalstatus(approved,suspended,withdrawn,lapsed,conditional,disapprovedetc.)

o Performancemonitoringe.g.measures,targets,KPIs,scorecards,dashboards,ratings,surveyso Retaineddocumentedinformationandactionsarisingfromtheevaluationse.g.approvedsupplierlist,auditreports,

performancereports,riskassessment,requestsforinformation(RFI),dispositionedapprovalrequests,certificates,gapanalysis,correctiveactionreports

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 36 Revision:2.1

8.4.1.1 Thingstoconsider:• Definedprocess,responsibilitiesandauthorityforexternalproviderapproval,includingforexamplesupplierselectionandapproval

process,rolesandaccountabilitystatements,levelsofauthorization,scopeofapproval(products,services,processes,commoditiesetc.),changestoapproval(approved,suspended,withdrawn,lapsed,conditonal,disapprovedetc.),monitoringandmeasurement(performancereviews,surveillanceactivityetc.)

• Maintainedregisterofexternalprovidersincludingforexampleapprovedvendorlist,approvedsupplierlist,vendordatabase,scopestatements,approvalstatus

• Periodicreviewofexternalproviderperformanceincludingconformity,on-timedeliveryandotherpertinentmeasures(KPIs,targets,measures,values),performancereviews,managementreview,dashboards,flightdecks,reportsetc.,andarrangementsfortakingactionwhenrequirementsarenotmete.g.containment,correction,rootcause,correctiveaction,effectivenessofresolution,changetoapprovalstatus

• Definedrequirementsforcontrollingdocumentedinformationcreatedbyand/orretainedbyexternalprovidersincluding:o Communicatione.g.contractconditions,flowdownrequirements,purchaseorderterms,mandatoryinstructionso Documenttypes(maintained/retained)andcontrol(storage,protection,retrieval,media,retentionperiodsetc.)

8.4.2 TypeandExtentofControlThingstoconsider:• Organizationapproachtowardsensuringthatexternallyprovidedprocesses,productandservicesdonotimpactuponthedeliveryof

conformingproducttothecustomer,takingintoaccount:o ExternallyprovidedprocessesremainwithintheorganizationsQMScontrole.g.documentedinformationisalignedtoensure

commoninputs,outputs,controls,ownership,governanceetc.,betweentheorganizationrequirementsandthosethatareusedtointerfacewiththeexternalprovider

o Thedefinitionofcontrolsappliedtoanexternalproviderandthoseappliedtotheresultingoutpute.g.definedexternalproviderrequirementsandalignmentwiththecontrolsappliedtothesubsequentoutput

o Considerationof:§ Theimpactofexternalprovisionandtheorganizationsabilitytoconsistentlymeetcustomer,statutoryandregulatory

requirementse.g.thosedefinedincontracts,legislationandregulations§ Theeffectivenessofthecontrolsappliedtotheexternalprovidere.g.requirements,selectionandapproval,verification

activities,performancemonitoring,correctiveaction,continualimprovemento Determinationofverificationactivitiesaccordingtotherisksidentifiedbytheorganization,thatcaninclude:

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 37 Revision:2.1

§ Reviewofdocumentedinformatione.g.certificatesofconformity,testreports,releasecertificates,regulatorycertificates(EASAForm1,FAA8130etc.),processcontroldocuments,productionprocessverificationactivity(see8.5.1.3)etc.

§ Inspectionandauditattheexternalproviderspremisese.g.sourceinspection,oversight,witness,sampling,3rdpartyverification,2ndpartyaudits,surveillanceactivity,processmonitoring,productaudits

§ ReviewofProductionPartApprovalProcess(PPAP)datae.g.controlplan,product/processcharacteristics,statisticalprocesscontrol,riskassessment,measurementsystemanalysis,verificationresults,PPAPfileetc.)

§ Inspection/verificationuponreceipte.g.activityundertakenbytheorganizationtoconfirmthatoutputsmeetstatedrequirementsthatcaninclude,definedlevelsofreceiptinspection,physicalinspectionoftheproducts,confirmationoftheserviceprovided,samplinginaccordancewithadefinedplan

§ Reviewofdelegationforproductverificatione.g.requirements,criteria,scope,registerofdelegations,levelsofauthority/approvals,authorizedsignatories,periodicmonitoringbytheorganization(oversight,witness,requalificationetc.)

§ Customerverificationatanylevelofthesupplychaine.g.oversight,witnessing,inspection,documentationreview,productaudit,productacceptancetesting

o Inspectionorperiodictestingwhenthereisahighriskofnonconformity,includingcounterfeitpartse.g.identification,detectionstrategies,communication,MRPindication/signal/alerts,inspectionlevels,verificationandtestmethodologies(partmarkings,visualfeatures,inspectionofattributes,functionaltest/validationetc.)

o Controlofproductsreleasedforproductionusependingcompletionofverificationactivitye.g.identification,communication,documentannotation,traceability,authorizationtoproceed,electronicsystem(MRP)updates,tracking/recallarrangements

o Utilizationoftestreportsduringverificationtoconfirmproductrequirementshavebeenmete.g.evaluationoftestreportdata(performancevalues,materialproperties,chemicalcomposition,productattributes,functionalcharacteristicsetc.)

o Validationoftestreportaccuracywhenrawmaterialhasbeenidentifiedasasignificantoperationalriske.g.provingthattestreportdatameetsproductrequirementsbyconductingsamplevalidationactivitysuchasfunctionaltesting,performancetesting,destructivetesting,laboratoryanalysisoftestspecimensetc.

8.4.3 InformationforExternalProvidersThingstoconsider:• Organizationarrangementstoensuretheadequacyofrequirementspriortocommunicatingwiththeexternalprovidere.g.clarityof

requirements,clearunderstanding,unambiguousdefinition,identificationofpotentialissues• Organizationcommunicationtotheexternalproviderincluding:

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 38 Revision:2.1

o Requirementsoftheprocesses,productsandservicesthatarebeingprovisioned(identity,description,partnumbers,quantity,deliveryschedule,serviceparametersetc.),togetherwiththeidentificationofanytechnicaldata(standards,specifications,drawings,processrequirements,instructionsetc.)

o Approvalofproducts,services,methods,processes,equipmentandreleasee.g.criteria,requirements,acceptance,buyoff,delegations,permissions,authorizations,licenses,conditions

o Competencyrequirementsandpersonqualificatione.g.skills,training,assessment,evaluation,levelsofqualification,periodicre-assessment,maintainedcompetence

o Interactionse.g.engagement,communication,reviews,meetings,exchangeofdocumentedinformation,notices,directives,feedback

o Controlandmonitoringofperformancee.g.datacapture,measures,targets,KPIs,scorecards,dashboards,ratings,surveyso Verificationorvalidationactivitiesattheexternalproviderpremises(includingthecustomer)e.g.intentforsourceinspection,

oversight,witness,sampling,3rdpartyverification,2ndpartyaudits,surveillance,processmonitoring,productaudits,functionaltesting,performancetestingetc.

o Controlofdesignanddevelopment(see8.3)e.g.requirements,statementofwork,planning,controls(review,verification,validationetc.),delegations,maintained/retaineddocumentedinformation,designanddevelopmentchanges

o Specialrequirements,criticalitems,keycharacteristicse.g.specifications,performancecriteria,riskmanagement,identification,traceability,enhancedproduction/processcontrols,monitoringandmeasurementarrangements,fixedprocessapprovals,productqualification,changecontrol

o Test,inspectionandverificatione.g.requirements,instructions,acceptancecriteria,equipment,productionprocessverification(see8.5.1.3),retaineddocumentedinformation

o Useofstatisticaltechniquesforproductacceptance,includingacceptanceofinstructionse.g.applicationof;statisticalprocesscontrol(SPC),processmonitoring,samplingactivityandorganizationacceptanceofexternalproviderarrangements(review,approval,buyoffetc.)

o Theneedto:§ ImplementaQMSe.g.statementofrequirementsthatcanincludealignmentoftheexternalprovidersQMSto

recognizedexternalstandard(s),requirementtomaintainrelevantapprovals/certificationetc.§ Usecustomerdesignated,orapprovedexternalproviderse.g.statementofrequirementsrelatingtoexternalprovisionof

processes(includingspecialprocesses),productsandservicesfromapprovedsuppliers,includingthosefloweddownbytheorganizationscustomerwhereapplicable

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 39 Revision:2.1

§ Notifynonconformingprocesses,productsandservicesandobtainapprovalfortheirdispositione.g.statementofrequirementsrelatingthecontrolofnonconformingoutputsthatcanincludeprocessrequirements,controlarrangements,requireddocumentedinformation,reportingmethod,responsibilitiesandauthorities(includingdelegationifapplicable),implementingdispositioninstructions,nonconformitymarking(e.g.concessionnumbersifapplicable)etc.

§ Preventtheuseofcounterfeitparts(see8.1.4)e.g.statementofrequirementsthatcanincludetheneedtoconductariskassessment,useofapprovedexternalproviders,verificationandtestmethodologies,preventingre-entryintothesupplychain(detection,segregation,quarantine,containment,dispositionetc.),reportingoccurrences,managingobsolescenceetc.

§ Notifychangestoprocesses,products,services,includingchangeofsource/locatione.g.statementrelatingtowhatconstitutesachange(includingchangestosourceandmethod)andtherequirementsforcontrol,reporting,obtainingapproval,actingupontheorganizationsresponseandmaintainingprocess/product/serviceconfigurationasapplicable

§ Flowdownrequirementse.g.specificrequirementsrequiredbytheorganization,ortheorganization’scustomer(asrequired)suchasmaintainingapprovals,useofapprovedexternalsub-tierproviders,verificationrequirements,notificationrequirements,reportingrequirementsetc.

§ Providetestspecimensfordesignapproval,inspection/verification,investigationorauditinge.g.statementofrequirementfortheprovisionoftestspecimens/pieces/artifacts,includingtype,identification,configuration,supportingdocumentedinformation,releaseconditions

§ Retaindocumentedinformation,includingretentionperiodsanddispositionrequirementse.g.statementofrequirementsrelatingtothecontrolofrecordsthatcanincludetype,media,backup,archive,storage,protection,preservation,access,retrieval,retentionperiods,disposal,returntotheorganization,security,dataprotection(electronicdocumentedinformation)etc.

o Rightofaccesse.g.acceptanceoftheorganizationsrightofaccesstotheexternalproviderspremisestoattendmeetings,conductperformancereviews,reviewworkinprogress,conductinvestigations,resolveproblems,conduct2ndpartyaudits,supportimprovementactivity,deliveryoftraining

o Ensuringpersonsareawareof:§ Theircontributiontoproduct/serviceconformitye.g.individualaccountability,understandingrequirements,compliance

toprocess,theneedtocontrolchanges,reportingofnonconformance§ Theircontributiontoproductsafetye.g.individualaccountability,compliancetoprocess,attentiontodetail,knowledge

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 40 Revision:2.1

ofproductendusage,potentialimpactrelatingtoproductissues§ Theimportanceofethicalbehaviore.g.codeofconduct,management/employeeworkingrelationships,fairtreatment,

employeeworkrecognition,confidentialreportingmechanisms,protectinganonymity,noblameculture8.5 ProductionandServiceProvision8.5.1 ControlofProductionandServiceProvision

Thingstoconsider:Implementationandcontrolofproductionandserviceprovisionincluding:• Availabilityofdocumentedinformationthatdefinesproduct/servicecharacteristics,activitiesandresultstobeachievede.g.works

orders,digitalproductdefinitiondata,designdefinitions,arrangementdrawings,component/assemblydrawings,keycharacteristics,processflowcharts,partslists,technicalstandards,materialspecifications,processspecifications,manufacturingplans,inspectionplans,productioninstructions,processdatacards,standardoperatingprocedures,setupdiagrams,toolinglists,gagelists,machinetoolprograms,batchcards,processcards,routers,travelers,controlplans,qualityplans,testschedulesproduction/serviceparameters,serviceinstructions,servicelevelagreements,inspection/verificationprocedures,acceptancecriteria

• Availabilityofmonitoringandmeasuringresourcese.g.persons,equipment,tooling,gaging,instrumentation,facilities(includingenvironmentalcontrols),machinetool/CoordinateMeasuringMachine(CMM)probing/profiling/camera/lasersystems,softwareprograms

• Implementationofmonitoringandmeasuringactiviese.g.o Inspection/verificationplanning,identificationofinspection/verificationpoints(sequencing)o Criteriaforacceptance/rejection,specifications,standards,performanceparameters,pass/faillimits,acceptancestandards,

visualaidso Equipmentrequirements(tools,gages,fixtures,machinetools,CMMsetc.)andinstructionsforuse(setupsheets,visualaids,

softwareprograms,inspection/verificationprocedures,operatinginstructions,standardoperatingprocedures,operatortrainingetc.)

o Retainedmeasurementresultse.g.inspection/verificationhistorycards,CMMreports,pass/failindicators,testresults,verificationreports,electronicdatacapture,FAI(see8.5.1.3)

o Samplinge.g.riskassessment,statisticalrationale/basis(AcceptanceQualityLimits(AQL),UnacceptableQualityLevel(UQL),establishedprocesscapability),samplingplan(producttype,features,characteristics,batch/lotsize,samplesize,inspectionlevels,typeofinspection),defectiveproduct(hold,reinspect,reprocess)

• Operatinginfrastructureandenvironment(see7.1.3&7.1.4)e.g.

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 41 Revision:2.1

o Buildings(manufacturing/assemblyplants,testfacilities,laboratories,servicecenters,officesetc.)o Utilities(electricity,gas,water,compressedairetc.)o Equipment(machinetools,jigs,fixtures,tooling,molds,workstations,materialhandlingetc.)o ITinfrastructure(networks,servers,backupsystems,hardware,softwareprogramsetc.)o Physicalenvironmente.g.space,ergonomics,workplaceorganization(5S),visualmanagement(layout,zoning,signingetc.)

cleanliness,restrictedareas,electrostaticdischarge,lighting,temperature,humidity,ventilation,noise,FODprevention,wastereduction(Lean)

• Appointmentofcompetentpersons(see7.2)e.g.competencyrequirements(education,training,skills,knowledge,experience),alignmenttorole(roleprofiles,jobdescriptions,accountabilitystatements,skillsmatrices),evidenceofability(assessment,evaluation,qualification,requalification)etc.

• Validationandrevalidationofspecialprocesses(see8.5.1.2)e.g.requirements(standards,specifications,instructions),validationactivity(methods,techniques,instructions,datacards,acceptancecriteriaetc.),approvalarrangements(facilities,plant,equipment,process,documentsetc.),processmonitoring,equipmentcalibration,personqualification(training,skills,assessment,levelsofauthorizationetc.),periodicrevalidationincludingchangecontrol

• Preventionofhumanerrore.g.riskassessment,training,competencyassessment,qualification,automation,errorproofing,mistakeproofing,humanfactors,effectivecorrectiveaction,readacross

• Release,deliveryandpostdeliveryactivities(see8.5.5&8.6)e.g.o Release/deliverye.g.verificationthatrequirementshavebeenmet,customeracceptancetesting(ifrequired),product

certification/qualification,approvalfromtherelevantauthoritywhenrequirementshavenotbeenmet(concession,deviation,waiveretc.),releaseauthorizations,certificatesofconformity,releasecertificates,regulatorycertificates(EASAForm1,FAA8130etc.),dispatchdocuments(consignmentnotes,deliverynotes,shippingnotesetc.),exportcontrollicenses

o Postdeliverye.g.requirements(organization,customer,statutory,regulatory),dealingwithcustomerfeedback(communication,compliments,complaints,issues,problemresolution,satisfaction,warranty,returns,rejections),recallnotices,product/servicesupport(queries,training,aftersales,maintenance,servicing),collection/analysisofin-servicedata,lessonslearned

• Workmanshipcriteriae.g.standards,samples,artifacts,visualaids,photographs,videos,diagrams,illustrations,comparitors,acceptance/rejectioncriteria

• Accountabilityforproductsduringproductione.g.parts/batchquantities,serialisation,quantityadjustments,(splitbatches/orders,nonconformingproducts),MRPupdates,controlboardupdates,worktracking/booking,WorkInProgress(WIP)monitoring

• Criticalitemse.g.identification,traceability,production/processcontrol,monitoringandmeasurementarrangements,focusonkey

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 42 Revision:2.1

characteristics,fixedprocessapprovals,productqualification,changecontrol• Methodstomeasurevariabledatae.gverification/inspectionplans,methods,instructions,techniques,identificationofgenerictooling,

equipment,gages,instrumentation,identificationofunique‘specialtoproduct’equipment,applicationofmachinetool/CMMprobing/profiling/camera/lasersystems

• Planningforinprocessinspection/verificationwhenverificationcannotbeperformedatalaterstageforexamplepriortoprocessingwhenfeaturesbecomehidden(coating,painting,surfacetreatment),priortoassemblywhenfeaturesbecomeinnaccessible(sub-assemblies,finalassemblies,modules,finalproduct)

• Evidencethatalloperationshavebeencompletede.g.accountingforworkviatheuseofsignatures,stamps,electronicworkbookingetc.,orotherwisedocumentedandauthorizedwherecompletionofanoperationisnotinaccordancewithinstructionse.greason,dispositionandauthorization/approval

• Prevention,detectionandremovalofFODe.g.o Prevention(campaigns,signage,briefings,training,bulletins,alerts,riskassessment,controlplans,lessonslearnedetc.)o Detection(inspection,instrascopes,boroscopes,x-raytechniques,customereyesoverchecksetc.)o Removal(cleaning,purging,disassemblyetc.)

• Monitoringofutilitiesandsupplies(electricity,gas,water,compressedair,chemicalproductsetc.)thathaveanimpactontheproductincluding;contamination,cleanliness,chemicalstrength/constituency,pressure,voltagesupplyetc.andcontrolarrangmentse.g.instructions,procedures,frequencyofchecks,analysisofresults,corrections/adjustments

• Productsreleasedforsubsequentproductionusependingcompletionofmeasuringandmonitoringactivitye.g.identification,communication,documentannotation,traceability,authorizationtoproceed,electronicsystem(MRP)updates,tracking/recallarangements

8.5.1.1 ControlofEquipment,Tools,andSoftwareProgramsThingstoconsider:• Organisationarrangementstocontrolequipment,toolsandsoftwareprogramsusedwithinproductionincluding:

o Validateequipment,toolsandsoftwareprogramse.g.instructions,validationcriteria,capabilitystudies,productiontrials,prototypes,testpieces,tooling/equipmentproving,componentproving,productionacceptance,PFMEA,GageR&R,controlcharts,productionprocessverification(see8.5.1.3),acceptancetesting,equipmentcommissioning,verificationresults,releasetoproduction

o Maintainequipment,toolsandsoftwareprogramse.g.identity,configuration(version/issuecontrol),changecontrol(includingauthorization),plannedmaintenance,servicing,calibration,softwareprotectionandbackup

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 43 Revision:2.1

o Storageofequipmentandtoolinge.g.identity,environmentconditions(space,cleanliness,housekeeping,temperature,humidityetc.),preservationandprotection(corrosion,deterioration,degradation,damage/lossprevention),assetregisters,conditionchecks,segregation(includingequipmentandtoolingbelongingtoothers–see8.5.3)

8.5.1.2 ValidationandControlofSpecialProcessesThingstoconsider:• Organisationarrangementsforvalidationandcontrolofspecialprocesses(e.g.heattreatment,chemicalprocessing,composites,

nonconventionalmachining,nondestructivetesting,joining,coating,surfaceenhancement)including:o Criteriaforprocessreviewandapprovale.g.technicalrequirements,verificationandvalidationactivity,approvalarrangements

(process,documents,personsetc.)o Conditionstomaintainprocessapprovale.g.compliancechecks,processoversight,specialprocessaudits,requalificationof

persons,plannedmaintenance,processmonitoring,processsurveillance,changecontrolo Approvaloffacilitiesandequipmente.g.plant(furnaces,machinetools,processlinesetc.),equipment,tooling,process

consumables,maintenanceregime,calibrationrequirements,testpieces,standards,specificationso Qualificationofpersonse.g.skillsmatrices,training,oversight,workplaceassessment,competencyassessment,levelsof

authorizationo Methodsandprocedurese.g.techniques,instructions,datacards,standardoperatingprocedures,diagrams,setupsheets,

acceptancecriteriao Retaineddocumentedinformation(thosedocumentsneededtodemonstratethatprocesseshavebeencarriedoutasplanned)

e.g.personqualification,processcards,routecards,furnacecharts,laboratoryreports,testpieces,testresults,x-rayfilm,maintenancerecords,calibrationrecords,consumabletraceability,verificationresults

8.5.1.3 ProductionProcessVerificationThingstoconsider:• Organisationarrangementstoverifythattheproductionprocessisabletoproduceproductsthatmeetrequirements,includingfor

example:o Capacity/capabilitystudiese.g.determinationofproducibility,methods,trials,tooling/equipmentproving,componentproving,

productionratereadiness,retaineddocumentedinformation(verificationresults,assessment/analysisreports,data,informationetc.)

o ProductionProductApprovalProcess(PPAP)e.g.definedproductionrun,controlplan/methods,resources,product/processcharacteristics,statisticalprocesscontrol,riskassessment(PFMEA),measurementsystemanalysis(GageR&R),retained

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 44 Revision:2.1

documentedinformation(timingplan,controlplan,verificationresults,data,information,PPAPfileetc.)o Firstarticleinspection(FAI)e.g.:

§ Provisionofresources(persons,facilities,equipmentetc.)§ SelectionofarepresentativeFAIitem(newproductintroductionorexistingproductsubjecttochange)§ IdentificationoftheFAIitem(PlanningfortheFAIitem,MRPsignalfortheFAIitem,uniqueidentifieretc.)§ Verification/inspectionactivity(requirements,characteristics,features,specialprocesses,materials,softwareprograms,

gages,tooling,equipment,drawings,specificationsetc.)§ Provisionforre-verification(requirementsnotachieved,identifiednonconformance,selectionofalternativeitem)

• Retaineddocumentedinformation(inspectionresults,testcertificates,certificatesofconformity,laboratoryreports,gage/equipmenttraceability,methodofmanufacture,acceptanceetc.)

8.5.2 IdentificationandTraceabilityThingstoconsider:• Organizationsuseofsuitablemeanstoidentifyproductsandservicese.g.physicalpartmarking,labeling,tags,barcodes,signage,visual

indicators,partsegregation,laydownareas,storageracks• Identificationoftheactualversusrequiredconfigurationoftheproducts/servicese.g.MRPsystem,BOM,partslist,revisionstatus,

changecontrol• Controlarrangmentsforacceptanceauthoritymediaforexample:

o Signaturese.g.recognizedimpressionalignedtoanauthorizedsignatorylisto Stampcontrole.g.uniqueidentification,stampissue,scopeofauthority,stampwithdrawal,registerofholders,stamplegibilityo Electronicsignaturese.g.userid,passwords,permissions,restrictions,userprofile,scopeofauthority

• Uniqueidentificationwhentraceabilityisarequirement,togetherwithretaineddocumentedinformationtoenable:o Identificationandtraceabilitythroughouttheproductlifee.g.lifecycleplanning,acquisitionofrawmaterials,manufacturing

history,serialization,batchcontrol,configurationstatus,productrelease,productionscrap,maintenancehistory,in-servicehistory,endoflifedisposal

o Traceabilityofassemblies(components,sub-assemblies,modules,finalassembliesetc.),BoM,configurationcontrol,assemblytrees,MRPsystems

o Retrievalofproduction(manufacture,assembly,inspection,testetc.)recordse.g.arrangementsforstorage,whereabouts,location,availability,methodofretrieval,permissions,timeliness

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 45 Revision:2.1

8.5.3 PropertyBelongingtoCustomersofExternalProvidersThingstoconsider:• Organizationsarrangementsrelatingtopropertybelongingtocustomersorexternalprovidersincludingforexamplematerials,

consumables,parts,tooling,equipment,facilities,intellectualproperty(standards,specification,drawings,manuals,data,etc.),returnablepackaging,productsreturnedforwarranty/servicing/investigation

• Controlofpropertybelongingtocustomersorexternalproviderse.g.ownership,identification,managementofinventory,verification,storage,protection,preservation,restrictions,security,segregation

• Retentionofdocumentedinformationandreportingoccurrenceswhenpropertyisfoundtobelost,damagedorunsuitableforusee.g.missing,broken,unserviceable,inventorydiscrepencies,shelflifeexpiration,deterioration,illegibility,obsolescence,misconfiguration

8.5.4 PreservationThingstoconsider:• Organizationapproachtowardspreservingoutputsduringproductionorserviceprovision,includingforexample:

o Identificatione.g.markings,labelling,tags,barcodes,routers,configurationstatus,traceabilityindicatorso Generalhandlinge.g.instructions,equipmentavailability,training,damagepreventiono Contaminationcontrole.g.instructions,preventionofforeignobjects(FOD),protectionarrangements(corrosion,deteroriation,

degradation),damageprevention,preventionofcrosscontamination(materials,consumables,mistakeproofing)o Packaginge.g.instructions,materials/equipmentavailability,training,customerrequirements,FODprevention,re-preservation

arrangements(timelimitations),reusablecontainerso Generalstoragee.g.stockholdingmethods,environmentcontrols(temperature,humidity,cleanliness),stockrotationmethods

suchasfirstinfirstout(FIFO),inventorycontrolchecks,conditionchecks,shelflifecontrol,accessrestrictions,security,housekeeping

o Transmissionortransportatione.g.internal/externalmovements,handling,protection,movementtickets,transportlabels,deliverynotes,specialinstructions,electronictransfers,trackingandtraceability

o Protectione.g.instructions,cleaning,useofinhibitors,preventionoflossordamage,materials/equipmentavailability• Organizationapproachtowardspreservingoutputsinaccordancewithspecificationsandapplicablestatutoryandregulatory

requirements,including:o Cleaninge.g.instructions,sequencing,resourceso Controlofforeignobjects(FOD):

§ Preventione.g.campaigns,signage,briefings,training,bulletins,alerts,riskassessment,controlplans,lessonslearned,

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 46 Revision:2.1

§ Detectione.g.inspection,instrascopes,boroscopes,x-raytechniques,customereyesoverchecks,§ Removale.g.cleaning,purging,disassembly

o Specialhandlingandstorageforsensitiveproductse.g.storagelife,environmentcontrols(temperature,humidity,cleanliness),electrostaticsensitivedevices(ESD),cleanrooms,accessrestrictions,security,segregation

o Marking,labeling,safetywarningsandcautionse.g.identification,instructions,notices,advisory’s,signageo Shelflifecontrolandstockrotatione.g.FIFO,labeling,expirydates,packagingindicators,inventorycontrolchecks,stockcontrol

systemo Specialhandlingandstorageforhazardousmaterialse.g.riskassessment,segregation,accessrestrictions,security,

housekeeping,environmentcontrols(temperature,humidity,cleanliness),personalprotectiveequipment(PPE),recordsofinventory,controlofsubstanceshazardoustohealth(COSHH)

8.5.5 Post-DeliveryActivitiesThingstoconsider:• Organizationdeterminationofpost-deliveryactivitiestakingintoaccount:

o Statutoryandregulatoryrequirementse.g.legislation,regulations,directiveso Undesiredconsequencese.g.riskassessment,impactanalysis,failureinservice,recallnotices,warrantyconsiderationso Product/serviceusee.g.product/servicesupport,lifecyclemanagement,aftersales(maintenance,sparepartsprovision,help

line,callcenters),technicaldocumentationo Customerrequirementse.g.contractualarrangements,installation,commissioning,handover,training,customersupporto Customerfeedbacke.g.surveyresults,compliments,complaints,lessonslearned,voiceofthecustomer,satisfaction

indicators/ratings,returns/rejections,warrantyclaimso Collectionandanalysisofin-servicedatae.g.performance,reliability,engineeringpredictions,lessonslearned,operating

characteristics,inserviceevents,serviceintervals,servicelife,yieldrates,servicelevelagreements,servicecostso Provisionoftechnicaldocumentatione.g.technicalpublications,operatinginstructions,maintenancepolicies,maintenance,

repairandoverhaulmanuals,controlarrangements(creationissue,updates,communication,distribution)o Controlofoff-sitework(installation,commissioning,maintenance,fieldsupportetc.)e.g.liaison,communication,contracts,

orders,resources,instructions,training,qualifiedpersons,recordsofworkaccomplished,workdeviations,controlofassets(tooling,equipment,parts,consumables,data)

o Product/customersupport:§ Queriese.g.pointsofcontact,helpline,callcenters,frequentlyaskedquestions(FAQs)

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 47 Revision:2.1

§ Traininge.g.provision(classroom,e-learning),manuals,exhibits,webinars§ Warrantiese.g.contractualobligations,servicelevelagreements§ Maintenancee.g.manuals,training,serviceintervals,resources,sparepartsprovision,repairschemes,servicebulletins§ Replacementpartse.g.attritionrates,servicehistory,predictedusage,illustratedpartscatalogues,linesidestock

(standardparts),billofmaterial(BoM),partsavailability,leadtimes§ Resourcese.g.people,materials,parts,consumables,information,equipment,financing§ Obsolescencee.g.riskmanagement,sustainability,sourcingstrategies,sourceofsupply,protectthecustomer,MRP

indicators,servicelifeplanning• Organizationarrangementsfordealingwithproblemsdetectedafterdeliverye.g.reactiontotheproblem,containment,correction,

investigation,problemresolution,correctiveaction,readacross,communication,reporting,lessonslearned8.5.6 ControlofChanges

Thingstoconsider:• Organizationapproachtowardsthereviewandcontrolofproduction/servicechangetoensurecontinuingconformitywith

requirementstakingintoaccount:o Reasonforchange(plannedandunplanned)o Temporaryorpermanentchangeo Review,assessmentandevaluationofthechangeproposalo Verificationorvalidationofthechangepriortoimplementationo Approvalofthechangebytheappropriateauthority(includingthecustomer/regulatorwhererequired)o Implementationofthechange(e.g.implementationplan,reviseddocumentedinformation,communication)o Confirmationthatthechangehasbeenincorporated,includingupdatestotheQMSasrequiredo Monitoringofeffectivenessofthechange

• Changestoproductionandserviceprovisioncaninclude,changesto:o Customer/regulatoryrequirementso Sourceandprocessmethodso Softwareusedtocontroloperationalactivity(e.g.machinetool/inspection/testprograms)o Drawings,diagrams,illustrationso Process/workinstructionso Machines,tools,jigs,fixtures,equipment,gages

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 48 Revision:2.1

o Materials,consumableso Maintenanceregimeso Serviceintervalso Authorizedperson(s)o Specialprocessparameterso Enduserdocuments(instructions,technicalpublicationsetc.)o Handling,storage,packaging,preservationrequirementso Operator/endusertrainingo Environmentalcontrols

• Theidentificationoftheperson(s)authorizingthechangesuchas:name,signature,role,useridentification,stampimpression,etc.,togetherwiththescope,delegationsandauthoritystatusrelatingtothechange(s)beingmade.

• Theretentionofdocumentedinformationrelatingtotheauthorizedchangesuchaschangenotices,alterationrequests,coordinationmemos,revisedinstructions,qualityplan,deviationrequests,verification/validationresults,reviewminutes,implementationplan,togetherwithanyactionsarisingfromtheidentifiedchange.

8.6 ReleaseofProductsandServicesThingstoconsider:• Implementationoftheorganizationsplannedarrangementsatappropriatestagestoverifythatproductandservicerequirementshave

beenmet.Plannedarrangementscanincludedesignverification/validation(modelling,simulations,experiments,trials,prototypes,functionaltesting,performancetesting),inspection(stage,final,firstarticle),examination(destructive/non-destructive),ProductionPartApprovalProcess(PPAP),customeracceptancetesting,productcertification/qualification,thirdpartyqualification(regulator,society,independentbodyetc.),

• Ensuringthattheplannedarrangementshavebeensatisfactorilycompletedpriortoreleaseoftheproductorservice,orobtainingapprovalfromtherelevant(internal/external)authorityorcustomerwhenplannedarrangementshavenotbeencompletedpriortoreleasee.g.concession,deviation,waiver,contractamendment,dispensation,releaseendorsement

• Recordstoprovideevidencethatacceptancecriteriahasbeenmete.g.certificateofconformity,releasecertificate,regulatorycertificate(EASAForm1,FAA8130etc.)

• Traceabilitytotheperson(s)authorizingthereleasesuchasname,authorizedsignatories,useridentification,stampimpressionetc.,includingtheirauthoritystatus(releasesignatory,certifyingstaff,scopeofauthorizationetc.)

• Recordstoprovideevidencethatproductqualificationmeetsdefinedrequirementse.g.inspectionhistory,testreports,

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 49 Revision:2.1

verification/validationreports,acceptancedocuments,qualificationcertificates• Accompanyingdocumentationatthepointofdeliverye.g.certificateofconformity,releasecertificate,regulatorycertificate(EASAForm

1,FAA8130etc.),despatchdocuments(consignmentnotes,deliverynotes,shippingnotesetc.)exportcontrollicences,userinstructions,handlinginstructions,maintenanceinstructions,logbooks,technicalpublications

8.7 ControlofNonconformingOutputs8.7.1 Thingstoconsider:

• Arrangementsforcontrollingnonconformingoutputs,includingproducts,servicesorprocessesidentifiedinternallyorexternallyincluding:

o Organizationsmaintaineddocumentedinformation(see4.4.2)thatdefinesthenonconformitycontrolprocess(e.g.documentedprocedure)including:

§ Responsibilityandauthorityforthereviewanddispositionofnonconformingoutputse.g.Authorizedsignatories,nonconformancecontrolauthorities,delegatedtechnicalauthorities,MaterialReviewBoard(MRB),customer

§ Theprocessforapprovingpersonswhodispositionnonconformingoutputse.g.organizationrole,knowledge,experience,demonstratedcompetence,qualification,requalification,includingscopeofauthority(type,product,project,serviceetc.)

§ Takingactiontocontaintheeffectofthenonconformitysuchasreaction/responsetimes,impact/riskassessment,stopandfix,readacrosstosameorsimilarproducts,servicesandprocessesthatmaybeimpacted(newproductintroduction,workinprogress,dispositionofservice,stockheldinstorage,productheldbyexternalproviders,productintransit,productalreadydeliveredtothecustomeretc.)

§ Therequirementsfortimelyreportingofthenonconformitytorelevantinterestedpartiesrelatingtoproductsorservicesalreadydeliverede.g.respectivetimescaleforreporting,responsibilityforreporting,methodofreporting(internal/externalcommunication,notifications,correctiveactionreports,recallnotices,safetyalertreports,mandatoryoccurrencereports,servicebulletins,advisory’setc.),acknowledgementofreport(s)

§ Thearrangementsfordefiningcorrectiveactionsfornonconformingoutputsdetectedafterdelivery(see10.2)i.e.reactiontothenonconformity,evaluationofnecessaryaction(s),implementationandmonitoringofidentifiedaction(s),reviewofeffectivenessandsustainmentofaction(s)taken

o Implementationoftheorganizationsdocumentednonconformitycontrolprocess(asabove)toensurethatappropriateactionistakenaccordingtothenatureofthenonconformity

o Organizationprovisionfordealingwithnonconformingoutputsincluding:

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 50 Revision:2.1

§ Correctionse.g.enablerework(correctionlabels,updatedinstructionsetc.)toachieveconformity,includingverificationoftheconformingstatusaftercorrection(checking,inspection,testingetc.)

§ Segregatione.g.physicalseparationofnonconformingproducts,useofdedicatedholdingareas(floorspace,racking,quarantinebonds,cages),identificationtoshownonconformingstatus(labels,markings,tags,annotationofdocumentedinformation,electronicsystem(MRP)updates)

§ Containmente.g.reaction/responsetimes,impact/riskassessment,stopandfix,communication,readacrosstosameorsimilarproducts,servicesandprocesses

§ Returnorsuspensionofoutputse.g.,stopshipment,suspendservice,productrecall,freezeinventory,blockstock,holdinstruction

§ Informingthecustomere.g.communication,notifications,alerts,bulletins,advisory’s§ Acceptanceunderauthorizedconcession(includingthecustomerwhenrequired)e.g.concession,waiver,deviation,

productionpermit,servicealleviationo Implementationofuse-as-is(accepteddeviationasidentified),orrepair(acceptedrestorationtoauseablecondition)in

accordancewithadisposition(e.g.concession,waiver,deviation):§ Approvedbyanauthorizeddesignresponsiblerepresentativewithintheorganization,or§ Approvedbyperson(s)withappropriatedelegatedauthority,and§ Authorizedbythecustomerifthenonconformityhasanimpactonthecontractrequirements

o Controlofproductsdispositionedasscrapincluding:§ Conspicuousmarking(e.g.scraplabels,tags,signs,paint)andpermanentmarkings(e.g.removalofproductidentity,

applyingscrapidentificationstatus),or§ Positivecontrol(e.g.segregation,dedicatedholdingareas(floorspace,racking,quarantinebonds,cagesetc.),annotation

ofdocumentedinformation,electronicsystem(MRP)updates§ Physicallyrenderedunusablebytheorganizationoracontrolledexternalprovider(e.g.productdestruction,removalof

keyfeatures,productdeformation,productmutilation,conversionbacktorawmaterial(revert,recycling)andcontrolofanyinherenthazadousmaterial/substances(asrequired)

o Controlofcounterfeitorsuspectcounterfeitparts(see8.1.4),topreventre-entryintothesupplychaine.g.labeling,marking,segregation,quarantine,containment,disposition,annotationofdocumentedinformation,electronicsystem(MRP)updates

8.7.2 Thingstoconsider:• Organizationarrangmentsforretainingdocumentedinformationthat:

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 51 Revision:2.1

o Describesthenonconformitye.g.statements,illustrations,reports,objectiveevidenceo Describesactionstakene.g.dispositionsrelatingtocorrection,concession,scrapo Describesanyconcessionsobtainede.g.acceptedconcession,waiver,deviation,productionpermit,servicealleviationo Identifiesthedecidingauthoritye.g.authorizedsignatories,nonconformancecontrolauthorities,delegatedtechnical

authorities,MaterialReviewBoard(MRB),customer

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 52 Revision:2.1

Clause# Considerations9. PERFORMANCEEVALUATION9.1 Monitoring,Measurement,AnalysisandEvaluation

9.1.1

GeneralThingstoconsider:• Identificationof“what”needstobemonitoredandmeasured,forexample:

o QMSandprocessperformancee.g.achievementofqualityobjectives,processperformanceindicators,targetsetting,auditresults(conformity/non-conformity,effectiveness),trends

o Operationalperformancee.g.product/serviceconformity,productionyield,rightfirsttime,partspermillion(PPM),defectsperunit(DPU),productcharacteristics,processparameters,processcapability,identificationofwaste,ontimedelivery,queuingtimes,resolutionofcustomerissues,serviceavailability,in-servicereliability,averageservicetime

o Deliveryofprojectstoplane.g.spendagainstbudget,milestoneachievemento Externalproviderperformance,e.g.product/serviceconformity,ontimedelivery,approvalratings,returns/rejections,invoice

querieso Customerperformancee.g.surveyresults,satisfactionindicators/ratings,compliments,complaintprofile,returns/rejections,

warrantyclaims,invoicequerieso Managementofriskandopportunitiestopreventandreduceundesiredeffects,enhancedesiredeffectsandachieve

improvement,e.g.identification,assessment,treatment• Determinationofthe“methods”usedformonitoring,measurement,analysisandevaluation,forexample:

o Datacaptureinstructions,e.g.procedures,qualityplans,datacollectionsheets,dataacquisitionsoftware,samplingtechniques,frequency

o Statisticalprocesscontrol(SPC)includingruncharts,histograms,controlcharts,Paretodiagrams,scatterdiagrams,causeandeffect(fishbone)diagrams,glyph/radarcharts,designofexperiments

o Projectearnedvalue,includingcostperformanceindex(CPI)andscheduleperformanceindex(SPI)o Managementofrisk,e.g.riskavoidance,riskelimination,takingrisk,risktreatment,risktransferandtakingrisktopursue

opportunitieso Performancegovernance,e.g.managementreview,processreviews/councils,performancereports,scorecards,flight

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 53 Revision:2.1

decks/dashboards

9.1.2

CustomerSatisfactionThingstoconsider:• Determinationofthe“methods”usedforobtaining,monitoringandreviewingcustomerperception,forexample:

o Customersurveys/questionnairesconductedbytheorganizationitselforviaanindependentresearchagency.NOTE:surveysmayrelatetoallcustomersorarepresentativesampleandthefrequencymayvaryaccordingtotherelationship

o Customerfeedbackrelatingtodeliveredproductsandservices,e.g.product/serviceconformity,ontimedelivery,reportcards,satisfactionindicators/ratings,compliments

o Customercommunication,e.g.facetofacemeetings,telephonecalls,useoffieldoperatives(installation/servicepersonnel),daytodayenquires(viainternalpersonnel),useofsocialmedia(websites,messageboards,on-linehelp),reportsfromdealers,distributors,integrators

o Complaintprofileincludingrejections,returns,warrantyclaims,invoicequeries,correctiveactionrequests,o Marketshareanalysisandevaluationofrepeatbusiness

• Arrangementsfordevelopingandimplementingplanstoimprovecustomersatisfactionincluding:analysisofresultsandtrends,inputtomanagementreview,addressingidentifiedproduct/servicedeficiencies(includingimprovementstotheQMS),assessingtheeffectivenessofactiontaken

9.1.3

AnalysisandEvaluationThingstoconsider:• Analysisofappropriatedataarisingfrommonitoringandmeasurement(see9.1.1)inordertoevaluate:

o QMSandprocessperformancee.g.achievementofqualityobjectives,processperformanceindicators,targetsetting,auditresults(conformity/non-conformity,effectiveness),trends

o Operationalperformance,e.g.product/serviceconformity,productionyield,rightfirsttime,partspermillion(PPM),defectsperunit(DPU),productcharacteristics,processparameters,processcapability,identificationofwaste,ontimedelivery,queuingtimes,resolutionofcustomerissues,serviceavailability,in-servicereliability,averageservicetime

o Externalproviderperformance,e.g.product/serviceconformity,ontimedelivery,approvalratings,returns/rejections,invoicequeries

o Customerperformance,e.g.surveyresults,satisfactionindicators/ratings,compliments,complaintprofile,returns/rejections,warrantyclaims,invoicequeries

o Ifplanninghasbeenimplementedeffectively,e.g.actionstoaddressrisksandopportunities,achievementofqualityobjectives,

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 54 Revision:2.1

planningofchangesandoperationalplanningo Informationonproductandserviceproblemsreportedbyexternalsources,e.g.alerts,advisories,directives,servicebulletinso Deliveryofprojectstoplan,e.g.spendagainstbudget,milestonedelivery,earnedvalue,costperformanceindex(CPI),schedule

performanceindex(SPI)o Effectivenessofactionstakentoaddressrisksandopportunities,e.g.riskassessment,risktreatment,residualriskvalues,

performancetrends,adoptionofnewpractices,launchingnewproducts/services,winningnewcustomers• Usingthedocumentedoutputofanalysisandevaluation,e.g.trendanalysisandreportsasaninputtomanagementreview(see9.3.2)

toenabletheorganizationtodeterminetheperformanceandeffectivenessoftheQMSandidentifyimprovementsNOTE:Analysisofthedatacanbeundertakenusingstatisticaltechniquestohelpbreakdowntheinformationandpresentitinaformatthatallowsadeterminationtobemadeofwhetheractionsareneeded.

9.2 InternalAudit

9.2.1

Thingstoconsider:• OrganizationarrangementsforconductinginternalauditsatplannedintervalstodemonstratethattheQMSconformsto:

o Theorganizationsownrequirements,e.g.policies,processes,procedures,instructions,specificationso Customerrequirements,e.g.floweddownbycontracto Statutoryandregulatoryrequirements,e.g.legislation,regulations,directiveso Applicableexternalstandards(including9100)

• OrganizationapproachtoknowinghowwelltheQMShasbeeneffectivelyimplementedandmaintainedforexampleby:o Obtainingauditresultso Reportingauditperformanceo Monitoringtrends,e.g.processstrengthsandweakness,repeatauditfindings,acknowledgedimprovement

9.2.2

Thingstoconsider:• Publicationofanauditprogram,oranumberofauditprogramsdependantuponthesizeandcomplexityoftheorganization,including

theidentification(title,scope,criteria)andfrequencyofeachaudit,e.g.monthly,quarterly,annually• Organizationmethodsusedforundertakingauditsforexampleauditselection,auditplanning,auditconduct,auditreporting,corrective

action,closeout,processmeasures,e.g.auditdeliveryagainsttheschedule,timetorespondtoanonconformity,timetocorrectanonconformity

• Applicationofriskbasedthinkingtoselectauditsandtheirfrequencyforexampleconsiderationof:

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 55 Revision:2.1

o Processesthatarecriticaltoproductandservicequalityo Complexprocessesthatrequireclosemonitoringandcontroltoensureconformityo Balanceacrossoperationalandnonoperationalprocesseso Processesthatutilizequalifiedpersonnelo Activitiesorprocessesthatoccuracrossmultiplelocationso Processesimpactedbyhumanfactorso Introductionofneworchangedprocesseso Changesaffectingtheorganizationo Statutoryandregulatoryissueso Processperformance,e.g.processconformity/nonconformity,escapestothecustomer,complaints,previousinternal/external

auditresults,identifiedrisk(see6.1and8.1.1)• Definitionofthecriteria,(policy,process,procedure,requirements)andscope(extent,boundaries,physicallocation,productline,

facility,department,activities,duration)foreachidentifiedaudit• Managementcommitment,authorizationandcontroloftheauditprogram(s)frominitialissue,togetherwiththeidentificationand

reason(s)forchange• Conductingofauditsontimeinaccordancewiththeschedule(previousandcurrentperiods)• Auditorresourcesincludingtraining/qualificationandselectionofauditorstoensureindependenceofthearea/processbeingaudited• Collectionofauditevidenceandreportingofauditfindingsincludingcorrectgradingoffindings(conformity,nonconformity,strengths,

opportunitiesforimprovement)• Routinereportingofauditresultstotherelevantmanagement(circulationofauditreport,closingmeeting/outbriefs,auditperformance

metrics),includinganinputtoManagementReview(see9.3.2)• Timelyundertakingofcorrections(fixtheissue)andcorrectiveactions(preventrecurrence)bytheorganization• Followupbytheauditortoensureeffectiveandtimelyimplementationofcorrectionandcorrectiveaction• Retentionofdocumentedinformationtoshowthatthattheauditprogramhasbeeneffectivelyimplemented(auditprogram,audit

plans,auditreports,questionnaires,auditevidence,correctiveaction,auditcloseout,auditorqualification,auditperformancemetrics)NOTE:Auditinformationcanbeinhardcopyformat,ormanagedwithinanITsystem

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 56 Revision:2.1

9.3 ManagementReview

9.3.1

GeneralThingstoconsider:• TopmanagementarrangementsforreviewingtheQMSatplannedintervalstoensurecontinuing:

o Suitability(fitforpurpose)o Adequacy(meetstheneedsoftheorganization)o Effectiveness(achievesintendedresults)

• Frequencyofreview,e.g.monthly,quarterly,sixmonthly,annual• Standalonerevieworcombinedwithotherbusinessactivities,e.g.strategicplanning,businessplanning,operationsmeetings,process

reviews/councils,functionalreviews• Representationatthereview,e.g.topmanagement,functionalmanagement,linemanagement,processowners,processchampions,

leadprocessusers,actionowners

9.3.2

ManagementReviewInputsThingstoconsider:• Statusofactions(open/closed)frompreviousmeeting(s),ageingprofileofopenactions,e.g.3months,6months,12months,greater

than1year• Changesarisingfrommonitoringinternal/externalissuesthatarerelevanttotheQMS(see4.1)• Identificationandevaluationofchangestointernal/externalrequirements,e.g.policies,processes,procedures,methods,instructions,

contracts,regulation,legislation,thatimpacttheQMS• QMSperformanceandeffectivenessincluding:

o Customersatisfaction(see9.1.2)andfeedbackfromotherinterestedparties,e.g.reportcards,indicators,ratings,complaints,compliments,mediareports

o Achievementofqualityobjectives(see6.2),includingstatusofplannedversusactualachievemento Organizationandexternalproviderprocessperformanceandproduct/serviceconformity(see4.4,8.4and8.6),e.g.flightdecks,

dashboards,scorecards,performanceindicators,performancetrends,rightfirsttime,ontimedelivery,escapestothecustomer,complaintprofile,returns/rejections

o Nonconformityandcorrectiveaction(see10.2)e.g.Paretoofnonconformitybytype,process,area,rootcauseetc.,readacrosstootherpartsoftheorganizationortoexternalparties,updateofrisks(see6.1and8.1.1),statusofcorrectiveactionimplementation

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 57 Revision:2.1

o Auditresults(see9.2),e.g.achievementoftheauditprogram(s),areasofgoodpractice,nonconformityprofile(number,type,process,area,significance),statusofcorrectiveaction,auditcloseout,externalauditfindings

o Adequacyofinternalresourcesandexternalproviders(see7.1)includingpeople(number,roles,competencyetc.),infrastructure(buildings,equipment,systems,transportetc.),workingenvironment(physicalandhumanfactors,monitoringandmeasuringequipment(availability,fitforpurpose,maintained)

o Effectivenessofactionstakentoaddressrisksandopportunities(see6.1),e.g.riskprofile,riskregister(s),statusofopen/closedactions,ageingprofileofopenactions,evaluationofeffectiveness(enhancedesirableeffects,prevent,reduceundesiredeffects,demonstratedimprovement)

o Identificationofopportunitiesforimprovement(see10.1),correctiveaction,goodpractice,bestpractice,innovation,lessonslearnedetc.,includingreadacrosstheorganizationandtoexternalproviders

9.3.3

ManagementReviewOutputsThingstoconsider:• Decisionsandactionsrelatingto:

o Opportunitiesforimprovement(see10.1)e.g.readacrossandimplementationofcorrectiveaction,goodpractice,bestpractice,innovation,lessonslearnedetc.

o ChangestotheQMS(see6.3),e.g.policies,processes,procedures,methods,instructionso Resourceneeds(see7.1),e.g.people(number,roles,training),infrastructure(buildings,equipment,systems,transport),

workingenvironment(physicalandhumanfactors),monitoringandmeasuringequipment(availability,fitforpurpose,maintained)

o Identifiedrisks(see6.1and8.1.1),e.g.addingnewrisksatanorganizationoroperationalleveltotheappropriateregister(s)andtheassignmentofresponsibilityandtreatment

• Documentedevidenceofthereview,e.g.attendance,agenda,presentations,meetingminutes,actions(list,owners,timescale),reports• Retentionofdocumentedinformationofthereviewandcommunicationoftherelevantcontentacrosstheorganization(see7.4)

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 58 Revision:2.1

Clause# Considerations10. IMPROVEMENT

10.1

GeneralThingstoconsider:• Organizationarrangementsforidentifyingopportunitiesforimprovementandtakingactioninorderto:

o Achieveintendedresultse.g.usinganalysisandtrendingoforganizationalperformance(see9.1.3)o Meetcustomerrequirementsandexpectationse.g.understandingcontractualarrangementsandimpliedcustomer

expectationso Enhancecustomersatisfactione.g.usingsatisfactionindicators/ratings,complaintsprofile,returns/rejections,warrantyclaims

(see9.1.2)• Organizationapproachtowardsundertakingimprovement,forexample:

o Focusonimprovingproductsandservices,includingfutureneedsandexpectations,e.g.investmentinlatesttechnologyandinnovation,improvingreliability,improvingyield,reducingcost,improvingon-timedelivery

o Correcting,preventingandreducingundesiredeffects,e.g.investigatingrootcause,preventingescapestothecustomer,actingoncustomerfeedbackand/orinservicereports

o ImprovingtheperformanceandeffectivenessoftheQMS,e.g.actingonprocessperformanceresults,understandingauditfindings,reducingwaste,processre-engineering,structuralre-organization,promotingbreakthroughprojects

10.2 NonconfomityandCorrectiveAction

10.2.1

Thingstoconsider:• Organizationsmaintaineddocumentedinformation(see4.4.2)thatdefinesthenonconfomityandcorrectiveactionmangement

processes,e.g.documentedprocedure• Organizationsreactiontononconformitieswhentheyarisefromsourcessuchasnonconformingproduct(identifiedduring

manufactureorpostdelivery),customercomplaints,auditfindings,warrantyclaims,etc.including:o Takingactiontocontrolandcorrectthenonconformity,e.g.understandingtheproblem,applyingimmediatecontainment

(protectthecustomer),undertakingcorrectionandcorrectiveaction.o Dealingwiththeconsequences,e.g.communicationwithintheorganization,contactinginterestedpartiesifrequired

(customers,regulators,externalproviders),applyingriskassessment

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 59 Revision:2.1

• Organizationsevaluationoftheneedtotakeactiontoeliminatethecauseofthenonconfomitytoensureitdoesn’texistelsewhere

andtopreventrecurrenceby:o Reviewingandanalyzingthenonconformitye.g.describingtheproblem(who,what,when,where,how,howmany),assigning

aproblemowner,launchinganinvestigation,collectinginformation,engagementwithotherswithproduct/processknowldegeo Usingappropriatemethodologies,e.g.rootcauseanalyisis,5Whys,Ishikawa(causeandeffectdiagram),Paretocharts,

processflowdiagram,8disciplinesofproblemsolving(8d),9stepstoproblemsolving(9s),FailureModeandEffectAnalysis(FMEA)etc.todeterminethecause(s)ofthenonconformity(includingthoserelatedtoHumanFactors)andextentoftheaction(s)needed,baseduponthepotentialeffectofthenonconformity

o Determiningifsimilarnonconformitiesexistorcouldoccurelsewheree.g.communicationwithinandoutsidetheorganization(asappropriate),verificationandquarantine(ifrequired)ofotherproductsthatcouldpotentiallybeimpacted(workinprogress,productheldbyexternalproviders,stockheldinstorage,productalreadydeliveredtothecustomer,productintransit,productrecall)

• Implementationandmonitoringofidentifiedaction(s)throughtheuseofactionplan(s),owner(s),resources,tasks,timescales,managementreviewetc.toensure:

o Timelycorrectiveactionistakenwithintheorganizationandanyexternalprovider(s)wherenecessaryo Furtheractionistakenwhentimelyandeffectivecorrectiveactionisnotachieved

• Reviewingtheeffectivenessofcorrectiveaction(s)taken,e.g.observingtheperformanceoftheprocess,reviewingdocumentedinformation,conductingcompliancechecks,orauditstoensure:

o Therootcausewasaddressedo Thatnootherissueswerecausedbythecorrectiveaction

NOTE:Establishingtheeffectivenessofcorrectiveactionmaytakeanappropriateamountoftimeaftercorrectiveactionhasbeenimplemented,toensurethatithasbeensustained• Conductingafollowuptoensuresustainment:

o Planningisupdatedifnecessary(see6.1)tocaptureanyidentifiedrisksandopportunitiesthatwerenotpreviouslyconsideredo ChangesareincorporatedintotheQMSifnecessary,(processdocuments,procedures,workinstructions,controlplan,service

plan,drawings,resourcesetc.)10.2.2 Thingstoconsider:

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 60 Revision:2.1

• Organizationarrangmentsforretainingdocumentedinformationtoshowthe:o Natureofthenonconfomities,e.g.source,type,quantity,frequency,severityo Action(s)taken,e.g.correctiveactiondocuments,actiontrackers,managementreviewoutputs,lessonslearned,riskregisters,

on-linesystemso Resultsofcorrectiveaction,e.g.openversusclosed,achievementofmilestones,performancemeasures(timetoclose,

problemrecurrence,costofnonquality)

10.3

ContinualImprovementThingstoconsider:• Organizationarrangementsforcontinuallyimprovingthesuitability,adequacyandeffectivenessoftheQMSconsideringthe:

o ResultsofAnalysisandEvaluation(see9.1.3)o OutputfromManagementReview(see9.3.3)

• Determinationofimprovementactivitybasedupon:o Externaldrivers,e.g.regulatorychanges,legislation,marketexpectations,competitoractivity,interestedpartyperceptions,

customersatisfaction,environmentalimpact,benchmarkingofbestpracticeo Internaldrivers,e.g.businessstrategy,achievementofobjectives,problemresolution,business/processperformance,process

capability,costreduction,reduceleadtime,right-first-time,on-timedelivery,improvedefficiency,lessonslearned,reducevariation,riskreduction,operationalbudgets,priortization,employeesuggestions

• Useofappropriatemethodologiesandtoolstoundertakecontinualimprovementincludingforexample:o Plan,Do,Check,Act(PDCA)approachtocontrolandcontinuouslyimprovemeprocessesandproducts:

§ Plan:definetheproblem,collectrelevantinformation/data,understandcurrentperformance,determinetherootcause,setimprovementobjectives/target,decideontimescale,determineresourcesandbudget

§ Do:developandimplementasolution,introduceaformofmeasurementtodeterminetheeffectivenessofthesolution,monitorprogress,reviewagainsttheplanandmakeanynecessaryadjustments

§ Check:measureperformance,compareoutcomeagainstexpectations,verifytoconfirmexpectedresultshavebeenachieved

§ Act:documenttheresults,communicatetheoutcome,makerecommendationsbasedonresultsachieved,assessandrepeatthecycleifrequired,updatethesystemsanddocumentationtosustaintheimprovement

o Leanapproachtoimprovecycletime,optimizeresourcesandeliminationofwasteusingsuchtechniquesas:workplaceorganization(5S),visualfactory,valuestreammapping,kaizenactivity,OverallEquipmentEffectiveness(OEE),TotalProductive

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 61 Revision:2.1

Maintenance(TPM),takttime,Just-in-Time(JiT),mistakeproofingetc.o Six-Sigmaapproachtoimproveprocesscapability,removingcausesofdefectsandreducevariationusingsuchtechniquesas:

theDefine,Measure,Analyze,Improve,Control(DMAIC)model,Define,Measure,Analyze,Design,Verify(DMADV)model,processmapping,Supplier,Input,Process,Output,Customer(SIPOC)diagram,causeandeffect(fishbone)diagrams,MeasurementSystemsAnalysis(MSA),gageR&R,runcharts,histograms,controlcharts,Paretodiagrams,scatterdiagrams,,glyph/radarcharts,DesignofExperiments(DoE),FailureModeandEffectsAnalysis(FMEA)etc.

• Organizationarrangementsformonitoringtheimplementationofimprovementactivitiesandtheeffectivenessoftheresults,e.g.dashboards,flightdecks,registers,milestonecharts,actiontrackers,performanceindexesandtrends,benefitscapture,on-linesystems,formalreporting,inputtomanagementreview

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 62 Revision:2.1

4.0 ChangeControl

Revision Detail Date1.0 Initialissuetosupportimplementationof9100:2016includingsections1,2and3(clauses4,5,6,9,

10)15Dec2016

1.1 Section3updatedwithnewcontentforclauses7.1,7.2,7.3,7.4,7.5,8.1,8.2and8.3. 13Apr20172.0 Section3updatedwithnewcontentforclauses8.4,8.5,8.6,8.7,contentpageaddedandwhole

documentupdatedtolandscapeversion20Jul2017

2.1 Documenttitlechangedto‘EvaluationGuidanceMaterial’andIAQGcopywritestatementadded 17Oct2017