as/en/jisq 9100:2016 evaluation guidance material · 2019-10-02 · as/en/jisq 9100 2016 (rev d)...

62
© 2017 IAQG All rights reserved AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 1 Revision: 2.1 AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL Prepared by the IAQG 9101 Writing Team

Upload: others

Post on 27-Jun-2020

35 views

Category:

Documents


5 download

TRANSCRIPT

Page 1: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 1 Revision:2.1

AS/EN/JISQ9100:2016EVALUATIONGUIDANCEMATERIAL

PreparedbytheIAQG9101WritingTeam

Page 2: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 2 Revision:2.1

ContentsPage# Section Title

3 1.0 Introduction4 2.0 Processauditingapproach4 2.1 Generalapproach5 2.2 Specificapproaches8

3.0

Auditconsiderations8 Clause4-ContextOfTheOrganization12 Clause5–Leadership15 Clause6–Planning18 Clause7–Support24 Clause8–Operation48 Clause9-PerformanceEvaluation54 Clause10-Improvement58 4.0 Changecontrol

Page 3: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 3 Revision:2.1

1.0 IntroductionThisdocumentprovidesgeneralguidanceforusebyauditteamswhenexecutingtheauditprocessdescribedbythe9101:2016standard,whenconducting9100:2016audits.

Thisguidanceisnotintendedtoaddto,subtractfrom,orinanywaymodifythestatedrequirements,buttoprovideexamplesandthoughtstimulationi.e.“thingstoconsider”,whenaskingquestionsandidentifyingobjectiveevidence.

NOTES:• Theguidancecontainedwithinthisdocumentcanalsobeusefulwhenpreparingandplanninganauditduringthecertificationcycle.

• Acceptablemeansofcompliancearenotlimitedtothoseitemslistedinthisdocument.

• Anyissuesidentifiedduringauditsaretobedocumentedagainst9100:2016requirements.

• 9100:2016clause0.3describestheprocessapproachdeveloping,implementing,andimprovingtheeffectivenessofaQualityManagementSystem

• 9101:2016clause0.2describestheprocessapproachwhenevaluatinganorganizationsQMS

• FurtherISOguidanceisalsoavailableasfollows:o Processapproachformanagementsystems-ISO/TC176/SC2/N544R3o ApplicationofISO9001:2015managementsystem-ISOTechnicalSpecificationISO/DTS9002

• Thisdocumentdoesnotprovideguidancefortheadditionalrequirementsdefinedby9110and9120.

Page 4: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 4 Revision:2.1

2.0 ProcessAuditingApproach2.1 GeneralApproachWhenevaluatinganorganization'sQMS,therearebasicquestionsthatshouldbeaskedofeveryprocess,forexample:• Istheprocessappropriatelydetermined?• Istheprocessidentifiedandappropriatelydefined,includingforexampleprocessinputs,outputs,resourcesandcontrols?• AreProcessOwnerandProcessUserresponsibilitiesassigned?• Istheprocessimplementedandmaintained?• Istheprocesseffectiveinachievingthedesiredresults?Otherquestionscouldinclude:• Whatistheprocess?• Whatisittryingtoachieve?• Whoisthecustomeroftheprocess?• Doestheprocessaddressapplicablecustomerspecificrequirements?• AreProcessUsercompetenciesidentified?• Istheprocessoperating,asdefined?• Whatisthedesiredlevelofperformance?• Haveprocessperformancetargetsbeenestablished,includingthosespecifiedbythecustomer?• Haveprocessmeasures(e.g.KPIs)beenestablishedandaremechanismsinplacetocollateandanalyzethedata?• Whatisthecurrentlevelofperformance?• Isprocessperformanceregularlyreviewedbytopmanagement?• Whereperformanceisnotbeingachieved,areimprovementplansinplace?

Page 5: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 5 Revision:2.1

2.2 SpecificApproaches

Thereareanumberofspecificauditapproachesthatcanbebuilt intotheauditplantoprovidefocusonkeyelementsoftheQMS, includingforexample:

2.2.1 CustomerFocus

The audit team should determine that customer satisfaction is being evaluated and appropriate actions are takenby the organization basedonavailable performance information (e.g., nonconformity data, corrective action requests, results of satisfaction surveys, complaints regardingproduct quality, OTD, service provision, responsiveness to customer and internal requests) provided by the organization's customers (e.g.,scorecards,reportcards).

2.2.2 OrganizationalLeadership

TheInterviewwithtopmanagementshouldbeusedtoevaluate:• Establishmentandcontinuedrelevanceoftheorganization'squalitypolicyandobjectives;• Establishmentofperformancemeasuresalignedtoqualityobjectives;• QMSdevelopment,implementation,andcontinualimprovement;• Topmanagementcommitment;• QMSperformanceandeffectiveness;• Performancetocustomerexpectations(e.g.,supplierrating,scorecard,auditresults);and• Actionstakentoaddressissuesthatarenotmeetingcustomerperformanceexpectations.

2.2.3 QMSPerformanceandEffectiveness

TheauditofQMSperformanceandeffectivenessshouldincludeareviewofthefollowing:• Processingofcustomercomplaints,customerfeedbackdata(e.g.,periodicperformancereportsreceivedfromcustomers),andotherrelevant

customerdata(e.g.,resultsofcustomersurveys);

Page 6: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 6 Revision:2.1

• ResultsandactionsfrominternalandexternalauditsoftheQMS,includingtheirassociatedrecords;• Stakeholderfeedback(e.g.,feedbackfromregulatoryauthoritiesorotherinterestedparties);• Processingofprocess/productnonconformities,includingreviewofassociatedcorrectiveactionsandevaluationontheeffectivenessofactions

taken;• Processingofpreventiveactions,includingevaluationontheeffectivenessofactionstaken;• Managementreviewconduct,includingassociatedrecords(e.g.,processinputs/outputs,actionstaken);• Internalperformancemonitoring,measurement,reporting,andreviewsagainststakeholderandinternalperformanceobjectivesandtargets,

includingcontinualimprovementactivitiesandassociatedrecords;• Organization’scurrentperformanceagainsttargets,includingcustomerspecifictargetsandassociatedrecordsofapplicableactionstakenwhere

targetsarenotbeingmet;and• Thestatusandeffectivenessoftheorganization'sprocessperformanceimprovementactivitiesandtheiroutcomesrelatedtoproductquality.

2.2.4 ProcessManagement

TheauditteamshouldconductQMSauditsusingamethodthatfocusesonprocessperformanceandeffectiveness;thisensuresthatpriorityisgiventothefollowing:• Reviewingtheorganization'sprocesses,theirsequenceandinteractions,theidentificationoffunctionsandassignmentofresponsibilities,and

performanceagainstrequirementsanddefinedmeasures,withfocusonprocessesthatdirectlyimpactthecustomer;• Reviewingtheprocessforvalidationandapprovalofprocessesandprocesschanges;• Reviewingtheavailabilityofresourcesandinformationrequiredtooperateandsupportassociatedactivities,includingappropriatetrainingand

competencyofpersonnel;• Reviewingtheprocess-basedmanagementtechniques,includingtheexaminationofprocessmeasures(e.g.,quality,tacttime,cycletime,output

effectiveness,controllimits,processcapabilitydetermination);• Reviewingplansinplacetoensureperformanceobjectives/targetsaremonitored,measured,andanalyzedinordertorealizetheplanned

activitiesandachievetheplannedresults(e.g.,verifyperformanceinformationavailability,percentageofnonconformingparts/products,percentageOTD);

Page 7: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 7 Revision:2.1

• Reviewingapplicableactiontakenwhenobjectives/targetsarenotmettopromotecontinualimprovement;and• Pursuingaudittrailsaddressingcustomerconcernsorrequestsforcorrectiveactions,performanceagainstobjectives,andrelevantprocess

controls.NOTE:Theaudit team should audit processes to sufficientdepth anddetail to evaluate if theorganization's processes are capableofmeetingplannedresultsandperformancelevels,includingapplicablecustomerspecifictargets.

2.2.5 ContinualImprovement

Theauditteamshouldevaluatetheorganization'sinterrelatedprocessesandactivitiesforcontinualimprovementoftheQMS,itsprocesses,theirconformity,andeffectivenessinorderto:

• Ensurefocusonissuesthatareimportanttotheorganization,theircustomers,andregulatoryauthorities;and

• Determinetheeffectivenessofanorganization'sapproachtocontinuallyimprovingprocessperformance.

NOTE:The organization should be able to demonstrate that they have a structured approach to achieve continual improvement of the QMS and itsprocesses.

Page 8: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 8 Revision:2.1

3.AuditConsiderationsThissectionlooksattherequirementsof9100:2016byclausenumberandprovidesexamplesoftypicalevidencethatcanbesoughtandconsideredbytheauditteamduringexecutionoftheauditplan.

Clause# Considerations4. CONTEXTOFTHEORGANIZATION

4.1

UnderstandingtheOrganizationanditsContextThingstoconsider:• Policystatement(s)regardingtheorganizationspurposeandstrategicdirection• Individualstrategyortacticalplandocumentswrittentounderpintheorganizationspoliciesandprovidearoadmaptoachieveits

futuregoals• Recordsofmeetingswherecontextisroutinelydiscussedandmonitored,e.g.aspartofthestructuredmanagementreviewprocess

orwithineachoftherespectivefunctionoftheorganization(Purchase,HR,Engineering,Sales,Financeetc.)• StructuredriskassessmentofExternalandInternalissues.• UseofPEST(Political,Economic,Social,Technological),PESTLE(Political,Economic,Social,Technological,Legal,Environmental)and

SWOT(Strengths,Weaknesses,Opportunities,Threats)analysistools• Documentedinformationthatdescribesthecontextoftheorganizationthatcouldbeincludedaspartofasinglesourceof

documentedinformation(sometimesreferredtoasaQualityManual)SourcesofinformationrelatingtoExternalandInternalissuescouldinclude:External:• Reportsrelatingtocompetitiveenvironment,newtechnologies,newmarkets,customerexpectations,supplierintelligence,

economicconditions,politicalconsiderations,investmentopportunities,socialfactorsetc.• Identificationoffactorsrelatingtochanginglegislationandregulations,includingenvironmentalimpact• Feedbackrelatingtoproduct/serviceperformanceandlessonslearned• RegisterofidentifiedexternalrisksandtheirtreatmentInternal:• Structureoftheorganization,identificationofrolesandresponsibilitiesandarrangementsforgovernance

Page 9: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 9 Revision:2.1

• Reports’showinghowwelltheorganizationisperforming• Statementsrelatingtotheorganizationsmission,visionandcorevalues• Emphasisplaceduponbusinessethicsandorganizationalcodesofconduct• Feedbackobtainedfromemployees,e.g.employeesurveyresults• Informationsystemsandthemechanismforcapturingandsharingknowledgeandlessonslearned• Organizationalcapabilitystudies,identificationofload/capacityandresourcerequirementstoachievedemand• Registerofidentifiedinternalrisksandtheirtreatment

4.2

UnderstandingtheNeedsandExpectationsofInterestedPartiesThingstoconsider:• Useofbrainstormingtechniquestoidentifyrelevantexternalandinternalinterestedparties,e.g.customers,partners,endusers,externalproviders,owners,shareholders,employees,tradeunions,governmentagencies,regulatoryauthorities,localcommunity

• Useofsurveys,networking,face-to-facemeetings,associationmembership,attendingconferences,lobbying,participationinbenchmarking,etc.,inordertogainstakeholderinformationandtheirrequirements

• Stakeholdermappingtodetermineimportanceusingthefour-quadrantmethod(power/influenceversusinterest/importancegrid).• UseofRACI(Responsible,Accountable,Consulted,Informed)methodtodeterminelevelofstakeholderengagement• FlowdownandcaptureofrequirementsrelevanttotheQMSdefinedincontracts,orders,statementsofwork,termsofbusinessetc.• Recordsofmeetingswhereinterestedpartiesandtheirrequirementsareroutinelydiscussedandmonitored,e.g.aspartofthestructuredmanagementreviewprocess,orwithineachoftherespectivefunctionoftheorganization(Purchase,HR,Engineering,Sales,Financeetc.)

NOTE:TheorganizationdeterminestherelevantinterestedpartiesandtheirrequirementsrelatingtotheactivitiesoftheorganizationandtheimpactontheirQMS

4.3

DeterminingtheScopeoftheQualityManagementSystemThingstoconsider:• Organizationevaluationof:

o Externalandinternalissues(see4.1)o Requirementsofrelevantinterestedparties(see4.2)o Itsproductsandservices

• ConsiderationofboundariesandapplicabilityoftheQMScaninclude:

Page 10: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 10 Revision:2.1

o Rangeofproductsandserviceso Differentsitesandactivitieso Externalprovisionofprocesses,productsandserviceso Commonsupportprovidedbycentralizedfunctionso InfrastructureoftheQMSsuchasdistribution,accessibility,differentlayers(process,procedure,instructions,sitespecific

requirements)• DocumentwrittenandmaintainedbytheorganizationthatdescribesthescopeoftheQMS,including:

o Astatementofthetypesofproductsandservicescoveredo Assessmentoftheapplicabilityofthe9100requirementso Justificationforany9100requirement(s)thatis/aredeterminedas‘notapplicable’

• Alignmentbetweenthedocumentedscopeoftheorganization’sQMSandtheiragreedscopeofcertification4.4 QualityManagementSystemanditsProcesses

4.4.1

Thingstoconsider:• IdentificationoftheprocessesneededfortheQMS,includingtheirsequenceandinteraction,e.g.processframework,process

model,processgroupings,processflowdiagram,processmapping,valuestreammapping,Turtlediagrams,SIPOC(Supplier,Input,Process,Output,Customer)charts,processcards

• Informationneededtoensureeffectiveoperationandcontroloftheprocesses,e.g.definedprocessrequirements(shall),goodpractice(should),definedroles,requiredcompetencies,associatedtraining,guidancematerial

• Expectedinputsandoutputsfromeachoftheidentifiedprocesses,togetherwithassignmentofresponsibilitiesandauthoritiese.g.ProcessOwner,ProcessChampion,LeadProcessUser,ProcessUser.

• Criteriaandmethodtoensureeffectiveoperationandcontroloftheidentifiedprocesses,e.g.processmonitoringindicators,processperformanceindicators,targetsetting,datacollection,performancetrends,internal/externalauditresults

• Arrangementsforgoverningtheprocessese.g.processownerreviews,processcouncils,flightdeck/dashboards,risksandopportunitiesrelatingtotheprocess,resourceneeds,usertraining/competency,continualimprovementinitiatives,frequencyofreviews,agenda,minutes,actions

• Organizationapproachtowardscontinualimprovementandactiontakenwhenprocessperformanceisnotmeetingintendedresults• Captureofcustomer,statutoryandregulatoryrequirementsandthemethodusedtobuildtheseintotheorganizationsQMS,e.g.

requirementscapture,gapanalysis,requirementsembeddedintotheprocessdefinition,assignedcontractassuranceinstructions,formallinkstoinformation,useofspecifieddocumentation

Page 11: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 11 Revision:2.1

4.4.2

Thingstoconsider:• Documentationcreatedandmaintainedbytheorganizationtosupporttheoperationoftheprocesses,e.g.manuals,expositions,

handbooks,documentedprocedures,workinstructions,guidancematerial,datacards,physicalsamples,ITsystems(includingintranetandinternet),universal/bespokesoftware,templates,forms

• Documentationidentifiedandretainedbytheorganizationtoshowthatprocessesarecarrieditasplanned,e.g.physicalhardcopyrecords,electronicmedia(dataservers,harddrives,compactdiscs,flashdrivesetc.)

• Specificdocumentationcreatedandmaintainedbytheorganizationthatincludesadescriptionofrelevantinterestedparties(see4.2),scopeoftheQMSincludingboundariesandapplicability(see4.3),descriptionoftheprocessesneededfortheQMStogetherwiththeirsequence,interactionandapplicationandassignmentofresponsibilitiesfortheprocesses.

NOTE:TheorganizationcancompilethisspecifiedinformationtogetherinasinglerepositoryandmayrefertoitasaQualityManual.

Page 12: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 12 Revision:2.1

Clause# Considerations5. LEADERSHIP5.1 LeadershipandCommitment

5.1.1

GeneralThingstoconsider:• TopmanagementcommitmenttowardstheQMStodemonstratethattheyhaveapresenceintheorganization,providedirection,

leadbyexample,makedecisionsandhave:o TakenaccountabilityfortheeffectivenessoftheQMSe.g.establishedmeasures,system/processperformancemonitoring,

managementreview,realizationofplannedactivities,achievementofplannedresultsandtakingactionwhenprocessperformanceisnotmeetingintendedresults

o Establishedandmaintainthequalitypolicyandobjectivesalignedtothestrategicdirectione.g.contextoftheorganization,external/issues(see4.1)

o Integratedqualityrequirementsintotheorganizationsbusinessprocessese.g.systemarchitecture,businessmodel,processmodel,organizationfootprint,functionalalignment(Engineering,Purchasing,IT,Finance,HRetc.)

o Promotedtheprocessapproachandriskbasedthinkinge.g.processmodeling,processmapping,inputs,outputs,activities,interactions,interfaces,resources,controls,riskmanagement(identification,severity,ownership,treatmentetc.)

o Supportedprocessownersintheirprocessmanagementactivitiese.g.deployment,governance,processevaluation,processimprovement

o Enabledtheresources(includingpeople)requiredforaneffectiveQMSe.g.resourceplanning,workload,priorities,constraints,balance,organizationflexibility,businessbenefits,organizationgrowth

o CommunicatedtheimportanceofconformitytotheQMSandeffectivequalitymanagemente.g.meetings,briefs,e-mail,intranet,campaigns,roadshows,focusedtraining,voiceoftheregulator/customer,consequenceofnonconformity

o Createdanenvironmentforcontinualimprovemente.g.§ Proactive-product/service/processimprovementinitiatives,improvementprojects,wastereduction(lean),process

re-engineering,costreductionetc.§ Reactive-actingonprocessperformanceresults,auditfindings,complaints,escapesetc.

Page 13: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 13 Revision:2.1

o Supportedotherrelevantmanagementrolese.g.organizationhierarchy,trust,empowerment,responsibledelegation,coaching,sharingknowledge,removingbarriers,routetoescalation

5.1.2

CustomerFocusThingstoconsider:• Topmanagementcommitmenttowardscustomerfocustodemonstratethat:

o Externalrequirementsaredeterminedunderstoodandmete.g.contracts,legislation,benchmarking,surveys,customersatisfaction,marketintelligence,futuretrends,customerexpectations

o Risksandopportunities(see6.1)e.g.competition,capability,resourcing,barrierstomarket,investment,businesscontinuity,innovation,futuretrends,newtechnology,newproducts/services,buildingoncurrentstrengthsaredeterminedandaddressed

o Focusonenhancingcustomersatisfaction(see9.1.2)ismaintainede.g.buildingrelationships,conductingsurveys,customerfeedback,customercommunication,customerperformance,complaintprofile,evaluationofrepeatbusinessandidentifyingopportunitiesforstrengtheningtheorganizationreputationandmarketpresence

o Customerperception(determinedbythecustomer)andcustomersatisfaction(measuredbytheorganization)arealignedo Productandserviceconformityandon-timedeliveryperformanceismeasurede.g.definingperformancecriteria,flowdown

acrosstheorganization,settingtargets,datacapture,datareporting,managementreviewo Actionistakenwhenproductandserviceconformityandon-timedeliveryperformanceisnotachievede.g.ownership,

containment,rootcause,correctiveaction,continualimprovement

Page 14: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 14 Revision:2.1

5.2 Policy

5.2.1

EstablishingtheQualityPolicyThingstoconsider:• Topmanagementprovisionforestablishingthequalitypolicyincluding:

o Appropriatenessandalignmenttothecontextoftheorganization(see4.1),e.g.purpose,strategicdirection,mission,vision,ethicalprinciples,businessreputation,corevalues,business/functionalpolicies,codesofconductetc.

o Establishmentofaframeworkforsettingqualityobjectives(see6.2),e.g.businessplanning,corethemes,strategicenablers,keyperformanceindicators(KPIs),milestones,resources

o Commitmenttosatisfyapplicableinternal/externalrequirementse.g.industry,business,customer,statutory,regulatoryandotherinterestedparties(see4.2)

o CommitmenttowardscontinuousimprovementoftheQMSe.g.businessperformance,processmaturity,processeffectiveness,customerexpectations,investment,organizationalgrowth,scopeapplication

• Topmanagementprovisionforimplementingandmaintainingthequalitypolicy(see5.2.2)

5.2.2

CommunicatingtheQualityPolicyThingstoconsider:• Arrangementsforcommunicatingthequalitypolicywithinandoutsideoftheorganizationincluding:

o Availabilityacrosstheorganizationandmethodsusedtodeploye.g.hardcopiesdisplayedinprominentplaces,electroniccopiesviaintranet/internetsites,identificationanddistributiontorelevantinterestedparties

o Awareness,understandingandapplicationacrosstheorganization(see7.3and7.4)e.g.communicationmethods(meetings,briefs,e-mail,websites),respondingtofeedback,providinginterpretation,translationintootherlanguages(asrequired),timingandfrequency

• Maintainingdocumentedinformationtodemonstratethatthequalitypolicyiscontrolled,reviewedperiodicallyforcontinuingsuitability,updatedasrequiredandsubjecttofurthercommunicationaccordinglye.g.ownership,review,validity,applicability,distribution,feedback

Page 15: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 15 Revision:2.1

5.3

OrganizationalRoles,Responsibilities,andAuthoritiesThingstoconsider:• Topmanagementprovisionforassigningrelevantrolesandresponsibilities(i.e.tasksallocatedtoarole)andauthorities(i.e.

permissionsallocatedwithintherole).• Assignmentofrelevantroles,responsibilitiesandauthoritiesacrosstheorganizatione.g.topmanagement,management

representative,functionalleaders,headsofdepartments,processowners,leadprocessusers,endusersetc.relatingto:o ConformanceoftheQMStothe9100standard(see4.3)o Deliveryofprocessoutputresults(see4.4.1)o ReportingofQMSperformanceandimprovementopportunities(see9.3)o Promotingcustomerfocus(see5.1.2)o MaintainingtheintegrityoftheQMSwhenchangesoccur(see6.3)

• Arrangementstodemonstratethatrelevantroles,responsibilitiesandauthoritiesarecommunicatedandunderstood,e.g.organizationchart,resourceallocation,roleprofiles,accountabilitystatements,jobdescriptions,termsofreference,training,competence,qualification,performancereview

• Appointmentofaspecificmanagementrepresentativewhohas:o Responsibilityandauthorityforoversightoftheabovee.g.assignmentofspecificduties,definedresponsibilities,clear

accountabilitieso Organizationalfreedomandunrestrictedaccesstotopmanagementtoresolvequalitymanagementissues,includingliaison

withexternalparties(asappropriate)e.g.organizationalstructure,hierarchy,reportinglines,conflictsofinterest,independence,recognizedauthority

Page 16: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 16 Revision:2.1

Clause# Considerations6. PLANNING6.1 ActionstoAddressRisksandOpportunities

6.1.1

Thingstoconsider:• OrganizationdeterminationoftherisksandopportunitieswhenplanningfortheQMSarisingfrom:

o Externalandinternalissues(see4.1)o Requirementsofrelevantinterestedparties(see4.2)

• Organizationarrangementsforaddressingtheidentifiedrisksandopportunities,includingthosethathaveanimpacton:o TheQMSachievingitsintendedresultse.g.conformitytorequirements,realizingobjectives,meetingperformancetargetso Enhancingthedesirableeffectse.g.developingopportunities,creatingnewpossibilities,exploringnewmarkets,expanding

thecustomerbase,organizationgrowtho Preventingorreducingundesiredeffectse.g.proactiveriskmanagement,focusingonriskreduction,takingpreventive

measureso Achievingimprovemente.g.meetingtargets,removingnon-valueaddedactivity(waste),positiveperformancetrends

6.1.2

Thingstoconsider:• Organizationarrangmentsfor:

o Planningactionstoaddresstheidentifiedrisksandopportunitiestoensurethatappropriateprocesscontrolsareinplacee.g.ownership,documentedinformation,instructions,methods,verificationactivity,processmonitoring,performancemeasuresandtrends,training,competency

o IntegratingandimplementingactionsintotheQMSe.g.captureoflessonslearned,sharinggoodpractice,processreviews,processupdates,changecontrol,communication

o Evaluatingtheeffectivenessoftheactionstakenincluding;managementreview,effectonproductandserviceconformity,achievingplannedactivitiesandplannedresults,monitoringoftrends,effectoncustomersatisfactionetc.

• Organizationapproachtomanagingrisksandopportunitiestakingintoaccount:o Levelofactivityproportionatetothesizeandcomplexityoftheorganizationandtheimpactontheconformityofproducts

andserviceso Applicationofriskmanagementtechniquesincluding;riskmanagementplan,riskidentification,likelihood/impactof

occurrence,severityofoutcome(e.g.high,medium,low),riskownership,risktreatment,residualrisk,continualmonitoring

Page 17: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 17 Revision:2.1

etc.o Takingadvantageofnewopportunitiesbybuildingoncurrentstrengths,anticipatingfuturetrends,introducingnew

technology,developingnewproductsorservices,openingnewmarkets,attractingnewcustomersetc.NOTE:Therearemanytoolsandmethodoligiesthatanorganizationcanadopttohelpmanagerisksandidentifyopportunitiesincluding;learningfromthepast(LessonsLearned),PEST(Political,Economic,Social,Technological),PESTLE(Political,Economic,Social,Technological,Legal,Environmental)SWOT(Strengths,Weaknesses,Opportunities,Threats),FMEA(FailureModesandEffectsAnalysis,brainstormingtechniques,BCM(BusinessContinuityManagement),benchmarkingetc.

6.2 QualityObjectivesandPlanningtoAchieveThem

6.2.1

Thingstoconsider:• Organizationarrangementsforsettingqualityobjectivesatvariouslevelsacrosstheorganizationincluding:

o Objectivesestablishedwithintherelevantfunctions(Engineering,Purchasing,Finance,HumanResources,Quality,ITetc.),linkedtotherespectivefunctionalstrategy,e.g.functionaldirection,processimprovement,milestonesetc.

o Objectivesestablishedwithinrelevantprocessesthatcouldinclude;directiondefinedbytheprocessownerrelatingtotheaimsoftheprocess,e.g.whattheprocessistryingtoachieve,resourcesneeded,timescalesetc.

o Objectivesestablishedandsuitablycascadedatvariouslevelsacrosstheorganizationstructuree.g.topmanagement,middlemanagement,supervisors,departments,projects,groups,individuals

• Maintainingdocumentedinformationtodemonstratethatthequalityobjectivesare:o Consistentwiththequalitypolicy(see5.2.1)toensurethattheyunderpinthestrategicdirectionoftheorganization,

supporttheneedsandexpectationsofinterestedpartiesandenhancecustomersatisfactiono Specificandmeasurableinordertogivecleardirectionastowhatisrequiredandtheexpectedoutcome,oftenreferredto

asSMART(Specific,Measurable,Attainable,RealisticandTime-bound)objectiveso Alignedtoapplicablerequirementse.g.thosedefinedbytheorganization,customerorregulatorandrelevanttothe

conformityofproductsandserviceso Monitoredusingsuitablemeanstoensurethattheobjectivesarebeingmete.g.BusinessPlanDeployment(BPD)charts,

dashboards,matrices,reports,progresscharts,trafficlightcharts,managementreviewo Communicatedattherelevantlevelswithintheorganizationtoensurethatteamsandindividualsareawareoftheir

importanceandcontributiono Updatedaccordinglytodemonstrateprogressandtotakeaccountofchangingcircumstancesthatcouldresultinnew,

Page 18: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 18 Revision:2.1

expanded,amended,cancelledobjectivesetc.

6.2.2

Thingstoconsider:• Organizationplanningtoachieveitsestablishedqualityobjectivesincludingthedeterminationof:

o Whatwillbedonee.g.clearlyestablishedobjectivesacrosstheorganizationatvariouslevels(functional,process,department,team,individual)thataremeasurable-see6.2.1.

o Resourcesrequiredtodelivertheobjectivese.g.numberandcompetencyofpeople,adequateinfrastructure,suitableworkingenvironment,organizationalknowledge,investment,budgets,externalprovision

o Responsibilityforachievingobjectivesatthevariouslevelswithintheorganizatione.g.topmanagement,functionalleadership,departmentleadership,processowners,teams,individuals

o Timescalesforachievingobjectivese.g.steppedachievement(weekly,monthly,quarterly),specificmilestones(defineddatesorperiods),annualachievement

o Methodsusedtoevaluatetheresultse.g.periodiccomparisonofperformanceagainstestablishedtargetsorexpectationsduringmanagementreview,functionalreviews,processcouncils,departmentalreviews,personaldevelopmentreviews(appraisals),teammeetings

6.3

PlanningofChangesThingstoconsider:• TheapproachtakenbytheorganizationtoensurethatchangestotheQMSareplannedandimplementedinacontrolledmanner

takingintoaccount:o Thereasonforthechange;e.g.contextoftheorganization(see4.1),needsofinterestedparties(see4.2),customer

feedback,complaintsanalysis,auditresults,performancetrends,riskreductionorrealizinganopportunity(see6.1),continualimprovement,organizationgrowth,launchofnewproducts/services,organizationrestructuringetc.

o Assessingthepurposeofthechange(s)andpotentialimpactusingariskbasedthinkingapproachtoensuretheintegrityoftheQMSismaintainedi.e.focusonpriorities,avoiddisruption,ensurebusinesscontinuity,maintainproduct/servicereliability,protectthecustomer,maintaincapability,continuetomeetinternal/externalrequirementsetc.

o Theresourcesrequiredtoenablethechangesuchaspeople,knowledgeacquisition,infrastructure,environment,budget,trials/tests,ongoingmonitoring,structuredreviewsetc.

o Determinationofresponsibilityandauthorityforthechangee.g.processowner,leadprocessusers,endusersetc.,includingthenecessarycommunication,trainingandongoingreviewtoensurethechangeiseffective(i.e.theplannedactivitiescontinuetoberealizedandplannedresultsarebeingachieved)

Page 19: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 19 Revision:2.1

Clause# Considerations7. SUPPORT7.1 Resources

7.1.1

GeneralThingstoconsider:• Organizationdeterminationoftheresourcesneededfortheestablishment,implementation,andcontinualimprovementoftheQMS

takingintoaccount:o Resourceplanning,includingload/capacitybalance,makeversusbuyanalysiso Utilizationofexistinginternalresourcese.g.people,facilities,materials,equipment,finance,capability,capacity,informationo Utilizationofresourcesprovidedbyexternalproviderse.g.processes,products,services

7.1.2 PeopleThingstoconsider:• OrganizationapproachforprovidingthenecessarypeopletoensureeffectiveimplementationandcontroloftheQMSe.g.

organizationalknowledge,capability,skillset,competency,experience,workforceagility,recruitment(temporary,permanent),termsofemployment

7.1.3

InfrastructureThingstoconsider:• Determinationoftherequiredinfrastructuretoenablethestrategicandoperationalneedsoftheorganizationincluding:

o Buildingse.g.manufacturing/assemblyplants,testfacilities,laboratories,servicecenters,officeso Utilitiese.g.electricity,gas,water,compressedairo Equipmente.g.machinetools,jigs,fixtures,tooling,workstations,ITequipment,softwareprograms,assetcareo Transportatione.g.materials/equipmenthandling(internal/external),packaging/protection,hazardoussubstanceso Informationandcommunicationtechnologye.g.ITinfrastructure(servers,backupsystems,businesscontinuity,accessibility,

security,networks)• Considerationoffacilitiesmanagemente.g.organizationalfootprint,realestateplanning,sustainability(energyusage,emissions),

statutoryinspections,restrictedareas,hardservices(infrastructuremaintenanceanddisposal),softservices(cleaning,catering,security)

Page 20: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 20 Revision:2.1

7.1.4

EnvironmentfortheOperationofProcessesThingstoconsider:• Determinationoftherequiredenvironmenttoenablethestrategicandoperationalneedsoftheorganizationincluding:

o Socialenvironmente.g.ethics,equality,diversity,behaviors,policies,codesofconduct,o Psychologicalenvironmente.g.workload,workinghours,timeconstraints,conflictingdemands,flexibleworking,healthand

wellbeing,confidentialreportinglineso Physicalenvironmente.g.space,ergonomics,workplaceorganization(5S),visualmanagement(layout,zoning,signingetc.)

cleanliness,restrictedareas,electrostaticdischarge,lighting,temperature,humidity,ventilation,noise,FODprevention,wastereduction(Lean)

7.1.5 MonitoringandMeasuringResources

7.1.5.1

GeneralThingstoconsider:• Determinationoftheresourcesrequiredtoenablevalidandreliableresultswhenmonitoring(checking,inspectingobservingor

witnessingactivity)andmeasurement(determinationofphysicalcharacteristics)isappliedtoverifyorvalidateprocess,activity,productandserviceconformity

• Monitoringandmeasuringcanvarydependinguponthecomplexityandneedsoftheorganization,requirementsoftheoutputandexpectationsofthecustomer

• Theselectionandapplicationofappropriateresourcesafterconsideringthecharacteristics,featuresandattributesoftheactivitiesbeingundertaken

• Monitoringandmeasurementresourcescanincludesuitablepeople,facilities,instruments,gages,equipment,devicesetc.,thatarebothinternalorexternaltotheorganization

• Maintenanceofthemonitoringandmeasurementresourcestoensurecontinuedadequacyincludingforexample:o Peopletrainingandcompetencyprovisiono Facilitymaintenanceregimeso Assetcareprogramso Calibrationarrangements

• Documentedinformationtodemonstratethattheresourcesutilisedcontinuetobeappropriateandadequateincludingforexample:plans,schedules,databases,systems,registers,certificates,traceabilitytorecognizedstandards(whererequired)

Page 21: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 21 Revision:2.1

7.1.5.2 MeasurementTraceabilityThingstoconsider:• Organizationapproachinunderstandingandapplyingthenecessaryarrangementsinordertovalidatemeasurementresultsfor

example:o Identificationandlocationofmeasurementequipment(organization,personal,customerowned)o Identificationofcalibrationequipment(gagenumber,serialnumber,referencenumber)o Methodsofcalibration(procedures,instructions,guides)o Measurementstandardsandacceptancecriteria(organization,national,international)o Frequencyofcalibrationandrecall(timeperiod,advancenotice,duedates,overduenotices)o Suitableenvironmentalconditions(cleanliness,temperature,humidity)o Calibrationstatus(marking,tagging,labeling,colorcoding)o Protectionandpreservation(storageconditions,preventionofadjustment,protectionfromdamage)o Measurementcapability(intervalanalysis,gageRepeatability&Reproducibility(R&R),measurementuncertainty,

measurementsystemanalysis)• Arrangmentsformaintainingdocumentedinformationincludingforexample:

o Equipmentregister(type,identification,location,method,frequency,acceptancecriteria)o Recordsofcalibration(identity,description,date,condition,results,authority,interval,nextduedate,adjustments,limitations

ofuse)• Actiontakenwhenmeasuringequipmentisfoundtobeunfitforitsintendedpurposeanditsimpactonpreviousmeasurementresults

(errorreports,investigation,previousequipmentuseage,riskassessment,reporting,productrecall)7.1.6 OrganizationalKnowledge

Thingstoconsider:• Organizationalapproachtowardsthedeterminationandcaptureofknowledgeincludingforexample:

o Planningforknowledgecapture(knowledgemanagementplan,presentandfutureneeds,sustainability,knowledgesources,approach,scope,ownership,timeliness,knowledgedistribution)

o Knowledgegainedfromexperience(personaldevelopment,successionplanning,subjectmatterexpertise,mentoringprograms,coaching,peerreviews,organizationinsight,businessexposure)

o Knowledgeacquisition(intellectualproperty,patents,lessonslearnedovertime,explicitandtacitknowledge,professionalmembership,industryassociations,benchmarking,conferences,communitiesofpractice,interestedparties,journals,

Page 22: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 22 Revision:2.1

academia)• Maintainandshareorganizationalknowledge(knowledgemanagementsystem,electronicmedia,intranet,database,repository,

libraries,communitiesofpractice,newcomermentoring,subjectmatterexperts,masterclasses,continualimprovement,sharinggoodpractice,QMSupdates)

7.2 CompetenceThingstoconsider:• Organisationdeterminationofcompetencyrelatingtoperson(s)androle(s)suchas:

o Definedcompetencyrequirements,includinganyspecifiedinduction,training,assessment,evaluationandqualification,roleprofiles,jobdescriptions,accountabilitystatements,skillsmatrices,competencypacks,performancecriteria,developmentframeworks

o Understandingresourceneedsandrelatedcompetency(trainingneedsanalysis,gapanalysis)o Provisioningthenecessarycompetencythroughreassignmentordevelopmentofexistingperson(s),oracquiringcompetent

resourceexternallyo Evidenceofabilitybasedoneducation,training,skills,knowledge,experience,professionalmembershipetc.o Periodicassessmentandevaluationofcompetencytoensurecontinuedadequacyandeffectiveness

• Evidenceofcompetency,includingforexample;trainingcertificates,recordsofachievement,qualificationstatements,IndividualDevelopmentPlans(IDP),ContinuedProfessionalDevelopment(CPD),BodyofKnowledge(BoK),Onthejobtraining(OtJ),logbooks,workplaceassessments

7.3 AwarenessThingstoconsider:• Demonstrationofperson(s)awarenessof:

o Thequalitypolicy(see5.2)andrelevantqualityobjectives(see6.2)througheffectivetwowaycommunicatione.g.prominence,briefings,visualdisplay,intranet,newsletters,employeeengagement,accessandavailability

o TheircontributiontotheeffectivenessoftheQMS,andthebenefitsofimprovedperformancee.g.BusinessPlanDeployment(BPD),flowdownofobjectives,appraisals,performanceratings,Quality/Cost/Delivery(QCD)performancemonitors

o Relevantdocumentedinformation(includingchanges),theircontributiontoproduct/serviceconformityandtheimplicationofnotconformingwithrequirementse.g.accesstotheQMS,navigationofrequirements,notificationofchanges,‘what’snew’notices,nonconformancereporting,understandingofculpabilityandconsequences

o Theircontributiontoproductsafetye.g.individualaccountabilities,compliancetoprocess,attentiontodetail,safety

Page 23: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 23 Revision:2.1

awarenesstrainingincludingproductendusageandpotentialimpactrelatingtoproductissues,noticeboards,safetyalerts,posters

o Theimportanceofethicalbehaviore.g.codeofconduct,internalmanagement/employeeworkingrelationships,fairtreatment,employeeworkrecognition,confidentialreportingmechanisms,protectinganonymity,noblameculture,awarenesscampaigns,noticeboards,posters,trainingprograms

7.4CommunicationThingstoconsider:• Useofacommunicationstrategy,policy,plan• Organisationapproachtowardsinternalandexternalcommunicationtakingintoaccount:

o Whatistobecommunicatede.g.policy,objectives,QMSrequirements,processes,organizationperformance,customersatisfaction,changesetc.

o Whentocommunicatee.g.frequency,importance,significance,scheduled,adhoc,shiftcoverageo Whotocommunicatewithe.g.relevantinterestedparties:

§ External(customers,regulators,stakeholders,agencies,localcommunity,investors,externalproviders,media)§ Internal(employees,employeerepresentatives,contractors)

o Howcommunicationoccurse.g.meetings,briefings,notices,e-mails,telephone,text,intranet,internet,directives,visualmanagement,campaigns,socialmedia,alerts,bulletins,WebEx’s,webinars,pressrelease,newsletters

o Whoundertakesthecommunication:managers,supervisors,teamleaders,teammembers,employeerepresentatives,corporatecommunication,publicrelations,marketing,externalproviders

• Methodstoenable‘twoway’communicationasappropriateinordertoverifyunderstandingandcapturefeedback7.5 DocumentedInformation7.5.1 General

Thingstoconsider:• ArrangementsforensuringthattheQMSincludesdocumentedinformationappropriatetotheorganization(size,products,services,

processes,complexity,competencyetc.)requiredby:o 9100standardo Organizationsownrequirements

Page 24: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 24 Revision:2.1

7.5.2 CreatingandUpdatingThingstoconsider:• Arrangementsforcreatingandupdatingdocumentedinformationincluding:

o Identificationanddescriptione.g.subjecttitle,documentidentification,date,author,referencenumber,versionnumber,changehistory,retentioncategory,retentionperiod,securityclassification,exportcontrol,intellectualpropertyclassification,copyright

o Formate.g.language,softwareversion,layout,pagination,style,fonts,resolution,hyperlinksandmediae.g.paper,magnetic,electronicoropticalcomputerdisc,photograph,mastersample,ITsystem,portablemedia,microfiche,X-rayfilm

o Reviewandapprovale.g.reviewperiods,permissions,authorizedpersons,reviewmethods(interestedparties,draftrelease,ballot,buyoff,finalrelease)

7.5.3 ControlofDocumentedInformation7.5.3.1 Thingstoconsider:

• Arrangementsforcontrollingdocumentedinformationrequiredby:o 9100standardo Organizationsownrequirements

• Availabilitye.g.documentaccessibility(hardcopy,electronicmedia),readilyavailableatthepointofuse• Suitabilitye.g.format,mediasuitabletotheenvironment,easeofunderstanding,language,interpretation• Protectione.g.documentauthentication,documentmarkings(official,secret,restricted,confidential,private,sensitive,classified,

unclassified),accesscontrols(individual,rolespecific),physicalsecurity(masterdocuments,serverrooms,libraries)ITsecurity(UserID,password,servers,download,backup,encryption,‘readonly’,‘read/write’),protectionfromcorruptionandunintendedalterations

7.5.3.2 Thingstoconsider:• Documentcontrolarrangementsincluding:

o Distribution(internal/external)e.g.infrastructure,networks,portals,documentregisters,pointsofcontact,transmissiono Accessandretrievale.g.location,availability,timeliness,remoteaccesso Storageandpreservatione.g.physicalenvironment,protectionfromfireandflood,filingarrangements,physical/ITsecurity,

sustainability,redundanthardware/softwaretoaccesslegacymedia,archivearrangementso Changecontrole.g.documentreview,changenotices,versioncontrol,changehistory,re-issue,communication,distributiono Retentione.g.organization/legal/contractualretentionperiods,storage,preservation,backup,retentionofknowledge,

disposal

Page 25: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 25 Revision:2.1

o Obsolescencee.g.withdrawal,replacement,legacyarchiveandsuitableidentification(“forinformationonly”,“nottobeusedafter….”,“uncontrolledcopy”,“forreferencepurposesonly”etc.

o Electronicdataprotectione.g.securitypolicy,systemaccessprofiles,passwordrules,storageandback-uppolicyincludingprotectionfromloss,unauthorizedchanges,unintendedalteration,corruption,physicaldamage

• Identificationandcontrolofexternaldocumentedinformatione.g.ownership,customerproperty,whereabouts,accessability,pointsofcontact,changenotices,distribution,return,disposal,documentmarkings,licensing,copyright,linkstoexternalrepositories

Page 26: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 26 Revision:2.1

Clause# Considerations8. OPERATION8.1 OperationPlanningandControl

Thingstoconsider:• Organizationapproach,includingrelevantfunctionengagement(e.g.planning,engineering,operations,purchasing,quality)toplan,

implementandcontroloperationalactivity,takingintoaccount:o Actionsarisingfromorganizationrisksandopportunities,qualityobjectivesandchangestotheQMS(seeclause6)o Provisionfordeterminingrequirementsrelatingtotheorganizationandinterestedparties(see8.2.2)o Establishmentofprocesscriteriae.g.inputs,outputs,controls,resources,sequence,interaction,measures,responsibilities(see

4.4)o Establishmentofcriteriaforacceptanceofproductsandservicese.g.measurement,monitoring,inspection,verification,

validation,test,statisticaltechniqueso Determinationofinternal/externalresourcestoensureconformityandon-timedeliverye.g.people,facilities,materials,

equipment,finance,information,outsourcedprocesses,productsandservices,includingtheuseofsystemssuchasEnterpriseResourcePlanning(ERP),MaterialPlanningSystem(MPS),MaterialResourcePlanning(MRP),load/capacitymodels,salesorderreview,forecastingtechniques,processsimulation

o Controloftheprocessesinaccordancewithcriteriatodemonstratethatintendedoutputsaremetandimprovementsidentifiede.g.processachievement,expectedoutcome,opportunitiesforimprovement

o Othermethodsdeterminedbytheorganizationsuchasanintegratedphasedprocessapproache.g.AdvancedProductQualityPlanning(APQP),ProductionPartApprovalProcess(PPAP)

• Determinationandretentionofdocumentedinformationtoensureprocesscontrole.g.operationalprocessdefinition,productionplanningandcontrol,operationalinstructions,productionsystem,productdatamanagement,qualityplans

• Determinationandretentionofdocumentedinformationtodemonstrateconformitye.g.processoutputrecords,verification/validationresults,scheduleadherence,performancemeasures,managementreviewoutput

• Determinationoftheprocessesandcontrolsneededto:o Managecriticalitems(itemsthathaveasignificanteffectontheprovisionanduseoftheproductsandservices–see91003.2)

e.g.partsclassification,criticalpartplans,sourceandmethodchange,traceability,servicelifemanagemento Managekeycharacteristics(anattributeorfeaturewhosevariationhasasignificanteffectonproductfit,form,function,

Page 27: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 27 Revision:2.1

performance,servicelife,orproducibility–see91003.3)e.g.componentproving,processcontrolplan,measurementandevaluationtechniques,reviewanddisposition

• Determinationoftheprocessandresourcesneededtomanageuseandmaintenanceoftheproductorservicee.g.servicenetwork,technicalservices,technicalpublications,problemresolutionsystem,feedbackmechanisms,productsupport,aftermarketsupport

• Determinationandcontroloftheproductsandservicestobeobtainedfromexternalproviderse.g.resourceplanning,load/capacitybalance,makeversusbuyanalysis,outsourcingpolicy,outsourcingprocesses,sourcechange(includingpermanentortemporaryworktransfertoanexternalprovider,orfromoneexternalprovidertoanother)

• Establishmentofcontrolsneededtopreventdeliveryofnonconformingproductsandservicese.g.processcontrol,verification/validationactivity,componentproving,riskmanagement,customereyes,delegatedproductreleaseverification

• Appropriateplanningandmanagementofproductandserviceprovisione.g.projectplanning,projectmanagement,programmanagementtakingintoaccount:projectmanagementplans,qualityplans,workbreakdownstructure,sequencing,phases,deliverables,dependencies,constraints,risks,resources,achievementofmilestones

• Controlofplannedandunintendedchanges(e.g.torequirements,processes,sources,methods,schedule,volumes)andconsideringcommunication,impactandmitigationofrisks,pre-postchangeevaluationetc.

8.1.1

OperationalRiskManagementThingstoconsider:• Organizationarrangementsformanagingriskwithinoperations(planningandcontrol,projectmanagement,requirementsforproducts

andservices,designanddevelopment,externalprovision,productionandserviceprovision),takingintoaccount:o Assignmentofresponsibilitiese.g.riskowners,facilitators,champions,coordinators,managers,actionownerso Riskcriteriae.g.criteriarelatingtothemagnitudeofrisk,probability/likelihood(veryhigh,high,medium,low,verylow),

impact/consequence(critical,significant,marginal,negligible)o Riskactivity:

§ Identificatione.g.scope,riskidentity,“if/then”statements,referencenumbers§ Assessmente.g.risktreatment,riskanalysis,applicationofcriteria,risklevels,scoring,ranking,prioritization,keyrisks,

riskassessmentmatrix,riskregister§ Communicatione.g.stakeholderengagement,monitoringandmeasurement,riskaggregation,riskstatus,risktransfer,

riskreports,riskmanagementsystems§ Managementofactionse.g.planning,identity,ownership,timescales,review,riskanalysisupdate§ Riskacceptancee.g.residualrisk,AsLowAsReasonablyPractical(ALARP),rationalrisk,proceedatrisk,contingency

Page 28: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 28 Revision:2.1

plans,readacrossofriskoutcomestoothersimilarprojects,activities,situations8.1.2 ConfigurationManagement

Thingstoconsider:• Organizationapproachtothemanagementofconfiguration,takingintoaccount:

o Productidentityandtraceabilitye.g.BillofMaterial(BoM),GeneralArrangement(GA),productbreakdownstructure,illustratedpartscatalog,partnumbering,partlists,‘asproduced’condition,configuration(IT)system,standardparts(catalog,offtheshelf),productlogbook,ProductLifecycleManagement(PLM),ProductDataManagement(PDM)

o Identifiedchangese.g.alterationrequests,noticeofchange,amendments,deviations,waivers,partrevisionchanges,partnumberchanges,changecategories,servicebulletins,modificationbulletins,airworthinessdirectives,engineeringcommunicationnotices

o Documentedinformationisconsistentwithproduct/serviceattributese.g.regulations,definitions,drawings,specifications,standards,models,componentproving,productioninstructionsandrecords(manufacturing,assembly,test,repair),productchangerecords,authorizationofchange,typecertificates

8.1.3

ProductSafetyThingstoconsider:• Arrangementsforplanning,implementingandcontrollingtheprocessesneededtoensureproductsafety(abilityforaproductto

performtoitsintendedpurposewithoutcausingunacceptableriskofharmtopersonsordamagetoproperty–see91003.4)includingforexample:

o Assessmentsofhazardsandmanagementofriske.g.riskassessment,DesignFailureModesandEffectsAnalysis(DFMEA),ProcessFailureModesandEffectsAnalysis(PFMEA),safetyanalysis,FailureModesEffectsandCriticalityAnalyses(FMECA),faulttreeanalysis

o ManagementofSafetycriticalitemse.g.monitoringcontrolplan,criticalpartplans,inspectionandserviceintervals,componentlifing,cycliclife,lifemanagementplans

o Analysis,reportingandcommunicationofoccurredeventse.g.datacollectionandanalysis,internalescalationprocess,mandatoryreporting,safetyalertreports,in-servicereliability,operatingperformanceandtrends,lessonslearned,sharedindustryexperience,productsafetyreviews,airworthinessdirectives

o Trainingandcommunicatione.g.productsafetypolicy,promotingasafetyculture,productintegritytraining,productsafetytraining,awarenesscampaigns,safetynotices,safetyalerts,individualreportingmechanisms

Page 29: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 29 Revision:2.1

8.1.4

PreventionofCounterfeitPartsThingstoconsider:• Arrangementsforplanning,implementingandcontrollingtheprocessesneededtopreventcounterfeitofsuspectcounterfeitparts

(unauthorizedcopy,imitation,substituteormodifiedpartwhichisknowlinglymisrepresented–see91003.1)includingforexample:o Planninge.g.policy,counterfeitpartsplan,governancearrangements,riskassessment,detectionstrategies,counterfeitsources,

reporting,training,engagementwithexternalproviders,communicationo Functionaltraining(prevention,mitigation,detection,dispositionandreporting):o Procuremente.g.trustedsourceselectiono Inspectione.g.preventionofcounterfeititems(visual/test)o Engineeringe.g.Obsolescencemanagemento Awarenesse.g.campaigns,posters,alertso Obsolescencemonitoringe.g.designdecisionsandpartselectionappropriateforservicelifeofproducto Partacquisitione.g.OriginalEquipmentManufacturers(OEM),authorizeddistributors,otherapprovedsourceso AssuringtraceabilitytoOEM,authorizedmanufacturer,approvedexternalprovidero Verificationandtestmethodologiese.g.partmarkings,visualfeatures,inspectionofattributes,functionaltestandvalidation,

packagingo Preventingre-entryintothesupplychain(see8.7.1)e.g.labeling,marking,segregation,quarantine,containment,disposition,

lessonslearned,annotationofdocumentedinformation,electronicsystem(MRP)updateso Reportinge.g.occurrences,events,externalintelligencegathering(databases,notifications,reporting),readacrosstoother

products/services,escalation,riskstointerestedparties,externalcommunication8.2 RequirementsforProductsandServices

8.2.1

CustomerCommunicationThingstoconsider:• Organizationapproachtowardscommunicatingwiththecustomerinrelationto:

o Product/serviceinformatione.g.promotionalmaterial,telephone,journals,conferences,seminars,exhibitions,tradeshows,catalogues,brochures,marketingcampaigns,socialmedia(websites,messageboards)

o Handlingenquiries,contractsororderse.g.engagement,support,pointsofcontact,accessibility,helpline,callcenters,internet‘contactus’,customerledger

o Handlingchangestoenquires,contractsororderse.g.changenotices,amendments,ledgerupdates

Page 30: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 30 Revision:2.1

o Collatingcustomerfeedback(see9.1.2)e.g.satisfactionsurveys,questionnaires,meetings,socialmedia,forums,events,complaintprofile,reportcards

o Handlingcustomerpropertye.g.registers,identification,location,preservation,maintenance,storageo Contingencyactions(ifrequired)e.g.backupplans,alternativearrangements,proactiveengagement,considerationofrisk

8.2.2 DeterminingtheRequirementsforProductsandServicesThingstoconsider:• Determinationofrequirementsincluding:

o Definedrequirementstakingintoaccount:§ Statutoryandregulatoryrequirementse.g.Airworthinessregulations,ITAR/EAR,REACH,CS-25international/national

legislation,certificationassurance(CE,ULetc.)§ Organizationrequirementse.g.product/serviceknowledge,understandingofcustomerstatedandimpliedneeds,lessons

learned,derivedrequirementsfrompreviousprojects,conceptualthinking,operationalparameters(producibility,safety,reliability,obsolescence,disposaletc.)

§ Specialrequirementse.g.requirementsidentifiedashighrisk,suchasproductorprocesscomplexity,pastexperience(lessonslearned),product/processmaturity,boundariesofperformance,technicalorprocesscapability,identificationofcriticalitems(see91003.5)

o Organizationabilitytomeetrequirementse.g.internal/externalresources(people,facilities,materials,equipment,finance,information),capability,capacity,expertise,product/servicevalidation,operationalrisk(newtechnology,shortleadtimes,resourceconstraints,interdependencies)

8.2.3 ReviewoftheRequirementsforProductsandServices8.2.3.1 Thingstoconsider:

• Organizationreviewofrequirementscoordinatedwithapplicablefunctions(e.g.sales,planning,engineering,operations,purchasing,finance,legal,quality),beforecommittingtosupplyproductsandservicesincluding:

o Requirementsspecifiedbythecustomere.g.withinenquiries,contracts,purchaseorders,statementsofwork,memosofunderstanding,lettersofintent,termsandconditions

o Requirementsimpliedbythecustomere.g.useofpreviouscontractreviewactivity,customerpreferences,customerfeedback,organization/industryknowledge,lessonslearned,marketplace/competitorintelligence,inserviceperformancedata,previousexperience,readacrossfromothersimilarproductsorservices

o Requirementsfordeliveryandpostdeliverye.g.logistics,shipping,transport,installation,commissioning,training,handover,

Page 31: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 31 Revision:2.1

customeracceptance,warranties,repairs,customersupporto Organization(e.g.policies,processes,procedures,standards,specifications,rules,codesofpractice)and,statutory/regulatory

requirements(e.g.airworthinessregulations,ITAR/EAR,REACH,CS-25international/nationallegislation,certificationassurance(CE,ULetc.)

• Resolutionofcontractororderrequirementsthathavebeensubjecttochangee.g.identificationofthedifferences,communicatingwiththecustomer,clarificationandunderstanding,impactassessment,mutuallyacceptableoutcome

• Confirmationofrequirementswhentheyarenotdocumentede.g.clarification,understanding,acknowledgementandsubsequentagreementintheformofretaineddocumentedinformation

8.2.3.2 Thingstoconsider:• Retentionofdocumentedinformationtocapturetheresultsofthereviewincludinganynewrequirementsorchangese.g.recordof

contractreview,includingforexamplecustomer,reference,date,persons,resources,conventional/specialrequirements,risksoutcome,changes

8.2.4 ChangestoRequirementsforProductsandServicesThingstoconsider:• Amendeddocumentedinformationandcommunicationofchangedrequirementse.g.updatedcontractreviewrecords,amended

orders/contracts,memos,changenotices,quailityplans,meetingminutes,togtherwithcommunicationtorelevantinterestedparties(personswithinoroutsidetheorganizationthatmaybeimpactedbythechange)

8.3 DesignandDevelopmentofProductsandServices8.3.1 General

Thingstoconsider:• Organizationarrangementsforestablishing,implementingandmaintainingadesignanddevelopmentprocessincludingplanning,

inputs,controls,outputsandchanges(see8.3.2to8.3.6)8.3.2 DesignandDevelopmentPlanning

Thingstoconsider:• Organizationapproachtowardsplanningdesignactivityincluding:

o Scopeofthedesigne.g.requirementscapture(see8.2.2)rationale,assumptions,objectives,complexity,size,detail,timescales,criticality,constraints,risks,producibility,maintainability

o Stagesofthedesignprocess,distinctactivitiesandreviewe.g.workbreakdownstructure,workpackages(tasks,resources,responsibilities,content,inputs/outputs),conceptdesign,preliminarydesign,detaildesign,reviewgates(preliminarydesign

Page 32: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 32 Revision:2.1

review,detaildesignreview,criticaldesignreview)o Verificationandvalidationactivitye.g.checks,trials,tests,simulations,demonstrationsrequiredtoensurerequirementsaremeto Responsibilitiesandauthoritiese.g.roleprofiles,accountabilitystatements,delegationofauthority,levelsofapproval,register

ofauthorityandapprovals,authorizedsignatorieso Internal/externalresourcese.g.knowledgeacquisition,people,competency,investment,funding,facilities,equipment,

innovation,technology,interestedparties(customers,externalproviders,researchestablishments),information(principles,standards,rules,codes)

o Organizationalinterfaces(persons/functions)e.g.sales,projectmanagement,production,procurement,quality,finance,customers,endusers

o Subsequentprovisionandapplicatione.g.forwardthinking,readacross,re-useofnewtechnologies,product/servicederivatives,sustainability,recognizingobsolescence

o Levelsofcontrolrequiredorimpliedbyinterestedparties(customers,regulators,endusersetc.)e.g.customeracceptance,safetychecks,riskmanagement,verification/validationactivity,productcertification

o Requireddocumentedinformatione.g.designplan,designreviews,designoutputs(specifications,schemes,drawings,models,data,reports),controlplans,certificates

8.3.3 DesignandDevelopmentInputsThingstoconsider:• Determinationofinputrequirementstogetherwiththeretentionofdocumentedinformationincluding:

o Functionalandperformancerequirementse.g.customerneeds,operatingcharacteristics,performanceparameters,safety,usability,reliability,maintainability

o Informationtransfere.g.readacrossfromothersimilardesigns,lessonslearned,performancedata,in-servicedata,customerfeedback,externalproviderfeedback,bestpractice,benchmarking

o Statutoryandregulatoryrequirementse.g.legislation,regulations,directiveso Standardsorcodesofpracticee.g.policies,standards,specifications,rulesandaids,protocols,guidance,industrycodeso Consequencesoffailuree.g.riskprofile,DFMEA,safetyanalysis,FMECA,faulttreeanalysis,lessonslearnedo Consequenceofobsolescencee.g.sourceofsupply,sustainability,prohibitedmaterialsandsubstances,conflictminerals,

specialitymaterials,proprietaryparts,exposureofcounterfeito Adequacyofinputse.g.clear,complete,unambiguous,understandable,transmittableo Conflictinginputsareresolvede.g.functionalconsensus,communicatingwithinterestedparties,contract/orderamendment,

Page 33: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 33 Revision:2.1

interestedpartyagreement,mutuallyacceptableoutcome8.3.4 DesignandDevelopmentControls

Thingstoconsider:• Determinationofcontrolstogetherwiththeretentionofdocumentedinformationincluding:

o Definitionofoutcomese.g.designintent,functionalandperformancerequirements,customer/enduserexpectationso Designreviewe.g.functionalrepresentation(customer,engineering,production,quality,projectmanagementetc.),review

gates(preliminarydesignreview,detaildesignreview,criticaldesignreview),commercial/technicalconsiderations,authorizedprogressiontonextstage(proceedtonextgate)

o Verificationactivitye.g.modeling,simulations,alternativecalculations,comparisonwithotherprovendesigns,experiments,tests,specialistoverview(stress,performanceetc.)

o Validationactivitye.g.functionaltesting,performancetesting,trials,prototypes,demonstrations,simulationso Managementofactionsarisingfromdesignreview,verificationorvalidatione.g.actionregisters,ownership,timescales,

escalation,changestoriskprofile8.3.4.1 Thingstoconsider:

• Planningandcontrollingverificationandvalidationtestsincluding:o Testrequirementse.g.productidentity,objectives,schedules,plans,specifications,resources(persons,equipment,

infrastructure),parameters,conditions,acceptancecriteria(pass,failetc.)o Testmethodse.g.instructions,procedures,schedule,checklists,logbooks,diaries,datacaptureo Configuratione.g.productconfiguration(item,assembly,module,product)andtestconfiguration(facilities,equipment,

conditions,software,controlledmonitoringandmeasuringdevices–see7.1.5)o Retaineddocumentationtodemonstratethatthe:

§ Testplanandtestprocedureshavebeenobservedandcriteriahasbeenmet§ Designmeetsthespecifiedrequirementsforallidentifiedoperationalconditionse.g.reports,calculations,testresults,

data,reviews,outputs8.3.5 DesignandDevelopmentOutputs

Thingstoconsider:• Outputsmeetinputrequirementse.g.checklists,designreviewrecords,authorizationtoproceed,customeracceptance,product

certification• Outputsareadequateforproductandserviceprovisione.g.standards,specifications,schemes,drawings,models,partlists,materials,

Page 34: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 34 Revision:2.1

methods,manufacturinginstructions,technicalpackages,tooling,machineprograms,preservation,handling,packaging,specialisttraining,userinstructions,servicemanuals,repairschemes,externalprovision,togetherwith:

o Referencetomonitoringandmeasuringequipmente.g.inspectionequipment,gages,instruments,environmento Acceptancecriteriae.g.product/servicespecification,limits,tolerances,qualityacceptancestandardso Product/servicecharacteristicse.g.keycharacteristics,customercriticalfeatures,interfacefeatures,inspections,service

intervals,operatingcharacteristics,o Criticalitems(see91003.2)e.g.identification,keycharacteristics,specialhandling,serviceintervals,componentlifing,cycliclife,

lifemanagementplans,sourceandmethodchange,traceability• Outputsareapprovedpriortoreleasee.g.scopeofauthorization,authorizedpersons,levelsofauthorization,methodofauthorization

anddocumentedinformationisretained8.3.6 DesignandDevelopmentChanges

Thingstoconsider:• Organizationapproachfortheidentification,reviewandcontrolofchangesincluding:

o Implementationofaprocesstonotifythecustomerwhenchangesaffectthecustomerrequiremente.g.customercommunication,notificationsofchange,requestsfordeviation,contractamendments

o Configurationcontrole.g.alterationrequests,noticeofchange,amendments,deviations,waivers,concessions,partrevisionchanges,partnumberchanges,changecategories,servicebulletins,modificationbulletins,airworthinessdirectives,engineeringcommunicationnotice,productchangeboards

• Retaineddocumentedinformationthatincludeschangehistory,evaluationofchangeresults,authorizationofchangeandactionstakeninrelationtosubsequentactivitiesthatareimpactedbythechange

8.4 ControlofExternallyProvidedProcesses,ProductsandServices8.4.1 General

Thingstoconsider:• Organizationarrangementsforensuringexternalprovidedprocesses,productandservicesconformtorequirementswhen:

o Incorporatedintotheorganizationsownproductsandservicese.g.boughtoutfinished/completeproducts/serviceso Provideddirectlytotheorganizationscustomere.g.directship,directdeliveryo Aprocessorpartofaprocessisprovidede.g.outsourcing,offload,operationalprocessing(conventional/specialprocessesetc.),

supportprovidedon-siteattheorganizationpremises(ITsupport,facilitiesmanagementetc.)

Page 35: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 35 Revision:2.1

• Takingintoaccount:o Retainingresponsibilityforexternalprovisionfromallsources,includingthosedefinedbythecustomere.g.organizationtakes

fullresponsibility,irrespectiveofanysourcesidentifiedbythecustomero Theuseofapprovedexternalprovidersincludingthosedesignatedbythecustomere.g.

§ Sourcingfromanapprovedsupplierlistwheretheorganizationsapprovalscopematchestherequirements§ Sourcinginaccordancewithcustomerinstructions(communications,contracts,memorandaetc.)

o Theidentificationandmanagementofrisks(see8.1.1)associatedwith:§ Externalprovisione.g.assignmentofresponsibilities,riskcriteria,riskidentification,riskassessment,communication,

managementofactions,riskacceptance§ Selectionanduseofexternalprovidersconsideringsuchthingsas:structure(ownership,parentcompany,subsidiaries,

locationetc.),marketintelligence,customerinput,financialstability,capability,termsandconditions,ratings,approvals(customer,regulator,3rdpartyetc.)

o Theapplicationofappropriatecontrolstoensurethatexternalproviderscontroltheirdirectandsub-tierexternalproviderse.g.approvals,contractconditions,flowdownrequirements,purchaseorderterms,mandatoryinstructions,nondisclosureagreements,statutory/regulatorydirectives,delegations

• Applyingcriteriafortheevaluation,selection,monitoringandre-evaluationofexternalprovidersincluding:o Commoditystrategye.g.products,services,categories,interestedpartyrequirementso Sourcingstrategye.g.policies,scope,goals,objectives,requirementscapture,riskso Selectioncriteriae.g.profile,portfolio,location,pre-requisites,capability,marketintelligence,strategicalignment,financial

stability,ethicalperformance,externalapprovals,externallyavailabledataandinformation(reports,disclosures,satisfactionindexes,performanceratings,reputation,otherpartyfeedbacketc.),

o Approvalmethodologye.g.approvalrequests(new,extendedetc.),scopeofapproval,externalapprovals/certifications(customer,regulator,3rdpartyetc.),authorizedpersons,2ndpartyauditactivity,approvalstatus(approved,suspended,withdrawn,lapsed,conditional,disapprovedetc.)

o Performancemonitoringe.g.measures,targets,KPIs,scorecards,dashboards,ratings,surveyso Retaineddocumentedinformationandactionsarisingfromtheevaluationse.g.approvedsupplierlist,auditreports,

performancereports,riskassessment,requestsforinformation(RFI),dispositionedapprovalrequests,certificates,gapanalysis,correctiveactionreports

Page 36: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 36 Revision:2.1

8.4.1.1 Thingstoconsider:• Definedprocess,responsibilitiesandauthorityforexternalproviderapproval,includingforexamplesupplierselectionandapproval

process,rolesandaccountabilitystatements,levelsofauthorization,scopeofapproval(products,services,processes,commoditiesetc.),changestoapproval(approved,suspended,withdrawn,lapsed,conditonal,disapprovedetc.),monitoringandmeasurement(performancereviews,surveillanceactivityetc.)

• Maintainedregisterofexternalprovidersincludingforexampleapprovedvendorlist,approvedsupplierlist,vendordatabase,scopestatements,approvalstatus

• Periodicreviewofexternalproviderperformanceincludingconformity,on-timedeliveryandotherpertinentmeasures(KPIs,targets,measures,values),performancereviews,managementreview,dashboards,flightdecks,reportsetc.,andarrangementsfortakingactionwhenrequirementsarenotmete.g.containment,correction,rootcause,correctiveaction,effectivenessofresolution,changetoapprovalstatus

• Definedrequirementsforcontrollingdocumentedinformationcreatedbyand/orretainedbyexternalprovidersincluding:o Communicatione.g.contractconditions,flowdownrequirements,purchaseorderterms,mandatoryinstructionso Documenttypes(maintained/retained)andcontrol(storage,protection,retrieval,media,retentionperiodsetc.)

8.4.2 TypeandExtentofControlThingstoconsider:• Organizationapproachtowardsensuringthatexternallyprovidedprocesses,productandservicesdonotimpactuponthedeliveryof

conformingproducttothecustomer,takingintoaccount:o ExternallyprovidedprocessesremainwithintheorganizationsQMScontrole.g.documentedinformationisalignedtoensure

commoninputs,outputs,controls,ownership,governanceetc.,betweentheorganizationrequirementsandthosethatareusedtointerfacewiththeexternalprovider

o Thedefinitionofcontrolsappliedtoanexternalproviderandthoseappliedtotheresultingoutpute.g.definedexternalproviderrequirementsandalignmentwiththecontrolsappliedtothesubsequentoutput

o Considerationof:§ Theimpactofexternalprovisionandtheorganizationsabilitytoconsistentlymeetcustomer,statutoryandregulatory

requirementse.g.thosedefinedincontracts,legislationandregulations§ Theeffectivenessofthecontrolsappliedtotheexternalprovidere.g.requirements,selectionandapproval,verification

activities,performancemonitoring,correctiveaction,continualimprovemento Determinationofverificationactivitiesaccordingtotherisksidentifiedbytheorganization,thatcaninclude:

Page 37: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 37 Revision:2.1

§ Reviewofdocumentedinformatione.g.certificatesofconformity,testreports,releasecertificates,regulatorycertificates(EASAForm1,FAA8130etc.),processcontroldocuments,productionprocessverificationactivity(see8.5.1.3)etc.

§ Inspectionandauditattheexternalproviderspremisese.g.sourceinspection,oversight,witness,sampling,3rdpartyverification,2ndpartyaudits,surveillanceactivity,processmonitoring,productaudits

§ ReviewofProductionPartApprovalProcess(PPAP)datae.g.controlplan,product/processcharacteristics,statisticalprocesscontrol,riskassessment,measurementsystemanalysis,verificationresults,PPAPfileetc.)

§ Inspection/verificationuponreceipte.g.activityundertakenbytheorganizationtoconfirmthatoutputsmeetstatedrequirementsthatcaninclude,definedlevelsofreceiptinspection,physicalinspectionoftheproducts,confirmationoftheserviceprovided,samplinginaccordancewithadefinedplan

§ Reviewofdelegationforproductverificatione.g.requirements,criteria,scope,registerofdelegations,levelsofauthority/approvals,authorizedsignatories,periodicmonitoringbytheorganization(oversight,witness,requalificationetc.)

§ Customerverificationatanylevelofthesupplychaine.g.oversight,witnessing,inspection,documentationreview,productaudit,productacceptancetesting

o Inspectionorperiodictestingwhenthereisahighriskofnonconformity,includingcounterfeitpartse.g.identification,detectionstrategies,communication,MRPindication/signal/alerts,inspectionlevels,verificationandtestmethodologies(partmarkings,visualfeatures,inspectionofattributes,functionaltest/validationetc.)

o Controlofproductsreleasedforproductionusependingcompletionofverificationactivitye.g.identification,communication,documentannotation,traceability,authorizationtoproceed,electronicsystem(MRP)updates,tracking/recallarrangements

o Utilizationoftestreportsduringverificationtoconfirmproductrequirementshavebeenmete.g.evaluationoftestreportdata(performancevalues,materialproperties,chemicalcomposition,productattributes,functionalcharacteristicsetc.)

o Validationoftestreportaccuracywhenrawmaterialhasbeenidentifiedasasignificantoperationalriske.g.provingthattestreportdatameetsproductrequirementsbyconductingsamplevalidationactivitysuchasfunctionaltesting,performancetesting,destructivetesting,laboratoryanalysisoftestspecimensetc.

8.4.3 InformationforExternalProvidersThingstoconsider:• Organizationarrangementstoensuretheadequacyofrequirementspriortocommunicatingwiththeexternalprovidere.g.clarityof

requirements,clearunderstanding,unambiguousdefinition,identificationofpotentialissues• Organizationcommunicationtotheexternalproviderincluding:

Page 38: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 38 Revision:2.1

o Requirementsoftheprocesses,productsandservicesthatarebeingprovisioned(identity,description,partnumbers,quantity,deliveryschedule,serviceparametersetc.),togetherwiththeidentificationofanytechnicaldata(standards,specifications,drawings,processrequirements,instructionsetc.)

o Approvalofproducts,services,methods,processes,equipmentandreleasee.g.criteria,requirements,acceptance,buyoff,delegations,permissions,authorizations,licenses,conditions

o Competencyrequirementsandpersonqualificatione.g.skills,training,assessment,evaluation,levelsofqualification,periodicre-assessment,maintainedcompetence

o Interactionse.g.engagement,communication,reviews,meetings,exchangeofdocumentedinformation,notices,directives,feedback

o Controlandmonitoringofperformancee.g.datacapture,measures,targets,KPIs,scorecards,dashboards,ratings,surveyso Verificationorvalidationactivitiesattheexternalproviderpremises(includingthecustomer)e.g.intentforsourceinspection,

oversight,witness,sampling,3rdpartyverification,2ndpartyaudits,surveillance,processmonitoring,productaudits,functionaltesting,performancetestingetc.

o Controlofdesignanddevelopment(see8.3)e.g.requirements,statementofwork,planning,controls(review,verification,validationetc.),delegations,maintained/retaineddocumentedinformation,designanddevelopmentchanges

o Specialrequirements,criticalitems,keycharacteristicse.g.specifications,performancecriteria,riskmanagement,identification,traceability,enhancedproduction/processcontrols,monitoringandmeasurementarrangements,fixedprocessapprovals,productqualification,changecontrol

o Test,inspectionandverificatione.g.requirements,instructions,acceptancecriteria,equipment,productionprocessverification(see8.5.1.3),retaineddocumentedinformation

o Useofstatisticaltechniquesforproductacceptance,includingacceptanceofinstructionse.g.applicationof;statisticalprocesscontrol(SPC),processmonitoring,samplingactivityandorganizationacceptanceofexternalproviderarrangements(review,approval,buyoffetc.)

o Theneedto:§ ImplementaQMSe.g.statementofrequirementsthatcanincludealignmentoftheexternalprovidersQMSto

recognizedexternalstandard(s),requirementtomaintainrelevantapprovals/certificationetc.§ Usecustomerdesignated,orapprovedexternalproviderse.g.statementofrequirementsrelatingtoexternalprovisionof

processes(includingspecialprocesses),productsandservicesfromapprovedsuppliers,includingthosefloweddownbytheorganizationscustomerwhereapplicable

Page 39: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 39 Revision:2.1

§ Notifynonconformingprocesses,productsandservicesandobtainapprovalfortheirdispositione.g.statementofrequirementsrelatingthecontrolofnonconformingoutputsthatcanincludeprocessrequirements,controlarrangements,requireddocumentedinformation,reportingmethod,responsibilitiesandauthorities(includingdelegationifapplicable),implementingdispositioninstructions,nonconformitymarking(e.g.concessionnumbersifapplicable)etc.

§ Preventtheuseofcounterfeitparts(see8.1.4)e.g.statementofrequirementsthatcanincludetheneedtoconductariskassessment,useofapprovedexternalproviders,verificationandtestmethodologies,preventingre-entryintothesupplychain(detection,segregation,quarantine,containment,dispositionetc.),reportingoccurrences,managingobsolescenceetc.

§ Notifychangestoprocesses,products,services,includingchangeofsource/locatione.g.statementrelatingtowhatconstitutesachange(includingchangestosourceandmethod)andtherequirementsforcontrol,reporting,obtainingapproval,actingupontheorganizationsresponseandmaintainingprocess/product/serviceconfigurationasapplicable

§ Flowdownrequirementse.g.specificrequirementsrequiredbytheorganization,ortheorganization’scustomer(asrequired)suchasmaintainingapprovals,useofapprovedexternalsub-tierproviders,verificationrequirements,notificationrequirements,reportingrequirementsetc.

§ Providetestspecimensfordesignapproval,inspection/verification,investigationorauditinge.g.statementofrequirementfortheprovisionoftestspecimens/pieces/artifacts,includingtype,identification,configuration,supportingdocumentedinformation,releaseconditions

§ Retaindocumentedinformation,includingretentionperiodsanddispositionrequirementse.g.statementofrequirementsrelatingtothecontrolofrecordsthatcanincludetype,media,backup,archive,storage,protection,preservation,access,retrieval,retentionperiods,disposal,returntotheorganization,security,dataprotection(electronicdocumentedinformation)etc.

o Rightofaccesse.g.acceptanceoftheorganizationsrightofaccesstotheexternalproviderspremisestoattendmeetings,conductperformancereviews,reviewworkinprogress,conductinvestigations,resolveproblems,conduct2ndpartyaudits,supportimprovementactivity,deliveryoftraining

o Ensuringpersonsareawareof:§ Theircontributiontoproduct/serviceconformitye.g.individualaccountability,understandingrequirements,compliance

toprocess,theneedtocontrolchanges,reportingofnonconformance§ Theircontributiontoproductsafetye.g.individualaccountability,compliancetoprocess,attentiontodetail,knowledge

Page 40: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 40 Revision:2.1

ofproductendusage,potentialimpactrelatingtoproductissues§ Theimportanceofethicalbehaviore.g.codeofconduct,management/employeeworkingrelationships,fairtreatment,

employeeworkrecognition,confidentialreportingmechanisms,protectinganonymity,noblameculture8.5 ProductionandServiceProvision8.5.1 ControlofProductionandServiceProvision

Thingstoconsider:Implementationandcontrolofproductionandserviceprovisionincluding:• Availabilityofdocumentedinformationthatdefinesproduct/servicecharacteristics,activitiesandresultstobeachievede.g.works

orders,digitalproductdefinitiondata,designdefinitions,arrangementdrawings,component/assemblydrawings,keycharacteristics,processflowcharts,partslists,technicalstandards,materialspecifications,processspecifications,manufacturingplans,inspectionplans,productioninstructions,processdatacards,standardoperatingprocedures,setupdiagrams,toolinglists,gagelists,machinetoolprograms,batchcards,processcards,routers,travelers,controlplans,qualityplans,testschedulesproduction/serviceparameters,serviceinstructions,servicelevelagreements,inspection/verificationprocedures,acceptancecriteria

• Availabilityofmonitoringandmeasuringresourcese.g.persons,equipment,tooling,gaging,instrumentation,facilities(includingenvironmentalcontrols),machinetool/CoordinateMeasuringMachine(CMM)probing/profiling/camera/lasersystems,softwareprograms

• Implementationofmonitoringandmeasuringactiviese.g.o Inspection/verificationplanning,identificationofinspection/verificationpoints(sequencing)o Criteriaforacceptance/rejection,specifications,standards,performanceparameters,pass/faillimits,acceptancestandards,

visualaidso Equipmentrequirements(tools,gages,fixtures,machinetools,CMMsetc.)andinstructionsforuse(setupsheets,visualaids,

softwareprograms,inspection/verificationprocedures,operatinginstructions,standardoperatingprocedures,operatortrainingetc.)

o Retainedmeasurementresultse.g.inspection/verificationhistorycards,CMMreports,pass/failindicators,testresults,verificationreports,electronicdatacapture,FAI(see8.5.1.3)

o Samplinge.g.riskassessment,statisticalrationale/basis(AcceptanceQualityLimits(AQL),UnacceptableQualityLevel(UQL),establishedprocesscapability),samplingplan(producttype,features,characteristics,batch/lotsize,samplesize,inspectionlevels,typeofinspection),defectiveproduct(hold,reinspect,reprocess)

• Operatinginfrastructureandenvironment(see7.1.3&7.1.4)e.g.

Page 41: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 41 Revision:2.1

o Buildings(manufacturing/assemblyplants,testfacilities,laboratories,servicecenters,officesetc.)o Utilities(electricity,gas,water,compressedairetc.)o Equipment(machinetools,jigs,fixtures,tooling,molds,workstations,materialhandlingetc.)o ITinfrastructure(networks,servers,backupsystems,hardware,softwareprogramsetc.)o Physicalenvironmente.g.space,ergonomics,workplaceorganization(5S),visualmanagement(layout,zoning,signingetc.)

cleanliness,restrictedareas,electrostaticdischarge,lighting,temperature,humidity,ventilation,noise,FODprevention,wastereduction(Lean)

• Appointmentofcompetentpersons(see7.2)e.g.competencyrequirements(education,training,skills,knowledge,experience),alignmenttorole(roleprofiles,jobdescriptions,accountabilitystatements,skillsmatrices),evidenceofability(assessment,evaluation,qualification,requalification)etc.

• Validationandrevalidationofspecialprocesses(see8.5.1.2)e.g.requirements(standards,specifications,instructions),validationactivity(methods,techniques,instructions,datacards,acceptancecriteriaetc.),approvalarrangements(facilities,plant,equipment,process,documentsetc.),processmonitoring,equipmentcalibration,personqualification(training,skills,assessment,levelsofauthorizationetc.),periodicrevalidationincludingchangecontrol

• Preventionofhumanerrore.g.riskassessment,training,competencyassessment,qualification,automation,errorproofing,mistakeproofing,humanfactors,effectivecorrectiveaction,readacross

• Release,deliveryandpostdeliveryactivities(see8.5.5&8.6)e.g.o Release/deliverye.g.verificationthatrequirementshavebeenmet,customeracceptancetesting(ifrequired),product

certification/qualification,approvalfromtherelevantauthoritywhenrequirementshavenotbeenmet(concession,deviation,waiveretc.),releaseauthorizations,certificatesofconformity,releasecertificates,regulatorycertificates(EASAForm1,FAA8130etc.),dispatchdocuments(consignmentnotes,deliverynotes,shippingnotesetc.),exportcontrollicenses

o Postdeliverye.g.requirements(organization,customer,statutory,regulatory),dealingwithcustomerfeedback(communication,compliments,complaints,issues,problemresolution,satisfaction,warranty,returns,rejections),recallnotices,product/servicesupport(queries,training,aftersales,maintenance,servicing),collection/analysisofin-servicedata,lessonslearned

• Workmanshipcriteriae.g.standards,samples,artifacts,visualaids,photographs,videos,diagrams,illustrations,comparitors,acceptance/rejectioncriteria

• Accountabilityforproductsduringproductione.g.parts/batchquantities,serialisation,quantityadjustments,(splitbatches/orders,nonconformingproducts),MRPupdates,controlboardupdates,worktracking/booking,WorkInProgress(WIP)monitoring

• Criticalitemse.g.identification,traceability,production/processcontrol,monitoringandmeasurementarrangements,focusonkey

Page 42: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 42 Revision:2.1

characteristics,fixedprocessapprovals,productqualification,changecontrol• Methodstomeasurevariabledatae.gverification/inspectionplans,methods,instructions,techniques,identificationofgenerictooling,

equipment,gages,instrumentation,identificationofunique‘specialtoproduct’equipment,applicationofmachinetool/CMMprobing/profiling/camera/lasersystems

• Planningforinprocessinspection/verificationwhenverificationcannotbeperformedatalaterstageforexamplepriortoprocessingwhenfeaturesbecomehidden(coating,painting,surfacetreatment),priortoassemblywhenfeaturesbecomeinnaccessible(sub-assemblies,finalassemblies,modules,finalproduct)

• Evidencethatalloperationshavebeencompletede.g.accountingforworkviatheuseofsignatures,stamps,electronicworkbookingetc.,orotherwisedocumentedandauthorizedwherecompletionofanoperationisnotinaccordancewithinstructionse.greason,dispositionandauthorization/approval

• Prevention,detectionandremovalofFODe.g.o Prevention(campaigns,signage,briefings,training,bulletins,alerts,riskassessment,controlplans,lessonslearnedetc.)o Detection(inspection,instrascopes,boroscopes,x-raytechniques,customereyesoverchecksetc.)o Removal(cleaning,purging,disassemblyetc.)

• Monitoringofutilitiesandsupplies(electricity,gas,water,compressedair,chemicalproductsetc.)thathaveanimpactontheproductincluding;contamination,cleanliness,chemicalstrength/constituency,pressure,voltagesupplyetc.andcontrolarrangmentse.g.instructions,procedures,frequencyofchecks,analysisofresults,corrections/adjustments

• Productsreleasedforsubsequentproductionusependingcompletionofmeasuringandmonitoringactivitye.g.identification,communication,documentannotation,traceability,authorizationtoproceed,electronicsystem(MRP)updates,tracking/recallarangements

8.5.1.1 ControlofEquipment,Tools,andSoftwareProgramsThingstoconsider:• Organisationarrangementstocontrolequipment,toolsandsoftwareprogramsusedwithinproductionincluding:

o Validateequipment,toolsandsoftwareprogramse.g.instructions,validationcriteria,capabilitystudies,productiontrials,prototypes,testpieces,tooling/equipmentproving,componentproving,productionacceptance,PFMEA,GageR&R,controlcharts,productionprocessverification(see8.5.1.3),acceptancetesting,equipmentcommissioning,verificationresults,releasetoproduction

o Maintainequipment,toolsandsoftwareprogramse.g.identity,configuration(version/issuecontrol),changecontrol(includingauthorization),plannedmaintenance,servicing,calibration,softwareprotectionandbackup

Page 43: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 43 Revision:2.1

o Storageofequipmentandtoolinge.g.identity,environmentconditions(space,cleanliness,housekeeping,temperature,humidityetc.),preservationandprotection(corrosion,deterioration,degradation,damage/lossprevention),assetregisters,conditionchecks,segregation(includingequipmentandtoolingbelongingtoothers–see8.5.3)

8.5.1.2 ValidationandControlofSpecialProcessesThingstoconsider:• Organisationarrangementsforvalidationandcontrolofspecialprocesses(e.g.heattreatment,chemicalprocessing,composites,

nonconventionalmachining,nondestructivetesting,joining,coating,surfaceenhancement)including:o Criteriaforprocessreviewandapprovale.g.technicalrequirements,verificationandvalidationactivity,approvalarrangements

(process,documents,personsetc.)o Conditionstomaintainprocessapprovale.g.compliancechecks,processoversight,specialprocessaudits,requalificationof

persons,plannedmaintenance,processmonitoring,processsurveillance,changecontrolo Approvaloffacilitiesandequipmente.g.plant(furnaces,machinetools,processlinesetc.),equipment,tooling,process

consumables,maintenanceregime,calibrationrequirements,testpieces,standards,specificationso Qualificationofpersonse.g.skillsmatrices,training,oversight,workplaceassessment,competencyassessment,levelsof

authorizationo Methodsandprocedurese.g.techniques,instructions,datacards,standardoperatingprocedures,diagrams,setupsheets,

acceptancecriteriao Retaineddocumentedinformation(thosedocumentsneededtodemonstratethatprocesseshavebeencarriedoutasplanned)

e.g.personqualification,processcards,routecards,furnacecharts,laboratoryreports,testpieces,testresults,x-rayfilm,maintenancerecords,calibrationrecords,consumabletraceability,verificationresults

8.5.1.3 ProductionProcessVerificationThingstoconsider:• Organisationarrangementstoverifythattheproductionprocessisabletoproduceproductsthatmeetrequirements,includingfor

example:o Capacity/capabilitystudiese.g.determinationofproducibility,methods,trials,tooling/equipmentproving,componentproving,

productionratereadiness,retaineddocumentedinformation(verificationresults,assessment/analysisreports,data,informationetc.)

o ProductionProductApprovalProcess(PPAP)e.g.definedproductionrun,controlplan/methods,resources,product/processcharacteristics,statisticalprocesscontrol,riskassessment(PFMEA),measurementsystemanalysis(GageR&R),retained

Page 44: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 44 Revision:2.1

documentedinformation(timingplan,controlplan,verificationresults,data,information,PPAPfileetc.)o Firstarticleinspection(FAI)e.g.:

§ Provisionofresources(persons,facilities,equipmentetc.)§ SelectionofarepresentativeFAIitem(newproductintroductionorexistingproductsubjecttochange)§ IdentificationoftheFAIitem(PlanningfortheFAIitem,MRPsignalfortheFAIitem,uniqueidentifieretc.)§ Verification/inspectionactivity(requirements,characteristics,features,specialprocesses,materials,softwareprograms,

gages,tooling,equipment,drawings,specificationsetc.)§ Provisionforre-verification(requirementsnotachieved,identifiednonconformance,selectionofalternativeitem)

• Retaineddocumentedinformation(inspectionresults,testcertificates,certificatesofconformity,laboratoryreports,gage/equipmenttraceability,methodofmanufacture,acceptanceetc.)

8.5.2 IdentificationandTraceabilityThingstoconsider:• Organizationsuseofsuitablemeanstoidentifyproductsandservicese.g.physicalpartmarking,labeling,tags,barcodes,signage,visual

indicators,partsegregation,laydownareas,storageracks• Identificationoftheactualversusrequiredconfigurationoftheproducts/servicese.g.MRPsystem,BOM,partslist,revisionstatus,

changecontrol• Controlarrangmentsforacceptanceauthoritymediaforexample:

o Signaturese.g.recognizedimpressionalignedtoanauthorizedsignatorylisto Stampcontrole.g.uniqueidentification,stampissue,scopeofauthority,stampwithdrawal,registerofholders,stamplegibilityo Electronicsignaturese.g.userid,passwords,permissions,restrictions,userprofile,scopeofauthority

• Uniqueidentificationwhentraceabilityisarequirement,togetherwithretaineddocumentedinformationtoenable:o Identificationandtraceabilitythroughouttheproductlifee.g.lifecycleplanning,acquisitionofrawmaterials,manufacturing

history,serialization,batchcontrol,configurationstatus,productrelease,productionscrap,maintenancehistory,in-servicehistory,endoflifedisposal

o Traceabilityofassemblies(components,sub-assemblies,modules,finalassembliesetc.),BoM,configurationcontrol,assemblytrees,MRPsystems

o Retrievalofproduction(manufacture,assembly,inspection,testetc.)recordse.g.arrangementsforstorage,whereabouts,location,availability,methodofretrieval,permissions,timeliness

Page 45: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 45 Revision:2.1

8.5.3 PropertyBelongingtoCustomersofExternalProvidersThingstoconsider:• Organizationsarrangementsrelatingtopropertybelongingtocustomersorexternalprovidersincludingforexamplematerials,

consumables,parts,tooling,equipment,facilities,intellectualproperty(standards,specification,drawings,manuals,data,etc.),returnablepackaging,productsreturnedforwarranty/servicing/investigation

• Controlofpropertybelongingtocustomersorexternalproviderse.g.ownership,identification,managementofinventory,verification,storage,protection,preservation,restrictions,security,segregation

• Retentionofdocumentedinformationandreportingoccurrenceswhenpropertyisfoundtobelost,damagedorunsuitableforusee.g.missing,broken,unserviceable,inventorydiscrepencies,shelflifeexpiration,deterioration,illegibility,obsolescence,misconfiguration

8.5.4 PreservationThingstoconsider:• Organizationapproachtowardspreservingoutputsduringproductionorserviceprovision,includingforexample:

o Identificatione.g.markings,labelling,tags,barcodes,routers,configurationstatus,traceabilityindicatorso Generalhandlinge.g.instructions,equipmentavailability,training,damagepreventiono Contaminationcontrole.g.instructions,preventionofforeignobjects(FOD),protectionarrangements(corrosion,deteroriation,

degradation),damageprevention,preventionofcrosscontamination(materials,consumables,mistakeproofing)o Packaginge.g.instructions,materials/equipmentavailability,training,customerrequirements,FODprevention,re-preservation

arrangements(timelimitations),reusablecontainerso Generalstoragee.g.stockholdingmethods,environmentcontrols(temperature,humidity,cleanliness),stockrotationmethods

suchasfirstinfirstout(FIFO),inventorycontrolchecks,conditionchecks,shelflifecontrol,accessrestrictions,security,housekeeping

o Transmissionortransportatione.g.internal/externalmovements,handling,protection,movementtickets,transportlabels,deliverynotes,specialinstructions,electronictransfers,trackingandtraceability

o Protectione.g.instructions,cleaning,useofinhibitors,preventionoflossordamage,materials/equipmentavailability• Organizationapproachtowardspreservingoutputsinaccordancewithspecificationsandapplicablestatutoryandregulatory

requirements,including:o Cleaninge.g.instructions,sequencing,resourceso Controlofforeignobjects(FOD):

§ Preventione.g.campaigns,signage,briefings,training,bulletins,alerts,riskassessment,controlplans,lessonslearned,

Page 46: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 46 Revision:2.1

§ Detectione.g.inspection,instrascopes,boroscopes,x-raytechniques,customereyesoverchecks,§ Removale.g.cleaning,purging,disassembly

o Specialhandlingandstorageforsensitiveproductse.g.storagelife,environmentcontrols(temperature,humidity,cleanliness),electrostaticsensitivedevices(ESD),cleanrooms,accessrestrictions,security,segregation

o Marking,labeling,safetywarningsandcautionse.g.identification,instructions,notices,advisory’s,signageo Shelflifecontrolandstockrotatione.g.FIFO,labeling,expirydates,packagingindicators,inventorycontrolchecks,stockcontrol

systemo Specialhandlingandstorageforhazardousmaterialse.g.riskassessment,segregation,accessrestrictions,security,

housekeeping,environmentcontrols(temperature,humidity,cleanliness),personalprotectiveequipment(PPE),recordsofinventory,controlofsubstanceshazardoustohealth(COSHH)

8.5.5 Post-DeliveryActivitiesThingstoconsider:• Organizationdeterminationofpost-deliveryactivitiestakingintoaccount:

o Statutoryandregulatoryrequirementse.g.legislation,regulations,directiveso Undesiredconsequencese.g.riskassessment,impactanalysis,failureinservice,recallnotices,warrantyconsiderationso Product/serviceusee.g.product/servicesupport,lifecyclemanagement,aftersales(maintenance,sparepartsprovision,help

line,callcenters),technicaldocumentationo Customerrequirementse.g.contractualarrangements,installation,commissioning,handover,training,customersupporto Customerfeedbacke.g.surveyresults,compliments,complaints,lessonslearned,voiceofthecustomer,satisfaction

indicators/ratings,returns/rejections,warrantyclaimso Collectionandanalysisofin-servicedatae.g.performance,reliability,engineeringpredictions,lessonslearned,operating

characteristics,inserviceevents,serviceintervals,servicelife,yieldrates,servicelevelagreements,servicecostso Provisionoftechnicaldocumentatione.g.technicalpublications,operatinginstructions,maintenancepolicies,maintenance,

repairandoverhaulmanuals,controlarrangements(creationissue,updates,communication,distribution)o Controlofoff-sitework(installation,commissioning,maintenance,fieldsupportetc.)e.g.liaison,communication,contracts,

orders,resources,instructions,training,qualifiedpersons,recordsofworkaccomplished,workdeviations,controlofassets(tooling,equipment,parts,consumables,data)

o Product/customersupport:§ Queriese.g.pointsofcontact,helpline,callcenters,frequentlyaskedquestions(FAQs)

Page 47: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 47 Revision:2.1

§ Traininge.g.provision(classroom,e-learning),manuals,exhibits,webinars§ Warrantiese.g.contractualobligations,servicelevelagreements§ Maintenancee.g.manuals,training,serviceintervals,resources,sparepartsprovision,repairschemes,servicebulletins§ Replacementpartse.g.attritionrates,servicehistory,predictedusage,illustratedpartscatalogues,linesidestock

(standardparts),billofmaterial(BoM),partsavailability,leadtimes§ Resourcese.g.people,materials,parts,consumables,information,equipment,financing§ Obsolescencee.g.riskmanagement,sustainability,sourcingstrategies,sourceofsupply,protectthecustomer,MRP

indicators,servicelifeplanning• Organizationarrangementsfordealingwithproblemsdetectedafterdeliverye.g.reactiontotheproblem,containment,correction,

investigation,problemresolution,correctiveaction,readacross,communication,reporting,lessonslearned8.5.6 ControlofChanges

Thingstoconsider:• Organizationapproachtowardsthereviewandcontrolofproduction/servicechangetoensurecontinuingconformitywith

requirementstakingintoaccount:o Reasonforchange(plannedandunplanned)o Temporaryorpermanentchangeo Review,assessmentandevaluationofthechangeproposalo Verificationorvalidationofthechangepriortoimplementationo Approvalofthechangebytheappropriateauthority(includingthecustomer/regulatorwhererequired)o Implementationofthechange(e.g.implementationplan,reviseddocumentedinformation,communication)o Confirmationthatthechangehasbeenincorporated,includingupdatestotheQMSasrequiredo Monitoringofeffectivenessofthechange

• Changestoproductionandserviceprovisioncaninclude,changesto:o Customer/regulatoryrequirementso Sourceandprocessmethodso Softwareusedtocontroloperationalactivity(e.g.machinetool/inspection/testprograms)o Drawings,diagrams,illustrationso Process/workinstructionso Machines,tools,jigs,fixtures,equipment,gages

Page 48: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 48 Revision:2.1

o Materials,consumableso Maintenanceregimeso Serviceintervalso Authorizedperson(s)o Specialprocessparameterso Enduserdocuments(instructions,technicalpublicationsetc.)o Handling,storage,packaging,preservationrequirementso Operator/endusertrainingo Environmentalcontrols

• Theidentificationoftheperson(s)authorizingthechangesuchas:name,signature,role,useridentification,stampimpression,etc.,togetherwiththescope,delegationsandauthoritystatusrelatingtothechange(s)beingmade.

• Theretentionofdocumentedinformationrelatingtotheauthorizedchangesuchaschangenotices,alterationrequests,coordinationmemos,revisedinstructions,qualityplan,deviationrequests,verification/validationresults,reviewminutes,implementationplan,togetherwithanyactionsarisingfromtheidentifiedchange.

8.6 ReleaseofProductsandServicesThingstoconsider:• Implementationoftheorganizationsplannedarrangementsatappropriatestagestoverifythatproductandservicerequirementshave

beenmet.Plannedarrangementscanincludedesignverification/validation(modelling,simulations,experiments,trials,prototypes,functionaltesting,performancetesting),inspection(stage,final,firstarticle),examination(destructive/non-destructive),ProductionPartApprovalProcess(PPAP),customeracceptancetesting,productcertification/qualification,thirdpartyqualification(regulator,society,independentbodyetc.),

• Ensuringthattheplannedarrangementshavebeensatisfactorilycompletedpriortoreleaseoftheproductorservice,orobtainingapprovalfromtherelevant(internal/external)authorityorcustomerwhenplannedarrangementshavenotbeencompletedpriortoreleasee.g.concession,deviation,waiver,contractamendment,dispensation,releaseendorsement

• Recordstoprovideevidencethatacceptancecriteriahasbeenmete.g.certificateofconformity,releasecertificate,regulatorycertificate(EASAForm1,FAA8130etc.)

• Traceabilitytotheperson(s)authorizingthereleasesuchasname,authorizedsignatories,useridentification,stampimpressionetc.,includingtheirauthoritystatus(releasesignatory,certifyingstaff,scopeofauthorizationetc.)

• Recordstoprovideevidencethatproductqualificationmeetsdefinedrequirementse.g.inspectionhistory,testreports,

Page 49: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 49 Revision:2.1

verification/validationreports,acceptancedocuments,qualificationcertificates• Accompanyingdocumentationatthepointofdeliverye.g.certificateofconformity,releasecertificate,regulatorycertificate(EASAForm

1,FAA8130etc.),despatchdocuments(consignmentnotes,deliverynotes,shippingnotesetc.)exportcontrollicences,userinstructions,handlinginstructions,maintenanceinstructions,logbooks,technicalpublications

8.7 ControlofNonconformingOutputs8.7.1 Thingstoconsider:

• Arrangementsforcontrollingnonconformingoutputs,includingproducts,servicesorprocessesidentifiedinternallyorexternallyincluding:

o Organizationsmaintaineddocumentedinformation(see4.4.2)thatdefinesthenonconformitycontrolprocess(e.g.documentedprocedure)including:

§ Responsibilityandauthorityforthereviewanddispositionofnonconformingoutputse.g.Authorizedsignatories,nonconformancecontrolauthorities,delegatedtechnicalauthorities,MaterialReviewBoard(MRB),customer

§ Theprocessforapprovingpersonswhodispositionnonconformingoutputse.g.organizationrole,knowledge,experience,demonstratedcompetence,qualification,requalification,includingscopeofauthority(type,product,project,serviceetc.)

§ Takingactiontocontaintheeffectofthenonconformitysuchasreaction/responsetimes,impact/riskassessment,stopandfix,readacrosstosameorsimilarproducts,servicesandprocessesthatmaybeimpacted(newproductintroduction,workinprogress,dispositionofservice,stockheldinstorage,productheldbyexternalproviders,productintransit,productalreadydeliveredtothecustomeretc.)

§ Therequirementsfortimelyreportingofthenonconformitytorelevantinterestedpartiesrelatingtoproductsorservicesalreadydeliverede.g.respectivetimescaleforreporting,responsibilityforreporting,methodofreporting(internal/externalcommunication,notifications,correctiveactionreports,recallnotices,safetyalertreports,mandatoryoccurrencereports,servicebulletins,advisory’setc.),acknowledgementofreport(s)

§ Thearrangementsfordefiningcorrectiveactionsfornonconformingoutputsdetectedafterdelivery(see10.2)i.e.reactiontothenonconformity,evaluationofnecessaryaction(s),implementationandmonitoringofidentifiedaction(s),reviewofeffectivenessandsustainmentofaction(s)taken

o Implementationoftheorganizationsdocumentednonconformitycontrolprocess(asabove)toensurethatappropriateactionistakenaccordingtothenatureofthenonconformity

o Organizationprovisionfordealingwithnonconformingoutputsincluding:

Page 50: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 50 Revision:2.1

§ Correctionse.g.enablerework(correctionlabels,updatedinstructionsetc.)toachieveconformity,includingverificationoftheconformingstatusaftercorrection(checking,inspection,testingetc.)

§ Segregatione.g.physicalseparationofnonconformingproducts,useofdedicatedholdingareas(floorspace,racking,quarantinebonds,cages),identificationtoshownonconformingstatus(labels,markings,tags,annotationofdocumentedinformation,electronicsystem(MRP)updates)

§ Containmente.g.reaction/responsetimes,impact/riskassessment,stopandfix,communication,readacrosstosameorsimilarproducts,servicesandprocesses

§ Returnorsuspensionofoutputse.g.,stopshipment,suspendservice,productrecall,freezeinventory,blockstock,holdinstruction

§ Informingthecustomere.g.communication,notifications,alerts,bulletins,advisory’s§ Acceptanceunderauthorizedconcession(includingthecustomerwhenrequired)e.g.concession,waiver,deviation,

productionpermit,servicealleviationo Implementationofuse-as-is(accepteddeviationasidentified),orrepair(acceptedrestorationtoauseablecondition)in

accordancewithadisposition(e.g.concession,waiver,deviation):§ Approvedbyanauthorizeddesignresponsiblerepresentativewithintheorganization,or§ Approvedbyperson(s)withappropriatedelegatedauthority,and§ Authorizedbythecustomerifthenonconformityhasanimpactonthecontractrequirements

o Controlofproductsdispositionedasscrapincluding:§ Conspicuousmarking(e.g.scraplabels,tags,signs,paint)andpermanentmarkings(e.g.removalofproductidentity,

applyingscrapidentificationstatus),or§ Positivecontrol(e.g.segregation,dedicatedholdingareas(floorspace,racking,quarantinebonds,cagesetc.),annotation

ofdocumentedinformation,electronicsystem(MRP)updates§ Physicallyrenderedunusablebytheorganizationoracontrolledexternalprovider(e.g.productdestruction,removalof

keyfeatures,productdeformation,productmutilation,conversionbacktorawmaterial(revert,recycling)andcontrolofanyinherenthazadousmaterial/substances(asrequired)

o Controlofcounterfeitorsuspectcounterfeitparts(see8.1.4),topreventre-entryintothesupplychaine.g.labeling,marking,segregation,quarantine,containment,disposition,annotationofdocumentedinformation,electronicsystem(MRP)updates

8.7.2 Thingstoconsider:• Organizationarrangmentsforretainingdocumentedinformationthat:

Page 51: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 51 Revision:2.1

o Describesthenonconformitye.g.statements,illustrations,reports,objectiveevidenceo Describesactionstakene.g.dispositionsrelatingtocorrection,concession,scrapo Describesanyconcessionsobtainede.g.acceptedconcession,waiver,deviation,productionpermit,servicealleviationo Identifiesthedecidingauthoritye.g.authorizedsignatories,nonconformancecontrolauthorities,delegatedtechnical

authorities,MaterialReviewBoard(MRB),customer

Page 52: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 52 Revision:2.1

Clause# Considerations9. PERFORMANCEEVALUATION9.1 Monitoring,Measurement,AnalysisandEvaluation

9.1.1

GeneralThingstoconsider:• Identificationof“what”needstobemonitoredandmeasured,forexample:

o QMSandprocessperformancee.g.achievementofqualityobjectives,processperformanceindicators,targetsetting,auditresults(conformity/non-conformity,effectiveness),trends

o Operationalperformancee.g.product/serviceconformity,productionyield,rightfirsttime,partspermillion(PPM),defectsperunit(DPU),productcharacteristics,processparameters,processcapability,identificationofwaste,ontimedelivery,queuingtimes,resolutionofcustomerissues,serviceavailability,in-servicereliability,averageservicetime

o Deliveryofprojectstoplane.g.spendagainstbudget,milestoneachievemento Externalproviderperformance,e.g.product/serviceconformity,ontimedelivery,approvalratings,returns/rejections,invoice

querieso Customerperformancee.g.surveyresults,satisfactionindicators/ratings,compliments,complaintprofile,returns/rejections,

warrantyclaims,invoicequerieso Managementofriskandopportunitiestopreventandreduceundesiredeffects,enhancedesiredeffectsandachieve

improvement,e.g.identification,assessment,treatment• Determinationofthe“methods”usedformonitoring,measurement,analysisandevaluation,forexample:

o Datacaptureinstructions,e.g.procedures,qualityplans,datacollectionsheets,dataacquisitionsoftware,samplingtechniques,frequency

o Statisticalprocesscontrol(SPC)includingruncharts,histograms,controlcharts,Paretodiagrams,scatterdiagrams,causeandeffect(fishbone)diagrams,glyph/radarcharts,designofexperiments

o Projectearnedvalue,includingcostperformanceindex(CPI)andscheduleperformanceindex(SPI)o Managementofrisk,e.g.riskavoidance,riskelimination,takingrisk,risktreatment,risktransferandtakingrisktopursue

opportunitieso Performancegovernance,e.g.managementreview,processreviews/councils,performancereports,scorecards,flight

Page 53: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 53 Revision:2.1

decks/dashboards

9.1.2

CustomerSatisfactionThingstoconsider:• Determinationofthe“methods”usedforobtaining,monitoringandreviewingcustomerperception,forexample:

o Customersurveys/questionnairesconductedbytheorganizationitselforviaanindependentresearchagency.NOTE:surveysmayrelatetoallcustomersorarepresentativesampleandthefrequencymayvaryaccordingtotherelationship

o Customerfeedbackrelatingtodeliveredproductsandservices,e.g.product/serviceconformity,ontimedelivery,reportcards,satisfactionindicators/ratings,compliments

o Customercommunication,e.g.facetofacemeetings,telephonecalls,useoffieldoperatives(installation/servicepersonnel),daytodayenquires(viainternalpersonnel),useofsocialmedia(websites,messageboards,on-linehelp),reportsfromdealers,distributors,integrators

o Complaintprofileincludingrejections,returns,warrantyclaims,invoicequeries,correctiveactionrequests,o Marketshareanalysisandevaluationofrepeatbusiness

• Arrangementsfordevelopingandimplementingplanstoimprovecustomersatisfactionincluding:analysisofresultsandtrends,inputtomanagementreview,addressingidentifiedproduct/servicedeficiencies(includingimprovementstotheQMS),assessingtheeffectivenessofactiontaken

9.1.3

AnalysisandEvaluationThingstoconsider:• Analysisofappropriatedataarisingfrommonitoringandmeasurement(see9.1.1)inordertoevaluate:

o QMSandprocessperformancee.g.achievementofqualityobjectives,processperformanceindicators,targetsetting,auditresults(conformity/non-conformity,effectiveness),trends

o Operationalperformance,e.g.product/serviceconformity,productionyield,rightfirsttime,partspermillion(PPM),defectsperunit(DPU),productcharacteristics,processparameters,processcapability,identificationofwaste,ontimedelivery,queuingtimes,resolutionofcustomerissues,serviceavailability,in-servicereliability,averageservicetime

o Externalproviderperformance,e.g.product/serviceconformity,ontimedelivery,approvalratings,returns/rejections,invoicequeries

o Customerperformance,e.g.surveyresults,satisfactionindicators/ratings,compliments,complaintprofile,returns/rejections,warrantyclaims,invoicequeries

o Ifplanninghasbeenimplementedeffectively,e.g.actionstoaddressrisksandopportunities,achievementofqualityobjectives,

Page 54: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 54 Revision:2.1

planningofchangesandoperationalplanningo Informationonproductandserviceproblemsreportedbyexternalsources,e.g.alerts,advisories,directives,servicebulletinso Deliveryofprojectstoplan,e.g.spendagainstbudget,milestonedelivery,earnedvalue,costperformanceindex(CPI),schedule

performanceindex(SPI)o Effectivenessofactionstakentoaddressrisksandopportunities,e.g.riskassessment,risktreatment,residualriskvalues,

performancetrends,adoptionofnewpractices,launchingnewproducts/services,winningnewcustomers• Usingthedocumentedoutputofanalysisandevaluation,e.g.trendanalysisandreportsasaninputtomanagementreview(see9.3.2)

toenabletheorganizationtodeterminetheperformanceandeffectivenessoftheQMSandidentifyimprovementsNOTE:Analysisofthedatacanbeundertakenusingstatisticaltechniquestohelpbreakdowntheinformationandpresentitinaformatthatallowsadeterminationtobemadeofwhetheractionsareneeded.

9.2 InternalAudit

9.2.1

Thingstoconsider:• OrganizationarrangementsforconductinginternalauditsatplannedintervalstodemonstratethattheQMSconformsto:

o Theorganizationsownrequirements,e.g.policies,processes,procedures,instructions,specificationso Customerrequirements,e.g.floweddownbycontracto Statutoryandregulatoryrequirements,e.g.legislation,regulations,directiveso Applicableexternalstandards(including9100)

• OrganizationapproachtoknowinghowwelltheQMShasbeeneffectivelyimplementedandmaintainedforexampleby:o Obtainingauditresultso Reportingauditperformanceo Monitoringtrends,e.g.processstrengthsandweakness,repeatauditfindings,acknowledgedimprovement

9.2.2

Thingstoconsider:• Publicationofanauditprogram,oranumberofauditprogramsdependantuponthesizeandcomplexityoftheorganization,including

theidentification(title,scope,criteria)andfrequencyofeachaudit,e.g.monthly,quarterly,annually• Organizationmethodsusedforundertakingauditsforexampleauditselection,auditplanning,auditconduct,auditreporting,corrective

action,closeout,processmeasures,e.g.auditdeliveryagainsttheschedule,timetorespondtoanonconformity,timetocorrectanonconformity

• Applicationofriskbasedthinkingtoselectauditsandtheirfrequencyforexampleconsiderationof:

Page 55: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 55 Revision:2.1

o Processesthatarecriticaltoproductandservicequalityo Complexprocessesthatrequireclosemonitoringandcontroltoensureconformityo Balanceacrossoperationalandnonoperationalprocesseso Processesthatutilizequalifiedpersonnelo Activitiesorprocessesthatoccuracrossmultiplelocationso Processesimpactedbyhumanfactorso Introductionofneworchangedprocesseso Changesaffectingtheorganizationo Statutoryandregulatoryissueso Processperformance,e.g.processconformity/nonconformity,escapestothecustomer,complaints,previousinternal/external

auditresults,identifiedrisk(see6.1and8.1.1)• Definitionofthecriteria,(policy,process,procedure,requirements)andscope(extent,boundaries,physicallocation,productline,

facility,department,activities,duration)foreachidentifiedaudit• Managementcommitment,authorizationandcontroloftheauditprogram(s)frominitialissue,togetherwiththeidentificationand

reason(s)forchange• Conductingofauditsontimeinaccordancewiththeschedule(previousandcurrentperiods)• Auditorresourcesincludingtraining/qualificationandselectionofauditorstoensureindependenceofthearea/processbeingaudited• Collectionofauditevidenceandreportingofauditfindingsincludingcorrectgradingoffindings(conformity,nonconformity,strengths,

opportunitiesforimprovement)• Routinereportingofauditresultstotherelevantmanagement(circulationofauditreport,closingmeeting/outbriefs,auditperformance

metrics),includinganinputtoManagementReview(see9.3.2)• Timelyundertakingofcorrections(fixtheissue)andcorrectiveactions(preventrecurrence)bytheorganization• Followupbytheauditortoensureeffectiveandtimelyimplementationofcorrectionandcorrectiveaction• Retentionofdocumentedinformationtoshowthatthattheauditprogramhasbeeneffectivelyimplemented(auditprogram,audit

plans,auditreports,questionnaires,auditevidence,correctiveaction,auditcloseout,auditorqualification,auditperformancemetrics)NOTE:Auditinformationcanbeinhardcopyformat,ormanagedwithinanITsystem

Page 56: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 56 Revision:2.1

9.3 ManagementReview

9.3.1

GeneralThingstoconsider:• TopmanagementarrangementsforreviewingtheQMSatplannedintervalstoensurecontinuing:

o Suitability(fitforpurpose)o Adequacy(meetstheneedsoftheorganization)o Effectiveness(achievesintendedresults)

• Frequencyofreview,e.g.monthly,quarterly,sixmonthly,annual• Standalonerevieworcombinedwithotherbusinessactivities,e.g.strategicplanning,businessplanning,operationsmeetings,process

reviews/councils,functionalreviews• Representationatthereview,e.g.topmanagement,functionalmanagement,linemanagement,processowners,processchampions,

leadprocessusers,actionowners

9.3.2

ManagementReviewInputsThingstoconsider:• Statusofactions(open/closed)frompreviousmeeting(s),ageingprofileofopenactions,e.g.3months,6months,12months,greater

than1year• Changesarisingfrommonitoringinternal/externalissuesthatarerelevanttotheQMS(see4.1)• Identificationandevaluationofchangestointernal/externalrequirements,e.g.policies,processes,procedures,methods,instructions,

contracts,regulation,legislation,thatimpacttheQMS• QMSperformanceandeffectivenessincluding:

o Customersatisfaction(see9.1.2)andfeedbackfromotherinterestedparties,e.g.reportcards,indicators,ratings,complaints,compliments,mediareports

o Achievementofqualityobjectives(see6.2),includingstatusofplannedversusactualachievemento Organizationandexternalproviderprocessperformanceandproduct/serviceconformity(see4.4,8.4and8.6),e.g.flightdecks,

dashboards,scorecards,performanceindicators,performancetrends,rightfirsttime,ontimedelivery,escapestothecustomer,complaintprofile,returns/rejections

o Nonconformityandcorrectiveaction(see10.2)e.g.Paretoofnonconformitybytype,process,area,rootcauseetc.,readacrosstootherpartsoftheorganizationortoexternalparties,updateofrisks(see6.1and8.1.1),statusofcorrectiveactionimplementation

Page 57: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 57 Revision:2.1

o Auditresults(see9.2),e.g.achievementoftheauditprogram(s),areasofgoodpractice,nonconformityprofile(number,type,process,area,significance),statusofcorrectiveaction,auditcloseout,externalauditfindings

o Adequacyofinternalresourcesandexternalproviders(see7.1)includingpeople(number,roles,competencyetc.),infrastructure(buildings,equipment,systems,transportetc.),workingenvironment(physicalandhumanfactors,monitoringandmeasuringequipment(availability,fitforpurpose,maintained)

o Effectivenessofactionstakentoaddressrisksandopportunities(see6.1),e.g.riskprofile,riskregister(s),statusofopen/closedactions,ageingprofileofopenactions,evaluationofeffectiveness(enhancedesirableeffects,prevent,reduceundesiredeffects,demonstratedimprovement)

o Identificationofopportunitiesforimprovement(see10.1),correctiveaction,goodpractice,bestpractice,innovation,lessonslearnedetc.,includingreadacrosstheorganizationandtoexternalproviders

9.3.3

ManagementReviewOutputsThingstoconsider:• Decisionsandactionsrelatingto:

o Opportunitiesforimprovement(see10.1)e.g.readacrossandimplementationofcorrectiveaction,goodpractice,bestpractice,innovation,lessonslearnedetc.

o ChangestotheQMS(see6.3),e.g.policies,processes,procedures,methods,instructionso Resourceneeds(see7.1),e.g.people(number,roles,training),infrastructure(buildings,equipment,systems,transport),

workingenvironment(physicalandhumanfactors),monitoringandmeasuringequipment(availability,fitforpurpose,maintained)

o Identifiedrisks(see6.1and8.1.1),e.g.addingnewrisksatanorganizationoroperationalleveltotheappropriateregister(s)andtheassignmentofresponsibilityandtreatment

• Documentedevidenceofthereview,e.g.attendance,agenda,presentations,meetingminutes,actions(list,owners,timescale),reports• Retentionofdocumentedinformationofthereviewandcommunicationoftherelevantcontentacrosstheorganization(see7.4)

Page 58: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 58 Revision:2.1

Clause# Considerations10. IMPROVEMENT

10.1

GeneralThingstoconsider:• Organizationarrangementsforidentifyingopportunitiesforimprovementandtakingactioninorderto:

o Achieveintendedresultse.g.usinganalysisandtrendingoforganizationalperformance(see9.1.3)o Meetcustomerrequirementsandexpectationse.g.understandingcontractualarrangementsandimpliedcustomer

expectationso Enhancecustomersatisfactione.g.usingsatisfactionindicators/ratings,complaintsprofile,returns/rejections,warrantyclaims

(see9.1.2)• Organizationapproachtowardsundertakingimprovement,forexample:

o Focusonimprovingproductsandservices,includingfutureneedsandexpectations,e.g.investmentinlatesttechnologyandinnovation,improvingreliability,improvingyield,reducingcost,improvingon-timedelivery

o Correcting,preventingandreducingundesiredeffects,e.g.investigatingrootcause,preventingescapestothecustomer,actingoncustomerfeedbackand/orinservicereports

o ImprovingtheperformanceandeffectivenessoftheQMS,e.g.actingonprocessperformanceresults,understandingauditfindings,reducingwaste,processre-engineering,structuralre-organization,promotingbreakthroughprojects

10.2 NonconfomityandCorrectiveAction

10.2.1

Thingstoconsider:• Organizationsmaintaineddocumentedinformation(see4.4.2)thatdefinesthenonconfomityandcorrectiveactionmangement

processes,e.g.documentedprocedure• Organizationsreactiontononconformitieswhentheyarisefromsourcessuchasnonconformingproduct(identifiedduring

manufactureorpostdelivery),customercomplaints,auditfindings,warrantyclaims,etc.including:o Takingactiontocontrolandcorrectthenonconformity,e.g.understandingtheproblem,applyingimmediatecontainment

(protectthecustomer),undertakingcorrectionandcorrectiveaction.o Dealingwiththeconsequences,e.g.communicationwithintheorganization,contactinginterestedpartiesifrequired

(customers,regulators,externalproviders),applyingriskassessment

Page 59: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 59 Revision:2.1

• Organizationsevaluationoftheneedtotakeactiontoeliminatethecauseofthenonconfomitytoensureitdoesn’texistelsewhere

andtopreventrecurrenceby:o Reviewingandanalyzingthenonconformitye.g.describingtheproblem(who,what,when,where,how,howmany),assigning

aproblemowner,launchinganinvestigation,collectinginformation,engagementwithotherswithproduct/processknowldegeo Usingappropriatemethodologies,e.g.rootcauseanalyisis,5Whys,Ishikawa(causeandeffectdiagram),Paretocharts,

processflowdiagram,8disciplinesofproblemsolving(8d),9stepstoproblemsolving(9s),FailureModeandEffectAnalysis(FMEA)etc.todeterminethecause(s)ofthenonconformity(includingthoserelatedtoHumanFactors)andextentoftheaction(s)needed,baseduponthepotentialeffectofthenonconformity

o Determiningifsimilarnonconformitiesexistorcouldoccurelsewheree.g.communicationwithinandoutsidetheorganization(asappropriate),verificationandquarantine(ifrequired)ofotherproductsthatcouldpotentiallybeimpacted(workinprogress,productheldbyexternalproviders,stockheldinstorage,productalreadydeliveredtothecustomer,productintransit,productrecall)

• Implementationandmonitoringofidentifiedaction(s)throughtheuseofactionplan(s),owner(s),resources,tasks,timescales,managementreviewetc.toensure:

o Timelycorrectiveactionistakenwithintheorganizationandanyexternalprovider(s)wherenecessaryo Furtheractionistakenwhentimelyandeffectivecorrectiveactionisnotachieved

• Reviewingtheeffectivenessofcorrectiveaction(s)taken,e.g.observingtheperformanceoftheprocess,reviewingdocumentedinformation,conductingcompliancechecks,orauditstoensure:

o Therootcausewasaddressedo Thatnootherissueswerecausedbythecorrectiveaction

NOTE:Establishingtheeffectivenessofcorrectiveactionmaytakeanappropriateamountoftimeaftercorrectiveactionhasbeenimplemented,toensurethatithasbeensustained• Conductingafollowuptoensuresustainment:

o Planningisupdatedifnecessary(see6.1)tocaptureanyidentifiedrisksandopportunitiesthatwerenotpreviouslyconsideredo ChangesareincorporatedintotheQMSifnecessary,(processdocuments,procedures,workinstructions,controlplan,service

plan,drawings,resourcesetc.)10.2.2 Thingstoconsider:

Page 60: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 60 Revision:2.1

• Organizationarrangmentsforretainingdocumentedinformationtoshowthe:o Natureofthenonconfomities,e.g.source,type,quantity,frequency,severityo Action(s)taken,e.g.correctiveactiondocuments,actiontrackers,managementreviewoutputs,lessonslearned,riskregisters,

on-linesystemso Resultsofcorrectiveaction,e.g.openversusclosed,achievementofmilestones,performancemeasures(timetoclose,

problemrecurrence,costofnonquality)

10.3

ContinualImprovementThingstoconsider:• Organizationarrangementsforcontinuallyimprovingthesuitability,adequacyandeffectivenessoftheQMSconsideringthe:

o ResultsofAnalysisandEvaluation(see9.1.3)o OutputfromManagementReview(see9.3.3)

• Determinationofimprovementactivitybasedupon:o Externaldrivers,e.g.regulatorychanges,legislation,marketexpectations,competitoractivity,interestedpartyperceptions,

customersatisfaction,environmentalimpact,benchmarkingofbestpracticeo Internaldrivers,e.g.businessstrategy,achievementofobjectives,problemresolution,business/processperformance,process

capability,costreduction,reduceleadtime,right-first-time,on-timedelivery,improvedefficiency,lessonslearned,reducevariation,riskreduction,operationalbudgets,priortization,employeesuggestions

• Useofappropriatemethodologiesandtoolstoundertakecontinualimprovementincludingforexample:o Plan,Do,Check,Act(PDCA)approachtocontrolandcontinuouslyimprovemeprocessesandproducts:

§ Plan:definetheproblem,collectrelevantinformation/data,understandcurrentperformance,determinetherootcause,setimprovementobjectives/target,decideontimescale,determineresourcesandbudget

§ Do:developandimplementasolution,introduceaformofmeasurementtodeterminetheeffectivenessofthesolution,monitorprogress,reviewagainsttheplanandmakeanynecessaryadjustments

§ Check:measureperformance,compareoutcomeagainstexpectations,verifytoconfirmexpectedresultshavebeenachieved

§ Act:documenttheresults,communicatetheoutcome,makerecommendationsbasedonresultsachieved,assessandrepeatthecycleifrequired,updatethesystemsanddocumentationtosustaintheimprovement

o Leanapproachtoimprovecycletime,optimizeresourcesandeliminationofwasteusingsuchtechniquesas:workplaceorganization(5S),visualfactory,valuestreammapping,kaizenactivity,OverallEquipmentEffectiveness(OEE),TotalProductive

Page 61: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 61 Revision:2.1

Maintenance(TPM),takttime,Just-in-Time(JiT),mistakeproofingetc.o Six-Sigmaapproachtoimproveprocesscapability,removingcausesofdefectsandreducevariationusingsuchtechniquesas:

theDefine,Measure,Analyze,Improve,Control(DMAIC)model,Define,Measure,Analyze,Design,Verify(DMADV)model,processmapping,Supplier,Input,Process,Output,Customer(SIPOC)diagram,causeandeffect(fishbone)diagrams,MeasurementSystemsAnalysis(MSA),gageR&R,runcharts,histograms,controlcharts,Paretodiagrams,scatterdiagrams,,glyph/radarcharts,DesignofExperiments(DoE),FailureModeandEffectsAnalysis(FMEA)etc.

• Organizationarrangementsformonitoringtheimplementationofimprovementactivitiesandtheeffectivenessoftheresults,e.g.dashboards,flightdecks,registers,milestonecharts,actiontrackers,performanceindexesandtrends,benefitscapture,on-linesystems,formalreporting,inputtomanagementreview

Page 62: AS/EN/JISQ 9100:2016 EVALUATION GUIDANCE MATERIAL · 2019-10-02 · AS/EN/JISQ 9100 2016 (Rev D) Evaluation Guidance Material 3 Revision: 2.1 1.0 Introduction This document provides

©2017IAQGAllrightsreserved

AS/EN/JISQ91002016(RevD)EvaluationGuidanceMaterial 62 Revision:2.1

4.0 ChangeControl

Revision Detail Date1.0 Initialissuetosupportimplementationof9100:2016includingsections1,2and3(clauses4,5,6,9,

10)15Dec2016

1.1 Section3updatedwithnewcontentforclauses7.1,7.2,7.3,7.4,7.5,8.1,8.2and8.3. 13Apr20172.0 Section3updatedwithnewcontentforclauses8.4,8.5,8.6,8.7,contentpageaddedandwhole

documentupdatedtolandscapeversion20Jul2017

2.1 Documenttitlechangedto‘EvaluationGuidanceMaterial’andIAQGcopywritestatementadded 17Oct2017