trial sms assessment 4 th arast meeting bangkok, thailand 21 to 25 february 2011
TRANSCRIPT
![Page 1: Trial SMS Assessment 4 th ARAST Meeting Bangkok, Thailand 21 to 25 February 2011](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649e9d5503460f94b9e154/html5/thumbnails/1.jpg)
Trial SMS Assessment
4th ARAST Meeting
Bangkok, Thailand
21 to 25 February 2011
![Page 2: Trial SMS Assessment 4 th ARAST Meeting Bangkok, Thailand 21 to 25 February 2011](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649e9d5503460f94b9e154/html5/thumbnails/2.jpg)
Objective
To Report on the Trial SMS Assessment completed by COSCAP in 2010
COSCAP 4th ARAST 2011 2Trial SMS Assessment
![Page 3: Trial SMS Assessment 4 th ARAST Meeting Bangkok, Thailand 21 to 25 February 2011](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649e9d5503460f94b9e154/html5/thumbnails/3.jpg)
Background
Effective implementation of SMS is key to achieving the desired safety benefits
Service Providers will implement SMS to meet State requirements
Civil Aviation Authorities will conduct oversight to verify effective implementation
Traditional oversight audit and inspections methods measure compliance (i.e., implementation and use)
Traditional oversight audit and inspections methods not well-suited to measure effectiveness of SMS processes
COSCAP 4th ARAST 2011 3Trial SMS Assessment
![Page 4: Trial SMS Assessment 4 th ARAST Meeting Bangkok, Thailand 21 to 25 February 2011](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649e9d5503460f94b9e154/html5/thumbnails/4.jpg)
Assessment Method
Based on ICAO Safety Management Manual SMS Gap Analysis
Preparation ~~ Development of method and assessment tools Conduct of Assessment ~~ Four Parts:
Completion of a Documentation Review Guide by the Air Operator
Review of the completed Review Guide and Company documentation by the Assessment Team
On-site assessment
Report Completion
COSCAP 4th ARAST 2011 Trial SMS Assessment 4
![Page 5: Trial SMS Assessment 4 th ARAST Meeting Bangkok, Thailand 21 to 25 February 2011](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649e9d5503460f94b9e154/html5/thumbnails/5.jpg)
ICAO Safety Management Manual (DOC 9859)
Provides guidance on the features of an SMS SMM includes a “Gap Analysis” table to assist implementation “Gap Analysis” table identifies features that an SMS should
have COSCAP developed SMS “Expectations” from the Gap
Analysis
COSCAP 4th ARAST 2011 Trial SMS Assessment 5
![Page 6: Trial SMS Assessment 4 th ARAST Meeting Bangkok, Thailand 21 to 25 February 2011](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649e9d5503460f94b9e154/html5/thumbnails/6.jpg)
Gap Analysis ~~ SMS Assessment Expectations
Chapters 3, 9 &10Does the [organization] have reactive processes that provide for the capture of information relevant to safety and risk management?
Chapters 9 &10Has the service provider developed training relevant to reactive methods of safety data collection?
Chapters 9 &10Has the service provider developed communication relevant to reactive methods of safety data collection?
COSCAP 4th ARAST 2011 Trial SMS Assessment 6
Chapters 3, 9 & 103.5.3; 3.5.12; 9.9.510.4.1
3) The organization has reactive processes that provide for the capture of information relevant to safety and risk management
Chapters 9 & 104.4.6; 9.11.6; 9.11.710.4.2
4) The service provider has developed training relevant to reactive methods of safety data collection
Chapters 9 & 107.2.8; 9.11.2; 9.11.79.12.2
5) The service provider has developed communication relevant to reactive methods of safety data collection
![Page 7: Trial SMS Assessment 4 th ARAST Meeting Bangkok, Thailand 21 to 25 February 2011](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649e9d5503460f94b9e154/html5/thumbnails/7.jpg)
Assessment Documentation
Outline of Trial Assessment (pdf) Expectations (pdf) Documentation Review Guide (pdf) Assessment Worksheet (sample)
COSCAP 4th ARAST 2011 Trial SMS Assessment 7
![Page 8: Trial SMS Assessment 4 th ARAST Meeting Bangkok, Thailand 21 to 25 February 2011](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649e9d5503460f94b9e154/html5/thumbnails/8.jpg)
Assessment Conduct
Review of Company safety management documents & records
Review of actual processes Interviews with personnel (President, middle management &
staff) Observation of actual work practices
COSCAP 4th ARAST 2011 Trial SMS Assessment 8
![Page 9: Trial SMS Assessment 4 th ARAST Meeting Bangkok, Thailand 21 to 25 February 2011](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649e9d5503460f94b9e154/html5/thumbnails/9.jpg)
Evaluation of Assessment Method
Generally good Expectations
Some revision of expectations required for clarity
Eliminate some duplicate expectations Worksheets
Reduce number of sample questions
Clarify sample questions
Improve alignment of questions to their expectation
COSCAP 4th ARAST 2011 Trial SMS Assessment 9
![Page 10: Trial SMS Assessment 4 th ARAST Meeting Bangkok, Thailand 21 to 25 February 2011](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649e9d5503460f94b9e154/html5/thumbnails/10.jpg)
Evaluation of Assessment Method
Model report required Assessment Team training required Guidance for Assessment timeline required
Formal initiation by CAA
Notification of Service Provider
Service Provider requires time to prepare Review Guide (minimum 30 days)
Completed Review guide and Service provider documentation needed by Assessment Team 30 days prior to on-site review
Assessment Team face-to-face meetings to complete final review preparations
COSCAP 4th ARAST 2011 Trial SMS Assessment 10
![Page 11: Trial SMS Assessment 4 th ARAST Meeting Bangkok, Thailand 21 to 25 February 2011](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649e9d5503460f94b9e154/html5/thumbnails/11.jpg)
SMS Assessment ~~ Next Steps
Revise COSCAP SMS Assessment tools Conduct second Trial Assessment Refine models Develop related COSCAP Guidance material Distribute to COSCAP members Provide support and training as requested
COSCAP 4th ARAST 2011 Trial SMS Assessment 11
![Page 12: Trial SMS Assessment 4 th ARAST Meeting Bangkok, Thailand 21 to 25 February 2011](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649e9d5503460f94b9e154/html5/thumbnails/12.jpg)
Assessment TeamCOSCAP Kim Trethewey
Len CormierNie Junjian
AAPA Martin Eran-TaskerAirbus Bruno DelpratBoeing Dr. Bill RankinCAA Singapore Huei Wang Kang
Capt Michael HohMichael Pang
Center Aviation Safety Technology, China Zhang Yuan
CAA State of Operator Four CAA InspectorsIATA Tony HoustonKorean Air Lines Peter BlakeSR Technics Eduard Schweizer
COSCAP 4th ARAST 2011 Trial SMS Assessment 12
![Page 13: Trial SMS Assessment 4 th ARAST Meeting Bangkok, Thailand 21 to 25 February 2011](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649e9d5503460f94b9e154/html5/thumbnails/13.jpg)
Appreciation
Tremendous support from COSCAP Safety Partners Airbus Association of Asia Pacific Airlines (AAPA) Boeing Commercial Airplanes International Air Transport Association (IATA) Korean Air Lines SR Technics
COSCAP 4th ARAST 2011 Trial SMS Assessment 13
![Page 14: Trial SMS Assessment 4 th ARAST Meeting Bangkok, Thailand 21 to 25 February 2011](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649e9d5503460f94b9e154/html5/thumbnails/14.jpg)
Comments ~~ Questions
Kim Trethewey
Chief Technical Advisor
COSCAP North Asia
COSCAP 4th ARAST 2011 Trial SMS Assessment 14