from notechs to lh behaviour markers an … notechs to lh behaviour markers an implementation case...

From NOTECHS to From NOTECHS to LH Behaviour Markers LH Behaviour Markers

An Implementation Case StudyAn Implementation Case Study

Cpt. Harry Neb,Lufthansa German AirlinesFRA PC/LHuman Factors Operational Input

BR&TE

Royal Aeronautical Society - Human Factors Group Seminar on Assessment & Accreditation, London April 30, 2004

Hans-Jürgen Hörmann, PhDBoeing Research & Technology EuropeMadrid, SpainSafety and Human Factors

OverviewOverview

BR&TE

Principles of Behaviour Assessment in NOTECHS/JARTEL

Development of Lufthansa‘s New Behaviour Marker System

Crosslinking NOTECHS to LH Interpersonal Competences

The Implementation Case: Lufthansa‘s Evaluation Method

Usability Study

Summary

BR&TE

H.-J. Hoermann, NOTECHS & LH-IPCs, RAeS, Apr. 2004

Research & Technology Europe

NOTECHS/JARTELNOTECHS/JARTELPrinciplesPrinciples andand PracticePractice ofof BehaviourBehaviour

MarkerMarker ApplicationApplication

HansHans--Juergen Hoermann, PhDJuergen Hoermann, PhDBoeing Research & TechnologyBoeing Research & Technology EuropeEurope

RAeS, April 30, 2004RAeS, April 30, 2004

BR&TE


Why NOTECHS?

JAR OPS (2001) 1.940, 1.945, 1.955, and 1.965, asking for an evaluation of flight crews’ CRM skills.

“the flight crew must be assessed on their CRM skills in accorda“the flight crew must be assessed on their CRM skills in accordance nce with a methodology acceptable to the Authority and published in with a methodology acceptable to the Authority and published in the the Operations Manual.” (1.965)Operations Manual.” (1.965)

“The purpose of such an assessment is to: provide feedback to th“The purpose of such an assessment is to: provide feedback to the e crew collectively and individually and serve to identify retraicrew collectively and individually and serve to identify retraining; and ning; and be used to improve the CRM training system.” (1.965). be used to improve the CRM training system.” (1.965).

BR&TE


NOTECHS/JARTEL Project Team

IMASSA

sofréavia

NOTECHS 1997 NOTECHS 1997 -- 19981998JARTEL 1998 JARTEL 1998 -- 20012001

BR&TE


Initial Setup of NOTECHS

For the purposes of the project, non-technical skills (NoTechS) weredefined as...

“the cognitive and social skills of flight crew members in the the cognitive and social skills of flight crew members in the cockpit, not directly related to aircraft control, system cockpit, not directly related to aircraft control, system management, and standard operating procedures (SOPs)” management, and standard operating procedures (SOPs)” (Avermaete, 1996)

Requirements for NOTECHS…to assess the skills of an individual pilot, rather than a crewassessments based on observable behavioursprecisely defined, culturally robust, reliable, and practicableto be suitable for use across Europe by both large and small operators

BR&TE


NOTECHS Development

Information Sources

Flin & Martin (1998, 2001) review of existing BM systemsIn-depth literature study on selected NTS conceptsAirlines’ current practices- Air France- KLM (WILSC / SHAPE)- Dutch CAA (RLD)- University of Texas Line-LOS Checklist- Lufthansa (Quick Reference List)

BR&TE


The NOTECHS System

Situation awarenessCo-operation Decision

makingLeadership &

management skills

Team building & maintaining

Supporting others

Conflict solving

Considering others

Helps other crew members in demanding situations

Offers assistance

Non-technical skills

Category

Element

Behaviour

(Avermaete & Kruijsen, 1998)

BR&TE


The NOTECHS Framework

Categories Elements

System awareness

Environmental awareness Situation Awareness Anticipation

Use of authority/assertiveness Maintaining standards

Planning and co-operation

Leadership & Management

skills Workload management

Team building and maintaining Consideration of others

Supporting others

Co-operation

Conflict solving

Problem definition/diagnosis Option generation

Risk assessment/option choice

Decision Making

Outcome review

(Rhona Flin et al., 2003)

BR&TE


JARTEL – The NOTECHS Health Check

Influence of cultural differences on the use of NOTECHS across EuropeInter-rater agreement and consistency when assessing the same scenariosApplicability and usefulness in real check situations

Study 1: Experimental approachN = 105 IPs from all over Europe assessing eight recorded scenarios with good and bad examples of NTS behaviours after a half day familiarization course

Study 2: Operational approachN = 25 IPs from five airlines participated in the feasibility check of applying NOTECHS in real training and check situations

BR&TE


NOTECHS Principles

The 5 The 5 principlesprinciples ofof NOTECHS NOTECHS applicationapplication in in practicepractice::

Only observable behaviour is to be assessed

Need for a technical consequence

Repetition required

Rating shall distinguish between acceptable and unacceptable NTS

Explanation required

BR&TE


Basics of Behaviour Observation

Behaviour is defined in the dictionary asaction or reaction of something under specified circumstances. action or reaction of something under specified circumstances.

Overt behaviours• Actions that others can directly observe, like activities, movements,

postures, gestures, positions, facial expressions, speech, or social interactions

Covert behaviours• Processes, that cannot be directly observed, like thoughts, cognition,

feelings, physiological responses

Behaviour observation is the planned collection of behavioural data while subjects are engaged in various activities in certain situations with the intention to assess the collected behavioural data against specified standards.

BR&TE


NOTECHS Application – An Example

Results of an airline audit - NOTECHS to identify training needs(Study conducted by German Aerospace Center (DLR), 02/2000)

0

1

2

3

4

5

Teambuilding

Consider. others

Support. others

Authority & assertiv

.

Providing stand.

Planning & coordin.

Workload mngmt.

SA systems

SA ext. environm.

SA time

Problem defin.

Option generat.

Risk assessm.

Outcome review

Average scores of 17airline pilots

5-po

int R

atin

g Sc

ale

Behavioural Elements of NOTECHS

BR&TE



Results of an airline audit - NOTECHS to examine training effects(Study conducted by German Aerospace Center (DLR), 02/2000)

0

1

2

3

4

5

Teambuilding

Consider. others

Support. others

Authority & assertiv

.

Providing stand.

Planning & coordin.

Workload mngmt.

SA systems

SA ext. environm.

SA time

Problem defin.

Option generat.

Risk assessm.

Outcome review

Pre CRMPost CRM

Behaviour Elements of NOTECHS

5-po

int R

atin

g Sc

ale

BR&TE



Results of an airline audit - NOTECHS to identify training needs(Study conducted by German Aerospace Center (DLR), 02/2000)

0

1

2

3

4

5

Co-operation

Leadersh. & Mngmt.

Situation Awareness

Decision Making

Pre CRMPost CRM

5-po

int R

atin

g Sc

ale

Behaviour Categories of NOTECHS

BR&TE


References

Aberdeen website: www.psyc.abdn.ac.uk/serv02.htmEAAP website: www.eaap.netSofreavia website: www.sofreavia.fr/JARTEL

Avermaete, J. van & Kruijsen, E. (Eds.) (1998). NOTECHS. The Evaluation NOTECHS. The Evaluation of Nonof Non--Technical Skills of MultiTechnical Skills of Multi--Pilot Aircrew in Relation to the JARPilot Aircrew in Relation to the JAR--FCL FCL Requirements.Requirements. Final Report NLR-CR-98443. Amsterdam: NLR.

Flin, R. et al. (2003). Development of the NOTECHS (non-technical skills) system for assessing pilots’ CRM skills. Human Factors in Aerospace Safety, 3(2), 97-119.

Goeters, K.M. (Ed.) (2004). AviationAviation Psychology Psychology –– Practice and ResearchPractice and Research. Aldershot: Ashgate.

Hoermann, H.J. (2001). Cultural variations in the perception of crew behaviour in multi-pilot aircraft. Le Le TravailTravail HumainHumain,, 64(3), 247-268.

O’Connor, P. et al. (2002). Developing a method for evaluating CRM skills: A European perspective. International Journal of Aviation PsychologyInternational Journal of Aviation Psychology, 12(3), 265-288.

What Next?What Next?

BR&TE

Principles of Behaviour Assessment in NOTECHS/JARTEL

Development of Lufthansa‘s New Behaviour Marker System

Crosslinking NOTECHS to LH Interpersonal Competences

The Implementation Case: Lufthansa‘s Evaluation Method

Usability Study

Summary

from notechs to lh behaviour markers an … notechs to lh behaviour markers an implementation case...

Documents