interview analysis - european union agency for … good practice.… · accident investigation by...

Facilitating the Use of New Approaches in Accident Investigation by NIBs

Interview Analysis (Document version 4.1: 29/11/2012)

ERA/2011/SAF/NP/02

Prepared by:

C.W. Johnson1, S. Reinartz2 and M. Rebentisch2,

1. School of Computing Science, University of Glasgow, G12 8RZ, Scotland, UK. [email protected] (Email), +44 141 330 6053 (Tel.), +44 141 330 4913 (Fax)

2. European Railway Agency, 120 Rue Marc Lefrancq, BP 20392, FR 59307, Valenciennes Cedex, France. {[email protected],Michael.REBENTISCH}@era.europa.eu

mailto:[email protected],Michael.REBENTISCH%[email protected]

New Approaches in Accident Investigation by NIBs Johnson, Reinartz and Rebentisch

2

Table of Contents

Executive Summary .......................................................................................................................................................... 4

Part One: Good Practices for Accident Investigation ................................................................................................... 7

1.1 Introduction ............................................................................................................................................................ 7

1.2. Immediate Facts of the Occurrence ..................................................................................................................... 9

1.3. Further Factual Information Gathering ............................................................................................................ 10

1.4. Reconstruction of the Occurrence ...................................................................................................................... 11

1.5. Analysis ................................................................................................................................................................. 12

1.6. Recommendations ................................................................................................................................................ 13

1.7. Facing Future Challenges ................................................................................................................................... 14

1.8. Summary and Conclusions for Part One ........................................................................................................... 15

Part Two: Detailed Analysis and Next Steps ................................................................................................................ 16

2.1. Introduction to Part Two .................................................................................................................................... 16

2.2. Investigators from Diverse Technical Backgrounds ......................................................................................... 16

2.3 Consistency in Training across NIBs .................................................................................................................. 17

2.4. Consistency of Techniques within NIBs ............................................................................................................. 19

2.5 Using Handbooks for Investigators ..................................................................................................................... 20

2.6 Checklists and Questionnaires ............................................................................................................................. 22

2.7 Pre-Investigation Meetings to Coordinate/Structure an Investigation ............................................................ 25

2.9 Guidelines on Access to Data Recorders ............................................................................................................. 31

2.10 The Diversity of Analysis Techniques ............................................................................................................... 32

2.11 Analysis of Human Factors/Organisational factors in investigations ............................................................ 34

2.12 Monitoring of Recommendations ...................................................................................................................... 35

2.13 Linking Recommendations to Stakeholders beyond the NSA ........................................................................ 38

2.14 Sharing European/Internal Systems and Resources ........................................................................................ 38

2.15 Linking Evidence and Analysis to Recommendations ..................................................................................... 40

2.16 Approaches to Time and Resources .................................................................................................................. 41

2.17 Conclusions .......................................................................................................................................................... 41

Appendix A: Additional Good Practice Identified in the NIB Workshop ............................................................. 44

A.1 Introduction ...................................................................................................................................................... 44

A.2 Immediate Facts of the Occurrence ............................................................................................................... 45

A.3 Further Factual Information Gathering ........................................................................................................ 45

A.4 Reconstruction of the Occurrence .................................................................................................................. 46

A.5 Analysis ............................................................................................................................................................. 46


3

A.6 Recommendations ............................................................................................................................................ 47

A.7 Conclusions ....................................................................................................................................................... 49

Appendix B – Statement of Risks and Areas for Further Work ............................................................................ 50

B.1 Statement of Risks and Other Constraints on Future Projects.................................................................... 50

B.2 Suggestions for Further Work to Promote Good Practices .......................................................................... 51


4

Executive Summary Article 21(7) of the Railway Safety Directive calls on the national investigating bodies (NIB) to actively exchange views and experience for the purpose of developing common investigation methods and adapting these to the development of technical and scientific progress. This report presents a number of good practices for accident investigation that were identified during structured interviews with investigators from Member States. The following pages explain how these recommendations were derived from the interviews with NIBs. Further links are drawn between the existing good practices and the roadmap for future investigation techniques that was developed in a previous ERA study (see ERA/2009/SAF/NP/02). Many of the NIBs that supported the present project already use techniques that are very similar to, and hence they can be easily integrated, with those identified in this previous ERA roadmap. The first part of the report recommends the following good practices:

1. If in Doubt, Visit the Site 2. Convene a Decision-to-Investigate Meeting 3. Develop a Preliminary Evaluation Report. 4. Provide Guidance for Interviews 5. Develop Flexible Checklists 6. Convene Weekly Review Meetings. 7. Build and Maintain a Timeline of Events 8. Exploit Flexible, Low Cost Reconstruction Tools 9. Link the Reconstruction to the Evidence 10. Document and Test Initial Hypotheses 11. Refine Initial Hypotheses using Change Analysis. 12. Use both Forward and Backwards Analysis 13. For High Risk Mishaps Document the Analysis 14. Challenge the Analysis through Peer Review 15. Ensure Sufficient Technical Support for Human and Organisational Issues 16. Identify if Recommendations Duplicate Previous Findings 17. Develop Expertise in Searching European Systems. 18. Document the Relationship between Recommendations and Causes 19. Support the Introduction of More Advanced Investigation Techniques.

The second part of the report summarises more detailed observations and identifies directions for further work from the comments made by the investigators during the NIB interviews:

1. Investigators come from a very diverse set of backgrounds. It remains to be seen whether

similar diversity will continue now that the NIBs have been established. If so, then training will continue to be very important as new investigators join NIBs.

2. Different countries offer very different forms of training to their investigators. However,

there is still a considerable overlap. Hence, costs might be reduced and consistency increased if courses could be shared between groups of Member States.


5

3. Investigators use many different tools and techniques to support their work. A previous ERA project (see ERA/2009SAF/NP/2 and Figure 2 of this document) has provided a roadmap to increase consistency. This approach is appropriate because many of the existing techniques used by NIBs fit well within this framework.

4. Investigators often rely on handbooks to guide their work, frequently developed from existing documents in the aviation industry. Future work might integrate these into a common reference document for railway investigators across Member States.

5. Many investigators use checklists to ensure that they follow consistent approaches to their work. Future work might integrate these into common forms, for example for gathering evidence or conducting interviews.

6. Investigators often convene meetings to coordinate an investigation; however, there is

considerable variation in the timing and purpose between NIBs. Such different investigation processes seem natural, given the diversity across Member States. . It is clear that several NIBs use preliminary meetings as an important way of coordinating their efforts across the rest of the investigation lifecycle. The aim here should not be to recommend one approach for all NIBs; but the purpose of the study is to make other NIBs aware of useful processes such as these.

7. The interviews revealed significant differences in the use of data recorders to gather

evidence across Member States. These systems can provide significant insights into the events leading to an incident. The US NTSB has provided guidelines for their use across North America and it might be helpful to consider this material as a basis for their use across Europe.

8. Investigators use different tools and techniques to help identify hypotheses. Good practice can encourage these approaches and help to link them to the framework for future investigatory practice, identified in the previous ERA project (see ERA/2009SAF/NP/2 and Figure 2 of this document).

9. Differences persist in the degree to which human factors experts support accident investigations across Member States. This is an area where further work may be required to encourage consistency and identify specific good practices that support the work of the NIBs.

10. The interviews again revealed a wide diversity of practice. However, there seems to be a

consensus that greater support is required for human factors issues. Typically, this may involve the recruitment of HF experts or contracting external experts for specific investigations1.Further guidance may be required from ERA in the following areas:

1 ERA is supporting this approach by disseminating a Europe-wide call for HF experts interested in assisting

NIBs


6

How to assess the quality of an external expert?

How to audit their findings when the NIB lacks expertise?

How to ensure that the voice of the external expert is not lost when they may be excluded from key stages of an investigation that typically only include NIB staff?

11. Some NIBs monitor the implementation of their recommendations. This provides

feedback on the effectiveness of their interventions.

12. Many NIBs stress the importance of being able to issue interim recommendations before the publication of a final report if they believe that urgent actions are required by particular stakeholders. Later on this will be confirmed or cancelled in the final investigation report. This is general good practice.

13. The previous ERA project has argued that data mining techniques can be used to support

‘next generation’ investigation techniques – for instance, by identifying similar incidents in other Member States to determine whether there is a pattern across a number of incidents. These utility of these ideas are confirmed by some of the interviews in this project.

14. The link between evidence and analysis is an important issue in other areas of accident

investigation and especially in North America. It will become more important if audit mechanisms are introduced to assess the effectiveness of recommendations. Industry stakeholders must be convinced that the findings are justified given available evidence; hence the analysis must be transparent. There is also a European dimension to this issue given that other NIBs must trust the judgements of other bodies if they are to learn from the findings of investigations in other Member States.

15. In implementing the good practices summarised in both parts of this report, it is important

not to lose sight of the impact that such changes might have on both the cost of an investigation and the timeliness of subsequent recommendations.

The European Railway Agency wishes to thank the investigators from the

participating National Investigation Bodies for their contributions to this work.


7

Part One: Good Practices for Accident Investigation

1.1 Introduction

The aim of this project is to identify ‘good practice’ in the investigation of railway accidents across Member States. Good practice is interpreted to reflect those techniques that support the timely identification of consistent recommendations following a thorough review of relevant evidence. Structured interviews were conducted with individual NIBs to learn about the techniques used during their investigations. The first part of this report presents the key messages from the interviews. The second part then provides a more detailed summary of the topics that were discussed during these interviews with investigators.

1: Safety occurrence notification

2: Immediate facts of the occurrence

3: Decision to investigate

12: Consultation

5: “Sufficient” factual information

4: Further factual information gathering

6: Reconstruction of the occurrence

7: Occurrence scenario

8: Analysis

11: Draft report

10: Recommendations

9: Causal factors

13: Final report

14: Publication and Monitoring

Figure 1: ERA Generic Occurrence Investigation Process

The first part of this report is structured according to the ERA Generic Occurrence Investigation Process, illustrated in Figure 1. We focus on phases 2 to 13 dealing with the collection of evidence through to the final report.


8

A further aim for this study is to link the existing good practices of NIBs to the roadmap for future investigation techniques that was developed as part of the previous ERA study (see ERA/2009/SAF/NP/02).

Figure 2: Overview of the Proposed Approaches and Roadmap for Future Harmonisation

Figure 2 provides an overview of the results of this work. As we shall see, many of the NIBs that supported the present project already use techniques that are very similar to those proposed in approaches A and B from the following diagram.

Events and

Causal

Factors

Charting

PEACE

Accimaps

Cognitive

Interviews

Change

Analysis

A:

Causal Sequence

Model

B:

Organisational

Model

C: Next Generation Model

Low cost

approach

intended for

simpler

mishaps.

Higher

cost

approach

suitable

for more

complex

mishaps

Additional

analytical tools

for

dissemination

and integration

with other

Member States.

Data mining

techniques

Advanced

simulation

tools for

reconstruction

and training

OR


9

1.2. Immediate Facts of the Occurrence

The second phase of the ERA Generic Occurrence Investigation Process focuses on gathering the

evidence in the aftermath of a mishap. This evidence informs the decision to investigate an

incident or accident. The following ‘good practices’ were identified by analysing NIB’s interview

responses about their activities during this phase of an investigation. It is important to stress that

there good practices will not be equally applicable to all Member States; however, they are based

on the practical experience of investigatory agencies.

Good Practice 1: If in Doubt, Visit the Site In most cases there should be an assumption to go on site. It is better to find out about an occurrence before making any decision not to investigate, because often too little information is available when a call comes in. It is useful to convene meetings approximately once every thirty minutes while on site to coordinate the initial stages of evidence gathering.

Good Practice 2: Convene a Decision to Investigate Meeting It is useful to hold a meeting shortly after the on-site visit to discuss the investigation plan. This meeting can also help to coordinate with senior management and any other member of staff in the NIB, who has specialist experience that is considered necessary for the investigation.

Good Practice 3: Develop a Preliminary Investigation Report. A Preliminary Investigation Report summarises what is known to date and sketches an initial time line of events. The development of this report can be guided by a template, which is not a restrictive checklist to be ticked off, but guides the investigator to categorise the case and develop an initial plan of investigation, feedback from on-site activities, references to previous NIB recommendations relevant to the case. The decision on the next steps will be based on this report, it may be necessary to specify what questions still need to be addressed before a final decision on the way forward is made. The report should include a further investigation plan and proposed budget, as appropriate. The report may also document urgent safety warnings and recommendations that should be issued before the final report is published.


10

1.3. Further Factual Information Gathering

The activities in the next phase of the Generic Occurrence Investigation Process following the

decision to investigate focus on gathering further evidence. It may involve securing company

records and other documentation. The aim of this phase is to produce sufficient factual

information to support the reconstruction of the occurrence. Investigators may also generate

preliminary hypotheses that guide the identification of evidence to be secured; however, care is

required to avoid premature commitment to particular hypotheses that might then have led

investigators to ignore alternative causes.

Good Practice 4: Provide Guidance for Interviews Specific training should be provided for investigators in interview techniques, using approaches such as PEACE (P – Preparation and planning, E – Engage and explain, A – Account, C – Closure, E – Evaluate) or Cognitive Interviews, as shown in Figure 2. Alternatively, questionnaires can be used to suggest the approach to be taken with different groups of witnesses. The team should agree on the ordering of interviews and on who to meet.

Good Practice 5: Develop Flexible Checklists Checklists support many phases of investigation and can be tailored to particular types of event. They can be used for different purposes: to safeguard the scene of an incident; to identify questions for interviews; to list evidence that should be gathered; to support quality control measures; to identify allocation of responsibilities etc. However, they must not be allowed to become so long and unwieldy that they hinder an investigation. Ideally they should fit on a single side of A4.

Good Practice 6: Convene Weekly Review Meetings. Each week the investigation team should convene to provide updates on the progress of an investigation. Others within the NIB can also be invited to attend if their input is required and to provide quality control through peer review. Although staff are encouraged to be physically present for these meetings, video conferencing facilities are also useful; or may be necessary.


11

1.4. Reconstruction of the Occurrence

The next phase of the ERA Generic Occurrence Investigation Process helps to reconstruct the

events leading to an accident or incident. The objective is to produce an occurrence scenario that

can be agreed upon by the different parties involved in an investigation.

Good Practice 7: Build and Maintain a Timeline of Events As evidence is secured, begin to map out a timeline of events that is shared by all investigators. Update and review the timeline as more information becomes available and review it during weekly meetings to see if anything is missing or an event is in the wrong order.

Good Practice 8: Exploit Flexible, Low Cost Reconstruction Tools There are many general purpose tools that can be used to build timelines and flow charts. These include Post-it notes but also software such as Excel and Visio. These need to be flexible enough so that it is easy for investigators to change the timeline as more evidence is received. Analytical techniques, such as Events and Causal Factors charting and Accimaps, provide methodological support for reconstruction. More advanced approaches include the use of simulation techniques to animate the events leading to an accident.

Good Practice 9: Link the Reconstruction to the Evidence It is important to show how the events in a timeline are supported by the available evidence. Some NIBs have software systems that track the evidence, others use manual tracking systems.

Good Practice 10: Document and Test Initial Hypotheses. Reconstructions can be used to test initial hypotheses to determine whether or not they are consistent with the timeline leading to an accident. Additional studies may be required to confirm the initial evidence. Data mining techniques can also be used to identify previous, similar incidents.


12

1.5. Analysis

After the reconstruction of an occurrence, the next phase in the Generic Occurrence Investigation

Process helps to analyse the incident and to refine initial hypotheses. The aim is to produce the

causal factors that led to a mishap.

Good Practice 11: Refine Initial Hypotheses using Change Analysis. Change Analysis uses a process of elimination to identify any subsystems whose behaviour deviated from expected norms. In other words, causal hypotheses can be generated by comparing the performance observed prior to the accident with the performance expected, for example, in applicable guidance or rule books.

Good Practice 12: Use both Forward and Backwards Analysis Causes can be identified by looking at the outcome of the accident and then working backwards through time looking for those factors that contributed to the accident. This creates problems if the analysis stops too soon and does not consider underlying causes. To prevent this, a second analysis should start well before the incident and work forwards towards the point where the accident occurred. This simulates how the occurrence unrolled for the actors involved. By using both approaches, investigators may develop a deeper understanding of the accident.

Good Practice 13: For High Risk Mishaps, Document the Analysis Some incidents are relatively straightforward, such as level crossing accidents. These do not require sophisticated forms of analysis. However, other high probability or high consequence mishaps require more detailed analysis. Many NIBs now use analysis techniques such as Events and Causal Factors diagrams, Accimaps, STEP, STAMP, MTO, HPES, SHEL, and TRIPOD etc. A key benefit of these approaches is that they can be documented and shown to other investigators to illustrate the analytical processes that support particular recommendations.

Good Practice 14: Challenge the Analysis through Peer Review In-house discussions help to validate the findings of any analysis before they are released to stakeholders. In some countries, it is important to present the findings to interested parties before they are made public. These techniques help to improve the coherence of any recommendations and help to avoid subsequent challenges to the content of a final report. A key objective is to present the causal hypotheses and see if they hold up to peer review.

Good Practice 15: Ensure Sufficient Technical Support for Human and Organisational Issues Several NIBs now routinely use external experts in human factors and organisational issues to support their analysis of occurrences. This is particularly important when safety management concerns are raised by an accident or incident. It is important to establish a list of well-qualified experts before the occurrence is reported.


13

1.6. Recommendations

The final phase of the ERA Generic Occurrence Investigation Process considered in this report

helps to identify the recommendations that reduce the likelihood or mitigate the consequences of

any future incident. The outcomes from this phase help to structure the development of the

draft report.

Good Practice 16: Identify if Recommendations Duplicate Previous Findings Several NIBs have databases or registers of previous recommendations so they can search to see if similar problems have occurred in the past. If they discover similarities then they can ask the national safety authority (NSA) to reinforce the importance of implementing these recommendations and to take any other appropriate actions.

Good Practice 17: Develop Expertise in Searching European Systems. The development of the ERAIL2 system provides a common interface for information about rail occurrences across Europe. It can, therefore, extend or serve as an alternative for national systems to identify similar recommendations made by other Member States and may also help to suggest areas for closer involvement with neighbours and with ERA.

Good Practice 18: Document the Relationship between Recommendations and Causes It is important to check that recommendations are supported by the previous stages in the investigation process. In the final report it is good to provide clear links/references between the recommendations and the identified causes. In the interviews, NIBs reported that this helps communication with the NSAs; industry will be more likely to implement a finding if they also believe that it will be effective.

2 The Agency’s European Railway Accident Information Links (ERAIL) system


14

1.7. Facing Future Challenges

ERA has identified a range of leading techniques that can increase consistency across NIBs.

Several of the interviewees described plans to introduce more advanced techniques, for example

by training staff in interview protocols or in causal analysis methods.

19: Support the Introduction of More Advanced Investigation Techniques. Previous ERA projects have recommended a simplified approach that integrates cognitive interviews or the PEACE interview methods mentioned in Good Practice 4 with one of the simpler causal analysis techniques in Good Practice 13, such as Events and Causal Factors modelling. A more advanced approach includes PEACE or Cognitive Interviews followed by a more structured analysis technique, such as STAMP, again mentioned in Good Practice 13, with a hypothesis generation technique such as the Change Analysis described in Good Practice item 11.


15

1.8. Summary and Conclusions for Part One

The first part of this report has presented a range of ‘good practices’ that were identified during structured interviews with investigators from different Member States. These can be summarised as follows:

1: If in Doubt, Visit the Site 2: Convene a Decision to Investigate Meeting 3: Develop a Preliminary Evaluation Report. 4: Provide Guidance for Interviews 5: Develop Flexible Checklists 6: Convene Weekly Review Meetings. 7: Build and Maintain a Timeline of Events 8: Exploit Flexible, Low Cost Reconstruction Tools 9: Link the Reconstruction to the Evidence 10: Document and Test Initial Hypotheses 11: Refine Initial Hypotheses using Change Analysis. 12: Use both Forward and Backwards Analysis 13: For High Risk Mishaps Document the Analysis 14: Challenge the Analysis through Peer Review 15: Ensure Sufficient Technical Support for Human and Organisational Issues 16: Identify if Recommendations Duplicate Previous Findings 17: Develop Expertise in Searching European Systems. 18: Document the Relationship between Recommendations and Causes 19: Support the Introduction of More Advanced Investigation Techniques.

The second part of this report builds on the previous list and looks in more depth at the topics that were mentioned by investigators during the interviews.


16

Part Two: Detailed Analysis and Next Steps

2.1. Introduction to Part Two

The second part of this report presents a more detailed analysis of the structured interviews (11 interviews in total) that were conducted with 10 individual NIBs. The aim of these meetings was to learn about of their current investigatory practice. The questions were carefully selected after discussion with project staff. A qualitative, open-ended approach was perceived to be the most flexible means of identifying good practice and lessons learned from experience across NIBs. The structured interviews prompted representatives of an NIB to discuss a number of topics relating to their investigations. Following the interviews, the interviewees were given the opportunity to amend and approve the written protocols of the interviews. These protocols were the basis for the analysis work. It is also important to recognise that these protocols may not reflect the opinions of all investigators within the NIB of a Member State.

2.2. Investigators from Diverse Technical Backgrounds

Observation: Investigators come from a very diverse set of backgrounds. It remains to be seen whether similar diversity will continue now that the NIBs have been established. If so, then training will be very important as new investigators join NIBs3.

The interviews revealed the varied backgrounds of the investigators. Most were previously employed within the railway industry, either by the railway undertakings (RUs) or the infrastructure managers (IMs) or the NSA prior to joining the NIB. More than half of the interviewees had between 5 and 10 years’ experience with their investigation body. One conclusion that might be drawn from the interviews is that the recruitment profile of NIBs may change once they have become more established. This issue is addressed in more detail in the next section which deals with training requirements. Another conclusion from the interviews is that there is a need to encourage new ideas by recruiting investigators with expertise in other domains, including human factors or software engineering. This would reflect possible changes in the future focus of investigations.

Many NIB’s had been established in the last decade and several interviewees have been in the NIB from the start. The interviews illustrated the recruitment that was undertaken by many states following the drafting of the Railway Safety Directive.

3 C.W. Johnson, Competency Management Systems to Support Accident and Incident Investigators. In C.G. Muniak

(ed.), Proceedings of the 29th International Systems Safety Society, Las Vegas, USA 2011, International Systems Safety Society, Unionville, VA, USA, 2011.


17

Figure 3: NIB Interviewee by Year of Recruitment (one NIB did not mention this) The investigators came from a range of backgrounds. Some had been aviation investigators, others came from RUs or their Ministry of Transport, some of the investigators worked for infrastructure managers. Some investigators had general training in health and safety but no previous investigatory expertise. Q helps to manage the NIB with a background as a rail engineer that included training as a traffic controller and train driver. However, in the interview he referred to his expertise gathered as an ISO 9001 auditor. Others, such as S, had professional qualifications in aviation engineering and investigation. V had worked as an accident investigator for the state railways before joining the NIB. He had studied a module on transport safety and accident investigation at University. This was followed by a master’s degree specialising in transport safety but he had no previous training in human factors (HF) or investigation. R described his degree in electrical engineering and 20 years of experience as a train dispatcher as the best training for the investigation work. However, he had also attended the Canadian HF and investigation courses. Others have a more varied background; they were previously employed by the NSA and on detachment to the Safety Board, a forerunner of the NIB. They also referred to previous experience in traffic control with the infrastructure manager that provided some exposure to accident investigation.

2.3 Consistency in Training across NIBs

Observation: Different countries offer very different forms of training to their investigators. However, there is still a considerable overlap. Hence, costs might be reduced and consistency increased if courses could be shared between groups of Member States.

The interviews illustrated diverse approaches towards the training of investigators. Many NIBs reported the provision of formal training programmes. For example, Z sketched a six-month

0

1

2

2004 [R] 2005 2006 [P, V] 2007 [T, Z] 2008 [S, W] 2009 [Z, Q] 2010 [Y]

Survey Participants Recruited


18

induction process that included mentoring. Several NIBs mentioned external courses. R has been to two Canadian courses4. Others had been on the Cranfield Rail Accident Investigation Training courses5. These included Z who felt that the case studies on the three week programme were particularly useful. This NIB now has its own training courses with another University run in conjunction with the infrastructure manager. They also provide training in interview techniques with the Police or external consultants. The training included on the job mentoring, Police courses, training from external consultants, training provided by the IM, ERA, the US NTSB and the International Maritime Organisation.

NIBs use different approaches to train their staff. For example, T described how most of their training was given by the infrastructure manager at their training centre. These tended to focus on operational and technical subjects. However, the NIB also included a theoretical course at a university in the training. The respondent also described training courses offered by ERA. In this case, they explicitly mentioned a half day session on human factors. T observed that training was becoming a more significant issue because they had recently recruited several new investigators. They had, therefore, developed a training plan that included a degree of mentoring as well as the more formal University courses, mentioned above. One colleague will also be visiting the NTSB in Williamsburg. He will then report back to the rest of the NIB. V used in-house resources for the initial internal training. External trainers were used for railway traffic rules, ETCS-GSM(R), judicial recorders. They also recommended the ERA course on accident investigation. This NIB linked training provision to the need for an adequate budget. With more funding, two colleagues will go to the Cranfield training course; there will be refresher training on traffic rules. This NIB will repeat training on data recorders and on dangerous goods. Again they talked about future requirements for new recruits with an expectation that they will have a minimum of 5 years’ experience in the railway sector plus special training or a Master’s degree in railway engineering or transportation safety. Y uses on the job training supplemented by internal courses on interview techniques. This lasts one week and is intensive. Although it is in-house it also benefits from external trainers. There are further courses on photography and report writing. This NIB provides training in analysis methods through police courses and on-site work. These comments contrast with the response from S; they do not have any formal training in rail accident investigation. They have built on ideas and concepts introduced as part of a course on maritime accident investigation. They stressed the importance of an understanding of human factors; this builds on T’s comments about the need for ERA support. This response is also important because it stresses the need for annual refresher training; especially for a small NIB with a low incidence of major accidents, as the opportunities for learning on the job are limited. This is similar to the comments from V, cited above, about the need for refresher training. They also proposed that staff might be seconded to other NIBs to gain more experience.

4 http://www.tsb.gc.ca/eng/

5 http://www.cranfield.ac.uk/soe/departments/airtransport/csaic/rail-accident-investigation-

courses.html?id=RailProfsky


19

P is similar to S in that they too lack any formal training in accident investigation. However, they also describe attempts to learn more through their individual initiative and by the Cranfield course. They stressed the utility of training for interview techniques provided by the Police, but also strongly supported the principle of ‘learning by doing’. Some NIB’s have a completely different attitude to training and focus on specific themes rather than on more generic competencies. For example, Q created a training course focussed on level crossings. This NIB also developed a training programme on the analysis of information gathered from data recorders and provides training on quality and safety management; all investigators become competent auditors. They argue that their employees cannot know every technical detail of the railway industry. Although staff have technical knowledge in their respective specialist areas of the railways (operation, infrastructure technology, rolling stock engineering and control command technology), the work of the investigators requires wider knowledge. Investigators must ensure that experts can do the work. Some of the NIBs have framework contracts with known suppliers of external expertise. A key finding is the amount of duplication between NIBs offering internal courses. Some reiterate the utility of external courses (ERA, Cranfield etc.); others do not mention them at all. Some NIBs focus entirely on more technical issues. These observations suggest a number of directions for future good practice – through mechanisms for sharing the costs of training and through the exchange of information about positive and negative experiences with external courses between NIBs.

2.4. Consistency of Techniques within NIBs

Observation: Investigators use many different tools and techniques to support their work. Previous ERA projects (see ERA/2009SAF/NP/2 and Figure 2 of this document) have provided a roadmap to increase consistency. This approach is appropriate because many of the existing techniques used by NIBs fit well within this framework.

X states that they do not any special tools for the collection of evidence so the NIB relies on the investigators’ skill, experience and judgement; there is a concern to ensure that tools do not limit thinking. X also reveals that there are no common techniques for reconstruction of the accident; the format depends on the preferences of the investigator. Microsoft Excel and Word are used during reconstruction. There is specialist software for modelling derailments but there are problems with the license. There is a concern that the output from software tools often cannot be used in evidence. The NIB explains that they had tried to use more formal software tools but this was abandoned as the benefits were not perceived to have justified the efforts involved. However, X would be interested in sharing the development costs of more consistent tools and techniques with other Member States. The need to find useful tools and techniques is referred to many times in the interviews. S is planning to create forms using templates from the aviation and maritime domain.


20

S also mentioned the difficulty of assessing the competence of external consultants; a small country has the advantage that personal contacts are an effective way to find experts. The role of expertise and individual judgement is apparent across the interviews. R describes how the task of collecting evidence is divided between three investigators. The allocation is determined by the team leader. R describes how deciding what to collect and who to interview is based on experience and the situation surrounding the accident. Usually the train driver will be interviewed on-site. The recorded communications between the dispatcher and the driver are collected. Some NIBs describe how they have used quality audit mechanisms (see previous section) to review the consistency of individual decision making but this is unusual. The consistency of investigatory practices can also be influenced by the role of external agents. For instance, Q describes how the NIB lacks technical equipment for the measurement of the track, wheel sets, etc. Instead, they rely on the RUs and IMs to provide support services. This can be complicated when there are different views on the scope of their obligations, for example, whether or not the use of a rail measurement train is necessary. The scope of the investigation is determined by the NIB, but external parties have influence. Further information about differences across Europe is provided by P. They describe how the decision not to investigate may be taken on site. In most cases, they will deploy because they often lack sufficient information to make a clear judgement when the call first comes in. Once on site, investigators have a better feeling for an occurrence. Some NIBs seem to almost always go on-site for a case that meets the criteria to investigate; others go on-site to find out if they need to investigate.

2.5 Using Handbooks for Investigators

Observation: Investigators often rely on handbooks to guide their work, often developed from existing documents in the aviation industry. Future work might integrate these into a common reference document across Member States.

The interviews provided some insights into tools and techniques that had been developed to encourage consistency and guide the work of investigators. Later sections will deal with checklists and forms. However, some NIBs seem to benefit from the development of an investigators’ handbook. In some Member States, this reflects national policy. For example, in X’s NIB all processes and actions are covered by rules. The use of documented procedures and regulations does not, however, guarantee a successful outcome. For example, Q describes how the NIB has to follow general administrative regulations. They must request information from the parties involved in a mishap. These parties are then obliged to provide answers. Other states have begun to develop investigatory handbooks for use within the NIB. V has a manual of their investigation process. As in other Member States, this has been extended from guidance that was originally developed in the aviation sector.


21

Interviewee Use of investigation handbook/documentation from aviation

Comment

S No forms exist yet; however, this is planned based on the notification forms for aviation and maritime modes.

There is an opportunity to share good practice between NIBs who already have these documents with those who are still developing them

T There is a handbook for investigation (other transport modes also have investigation handbook, currently working on a generic handbook with supplements for each mode). Organisation of teams/work is also covered in the investigation handbook. The external experts work independently and pass on their reports to the NIB. At least two NIB staff will go on-site.

Not quite a technique, but it seems to be the trend to have an NIB notebook, with multi-modals shared handbooks seem to be the way forward as here.

U The NIB has its own database of all investigations and under license uses a database for logging all physical evidence – the NIB is developing a user specification for its own evidence database to better meet own requirements and reduce costs possibly in cooperation with the marine and aviation investigation bodies. The use of an evidence logging database is considered to be good practice by the NIB and simplifies cooperation with the police.

Not just handbooks and forms but there are opportunities to share software developed in other modes.

V There is a manual of the overall process based on the aviation sector. One general chapter and then specific sections on accidents types. The manual requires improvement as it is not fit for purpose. There is an ICAO audit in the near future; this will lead to a new more integrated manual. NIB has tried to improve investigation process from case to case; however documentation of improvements has not been maintained.

See previous comments on the utility of manuals shared across modes.

Z The techniques used by the NIB have been adapted from aviation investigations. Use the STEP method, plot actors against time and identify the important boxes. The results are put into an excel sheet, which logs the time information from different sources for the course of events. Next step is to make it more useful to use. On site the investigators can access server via their telephones to obtain checklists, manual and procedures.

Table 1: Transfer of Manuals and Documentation from Other Modes One interview described how their guidance will be revised following an audit from the International Civil Aviation Organisation (ICAO). They hope that this will lead to a more integrated manual. This NIB has gradually made improvements to their investigation processes. However, the documentation has not been maintained to record the changes. This theme is repeated by several NIBs. For instance, T reports that they also have an investigation handbook shared across a multi-modal agency. They are working on supplements for each transport mode. This investigator concluded that their investigation techniques and procedures have been developing


22

over time. There is an internal review of an investigation when the work has been completed and a quality meeting to consider improvements. The handbook includes quality measures, these relate mostly to the use of resources, what was missed etc. Again the common concerns to develop and revise investigation handbooks might create opportunities for ERA to maximise shared resources.

2.6 Checklists and Questionnaires

Observation: many investigators use checklists to ensure that they follow consistent approaches to their work. Future work might integrate these into common forms, for example to gathering evidence or conducting interviews.

One of the most widespread means of encouraging consistency within NIBs and between investigations is through the use of checklists and questionnaires. Interviewee Purpose of Checklist/ Questionnaire Comment

P NIB has a checklist to follow when occurrences are registered to ensure all necessary information is recorded: telephone numbers of contact people etc. Under development are checklists for different types of accidents.

Good practice to tailor forms for different types/severity of accidents.

R A checklist is under development providing a template for data collection for different types of occurrences. This still has to be tested.

A number of NIB see the need or have developed checklists/aide memoire, there is a stress on flexibility as cases are not always the same.

S There are no checklists at present; examples from other NIBs have been examined. It is planned to develop these as part of the procedure book that must be developed for the rail mode (as for other modes).

T At the moment there is no checklist for preparing interviews. Checklists (for all modes) are under development, in draft and will be optimised for the NIB work. These are considered to be very useful – need to know what to look for and also what may be missed when collecting evidence. The checklist will give guidance on this.

ERA might support the development of checklists for use across NIBs.

U The investigation team prepares a Preliminary Examination report (PER). This includes a briefing on the incident – a summary of what is known to date (issues identified), and a time line of events. There is a series of guidance questions (this is not considered as a checklist to be ticked off, nor intended to be exhaustive) as to what is required for the PER – a categorisation of the case and initial plan of investigation, feedback on on-site activities, references to previous NIB recommendations relevant to the case – this may include a

This NIB again stresses the need for flexibility hence does not provide a checklist for the PER.


23

follow-up on the implementation of these recommendations.

V It is planned to develop special sections in a checklist for different accident types together with the other sections: overview/injuries/dead/vehicles involved/ location/weather/traffic rules/involved organisations. In addition there are sheets for the site plan, further notes, details of staff, personal details of witnesses; documentation of interviews; instructions for internal use; recordings (seldom used); details of traction units etc. The documentation sheets are more than just a checklist. Details on organisation are also documented. Some information is made available to others e.g. the site plan for the police; witness information is not available to others.

W The investigation manager oversees the work plan etc.; there is a checklist to check steps: e.g. timeline, causal analysis – it easier to see if something is missing.

X In the Member State all processes and actions are governed by a very detailed rulebook; these rules should be known by the investigators. The NIB keeps track of the evidence by scanning it and collecting it in a database. There are no special checklists to guide the investigators through the investigation.

This reflects important cultural influences; everything is covered by rules. They also illustrate differences between rules and checklists; rules are probably high-level

Y Checklists (mainly for use on-site) for different types of accident are under development – even if each case is different, there is a similar approach for a particular type e.g. collision. Y personally does not value checklists nor use them himself, he considers relying on knowledge and expertise is the better approach. The main guidance comes from the need to determine what happened and what did not happen. What to collect depends on the type of accident, particularly for derailments. Deciding enough evidence has been collected: partly through the on-site checklists that are being developed. Follow a shopping list approach – what information do you need/ what have you got. Time constraints: 2 of the 3 months schedule for evidence collection – it can be quite demanding to stop collecting.

Z Project management template for “driving” plan/progress, plan template with milestones and allocation of responsibilities. The checklist for on-site is under discussion because it is too large and unwieldy, it is helpful for preparations and checking back afterwards as it is comprehensive, but not useful on site. Next step is to make it more useful to use. On-site the investigators can access the server via their telephones to obtain checklists, manual and procedures. This is all input for the checklist, but this is now too cumbersome to use on-site. They stress the need for short but effective checklists for on-site and the quality control


24

approach has been introduced for recommendations.

Table 2: Overview of Checklists Used by NIBs Checklists for interviews: X describes how they use questionnaires to guide the interview process. These are based on the experience of investigators. There are different questionnaires for different interviewees. However, several NIBs also stressed the need for flexibility during the interview process hence it is to be expected that investigators will have to divert from these scripts during most investigations. Checklists for gathering evidence: R goes slightly further by providing their teams with checklists that cover a wide range of evidence gathering activities including the use of photos, video recordings and minutes of site visit. All of these details are then recorded in an event database. Several interviewees felt that it was difficult to create a single checklist that might be used across many different events. Some NIBs have created different checklist templates to support data collection for different types of occurrences. P uses checklists to ensure all necessary information is recorded. These forms also include important information, such as the telephone numbers of contact people etc. They are developing different checklists for different types of accidents such as runaway train, brake failures etc. Other NIBs, such as T are developing draft checklists from other modes so that they can be optimised for rail investigations. These are considered to be very useful – investigators need to know what to look for and also what may be missed when collecting evidence. Checklists for specific accident types: NIB V provides its investigators with a ‘tailored checklist’ for level crossing related accidents. For each incident, data must be collected under the following categories:

Incident overview;

injuries/fatalities;

vehicles involved;

location;

weather;

traffic rules;

organizations involved.

In addition there are sheets for the site plan, further notes, details of staff, personal details of witnesses; documentation of interviews; instructions for internal use; recordings (seldom used); details of traction units etc. Level crossing accidents are typical of situations where a template is appropriate; they are regarded as routine investigations. Some information is made available to others e.g. the site plan is passed to the police. However, witness information is not available to other agencies. During the interview, Q expressed concern that the checklists must be practical to use and therefore should not be too large. Z also uses checklists; however, this NIB uses them to coordinate the management of investigations, as a progress plan with milestones and for the allocation of responsibilities. Staff can look at the chart and easily find out which activities have


25

been ‘ticked off’. This feeds into a quality control questionnaire. They use the same approach to test out their recommendations; investigators have to follow a number of validation steps before releasing their findings. Z uses checklists for on-site investigation but is worried that existing documents have become too cumbersome; it is helpful for preparations but is less useful on site. They are, therefore, reviewing the existing version of the checklist. Investigators can access a remote server via their telephones to obtain checklists, manual and procedures. Y describes how checklists are mainly used on-site. They also are developing specific forms for different types of accident; even if each case is different, there is a similar approach for a particular type e.g. collision. Y describes how checklists help to determine what happened and what did not happen. In other words, they provide a first step towards a causal hypothesis. The priority is to determine what needs to be collected before the train is removed from the site, e.g. data will be lost if the data recorder is isolated from a power supply, switch positions may change. Data may also be lost if the download is not carried out correctly, or data may be destroyed by magnets – this is well-known by drivers, who have access to strong magnets. Such problems are particularly significant when accidents occur at night; this can lead to delays before investigators can work on-site. Checklists have been used throughout many different phases of the investigation lifecycle. For example, W describes how Final Analysis Reviews (FAR) are guided by an explanation of all aspects of the investigation. These documents are developed by the lead investigator using a standard NIB template that provides the initial structure for a final report. The FAR template includes: a timeline, a summary of all evidence - both relevant and discounted, an explanation of the causal analysis, previous relevant recommendations, relevant action already taken – all to be agreed at the meeting. This approach has been used for a number of years and the NIB is now reviewing the approach to improve the ways in which they identify recommendations.

2.7 Pre-Investigation Meetings to Coordinate/Structure an Investigation

Observation: Investigators often convene meetings to coordinate an investigation but there are considerable differences in the timing and purpose across countries. This variation in investigation processes seems natural, given the diversity across Member States. The aim here should not be to recommend a single process across all NIBs. Instead, it is clear that several NIBs use preliminary meetings as an important way of coordinating their efforts across the rest of the investigation lifecycle. And the purpose of the study is to make other NIBs aware of useful processes such as these.

The NIBs in the interviews described a number of mechanisms that have been successfully used to coordinate their work. Both U and Z describe how meetings are held between several different investigators to agree on the planned course of action before a full investigation is launched. Investigations should be divided into steps or work packages to clarify critical tasks and to identify


26

milestones. T holds meetings after an on-site inspection. This initial, start-up meeting is used to discuss an investigation plan. After this meeting, the investigators identify what to look at, which experts should be involved and also a budget. These meetings are important because they avoid some of the limitations with the overuse of checklists. Individuals can discuss appropriate actions within the context of a particular mishap. W describes a Preliminary Examination Report (PER) that provides a briefing on the incident. It presents a summary of what is known with a time line of events. The PER also contains a series of guidance questions. However, they stress that it is not considered to be a checklist that is ticked off. Nor is it intended that the PER is an exhaustive account of the accident. It provides the basis for deciding to investigate. This decision is made by the Investigator in charge based on the PER. However, a preliminary meeting is used to coordinate this decision. As with T, an investigation plan is created. However, there has to be considerable flexibility. Elements of this ‘strategic remit’ are shared with stakeholders from industry for comment. The tactical remit is a living document during the investigation. It may begin to identify possible causes; hence it is only shared internally. Any member of the team can request a revision of the remit as the investigation progresses. Interviewee Monday meetings or similar Purpose/comment

P “Monday” meetings are useful ways of learning from experience and reducing the chance that something is forgotten. When tasks for investigating are allocated to investigators, each has access to necessary documents for the work. In addition there are the regular team meetings and internal communication (all work in the same building). Analysis and conclusions are checked over in team meetings.

Meetings and internal communication are eased by working in the same building. This is less easy for other NIBs who are distributed across multiple locations.

Q At the end of the investigation there are internal final meetings.

R As there is no special training in interviews; the NIB holds internal meetings to prepare for interviews. There are business meetings to discuss scenarios, investigation team plus others – more views on the situation may be beneficial. Sometimes there is disagreement, but this is considered good for development of investigation. Continue until all are in agreement – build a joint opinion. Differences may remain; these will be followed up through emails, phone, more meetings, or via the database. External experts may take part in discussion meetings as necessary, but normally the reconstruction is not discussed with outsiders. At the main office, 2-3 people read the draft report, give comments and pass it to the investigator in charge and the head of the NIB for general comments; all comments are returned to the regional office, where it is decided whether the

Inclusion of external experts in team meetings is important if their findings are to be integrated into an investigation.


27

comments are taken into final report. External experts may take part in business meetings.

S Particularly for large accidents there are regular multi-expert meetings; the investigator in charge coordinates these meetings. External experts have direct access to relevant documents; these are shared during the meetings. Police and other investigating parties (IM/RU are only involved during evidence gathering on-site. After this there is no further involvement. When factual part of analysis is complete, lead- investigator discusses ideas on “why” in meetings. All involved in the investigation are kept informed on progress through discussions in common meetings.

T There will be meetings to discuss the case, possibly before but certainly after the on-site visit there will be an initial/start-up meeting to discuss the investigation plan, after which the investigator will set up a simple investigation plan indicating what to look at, which experts should be involved and also a budget. Meetings are held - milestones for decisions on whether enough evidence has been collected, whether the investigation plan – this is not too detailed - needs to be modified; question whether they need to go to any other direction. The investigator in charge keeps the overview and ensures that the plan is followed and updated as necessary. Decide that sufficient evidence has been gathered: this is determined by a “factual” meeting, although it is of course still possible to gather more documentary evidence later on. In an internal meeting or workshop, the team review the events over time on a white board to look for hypotheses – they may run ideas through the chairmen or other non-rail colleagues. If the team think they have not clarified what happened they may decide it is necessary to use test runs to find this out. Deciding on why it happened is more difficult, there is seldom disagreement on what happened. There is a factual meeting, when the team think they have all the necessary facts. RU/IM and principle actors involved e.g. train driver are always invited to the factual meeting. Team reaches agreement on the recommendations through discussion at the internal meetings. There is a formal procedure if agreement cannot be reached – a procedure of voting – as yet it has not been necessary to follow this procedure (The executive

This NIB uses many different meetings; these include the driver signaller and other principle actors


28

Director cannot overrule the chairman, this is one of the measures to ensure independency.) Discussion of draft report The team discuss the draft report with the NSA, staff from other modes may be asked informally to check over report; however, this depends on workload.

U Video conferences are used to assist working between the 2 locations, e.g. for all team Monday meetings, investigation reviews etc. The preliminary investigations are reviewed by the entire NIB in the regular Monday team meetings; (all staff are encouraged to be in the office on Mondays). The process is similar for all investigations; however, content will vary depending on size/seriousness of case. When the investigation team is clear on the causal analysis and after investigation manager has carried out a step-by-step check that all work is complete, there will be the final analysis review (FAR) meeting. This is a critical milestone where the team will effectively and constructively challenge the investigation and results of the analysis.

Good for NIBs with distributed teams, the other examples do not discuss cases with all teams just the team handling the investigation and the IIC; good practice to involve teams from other locations, peer review approach.

V On-site there is a team discussion to divide up tasks for collecting evidence, talk to involved parties, call up stations to secure further evidence/collect data as necessary then recurrent meetings (every 30 minutes) to discuss what has been discovered to guide next steps and determine what still needs to be collected, photographed, particularly close-up shots. Special informal meetings at defined time each week with all heads of sector, head of rail investigation unit, and DG, plus legal and communications colleagues – peer review group. The draft investigation report is basis for meeting, presented by head of sector (studied in detail/discussed with lead investigator beforehand). Members of the meeting check the whole process and make suggestions about improvements – clarity of report, explanations given by investigator. The meeting does not give a formal decision, but it is clear what still needs to be done in the investigation/report. After consultation period there is a closing meeting when opinions on the report have been collected, where the NSA is present as well as all involved parties. This is a meeting to exchange opinions, can explain, change views of others including on recommendations. Most of the discussion/comments relate to the recommendations; it is often possible to find better measures to the problem in the recommendations.

The 30 minute review is good practice as it provides structure and direction to the on-site activities.


29

W Each step is documented/ recorded: the meeting minutes, power point presentations, drafts of reports, photos, collected documents and scanned records, results of analyses are all stored in the NIB’s own database. The principal investigators, deputy chief investigator and chief investigator then decide on the response to the case: full investigation, lesser investigation, no investigation, letter to involved railway on specific aspect or a record of the case for the coroner’s court, The lead investigator (site phase) will document the comments from the meeting. A checking process is embedded in the various meetings; the lead investigators should be making sure that work is checked. The investigation manager discusses the investigation with the lead investigator as work progresses. This means the investigation manager cannot be a fully independent checker. The need for adequate checking is one reason why the principle inspector asks other members of his team to act as peer reviewers – they attend some of the milestone meetings.

Perhaps need to distinguish between checking (technical/quality) and peer reviews – define what the purpose of each is and build this in to the good practice.

X The RU and the IM are represented in the investigation committee, but not the police. Gathering evidence and the reconstruction of the occurrence are performed in close cooperation. Tests are performed together. There are no additional regular meetings. At the end of the analysis a whole day meeting

takes place for discussion, also on weak and strong

points within the analysis.

Y Tasks are divided amongst team; there are weekly team meetings to see how the investigation is progressing. The team leader needs to keep an overview. Involving other investigating parties: Only during the on-site meetings. One week after the accident, NIB meets with management of the involved parties to explain what is expected from them. There is no exchange/sharing of information during the investigation

Z After the on-site visit there is a team meeting to decide on the investigation; previous reports on similar cases and the possibility to learn something from the investigation are considered in this decision. The team also have Monday meetings where all

Peer contributions also from those not involved in investigation


30

current investigations are considered. Starting from this meeting, the investigation procedure follows a project management basis. There is a need to fit the management of the investigation to the “size” of the case. The NIB organise a meeting with involved parties to organise gathering evidence; a list of questions and list of documents to be supplied is sent to participants. Other parties appreciate this meeting to discuss what the NIB is looking at. It may be necessary to resort to formal letters to obtain documents. Monday /whole NIB meetings give feedback of experience on what evidence should be collected, there is also input from experts brought into the case. In the early stages of the investigation there are meetings to discuss the events in the course of the accident, to look for the cause. When all facts have been collected, there is a meeting to share the same understanding of the case and there is a meeting when the analysis has been completed, one issue to be discussed is how the cause should be described in the report. Agreement with external experts: Open meetings discuss and weigh up the situation. Emphasis is given to having open meetings; in one example case, experts from the fire research organisation.

Table 3: Overview of NIB Meetings Many NIBs hold meetings that help to coordinate an investigation well beyond the preliminary sessions described in the previous paragraph. In particular, many respondents describe ‘Monday Meetings’ to discuss the progress of active investigations. Z holds these sessions to coordinate the immediate team and also make sure that others in the NIB are aware of the status of the investigation. W uses Monday meetings to follow-up the preliminary investigations described in the previous section; all staff are encouraged to be in the office on Mondays. P describes how there can be difficult decisions during the course of an investigation; Monday meetings and quality management processes help to address these concerns. R describes the use of business meetings to discuss scenarios. These are open to other members of the NIB. They stress that having wider views can be beneficial to the analysis; however, it can also lead to disagreement. If this is controlled and focussed it can improve the quality of a final report. Although the aim is to encourage agreement through debate, differences can remain and so an initial meeting may be followed up by emails, phone calls, by further meetings and by the exchange of more material through the NIBs database. V describes how in most cases the team come to a common viewpoint on the course of events, it is possible for different viewpoints to be taken into consideration, if necessary these will be documented in the report. However, they also


31

describe how investigators have been divided over the causes of accidents. These differences could not be resolved through the processes described above and resulted in divergent views in the final analysis. W uses video conferences to coordinate work when teams of investigators can are based at different locations. These video facilities help to ensure that all staff have the opportunity to participate in Monday meetings, investigation reviews etc. The term ‘Monday meeting’ is used in several interviews. However, it is clear that NIBs hold these meetings on different days of the week. V holds ‘special informal meetings’ at a predetermine time each week. However, membership is limited to heads of sector. The meetings are also open to legal and communications teams as well as a peer review group. This approach may be necessary for larger NIBs when it is not possible for all staff to participate even using video conferencing facilities. These meetings are particularly important for quality control with the final report, checking whether further explanations are required.

2.9 Guidelines on Access to Data Recorders

Observation: The interviews revealed significant differences in the use of data recorders to gather evidence across Member States. These systems can provide significant insights into the events leading to an incident. The US NTSB has provided guidelines for their use across North America and some of this material might be useful for NIBs in Europe.

X describes how trains are not equipped with specialist data recorders in their Member State. Other on-board systems can be used to infer the speed and power of the engine while a small number of stations have electronic interlocking systems that can be reviewed. Most other NIBs participating in the interviews could access data recorders – even if they had to rely on third parties. For example, T cannot themselves download the data and must request help from the Infrastructure Manager. This applies both to on-board systems and to signalling records and voice recordings. Other NIBs rely on RUs to provide access to on-board systems. The investigators in P’s NIB are permitted to observe the retrieval process, which is conducted by the IM and RU personnel. Printouts of the data are then supplied to the NIB. Some NIBs have the software to read the data, but not to analyse the data. In other cases, NIBs have contracts with third parties that enable them to directly read recorders. Y employs a company to conduct the technical analysis of the raw data from recorders and voice recordings. These are documented and the analysed data is not discussed with other parties. However, this company has no access to any evidence unless there is a member of the NIB present. Z can take away the data recorder or, for newer locomotives, can download data onto USB sticks. Like Y, they use an external company to analyse the data, but in this case the NIB provides the external organisation with an electronic copy. R identifies a host of problems in accessing data recorders. These include lack of technical knowledge, remote locations, different types of recorders, problems in obtaining software to download and decode the data etc. In consequence, the NIB can take possession of the recorder


32

or the IM/RU will bring it to them. There have not yet been any cases of tampering discovered by the NIB. Other NIBs have the ability to directly decode data recorders. For instance, Q has details of the location of the data recorder for different locomotives. They have software to decode two of the common types of recorder and will request external assistance for other makes. However, it is important to stress that data is often lost by inappropriate responses to an accident, for instance, inadvertently shutting down the recorders or conversely forgetting to shut down loop systems that then record over critical parameters. P describes how they have a homemade catalogue of data recording devices. This was developed after investigators received external training on the use of rail data recorders. The catalogue helps staff to ensure that data is protected in the aftermath of an incident. This NIB also has the ability to download and evaluate data on-site using laptops and a mobile office vehicle. The NIB cooperates with their NSA to ensure that when a new vehicle is authorised, the owner provides the NIB with the necessary cables and software to access on-board data recorders. Y has access to video data from cameras installed at the front of train. This data is particularly useful in reconstructions human factor analyses. They help to determine what a driver might have seen, although they do not provide confirmation of what the driver actually observed.

2.10 The Diversity of Analysis Techniques

Observation; Investigators use different tools and techniques to help identify hypotheses. Good practice can encourage these approaches and help to link them to the framework for future investigatory practice, identified in previous ERA projects (see ERA/2009SAF/NP/2 and Figure 2 of this document).

The interviews revealed that accident reconstructions help to test initial hypotheses. Investigators determine whether or not their ideas are consistent with the timeline of events leading to an accident. A process of elimination can also identify those technical subsystems whose behaviour deviated from expected norms. In other words, causal hypotheses are generated by comparing the performance leading up to the to the accident - whether observed, recorded or reconstructed - recorded or reconstructed with the performance expected, for example, in applicable technical specifications, operational procedures or rule books. Other techniques involve forward and backwards analysis of the course of events. Causes can be identified by looking at the outcome of the accident then work backwards through time looking for those factors that contributed to the accident. This creates problems if the analysis stops too soon and does not consider underlying causes. To prevent this, a second analysis should start well before the incident and work forwards towards the point where the accident occurred. This simulates how the occurrence unrolled for the actors involved. By using both approaches, investigators may develop a deeper understanding of the accident.


33

There are differences in the ways in which NIBs use these various analysis techniques. P described an informal in-house approach driven by discussions and the exchange of expert views. Hypotheses are set up and tested. However, it is recognised that this requires skill and expertise that must be learned over time by new staff. Other NIBs use structured approaches. For instance, S constructs a time line on a wall chart using post-it notes. This informal approach is highly flexible. Time lines can be constructed in a similar way for many different accidents in signalling centres, dispatcher, driver incidents etc. Although they mainly help to reconstruct what happened, the resulting diagrams can also support hypothesis generation. Other NIBs use more complex methods. Q creates a timeline of the event in a suitable form (such as flowchart or fault tree). T and Z use Sequential Timed Event Plotting (STEP). This was adapted from the tools used by aviation investigators. The actors and events are logged within an Excel spread sheet. This provides a more structured extension to the informal timelines that are being used by most NIBs. Strong similarities can be seen between this approach and the analysis techniques used by T. Rather than using STEP, they use the Man-Technology-Organisation (MTO) structuring technique that has also informed the work of the Canadian Transportation Safety Board6. Rather than Excel, T relies on Visio when conducting the analysis of a mishap. They describe how the MTO technique is augmented by the use of the Human Performance Enhancement System (HPES)7. The investigator uses these techniques to start from the adverse event and works backwards following the approach described in the opening paragraph of this section. They identify the barriers and compare what was done with what should have been done. This ‘normative’ approach may make unrealistic assumptions about what operators can do in particular contexts. However, the results of any analysis are validated in consultation with the NSA; staff from other modes may be asked informally to check over the report; however, this depends on workload. This extended peer review forms an important part of the analysis used by many NIBs, including V. All of the approaches described here can be accommodated within the proposals of the previous ERA project (ERA/2009/SAF/NP/02) illustrated in Figure 2. Analysis Technique Reference

Sequential Timed Event Plotting (STEP) Z

Man-Technology-Organisation (MTO) T

Human Performance Enhancement System (HPES) T

Post-its/Timelines S

Integrated Safety Investigation Methodology (ISIM) R

SHEL – Software –Hardware – Environment - Liveware R

TRIPOD Y

STAMP (Systems-Theoretic Accident Model & Processes) Y

TOON Animation Software Y

Flowchart Q

Fault Tree Q

Table 4: Diversity of analysis techniques

6 Rollenhagen, C. (1995). MTO - en introduktion. Samband Människa, Teknik och Organisation. Lund:

Studentlitteratur. 7 Human Performance Enhancement System (HPES) Co-ordinator Manual, Institute of Nuclear Power Operations INPO

86-016, December (1987).


34

Table 4 illustrates the range of analytical tools identified from the interview protocols – it is remarkable that no single technique is used by more than one NIB, with the possible exception of Post-It notes. Some NIBs use various techniques for different aspects of their analysis. For instance, R conducts reconstructions using wall charts, they use the Integrated Safety Investigation Methodology (ISIM) tool, they will also draw sketches, conduct verification tests and interview data, for instance to determine viewing times for bicyclist approaching level crossing. The results of these different tools are integrated; however, they do not use specialist software. In addition to the ISIM toolset they also use the SHEL model to examine the behaviour of train drivers. This decision was influenced by two investigators participating in the training courses offered by the Canadian Transport Safety Board. Y uses a number of software tools. These include a general drawing package for developing timelines. They also have digital systems to replay the traffic controller’s view of train movements. They conduct detailed analysis using TRIPOD and STAMP. This NIB can also commission simulations if they are needed. However, they note that these are expensive and more often used to show the results of the investigation rather than as an analysis tool. They only have the expertise to focus on a few topics and hence the board helps to determine the scope of an investigation when they agree on the composition of a team. If TRIPOD is used then there will always be 1-2 experts; including a human factors specialist, in the team. Ergonomic experts may be brought in. Support may also be needed to look at technical and operational issues.

2.11 Analysis of Human Factors/Organisational factors in investigations

Observation; differences persist in the degree to which human factors experts support accident investigations across member states. This is an area where further work may be required to encourage consistency and identify specific good practices that support the work of the NIBs.

Observation; The interviews again revealed a wide diversity of practice. However, there seems to be a consensus that greater support is required for human factors issues. Typically, this may involve the recruitment of external experts and further guidance is required from ERA in the following areas: how to assess the quality of an external expert? How to audit their findings when the NIB lacks expertise? How to ensure that the voice of the external expert is not lost when they may be excluded from key stages of an investigation that typically only include NIB staff?

Some NIBs have sufficient expertise ‘in-house’ to use sophisticated analysis techniques to support the causal analysis of mishaps. Other NIBs will recruit specialist ergonomists and human factors experts if they feel that this form of input is required during an investigation. For instance, S explains that technical faults are not investigated further when it is decided that the human element was the main element in the accident. As we have seen, R uses SHEL but they too lack human factors expertise. They also want to increase their expertise in ‘organisational aspects’.


35

This NIB can request documentation on the safety management system and policy from companies involved in an accident. They will then identify experts that can assist in an investigation when the budget permits. As noted in previous sections, interviewees are concerned about ways of assessing the expertise of external agents. W reiterates these concerns. They too can request information about safety management. However, as U explained they are concerned that smaller organisations do not understand the need for an SMS; for larger RUs it is more a question of the delivery of the SMS, how they implement the SMS in practice. This NIB has made good use of experienced staff in the regulator. They also use the published standards regarding safety management systems to support their analysis and recommendations. P describes how they are only at the beginning of the introduction of human factors into their work. They have new national legislation on interviews that encourage them to look more closely at human performance issues. External human factors experts have not yet been used. P has had difficulty in obtaining access to RU and IM documents. This makes it hard for them to assess safety management systems and organisational concerns. This NIB specifically advocated that ERA sustains the work of the human factors network and encourages seminars to exchange good practice with other NIBs. They also support a memorandum of understanding between Member States for cooperation between NIBs, possibly as part of an amendment to the Railway Safety Directive. The NIB of interviewee Y includes a group with five human factors experts. They take part and support investigations for all accident modes. One of these human factors specialists is usually included in each investigation team. Another of their colleagues will independently review the investigation analysis. Recently, the NIB of U has recruited a human factors specialist to train as an investigator.

2.12 Monitoring of Recommendations

Observation; some NIBs monitor the implementation of their recommendations. This provides feedback on the effectiveness of their interventions.

Under the provisions of the Railway Safety Directive, NIBs address their recommendations to the NSAs. The NSA is responsible for “taking the necessary measures to ensure that the safety recommendations issued by the investigating bodies are duly taken into consideration, and, where appropriate, acted upon”8. The NSAs are tasked with ensuring that the safety recommendations are taken into consideration and, as necessary, actions such as amendments to rules and procedures are undertaken by the RUs and IMs. This separation of responsibilities is important

8 Article 25 92) Railway Safety Directive


36

because it enables NIBs to make recommendations without necessarily specifying prescriptive measures or considering the costs and consequences of implementation; their focus is on learning from incidents and accidents. Some NIBs monitor the implementation of their recommendations. For example, Y describes how they follow-up the actions that the involved parties have taken in response to the recommendations in the investigation report; have the recommendations been taken implemented as intended? This NIB also mentions that their handling of recommendations will change as, in the future the parties addressed by the recommendations must report back to the Minister on their actions. Z also follows status reports issued by the NSA on recommendations, which are classified as open, pending, or closed. Z also notes that if recommendations are repeated, then they will explicitly check the status of previous recommendations, for example to determine if it should be rephrased. However, the NIB can do little when recommendations are not implemented.

Table 5: UK Air Accident Investigation Branch Recommendation Summary (2010) Table 5 shows the status of recommendations made by the UK Air Accident Investigation Branch This data provides important insights into the safety management practices that protect an industry. A large or growing number of recommendations for which a response is still awaited can indicate a breakdown in communication between the safety authority and key stakeholders. Conversely, a high number of accepted recommendations can indicate a close working relationship across an industry in the aftermath of adverse events. NIBs might be encouraged to follow a similar approach in assessing the impact of their recommendations. The UK Office of the Rail Regulator provides the public with a summary of their responses to recommendations made by the RAIB, illustrated in Figure 4. However, they do not break down these responses into the more detailed analysis that is common in the aviation investigation boards across Europe.


37

Figure 4: UK Office of the Rail Regulator Response to Recommendations (2012) 9 Figure 5 illustrates the recommendation area provided by ERA’s European Railway Accident Information Links (ERAIL) system. One NIB mentioned that they were waiting for the tool to go fully live. This has now been achieved on the 12/07/2012.

Figure 5: ERA ERAIL Recommendation Interface10

W describes how the NIB and the Regulator informally discuss whether the industry’s response really addresses the NIB recommendation or causal factors before deciding whether to accept or reject the industry response. This is still the Regulator’s decision; however, the Regulator can only

9 Available on http://www.rail-reg.gov.uk/server/show/nav.2517

10 Available on http://erail.era.europa.eu/


38

enforce/prosecute those recommendations covered by the health and safety legislation, unless a more extreme response is justified to revoke a safety certificate or authorisation. Elements of this informal approach are also seen in P’s interview; after one year feedback is supplied by the NSA about the implementation of recommendations. However, no checks on the safety-relevant contribution of the recommendation are made. There are meetings with the IM where feedback is provided on what measure the IM has taken in order to discuss any difficulties that have arisen. In contrast, T states that monitoring the implementation of recommendations is not part of their remit they emphasise that it is the task of the NSA to follow this up. They will, however, analyse the implementation of previous recommendations if a similar incident recurs. They will then report to the Government. W checks a database of recommendations, and if there is a repetition then the NIB needs to consider why their previous recommendation has not been implemented or taken effect. The development of such databases is an example of good practice that can be encouraged across Member States. V also has a register of previous recommendations. . There is no formal way of assessing the implementation of recommendations and the addressee is not obliged to respond within a time period. R has no way to assess whether recommendations lead to improved safety. They are dependent on NSA annual reports (to ERA) for feedback.

2.13 Linking Recommendations to Stakeholders beyond the NSA

Observation: Many NIBs stress the importance of being able to issue interim recommendations before the publication of a final report if they believe that urgent actions are required by particular stakeholders. Later on this will be confirmed in the final investigation report. This is general good practice.

During the interview, Q described how recommendations are reviewed for internal quality control at the headquarters and then checked by superiors inside the NIB. Drafts are sent to stakeholders for comments. However, they emphasise that they limit the number of recommendations. Generally, recommendations are addressed to the NSA. Other parties involved in the occurrence are also informed by the NIB. Other NIBs use a hybrid approach to consultation. W takes care to explain the purpose of the recommendations to the regulator. They explain what the recommendation should achieve. The NIB believes that a recommendation will be implemented by industry if they also think it will be effective. W found it helpful to include an explanation of the intent of the recommendation in the report to avoid discussion over the meaning of the recommendation wording, which tended to divert attention from addressing the safety needs.

2.14 Sharing European/Internal Systems and Resources

Observation: Previous ERA projects have suggested that data mining techniques can be used to support ‘next generation’ investigation techniques – for instance, by identifying similar incidents


39

to determine whether there is a pattern across a number of incidents. The utility of these ideas are confirmed by some of the interviews in this project.

Reviewing previous recommendations Many NIBs search for previous recommendations using databases that they have created and maintained themselves. For instance, R has developed their own internal database based on key word searches e.g. sidings. This application was first made available in 2006. It also provides basic data on any occurrences that were not investigated. P also uses an internal system, augmented by direct personal contact with countries in their region to identify whether or not similar accidents have happened in the past. However, for most NIBs there is no systematic approach to consider recommendations by other NIBs. V used of Excel tables to search for similar cases based largely on the location and type of accident. They would prefer to use ERAIL rather than to develop their own database to avoid unnecessary duplication. Z maintains their own Excel sheet of all previous investigations and recommendations. This supplements the memories of staff. Z will also conduct internet searches, they will access an NSA database, and they will consult ERADIS and also plan to use ERAIL. Table 6 provides an overview of the different search tools that were mentioned during the elicitation exercises.

Tools Cited Reference

Excel V, Z

ERAIL (Planned) V, Z, Y, S

ERADIS X, Z, S, Y

NSA Database Z, Y

NIB Database T, R, Q, W, P

Other national Databases W

Table 6: Distribution of Tools Cited During Interviews


40

Figure 6: ERADIS Interface11 Now that ERAIL has come into operation, it is hoped that the NIBs will also find this a useful tool to consult the investigations and particularly the recommendations of other NIBs.

Tracking evidence Although W already uses database for logging all physical evidence, they are now developing their own user specification for a new system, which will be tailored to their detailed requirements. They plan to reduce costs by sharing the development with the marine and aviation investigation bodies. The use of such a tool simplifies cooperation with the police. V is also looking for a system to help track evidence. At present, the team secretary uploads all data to a digital database/map. V does not have an internal evidence tracking system. They rely on paper documentation used to record numbered pieces of evidence with tags on evidence. They use lists of collected evidence and rely on the investigator’s memories to keep an overview of evidence. Hence there is considerable scope for the exchange of tools and techniques to help record evidence and then to link that evidence through subsequent analysis to the identification of recommendations. In the future, ERA might support these initiatives by developing shared tools cross different Member States. Recording causes of accidents: W has a database of incidents and causal factors. Q also has an internal database with more than 100,000 events since 2000. A bi-annual ‘systematic review’ of previous incidents is planned.

2.15 Linking Evidence and Analysis to Recommendations

The issue of transparency was raised by several, but not by all interviewees

Observation: The link between evidence and analysis is an important issue in other modes and especially in North America. It will become more important if audit mechanisms are introduced to assess the effectiveness of recommendations. Industry stakeholders must be convinced that the findings are justified given available evidence; hence the analysis must be transparent. There is also a European dimension to this issue given that other NIBs must trust the judgements of other agencies if they are to learn from the findings of investigations in other Member States.

11 Available on https://pdb.era.europa.eu/


41

2.16 Approaches to Time and Resources

Observation: It is important to continually review the cost of an investigation and the timeliness of subsequent recommendations.

Y discussed the impact of time constraints on their work. He described how investigations are organised around a 12 month timescale the ‘3522 model: referring to the number of months allocated to the complete investigation process. They have 3 months to collect evidence from accident both on-site and from interviews – this leads to a proposal to the board indicating the main questions that need to be considered in the investigation. 5 Months are for the analysis, 2 for the draft report and 2 for the consultation phase. However, the new head of the NIB aims to reduce this to 6 months by increasing the size of investigation teams; with 3 to 5 people as a norm and 10 to 12 for more complex cases. The NIB plans will focus more on investigating the types of accidents as required by the Railway Safety Directive in order to reduce the investigation time.

2.17 Conclusions

This report has identified good practices that emerged during a series of structured interviews with NIBs from 10 Member States. We have also identified key lessons that can help to increase consistency across Member States, for example in the use of common analysis techniques. We have identified areas of existing weakness; for instance in human factors and organisational analysis, which should be addressed by European initiatives. In particular, this second part of the report has presented detailed observations that emerged during the interviews. These can be summarised as follows:

1. Investigators come from a very diverse set of backgrounds. It remains to be seen whether

similar diversity will continue now that the NIBs have been established. If so, then training will be very important as new investigators join NIBs.

2. Different countries offer very different forms of training to their investigators. However,

there is still a considerable overlap. Hence, costs might be reduced and consistency increased if courses could be shared between groups of Member States.

3. Investigators use many different tools and techniques to support their work. Previous ERA projects (see ERA/2009SAF/NP/2 and Figure 2 of this document) have provided a roadmap to increase consistency. This approach is appropriate because many of the existing techniques used by NIBs fit well within this framework.


42

4. Investigators often rely on handbooks to guide their work, often developed from existing documents in the aviation industry. Future work might integrate these into a common reference document across Member States.

5. Many investigators use checklists to ensure that they follow consistent approaches to their work. Future work might integrate these into common forms, for example to gathering evidence or conducting interviews.

6. Investigators often convene meetings to coordinate an investigation but there are

considerable differences in the timing and purpose across countries. This variation in the investigation processes seems natural, given the diversity across Member States. The aim here should not be to recommend a single process across all NIBs. Instead, it is clear that several NIBs use preliminary meetings as an important way of coordinating their efforts across the rest of the investigation lifecycle. And the purpose of the study is to make other NIBs aware of useful processes such as these.

7. The interviews revealed significant differences in the use of data recorders to gather

evidence across Member States. These systems can provide significant insights into the events leading to an incident. The US NTSB has provided guidelines for their use across North America and some of this material might be useful for NIBs in Europe.

8. Investigators use different tools and techniques to help identify hypotheses. Good practice can encourage these approaches and help to link them to the framework for future investigatory practice, identified in previous ERA projects (see ERA/2009SAF/NP/2 and Figure 2 of this document).

9. Differences persist in the degree to which human factors experts support accident investigations across member states. This is an area where further work may be required to encourage consistency and identify specific good practices that support the work of the NIBs.

10. The interviews again revealed a wide diversity of practice. However, there seems to be a

consensus that greater support is required for human factors issues. Typically, this may involve the recruitment of external experts and further guidance is required from ERA in the following areas: how to assess the quality of an external expert? How to audit their findings when the NIB lacks expertise? How to ensure that the voice of the external expert is not lost when they may be excluded from key stages of an investigation that typically only include NIB staff?

11. Some NIBs monitor the implementation of their recommendations. This provides

feedback on the effectiveness of their interventions.

12. Many NIBs stress the importance of being able to issue interim recommendations before the publication of a final report if they believe that urgent actions are required by


43

particular stakeholders. Later on this will be confirmed in the final investigation report. This is general good practice.

13. Previous ERA projects have suggested that data mining techniques can be used to support

‘next generation’ investigation techniques – for instance, by identifying similar incidents to determine whether there is a pattern across a number of incidents. The utility of these ideas are confirmed by some of the interviews in this project.

14. The link between evidence and analysis is an important issue in other modes and especially

in North America. It will become more important if audit mechanisms are introduced to assess the effectiveness of recommendations. Industry stakeholders must be convinced that the findings are justified given available evidence; hence the analysis must be transparent. There is also a European dimension to this issue given that other NIBs must trust the judgements of other agencies if they are to learn from the findings of investigations in other Member States.

15. In implementing the good practices summarised in both parts of this report, it is important

not to lose sight of the impact that such changes might have on both the cost of an investigation and the timeliness of subsequent recommendations.

Above all, our interviews revealed a concern across most Member States to share resources across Europe. Investigation tools for different types of accidents could be developed for all NIBs which would be more efficient. Also cooperation in the field of laboratories (technical analysis, crash test) would be good. In these areas and the others mentioned above there is considerable scope for ERA to play a leading role in the future of European railway accident investigation.


44

Appendix A: Additional Good Practice Identified in the NIB Workshop

A.1 Introduction

In order to validate the good practices for accident investigation, the key findings in this report

were discussed with a group of NIBs. This section summarises the main outcomes and identifies

additional good practice reported by the investigators during the course of this workshop.

1: Safety occurrence notification

2: Immediate facts of the occurrence

3: Decision to investigate

12: Consultation

5: “Sufficient” factual information

4: Further factual information gathering

6: Reconstruction of the occurrence

7: Occurrence scenario

8: Analysis

11: Draft report

10: Recommendations

9: Causal factors

13: Final report

14: Publication and Monitoring

Figure A1: ERA Generic Occurrence Investigation Process

The following sections structure the reported good practice according to the phases in the ERA

Generic Occurrence Investigation Model, illustrated in Figure 2.


45

A.2 Immediate Facts of the Occurrence

1. The decision to investigate is not a simple binary choice. In other words, investigators may

decide to launch an initial investigation of a minor incident using fewer resources than

would be allocated to a major investigation.

2. It is important to decide which interviews will need to be fully transcribed. It takes

considerable time and resources to document the outcome of lengthy interviews. Hence

some NIBs will only transcribe critical sections of some interviews.

3. It is important to identify the time within which a preliminary review meeting will be held.

Some NIBs require a preliminary briefing at the next Monday meeting but this imposes

tight deadlines on staff. Others require a briefing within three weeks of an incident being

reported.

4. During the initial stages of an investigation, some NIBs draft a remit document that

describes the scope of the investigation. This is agreed with senior staff. It is important

that this is a living document that is maintained to reflect changes in an investigation

when, for example, more evidence becomes available. In some cases, the remit will

describe alternative ways forward; “if we find that X happened then we will study Y, if not

then the investigation will focus on Z”;

5. It is considered important to involve industry in the early stages of an investigation as they

are then motivated to support and implement subsequent recommendations.

6. Multi-modal call centres can be created to handle the initial notification of an incident.

Duty staff are trained to ask relevant questions depending on the mode –rail, aviation,

road etc. - and the type of occurrence that is being notified.

7. In some countries, the NIB examines industry logs to determine whether or not incidents

have occurred that have not been notified to them.

8. Some NIBs have begun to make use of ‘associate investigators’ or ‘accredited agents’.

These are individuals in the industry who are trained by the NIB but who are not members

of their permanent staff. They are often located in geographical regions where the NIB

does not have a full-time presence. If an incident is notified in a remote region, then the

associate investigator can respond and be on-site to secure evidence many hours before it

would be possible for a full-time investigator to reach the site.

A.3 Further Factual Information Gathering


46

9. It is important to identify key milestones within the phases in an investigation where

progress can be reviewed. These milestones can either be based on time – scheduled after

N weeks, or can be based on key events – for example when the results of specific tests are

available.

10. There should be no ‘single point of failure’ in an investigation. All key decisions and

documents should be reviewed by at least one other person in addition to the investigator

involved in taking a decision or writing a report.

11. In some member states, it is useful to share factual information learned from evidence

with industry; however, this should NOT include information from witness statements and

interview transcripts.

A.4 Reconstruction of the Occurrence

12. Videos now provide a useful means of reconstructing an incident. Cameras can be placed

on trains to record critical information and provide a direct view of what happens to

components and subsystems under conditions that are similar to those in an incident.

13. If video evidence and similar techniques are used to reconstruct an incident, it is important

to work with the IM and RU to conduct a risk assessment. This helps to ensure that the

process of reconstructing an adverse event does not lead to another incident.

14. Video evidence is particularly useful for reconstructing what a driver saw in the moments

leading to an accident. It cannot provide an exact impression – for example of particular

weather conditions; however it improves the investigators’ understanding of the

information available to the driver as the event developed.

15. It is important to remember that any reconstruction created after an accident can be

influenced by hindsight; investigators know that an accident has taken place so it can be

difficult to imagine what key participants experienced without the prior knowledge that an

accident would happen.

16. Working with the involved companies can help during a reconstruction. Both the NIB and

the IM/RU can gain new insights into the causes of an incident by working together.

A.5 Analysis

17. A final analysis review should be convened during which the investigator(s) will present the

entire investigation process. This should also include a summary of the evidence that was

NOT used during the analysis. This review may identify the need for further analysis. It is

more efficient if the structure of this review follows the template of the final report.


47

18. A useful approach to tracing the causes of an incident is to ask WHY? A rule of thumb is to

ask WHY at least five times for each contributory factor.

19. The knowledge and analysis techniques applied by human factors specialists can lead to

increased confidence on conclusions about human error and how this may or may not

have contributed to the occurrence. This works in two ways – they can help exclude

human error as the sole cause when otherwise the driver/signaller may be blamed without

further consideration of the underlying causes. Equally, in those situations where human

error is shown to be a major contributory factor, human factors experts can examine how

or why the error was made.

20. Some NIBs will include diagrammatic representations of the analysis in their reports, such

as STEP diagrams, to justify the conclusions. As these diagrams can be extensive and

complex, simplified versions of the diagrams will be used in the reports.

21. Some NIBs do not include analytical diagrams in their reports but instead, they will keep

them with the documents gathered during an investigation so that they can go back at a

later date and better understand why they produced particular conclusions, for example,

in the investigation of similar incidents in the future.

22. Analytical diagrams can also be kept to reflect what an NIB was thinking at different stages

of an investigation.

23. Some NIBs have used barrier analysis. In using this particular tool, it is often important to

look at barriers that worked in the way intended and yet did not prevent the accident, for

instance if a driver failed to respond to a warning.

24. During any analysis it is also important to stress the measures that worked well, for

example in reducing the consequences of an incident, and not just to focus on the

measures that failed.

A.6 Recommendations

25. The recommendations from an investigation should be sent out for consultation with key

stakeholders. Industry representatives can provide initial feedback without undermining

the independence of an investigation.

26. Some investigations lead to bulletins and safety alerts rather than to full formal reports. It

is good practice to produce these publications for less severe incidents that create safety

concerns. However, some of the NIBs lack the resources that are required to produce

these additional documents, especially when they are not required by national legislation.


48

27. Techniques such as a ‘most wanted list’ of recommendations may irritate some sectors of

industry. However, some NIBs find it useful to have a ‘watch list’ of topics and concerns

that require further attention.

28. Some NIBs12 have developed a checklist to assess the quality of recommendations. These

include such questions as: how do we follow up implementation? Is the recommendation

identifying a problem or does it also identify specific solutions? Is it possible to understand

the wording of a recommendation without reading an entire report? Have you followed

the NIBs procedure in creating recommendations?

29. Multi-modal NIBs find it is useful to have cross-modal meetings to discuss

recommendations in some incidents, particularly when the domains share similar

problems, e.g. with human fatigue.

30. Some recommendations may have been included in previous reports. If this is the case

then it is good practice for new reports to reference the earlier recommendations and to

indicate to the readers that this is a repeated, unresolved concern.

31. Some NIBs have two forms of recommendation – the formal findings in a report at the end

of an investigation and other more immediate recommendations that are issued earlier in

the investigation, some even be made at the accident site. These immediate

recommendations can be published electronically – for instance via a web site. However,

other NIBs have avoided the use of immediate recommendations because they do not

easily fit inside the existing mechanisms of consultation and coordination with the NSA and

other stakeholders.

32. Some NIBs stated that it is useful for them to work with NSAs in following the

implementation of recommendations. Cases have been reported where, companies that

have been given N years to complete a recommendation will apply to the NSA just before

this deadline and ask for an extension. Experience has shown that is far more likely to be

granted by the NSA several years after an accident than if the company make the request

immediate following the publication of the investigation report.

33. Some recommendations are more effective than others. NIBs should monitor the impact

of their recommendations and also share experience between Member States on which

recommendations have had the biggest impact on their industries.

12 NIB Norway has provided a checklist and a procedure on recommendations to ERA; these will be translated

into English and made available to all NIBs.


49

A.7 Conclusions

During the workshop a number of further issues were discussed. These can be summarised as

follows:

A. In the future, it would be useful to consider the good practices that are best suited to

different NIBs. For instance, approaches that are useful for larger NIBs may not be

practical for smaller investigation bodies. Good practices that support small teams may

differ from those that are useful for large teams. An example would be the need for

smaller teams to contract out more of the work in an investigation.

B. It would be useful for ERA to establish a record of key investigation and analysis skills

possessed by particular NIBs. During the meeting, Member States presented a diverse

range of techniques that they had used – including analytical tools and video evidence as

well as quality control mechanisms. A list of these existing skills would be useful when

other NIBs needed help – for instance, in shooting video evidence on site, NIBs could ask

what equipment has been used etc.

C. There are dozens of tools that can be used in a reconstruction; some focus on locos,

signalling, fire etc. it would be useful to have an NIB action to review what is available and

what is useful across different types of investigation.

D. Some NIBs have modified existing analytical tools for their own use; others have good

techniques for reconstruction which have no formal name. In the future, it might be useful

to compare these approaches in more detail.

E. Many NIBs have developed quality control processes for their investigations. Some have

argued that procedures plus checklists provide a framework for such processes. Quality

control procedures might be discussed in a plenary session or workshop.

F. In aviation, stakeholders must provide feedback on the actions they will take to implement

recommendations within 30 days; within a further 90 days the investigation bodies must

state whether this is an acceptable means of compliance. Although this is not the

approach advocated within the Railway Safety Directive some thought could be given to

similar techniques being trialled in the future within the railway industry.


50

Appendix B – Statement of Risks and Areas for Further Work

This section uses the insights derived from the structured interviews and the subsequent NIB

meeting to identify future risks for work to promote good practice in accident investigation. It

then presents areas for further work to achieve the Agency objectives to assimilate new methods

into the existing good practice currently applied by the NIB; both to retain valuable and pragmatic

approaches to investigation and to promote acceptance of these innovative methods by the NIB.

B.1 Statement of Risks and Other Constraints on Future Projects

The work described in this report has helped to identify a number of constraints and risks that

should inform and guide future work in this area. The can be summarised as follows:

Need to Understand and Support Differences between NIBs.

During the course of this project, the point was made on many occasions that NIBs are very different across Member States. There are differences in terms of the size and nature of the railway networks. There are differences in terms of the demands placed on and the resources available to NIBs. In consequence, NIBs have developed good practices that are well suited to the particular context that they work in. Although all NIBs operate within the provisions of the Railway Safety Directive, there are strong differences across the various legal systems in Europe. Techniques that have worked well in large NIBs cannot always be transferred successfully to smaller NIBs and vice versa. Future work in this area must be sensitive to the different requirements across member states otherwise there is a risk that techniques will be advocated that cannot be used by different NIBs.

Need to Support Regional Differences and Regional Initiatives. NIB plenaries provide good opportunities to talk about common problems and exchange good practices. It is also important to promote regional groupings where NIBs meet more frequently. This is important because it encourages cooperation; especially when investigations may involve more than one member state. The meetings of the German speaking NIBs provide an example of such initiatives. In other industries, for example Air Traffic Management, these regional groupings are become more important as a means of addressing the different needs in particular areas of Europe. If this is not done then there is a risk that important opportunities will be lost to encourage the mutual exchange of good practices.

Need to Understand Different Terminology. During the structured interviews and the subsequent analysis, it became clear that a number of common terms were being used in slightly different ways by different NIBs. For instance, some NIBs only use the term ‘recommendation’ to refer to the final list of actions that are identified in the formal report. Other NIBs also refer to ‘recommendations’ that can be made immediately after an accident. In the ERA generic framework for the investigation process,


51

recommendations are identified towards the end of the investigation process. The different uses of common terms reflect important differences between NIBs and future work in the exchange of good practices must be sensitive to the ways in which common terms are used to describe different practices.

Need to Understand the Importance of Time and of Money. In promoting good practices, it is important to remember that NIBs operate under financial constraints. There is also a requirement to publish recommendations as soon as possible, without undermining the quality of a report, so that stakeholders can take appropriate measures to avoid the recurrence of any incident. These constraints are likely to become increasingly important as many Member States address fiscal deficits. If these issues are not considered then there is a risk that techniques will be promoted that cannot be used within the existing resources available to many NIBs.

B.2 Suggestions for Further Work to Promote Good Practices

During the structured interviews and the subsequent NIB meeting, a number of areas for further

work were identified. These are itemized in the following list:

Hold Additional NIB Technical Presentations. The NIB meeting to consider the work in this project helped to identify a host of good practices. These extended the scope of the structured interviews and helped to confirm the utility of the results. However, it also demonstrated the need to provide additional time for NIBs to present the techniques that they use outside the context of particular investigations. The presentations by NIBs on the methods that they use provided useful first-hand experience that can be passed on to other Member States. These methodological issues can be hidden by the details of a particular investigation in other meetings that are specifically focussed on individual incidents. In the future there should be additional opportunities for NIBs to show the techniques they use – for example in video recording evidence or in the application of causal analysis tools.

Hold Additional Meetings on Phases of Investigation.

This builds on the previous recommendation. The NIB meeting that was held as part of this project was scheduled to last one day. It covered many different phases of an investigation from the gathering of evidence through to the generation of recommendations. However, it was clear from the NIB feedback that discussions could have continued far longer than the time available. In particular, future sessions could be held to focus specifically on the exchange of good practice for interviews, on causal analysis tools and on techniques that help to identify effective recommendations.

Establish a Registry of NIB Special Expertise. This project identified that NIBs have a great range of existing expertise in interview techniques, in different approaches to causal analysis, to quality control in the development of recommendations and final reports. The NIB meeting also revealed key strengths in human


52

factors and in the analysis of video evidence. A future project might create a shared resource or web site that provides a list of expertise and contact information within NIBs so that other Member States might seek assistance when using new approaches.

Promote Comparisons of Checklists and Other Documentation. During the structured interviews, the NIBs revealed that their work is guided by many different forms, checklists and handbooks. These are often exchanged following Plenary meetings on an ad hoc basis. In the future, a project might be launched to create an inventory of useful documentation so that some of them might be translated and shared between different NIBs. The aim would not be for ERA to publish ‘recommended’ checklists but to provide Member States with access to the documents that NIBs have already found useful.

Hold a Review of Quality Control and Audit Techniques. Many NIBs referred to the tools and techniques they used to ensure quality control during an investigation. These included checklists to monitor the procedures used in each step of an investigation. In other Member States, meetings were held at key stages in an investigation and peer review was used to assess interim findings. A more detailed study might be launched to focus on quality control and audit. As mentioned in previous sections, it is unlikely that all NIBs will use or need the same techniques during different phases of an investigation. However, all Member States can benefit from quality control mechanisms that increase confidence in the investigatory processes used around Europe.

interview analysis - european union agency for … good practice.… · accident investigation by...

Documents