AYR COLLEGE

Flight Deck Evolution – The

Path to Automation and

Beyond HND Aeronautical Engineering Graded Unit

Jon Galbraith

S10172585 2011/2012

An appraisal of flight deck instrumentation, from mechanical and electro-mechanical instruments,

through to modern TFT displays and beyond. Modern voice recognition technology will be discussed and

assessed for suitability in the modern flight deck.

~ 1 ~

Table of Contents

FLIGHT DECK EVOLUTION – THE PATH TO AUTOMATION AND BEYOND .......... 2

PROJECT BRIEF .......................................................................................................................................................... 2 PROJECT SPECIFICATION AND GANTT CHART ............................................................................................................ 3 AN INTRODUCTION .................................................................................................................................................... 5 INSTRUMENTS – THE BASICS ..................................................................................................................................... 6 ELECTRONIC DISPLAYS ........................................................................................................................................... 11

THE INTEGRATION OF A SYSTEM .................................................................................... 15

REQUIREMENTS ....................................................................................................................................................... 15 SAFETY ASSESSMENT .............................................................................................................................................. 16 WHAT IS CERTIFICATION? ....................................................................................................................................... 19 TESTING PRINCIPLES ............................................................................................................................................... 22

INTEGRATION OF VOICE RECOGNITION SOFTWARE ............................................... 24

THE REQUIREMENT ................................................................................................................................................. 24 SAFETY ASSESSMENT .............................................................................................................................................. 24 SYSTEM DESIGN ...................................................................................................................................................... 29 HUMAN FACTORS – STRESS ..................................................................................................................................... 31 TEST PROCEDURE - METHODOLOGY ........................................................................................................................ 32 TEST PROCEDURE - SUMMARY OF FINDINGS ........................................................................................................... 37

FLIGHT DECK EVOLUTION – THE PATH TO AUTOMATION AND BEYOND ........ 39

A CONCLUSION........................................................................................................................................................ 39 REFERENCES ............................................................................................................................................................ 40

APPENDICES

~ 2 ~

Flight Deck Evolution – The Path to

Automation and Beyond

Project Brief This project will look at the evolution of flight deck instrumentation and the possible

integration of new technology. The project will include:

- Appraisal of flight deck instrumentation

- Mechanical and Electro-mechanical instruments

- ‘Glass Cockpit’ and TFT Displays

- Developments and limitations of flight deck information

- Voice recognition software advantages/ disadvantages

- Human factors including stress

- Testing of software and results

As a subset of the overall remit of the project, modern technology, in particular voice recognition

and voice response software will be discussed and assessed for suitability in the modern flight

deck. The development of this software will be discussed with particular regard to safety,

certification and any issues faced with possible integration. A practical exercise in testing the

software will further assess the suitability of the software before a conclusion is made.

It is expected that the project shall take approximately thirty-one weeks to complete.

~ 3 ~

Project Specification and Gantt Chart The majority of this project will be compiled of research into flight deck instrumentation

from the past and present, voice recognition software, testing and certification of new

technology. The project specification is as follows:

Section A: Flight Deck Evolution Introduction and Current Instrumentation

Section B: Integration of a System

Section C: Integration of a Voice Recognition System (including test procedure)

Section D: Conclusion and References

~ 4 ~

14/11/2011

21/11/2011

28/11/2011

05/12/2011

12/12/2011

19/12/2011

26/12/2011

02/01/2012

09/01/2012

16/01/2012

23/01/2012

30/01/2012

06/02/2012

13/02/2012

20/02/2012

27/02/2012

05/03/2012

12/03/2012

19/03/2012

26/03/2012

02/04/2012

09/04/2012

16/04/2012

Mechanical instrum

ents, electro-mechanical instrum

ents andTFT displays

Hum

an factors with regard to Speech Recognition Idea

Certification requirements

Speech recognition systems

System safety, security and appraisal

Look at system requirem

ents and design

Possible acquirement of softw

are

Testing Procedures and types of test

Fault tree analysis and severity of problems

FMECAs and FM

EAs

Compiling an enablem

ent, control and override procedure

Compiling a test procedure

Performing the test procedure and recording results for norm

al range

Performing the test procedure and recording results for robustness

Writing a test report

Compile the test data and form

at into readable results

Conclude the report and complete references

In Progress

Actual Completion (If D

ifferent)

Plan of Action

~ 5 ~

An Introduction The aviation industry has fast become one of the most technologically advanced

industries in the world with ever increasing automation at the very forefront of the industry’s

continuing evolution. Aircraft have come a long way since the Wright brothers first took to the

skies back in 1903. Modern day rivalries between the likes of Boeing and Airbus continue to

contribute positively to the emergence of new technologies, and on-going space and military

research constantly supply new ideas and innovations to the industry. Through the years, aircraft

have become more aerodynamic, grown in size, flown faster, reached new heights, but perhaps

most importantly, become safer. In addition to producing more structurally sound,

aerodynamically, economically and environmentally efficient aircraft, the manufacturers have

also continued to modernise and develop one of the more major and important aspects with

regard to safety – the Flight Deck. Through the years there has been the transformation of

instruments from mechanical to electro-mechanical, and more recently from electro-mechanical

to ‘glass cockpit’. Automation is ever increasing as is the production of new technologies. With

the recent emergence of Voice Recognition aboard the Typhoon military jet, it would be thought

that such developments would also be made within the commercial aviation industry. However, a

lot of work has to be carried out when considering the integration of a system, it is not quite as

easy as ‘the system works, let’s pop it in’, there are lots of procedures to be completed before

integrating a system. These include requirements writing, safety assessment analysis, human

factors considerations, certification and rigorous testing of the system. Therefore, once these

procedures are completed, would the integration of a Voice Recognition system aboard

commercial aircraft actually be practical?

~ 6 ~

Flight Deck Instrumentation

Instruments – The Basics A standard instrument always consists of three main components:

- A sensor

- A processor

- An indicator

The senor is sensitive towards the measured parameter. It converts the parameter measured

into a quantity which is simple and easy to process. The processor takes the parameter input from

the sensor and prepares it for indication. The processor accounts for factors such as calibration,

fault detection, fault correction and signal amplification. Finally, the indicator displays the

measured parameter in the correct format. These three components are traditionally situated

within the same housing. Over time, however, both the processor and sensor were located within

an analogue or digital computer, away from the indicator housing.

Figure 1: Simple Flowchart of Instrument Components

Modern day developments include the positioning of the sensor at a location where the

conditions for measuring the parameter are ideal. A data bus has also been incorporated to send

the signals to a digital processor, this forms part of a computer network. The signals are then

processed and sent to the dedicated aircraft instruments, systems and displays. The instruments

themselves also feature glare shield technology. A glare-shield is a colourless shield which

covers the main instrument panels and protects them from direct sunlight.

Sensor Processor Indicator

~ 7 ~

The vast majority of modern day passenger aircraft have a similar layout of their instruments

in specific instrument panels. Radio and communication controls are found between the pilot and

co-pilot seats on a pedestal. The overhead panel houses the control panels for all of the aircraft’s

systems. Facing the flight crew is the main instrument panel. The uppermost central part mostly

houses the autopilot controls. The centre main instrument panel houses engine indicators. The

left and right main instrument panels house the instruments like airspeed indicator, altimeter etc.

and is identical for both pilots. A basic image of this can be found below in Figure 2.

`

Figure 2: Aircraft Instrument Panels

For Figure 2 the acronyms are as follows:

- OHP – Overhead Panel

- MCP – Mode Control Panel

- LMIP – Left Main Instrument Panel

- RMIP – Right Main Instrument Panel

- CMIP – Centre Main Instrument Panel

- PED – Pedestal

OHP

MCP

LMIP RMIP CMIP

PED

~ 8 ~

The basic ergonomic instrumentation principle is the ‘basic t-shape’ or ‘basic 6’

arrangement. It houses the six fundamental instruments:

- Artificial Horizon

- Heading Indicator

- Airspeed Indicator (ASI)

- Attitude Indicator

- Vertical Speed Indicator (VSI)

- Altitude Indicator

- Rate of Turn Indicator

Figure 3: Basic T-Shape Instrument Layout

The earliest flight deck instrumentation examples featured an endless amount of analogue

instruments all in very close proximity to each other, literally side by side. An example of the

flight deck of a Boeing 707 type aircraft can be seen overleaf in Figure 4.

~ 9 ~

Figure 4: Flight Deck of a Boeing 707

It can be seen in Figure 4 that there are two ‘basic-6’ arrangements, one for each pilot. In

the centre, the analogue engine instrumentation can be seen. To anyone without any aircraft

knowledge, or even those with some basic knowledge, the earliest flight deck examples may

seem a bewildering concept and the demand for pilots to be in control and supervise the

configuration of the aircraft must have been very high.

As evolution in the aircraft industry began, with airliners being the way forward, the

flight deck instrumentation progressed too, and although still very much an ‘analogue’ flight

deck, the example seen in Figure 5 overleaf, of a Boeing 737-200, already seems simpler and de-

cluttered somewhat.

~ 10 ~

Figure 5: Flight Deck of a Boeing 737-200

Figure 5 shows the most modern example of an analogue flight deck and its furthest

stage of development in the mechanical flight deck instrumentation phase before the birth of the

electronic era.

~ 11 ~

Electronic Displays

As the aviation industry moved forward, the technology used in the flight deck

progressed also, with the introduction of electronic display methods. There are several different

types of electronic display:

- Incandescent (Earliest)

- Light Emitting Diode (LED)

- Cathode Ray Tube (CRT)

- Liquid Crystal Display TFT (LCD)

Below in Figure 6 is an example of a typical seven-segment incandescent display.

Figure 6: Seven-Segment Display

An incandescent display is an electronic display used for displaying numerical characters

and decimals. The most common incandescent displays feature seven segments; hence they are

named seven segment displays. For the ease of reading, they are often, as shown in Figure 6,

slanted slightly. Their earliest principle of operation concerned the illumination of incandescent

light bulbs housed behind each of the seven segments.

~ 12 ~

More modern seven segment displays use LEDs (Light Emitting Diodes) for illuminating

the seven segments. In Figure 6 on the previous page, all of the 7 segments are illuminated,

producing a number 8. The decimal above is not illuminated in this case. LEDs work by

producing light when conducting electricity. The colour of an LED can be easily altered by

simply changing the cover placed over the segment. Examples of LEDs can still be found in the

flight deck today on, for example, the autopilot control panel as shown below in Figure 7.

Figure 7: LEDs on Boeing 737 Mode Control Panel (MCP)

~ 13 ~

The next step was the introduction of the CRT (Cathode Ray Tube) display system. This

began the EFIS (Electronic Flight Instrument System) principle, where information is displayed

on screens, with analogue instruments used only as a back-up. The displayed parameters

displayed on the EFIS are the: PFD (Primary Flight Display) which shows the artificial horizon,

heading, airspeed, altitude and vertical speed; the MFD (Multi-Function Display) which displays

information about the weather and navigational principles; and the EICAS (Engine Indicating

and Crew Alerting System) which displays all relevant engine instrumentation such as

temperatures, fuel levels and alerting systems should any problems occur. The principle of a

CRT itself, concerns the operation of an electron ‘gun’ firing at a fluorescent screen to produce

an image. Most televisions used this technology before the invention of LCDs (Liquid Crystal

Displays) and plasma technology. A diagram of a CRT can be seen in Figure 8 below with a

description following.

Figure 8: Cathode Ray Tube

The principle of the CRT is the operation of the negatively charged cathode and heater system.

The heater when active produces electrons, and due to the cathode also being negatively charged,

the electrons are repelled away from it in the direction of the fluorescent screen. The anodes

increase the velocity of the electrons due to their opposite charge. The deflecting coils then direct

the electron flow to hit the desired part of the fluorescent screen and this proportion of the screen

lights up. The fluorescent screen is made up of Phosphor dots or lines, which are coloured red,

green and blue. These 3 basic colours form the basis of all colours that can be produced by the

screen when mixed adequately.

~ 14 ~

The most modern development is that of the LCD (Liquid Crystal Display). The LCD

uses liquid crystals and the modulation of light to produce images on a screen. The liquid crystal

solution is held in between two transparent screens, through which light is projected through.

The light is either allowed to pass through the solution, or is blocked by it, thus producing the

images. They are of poorer quality from a resolution point of view, compared to CRTs but are

lighter and easier to flat-pack into a smaller area. In addition to this, LCD TFT displays are the

latest to be developed. These are essentially LCD screens that feature a TFT (Thin Film

Transistor). The thin film transistor improves the quality of the produced image, and hence are

features of the majority of EFIS’s integrated into modern aircraft. An example of an LCD TFT

can be seen in the Boeing 787 flight deck shown in Figure 9 below.

Figure 9: LCD TFT Screens on Boeing 787

It is due to this ever increasing technology that the pressure on pilots to monitor all

systems have decreased, and with aircraft becoming ever more increasingly automated, it is

important that adequate instrumentation is also modified to ensure that the aircraft remains as

safe as possible. It could be thought that increasing modifications and progression in HUDs

(Head-Up Displays) would be the next step in the progression of flight deck technology.

~ 15 ~

The Integration of a System

Requirements In reality, a system would not be integrated into the flight deck of the aircraft because it is

deemed a good idea. It must contribute positively in some way to the operation and/or

performance of the aircraft and/or personnel on board. There must be, therefore, a ‘requirement’

for the system to be there in the first place. Therefore, a set of System Requirements would be

compiled, usually by way of the customers stating ‘this is what we want and why’, or by way of

the manufacturer trying to sell the product in stating ‘this is why you need this package.’ The

requirements themselves need to be properly evaluated in terms of being; correct, feasible,

necessary, of a certain level of priority, non-ambiguous and finally verifiable (See Figure 10).

Thus, each requirement must be analysed, verified, well documented and derived.

Requirement System Requirements Definition

Correct Must contain a correct description of the intended functionality of the system

Feasible Must be realistic in terms of system integration into the intended environment

(Shows an acknowledgement of both capabilities and limitations)

Necessary Must outline the real requirement for the system in terms of why it should be

integrated

Prioritised Must outline the importance of the system in terms of essentiality

Unambiguous Must be no misunderstanding or misinterpretation of what the requirement is

Verifiable Must show the ability to perform tests and demonstrations of the system and

concludes whether these requirements are properly demonstrated in the system

Figure 10: Requirement definitions

~ 16 ~

Safety Assessment

Safety Assessment is perhaps the most important consideration when integrating a

software package into an aircraft. After all, if the system is unsafe, there is a high risk of

catastrophic loss of the aircraft and its occupants. There are differing methods of safety

assessment procedures, examples being FHAs, FMEAs and FMECAs.

An FHA (Functional Hazard Assessment) is a technique used to investigate the effects of

functional failures of parts of the system. The primary intention of an FHA is to identify the

hazardous function failure scenarios. The method of conducting an FHA is relatively

straightforward:

- For a suitable process, select functions in turn

- Define the behaviour and purpose(s) of the function

- Consider and cater for hypothetical failure modes*

- Determine the effects of these

- Determine associated risk factors**

- Record associated risk factors

- Justify the associated risk factors

*Failure modes include ‘Loss of function’, ‘Operation when not required’, ‘Incorrect operation’

etc.

**Risk factors include ‘sensitivity’ and ‘probability budget’

FHA results are usually displayed in a table as shown below:

Function Failure Condition Phase Effect Class Verification

- - - - - -

Figure 11: FHA Table

~ 17 ~

Another two methods which are very similar to one another are FMEAs and FMECAs.

An FMEA (Failure Mode and Effects Analysis) is a systematic approach to the identification of

risks and an analysis of determining possible failure modes whilst trying to prevent their

occurrence. An FMECA (Failure Mode, Effects and Criticality Analysis) however is basically an

extension of an FMEA, as in addition to covering the basics of an FMEA, it also takes into

account a criticality analysis. A criticality analysis is used to record the likelihood of failure

modes occurring against the severity and impact of their consequences:

‘The result highlights failure modes with relatively high probability and severity of

consequences, allowing remedial effort to be directed where it will produce the greatest value’

Figure 12: Quoted Explanation of FMECA

In addition to the above, the four fundamental aspects of a FMECA are as follows:

- Identification of Faults (i)

- Potential Effects the Fault can Cause (ii)

- Existing and/or Projected Control and/or

Compensation (iii)

- Summary of all Findings (iv)

(i)- Identifies any possible hazardous conditions that can develop

(ii)- Explains why there is a problem as a result of the condition

(iii)- Describes the actions which have to be taken in order to compensate for and/or control the

problem

(iv)- Conclusively states whether the situation is under control or whether further action is

required

The effects of a system can be defined as negligible, marginal, critical or catastrophic.

These effects can also be seen in the Criticality Level Pyramid which is shown overleaf in Figure

13 and a description follows.

~ 18 ~

Figure 13: Criticality Level Pyramid

Level A - Catastrophic: often results in complete loss of the aircraft, passengers and

crew. It can happen by placing too much of a workload on the crew. An example would be a

function or system failing and all of a sudden the pilots could be too focussed on dealing with the

breakage that a catastrophic failure then occurs. The catastrophic event doesn’t necessarily result

from a faulty indication or failure of a system that removes important data, it could be that the

initial breakage could cause so much of a problem in terms of correction, that the pilots cannot

maintain control over the aircraft and the situation at that time.

Level B – Hazardous/Severe: may not involve the complete loss of life but can

potentially lead to such a serious condition.

Level C - Major: can cause loss of control of the aircraft and potentially cause injuries to

passengers and crew.

Level D - Minor: can cause some sort of detrimental effect, however the aircraft can often

overcome the situation and pilots can maintain control of the aircraft.

Level E – No Effects: results in no noticeable effect. As discussed earlier the differing

system safety assessment processes identify the criticality levels. If Level E is declared, then

there is a requirement for some sort of justification either by the manufacturer or customer.

~ 19 ~

What is Certification?

Certification is a means of protection. Certification ensures that the package considered

for integration provides security and means of protection. In other terms, it should not detriment

the safety of the aircraft, its occupants, the ground upon which it operates and the ground which

it flies over. It must not cause any potential dangers to any products or goods carried within

and/or within the vicinity of the aircraft. There should be no interference caused to external

equipment or systems.

There are two main parts to certification of a system. There is the TSOA (Technical

Standard Order Authorisation) and the TC (Type Certificate) or STC (Supplemental Type

Certification). The TSOA concerns the development of an aircraft avionic appliance. This can

include the likes of Weather Radars, Radio Communication Devices and other ‘boxes’ like

EGPWS (Enhanced Ground Proximity Warning System) etc. The TSOA concludes whether a

system is permitted for use on an aircraft in that it meets the minimum standard required for

installation. After approval with regards to the TSOA however, the system is not given the go

ahead just yet. Next, the system must comply with either a TC or an STC. Both are similar in that

they concern the actual system installation onto the aircraft. They differ in that a TC is with

regards to a new aircraft, whereas an STC is for the addition of a new system onto an already

existing aircraft that is already in service.

There is a lot more factors of consideration in certification than just TSOAs and STCs

though. With regards to integrating systems involving software, for example, a PSAC (Plan for

Software Aspects of Certification) must be compiled, which is one of Five Key Plans which will

be discussed later.

A PSAC is quoted as:

‘The primary means used by the certification authority for determining whether an

applicant is proposing a software life cycle commensurate with the rigor required for the level of

software.’

Figure 14: PSAC Definition Quote

~ 20 ~

In other words, a PSAC is the first stage in concluding whether the proposing body, is

putting forward a software development structure which is of an adequate severity for the

software intentions. It is generally around 20-30 pages in length and features certification

considerations, a system overview, and a software overview (See Figure 15 below). It will also

discuss system architecture, criticality and safety with some referencing to any other designs

and/or equipment. It should then be submitted to and approved by EASA (European Aviation

Safety Agency).

PSAC Sections PSAC Sections Explanation

Certification

Considerations

A summary of the certification principle which includes aspects such as

means of compliance relating to SAC (Software Aspects for

Certification). It should also define the proposed software level(s) and

includes the conclusion from the SSA (System Safety Assessment)

process, with the inclusion of possible software contribution to any

conditions involving failure

Software Overview Features a brief specification of the software functions with particular

concentration on proposed safety and partitioning concepts for example:

‘resource sharing, redundancy, multiple-version dissimilar software,

fault tolerances and timing/scheduling strategies’

System Overview Provides a system overview which should include a depiction of the

system functions and their distribution to both the hardware and software.

It should also provide information regarding safety features, hardware

interfaces, software interfaces system composition and any processing

units used

Figure 15: PSAC Sections and their Explanation

~ 21 ~

The second of the Five Key Plans is QA (Quality Assurance). This addresses the role

which QA plays throughout the whole development process from beginning to end. It also

ensures that all planning is coordinated and followed. It ensures that any criterion for transition is

conformed to. It should also: ‘address conformity reviews and inspections whilst providing

guidance and timelines for audits/reviews by QA (including checklists.)’

The third of the Five Key Plans is CM (Configuration Management). It concerns the

‘tight configuration control of the product, data item documents and artefacts of the lifecycle.’ It

will also depict the numbering and naming protocol along with the approach to revision.

The fourth of the Five Key Plans is SWDP (Software Development Plan). The SWDP

concerns all aspects involved in development including personnel, scheduling, deliverables,

reliance, stages of development and any involved organisations. It will also address the

development, surrounding environment and any equipment used. Lifecycle processes should be

mentioned also i.e. ‘requirements, design, coding, integration, testing, changes etc.’ It will

address the roles of the QA and CM and any deliverables.

The fifth and final aspect of the Five Key Plans is SWVP (Software Verification Plan)

addresses acceptance and analysis of the software, traceability and virtually every aspect of

testing and actual integration of the software. It will also consider the test surroundings and

equipment, re-verifications and reversion and also any resource materials and the project

management.

1.PSAC 2.QA 3.CM 4.SWDP 5.SWVP

Figure 16: Five Key Plans

~ 22 ~

Testing Principles

There are four standard types of test procedure when it comes to performing software

tests:

- Functional

- Normal Range

- Robustness

- Structural Coverage

Functional tests are those which cover every performance aspect of the software in terms

of functionality i.e. Does the software do absolutely everything that it is designed to do?

Functional tests cover everything from Low-Level tests to High-Level critical tests.

Normal Range testing is different in that it tests the functionality in terms of what the

design is. It tests that the software performs the tasks that it is intended for under normal

operating conditions. These tests can sometimes exercise the software and the basis is to make

sure the software operates the way you would want it to in an everyday scenario. It is common

that Normal Range test procedures have in fact already been covered by the functional tests, as

functional tests cover everything. There may be additions made to cover aspects, if any, not

covered by the functional test. This ensures everything within the ‘Normal Range’ of operation

has been covered.

Going a step further, Robustness testing involves ‘invented’ scenarios and really tests the

software’s ability to perform under abnormal and challenging conditions. For example an event

that, albeit would occur infrequently but in reality could still occur during real operation, is

created to push the software to its absolute limits. In comparison to a physical structure, it would

be tested until broken to observe the maximum load it can take. Effectively, this test tries to

‘break’ the software. It includes an error injection principle which states; ‘Things that will not

occur under normal operating conditions.’ This can include errors in the initial input of

information, data corruption, system interference, and ultimately any possible scenarios,

including those so extreme that they really should never be reached. It is effectively a test

procedure covering every ‘What if?’

~ 23 ~

Finally, Structural Coverage testing is ‘an analysis of how well you have written the

requirements and tested them.’ It is effectively ‘the final defence against software errors’ or the

‘exit criteria.’ In other words, it is more of an analytical procedure than an actual test. The

following section, which begins overleaf, will discuss the integration of a voice recognition

system with regards to requirements, safety and testing.

A testing procedure for voice recognition software can be found on page 29. This is the

test that was performed when testing the software to provide the results obtained within the

Appendices.

~ 24 ~

Integration of Voice Recognition

Software

The Requirement The flight deck of an aircraft often seems an array of endless switches, knobs, buttons

and screens displaying limitless amounts of in-depth information. It must be very demanding for

a pilot to try and troubleshoot any problems that may occur during a flight, or indeed to check all

aspects of aircraft configuration at a particular time, even more so under pressure during take-off,

landing or mid-air incident. Therefore, it would be thought that integrating voice recognition and

response software could ease the pressure on pilots allowing them to be in as relaxed a state as

possible during long transatlantic flights or even during pilot training.

Safety Assessment

It would be difficult to perform a full scale FHA and somewhat impractical, therefore a

sample table is shown below in Figure 17.

Function Failure

Condition

Phase Effect Class Verification

Pitch Nose Up Aircraft Pitches

up too early upon

take-off

Take-off Aircraft pitches

up at too low a

speed and stalls

at low altitude

Level A

Autopilot

Voice Control

Activation

The system is

activated when

not requested and

performs

unwanted

operations

Any

phase of

flight

Loss of control of

the aircraft at that

particular phase

of flight

Varies,

dependent

upon the

phase of

flight in

which it

occurs

Figure 17: Sample FHA Table for Voice Response Software

~ 25 ~

With regards to Figure 17 above, an FHA is a very substantial piece of documentation

which would cover every possible consideration and would be passed amongst many parties to

include unthought-of scenarios and possibilities. Due to time limitations only a very brief sample

is shown above to cover the basic methodology in writing a Functional Hazard Assessment.

So, in terms of safety, every possible parameter must be considered and covered by

FHAs, FMEAs, and FMECAs etc. This report will only provide an understanding of the basic

background and explanation of these as, due to limitations with time and personnel, it would not

be realistic to cover every possible consideration.

Another method in which safety is the main consideration is fault trees. They consist of a

list of possible parameters that would either combine to cause failure, or independently cause the

same failure as some other occurrences. So, for the voice recognition principle, there are two

examples of occurrences in which there is unintentional operation of the system and then the

overall failure condition, with a fault tree analysis following.

Example 1: The first consideration is how the software can be safely and securely activated in

such a way that it only processes intended inputs, and does not act upon any vocal occurrences

during a flight such as any casual or required conversation between the Captain, First Officer,

Cabin Crew and ATC (Air Traffic Control) etc. The problem here, therefore, is unintentional

activation via an unintended vocal input. Hence, the system could become very unpredictable in

that the vocal input acted upon could be virtually anything said during the flight, including

casual, non-flight related chat, dependent upon words spoken and heard by the system. An

unrealistic, emphasised example, for the purpose of explanation could be the First Officer saying

to the Captain, or vice versa, ‘I almost accidentally turned the autopilot off (or on) there!’ Now

assuming that ‘Autopilot Off’ or ‘Autopilot On’ are both commands, the system could interact

and actually turn the autopilot off or on at an undesirable time, which was the initial avoidance

made by the First Officer in this instance. It is due to this unpredictability that the Criticality

Level of such an occurrence could be variable dependent upon the phase of flight. I would

therefore class this problem as Level B: Hazardous/ Severe.

~ 26 ~

Example 2: Secondly, another problem occurrence could be the effects of Electro-Magnetic

Interference. At any one time, an aircraft has many, many signals and currents passing through

the extensive wiring system. With the operation of other systems such as the engines, APU

(Auxiliary Power Unit), power generators, radio signals etc. Electro-Magnetic Interference

would be a highly possible occurrence. This interference occurrence within the wiring system

could cause problems with performance within the system, and indeed any system causing

system failure, false inputs and/or corrupt data. Taking the same problem occurrence as above

into account – unintentional activation or deactivation of the autopilot – then ultimately the result

would be the same failure condition. There is a requirement for all wires within the system to be

there, and they are grouped together efficiently in terms of structural concepts and therefore this

problem could also be classed as Level B: Hazardous/ Severe dependent upon the interference

and failure condition.

~ 27 ~

If both of these occurrences are taken into account as taking place during a phase of flight

in which the pilots have a very limited timeframe, for example around 20-30 seconds, in which

to react to save the aircraft from a condition of inevitable, catastrophic loss of aircraft and

occupants, the severity of occurrence is made clearer. So, should either of these conditions occur

and turn the autopilot on unintentionally and the aircraft performs a manoeuvre in which the

result would be catastrophic loss of the aircraft and its occupants if uncorrected within a period

of around 20-30 seconds, it would be presumed that the pilots would seek to quickly deactivate

the system and perform corrective action manually to regain control of and ultimately save the

aircraft. However, in terms of the occurrence of Example 1, imagine the aircraft does not respond

to the ‘Autopilot Off’ command. Human factors considerations would suggest that the pilot

would have raised adrenaline levels due to stress (see page 28) , and would not use the same

vocal tone as under normal conditions, therefore would there be sufficient time to manually

disable the autopilot and regain control? In terms of Example 2, imagine the same problem

occurs, this time however, the autopilot is unintentionally activated as a result of Electro-

Magnetic Interference. It would therefore be assumed that the pilots would react in the same

manner, as the same problem has occurred. If the system fails to react then would there be

sufficient time to manually disable the autopilot and regain control? Although, imagine the

software does respond, to the ‘Autopilot Off!’ command this time, but the Electro-Magnetic

Interference again almost instantaneously re-activates the system due to a fault in the wiring

system, is there any way of the pilots overcoming such a problem at all, never mind within in the

20-30 second timeframe?

~ 28 ~

Thus, for the above conditions, a very brief fault tree is shown below in Figure 18:

Figure 18: Fault Tree Analysis

For the fault tree analysis in Figure 18 above:

1. Aircraft receives false input and acts upon an unintentional command

2. System does not have the ability to be overridden with vocal changes due to

adrenaline

3. Electro-Magnetic Interference occurs

4. There is no kill switch/ easy access circuit breaker to disable current within

the wires activating the system

5. AND Gates

6. OR Gate

In Figure 18, it can be seen that if occurrences 1 and 2 were to occur, or if occurrences 3

and 4 were to occur, then the result would be the loss of the aircraft and its occupants.

Therefore it would be thought that an efficient method of enablement, control and

override would be compiled to resolve this.

LOSS OF AIRCRAFT AND

OCCUPANTS

1. 2. 3. 4.

5. 5.

6.

~ 29 ~

System Design

Taking the example fault tree problem occurrence in Figure 18, there would be a

requirement to redesign the software to overcome the problem occurrences discussed. For this

example, it can be concluded that, in order to maintain aircraft safety and preserve life in this

situation, then there would be a requirement for changes made to the enablement logic, control

logic and override logic of the voice recognition system to ensure that the pilot can activate,

control and deactivate the system quickly and effectively. In order to achieve this, a secure

enabling method, efficient control method and effectual override functions are established.

Enablement: It would be initially thought that until proven that the voice recognition system is

effective, it should use a PTT (Push-To-Talk) system. This would involve the pilot having to

physically push and hold a button whilst giving a vocal command to activate the listening device.

This would, significantly reduce the occurrence of factor 1 considered in Figure 18 as the pilot

would tend not to give a false input whilst holding in the required activation button. However, if

proven that the voice recognition system is of a very high standard then it would develop into a

totally vocal dependent system, independent of any physical input. There would be the

requirement however, for a vocal activation security code word or sequence in order to

differentiate an intentional command from any other vocal noise during a flight such as ATC or

discussion between Flight Crew and Cabin Crew. The vocal activation security code word or

sequence would essentially ‘grab the system’s attention’.

Control: In the early stages of the development process, it would be thought that the

system would be initially speaker independent (see Figure 19). However, as the system is

developed and inevitably establishes a higher accuracy, it would be thought that a speaker

dependent (see Figure 19) system would be incorporated. However, this presents its own new

difficulties. The sheer number of pilots that fly one specific aircraft would present problems with

regards to how the system stores and remembers the different voices. Therefore, an effective

method of training the system would be desirable. One suggestion could be that the system

listens to and learns a pilot’s voice during a pre-flight checklist, which would not waste time as

the pre-flight operations will also be completed during this.

~ 30 ~

System Type System Type Description

Speaker Independent Functions regardless of the operator. This type of system is the

most favoured for everyday general usage. There is a decrease in

the accuracy rate and response rate therefore is not as accurate or

responsive as a Speaker Dependent system

Speaker Dependent Functions dependent upon the operator. This type of system is

required to be trained by the intended operator by detecting the

vocal properties, acoustic properties and speech patterns of that

particular speaker. There would be an increased response rate

and accuracy rate than the Speaker Independent system as a

result.

Figure 19: Speaker Independent and Speaker Dependent Systems

Override: It would be desirable for the system to feature an effective override method i.e. a

kill-switch, in the event of an emergency or system failure. In terms of a PTT system, the process

would be simple, release pressure from the button and the system stops listening, unless the

button becomes ‘stuck’. To overcome this, it could be thought that the button would be spring

loaded and not too tight against the orifice in which it is situated. When deactivated using this

method, if the aircraft is still performing a previous vocally activated manoeuvre or command,

then it could be overridden by the pilots following the procedures they would normally use in

overcoming that particular occurrence e.g. turn the autopilot off manually, if previously or

undesirably activated by voice. In terms of the Speaker Dependent system, a vocal command

would be used to deactivate the system. Although from a Human Factors point of view, stress

(see page 28) could cause changes to the vocal properties, acoustic properties and speech

patterns of that particular speaker which could lead to difficulties with vocal deactivation.

Therefore, it would be thought, that a kill-switch or circuit-break method would need to be

considered. This would allow the pilots to kill power to the system and regain normal control of

the aircraft whilst eliminating the Electro-Magnetic Interference problems discussed previous

due to the absence of current flowing through the wires.

~ 31 ~

Human Factors – Stress

There are many Human Factors aspects which would affect the proposed Voice

Recognition operation, and every single one of these factors would need to be taken into deep

consideration and analysed in terms of safety. To provide an example of the Human Factors

Considerations aspect of integrating a Voice Recognition package, a very brief discussion of the

effects of stress will follow.

Stress is the reaction of a person to mental and physical strain placed upon him/her. This

response to stress includes the release of chemical hormones for example adrenaline, into the

bloodstream. The person’s metabolism is increased which provides positive energy to the

muscular system. As a result, factors including blood sugar, heart rate, breathing, blood pressure

and sweating are increased. The factor which causes stress is known as a stressor and these can

either be physically, mentally or physiologically induced. An example of physical inducement

would be g-force or vibration. An example of mental inducement would be working with

difficult personal circumstances out-with the work environment. Finally an example of

physiological stress would be fatigue.

Stress can also be divided into two types, chronic and acute. It would be assumed that if a

pilot were experiencing/ diagnosed with chronic stress then he/she would be deemed unfit to take

control of an aircraft, therefore acute stress is more relevant in this instance. Acute stress is

induced by a sudden sense of danger. It triggers a ‘do or die’ scenario that is either real or falsely

perceived/exaggerated. It would be normal for a person to be able to effectively deal with acute

stress at that time, however if it is on-going it could become chronic.

In terms of the Voice Recognition system, it would be thought that any signs of excessive

mental or physical stress would have an effect on the competency of the pilot and his vocal

rhythms could be altered by this, and therefore the performance of the system would also be

affected as an incorrect input rarely achieves a correct output once it has been processed.

Examples of physical stress include tension in the muscles, shivers and the clench of the jaw,

similar to the phenomena experienced when on a thrill ride at a theme park for example. It would

be thought that any vocal input under these conditions would have a significantly reduced

accuracy and hence the operation of the system would suffer.

~ 32 ~

Test Procedure - Methodology Job Setup

Step Procedure Completed

(Tick)

1 Activate the power supply for the intended equipment

2 Perform the start-up procedure for the computer in which the software

package is installed

3 To activate Windows Speech Recognition follow these steps:

- Click ‘Start’

- Click ‘All Programs’

- Click ‘Accessories’

- Click ‘Ease of Access’

- Click ‘Windows Speech

Recognition’

4 Follow the on-screen instructions to check that the microphone intended for

use is plugged in and operational

Normal Range Test – Action

Step Procedure Completed

(Tick)

1 Follow the ‘Job Setup’ procedure

2 Complete the speech recognition tutorial (if required)*

3 Open the Microsoft Word program using the following steps:

- Click ‘Start’

- Click ‘All Programs’

- Click ‘Microsoft Office’

- Click ‘Microsoft Word (2010 etc.)’

4 Ensure that the cursor is flashing, signalling that it is prepared for a typed, or

vocal input

5 To activate the listening device, have the test performer say ‘Start Listening’

and record how many attempts it takes for the system to respond properly

6 Have each test performer, in turn, say each of the four predetermined

sentences (see Figure 21) three times and record how the output compares to

the original sentences

7 Have the test performers deactivate the listening device by saying ‘Stop

Listening’ and record how many attempts it takes for the software to respond

8 Perform the ‘Close Up’ procedure

~ 33 ~

* For the basis of comparison, during the first stage of Normal Range testing, the tutorial will

not be used. The second stage of the test will require the tutorial to be completed as the system is

trained to the voice of the test performer. The first and second tests will determine whether

training the system produces a higher percentage of accuracy than leaving the system un-trained

(see Figure 19).

Robustness Test – Action

Step Procedure Completed

(Tick)

1 Follow the ‘Job Setup’ procedure

2 Complete the speech recognition tutorial (if required, see previous)*

3 Open the Microsoft Word program using the following steps:

- Click ‘Start’

- Click ‘All Programs’

- Click ‘Microsoft Office’

- Click ‘Microsoft Word (2010 etc.)’

4 Ensure that the cursor is flashing, signalling that it is prepared for a typed, or

vocal input

5 Repeat the Normal Range Test – Action steps 5, 6 and 7,

incorporating the following factors:

- Background noise

- Unintentional command

- ATC transmission at the same time

- Quick speech

- Slow speech

- Shouting over increased

background noise

- Heightened stress

(Details/ideas of how to achieve these factors will follow the test procedure)

6 Perform the ‘Close Up’ procedure

~ 34 ~

Close Up

Step Procedure Completed

(Tick)

1 Close Microsoft Word by either clicking the Red ‘X’ at the top right of the

window or by right-clicking the ‘Microsoft Word’ icon on the toolbar and

clicking ‘Close’. There is a requirement to save the document for compiling

results later

2 Close Windows Speech Recognition by either clicking the Red ‘X’ at the top

right of the window or by right-clicking the ‘Windows Speech Recognition’

icon on the toolbar and clicking ‘Close’

3 Disconnect the microphone used

4 Perform the shut-down procedure for the computer used during testing

5 Disconnect the power supply to all equipment

Why and How to create Factors Affecting System Performance

Background Noise

The flight deck is by no means a quiet environment, after all aircraft can be travelling

through the air at speeds of up to and even beyond 500mph. To simulate the noise which would

be heard due to air passing over the flight deck exterior in terms of the test procedure, the most

practical method would be to conduct the test with the Flight Simulator program in operation,

with an aircraft in flight creating the appropriate simulated noise. The test would then be

conducted with the performer and microphone placed close to the output noise of the Flight

Simulator program.

Unintentional Command

There is the potential for the pilots to be unaware of the software status i.e. whether or

not the listening device is active. An example of a possible problem could be the Captain

briefing the First Officer of an upcoming flight procedure. The software could receive and

process this input and perform an unwanted manoeuvre too early. A sample sentence could be:

‘Keep an eye on airspeed, and remember, don’t bring the landing gear down too early!’

~ 35 ~

If processed as a command, the aircraft could bring the landing gear down and significant

structural damage could result. It would be thought however, that the pilots would be purposeful

in giving commands to the system and that their vocal tones would differ between giving

commands and general chat. This factor would then measure the sensitivity of the device.

ATC Transmission

Similar to the two previously discussed factors in that background noise in the form of an

ATC transmission to another aircraft could be picked up from the pilot’s earpiece as an

unintentional command by the system. So, for this part of the test, there would be a requirement

for two test performers, one acting as the pilot and one as ATC. The pilot would say the

predetermined sentence as outlined in the test and the ATC would say another command as if

talking to another aircraft, at the same time. This test is to see which, if any, command the

system follows or whether the system becomes confused.

Quick/Slowed Speech

Due to the effects of hypoxia or adrenaline, the voice of the pilot could be quickened or

slowed, producing an abnormal input to the system. This would be simulated by the test

performer speeding up/ slowing down their speech accordingly. This further tests the accuracy

and sensitivity of the system under abnormal conditions.

Shouting Over Increased Background Noise

Follows the same idea as background noise outlined previous although would involve

significantly increased audio output. This would simulate a situation in which the aircraft is in a

state of real danger and the pilots could be frantically trying to regain control of the aircraft. This

could be achieved by increasing the volume of the Flight Simulator program, having aircraft

warning systems going off and the test performers shouting over this in an attempt to process the

commands.

~ 36 ~

Heightened Stress

This would be to simulate a period of heightened stress (see page 28) for the pilots and

could be simulated via some form of physical test to induce breathlessness and a sign of

heightened stress. Simulation could be achieved by:

- Treadmill (ideally)

- Holding/ lifting a heavy object

- Arm wrestle

- Shadow boxing

- Star jumps

Next, a set of sentences and the test performers should be obtained. Now, ideally, this test

should be performed with a very high number of test performers from a very wide variety of

backgrounds and a wide range of different sentences and possible vocal inputs, but due to time

limitations and the conditions for the writing of this report, it would not be realistic to do so,

therefore the test will be performed with 4 sentences by 5 people.

Participant Participant Name

1 Frazer Hamilton

2 Kirsten Gallagher

3 Antony Templeman

4 Eric Mutasa

5 Gordon Keary

Figure 20: Test Performers

~ 37 ~

Statement Number Test Sentences

1 She sells sea shells on the seashore

2 Turn left heading 270

3 Full throttle

4 Climb flight level 360

Figure 21: Test Pre-determined Sentences

The results are then compiled into spread-sheets which feature as Appendices in this report.

Test Procedure - Summary of Findings

Before outlining and summarising the results of the test, it should be noted that

abnormalities are to be expected within the results. This is due to time limitations experienced

when performing the test involving Eric Mutasa. This test performer was only available to

perform the Normal Range tests and did not complete the Robustness section of the test.

Figure 22: Table of results

0

10

20

30

40

50

60

Statement 1 Statement 2 Statement 3 Statement 4

Statement

Average Percentage Accuracy

Average PercentageAccuracy

~ 38 ~

It can be seen in Figure 22, that Statement 1 clearly demonstrates the highest accuracy

level at 55%. This demonstrates that the software could understand and process this sentence

easier and could be symbolic of the system finding longer, flowing sentences easier to process.

There seems to be an alarming level of accuracy with regards to Statement 3 at just over 10%.

This could suggest that the system struggled with the much shorter, to the point, statement.

Statements 2 and 4 are of a similar accuracy and are essentially the midpoint in terms of length

of the sentences and accuracy. However, the more words there are the more accuracy there is

likely to be, so the results are typical of what would be expected in this case.

Looking at Appendix A and Appendix B, it can be seen that there is a very wide variety

of accuracy levels within every aspect of the test. There is no real consistency between the type

of test, and the test performers, and the results seem somewhat random. Although, in Appendix

C, with the exception of Participant 4, the results for Statements 1 and 2 show the only real signs

of consistency in terms of the statements. In terms of the Participants, the majority of the

participants follow similar performances relating to the graph in Figure 22 above, in that they

performed best in Statement 1, showed significant decrease in Statement 2, further decrease in

Statement 3 and an increase in Statement 4. Participants 1, 3, and 5 have accuracy levels of

around the same general area within 10% in all cases. This was expected as these performers

were all males of British origin with a mixture of Scottish and English accents. Participant 2

showed the same pattern but a significantly lower accuracy and was a female of British origin

with a Scottish accent. The reasons for the abnormalities with regard to Participant 4 were

discussed earlier, and it should be noted that he was also a male of British origin with an English

accent. Although during the Normal Range tests he performed, it can be seen from Appendix C

that he produced accuracies of a similar trend to that of Participants 1, 3 and 5.

As also discussed previous, this test would have been conducted with many more

participants from all over the world to rigorously test the software and give more accurate

results, but due to time limitations and the basis of completion of this report, a rough and brief

test was conducted to show the theory behind a test procedure to enhance the reader’s

understanding.

~ 39 ~

Flight Deck Evolution – The Path to

Automation and Beyond

A Conclusion Technology within the aviation industry has come a long way, and it can be clearly

observed that the flight deck of an aircraft is certainly at the forefront of modernisation and

increased automation used on board the most modern aircraft nowadays. There are limitations

however in what can be integrated; even the most proven technically advanced systems are put

through a rigorous safety assessment, certification and testing process before being seriously

considered for integration. Safety is top priority in aviation as any system failure could

potentially put the lives of many people at risk both aboard the aircraft, and on the ground. It is

thus that every possible outcome that can occur is assessed, analysed and considered in terms of

certification, safety assessment and incorporates the likes of human factors (stress etc.) and

failure modes. It is thus that, in terms of Voice Recognition Software, it can be concluded that it

is not a practical system for integration based on the results of the test carried out in this

particular report. The results in Figure 22 highlight that the system is not accurate enough and

there are too many uncertainties and erroneous outputs from the system for it to control an

aircraft. The system would need to be refined and updated to incorporate differing accents, inputs

and be somewhat resistant to background noise. Careful consideration would need to be given to

every possible variable factor within the system, its inputs and outputs to ensure that the system

operates accurately and safely. Military examples of voice recognition can be seen aboard the

Typhoon fighter aircraft, therefore voice recognition is certainly a valid and realistic system to be

considered for use aboard commercial aircraft. However, at this time, it remains simply a

concept, that could one day control the commercial aircraft of the future.

~ 40 ~

References

Websites and PDFs

Author unknown (2007) Aeronautics - Flight Instruments - Level 1,

http://www.allstar.fiu.edu/aero/fltmidinst.htm, Date accessed 12/10/2011

Author unknown (Date unknown) Instruments – General,

http://www.nordian.net/pdf/easa_instrumentation_demo.pdf, Date accessed 19/10/2011

Author unknown (Date unknown) Aircraft System Test Procedures Preparation & Release,

http://www.nasa.gov/centers/dryden/pdf/87455main_DCP-O-011.pdf, Date accessed 23/11/2011

Kelly, T.P and Wilkinson, P.J (Date unknown) Functional Hazard Analysis for Highly Integrated

Aerospace Systems, http://www-users.cs.york.ac.uk/~tpk/ieefha.pdf, Date accessed 02/02/2012

Author unknown (2010) Standard glossary of terms used in Software Testing,

http://www.astqb.org/educational-resources/glossary.php, Date accessed 02/02/2012

Various Authors (2009) Notice of Proposed Amendment (NPA) No 2009-12,

http://www.easa.eu.int/rulemaking/docs/npa/2009/NPA%202009-12.pdf, Date accessed

04/03/2012

Author unknown (2005) FAA Federal Aviation Regulations (FARS, 14 CFR),

http://www.flightsimaviation.com/data/FARS/part_25-1301.html, Date accessed 04/03/2012

Author unknown (1996) AC-251309 System Design and Analysis, http://www.astech-

engineering.com/systems/avionics/aircraft/faaac251309a.html, Date accessed 04/03/2012

Author unknown (2011) AN-656 Understanding the Operation of a CRT Monitor,

http://www.ti.com/lit/an/snla017/snla017.pdf, Date accessed 04/03/2012

Haddon, D.R and Whittaker, C.J (2002) Aircraft Airworthiness Certification Standards for Civil

UAVs, http://www.caa.co.uk/docs/1995/srg_acp_00016-01-120203.pdf, Date accessed

02/02/2012

Hilderman, V (2011) Do 178-B to Do 178-C Impact on Avionics Verification and Certification,

https://www.signup4.net/Upload/BOOZ14A/SAFE23E/Day1_03_MilSymposiumJune2011.pdf,

Date accessed 13/04/2012

Various authors (2006) Human Factors and Aviation Medicine, http://flightsafety.org/hf/hf_jan-

feb06.pdf, Date accessed 16/04/2012

~ 41 ~

Books

Pallett, E.H.J (1992) Aircraft Instruments & Integrated Systems, New Jersey: Prentice Hall

Baghai, T and Hilderman, V (2007) Avionics Certification: A Complete Guide to DO-178

(Software), DO-254 (Hardware), Leesburg: Avionics Communications Incorporated

Various Authors (2008) Pilot’s Handbook of Aeronautical Knowledge, Oklahoma: United States

Department of Transportation, Federal Aviation Administration, Airman Testing Standards

Branch

Various Authors (2000) FAA System Safety Handbook, United States of America: Federal

Aviation Administration

Various Authors (1987) Fault Tree Handbook, Washington D.C: Nuclear Regulatory

Commission

Various Authors (2001) Avionics Handbook, Washington D.C: CRC Press

Keary, G (Date unknown) Test Procedure, Prestwick