towards model-based ahmi development

* Université catholique de Louvain (UCL) Belgian Laboratory of Computer-Human Interaction (BCHI)

{juan.m.gonzalez, jean.vanderdonckt}@uclouvain.be† OFFIS e.V., Escherweg 2, 26127,

Oldenburg, Germany{luedtke, osterloh}@offis.de

Towards Model-Based AHMI Development

Juan Manuel Gonzalez-Calleros*, Jean Vanderdonckt*, Andreas Lüdtke†, Jan-Patrick Osterloh†

• Existing work on automated evaluation of AHMI • Some studies suggested that AHMI and software user interfaces

share several similarities:• In terms of interaction techniques for input/outputWilliges, R.C., Williges, B.H., and Fainter, R.G., Software interfaces for aviation systems, in

Human Factors in Aviation, E.L. Wiener and D.C. Nagel (Eds.) 1988, Academic Press: San

Diego: pp. 463-493. • In terms of automatic analysis of workloadEldredge, D., S. Mangold, and R.S. Dodd, A Review and Discussion of Flight Management

System Incidents Reported to the Aviation Safety Reporting System 1992, Battelle/U.S. Dept. of Transportation.

• In terms of automatic evaluation based on goal modelsIrving, S., Polson, P., Irving, J.E., A GOMS Analysis of the Advanced Automated Cockpit, Proc.

of ACM Conf. CHI’94, ACM Press, New York, 1994, pp. 344-350

Introduction

10 April 2023 HCI-AERO, November 3-5 2010

• Several challenges for Aircraft cockpit design, such as: • Introduction of new technologies break previous user experience.

Introduction


• New challenges for AHMI: analysis, design, implementa-tion, and evaluation

• Integrating evaluation in the loop of AHMI life cycle involves the use of pilots and a physical simulator. • Shortcomings: availability, cost, time,…

• Human project: new ways to substitute pilots and physical simulator are explored by coupling them to virtual simulator

• Our goal: to introduce UI automated evaluation (workload, execution time, usability guidelines) in the life cycle

Introduction


• Interactive Cooperative Objects (ICOs)• Used to model aircraft interactive systems (air traffic workstations,

civil aircraft cockpit and military aircraft cockpit)• Formal description for interactive cockpit applications using Petri

nets• Behavioural aspects of systems in the cockpit are modelled• Recent work on usability, reliability, and scalability

• Navarre, D., Palanque, Ph., Ladry, J.-F., Barboni, E., ICOs: A model-based user interface description technique dedicated to interactive systems addressing usability, reliability and scalability, November 2009, Transactions on Computer-Human Interaction (TOCHI) , Volume 16 Issue 4

• Problems • There is a limited set of widgets, User Interface models and

guidelines for the development.

State of the Art


• ARINC 661 Standard• Defines protocols to communicate the dialogue and the functional core

of cockpit display system:

• a user application is defined as a system that has two-way communication with the CDS (Cockpit Display System):

• Transmission of data to the CDS, possibly displayed to the flight deck crew• Reception of input from interactive items managed by the CDS

• A set of widgets is included as a recommendation

• Problems• No design guidelines for the widgets,

• No method to design UIs are considered in the standard

• Each manufacturer is free to implement their own understanding about the standard

• The ARINC standard is not used for primary cockpit applications (AHMI)

• Only deals with secondary applications involved in the management of the flight such as the ones allocated to the Multiple Control Display Unit

State of the Art


• A Methodology is suggested that is composed of:• Models (Traditional Widgets + AHMI specific UI elements)• Language (User Interface Description Language)• Method (Model-Based UI Design applied to the AHMI)• Software support

• A Language Engineering approach• Semantics as meta-models (UML class diagrams)• Syntax as XML Schemas• Stylistics is the visual syntax, in our context not used.

• A Structured Method compliant to the Cameleon Reference Framework for UI development• Just focus in the layer that concerns to the concrete description

model.

Model-Based AHMI Design


10 April 2023

Task and Domain Model

Model to Model

Abstract UI Model

Model to Model

Concrete UI Model

Code Generation

Final UI

Control


Physical Control Software Control

Physical interaction object 2D GUI 3D GUI

HCI-AERO, November 3-5 2010

10 April 2023


Model to Model

Abstract UI Model

Model to Model

Concrete UI Model

Code Generation

Final UI


• The Concrete UI Model (CUI) allows:• Specification AHMI presentation and behaviour independently of

any programming toolkit

Model-Based AHMI Design


X3D OpenGL


Model to Model

Abstract UI Model

Model to Model

Concrete UI Model

Code Generation

Final UI

• Integrating evaluation in the loop of the design of the AHMI imply the use of pilots and a simulator.

• Gonzalez Calleros, J.M., Vanderdonckt, J., Lüdtke, A., Osterloh, J.P., Towards Model-Based AHMI Automatic Evaluation, In: Proc. of 1st Workshop on Human Human Modelling in Assisted Transportation (HMAT'2010), Belgirate, Italy, June 30- July 2, 2010. Springer-Verlag, Berlin.

• Automated evaluation of• Static aspects (UI layout, position of objects)• Dynamic concepts (state of a button during the interaction, color of

the label).

• Any UI is represented in a UsiXML model submitted to automated evaluation (automatic or manual)

• Usability guidelines over the UI objects (distribution of the widgets composing the UI) are evaluated.

Benefits from Relying on a Model-Based Development


• Special attention was paid to those guidelines for standard certification and quality assurance and to express them in the Guideline Definition Language (GDL)

Evaluating the AHMI User Interface


http://www.usixml.orghttp://www.w3.org/2005/Incubator/model-based-ui/

Integrating UI Evaluation in a Simulation Environment


UI timeline


Selection of interaction objects


• Evaluating the AHMI User Interface

• Execution Time

• Workload

• Guidelines

Outline


Evaluation-Execution time


10 April 2023

Evaluation-Workload


10 April 2023

Evaluation-Execution Workload

1.2 + 3x(1.5 + 0.075) + 1.5 = 7.4254.6 + 4x(4.6 + 2.2) = 31.8

WorkloadRemember dateSelect daySelect monthSelect year

Execution Time Select daySelect monthSelect yearValidate

(Use mouse to point at object on screen1.5 second)+Execute a mental “step”.075 secondRetrieve a simple item from a long-term memory1.2 second

4.6Evaluation/Judgment (consider single aspect) + 2.2Discrete Actuation (button, toggle, trigger)


Some guidelines

• Cockpit display systems should at least be consistent with systems of our daily life [Singer 2002]

• Usability guidelines from ISO 9126 could be evaluated• Messages should follow always the nomenclature: first letter in

capital and the rest in lower case• AHMI display systems such as the consistency in the roll index in

the compass rose [Singer 2001]


• A model-based method for the development of the AHMI was presented allowing:• Structured development process (making design more explicit)• UI evaluation

• Traditional measurements can be assessed like UI workload and execution time,

• More complex automated evaluation based on guideines.

• Explore design options, for instance, modality of interaction of the UI can be object of evaluation.

• The original 2D rendering can be equally rendered in 3D.

• A future plan is to automatically generate the AHMI from its model and to submit it to run-time analysis.

Conclusion


towards model-based ahmi development

Technology

domain model model

abstract ui model model

concrete description

ui development

ahmi software

aircraft cockpit design

automated evaluation

theconcrete ui modelcui