software interfaces marek milosz professional network of master’s degrees in informatics as a...

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

1

SOFTWARE INTERFACES

Marek MiloszHMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

2

Agenda

HMI - 2

Types of software interfaces Material for interacting Interaction mechanisms

3

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

Text-based Interface, Text User InterfaceGraphical User Interface

Types of software interfaces

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

4

Text User Interface

TUI The characters displayed on the monitor

in line or lines (text) Semigraphics or pseudographics Until today: emoticons (SMS, Skype, e-

mail,...) Encoding: ASCII ASCII art

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

5

TUI - Emoticons

:) :-) ;-) :D :*

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

6

TUI - ASCII art (1)

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

7

TUI - ASCII art (2)

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

8

Graphical User Interface

GUI Uses: graphics objects User interaction with graphical objects Methods:

Graphics cursor + Keyboard Mouse Touch screen Light pen ...

HMI - 2

9

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

Interface objects

Material for interacting

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

10

Line of text

Type using a keyboard (physical or virtual)

Command line or text box Elements of I/O:

Forms Questions/answers Command line

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

11

Menus

HMI - 2

Set of options displayed on the screen Options visible

less recall - easier to use rely on recognition so names should be meaningful

Selection by: numbers, letters, arrow keys mouse combination (e.g. mouse plus accelerators)

Options often hierarchically grouped sensible grouping is needed

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

12

Forms to fill

HMI - 2

Primarily for data entry or data retrieval Screen like paper form Data put in relevant place Requires

good design obvious correction

facilities

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

13

Windows

HMI - 2

Areas of the screen that behave as if they were independent: can contain text or graphics can be moved or resized can overlap and obscure each other, or can be

laid out next to one another (tiled) Scrollbars – allow the user to move the

contents of the window up and down or from side to side

Title bar – describes the name of the window

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

14

Icons

HMI - 2

Small picture or image Represents some object in the interface

– often a window or action Windows can be closed down (iconised) Icons can be many and various

highly stylized realistic representations standarisation in a particular environment

Hard to develop in many industrial cases

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

15

Icons – standards

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

16

Pointers – cursors

HMI - 2

Shows point of input device activity

Texts vs. graphics Important component WIMP

style relies on pointing and selecting things

Uses mouse, trackpad, joystick, trackball, cursor keys or keyboard shortcuts

Wide variety of graphical images

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

17

Menus – kinds

HMI - 2

Menu Bar at top of screen (normally), menu drags down pull-down menu - mouse hold and drag down menu drop-down menu - mouse click reveals menu fall-down menus - mouse just moves over bar!

Contextual menu appears where you are pop-up menus - actions for selected object pie menus - arranged in a circle

easier to select item (larger target area) quicker (same distance to any option)

… but not widely used!

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

18

Menus - extras

HMI - 2

Cascading menus hierarchical menu structure menu selection opens new menu and so in ad infinitum

Keyboard accelerators key combinations - same effect as menu item two kinds

active when menu open – usually first letter active when menu closed – usually Ctrl + letter

usually different in different applications

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

19

Menus – design issues

HMI - 2

Which kind to use? What to include in menus? Words to use (action or description) How to group items? How to order items? Choice of keyboard accelerators

(possible conflicts)

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

20

Buttons

HMI - 2

Individual and isolated regions within a display that can be selected to invoke an action

Special kinds radio buttons

– set of mutually exclusive choices check boxes

– set of non-exclusive choices Parts of windows/forms

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

21

Toolbars

HMI - 2

Long lines of icons Fast access to common actions Often customizable:

choose which toolbars to see choose what options are on it

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

22

Ribbons

HMI - 2

Kind of toolbars Form of a set of toolbars placed on

several tabs Group and speed using of icons Mainly used in MS Office

23

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

Palettes and tear-off menus

Problem:menu not there when you want it!

Solution: tear-off and pin-up menus

– menu ‘tears off’ to become palette

palettes – little windows of actions – shown/hidden via menu option e.g. available shapes in drawing package

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

24

Dialogue boxes

HMI - 2

Information windows that pop up to inform of an important event or to request information

Dirrerent types: alert, error, informational

Different icons used Modal windows

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

25

Dialogue boxes - symbols

HMI - 2

25

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

26

Other window objects

HMI - 2

Text field Choice field Label List …

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

27

Physical objects for interacting

HMI - 2

Buttons Devices – controllers Accelerometer Cameras …..

28

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

Interaction mechanisms

HMI - 2

29

Direct and indirect interaction

HMI - 2PROfessional network of Master’s degrees in Informatics as a

Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

system

interface real system

immediate

feedback

real feedback

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

30

Initiative

HMI - 2

Who has the initiative?old question–answer –> computerWIMP interface –> user

WIMP exceptions …pre-emptive parts of the interface

Modal dialog boxes come and won’t go away! good for errors, essential steps but use with care

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

31

Type of interaction mechanisms

HMI - 2

Instructions Talk, conversation (usually with avatars) Direct manipulation of objects (virtual or

physical, visual objects) Exploration (virtual or physical

environment)

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

32

Type of interfaces

Command line (Command Line Interface, CLI)

WIMP (Windows - Icons - Menus - Pointer) Natural User Interface (NUI) Voice Touch Gesture Multimodal Augmented reality

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

33

Interface: CLI (1)

Command line Character cursor - keyboard (including

special keys F…) Specific residues: enter the text fields or

CAPCHA

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

34

Interface: CLI (2)

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

35

Interface: WIMP

Windows – Icons – Menus – Pointer

HMI - 2

36

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

Interface WIMP – precursor

Xerox Alto 1973 One of the first PC

using the Desktop Metaphor

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

37

Xerox Alto – file manager

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

38

Interface WIMP – Windows

HMI - 2

W1.0 – 1985; W3.0 – 1990, W95, W2000 Graphical, 16-bit multi-tasking shell on

top of an existing MS-DOS installation

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

39

Windows 3.0

HMI - 2

1990

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

40

Evolution of interfaces

HMI - 2

• Codified• StrictCLI

• Metaphor• Explorato

ryGUI • Direct

• IntuitiveNUI

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

41

NUI – Natural User Interface

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

42

MUI – Modern User Interface

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

43

Voice interface

HMI - 2

Hands-free and Eyes-free Disadvantage:

Noise Errors in identifying the command Snooping (lack of privacy of communication)

User Requirements: Hands free I do not need to look Close to customers

Problems

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

44

Voice interface - Avatars

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

45

Touch interface

HMI - 2

It uses the sense of touch It can work two ways: input and output Input:

Touchpad Touchscreen

Output: Gloves and clothes making pressure

Special I/O devices using the Braille alphabet

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

46

Touchpad

HMI - 2

Mouse replacement device The advantage - built-in and requires no

additional work surface as a mouse The alternative - trackball (inverted

mouse) or the trackpoint (joystick)

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

47

Touchscreen

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

48

Touchscreen – gestures (1)

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

49

Touchscreen – gestures (2)

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

50

Braille devices

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

51

Gesture interface

HMI - 2

It uses the human body to transmit commands

Devices: Gloves with sensors 2D cameras 3D cameras Cameras with structured light Motion controllers

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

52

Cameras with structured light

HMI - 2

Use structural lighting (e.g. points or grid) to obtain information about the environment (3D)

Examples: Kinect (2010) EyeTracker

Kinect: Cloud of points (infrared LED + infrared camera) ->

image depth The image -> an ordinary 2D camera Microphones (4) Head rotating by motor (vertical) Accelerometer

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

53

Kinect – cloud of points

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

54

Gesture interface – applications

HMI - 2

Entertainment Medical (no touch) Processing of large number of visual

data / graphics / multimedia

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

55

Augmented reality

HMI - 2

Overlaping the real and virtual images

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

56

Augmented reality - welding technology training

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

57

Groupware Interfaces

HMI - 2

Provide the simultaneous interaction of a large number of participants

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

58

Brain-Computer interface

HMI - 2

BCI Recognition signals the brain working and

using them to transfer commands to a computer

Electroencephalograph Threats:

The accuracy of reading and interpreting The variety of people Variability of states of a single man Ethics: reading thoughts, control his thoughts and

so on

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

59

BCI – first experiments

HMI - 2

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

60

Physical object design

HMI - 2

Many constraints: ergonomic – minimum button size physical – high-voltage switches are big legal and safety – high cooker controls context and environment – easy to clean aesthetic – must look good economic – … and not cost too much!

Physical vs. logical state (button: on/off)

61

PROfessional network of Master’s degrees in Informatics as a Second Competence – PROMIS (544319-TEMPUS-1-2013-1-FR-TEMPUS-JPCR)

Thank you for attention

Questions?

HMI - 2