1

Computer Graphics

Assistant Professor

Dr. Sana’a Wafa Al-Sayegh

2nd Semester 2008-2009

ITGD3107

University of Palestine

2

Chapter 8Part-3

Graphical User Interfaces and

Interactive Input Methods

ITGD3107 Computer Graphics

3

– Distinguishing characteristics of devices– Input devices

• standard

• research

– Output devices• video

• other

– Virtual devices– WIMP vs. Post-WIMP interfaces– Where do we go from here?

Display and Interaction Devices

4

Display and Interaction DevicesUser Interface

5

Hardware characteristics

– Absolute vs. relative

– Polled vs. interrupt-driven

– Discrete vs. continuous input

– Degrees of freedom (DOF)

• number of simultaneous, independent data values that arrive in one record

Potential problem areas

– Spatial resolution

– Registration and calibration

• Accuracy and repeatability

– Sample frequency (temporal resolution)

– Data synchronization

Abstractions

– Hardware level

– Logical level

Input Device Hardware

6

Device interface level

– Wired vs. wireless

– Connection

• Universal Serial Bus (USB)

• Bluetooth

– Power source

• AC power supply

• batteries

• mechanical motion

• connection to computer

– Type of data transferred

• binary or text

• floating point, integers, text, etc.

Input Device Hardware

7

Commonly used today

– Mouse-like devices

• mouse

• wheel mouse (up to 2 wheels offer extra DOF)

• trackball

• trackpad

– Keyboards

• one handed vs. two handed

• DataHand keyboard

Traditional Input Devices

8

Traditional Input Devices

WACOM Intuos3 tablet

Pen/Stylus

• Data provided

– Absolute position

– Pressure, distance from surface

• Tablets for desktop computers

– Alternative to mouse

• Tablet PCs (can be used as laptops or slates)

– Toshiba Portege M200, M400

– HP Tablet PC tc1100, tc4200

– Acer TravelMate C200, C310

• and many more…

• Palm-top devices

– HP iPaq Pocket PC

– Handspring, PalmOS™

– Sony Clie, Treo

– BlackBerry

– Nintendo DSNintendo DSHP iPaq HP tc1100

WACOM Cintiq 21UX

9

– Dial boxes• number of dials (1 DOF per dial)

– Joysticks• game pads

• flightsticks (2 or 3 DOF plus a myriad of buttons and sliders)

• Nintendo Wii’s controller

Traditional Input Devices

10

– Touchscreens

– Microphones

• wireless vs. wired

• headset

– Digital still and video cameras, scanners

• Sony Eye

• Uses basic image recognition to

track body movements as an input

to console games

• TrackIR by NaturalPoint

– MIDI devices

• input from electronic musical instruments

• more convenient than entering scores with just a mouse/keyboard

Traditional Input Devices

11

Use may become more common in future

– Electromagnetic trackers

– Acoustic-inertial trackers

– Infrared trackers

– Optical trackers

3D Input Devices

12

– Gloves• attach electromagnetic tracker to the hand

– Pinch gloves

– Mouselike

– Hybrid

3D Input Devices

13

Commercial failures– Spaceball

– Flymouse

Unsuccessful 3D Input Devices

14

Non-standard Input Devices– Reconfigurable devices – Tool handles/props, with attached sensors

Passive input devices– Would like to separate user from devices– Voice recognition without a headset

• not successful yet

– Image-based analysis• video camera trained on user

Some Current Input Device Research

15

– Part of a windowing system, UI toolkit, or 3D environment

– a combination of behavior and geometry

Motivation– Advanced hardware devices are expensive, and not always available for all

platforms

– Most users already know how to use traditional input devices (mouse & keyboard)

– It is inefficient to have to continuously switch devices• try to keep hands on the mouse or the keyboard

– Would like to perform complicated inputs with simple gestures

– You will implement a virtual trackball and other virtual devices in the Modeler assignment

Virtual Input Devices

16

2D widgets

– Windowing systems (e.g. X, Mac, Windows)

• window

• scrollbar

– UI toolkits (e.g. Java Swing/AWT, Motif, Windows Forms)

• button

• dialog box

• drawing area

• object handles

– Simulating hardware devices

• sliders as virtual dials

• windows as virtual tablet

Virtual Input Devices

17

3D widgets

– Ambiguity of gestures

• 2D mouse gesture 3D movement

• interface must make decisions

• complex geometry involved to make these decisions

– Fundamental differences between 2D and 3D graphics

• multiple coordinate systems

• hidden surfaces

• more complicated primitives (3D objects, not 2D windows)

– Combine geometry & behavior

• make sure that target users can infer the widget’s functionality based on it’s geometry

• reduce the cognitive distance between the function you are actually performing and the interaction you are doing

• virtual devices should show the actions they are designed to do

Virtual Input Devices

18

Virtual sphere rotation (Chen ’88)– Project mouse motions onto the surface of a sphere surrounding the

object (an “object trackball”)

– Construct two vectors from center of sphere to the surface of the sphere– Cross product of two vectors gives the axis around which to rotate– Normalized dot product gives the cosine of angle to rotate object

through– Used for “camera trackball” as well– You will implement this in Modeler!

Virtual Input Devices

19

Inherent difficulties of 3D input– Different coordinate systems

• world

• object

• camera

• UV coordinates on object’s surface

• screen

– More complex math• 3D points, vectors, transformation matrices

• ray casting, hidden surface calculations

– 2D view of 3D scene• information is missing in a flat display

• objects obscured or off screen

• spatial relationships difficult to perceive

Virtual Input Devices

20

Classifications

Stereo

Desktop– CRT– LCD flatpanel

Semi-Immersive Desktop

Semi-Immersive Wall– Single projector, often DLP (Digital Light

Processing)

Video Output Devices

21

Fishtank VRDepthCubeHead-mounted displays (HMD)

Video Output Devices

22

Virtual Retinal Display (VRD)

Virtual Sphere

Video Output Devices

23

Augmented Reality– Augmented reality devices

• Optical see-through or video-based

• research still going on

Microphone

Sony LDI-D100B

P5 Glove

Hand Tracker (IS900)

Diamond Touch Table Connector

Head Tracker (IS900)

Columbia’s MARS

24

Audio

– Do not underestimate the importance of sound!

– Speakers

– 3D spatial sound

– Headphones

Printers

– Style impact printing

– Plotters

– Ink jet

– Laser

– Slides/film

– Holographs

– Rapid prototyping systems and 3D raster scan devices

Other Output Devices

25

Multiple, Interconnected Devices and UIs per User

Office/Home

• wall displays + personal notepad

• video-tracking for user id, location

• continuous speech recognition + natural language understanding + intelligent information processing

• Furniture: chair is instrumented to help detect posture, adjust to the user’s preferred position

Health-care

• Electro-chemical monitors