CSCI 6363Human Computer Interaction

Class Introduction

and

User Interface HistoryXerox Star, 1981

Familiar Systems …

• What is the below?– Right, a computer

• Many (essentially all) courses concerned with, e.g., algorithms, architecture, operating systems, …

– How many courses have you had concerned with these?

Familiar Systems …

• What is the below?– Right, a human

• Not so many courses in your curriculum talk about

• However, without humans there would be no computer science!

Familiar Systems …

• So, the subject(s) of this course – human computer interaction– Computer system, human, software, hardware, programming, models, …

• And – most importantly, their interaction– Human-Computer Interaction: HCI

Familiar Systems …

• So, the subject(s) of this course – human computer interaction– Computer system, human, software, hardware, programming, models, …

• And – most importantly, their interaction– Human-Computer Interaction: HCI

Familiar Systems …

• So, the subject(s) of this course – human computer interaction– Computer system, human, software, hardware, programming, models, …

• Also, will be concerned with how to program display and interaction mechanisms common to user interface implementation

Familiar Systems …

• So, the subject(s) of this course – human computer interaction– Computer system, human, software, hardware, programming, models, …

• Also, will be concerned with how to program display and interaction mechanisms common to user interface implementation

• Even learn some things about the human– Cognition, performance, …

Ever seen this?

Ever seen this?

• What’s wrong here?

• To start with, it doesn’t provide the user feedback about whether the last action was correct

– It does not say it is the course schedule

• Then, well, I guess it’s a matter of “selecting a term”

What’s a module?• Is it like a semester?

OK,scroll down• But, why do you (the

user have to scroll down?

• Why isn’t the usual, rather the exceptional, choice the one on top?

• What is the user’s impression of the system and the organization using it?

– It’s harder than it should be

OK,scroll down• Besides impression

and satisfaction, what’s the cost?

• About 3 seconds per user

– If practiced– If no error

• 3 secs * 3 times * 20,000 users =

50 hours

• How long to do right?– ~ 0 hours

Why HCI and Interactive Systems?• The interface is all the user (customer, employee) sees!

– Efficiency, success, satisfaction …

• Practically …– In sw systems > 50% of code is ui code– And somebody has to program it

• Discipline wise …– ACM SIGCHI is 2nd largest conference– SIGGRAPH in 10’s of k, SIGCHI k’s … many just hundreds

• Bigger picture …– “External aids to cognition”, as critical for humans– Ability of computers to store and retrieve information may have deep

implications– History of computing systems leads to emphasis on interactive systems

• Context first

Overview of Interactive Systems

• Humans interact with computers primarily through displays (and input devices)

• In fact, lots of things going on that are part of the whole context of computing, humans and hci

– Human– Interaction

• Style, characterization, design

– Programming systems

• Other UTPA CS courses:– CSCI 6360: Graphics– CSCI 6361: Visualization

Programming systems-windows-graphics

Hardware andgraphics engines

3-DViewing

Transformation

Representation ofcurves and surfaces

“Models”,(Foley et al. term)

DrawingAlgorithms

Lights Rendering

User, i.e., Human“Interaction” “Perception” “Information”

An illusion: analog world represented

digitally

Programming is a craft

OpenSystem

DisplayClosedSystem

External Aids to Cognition

External Aids to Cognition

• A tool is a tool is a tool …– (arguably)– Whether mechanical or electronic (information)

• Levers multiply physical forces, information tools multiply …– Computer as “mind tool” …

• Don Norman: – “The power of the unaided mind is highly overrated. Without

external aids, memory, thought, and reasoning are all constrained. But human intelligence is highly flexible and adaptive, superb at inventing procedures and objects that overcome its own limits. The real powers come from devising external aids that enhance cognitive abilities. How have we increased memory, thought, and reasoning? By the invention of external aids: It is things that make us smart.” (Norman, 1993, p. 43)

Amplifying Cognition

• Humans think by interleaving internal mental action with perceptual interaction with the world

• This interleaving is how human intelligence is expanded– Within a task (by external aids)– Across generations (by passing on techniques and knowledge)

• Computers in general, and interactive systems in particular, are important class of external aid

External Cognition – A Demo

• In your head: 34 x 72

• Without special purpose algs, it’s a choreNeed to remember intermediate results:

34 x 2 and 34 x 70 and add them

• External aids make it easy to remember int. results– paper and pencil, easy

• Or do away with altogether – calculator, fast

External Aids

• In fact a long history of external aids

• Slide rule exploits addition of logs for multiplication– Analog device

• Computers are general purpose external aids

Ratio of Hu man Cost to Hardware Cost for Computing

• Many years ago computers were large expensive unreliable machines

• Von Neuman thought a few machines of capacity of Eniac would be enough for US

• Plugging in cables, or setting switches, was reasonable use of operator time (both training and operation)

• Cost of computer time relative to operator time was very high,

• And tasks that were performed were relatively simple

• A few calculations vs. interactive scientific visualization, or the human task of managing one's own "information space"

ENIAC

Ratio of Human Cost to Hardware Cost

• Interface styles have changed over time

• As cost of hardware has decreased, has resulted in increasing need for efficiency of human time utilization

Ratio of Human Cost to Hardware Cost

• Change in the relative cost of computer to human time,

– coupled with tasks that can be performed due the processing power are responsible for the change in interface style

– (put another way:)

• Power of computer essentially ALLOWS more computing power to be directed toward the interface

– Today, even "entry-level" PC's have "way more" processing power than needed for most task

• e.g., word processing

• Today, cost of hardware is essentially trivial compared to the cost of human user!

Ratio of Human Cost to Hardware Cost

• Moore’s law is the 800 pound elephant in the room

• But, no real change in change in interaction style since mid ’80’s

– 25 year’s!– Approaching a complete human

generation

• Rather, more computing devices

– Shneiderman notes and his current edition reflects this

• Also, connectivity of devices and storage

Context of User Interfaces• Graphics is key enabling technology in evolution of

computing environments: – Graphical user interfaces– Visual computing,

• e.g., desktop publishing, scientific visualization, information visualization

• Hardware revolution drives everything – ~ every 18 months, computer power improves by factor of 2 in price/performance

– Moore's Law– … and exponential growth generally – things are different …

• Graphics memory and network speeds are on even faster exponentials  

– graphics chips in particular have major improvements every six to nine months

Moore’s Law - CPUs• Hardware revolution drives everything

– ~ every 18 months, transistor count improves by factor of 2 in price/count – Moore's Law

Moore’s Law - CPUs• Hardware revolution drives everything

– ~ every 18 months, transistor count improves by factor of 2 in price/count – Moore's Law

Another Example of Exp. Growth• The Internet … by whatever metric

Another Example of Exp. Growth• And digital camera resolution ….

Taking Advantage of Exp. Growth?

On Exponential Growth …• Some “technology” is exponential in advancement• Other things are not …

– E.g., software and …

Moore’s Law Dead?

• … and transistor count not equal to power

Moore’s Law Dead?

• Transistor count not equal to performance

Parallelism and Moore’s Law

• Moore’s law is alive and well for graphics and may be fine too for “gpu computing”

– Because lends itself to parallelization

– Pipeline architecture works well with current approach to cg

• Now, gpu processing power significant fraction of cpu processing power in pc’s

• How to use parallel, here, gpu, architectures is not a new question

• Cf. Nvidia gpu programming language (CL)

And Ratio of Human Cost to Hardware Cost for Computing … and “Neural Power”

Hans Moravec, When will computer hardware match the human brain? Dec. 1997 http://www.frc.ri.cmu.edu/~hpm/

But, like, not …1. Promise of future fallacy2. Computer processing not (at all) like neural processing

(but, cf. this month’s CACM, “Cognitive Computing”)

Ratio of Human Cost to Hardware Cost

• Moore’s law is the 800 pound elephant in the room

• But, no real change in change in interaction style since mid ’80’s

– 25 year’s!– Approaching a complete human

generation

• Rather, more computing devices

– Shneiderman notes and his current edition reflects this

• Also, connectivity of devices and storage

Chronology of System Development(another take on style)

1945-1955 Pioneer

1955-1965 Historical

1965-1980 Traditional

1980-1995 Modern

1995-? Future

UI Paradigm batch processing command languages

full screen (menus, forms)

WIMP noncommand based

interfaces

Terminal Technology TTY typewriter

line oriented terminals (“glass

TTY” CRT) full screen terminals

graphical displays

desktop workstations

heavy portables

“Dynabook” multimedia I/O

easily portable

cellular modem

User Types experts, pioneers

technocrats

professional computerists

specialized groups without computer

knowledge

business professionals

hobbyists

everybody

Operating Mode one user

batch

central processing

time sharing

remote access single user PC’s

networked single user

embedded systems

Programming Language

machine language assembler high level languages

problem oriented languages

spreadsheets

nonimperative visual programming

Wise, 1997

W W W = Hypertext(a quick example of why vision and history matter)

W W W = Hypertext(a quick example of why vision and history matter)

• Real “power” interest in Internet only arose well after its initial implementation

– ARPANET– Relatively few people cared about ftp, bbs, …– At least “few” compared to current use of WWW

• Interest and use arose only after:– Specification of WWW

• How to go from one file/document to another via its address … the link!– And then only with design of a graphic user interface for it

• Text-based browsers not too neat (at least to non-computer types)• How to “navigate” easily and with use of pictures/images/icons

• Essential idea of “navigation” among documents dates to Vannevar Bush’s Memex, 1945

History: Vannevar Bush, MEMEX

• MIT professor 25 years, etc.– Claude Shannon (information theory) was student

• Roosevelt’s science advisor in WWII 

• Invented continuous intergraph or Differential Analyses– Essentially, Charles Babbage’s Difference Engine with shafts

driven by motors– Analog computing solutions to differential equations (gears, etc.)– Big and handmade

History: Bush's MEMEX, 1945

• "As We May Think", 1945– MEMory EXtender system– Atlantic Monthly!– available at:

http://www.theatlantic.com/atlantic/atlweb/flashbks/computer/bushf.htm

– “new knowledge not reaching the people who would benefit from it”

– Concerned about the explosion of scientific literature which made it impossible even for specialists to follow developments in a field,

– “A Memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory.”

– desk, slanting screens, buttons, levers, and keyboard

• “A memex looked like a desk with two pen-ready touch screen monitors and a scanner surface. It had several gigabytes of storage space filled with textual and graphic, indexed, information”

History: Bush's MEMEX, 1945

• Microfilm projectors for viewing different information

– uses mircofilm for storage – new material can be added via

microfilm or by direct entry via ‘‘dry photography'‘

• Supports indexing, cross referencing, keywords

– supports associative indexing via links and creation of ``trails'' which can later be followed

– allows annotations comments, and marginal notes .

– envisions multimedia i/o: other senses, such as, speech and audio

• Associative indexing– "The process of tying things together

is the important thing.“– New profession of ``trail blazing"– Trail building and trail following by

user

Editor’s foreword to Bush’s 1945 Atlantic Monthly article

As Director of the Office of Scientific Research and Development, Dr. Vannevar Bush has coordinated the activities of some six thousand leading American scientists in the application of science to warfare. In this significant article he holds up an incentive for scientists when the fighting has ceased. He urges that men of science should then turn to the massive task of making more accessible our bewildering store of knowledge. For years inventions have extended man's physical powers rather than the powers of his mind. Trip hammers that multiply the fists, microscopes that sharpen the eye, and engines of destruction and detection are new results, but not the end results, of modern science. Now, says Dr. Bush, instruments are at hand which, if properly developed, will give man access to and command over the inherited knowledge of the ages. The perfection of these pacific instruments should be the first objective of our scientists as they emerge from their war work. Like Emerson's famous address of 1837 on "The American Scholar," this paper by Dr. Bush calls for a new relationship between thinking man and the sum of our knowledge. --THE EDITOR

Some Interactive Systems History

Some Interactive Systems History

• Doug Englebart– A Turing Award for vision in interaction

• Ivan Sutherland– A early 1960’s “paint program” that foreshadowed much

• Alan Kay– The “personal computer” – also, dynabook, Apple II

• Today’s ubiquitous interaction style– Xerox Star and Alto– Apple Lisa and Macintosh

Douglas Engelbart, 1963

• Turing Award, 1998 – this is a big deal

• Augmentation of human intellect (1963) – “... increasing the capability of man to approach a complex

problem situation, gain comprehension to suit his particular needs, and to derive solutions to problems.”

– Recognized his ideas built on Bush's idea of a machine that would aid human cognition

– hierarchical structures for ordinary documents– group creation and problem solving

• NLS System (1965 1968):– outline editors for idea development – hypertext linking – tele-conferencing, word processing, e-mail

• System required:– mouse pointing device for on-screen selection: Invented the mouse

(1965) as a replacement for light pens for use in his NLS system– a one-hand chording device for keyboard entry – full windowing software environment – on-line help systems– concept of consistency in user interfaces

Sutherland’s Sketchpad• Ivan Sutherland

– “Pioneer” of … lots of things– Visualization– Graphics– Interaction– Still around

• Evans and Sutherland graphics

• First truly interactive graphics system, Sketchpad

– A fairly sophisticated “paint” (or drawing) program

• MIT, Ivan Sutherland’s 1963 Ph.D. thesis

– “Sketchpad, A Man-Machine Graphical Communication System”

• Available: www.cl.cam.ac.uk/techreports/UCAM-CL-TR-574.pdf

• Video: www.youtube.com/watch?v=mOZqRJzE8xg

• “Among most important works in computer science”

Ivan Sutherland using Sketchpad in 1963 CRT monitor, light pen and function-key panel

Ivan Sutherland’s Sketchpad, 1963

• Regarded as the first to implement much of what called “visualization”, “immersion”, and “virtual reality” (not to mention cg)

• Some quotes:

– ….. If the task of the display is to serve as a looking-glass into the mathematical wonderland constructed in computer memory, it should serve as many senses as possible.

– ….. By working with such displays of mathematical phenomena we can learn to know them as well as we know our own natural world. Such knowledge is the major promiseof computer displays.

– ….. The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such a room would be fatal. With appropriate programming such a display could literally be the Wonderland into which Alice walked.

Sutherland’s 1960’s equipment

• “Ultimate display”, 1965

• Sword of Damocles – 1st HMD– Actual camera-like metal shutters– Not OSHA approved

Advances in Computer Graphics for Interaction

• Input devices such as data tablets (1964)

• Display processors, capable of real time image manipulation (1968)

• Introduction of low cost graphics terminals (1970s)

• OS support for interactive graphics under timesharing systems

• New programming languages for embedded graphics support

• New applications flourished - computer aided math libraries (1968) . - 3D computer graphics / molecular modeling (1968, 1977). - animation, movies, sequencing of images (1966, 1969)

 

The “Personal Computer”, 1

Dynabook (Alan Kay, 1977): Conceptualized notebook computer:

cardboard prototype

“...a self contained knowledge manipulator in a portable package the size of an

ordinary notebook...”

powerful processor, lots of memory, high resolution graphics, high fidelity audio

 

Altair 8800 (1975):Considered to be first microcomputer

2 MHz Intel 8080 with 256 bytes standard RAM and interfaced

with the user through octal front panel switches appeared in Popular Electronics home computer

people, could build for around $400

Altair 8800

The “Personal Computer”, 2

Apple II (1977):Was “easy to use”

Color graphics

High-level language embedded in ROM

4K of memory, more could be added to a maximum of 12K (if using the 4K chips) or 48K (if using the 16K chips)

Innovations at Xerox PARC in Late 1970s Xerox Alto

Comprehensive combination of many ideas developed at Xerox PARC and other pioneering systems:

• user conceptual model via the desktop metaphor .

• direct manipulation, property/option sheets to specify appearance of objects .

• what you see is what you get (WYSIWYG) . • universal generic commands that are used

throughout the system (e.g., move, copy, edit, delete).       

• high degree of consistency, look and feel . • modeless interaction and limited amount of

user tailorability  

(FYI) Xerox Alto

The Xerox Alto (1979):• it was always referred to as the "personal computer"• it used high resolution bit-mapped graphics, a mouse, menus, icons and other

features• a single user machine supplied with software for word processing, electronic

mail and other office tasks• the 15 inch display was bit-mapped and had 72-dpi resolution (about 1000 x

800)

1/2 meg of memory, 29 mb of disk storage, and a mouse about the minimum specifications to support a graphic interface

• pioneered developments of GUIs and applications text editing, drawing, document processing, e mail, windows, menus, scrollbars, mouse control and selection

       simultaneous file storage• featured the world's first What-You-See-Is-What-You-Get (WYSIWYG)   • lead to the creation of Ethernet, a local area networking protocol, to link Altos

within PARC (1975)

Screen shots of an Alto

Xerox Alto

• Designed for business professionals in an office environment.

• Contained all of the Alto's capabilities plus multilingual software, the Mesa programming language, and interim file servers

• The most distinguishing features of the Star were technologies like bitmapped screens, windows, a mouse-driven interface, and icons.

• The system allowed users to create complex documents by combining computing, text editing and graphics, and to access file servers and printers around the world through simple point-and-click actions

 

The Xerox Star (1981), 1"The best way to predict the future is to invent it."

--Alan Kay

The Xerox Star (1981), 2

Star did not turn out to be a commercial success.

The reasons for failure:• it was ahead of its time and the full potential of the

software was only visible to the PARC developers

• the software was designed for users with no computer knowledge, it became very demanding on the hardware

• perceived as too slow unreasonable speed because of memory consumption and the number of hardware cycles needed

• costly, $16,595 in 1981; IBM introduced a cheaper machine .

• limited functionality: no spreadsheet • lacked an open architecture

• no 3rd party application development: because the programming language was never publicly released.

• direct manipulation was overly used and not the best in some cases

 

(fyi) Xerox Star Design Goals (1)

FAMILIAR USER'S CONCEPTUAL MODEL        as much as possible, the functionality of the system would be conveyed to the user by representing

system objects by familiar concrete objects        that is, if possible, metaphor would be used to convey the capabilities of the system       after all, this was a personal computer for office use, so the designers settled on the metaphor of

computer system as desktopknowledge about how to use office equipment might be used in using and learning the system

 

USE ‘SEEING’ AND ‘POINTING’ VS. ‘REMEMBERING’ AND ‘TYPING’        everything that is relevant to a task is visible on the screen        their rationale was couched in terms of the human visual and memory systems        basically, if possible, it's more efficient not to have to remember the set of all commands and objects        and then they made a very interesting statement, that I'll elaborate on later:

"A subtle thing happens when everything is visible: the display becomes reality. The user model becomes identical with what is on the screen. Objects can be understood purely in terms

of their visible characteristics. Actions can be understood purely in terms of their effects on

the screen."

- this has come to be called "direct manipulation"

(fyi) Xerox Star Design Goals (2)

WHAT YOU SEE IS WHAT YOU GET (WYSIWYG)UNIVERSAL COMMANDS

– as many commands as possible should be able to be used in the same– way throughout the system– help, undo

CONSISTENCY– mechanisms should be used in the same way wherever they occur– for example, the left mouse button always selects

SIMPLICITY– when possible– but novices and experts use the same system– in Alan Kay's words: "simple things should simple, complex things

should be possible"

MODELESS INTERACTION– in practice, select object, then action– for example, no delete mode

USER TAILORABILITY– well, if these design goals (except for the last one) CONJURED UP for you

the Apple Lisa and Macintosh, that's good because it was after a trip to Xerox PARC, that Steve Jobs conjured up the Apple Lisa and Macintosh (and, I understand, entered into licensing agreements with Xerox)

 

indeed, these goals have become for most designers the BASIC TENETS OF GRAPHIC INTERFACE DESIGN

Apple Computer - Lisa

Apple Lisa (1983)

      “after a visit to Xerox …”

based on ideas from Xerox Star

      could be used as an office tool but also individuals

at $10,000 was cheaper than the Star,

but... also failed...

Apple Computer - Macintosh

Apple Macintosh (1984):

same ideas from the Lisa, but much improved

Reasons for success: –         cheaper: only $2500 . –        previous systems prepared the market . –        ‘‘second generation,'' many problems

with Lisa were fixed . –        open architecture and powerful

developers toolkit . –        published interface guidelines: consistent

look and feel for apps . –        excellent graphics and emergence of

desktop publishing . –        Apple had marketing expertise,

distribution channels, and experienced computer sales and support staff

 

Ware’s “Ultimate Display”

A question to ponder …• Recall, ui history

– Nothing much new lately (20 years since wimp)

• Is this because ui (or is) design awaits a fundamentally new way of conceptualizing the use of computers?

• A fact:– Second language use acquired after some young, e.g., 5-8 years

old, is fundamentally different than language skills acquired earlier• E.g., translation vs. thinking, forming phonemes

• A way that can only be conceptualized by a person (or really some critical mass of people) who have fundamentally (neurologically) different ways of perceiving computers?– Has it happened already?– When might that happen?– How might it happen?

“Human Computer Interaction”Course overview

• Shneiderman text …

Usability of Interactive Systems

• Introduction– What is an interface?– Examples of good and bad design– Principles of design

• About the field– Evolution of hci and interface design

• Interaction Design: – Some principles through examples

• Usability – What, why, examples, motivation

• Universal Usability

• Goals for the Profession

Theories, Principles, Guidelines

• Guidelines:– Navigating interface, organizing display– Getting user’s attention, data entry

• Principles:– “Rules that distill out the principles of

effective user interfaces”– Determine users’ skill level– Identify tasks– Choose an interaction style– “Golden rules of interface design”– Integrating automation and human

control

• Theories and models:– Levels of analysis theories– Stages-of-action models– GOMS and keystroke-level model– Consistency through grammars– Widget level – Context of use

Theories

Guidelines

Principles

Managing Design Processes• Organizational design to support usability

– Shneiderman talks about both “design” and organizational context in which it occurs

• Carroll and Rosson’s characterization of design– “radically transformational”

• Shneiderman’s “three pillars of design”– Guidelines documents and processes– User interface software tools– Expert reviews and usability

• Development methodologies– IBM: Ease of Use, Lucid

• “Ethnographic” user observation• Participatory design• Scenario development• Social impact statement for early design review• Legal Issues

Expert Reviews, Usability Testing, Surveys, and Continuing Assessment

• Introduction– Evaluation Plans, Acceptance Testing, and Life Cycle

• Expert reviews

• Usability testing and techniques– Goal is to “engineer” a good interface, constrained by time and cost

• Survey instruments

• Acceptance tests

• Evaluation during active use

• “Controlled psychologically oriented experiments”– Elements of science, as applied to interface evaluation

Command Line, Menu Selection, Form Fillin, Dialog Boxes

• What and why “frameworks (or theories) of interaction”– The big picture (again)

• Overview of interaction styles– Command line, menus, …, WIMP, … – Direct manipulation

• “Look and feel”

• Details:– “Theory, principles, and (especially) guidelines”– Menus, form fillin

Interaction Frameworks (Dix)

• Interaction:– Communication between

user and system

• Why have a framework?– Allows “precision” in

accounting for differences• E.g., gulfs of execution and

evaluation

– Presents global view• All elements receive

attention

Task

“wor

k on t

ask”

“comm

ands”

System

User

“gives”“per

form

s”

“feedback”

Physical System

Goals

Gulf of Execution

Gulf of Evaluation

Direct Manipulation (Hutchins et al.)

• Introduction– Big picture first, details and a theoretical account later

• Examples of Direct-Manipulation systems • Explanations of Direct Manipulation

– Hutchins et al. paper

 • Virtual Environments, or Immersive Interfaces

Balancing Function and Fashion User Manuals, Online Help, and Tutorials

• “Balancing Function and Fashion”– Error messages– Nonanthropomorphic design– Display design– Window design– Color

• “User Manuals, Online Help, and Tutorials”

Balancing Function & Fashion

• Interface design, both graphics and interaction, not yet “high art” (Shneiderman)

– Architecture and fashion are old, interfaces are not

• But, not too sure how far analogy goes …

– Usually, consider that interface design is about software engineering, usability, etc.

• Recall, early discussions– Perhaps better, computer interface design is

young

• Yet, there is style– And it elements can be examined

• Where “style” results from the set of decisions made about graphics design, type of interaction form, wording selected, etc., that are made

– Recall, guidelines constrain, etc.

“Look and Feel”

• “Look and feel”

– Lots of things you can interact with:• main WIMP components

(windows, menus, icons)• Buttons• Dialogue boxes• Palettes• … Collectively known as

widgets

• appearance + behavior = look and feel

And programming, too

And programming, too

• Will take two approaches to programming:

• 1. “Assembly language programming for user interfaces”– MS Windows API using c

• Yes, that’s c, not c++, not c#, … for the pedagogy• Focus on event driven programming – with no software layer

between programmer and windowing system event queue

• 2. A High level tool– QT

• Cross platform user interface programming tool• E.g., used in first Google Earth implementation

• Java, c++ (mfc) straightforward object oriented approach

Windows API

• Control flow is “event driven”– Vs. simply top to bottom

• All about function calls and interface elements– E.g., “get and set thumb of scroll bar”– ~700

• Will use C (vs. C++ or Java or …)– Want to get to layer of closest control

About Interaction Handling, Polling vs. Event - Driven

Sampling (polling) vs. event-driven software architectures 

Basic issue of how to get and act upon user actions 

Entails “looking at” (getting information from) input devicesReadln (...) -- get character information, “sits and waits”

With several asynchronous devices (mouse loc & button, keyboard, ...) need new method(s)

  

About Interaction Handling, Polling

 Polling to detect mouse:// general initial display, etc// use past and current values of some event indicator to see if (new) input

 old_val = get_val_of (mouse_button_1)

 WHILE (!quit)

{:

new_val = get_val_of (mouse_button_1)if (new_val != old_val)

{// do whatever to do with mouse clickold_val = new_val}

  // check other devices too, and act if need to }

 This is process intensive as always checkingMay be better to have scheme in which not just keep looking  

About Interaction Handling, Event Driven

Event Driven  // general initial display, etc 

WHILE (!quit){// program waits for user actionwait_on (user_action)  switch (user_action)

{case mouse_button_1_clicked:// do whatever to do with mouse click

 // other cases to handle user inputcase mouse_button_2_clicked:

:}

}  

wait_on()?

Event QueueWhen > 1 user action occurs before complete processing of

another eventput events in queue, the event queue

 This is basic manner in which MS windows handles user input:

there is an event for everything (click, resize window, move mouse into window,move at all, ...)

 and a message is generated for each, so in fact the messages are held in a queue:

 the window’s message queue

Overview

• Windows distribute messages to programs– A function (named in main program, e.g., wndproc) is called

(instantiated) to process events– Event-driven architecture

• Program driven control flow– Typically, programs are structured so that updating occurs when

window is redrawn upon receiving a WM_PAINT message

• E.g., scroll bars are an windows interface object that when user interacts with cause messages to be sent to programs event cue– Will examine those messages in detail as a first example

• Petzold’s text uses c programs to illustrate– Which is good because it’s real– However, requires fairly extensive experience with c to appreciate all

nuances

Windows Distributes Events to Appropriate Event Queues

 

Windows perspective:

 

program 1

queue 1

wndproc(){}

program n

queue n

wndproc(){}

Windows - distributes eventsProgram 1 Program 2 Program 3

Event queue 1 Event queue 2 Event queue 3

Windproc 1

Windproc 2

Windproc 3

Interaction Handling: Event Driven

Program perspective: Program handles events, thus event-driven programming // general initialization, first display While (!quit)

{// program waits for user actionwait_on (user_action) // just sit there until event occurs

 switch (user_action)

{case mouse_button_1_clicked

// do whatever upon mouse click 

// other cases to handle input (events)case mouse_button_2_clicked

:}

} 

When > 1 user action occurs before completing processing of another event,put event in queue, the event queue

 In fact there is a message for just about everything, mouse move, resize, etc.,

and events are passed to the program as messages and placed in the message queue

Function Called to Process Events(first look)

• Entry point is WinMain( )– Name of function, e.g.,

WindowProc, to handle events, etc., specified in main program

– Loop to “translate and dispatch” events

• Function, e.g.,WindowProc( ), is called repeatedly to handle events/messages– On and off stack, so must use static

variables to save values

Microsoft Windows

program

start

me

ssages

WinMain()- initialization- define windows- create windows- message loop

WindowProc()- process messages

Windows API

Windows Application

API Calls

Interaction Handling: MS Windows Code

Note similarity of Windows program to general structure: WndProc (HWND hwnd, UINT iMsg, WPARAM wParam, LPARAM lParam) { HDC hdc ; PAINTSTRUCT ps ; RECT rect ;  switch (iMsg) { case WM_CREATE : PlaySound ("hellowin.wav", NULL, SND_FILENAME | SND_ASYNC) ; return 0 ;  case WM_PAINT :

hdc = BeginPaint (hwnd, &ps) ; GetClientRect (hwnd, &rect) ; DrawText (hdc, "Hello, Windows 95!", -1, &rect, DT_SINGLELINE | DT_CENTER | DT_VCENTER) ;

EndPaint (hwnd, &ps) ; return 0 ;  case WM_DESTROY : PostQuitMessage (0) ; return 0 ; }  return DefWindowProc (hwnd, iMsg, wParam, lParam) ; } 

Each “case” represents an event, and what to do when it occurs

Should (soon) feel (more or less) comfortable with all aspects

End