New Computing In 2019 and Beyond - Opportunities,

Challenges, and ThreatsFromm Institute

Fall 2019 - Lecture 4 Bebo White - bebo.white@gmail.com

1

calendar

2

questions (1/3)

• “How does one protect a patent in blockchain?”

• “There is not ‘a’ blockchain, but innumerable different ones - correct?”

• “Who decides who gets one of the distributed ledgers?”

• “How is it ensured that all ledgers get all blocks?”

• “Does Hans Gosling use AI in his Gapminder program to analyze population and economic data trends?” (Trendalyzer)

3

questions (2/3)

• “How do I set up a blockchain? How do I get my blockchain on the Internet? Are there tools needed to create a blockchain?”

• “Can everyone see a blockchain?” (paraphrased)

4

questions (3/3)

• “Could you explain VPNs and if we should join one?”

• “Should all children be taught to code? How many years of Computer Science should a child have?”

5

6

7

8

what is one thing that these technologies have in common?

9

10

human-computer interaction (hci)

• Human

• the end user(s) of a computer-based operation

• one member of a dialog with a computer-based system

• distinguished from interactions between computer systems

• Computer

• a system containing some computer processing capability

• typically represented by a combination of hardware and software and maybe networked-based systems

• Interaction

• the process by which different parties communicate, exchange information, affect each other, and change each other

11

caveat

• HCI is one of the topics that I teach a full semester class on, so

• make sure that I don’t go too fast and make too many presumptions

• make sure that I give opportunities for questions and comments

12

13

re-thinking hci (1/2)

• in the past

• it was humans telling computers what to do and computers responding

• humans were the dominant actor

14

re-thinking hci (2/2)

• humans and computers can be equal partners

• in some instances, the computer might be dominant

• HCI - CCI (Computer-Computer Interaction) - UIs designed for computers

15

16

Computer Science

Psychology

HCI

Information Systems

Software EngineeringLanguages

Interaction Hardw are

Cognitive Social

Organizational

Cognitive Science Sociology

Artif icial Intelligence Ergonomics

Computer Science

Psychology

HCI

Information Systems

Software EngineeringLanguages

Interaction Hardw are

Cognitive Social

Organizational

Cognitive Science Sociology

Artif icial Intelligence Ergonomics

cognitive - model human computer (mhp)

• map human systems to computer systems

• driven by human capabilities and limitations

• humans have a perceptual processor, a cognitive processor, and a motor processor

• examples?

17

18

interaction models

• Implementation Model

• the actual way an interface works

• Manifest Model

• the way the interface presents its functioning to the user

• Mental Model

• a user’s perception of the interaction model

19

what do you think?

• Implementation model?

• Manifest model?

• Mental model?

What is this?

20

affordances

• strong clues to the operations of things

• when affordances are taken advantage of, the user knows what to do by just looking

• based upon experience

• often presented by images or icons

• _____ affords _____

21

22

direct manipulation

• Metaphor

• computer objects as visible, moveable entities

• Consequences

• items represented as icons

• items can be “picked up” and “moved” on a surface

• items can be “thrown out”

• items can be “copied”

23

bringing concepts together - wimp and/or gui• Windows

• Icons

• Menus

• Pointers

or

• Windows

• Icons

• Mice

• Pull-down menus

similar to

• GUI - Graphical User Interface

24

25

all well and good, but…

• WIMP, GUI, etc. are good for desktops, laptops

• maybe OK for ATMs, kiosks, etc.

• not so good for mobile phones, smart watches, etc.

• this course is about new technologies, so we need new interfaces

• what about interfaces for IOT?

• what about interfaces for large dataset manipulation?

• are you going to use a GUI for AI applications? (you already don’t)

• does MHP still work?

26

27

are such computer interfaces a part of our future?

• where are

• interaction models?

• the affordances?

• direct manipulation?

• does the MHP still apply? has the MHP been usurped/disrupted?

28

why do we see this so often?

• why is it successful?

• why can’t we be more like children in our computer interactions?

• is it natural?

29

natural user interfaces (nui)

• seeks to define a more natural way for humans to interact with technology

• refers to both sensory inputs such as in MHP but goes further to describe computing that is intelligent and contextually aware, with the ability to recognize a person’s face, environment, and intent, even emotions and relationships

• reflects the roles that computing plays in all aspects of life

• let’s assume that natural language interfaces are a given

30

31

mhp < nui

• 3 components of NUIs

• Invisible Computing - hardware virtually disappears

• Supporting Computing - supports natural human function rather than requiring humans to adapt to computing functions

• Adaptive Computing - along with ML intelligently recognizes and interprets human patterns to produce output based on relative context

32

what would be the best (most natural) interface?

33

did anyone say your phone?

• perhaps the most interface-intensive computer system that you own

• strong requirements for

• quick interactivity

• multiple “interaction modes”

• touch

• voice

• orientation

• etc.

34

but, how do you hold your phone? how this affect interaction?

35

how does your phone see you?

36

Are there other interaction methods to explore?

37

gestures as nui

• we cannot be limited to

• pointing

• touching

• sweeping

• zooming, pinching

• other traditional forms of traditional direct manipulation

• implications of gestures

• subtle communication

• cultural applications

• can add to flexibility and “richness”

38

the index finger - “the king of the fingers” (1/3)

39

the index finger - “the king of the fingers” (2/3)

40

the index finger - “the king of the fingers” (3/3)

41

gestural interfaces (1/2)

• extreme case of direct manipulation

• driven by sensors capable of detecting

• pressure

• light

• proximity

• acoustic

• motion

• orientation

• location

42

gestural interfaces (2/2)

• Bad

• heavy data input

• reliance on the visual

• reliance on the physical

• inappropriate for context

• Good

• more natural

• less visible hardware

• more flexibility

• more fun

43

44

45

haptic interaction

• relating to sense of touch and grasp

• the haptic senses work together with motor control systems to coordinate movement and enable perception

• cutaneous - temperature, texture, slip, vibration, force

• kinesthesia - location/configuration, motion, force, compliance

46

tactual stereognosis

• tactual = tactile = via sense of touch

• stereognosis - the mental perception of 3D by the senses; perceiving the form of solid objects by touch

• people are very good at tactual stereognosis

47

48

we have to talk about ar/vr/mr - not just for games

49

•augmented reality •virtual reality •mixed reality

50

51

what has happened to the desktop metaphor?

52

how many interaction styles are you seeing here?

53

we have to mention brain interfaces

54

55

56