new interaction techniques

New Interaction Techniques

Department of Computer and Information SciencesUniversity of Tampere, Finland

Department of Computer and Information SciencesUniversity of Tampere, Finland

January – June, 2003

Grigori Evreinov

www.cs.uta.fi/~grse

Engineering basics for Computer Interaction

Engineering basics for CI

TAUCHI MMIG G. Evreinov p 01_97 17.01.2003

http://www.ccs.neu.edu/home/fell/images/BBB/BBBphoto.jpeg

Device Capabilities and their Future

http://www.casio.com/labelprinters/product.cfm?product=3766&display=14&cid=5227

KP-C50 PC Tag Writer

Fossil Wrist PDA with Palm OShttp://www.fossil.com/


within just the last 24 months, myriad audio, video, PDA and cellular products

have equipped people not only to carry around data, images and audio but also

to swap devices between various types of hardware

new technologies include non-volatile flash memory cards and small disk

drives

flash memory cards have no moving parts and retain data in the absence of

power

memory is key to retaining complex data on a device

it enables storage of programs, audio and video files and provides users with

more efficient data compression methods

sufficient memory also allows devices to run applications that require large

amounts of memory to implement, like as Java etc



two major subjects are to develop very high density magnetic medium and very sensitive reading head technology using Giant Magnetic Resistance Effects [3]

Association of Super-Advanced Electronics Technology (ASET)



IBM has demonstrated a GMR (Giant Magneto Resistive) head with an areal density capability greater than 35.3 billion bits per square inch and laboratory demonstrations up to 130 Gbits/in2 have been reported in the industry, indicating that future disk drives could exhibit capacities at least two times higher than today

IBM Magnetic Hard Disc Drive Technology [5]



added functionality implemented in recent 2G and 2.5G terminals

source: UMTS (Universal Mobile Telecommunications System) Forum [1]



the above-mentioned multifunctional devices are based on a mobile phone

centric approach

new multimedia and mixed data services would create further opportunities for

UMTS devices to be complementary to other electronic devices

considering the high level of complexity entailed in integrated multifunctional

devices, a feasible approach is to enable traditional portable (consumer or

business user) devices to interwork with UMTS terminals implementing core

access functionality

examples would include a digital camera interworking with a UMTS terminal,

which would enable a user to transfer a digital image to the terminal for

incorporation in a multimedia message

the possible combinations are very wide ranging



many different applications are expected to be implemented in UMTS devices

for each application, corresponding additional components or elements need to

be employed

this will have some impact on the terminal design from a form factor

perspective

on the other hand, most users want to carry as small and as light a device as

possible even though new functionality or features are added

further miniaturization is one of the key issues and this requires further

miniaturization or integration of all related components on UMTS devices



several approaches for further integration and miniaturization

source: Nikkei Electronics No. 782, cit. in [1]

LSI - Large Scale Integration

LTCC - Low Temperature Co-fired Ceramics; integrate high frequency passive components into one ceramic substrate

MEMs - MicroElectroMechanical Systems an advanced technology that makes possible to integrate passive elements into semiconductorMEMS is also known as micromachine technology


Trends in component technology




Display technologysometimes the user may be concerned more with viewing a screen than with listening to an ear piece

the display represents the most important component in the future of communication

perhaps users will interact through the display in many different environmental conditions for almost all device applications

they will need to view high information content multimedia as well as the high bandwidth video

the display is also likely to function as an input device through the use of “soft keys”

for effective interaction between users and displays, a direct-view display must be as large as possible within the constraints of a portable device



no single display technology can currently satisfy all of requirements, like

resolution, contrast / brightness, illumination, colour, frame rate, interface,

bezel (non-display area), thickness, weight

the simplest displays, including for mobile applications, are passive matrix

displays

a passive matrix display is an array of pixels, each of which contains an

optical element that is sandwiched between column and row electrodes

passive addressing via the column and row electrodes puts limitations on

the achievable display resolution and levels of grey-scale that can be

programmed at each pixel



in STN Liquid Crystal Displays, the optical element is a Super Twisted

Nematic (STN) liquid crystal that modulates (between 180 to 260 degrees

resulting in better contrast) the transmission of light through polarisers

positioned at each side of the liquid crystal cell

STN materials have a sharp transmission-voltage response and a slow

switching speed (e.g. >100ms), and as such are well suited to binary

(black or white state) passive addressing, although 3-4 bit grey-scale can

be achieved

displays of this type are particularly suitable for text and simple graphics

display, and this is sufficient for many of today’s low-bandwidth

applications, while they have a limited viewing angle

these reflective displays are very low power and are commonly illuminated

by a (near white) LED, and are very cheap to manufacture



higher performance color STN LCDs offer desirable benefits for multimedia

applications, though the introduction of colour filters can reduce total

display brightness and increases the unit cost

transflective technology helps ensure that pixels make the most of both

ambient light and back-light sources

although not capable of matching the performance of TFT (Thin Film

Transistor) LCDs, the best color STNs of today can achieve 65,000 colors

for still images and 15 frames per second video at intermediate resolutions

one of the more interesting technological developments is the move to

plastic substrates; plastic STN LCDs offer lighter weight, greater impact

resistance and the option to have custom (e.g. non-rectangular) display

shapes



XtraViewTM Wide Viewing Angle Technology [9]

these transistors do not generate light

or color, an often-made mistake

this is where the liquid crystals (LC)

and their alignment come into play

the transistors control the orientation

of the LC, thus allowing them (LC) to

pass (or not pass) light from the

backlight



XtraViewTM Wide Viewing Angle Technology [9]

by having the electrodes on the same glass substrate, they allow the crystals to remain horizontal to the glass substrates in both the on and off statessuch design improves the viewing angle by passing the light through the crystals at their most efficient orientation – a horizontal orientation – thus dispersing the light more efficiently



organic electroluminescent (OEL) materials emit light in proportion to the current flowing through them, and have the advantages of high brightness and of being very thin

higher performance displays are composed of active matrix pixels

each pixel typically includes an optical element and switch

the switch is an active component such as a TFT (thin-film transistor) or a TFD (thin-film diode), and is addressed by column (data) and row (scan) lines

TFTs are normally fabricated from a thin-film of amorphous Silicon (a:Si); though complete construction of the TFT requires the deposition of several additional layers, including the addressing lines, today, this can be achieved with a minimum of five photolithographic masks, which keeps the cost of active matrix displays competitive



source: UMTS Forum [1]

four of the most common active matrix display cross-sectionsto maximize the use of ambient light, a single polariser can be used; micro-reflective structures and careful choice of colour filters can increase brightness at the expense of contrast ratio and of viewing angle



although more expensive to manufacture than passive displays, the active matrix pixel switch permits a larger total number of pixels in the display, higher resolution, higher contrast and accurate grey-scale pixel programming

in transmissive TFT LCDs, the optical element is usually a Twisted Nematic liquid crystal that modulates the transmission of light supplied by a back-light through orthogonal polarisers positioned at each side of the liquid crystal cell Twisted Nematic materials have a shallow transmission-voltage response and a fast switching speed (e.g. 25ms), and can therefore achieve 8-bit or higher RGB grey-levels (16 million colours) at 60 Hz updates (i.e. “true colour“ video)

high performance active OEL displays based on poly-silicon TFTs are being considered since more than one of them can be implemented at each pixel to implement a small current-mode driver circuit

this “pixel circuit“ is very power efficient and can minimize luminance non-uniformity across the display



source: Advanced Data Research, Japan (11/09/00) [1]

Low Temperature Poly-Silicon

organic electroluminescent

Super Twisted Nematic



Input devicesusability is a key issue affecting both the implementation of applications

and device design

unification of input methods is an important factor in realizing “easy to use”

user interfaces, but unification of new features could bring complexity to

users to understand which input device is doing which function and/or

feature

some UMTS devices will have similar input methods and components like

current mobile units (keypad and pointing device), others may employ

touch screens and voice recognition

devices should not only support the display of character encodings and

character sets in supporting internationalized content in local languages,

they must also allow for the input of text in those local languages



Keyboards and keypads a basic requirement for a mobile unit input device is to

employ at least ten keys for activating the phone and the line and for inputting telephone numbers

most current mobile phones employ between 14 and 17 keys, normally realized using carbon printed or gold flashed substrate combined with a carbon printed rubber sheet, poly-dome sheet or metal contact sheet

reliability is becoming an increasingly important factor as mobile phones change from voice-oriented to games-oriented usage

the minimum life cycle for the key panel has to guarantee at least one million contact cycles

sensitivity to moisture from the human body becomes an issue



switch technologies [10]

direct membrane/polydome switches

indirect full-travel membrane switch

printed circuit board contact patternsthe most important single design objective is to provide as many shorting paths as possible so best switch operation can be realized when the button is actuated



wire-free soft technology

the three modes of ElekTex™ sensor operation - position sensing (X-Y positioning), pressure measurement (Z sensing) and switch arrays – are normally achieved through four connections to each fabric interface

demo 1

http://www.electrotextiles.com/flash/tech_spec.shtml



Logistic Design (UK) Ltd. [12, 13]

1. rubber keypad (non-conductive) 2. spacer/Adhesive 3. membrane/polydome layer with

conductive ink 4. spacer/Adhesive 5. PCB 6. conductive ink

sample designs illustrating indirect polydome construction

this width should be same as length of key-travel + 0.2mm

Printed Circuit Board (PCB) design for use with membrane/polydome switches


TAUCHI MMIG G. Evreinov p 25_97 17.01.2003ICHIA Technologies Inc. [11]

the snap ratio (or click ratio) of any conductive rubber keypad directly affects the tactile feel realized by the operator

keypads with snap ratios of 40-60% have excellent tactile feel and relatively long life, while keypads with snap ratios below 30% have relatively weak tactile feel, but longer life

dual-durometer keypads also improve tactile feelthe snap ratio of any keypad can be calculated by working with the formula F1-F2

divided by F1, where F1 is the actuation force and F2 is the contact force



ICHIA Technologies Inc. [11]

a very general guideline that can be followed for developing good tactile

feel is to specify higher actuation forces for keypads with large keys than

those with small keys

this rule also applies to key heights: tall keys require higher actuation

forces than short keys



another typical guideline for actuation force is to specify a minimum actuation force of 80 - 100 grams for keys with heights of 10 - 15mm and a minimum actuation force of 150 - 175 grams for keys with heights of 15 - 25mm

care should be taken when designing tactile feel so a minimum return force of 30 grams is realized

this minimum return force will help greatly to eliminate the potential problem of sticking keys [11]

even though it is possible to use ten keys for writing emails or inputting characters, this would not be acceptable to users

other solutions have to be considered

today, several sub-systems and technologies are already available to support these requirements; some have already been used in market products



pen-input technology (touch screen, track pad and click) or voice

recognition technology could also improve usability as alternatives to

keypad-based input methods

with the trend towards even more innovative device features and designs

that go beyond conventional keypads - often incorporating icons,

pictograms, and symbols for interaction and inputting instead of keys



Pointing devicesbeside keypads, many different kinds of pointing devices have been

implemented onto mobile applications that allow the user to scroll the

menu or to select a subject on the display

source : ALPS Electric Co., Ltd [2]

currently available surface mounted devices (SMD input devices) for mobile phones: (a) top-faced slide switch with centre push; (b) side-faced slide switch with centre push; (c) small rotary encoder (Jog); (d) 4-directional switch with centre push; (e) very small rotary encoder



low-profile type TACT & mechatronic detection switches

slide potentiometers

small rotary encoder

film GlidePoint

colorless tablet with a high transparency of 88%; deadspace of 2.0 mm; micro dot spacers to maintain visibility

hollow shaft encoders for level control

source : ALPS Electric Co., Ltd [2]

8-Directional operating switches with thin center-push switch



pressure sensitive direction switches H01C 010/46 USA Pat. No 6,313,731

pressure sensitive direction devices provide a user interface to facilitate user navigation through increasingly complex menu structures

http://www.altavista.com Engineering basics for CI


the round grid pattern for directional navigation are shown, where the signal and output contact regions U, D, L, R, G are circumferentially displaced and arranged in a circular pattern

pressure sensitive direction switches H01C 010/46 USA Pat. No 6,313,731

capacitive pointing stick apparatus G09G 005/08 USA Pat. No 6,437,772



multi-directional symbol input [7]

the X-conductors (24) and the Y-conductors (25) are electrically connected to a Module of Measuring Touch Point Coordinates (30) which electrically interacts with a Module of Analysis of Lateral Movement Trajectory (31);an Interface Module (32) interacts with both the Module of Analysis of Lateral Movement Trajectory (31) and a Module of Mechanical Keypad (33) to which dome contacts (34) are connected

X-conductors (24) and Y-conductors (25) are placed on a flexible base plate (26); the membrane (28) and the contacts (29) are located under the base plate (26); the membrane (28) and the contacts (29) form a dome switch




contact switch capacitive magnetic reed ferrite core mercury contact



bent springthe bent spring, though more expensive, provides slightly better feedback than electrometric mat underlay (with domes under each key)

capacitance switch with a compression springresistance increases until switch closurethis poor responsiveness is disturbing for most situations, and unacceptable for repetitive use (due to the increased risk of RSI), so is rarely found in contemporary products




pressure-actuated pointing device [14]

the magnitude of the applied positive pressure gradient and point of pressure application on the finger pad determine the magnitude and direction of the cursor's displacement on the graphics screen



tongue touch keypad is the “keyboard” that

utilizes key chording

this device is available for quadriplegics

who need in computer access

the keypad contains a miniature circuit

board with a nine-button keypad and radio

transmitter that fits into a standard dental

retainer worn in the roof of the mouth

http://www.wheelchairnet.org/WCN_ProdServ/Docs/

TeamRehab/RR_97/9702art1.PDF

http://www.gerardpas.com/lrahm/gallery/si11.html


http://www.gerardpas.com/lrahm/gallery/si11.html


a pair of ear-microphones output signals were examined to detect the side of teeth-chattering, right or left at discriminator block [Hashimoto, Yonezawa and Itoh 15]



tonguepoint

is an isometric tongue pointing device was

developed in IBM Almaden Research Center [16]

a tonguepoint is a mouthpiece that, similar to a

dental night guard or a sports mouth guard, is

form fitted to each individual's upper teeth and

hard pallet

because of this fixture the user may relax at

normal jaw posture when wearing the mouthpiece

speaking with the tonguepoint inserted in the

mouth is also feasible

Die Zungensteuerung (PROTOS System)http://www.camt.de/



analog button

the analog button and testing software has also been designed in TAUCHI Unita pilot investigation was carried out to study behavior patterns in hand-eye coordination and some new strategies of their exploitationthe results suggest that there is potential for further development and applications of these alternative input devices to control by different entities (menu pointing, scrolling, etc.) of information environment



pointing devices can improve usability for specific applications and functionality

applications such as mobile gaming will require dedicated pointing devices to satisfy the “easy to play” principle for users

development trends for pointing devices focus on further miniaturization and the ability to deploy re-flow soldering techniques on current devices already employed in consumer electronic products

IR LED

IR-Photodiode

IR-Photodiode



Cameras Complementary Metal-Oxide Semiconductor (CMOS) image

sensors have been highlighted recently with a smaller size and

reduced weight as a candidate technology for integrating digital

camera capability into mobile phones

CMOS image sensors offer lower power consumption and a much

smaller physical integration area than the Charge-Coupled Device

(CCD) image sensors which are conventionally used for digital still

cameras and camcorders that require high picture quality

CMOS image sensors have been accepted only for certain

products that focus on low power consumption rather than picture

quality

demo 2

http://intron.kz.tsukuba.ac.jp/vrlab_web/floatingeye/floatingeye_e.html



the latest CMOS sensor technology could bring around 110,000-pixel

(that is, based on 352 x 288 pixels, they can provide Common source

Intermediate Format (CIF) compliant quality levels) with 1/7” optics, a

form factor of < 101010 mm3 and low consumption of < 100 mW

CCD could bring 350,000 ~ 380,000 -pixel with 1/6” optics

however, the physical integration area is rather bigger than that for

CMOS sensors as CCD requires 3-4 different supply voltages and power

consumption for CCD is still over 200 mW

the next opportunity for image sensors would be to satisfy the requirements of the PDA and notebook PC markets



an image sensor must achieve 640 x 480 dots, or Video Graphics Array

(VGA) compliant

CMOS and CCD image sensors will be competing technologies in sensor

market that needs products with a resolution of VGA-compliant quality

CMOS image sensors used in dark environments suffer deterioration in

colour production quality and increase of output noise

CCD image sensors offer better quality but rather high power

consumption as well as a larger integration area

CMOS image sensors are facing the challenge of improving picture

quality along with downsizing whilst CCD image sensors are facing the

challenge of reducing their size and power consumption



proximity detector for a seeing eye mouse

motion produces successive frames of translated patterns of pixel information, which are compared by autocorrelation to ascertain the direction and amount of movement [6]a hold feature suspends the production of movement signals to the computer, allowing the mouse to be physically relocated on the work surface without disturbing the position on the screen of the pointer



Solid-State Optical Mouse Sensor

source: Agilent Technology [3]

the HDNS-2000 is a reflective optical sensor that

measures changes in position by optically acquiring

sequential surface images (frames) and

mathematically determining the direction and

magnitude of movement

the sensor is designed to be used with the HDNS-

2100 (Lens), HDNS-2200 (LED Assembly Clip), and

HLMP-ED80 (High Light Output 639 nm LED)

this optical tracking engine has no precision optical

alignment

resolution is specified as 400 cpi (characters per

inch) at rates of motion up to 12 inches per second



source: Agilent Technology [3]



Fingertip Detector outputs a list of fingertips’ 2-D coordination only if fingertips were detected; Stroke Detector watches the alternation of the moving vectors of each fingertip; Keyboard Checker translates fingertip’s coordinates detected as stroke to user-defined key character

virtual keyboard with one CCD camera [17]Engineering basics for CI


a full-size virtual keyboard can be projected by light on to any surface [18, 19]

it can be integrated into mobile phones, laptops, tablet PCs or even sterile medical environments

the keyboard, manufactured by Developer VKB Inc, in Israel ( http://www.vkb.co.il/ )

the mini projector that detects user interaction with the surface also simulates a mouse pad (Hanover, Germany, CeBIT 2002)


http://www.vkb.co.il/


facial gesture musical interfaces [20]

the musculature of the face allows for fine motor control of actionsso it is interesting to explore the possibility of machine interfaces that are driven by facial actionbecause facial action is involved in both speech production and emotional expression, there is a rich space of intuitive gesture to sound mappings for face action

Demo 3



http://members.aon.at/mth/mocap/mocaptext.htmhttp://www.vicon.com/main/images/misc/sci_rehab2.jpghttp://ligwww.epfl.ch/~molet/pampers/EGCAS96/firstbig.jpeg, secondbig.jpeg

Body motions

[32]


http://members.aon.at/mth/mocap/mocaptext.htm

http://www.vicon.com/main/images/misc/sci_rehab2.jpg

http://www.vicon.com/main/images/misc/sci_rehab2.jpg

http://ligwww.epfl.ch/~molet/pampers/EGCAS96/firstbig.jpeg


human movement tracking technology [24]



muscle twitch switches are activated by muscle contraction they can be used with eyebrow movement and finger flexion [21]

magnet

reed switch



“Body Coupled FingeRing”: Wireless Wearable Keyboard [30] the transmitter (TX) mounted on the base of finger and the receiver (RX) mounted on the wrist



Samsung Scurry wearable keyboard Futurelooks.com http://www.futurelooks.com/features/events/comdex2k1vegas/pictures/the%20technology/pages/Samsung%20Scurry%20wearable%20keyboard.htm

Senseboard [http://www.senseboard.com/] works by tracking the muscle movements in the palm of the hand: when you extend your left pinky finger in midair and strike it down as if you were going to strike the "Q" key Senseboard displays the letter "Q" on the monitor

Samsung's Scurry works by attaching motion sensors to each finger; it doesn't detect muscle movement, but rather uses gyroscopic technology to detect angular movements of fingers through space

this approach works better, however, both devices are too bulky

[http://www.pcworld.com/news/article/0,aid,70568,00.asp ]


http://www.futurelooks.com/features/events/comdex2k1vegas/pictures/the%20technology/pages/Samsung%20Scurry%20wearable%20keyboard.htm




http://www.senseboard.com/


GestureWrist is a wristband-type input

device that recognizes hand gestures

and forearm movements

unlike DataGloves or other hand

gesture-input devices, all sensing

elements are embedded in a normal

wristband

GesturePad is a sensing module that

can be attached on the inside of

clothes, and users can interact with this

module from the outside

it transforms conventional clothes into

an interactive device without changing

their appearance


http://www.csl.sony.co.jp/person/rekimoto/gwrist/gband.jpg

http://www.csl.sony.co.jp/person/rekimoto/gwrist/


measuring wrist-shape, forearm movements and gestures [34]



clothes as communication surfaces [34]



SmartSkin: An Infrastructure for Freehand Manipulation on Interactive Surfaces [35]

demo4

demo5



demo6

the Gesture Wall [36-38] injected a 50 100 kHz signal into the body of the user through an electrode on the floor; the strengths of this signal, as capacitively received at electrodes placed in the four corners of the display, were used to track the position of a hand as it moved around the display surfacealthough this system was very sensitive to gesture, it required fairly stiff postural constraints on the part of the user - one hand forward and body back, since the entire body radiates the transmit signal, not just the hand to be tracked



tactile array sensor

top: exploded view showing sensor constructionbottom: side view showing the crossed layers of copper strips separated by silicone rubber spacers a protective rubber coating is added on the contact surface [26]



infant health monitoring system [27]



dynamic system for determining human

physical instability [28], the degree to

which his physical stability is impaired

without regard to the cause of impairment

signals of the sensors which are mounted

on the platform depend on the deviation of

the platform from the X and Y axes, when

the subject standing on the platform shifts

his weight



respiration and movement monitoring system [29]a monitoring system (10) includes a first sensor (12) for detecting the respiration and/or movements of an infant (14), and a sensor (18) for detecting the presence or movement of the infant or proximal objects (20); an accelerometric sensor (22) detects movements of a platform (16); an audio sensor (24) detects sounds associated with the infant or proximal objectsnone of the sensors are physically attached to the infant the high-impedance element and the sensor forming a voltage divider that produces from the signal a sensor voltage that is proportional to the impedance of the first sensor



PadGraph is a registrar of body motions based on capacitive sensors [22]

+ 5V

Ix

Gnd

Iy

elastic porous non-conductor

elastic conductor



HoloWall is a wall-sized computer display that consists of a glass wall with rear-projection sheet behind it a video projector displays images on the wall, while inputs are recognized with infrared (an array of IR LEDs) and a video camera with an IR filter (840 nm) installed behind the wallwhen a user moves a finger close enough to the screen (0-30 cm, depending on the threshold value of the recognition software), it reflects IR light and thus becomes visible to the camerathrough image processing technique, the finger shape can be separated from the background [33]



schematic of haptic interaction system based on Lorentz force magnetic levitation [40, 41]



Magnetic Imaging System of Virtual Objects in Haptic Space [42]a detection of density gradient of magnetic field through the small “probe-magnet” (5) coupled to the finger



experimental setup for magnetic imaging system1 - cardboard box; 2 - constant magnets; 3 - the probe magnet; 4 - a copying-paper; 5 - distance control (attenuation of magnetic field)



Communication With PC Input & Output capabilities



Joystick Port

source: Joysticks and other game controllers [43-45]

the joystick interface card was designed to be as simple and cheap as possiblethe card consisted only of bus interface electronics and four monostable multivibrators (in 558 chip) those monostable multivibrators were simple timer circuits which put out a pulse with width directly proportional to the joystick resistance valuethe pulse width was then measured using software loop



the joystick consists of two potentiometers with variable resistance value between 0 Ohm and 100 kOhm (in some joysticks up to 150 kOhm)the potentiometer resistances have the minimum values when the joystick is at the top left positionone end of the potentiometer is connected to +5V pin and the center pin is connected top the analogue input of the joystick

pin purpose1 potentiometer common (Joy A)2 button 1 (Joy A)3 X coordinate potentiometer (Joy A)4 button common (Joy A)5 button common (Joy B)6 Y coordinate potentiometer (Joy A)7 button 2 (Joy A)8 unused9 potentiometer common (Joy B)10 button 1 (Joy B)11 X coordinate potentiometer (Joy B)12 MIDI TXD (transmit) (computer-> midi)13 Y coordinate potentiometer (Joy B)14 button 2 (Joy B)15 MIDI RXD (midi -> computer)



to read the joysticks (or your slide potentiometer positions), you must first write a byte to port 201h, this triggers the 558 timer on the game adapterit doesn't matter what value you send, as long as you perform an I/O writeGame port 201h byte: | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 | | but4 | but3 | but2 | but1 | stk4 | stk3 | stk2 | stk1 | the most machine-independent way to sample the game port is to use a timerNOTE the time just before you trigger the 558(e.g., read the countdown register in Timer 0, you need pretty fine resolution and this timer performs 65535 counts every 55 ms) after triggering, sit in a loop reading port 201h and examining bits 0-3for those bits that have a joystick potentiometer attached, you'll see them sit for a while at 0, then become 1as each bit flips back to 1, note the time againwhen all bit 0-3 have flipped back to 1, you're almost donecompute elapsed time for each bit, and you end up with a value that is proportional to potentiometer position



potentiometers are normally 0-150k variable resistors, and according to the IBM

techref, the time is given by Time = 24.2e-6s + 0.011e-6s * R/Ohms

this equation does not accurately represent the real situation, where there are

differences in absolute components values

in reality you have to calibrate the joystick for the application you use

the most straightforward way to calibrate the stick for the program is to record

the values the joystick gives in extreme positions and in the center position

buttons can be read at any time just by reading port 201h and looking at bits 4-7

No triggering is required

button bits are normally 1; while a button is depressed, its bit will flip to 0



Parallel Port

the original IBM-PC's Parallel Printer Port (the Standard Parallel Port (SPP)) had a total of 12 digital outputs and 5 digital inputs accessed via 3 consecutive 8-bit ports in the processor's I/O space [46-47]•8 output pins accessed via the DATA Port •5 input pins (one inverted) accessed via the STATUS Port •4 output pins (three inverted) accessed via the CONTROL Port •The remaining 8 pins are grounded source: Use of a PC Printer Port for Control and Data Acquisition [46-47]

25-way Female D-Type Connector



various enhanced versions of the original specification have been introduced over the years

•Bi-directional (PS/2) •Enhanced Parallel Port (EPP) •Extended Capability Port (ECP)

each printer port consists of three port addresses; data, status and control portthese addresses are in sequential order; that is, if the data port is at address &H378 the corresponding status port is at &H379 and the control port is at &H37a

Printer Port Data Status Control LPT1 &H3bc &H3bd &H3be LPT2 &H378 &H379 &H37a LPT3 &H278 &H279 &H27a



Pin (25 pin connector) & Port (bit) Assignments on the three ports [46]



in normal printing STROBE is high

all outputs on the Data Port are true logic; that is, writing a logic one to a bit

causes the corresponding output to go high

however, the /SELECT_IN, /AUTOFEED and /STROBE outputs on the Control

Port have inverted logic; that is, outputting a logic one to a bit causes a logic

zero on the corresponding output

this adds some complexity in using the printer port, but the fix is to simply

invert those bits using the exclusive OR function prior to outputting

why the designers of the printer port used inverted logic?

assume you have a printer with no cable attached

an open usually is read as a logic one; thus, if a logic one on the SELECT_IN,

AUTOFEED and STROBE leads meant to take the appropriate action, an

unconnected printer would assume it was selected, go into the autofeed mode

and assume there was data on the outputs associated with the Data Port

the printer would be going crazy when in fact it wasn't even connected [46]



Printer Port - Typical Application [46]NOTE, 5V is an external source

Logic 1 on output DATA 0 (Data Port - Bit 0) causes LED to be offLogic 0 causes LED to turn on normally open push-button causes +5V (logic 1) to appear on input BUSY (STATUS PORT - Bit 7)when depressed, push-button closes and ground (logic 0) is applied to input Busy when idle (waiting), push-button is open and LED is offon depressing push-button, LED blinks on and off at nominally 5 pulses per second



Serial RS232 Port

what are the advantages of using serial data transfer rather than parallel?

the serial port transmits a '1' as -3 to -25 volts and a '0' as +3 to +25 volts

where as a parallel port transmits a '0' as 0v and a '1' as 5v

therefore the serial port can have a maximum swing of 50V compared to

the parallel port which has a maximum swing of 5 Volts

therefore cable loss is not going to be as much of a problem for serial

cables than they are for parallel

if the device needs to be mounted a far distance away from the computer

then 3 core cable (Null Modem Configuration) is going to be a lot cheaper

that running 19 or 25 core cablesource: http://www.beyondlogic.org/serial/serial.htm


http://www.beyondlogic.org/serial/serial.htm

http://www.beyondlogic.org/serial/serial.htm


many palmtop computers and microcontrollers have in built SCI – Serial Communications Interfaces

Serial Communication reduces the pin count of these MPU's

only two pins are commonly used, Transmit Data (TXD) and Receive Data (RXD) compared with at least 8 pins if you use a 8 bit Parallel method + Strobe

the serial transmission is used where one bit is sent at a time

IrDA-1 (the first infra red specifications) was capable of 115.2k baud and was interfaced into a UART (Universal Asynchronous Receiver / Transmitter)

the pulse length however was cut down to 3/16th of a RS232 bit length to conserve power



the electrical specifications of the serial port is contained in the EIA (Electronics Industry Association) RS232C standard, tt states many parameters such as - 1. a "Space" (logic 0) will be between +3 and +25 Volts 2. a "Mark" (Logic 1) will be between -3 and -25 Volts 3. the region between +3 and -3 volts is undefined 4. an open circuit voltage should never exceed 25 volts (in Reference to GND) 5. a short circuit current should not exceed 500mA, the driver should be able to handle this without damage



Abbreviation Full Name FunctionTD Transmit Data Serial Data Output (TXD)RD Receive Data Serial Data Input (RXD)CTS Clear to Send this line indicates that the Modem is ready to

exchange dataDCD Data Carrier Detect when the modem detects a "Carrier" from

the modem at the other end of the phone line,this Line becomes active

DSR Data Set Ready this tells the UART that the modem is ready toestablish a link

DTR Data Terminal Ready this is the opposite to DSR. This tellsthe Modem that the UART is ready to link

RTS Request To Send this line informs the Modem that the UART isready to exchange data

RI Ring Indicator goes active when modem detects a ringingsignal from the PSTN



above is the standard port addresses, which should work for most PC'sif IBM P/S2 has a micro-channel bus, then expect a different set of addresses and IRQ's



USB Port or Universal Serial Bus [50]the original motivation for the Universal Serial Bus (USB) came from three

interrelated considerations:connection of the PC to the telephonethe USB provides a ubiquitous link that can be used across a wide range of PC-to-telephone interconnectsease-of-usethe PC’s I/O interfaces, such as serial/parallel ports, keyboard /mouse /joystick interfaces, etc., do not have the attributes of plug-and-playport expansionthe lack of a bi-directional, low-cost, low-to-mid speed peripheral bus has held back the creative proliferation of peripherals such as telephone/fax/modem adapters, answering machines, scanners, PDA’s, keyboards, mice, etc. existing interconnects are optimized for one or two point productsas each new function or capability is added to the PC, a new interface has been defined to address this need



the more recent motivation for USB 2.0 stems from the fact that PCs have

increasingly higher performance and are capable of processing vast

amounts of data

at the same time, PC peripherals have added more performance and

functionality

user applications such as digital imaging demand a high performance

connection between the PC and these increasingly sophisticated

peripherals

USB 2.0 addresses this need by adding a third transfer rate of 480 Mb/s to

the 12 Mb/s and 1.5 Mb/s originally defined for USB

USB is a fast, bi-directional, isochronous, low-cost, dynamically attachable

serial interface that is consistent with the requirements of the PC platform

of today and tomorrow



the USB is a cable bus that supports data exchange between a host

computer and a wide range of simultaneously accessible peripherals

the attached peripherals share USB bandwidth through a host-scheduled,

token-based protocol

the bus allows peripherals to be attached, configured, used, and detached

while the host and other peripherals are in operation

the USB transfers signal and power over a four-wire cable

the signaling occurs over two wires on each point-to-point segment



USB data transfers take place between host software and a particular endpoint on a USB device, such associations are called pipesdata movement though one pipe is independent from the data flow in any other pipe, a given USB device may have many pipes; while one supports transporting data to the USB device, another supports transporting data from the USB device the USB architecture comprehends four basic types of data transfers:Control Transfers: used to configure a device at attach time and can be used for other device-specific purposes, including control of other pipes on the deviceBulk Data Transfers: generated or consumed in relatively large and bursty quantities and have wide dynamic latitude in transmission constraintsInterrupt Data Transfers: used for timely but reliable delivery of data, for example, characters or coordinates with human-perceptible echo or feedback response characteristicsIsochronous Data Transfers: occupy a prenegotiated amount of USB bandwidth with a prenegotiated delivery latency (also called streaming real time transfers)



Hubs are a key element in the plug-and-play architecture of the USB, serve to simplify USB connectivity from the user’s perspective and provide robustness at relatively low cost and complexityhubs are wiring concentrators and enable the multiple attachment characteristics of the USB

Hubs in a Desktop Computer Environment [50]



hubs can detect attach and detach at each downstream port and enable the distribution of power to downstream devices; each downstream port can be individually enabled and attached to either high-, full- or low-speed devices

a USB 2.0 hub consists of three portions:

the Hub Controller, the Hub Repeater, and the Transaction Translator

the Hub Repeater is a protocol-controlled switch between the upstream port and downstream ports, has reset and suspend/resume signaling

the Host Controller provides the communication to/from the host; hub-specific status and control commands permit the host to configure a hub and to monitor and control its ports

the Transaction Translator provides the support of full-/low-speed devices behind the hub, while transmitting all device data between the host and the hub at high-speed



Data Encoding/Decodingthe USB employs NRZI* data encoding when transmitting packetsin NRZI encoding, a “1” is represented by no change in level and a “0” is represented by a change in levelthe high level represents the J state on the data lines a string of zeros causes the NRZI data to toggle each bit timea string of ones causes long periods with no transitions in the data

Non Return to Zero Invert (NRZI) - a method of encoding serial data in which ones and zeroes are represented by opposite and alternating high and low voltages where there is no return to zero (reference) voltage between encoded bits, eliminates the need for clock pulses

a data stream and the NRZI Data Encoding

J



Universal Serial Bus Revision 2.0 specification [50, 51] (.zip, 6.5Mb, 650 pages!) provides the technical details to understand USB requirements and design USB compatible products (12/21/2000)

the Enhanced Host Controller Interface (EHCI) specification [52]describes the register-level interface for a Host Controller for the Universal Serial Bus (USB) Revision 2.0. The specification includes a description of the hardware/software interface between system software and the host controller hardware. Some key features of the EHCI specification are:

Full, Robust Support for all USB 2.0 Features

Low-risk support for Full- and Low-speed peripherals

System Power Management

Provides simple, robust solutions to USB 1.1 Host Controller Issues

Optimized for Best Memory Access Efficiency

Minimized Hardware Complexity

Support for 32 and 64-bit Addressing



[1] Key Components for 3G Devices, Report No. 15 from the UMTS Forum, January 2002 http://www.cs.berkeley.edu/~perj/3GPP/Documents/UMTS_Forum_3g_devices.pdf

[2] ALPS Electric Co., Ltd http://www.alps.co.jp/press/new2002/f0221-e.htm[3] Mouse sensors for optical navigation. Datasheets are available at:

http://www.semiconductor.agilent.com[4] Association of Super-Advanced Electronics Technology, http://www.aset.or.jp/seika_hdd_indexe.html[5] IBM Magnetic Hard Disk Drive Technology,

http://www.almaden.ibm.com/sst/html/leadership/leadership.htm[6] Gordon, et al, Proximity detector for a seeing eye mouse, Agilent Technologies, Inc. (Palo Alto, CA),

G09G 005/08, USA Pat 6,281,882 [7] Multi-directional symbol input, http://www.vitgn.com/[8] Subramanian, V. Fabrication of thin film transistors for Liquid Crystal Display applications. ESCI 577

Literature Review Report. http://www.personal.psu.edu/users/v/t/vts103/tft.doc[9] XtraViewTM Wide Viewing Angle Technology, http://www.necmitsubishi.com/markets-solutions/

financial/downloads/xtraview.pdf[10] Griffin, T. Haptic Feedback in Button Technologies, 1999,

http://tim.griffins.ca/writings/haptic_tech_body[11] Force / Travel Diagram, ICHIA Technologies Inc.

http://www.ichia.com/keypad/silicone/terminology/snap.htm[12] Logistic Design (UK) Ltd. www.logisticdesign.co.uk/data%20sheets/term.pdf[13] Logistic Design (UK) Ltd. www.logisticdesign.co.uk/data%20sheets/membrane.pdf[14] Gervais, J-Ph. A.F.M., Pressure-actuated pointing device, G09G 003/02, USA Pat. No 5,508,719[15] Hashimoto, M., Yonezawa, Y. and Itoh, K. New mouse-function using teeth-chattering and potential

around eyes for the physically challenged. In: Interdisciplinary Aspects in Computers Helping People with Special Needs. 5th Int. Conf. ICCHP'96 Linz, Austria, July 1996. R. Oldenbourg Verlag GmbH Munich, Germany, Part 1, pp. 93-98.

References


http://www.cs.berkeley.edu/~perj/3GPP/Documents/UMTS_Forum_3g_devices.pdf

http://www.alps.co.jp/press/new2002/f0221-e.htm




http://www.semiconductor.agilent.com/

http://www.aset.or.jp/seika_hdd_indexe.html

http://www.almaden.ibm.com/sst/html/leadership/leadership.htm

http://www.vitgn.com/

http://www.personal.psu.edu/users/v/t/vts103/tft.doc

http://tim.griffins.ca/writings/haptic_tech_body

http://www.ichia.com/keypad/silicone/terminology/snap.htm

http://www.logisticdesign.co.uk/data%20sheets/term.pdf

http://www.logisticdesign.co.uk/data%20sheets/membrane.pdf


[16] Salem, C. and Zhai, S. An Isometric Tongue Pointing Device, IBM Almaden Research Center http://www.almaden.ibm.com/cs/people/zhai/

[17] Nozomu MATSUI and Yoshikazu YAMAMOTO, A New Input Method of Computers with One CCD Camera: Virtual Keyboard, INTERACT’01, pp. 678-679, http://www.yy.ics.keio.ac.jp/~nozomu/research/vk/

[18] The full-size virtual keyboard, http://www.ananova.com/news/story/sm_548253.html , http://www.vkb.co.il/

[19] Kolsch, M. and Matthew Turk, M., Keyboards without Keyboards: A Survey of Virtual Keyboards, http://www.cs.ucsb.edu/research/trcs/docs/2002-21.pdf

[20] Facial Gesture Musical Interfaces, http://www.mis.atr.co.jp/~mlyons/mouthesizer.html[21] Muscle Twitch Switches, http://www.cs.wright.edu/bie/rehabengr/Switch1/twitch.htm[22] Evreinov G., Agranovski A., Yashkin A., Evreinova T. PadGraph. In: Human-Computer Interaction:

Communication, Cooperation, and Application Design, Vol. 2 of the Proc. of HCI International '99, Munich, Germany, August 22-26, 1999. Hans-Jorg Bullinger and Jurgen Ziegler (eds.) Lawrence Erlbaum Associates, Publishers Mahwah, New Jersey, London, 1999, pp. 985-989.

[23] Robert J.K. Jacob, John J. Leggett, Brad A. Myers, et al. An Agenda for Human-Computer Interaction Research: Interaction Styles and Input/Output Devices, http://citeseer.nj.nec.com/177873.htmlhttp://www.cs.tufts.edu/~jacob/papers/bit.pdf

[24] Mulder, A. Human movement tracking technology, 1994, http://www.cs.sfu.ca/~amulder/personal/vmi/, http://www.cs.sfu.ca/~amulder/personal/vmi/HMTT.pub.html

[25] Antifakos, S., Sensors, http://www.vision.ethz.ch/antifako/sensors.html[26] Pawluk, D.T.V., Son, J.S., Wellman, P.S., Peine, W.J. and Howe, R.D. A Distributed Pressure Sensor

For Biomechanical Measurements, Journal of Biomechanical Engineering, April, 1998. http://www.med.jhu.edu/somlab/dianne/refs.html

[27] Higgins, et al. Infant health monitoring system, 1996, A61B 005/020.5 USA Pat. 5,479,932[28] Zanakis, M.F. Dynamic system for determining human physical instability, 1999, A61B 005/103, USA

Pat. 5,919,150


http://www.almaden.ibm.com/cs/people/zhai/

http://www.yy.ics.keio.ac.jp/~nozomu/research/vk/

http://www.ananova.com/news/story/sm_548253.html










http://www.cs.ucsb.edu/research/trcs/docs/2002-21.pdf

http://www.mis.atr.co.jp/~mlyons/mouthesizer.html

http://www.cs.wright.edu/bie/rehabengr/Switch1/twitch.htm

http://citeseer.nj.nec.com/177873.html

http://www.cs.tufts.edu/~jacob/papers/bit.pdf

http://www.cs.sfu.ca/~amulder/personal/vmi/

http://www.cs.sfu.ca/~amulder/personal/vmi/HMTT.pub.html

http://www.vision.ethz.ch/antifako/sensors.html

http://www.med.jhu.edu/somlab/dianne/refs.html


[29] Teodorescu, et al. Respiration and movement monitoring system, 2000, G08B 023/00, USA Pat. 6,011,477

[30] FUKUMOTO, Masaaki, TONOMURA, Yoshinobu, "Body Coupled FingeRing": Wireless Wearable Keyboard, http://www.acm.org/sigchi/chi97/proceedings/paper/fkm.htm

[31] Alternative Control Technologies: Human Factors Issues, RTO NATO, 1998, RTO-EN-3 AC/323(HFM)TP/1, http://www.rta.nato.int/RDP.asp?RDP=RTO-EN-003ftp://ftp.rta.nato.int/PubFulltext/RTO/EN/RTO-EN-003/$$EN-003-ALL.pdf

[32] Alternative Control Technologies, 1998, RTO-TR-7 AC/323(HFM)TP/3, http://www.rta.nato.int/RDP.asp?RDP=RTO-TR-007ftp://ftp.rta.nato.int/PubFulltext/RTO/TR/RTO-TR-007/$$TR-007-ALL.PDF

[33] Rekimoto J., HoloWall, http://www.csl.sony.co.jp/person/rekimoto/holowall/[34] Rekimoto, J., GestureWrist and GesturePad: Unobtrusive Wearable Interaction Devices,

http://www.csl.sony.co.jp/person/rekimoto/papers/chi02.pdf[35] Jun Rekimoto, SmartSkin: An Infrastructure for Freehand Manipulation on Interactive Surfaces,

CHI2002, 2002, http://www.csl.sony.co.jp/person/rekimoto/papers/chi02.pdf, http://www.csl.sony.co.jp/person/rekimoto/smartskin/

[36] J. A. Paradiso, K. Hsiao, J. Strickon, J. Lifton, and A. Adler Sensor systems for interactive surfaces – References, http://www.research.ibm.com/journal/sj/393/part3/paradiso.pdf

[37] J. R. Smith, T. White, C. Dodge, J. Paradiso, N. Gershenfeld, and D. Allport, “Electric Field Sensing for Graphical Interfaces,” IEEE Computer Graphics and Applications 18, No. 3, 54 60 (1998), http://www.research.ibm.com/journal/sj/393/part3/ref21

[38] J. Paradiso, “The Brain Opera Technology: New Instruments and Gestural Sensors for Musical Interaction and Performance,” Journal of New Music Research 28, No. 2, 130 149 (1999), http://www.research.ibm.com/journal/sj/393/part3/ref21

[39] Aggarwal, J. K. and Cai, Q. Human Motion Analysis: A Review. In Proceedings of the IEEE Nonrigid and Articulated Motion Workshop 1997. IEEE, Piscataway, NJ, USA.


http://www.acm.org/sigchi/chi97/proceedings/paper/fkm.htm

http://www.rta.nato.int/RDP.asp?RDP=RTO-EN-003

ftp://ftp.rta.nato.int/PubFulltext/RTO/EN/RTO-EN-003/$$EN-003-ALL.pdf

http://www.rta.nato.int/RDP.asp?RDP=RTO-TR-007

ftp://ftp.rta.nato.int/PubFulltext/RTO/TR/RTO-TR-007/$$TR-007-ALL.PDF

http://www.csl.sony.co.jp/person/rekimoto/holowall/

http://www.csl.sony.co.jp/person/rekimoto/papers/chi02.pdf



http://www.csl.sony.co.jp/person/rekimoto/smartskin/

http://www.research.ibm.com/journal/sj/393/part3/paradaut.html#paradiso

http://www.research.ibm.com/journal/sj/393/part3/paradaut.html#hsiao

http://www.research.ibm.com/journal/sj/393/part3/paradaut.html#strickon

http://www.research.ibm.com/journal/sj/393/part3/paradaut.html#lifton

http://www.research.ibm.com/journal/sj/393/part3/paradiso.pdf

http://www.research.ibm.com/journal/sj/393/part3/paradiso.pdf

http://www.research.ibm.com/journal/sj/393/part3/ref21







[40] Peter J. Berkelman and Ralph L. Hollis, Magnetic Levitation Haptic Interfaces, http://www-2.cs.cmu.edu/~msl/haptic/haptic_desc.html

[41] P. J. Berkelman, Z. J. Butler, and R. L. Hollis, "Design of a Hemispherical Magnetic Levitation Haptic Interface Device," 1996 ASME IMECE, Atlanta, November 17-22, 1996, DSC-Vol. 58, pp. 483-488, http://www-2.cs.cmu.edu/~msl/publications/asme96.design.pdf

[42] Evreinov G. Magnetic Imaging System of Virtual Objects in Haptic Space. Computer Helping People with Special Needs. ICCHP2000. Proc. of the 7th International Conference on Computer Helping People with Special Needs. July, 2000, Karlsruhe, Germany, pp. 182-191.

[43] Clark, J. May the Force Feedback Be with You: Grappling with DirectX and DirectInput, 1998, http://www.microsoft.com/msj/defaultframe.asp?page=/msj/0298/force.htm&nav=/msj/0298/newnav.htm

[44] Tomi Engdal, Joysticks and other game controllers, 1996-1998 http://documents.epanorama.net/documents/joystick/

[45] PC Game Programmer's Encyclopedia, http://brand107.home.attbi.com/pc-gpe/[46] Anderson, P.H., Use of a PC Printer Port for Control and Data Acquisition, http://et.nmsu.edu/~etti/,

http://et.nmsu.edu/~etti/fall96/computer/printer/printer.html[47] Beyond Logic, (BUS & Ports) http://www.beyondlogic.org/[48] Intel’s Developer Site, http://developer.intel.com/[49] USB Developers, http://www.usb.org/developers/[50] Universal Serial Bus Specification Revision 2.0, http://www.usb.org/developers/data/usb_20.zip[51] USB-2-0.com, http://www.usb-20.com/what-is-usb-2-0.html[52] Enhanced Host Controller Interface (EHCI) specification,

http://developer.intel.com/technology/usb/download/ehci-r10.pdf[53] FLEXIBLE DISPLAYS, http://www.darpa.mil/MTO/displays/hds/Presentations/-

EICPresentationsSrg2000/FlexibleDisplays/index.html[54] Ferroelectric Liquid Crystal (FLC) Spatial Light Modulators (SLMs),

http://oldeee.see.ed.ac.uk/profiles/research/STR/research_projects/slm/slm.html[55] Ports’ tutorial, http://www.ctv.es/pckits/tutorial.html


http://www-2.cs.cmu.edu/~msl/haptic/haptic_desc.html

http://www-2.cs.cmu.edu/~msl/publications/asme96.design.pdf

http://www.microsoft.com/msj/defaultframe.asp?page=/msj/0298/force.htm&nav=/msj/0298/newnav.htm

http://documents.epanorama.net/documents/joystick/

http://et.nmsu.edu/~etti/

http://et.nmsu.edu/~etti/fall96/computer/printer/printer.html

http://www.beyondlogic.org/

http://developer.intel.com/

http://www.usb.org/developers/

http://www.usb.org/developers/data/usb_20.zip

http://www.usb-20.com/what-is-usb-2-0.html

http://developer.intel.com/technology/usb/download/ehci-r10.pdf

http://developer.intel.com/technology/usb/download/ehci-r10.pdf


The Next Lecture:

http://www.hash.com/users/navone/HTML/AlienSongDownload.htm


new interaction techniques

Documents

evreinov p

citauchi mmig

umts devices

grseengineering basics

business user devices

wide rangingengineering

java etcengineering

umts terminals