Skinput technology

Dept. of Computer Science & Engg., C.O.E.,Ambajogai Page 1

1) Introduction:

The world is going crazy over an invention, which is known as mobile

phones. The Mobile devices became popular in less time due some advantages they came up

with, like portability, flexibility, mobility and responsiveness. These devices easily get fit in

our pocket means we don’t need to carry any extra surface area with us. Devices with

significant computational power and capabilities can now be easily carried on our bodies.

However, their small size typically leads to limited interaction space and consequently

diminishes their usability and functionality. Since, we cannot simply make buttons and screens

larger without losing the primary benefit of small size.

Appropriating the human body as an input device is appealing not only

because we have roughly two square meters of external surface area, but also because much of

it is easily accessible by our hands (e.g., arms, upper legs, torso). Furthermore, proprioception –

our sense of how our body is configured in three-dimensional space – allows us to accurately

interact with our bodies in an eyes-free manner. We can use any part of our body as an input

surface but the for comfortable operation we need to use our arm as an input. Skinput is a

method that allows the body to be appropriated for finger input using a wearable bio-acoustic

sensor.

The technology was developed by Chris Harrison, Desney Tan, and Dan Morris,

at Microsoft Research's Computational User Experiences Group. Skinput is a combination of

three technologies which are pico-projector, bioacoustics sensors and Bluetooth. Pico-projector

will display mobile screen on our skin. As according to our need we tap on our body. After

tapping some vibrations are produced through our body, those ripples are captured by

bioacoustics sensors which are mounted armband. These armband is connected to the mobile

device by wireless connection i.e. Bluetooth. Mobile device consists of a software which

matches these vibration signal with the store signals and desired operation is performed. We

have use Support Vector Machine algorithm i.e. supervised learning algorithm to train our

software. At initial stage we have to store the signal data from each location of our arm which

is the reference signal for our software. Skinput employs acoustics, which take advantage of the

human body's natural sound conductive properties (e.g., bone conduction). This allows the

Skinput technology

Dept. of Computer Science & Engg., C.O.E.,Ambajogai Page 2

body to be annexed as an input surface without the need for the skin to be invasively

instrumented with sensors, tracking markers, or other items.

The description of the design of a wearable sensor for bio-acoustic signal

acquisition. Also the description of an analysis approach that enables skinput system to resolve

the location of finger taps on the body. In this we present working on skinput—a method that

allows the body to be appropriated for finger input using a wearable bio-acoustic sensor. When

coupled with a pico-projector, the skin can operate as an interactive supporting both input and

graphical output.

Skinput technology

Dept. of Computer Science & Engg., C.O.E.,Ambajogai Page 3

2)Skinput:

What Is Skinput

Skinput is a device that uses a pico projector to beam graphics onto a user’s

palm and forearm, transforming the skin into a computer interface. Skinput is a combination of

two words i.e Skin and Input. This technology uses largest part of our body which is skin as an

input surface for mobile gadgets. Chris Harrison and team of Microsoft research has developed

Skinput, a way in which your skin can become a touch screen device or your fingers buttons on

a MP3 controller.

Figure 2.1: Display on palm using Skinput Technology

Skinput represents one way to decouple input from electronic devices with the

aim of allowing devices to become smaller without simultaneously shrinking the surface area

on which input can be performed.

Skinput technology

Dept. of Computer Science & Engg., C.O.E.,Ambajogai Page 4

3)Working of Skinput:

Figure 3.1:Working of skinput

Skinput is a combination of three technologies which are pico-projector,

bioacoustics sensors and Bluetooth. Pico-projector will display mobile screen on our skin. As

according to our need we tap on our body. After tapping some vibrations are produced through

our body, those ripples are captured by bioacoustics sensors which are mounted armband.

These armband is connected to the mobile device by wireless connection i.e. Bluetooth.

3.1 Pico-Projector

Pico projectors are tiny battery powered projectors - as small as a mobile phone - or

even smaller: these projectors can even be embedded inside phones or digital cameras. Pico-

projectors are small, but they can show large displays. While great for mobility and content

Skinput technology

Dept. of Computer Science & Engg., C.O.E.,Ambajogai Page 5

sharing, pico-projectors offer low brightness and resolution compared to larger projectors. It is

a new innovation, but pico-projectors are already selling at a rate of about a million units a year

(in 2010), and the market is expected to continue growing quickly.

Figure 3.2: Pico-projector

We are using DLP (Digital Light Processing) , the idea behind DLP is to use

tiny mirrors on a chip that direct the light. Each mirror controls the amount of light each pixel

on the target picture gets (the mirror has two states, on and off. It refreshes many times in a

second - and if 50% of the times it is on, then the pixel appears at 50% the brightness). Color is

achieved by a using a color wheel between the light source and the mirrors - this splits the light

in red/green/blue, and each mirror controls all thee light beams for its designated pixel. So with

the help of tiny projector we will display required menu bar on our arm.

Skinput technology

Dept. of Computer Science & Engg., C.O.E.,Ambajogai Page 6

3.2 Bio-Acoustics

Acoustics is the interdisciplinary science that deals with the study of all

mechanical waves in gases, liquids, and solids including vibration, sound, ultrasound and

infrasound. A scientist who works in the field of acoustics is an acoustician while someone

working in the field of acoustics technology may be called an acoustical engineer. The

application of acoustics can be seen in almost all aspects of modern society with the most

obvious being the audio and noise control industries. Bioacoustics is a cross-disciplinary

science that combines biology and acoustics. Usually it refers to the investigation of sound

production, dispersion through elastic media, and reception in animals, including humans.

When a finger taps the skin, several distinct forms of acoustic energy are

produced. Some energy is radiated into the air as sound waves; this energy is not captured by

the Skinput system. Among the acoustic energy transmitted through the arm, the most readily

visible are transverse waves, created by the displacement of the skin from a finger impact.

When shot with a high-speed camera, these appear as ripples, which propagate outward from

the point of contact. The amplitude of these ripples is correlated to both the tapping force and to

the volume and compliance of soft tissues under the impact area. In general, tapping on soft

regions of the arm creates higher amplitude transverse waves than tapping on boney areas (e.g.,

wrist, palm, fingers), which have negligible compliance.

In addition to the energy that propagates on the surface of the arm, some

energy is transmitted inward, toward the skeleton. These longitudinal (compressive) waves

travel through the soft tissues of the arm, exciting the bone, which is much less deformable then

the soft tissue but can respond to mechanical excitation by rotating and translating as a rigid

body. This excitation vibrates soft tissues surrounding the entire length of the bone, resulting in

new longitudinal waves that propagate outward to the skin.

Skinput technology

Dept. of Computer Science & Engg., C.O.E.,Ambajogai Page 7

3.2.1 Transverse Wave Propagation :

Figure 3.3: Finger impacts displace the skin, creating transverse waves (ripples). The sensor is

activated as the wave passes underneath it.

3.2.2 Longitudinal Wave Propagation :

Figure 3.4: Finger impacts create longitudinal (compressive) waves that cause internal skeletal

structures to vibrate. This, in turn, creates longitudinal waves that emanate outwards from the

bone (along its entire length) toward the skin.

We highlight these two separate forms of conduction, transverse waves moving

directly along the arm surface, and longitudinal waves moving into and out of the bone through

soft tissues – because these mechanisms carry energy at different frequencies and over different

distances. Roughly speaking, higher frequencies propagate more readily through bone than

through soft tissue, and bone conduction carries energy over larger distances than soft tissue

conduction. While we do not explicitly model the specific mechanisms of conduction, or

Skinput technology

Dept. of Computer Science & Engg., C.O.E.,Ambajogai Page 8

depend on these mechanisms for our analysis, we do believe the success of our technique

depends on the complex acoustic patterns that result from mixtures of these modalities.

Similarly, we also believe that joints play an important role in making tapped locations

acoustically distinct. Bones are held together by ligaments, and joints often include additional

biological structures such as fluid cavities. This makes joints behave as acoustic filters. In some

cases, these may simply dampen acoustics; in other cases, these will selectively attenuate

specific frequencies, creating location specific acoustic signatures.

Figure 3.5: Arm band which consists of vibration sensor array

3.2.3 Bioacoustic Sensor:

The Minisense 100 is a low-cost cantilever-type vibration sensor loaded

by a mass to offer high sensitivity at low frequencies. The pins are designed for easy

installation and are solderable. Horizontal and vertical mounting options are offered as well as

a reduced height version. The active sensor area is shielded for improved RFI/EMI rejection.

Rugged, flexible PVDF sensing element withstands high shock overload. Sensor has excellent

Skinput technology

Dept. of Computer Science & Engg., C.O.E.,Ambajogai Page 9

linearity and dynamic range, and may be used for detecting either continuous vibration or

impacts.

Some features of Minisense 100 are given below:

High Voltage Sensitivity (1 V/g)

Horizontal or Vertical Mounting

Shielded Construction

Solderable Pins, PCB Mounting

Low Cost

< 1% Linearity

Up to 40 Hz (2,400 rpm) Operation Below Resonance

3.3 Bluetooth :

Bluetooth is a wireless technology standard for exchanging data over short

distances from fixed and mobile devices, creating personal area networks (PANs) with high

levels of security. It can connect several devices, overcoming problems of synchronization.

Bluetooth takes small-area networking to the next level by removing the need for user

intervention and keeping transmission power extremely low to save battery power.

Bluetooth is essentially a networking standard that works at two levels:

It provides agreement at the physical level -- Bluetooth is a radio-frequency standard.

It provides agreement at the protocol level, where products have to agree on when bits

are sent, how many will be sent at a time, and how the parties in a conversation can be

sure that the message received is the same as the message sent.

The low power limits the range of a Bluetooth device to about 10 meters, cutting

the chances of interference between your computer system and your portable telephone or

television. Even with the low power, Bluetooth doesn't require line of sight between

Skinput technology

Dept. of Computer Science & Engg., C.O.E.,Ambajogai Page 10

communicating devices. The walls in your house won't stop a Bluetooth signal, making the

standard useful for controlling several devices in different rooms.

Bluetooth can connect up to eight devices simultaneously. With all of those

devices in the same 10-meter radius, you might think they'd interfere with one another, but it's

unlikely. Bluetooth uses a technique called spread-spectrum frequency hopping that makes it

rare for more than one device to be transmitting on the same frequency at the same time..

So we are connecting armband and mobile device using Bluetooth

technology. So whatever data is received by the sensors are transferred to the mobile device.

That mobile device samples the data and compared it with the stored data and according to the

algorithm task is performed.

Skinput technology

Dept. of Computer Science & Engg., C.O.E.,Ambajogai Page 11

4)Advantages:

Easy to work: Skinput technology is very easy to understand and it’s very easy to use.

No interaction with the gadget : If we have to use any application of our mobile then we

reach to our pocket take out the device, unlock it and then go to the application. By

using Skinput we do not need any interaction with the gadget. We have to just tap our

finger and the desired function will performed by the system.

No worry about keypad : People with large fingers gets trouble while operating touch

screens. Using Skinput we get very large interaction surface area. So for such people

this problem will resolve.

Easy to access when your phone is not available.

Allows users to interact more personally with their device.

Through the use of a sense called proprioception after user learns where the locations

are on the skin they will no longer have to look down to use Skinput reducing people

looking down at their phone while driving.

It can be used for a more interactive gaming experience.

Skinput technology

Dept. of Computer Science & Engg., C.O.E.,Ambajogai Page 12

5)Disadvantages:

Skinput has its downfalls, especially due to fact of the BIG band that looks very easy to

put on. Many people would not wear a very big band around their arm for the day just

to have this product.

This technology only works on direct skin exposure. We cannot use full sleeves shirt

when we are using this technology.

The visibility of the projection of the buttons on the skin can be reduced if the user has

a tattoo located on their arm

Not enough research has been conducted on this product to test the possible skin

diseases or type of cancer one can get from using this product.

This technology might start up at very high cost which will not be affordable for the

common man.

Skinput technology

Dept. of Computer Science & Engg., C.O.E.,Ambajogai Page 13

6)Applications:

1. Mobile:We can use Skinput technology in any mobile device. We just need different

software for different mobiles like for mobiles which supports android operating system

requires android application operating system requires .jar or .sis software.

2. I-pods:We can use this technology in i-pods or other music devices which supports

Bluetooth technology. For such music devices we just need 4 or 5 different buttons. So

we can use our fingertips as input. Like this we can operate these devices without any

visual contact.

3. Game: In gaming devices we can use this technology. So without any joysticks or touch

screens we can play games very easily.

Skinput technology

Dept. of Computer Science & Engg., C.O.E.,Ambajogai Page 14

7)Future Implementation:

In order to assess the real-world practicality of Skinput, we are currently

building a successor to our prototype that will incorporate several additional sensors,

particularly electrical sensors and inertial sensors (accelerometers and gyroscopes). In addition

to expanding the gesture vocabulary beyond taps, we expect this sensor fusion to allow

considerably more accuracy—and more robustness to false positives—than each sensor alone.

This revision of our prototype will also allow us to benefit from anecdotal lessons learned since

building our first prototype: in particular, early experiments with subsequent prototypes suggest

that the hardware filtering we describe above can be effectively replicated in software, allowing

us to replace our relatively large piezoelectric sensors with micro-machined accelerometers.

Figure 7.1:Future Implementation

This considerably reduces the size and electrical complexity of our

armband. Furthermore, anecdotal evidence has also suggested that vibration frequency ranges

as high as several kilohertz may contribute to tap classification, further motivating the use of

broadband accelerometers. Finally, our multi-sensor armband will be wireless, allowing us to

explore a wide variety of usage scenarios, as well as our general assertion that always-available

input will inspire radically new computing paradigms.

Skinput technology

Dept. of Computer Science & Engg., C.O.E.,Ambajogai Page 15

8)Conclusion:

Skinput allows the human body as an input surface.

It describes a novel, wearable bio-acoustic sensing array that we built into an armband

in order to detect and localize finger taps on the forearm and hand.

This system performs well even when the body is in motion.

Skinput technology

Dept. of Computer Science & Engg., C.O.E.,Ambajogai Page 16

9)References:

1) Chris Harrison, Desney Tan, and Dan Morris “Skinput: Appropriating the Skin as an

Interactive Canvas” Microsoft Research 2011.

2) Chris Harrison, Scott E. Hudson “Scratch Input: Creating, Large Inexpensive, Unpowered

and Mobile Finger Input Surfaces”UIST 2008.

3) Amento, B.Hill, W.Terveen “The Sound of one Hand: A wrist- mounted bio-acoustic

fingertip gesture interface” CHI’02.

4) Thomas Hahn “Future Human Computer Interaction with special focus on input and output

techniques” HCI.

5) Burges, C.J. A Tutorial on Support Vector Machines for Pattern Recognition. Data Mining

and Knowledge Discovery, 2.2, 121-167.

6) Clinical Guidelines on the Identification, Evaluation, and Treatment of Overweight and

Obesity in Adults. National Heart, Lung and Blood Institute.

7) Deyle, T., Palinko, S., Poole, E.S., and Starner, T. Hambone: A Bio-Acoustic Gesture

Interface. In Proc. ISWC '07. 1-8.