RemoTouch:A System For Remote Touch Experience

Domenico Prattichizzo1,2 Francesco Chinello1 Claudio Pacchierotti1 Kouta Minamizawa31Dipartimento di Ingegneria dell’Informazione, Universita di Siena, Italia

2Istituto Italiano di Tecnologia, Genova Italia3Tachi Lab, University of Keio, Japan

Abstract— This paper presents some preliminary results onRemoTouch, a system allowing to perform experiences ofremote touch. The system consists of an avatar equipped with aninstrumented glove and a user wearing tactile displays allowingto feel the remote tactile interaction. The main features ofRemoTouch are that it is a wearable system and that a humanavatar is used to collect remote tactile interaction data. Newparadigms of tactile communication can be designed aroundthe RemoTouch system. Two simple experiences are reportedto show the potential of the proposed remote touch architecture.

I. INTRODUCTION

Humans have always attempted to extend their percep-tion abilities to enlarge the workspace of the human body.Consider, for instance, the role played by mobile phones foraudio modality. If one thinks about technology for recordingand playback of audio and video, portable devices cometo mind. It isn’t so for touch modality. Technology fortouching remote objects has typically been used in roboticsfor teleoperation and the devices used are very far frombeing portable. In telepresence, a robot is used as a slave inthe remote scenario and a haptic interface or an exoskeletonfeeds back contact forces letting the user perceive the remoteenvironment [1]. Current technology for telepresence is veryadvanced but it is not portable and is not low energyconsumption thus compromising the possibility to be used ineveryday life. in Fig. 1 an example of telepresence systemdeveloped at DLR is reported [2].

Fig. 1. Example of a teleperation system where the user teleoperates by theremote robot. Courtesy of Institute of Robotics and Mechatronics GermanAerospace Center (DLR),

A different approach to remote tactile perception is pre-sented in this work. It involves replacing the slave robot with

e-mail: prattichizzo,chinello,pacchierotti@dii.unisi.it, kouta@tachilab.org

a human operator and substituting the exoskeleton, or otherhaptic interfaces, with a simple and wearable tactile device.In the RemoTouch project the device in charge of recordingtactile perception is not a robot but a human avatar. It is ableto collect tactile, audio and video signals to be fed back to theremote user. This is what we refer to as “remote perception”in this paper.

Of course, the main difference with teleoperation is thatthe human avatar cannot be controlled as a robot but thishas nothing to do with RemoTouch which only deals withthe perception of the remote environment through humanavatars.

II. THE REMOTOUCH IDEA

Fig. 2. The functional scheme of the RemoTouch project. A remote humanavatar collects data to be fed back to the user.

The idea of RemoTouch is pictorially described in Fig. 2:the user (on the left) perceives the force feedback recordedby the human avatar (on the right). The human avatar wearsa glove equipped with force sensors, one per finger as inFig. 3. The measured contact force at the remote interactionis fed back to the user through simple and wearable tactiledisplays, one per finger, as in Fig. 3. The tactile displayis similar to those developed in [3] and has been adaptedto the idea of remote touch here presented. The tactiledisplay consists of two motors and a belt able to deformthe fingertip according to the contact force measured by theremote instrumented glove. RemoTouch is a project whichinvolves telepresence and communication between two users

19th IEEE International Symposium on Robot andHuman Interactive CommunicationPrincipe di Piemonte - Viareggio, Italy, Sept. 12-15, 2010

978-1-4244-7989-4/10/$26.00 ©2010 IEEE 676978-1-4244-7990-0/10/$26.00 ©2010 IEEE

Fig. 3. The glove instrumented with force sensors on the right and thetactile display (with the functional scheme) on the left.

without using a robotic avatar. So the target is to be ableto transfer the contact signal in the same way the voice andthe video image are sent to the remote user. Note that theforce feedback is tactile only and the kinesthetic feedbackis missed. Recently, results have been presented to studyto what extent the kinesthetic feedback can be substitutedwith tactile feedback only [4]. Practical experiments haveshown that this lack of feedback in force is well compensatedby the presence of other modalities like video and audiofeedback, which are extensively used in our experiments.A very important feature of RemoTouch is that the involvedtechnology is simple, wearable, low energy consumption andnot expensive. Of course this is a direct consequence ofhaving chosen a human avatar instead of a remote robotwhich is not a secondary point but the main point of ourproject.

III. THE REMOTOUCH SYSTEM

In this section the hardware and software parts of thesystem are described.

A. Tactile feedback

The device in charge of recording tactile perception is nota robot, as in teleoperation, but a human avatar. The humanavatar wears a glove equipped with a force sensor. Thepiezoresistive force sensor by FlexiForce (model A201) hasbeen chosen to equip each finger of the human avatar as inFig. 3. This force sensor is easily wearable due to its flexiblestructure and has been integrated with a glove to measurethe deformation of the human avatar’s fingertip during thecontact interaction. A standard protocol based on USB hasbeen used for communication with the computational unitand to provide the power supply.

Regarding the user, the force feedback recorded by thehuman avatar is presented to the user using a simple andwearable tactile display as shown in Fig. 3. It consists oftwo motors and a belt able to deform the fingertip accordingto the contact force measured by the remote instrumented

glove. When motors spin in opposite directions the beltapplies a perpendicular force on the user’s fingertip. Notethat this tactile display can also render tangential forces butthis is not used in RemoTouch since the force sensors onthe human hand measure only normal forces. The tangentialforces will be implemented on the next version of the system.For this reason two motors, instead of one, are used togenerate the normal force. The motors are current controlledand the torque command is sent by an embedded C++library. The force feedback is local to the fingerpad andthe kinesthetic feedback is missed. [4]. The actuators are acouple of DC motors of 5V and 500 mA maximum current.The maximum torque is about 5.80 N·cm so the maximumforce applied by the belt on the pad is about 3N. As far ascalibration is concerned, the same force sensor used for theremote instrumented glove has been also used to calibratethe tactile display. The force sensor was fixed between thebelt and the fingerpad, as if the same user was using boththe instrumented glove and the tactile display at the sametime. The relationship between the current to the two motorsof the tactile display and the resulting normal force appliedat the fingerpad and measured by the FlexiForce sensor isapproximately linear as shows in Fig. 4. The relationshipused to map the normal forces to motor currents is

c = 0.720f + 0.1551 (1)

where f , in N, is the normal force measured at the remoteinteraction point with the force sensor fixed to the glove ofthe remote avatar and c is the current, in dA, to be sent to themotors of the tactile display to render the remote measurednormal force at the contact.

B. Visual feedback

The remote user, who wears the tactile display, enhanceshis perception using head mounted video-glasses displayingwhat the head-mounted camera on the human avatar is

Fig. 4. Tactile display charateristic. Motor current (dA) and normal force(N) generated by the belt on the fingerpad.

677

capturing. The video signal is acquired by the human avatarcamera and transmitted to the user’s head mounted display.Particular attention has been paid to get coherence betweenthe point of view of the head mounted camera and thedisplay.

The visual feedback is a relevant part of the remote touchexperience. The experience perceived by the user is justifiedby a perceptive illusion similar to the one described for therubber hand in [5]. The main idea is to let the operatorperceive the remote objects through the human avatar. Usingonly the tactile modality, there is not enough informationabout the remote context. Experiments on brain cognitionshow that the human brain needs some knowledge aboutthe remote environment [6]. In particular the human avatarneeds to share what he’s seeing with the remote user usingthe same point of view of the 3D scene. The RemoTouchvision system consists of a camera, fixed on the head ofthe human avatar, and a head mounted display worn by theuser during the experience. Although the 3D display is animmersive experience for vision system, RemoTouch doesn’tuse a 3D vision system. This lack is due to the consistentcost of the 3D vision devices and the consequent increase inhardware complexity and portability reduction. The cameraposition and the multimedia glasses are crucial, they need tobe adapted to each single user since, as discussed in [6], thevisual information from the first person perspective plays animportant role in establishing the location of the perceivedbody relative to environmental landmarks and in defining theorigin of the body centered reference frame.

IV. TWO EXPERIENCES OF REMOTE TOUCH

Two experiences of remote touch are presented. One of theaims of the RemoTouch project is to improve the quality ofcommunication. Touch is important to communicate feelingsand emotions. The aim of the experiences was to test thetactile communication and a possible application where touchcould be useful. For these two reasons a family contest anda music training activity were selected. The first experienceshows RemoTouch performing the task of touching a baby.Think of a family where the mother with her child areat home and the father is far away. The mother, in thiscase, wears the instrumented glove to record and transmitthe interaction forces while touching her child. The motheralso uses a head mounted camera to feed back visual datato the remote husband. The father plays the role of theuser and, through the tactile display and the head mounteddisplay, perceives the tactile and visual experience recordedby his wife while touching their child (Fig. 5). Preliminarytests show that the realism of this remote experience largelyimprove with the tactile feedback.

The context chosen for the remote experience is veryimportant: having a tactile interaction with their own child in-volves intense emotions thus increasing the sense of presenceeven if important components like the kinesthetic feedbackare missed in the communication.

The second experience deals with playing and listening tomusic. This is another very involving experience in everyday

Fig. 5. An experience of remote touch in a context where the motherremotely touches the child and transmits the tactile interaction to the fatherwho is able to perceive the remote touch experience.

Fig. 6. The second experience of remote touch where a remote piano playertransmits the tactile interaction to another player who is able to perceivethe remote touch experience.

life. In the experiment, the human avatar plays the piano andrecords the tactile experience, with the instrumented glove,to be played back to the remote user wearing the tactiledisplays (Fig. 6). For audio feedback, a microphone is usedto capture the audio signal and in-ear headphones have beenused for play back.

To perform these experiences a TCP/IP LAN connectionhas been used. TCP protocol is optimized for accuratedelivery rather than timely delivery, and therefore, TCPsometimes incurs in delays while waiting for out-of-ordermessages or retransmissions of lost messages. Several tech-niques were used to synchronize the video flow and thetactile feedback recorded by the instrumented glove. Thecurrent implementation of RemoTouch uses synchronisingsignals but we plan to add the support, in the next future,for the Real-time Transport Protocol (RTP) running over theUser Datagram Protocol (UDP) that is the now-a-days usedprotocol for real-time applications such as Voice Over IP.

The video of the two experiences can be found athttp://remotouch.dii.unisi.it.

V. DISCUSSION

Transmitting tactile information can be used to enhancecommunication between humans. In particular we can com-municate different emotions which cannot be easily trans-mitted through commonly used communication modalitieslike audio and video. When RemoTouch is used as a tactile

678

communication system, the data-flow consists of recordingtactile interaction forces with the instrumented glove to betransmitted to the tactile display along with video and audiosignals.

Another very interesting scenario for RemoTouch is to usethis system not as a real-time communication system but asa system to record the many visual-tactile experiences to beplayed back not in real-time but at a different time. Theseexperiences can be shared with other people as it usuallyhappens for images or music in social networks.

A further interesting aspect is to build a database of tactileexperiences recorded through RemoTouch. The challengehere is to store tactile data to get information retrieval usingsimple search engines. We are planning to develop a tactiledatabase where users can submit their recorded experiencesusing tagging techniques as for audio and video files, e.g.the ID3 tag system for mp3 files.

VI. CONCLUSIONS

In this paper we presented RemoTouch, a system toperform remote touch experiences. Two example have beenpresented dealing with particularly emotional contexts.

An important feature of RemoTouch is that the involvedtechnology is low cost and with low energy consumption.Of course this is a consequence of choosing a human avatarwhich is not a secondary point, but the main point of ourproject. This work is still in its infancy and we believe thatmany other applications can be found for RemoTouch. Con-sider rehabilitation where RemoTouch can be used to easilycontrol the force distribution among fingers in hand graspingtasks for rehabilitation. Substituting the robotic avatar withthe human avatar means lower costs, a simplification of thesystem and a more simple approach for the user. On the otherhand, the kinesthetic feedback is missed, so a vision systemis used for any position feedback of the human avatar. Theweak point of the proposed remote touch architecture is thatthe performance, especially in terms of realism, of the overallsystem has been sacrificed in exchange of the portability andwearability of the devices.

We believe that new interaction paradigms based on tactilecommunication can be developed around RemoTouch forapplications in both telepresence and teleoperation. Regard-ing teleoperation, note that RemoTouch deals only with theremote perception while the action needed to control theremote avatar is an open and very interesting issue and willbe the object of future investigations.

VII. ACKNOWLEDGMENTS

The RemoTouch idea has been selected by the MinisteroItaliano dell’Innovazione as one of the Italian innovationswithin the “Italia degli Innovatori” initiative, organized bythe Italian Government for the Shangai 2010 Expo. Theauthors wish to thank the student Leonardo Meli for his helpin realizing the experiments.

This work is supported by the European Commissionunder CP grant no. 248587, “THE Hand Embodied”,

within the FP7-ICT-2009-4-2-1 program “Cognitive Sys-tems and Robotics”and CP grant no. 231143, “ExperimentHANDS.DVI” within the program “Echord”.

REFERENCES

[1] M. Bergamasco, B. Allotta, L. Bosio, L. Ferretti, G. Parrini, G.M.Prisco, F. Salsedo, and G. Sartini. An arm exoskeleton system forteleoperation and virtual environments applications. In Robotics andAutomation, 1994. Proceedings., 1994 IEEE International Conferenceon, pages 1449 –1454 vol.2, may 1994.

[2] P. Kremer, T. Wimbock, J. Artigas, S. Schatzle, K. Johl, F. Schmidt,C. Preusche, and G. Hirzinger. Multimodal telepresent control ofDLR’s rollin’ JUSTIN. In ICRA’09: Proceedings of the 2009 IEEEinternational conference on Robotics and Automation, pages 3096–3097, Piscataway, NJ, USA, 2009. IEEE Press.

[3] K. Minamizawa, S. Fukamachi, H. Kajimoto, N. Kawakami, andS. Tachi. Gravity grabber: wearable haptic display to present virtualmass sensation. In ACM SIGGRAPH 2007 emerging technologies,page 8. ACM, 2007.

[4] K. Minamizawa, D. Prattichizzo, and S. Tachi. Simplified design ofhaptic display by extending one-point kinesthetic feedback to multipointtactile feedback. IEEE Haptic Symposium, 2010.

[5] M. Botvinick and J. Cohen. Rubber hands’ feel’touch that eyes see.Nature, 391(6669):756–756, 1998.

[6] V.I. Petkova and H.H. Ehrsson. If I were you: perceptual illusion ofbody swapping. PLoS One, 3(12), 2008.

679