intech-markerless kinect based hand tracking for robot teleoperation

International Journal of Advanced Robotic Systems Markerless Kinect-Based Hand Tracking for Robot Teleoperation

Regular Paper

Guanglong Du, Ping Zhang*, Jianhua Mai and Zeling Li Department of Computer Science, South China University of Technology, P.R. China * Corresponding author E-mail: [email protected]

Received 9 Apr 2012; Accepted 23 May 2012 DOI: 10.5772/50093 2012 Du et al.; licensee InTech. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract This paper presents a realtime remote robotteleoperation method using markerless Kinectbasedhand tracking. Using this tracking algorithm, thepositions of index finger and thumb in 3D can beestimatedbyprocessingdepth images fromKinect.Thehand pose is used as a model to specify the pose of arealtime remote robots endeffector. This methodprovides away to send awhole task to a remote robotinstead of sending limited motion commands likegesturebased approaches and this method has beentestedinpickandplacetasks.Keywordsrobotmanipulator,markerless,Kinect1.Introduction

If a task is too complex for an autonomous robot tocomplete, thenhuman intelligence isrequired tomakeadecision and control the robot, especially when it is inunstructureddynamicenvironments.Furthermore,whenthe robot is in a dangerous environment, robotteleoperation may be necessary. Some humanrobotinterfaces (Yussof et al. [1]; Mitsantisuk et al. [2]) likejoysticks,dials and robot replicas,havebeen commonly

used, but these contacting mechanical devices requireunnatural hand and arm motions to complete ateleoperationtask.

Another way to communicate complex motions to aremote robot, which is more natural, is to track theoperatorhandarmmotionwhichisusedtocompletetherequired task using contacting electromagnetic trackingsensors, inertial sensors and gloves instrumented withanglesensors(Hircheetal.[3];Villaverdeetal.[4];Wangetal.[5]).However,thesecontactingdevicesmayhindernaturalhumanlimbmotions.

Becausevisionbasedtechniquesarenoncontactandlesshindrancetohandarmmotions,theyhavealsobeenused.Visionbased methods always use physical markersplaced on the anatomical body part (Kofman et al. [6];LathuilireandHerv[7];GuangLongDuetal.[8]).Thereare a lot of applications (Peer et al. [9] Borghese andRigiroli[10];Kofmanetal.[6])usingmarkerbasedhumanmotion tracking, however, because body markers mayhinderthemotionforhighlydexteroustasksandmaygetoccluded, this markerbased tracking is not alwayspractical. Thus, amarkerless approach seems better formanyapplications.

1Guanglong Du, Ping Zhang, Jianhua Mai and Zeling Li: Markerless Kinect-Based Hand Tracking for Robot Teleoperation

www.intechopen.com

ARTICLE

www.intechopen.com Int J Adv Robotic Sy, 2012, Vol. 9, 36:2012

Compared to imagebased trackingwhichusesmarkers,markerless isnot only less invasive, but also eliminatesproblemsofmarkerocclusion and identification (Verma[11]).Thus,markerlesstrackingmaybeabetterapproachfor remote robot teleoperation. However, existingmarkerless humanlimb tracking techniques have somanylimitationsthattheymaybedifficulttouseinrobotteleoperation applications. A lot of existing markerlesstracking techniques capture images and compute themotion later like a postprocess (Goncalves et al. [12];Kakadiarisetal.[13];Uedaetal.[14]RosalesandScarloff[15]). The markerless tracking has to performsimultaneouslyinrealtimeforremoterobotteleoperationwhen controlling continuous robotmotion.Toallow thehumanoperatortoperformhandarmmotionsforataskin a naturalwaywithout interruption, theposition andorientation of the hand and arm should be providedimmediately.Manytechniquescanonlyprovide2Dimageinformationof thehumanmotion (Koaraetal. [16];MacCormickandIsard[17])andthetrackingmethodscannotbe extended for accurate 3D jointposition data. Anendeffector of a remote robot would require the 3Dposition and orientation information of the operatorslimbjointcentreswithrespecttoafixedreferencesystem,and identifying human body parts in differentorientations has always been a significant challenge(Kakadiaris et al. [13];Goncalves et al.[12]; Triesch andMalsburg[18]).

For robot teleoperation, there is limited research onmarkerlesshumantracking.Mosttechniqueshavetriedtouse a humanrobot interface based on handgesturerecognitiontocontrolrobotmotion(Fongetal.[19];Huetal. [20];Moy [21]).Coquin et al. and Ionescu et al. [22]developedmarkerlesshandgesturerecognitionmethodswhichcanbeusedformobilerobotcontrolwhereonlyafew different commands are enoughlikego,stop,left,rightandsoon.However,forobject manipulation in 3D space, it is not possible toachievenaturalcontrolandflexiblerobotmotionusing

gesturesonly.Ifahumanoperatorwantstousegestures,he/sheneedstothinkofthoselimitedseparatecommandsthatthehumanrobotinterfacecanunderstandlikemoveup, down, forward and so on. A better way ofhumanrobotinteractionwouldbetopermittheoperatortofocusonthecomplexglobaltaskasahumannaturallydoeswhengraspingandmanipulatingobjectsin3Dspaceinsteadofthinkingaboutwhattypeofhandmotionsarerequired.Toachieve thisgoal,amethod thatallows theoperatortocompletethetaskusingthehandarmmotionsnaturally, providing the robot with information of thehandarm motion in realtime like the hand and armanatomicalpositionandorientation(Kofmanetal.[23]),isneeded.However,toachievetheinitialization,thehumanoperatormustassumeasimpleposturewithanunclothedarm in frontof adarkbackground,handplacedhigherthantheshoulder.Itisnotpossibletogetapreciseresultwith a complex background. In addition, the humanoperatorwouldfindithardtoworkincoldweatherasthearmisunclothed.Itisalsolimitedbecauseofthelightingeffect,i.e.,itisdifficulttousewhenitistoobrightortoodark.

This paper presents a method of remote robotteleoperation using markerless Kinectbased 3D handtracking of the human operator (Figure 1). MarkerlessKinectbased hand tracking is used to acquire 3Danatomicalpositionandorientation,andthenitsendsthedatatotherobotmanipulatorbyahumanrobotinterfacetoenabletherobotendeffectortocopytheoperatorhandmotion in realtime. This natural way to communicatewith the robot allows the operator to focus on the taskinsteadofthinkingintermsoflimitedseparatecommandsthat the humanrobot interface can understand likegesturebased approaches. Using the noninvasiveKinectbased tracking avoids the problem that physicalsensors,cablesandothercontactinginterfacesmayhindernaturalmotionsand that theremaybemarkerocclusionandidentificationwhenusingmarkerbasedapproaches.

Figure1.NoninvasiverobotteleoperationsystembasedontheKinect

2 Int J Adv Robotic Sy, 2012, Vol. 9, 36:2012 www.intechopen.com

2.HumanhandtrackingandpositioningsystemHumanhand trackingandpositioning is carriedoutbycontinuouslyprocessingRGBimagesanddepthimagesofan operator who is performing the hand motion tocompletearobotmanipulationtask.TheRGBimagesanddepthimagesarecapturedbytheKinectwhichisfixedinthefrontoftheoperator.

The Kinect has three autofocus cameras: two infraredcamerasoptimizedfordepthdetectionandonestandardvisualspectrumcamerausedforvisualrecognition.

2.1Kinectcoordinatesystem

InFigure2,anoperatorstandsinfrontoftheKinectandcontrolsa robot.We candefine theKinectcoordinateasshowninFigure2:axisXisupturned,axisYisrightwardandaxisZisvertical.TheKinectcancapturethedepthofanyobjects in itsworkspace. InFigure2we can see theindexfingertip(I),thethumbtip(T)andapartofthehandbetween the thumb and the index finger(B). EverydistancebetweentheKinectandI,B,TorUisdifferent.IandT are closest to theKinect and theupper armU isfurthest. The 3D position of B is used to control thepositionof the robot endeffector.The I,TandBof theoperatorareused to control theorientationof the robotendeffector.

Figure2.depthofobjects.K:Kinect;I:indexfingertip;T:thumbtip,B:apartofthehandbetweenthethumbandtheindexfinger;U:upperarm.

2.2Imagecaptureandsegmentationofhand

Inordertocatchthehandmotionusedforcontrollingtherobotmanipulator,weneedtoseparatethehandfromthedepth image. The arm is segmented from the body bythresholdingtherawdepthimage.

(a) (b) (c)

(d) (e) (f)Figure3.Segmentationofhandanddeterminationofthumbandindexfingertippositions


www.intechopen.com

A depth image D(i,j), shown in Figure 3a, records thedepthofall thepixelsofRGB imagewhich is shown inFigure3b.Assume that thedistancebetween thehumanoperatorandtheKinectisnotmorethanT(m)andthereisno other object between the human operator and theKinect.ForalliandjindepthimageD(i,j),thebodyimageCb(i,j)isthendividedas:

b , { , | ( , ) T ; ( , ) ;1,2,..., ; 1,2,..., }

C i j d i j d i j d i j Di n j m

(1)

Whered(i,j)isthepixelofdepthimageD;nisthewidthofDandmistheheightofD.When thehumanoperatorheldout thehand to controltherobotmanipulator,thearmiscloserthanthebody,wecan first compute the mean value M of all the body,includingthearm:

b1 1

( , )n m

i jC i j

M mn

(2)

ThenwecandividethearmregionA(i,j)asfollows:

b{ ( , ) | ( , ) & ( , ) }A d i j d i j C d i j M (3)ThearmregionAisshowninFigure3c.2.3DeterminationofthumbandindexfingertippositionsThe positions of thumb tip and indexfinger tip aredeterminedbyan image thatcontains thearm.ThearmregionA3d(x,y,z)canbereconstructedfromA(i,j)asshowninFigure3d.

For all 2Dpoints (i,j) in theA(i,j), the 3Dpoints canbecalculatedby:

A3d(x,y,z)=[i,j,d(i,j)] (4)

Thenproject the3DpointsA3d(x,y,z) to the faceYOZasshowninFigure3e.

AYOZ(y,z)=A3d(0,y,z)=[j,d(i,j)] (5)

Definetheminimizeprojectfunction f :

YOZ1,2,...( ) min ( ( , ))z mf y A y z (6)

Determinetheonemaximum(aty=y1)andtwominimum(aty=y2,y=y3)fortheminimizeprojectfunction f .Thenthe3DpointofIcanbereconstructedby:

'

'3d

1( , y2, (y2))

( , , ) y2(y2)

n

xx A x f

I x y z yz f

(7)

'

'3d

1( , y3, (y3))

( , , ) y3(y3)

n

xx A x f

T x y z yz f

(8)

'

'3d

1( , y1, (y1))

( , , ) y1(y1)

n

xx A x f

B x y z yz f

(9)

3.Positionmodel

Toavoid largescalemotionwhen theoperatorperformsmanipulation,we need to confine theworking space ofthe operator to a relatively small space. However, theworkingspaceoftheremoterobotshouldnotbelimited.Thismeansthemappingfromarelativelysmallplacetoanunconfined largespace isnecessary.Becauseofdirectmappingfromsmallspacetoalargerspace,themappingwilllosesomeprecision.Toavoidthisproblem,weadjustadifferentialpositioningmethodinthissituation.

Similartothemouseandthekeyboard,thepositionofthehandcanbecalculatedbytheincrementalmethod.Fromsection2,the3DpositionofB,TandIarecalculatedintheworldcoordinate,shownasFigure3.Theinitialpositionand orientation of the robot endeffector in the startingpointarealsostoredas the robot referencepositionandorientation, respectively. The position of the robottoolcontrol point on the endeffector is controlled bypositionBofthehumanoperator.

Define the 3D position of the I,T andB in the currentframe as I(x,y,z), T(x,y,z) and B(x,y,z), respectively.Define the length of the line segment jointing theindexfinger tip (I) and thumbtip (T) on the operatorhandasL(shownasFigure5),

L=||T(x,y,z)I(x,y,z)|| (10)


The 3D position of B in the last frame isB(x,y,z). TheendeffectorreferencepositioninthelastframeisP(x,y,z)andthenewendeffectorreferencepositionisupdated:

' '' * u

' '' uP P L L

P P L

(11)

Whereuisathresholdthatdetermineswhethertherobotkeepsmovingorpauses.WhenL=0,itmeanstheoperatorstopstocontroltherobot,shownasFigure4.

Figure4.Handpose

Figure5.Positioningmodel

Because is an adjustable parameter, theoretically thespacemanipulatedbytheoperatorisaninfinitespaceandwe can obtain coarsecontrol and finecontrol throughadjustmenttothevalueof .

4.OrientationModel

AsdescribedinFigure,theorientationoftheendeffectorisinaccordancewiththeorientationformedbythumbtip,indexfingertipand Thepartbetweenthethumbandtheindexfinger

Figure6.Orientationmodel

Theorientationoftheendeffectoriscalculatedusingthe3D positions of the I, T and B. In the mapping of the

operatorhandtotherobottoolcoordinatesystem,thelinefromBtothemidpointMofthelinesegment,whichjointstheindexfingertip(I)andthumbtip(T)ontheoperatorhand, ismapped to therobottoolaxisX (Figure4),andtheXYplaneisdefinedbyB,IandT.

Thismeansthatifweonlygetthetransformationmatrixfrom the coordinate system of the console to thecoordinate systemof theoperatorshand,wecanobtainthe transformation matrix from the base coordinatesystemtotheendeffector.Thedetailsofthederivationoftheorientationmatrixaregivenbelow:

Assuming the origin of the operators hand coordinatesystemisidenticaltotheoneinconsolecoordinatesystemandthetransformationmatrixisa3*3matrixM.LetPointA in the operators hand coordinate system transfer toPointAintheconsolecoordinatesystem,wehave:

A =MA (12)

In hand tracking and positioning, the unit vector1 2 3[ , , ]x x x 1 2 3[ , , ]y y y , 1 2 3[ , , ]z z z indirectionX,Y,

ZcanbemeasuredbyKinectyielding:

11 12 13 1

21 22 23 2

31 32 33 3

100

m m m xm m m xm m m x

(13)

11 12 13 1

21 22 23 2

31 32 33 3

010

m m m ym m m ym m m y

(14)

11 12 13 1

21 22 23 2

31 32 33 3

001

m m m zm m m zm m m z

(15)

Through(13),(14),(15),wecanget:

11 12 13 1 1 1

21 22 23 2 2 2

31 32 33 3 3 3

m m m x y zm m m x y zm m m x y z

(16)

As stated before, the transformation matrix from theconsole coordinate system to the operators handcoordinate system is identical to the one from the basecoordinatesystem to theendeffectorcoordinatesystem,andthetranslationrelationshipbetweentheendeffectorandthebasecoordinatesystemisalreadyyieldinginthe


www.intechopen.com

positioning model, so the transformation matrix oforientationis:

1 1 1 1

2 2 2 2

3 3 3 30 0 0 1

x y z px y z pM x y z p

(17)

Noticethatthe[ 1 2 3, ,p p p ]isthetranslationmatrixfromthebasecoordinatesystemtotheendeffector.

5.VirtualRobotManipulationSystem

We use a six degreeoffreedom industrial robot toperformthisexperiment,asshowninFigure7.Thetaskistograb the targetobjectwhich is in the robotsworkingspaceandthenplacetheobjectatthedestination.

Therearetwoworkingmodesfortherobot.Thefirstoneis to calculate the angle of every joint by reversingkinematic according to the position of the endeffector.After joints execute the entire requested angles, theendeffectorof thevirtual robot reaches thedestination.Thismode is suitable for a situationwhere no obstacleoccursintheworkspaceofthevirtualrobot.However,thesecond mode is suitable for the situation where anobstacleshowsupinthevirtualrobotsworkingspace.Inthismodethevirtualrobothastomovealongasafepath,whichensures thevirtualrobotwillnotcollidewith theobstacle.

In DH representation, Ai presents the homogeneouscoordinatetransformationmatrixfromcoordinatei1toi:

cos sin cos sin sin cossin cos cos cos sin sin

0 sin cos0 0 0 1

i i i i i i i

i i i i i i ii

i i i

llA r

(18)

Fora robotwith six joints, thehomogeneous coordinatetransformationmatrixfromthebasecoordinatesystemtotheendeffectorscoordinatesystemisdefinedas:

0 0 0 06 6 6 6

6 1 2 6... [ ]0 0 0 1n s a pT A A A

(19)

Where 06n is the row vector of the endeffector, 06s isthe pitch vector, 06a is the yaw vector and 06p is thepositionvector.Using(17),(19),wehave:

6T M (20)

Through (8) we can have the angle of six joints:1 2 6( , ,..., ) .

(a)

(b)

Figure7.Sixaxisrobotmanipulatorusedattheremoterobotsite

6.Experiments

Weevaluatedthealgorithmonourrobotplatform.Whentesting it, we built up an experimental environment ofteleoperation.Webuilt a setof emulation environmentsforthetechnicalrobotandasetofvirtualrealitysystemsbasedonvideoatthelocalsite.Theremotesiteistherealrobot in the working environment. In this experiment,considering the real environment of teleoperation, welimit bandwidth to 30kB/s and the delay time isapproximately3seconds.

To evaluate the Kinectbased teleoperation algorithmdescribed in this paper, we use C++ to develop aKinectbasedhumanrobotinterfacesystem(Figure8)andthis system is used for the teleoperation of a sixaxistechnical robot.Thisexperimental system includes threemodules:

1) Use the human hand tracking and positioningsystemtogetthehandimages,andthencalculatethe3DpositionsofT (the thumb tip), I (the indexfingertip) andB (thepartof thehandbetween the thumbandtheindexfinger). 2)Virtualrobotmanipulationsystemdrivesthevirtualrobotbasedon the joint angleswhich are calculatedthroughreversekinematic.Ifthecommandsaresafe,


theywillbe transmitted to the remotesite tocontroltherealrobot.3)Theremotesitewilltransmitthevideotothelocalsite and the video fuse system displays the virtualenvironment and the real environment. Then theedgesofthevirtualrobotcoverthevideoframewhichistransmittedfromtheremotesite.

In the experiment, the operator placed his hand in theworkspacetocontrolthevirtualrobot.Theorientationofthevirtualrobotsendeffectorcoincidedwiththehumanhand.Thepositionofthevirtualrobotsendeffectorwasadjusted bymoving the human hand through differentfacesofthedirectionspace,asshowninFigure3.

Asshown inFigure3, theway theoperatorcontrols therobot is natural and intuitive. Because of using anincrementalmethodwhichissimilartokeyboardcontrol,the operator is not required to make large scalemovementstocontroltherobot.

7.Result After reconstructing and controlling robots by reversekinematics, the precision of manipulation will decreasebecause of the transformation of the coordinate systemandsolvingoftheequationsset.

Figure showsthepositionandorientationoftherobotsendeffectorandtheoperatorshandduringteleoperationexperiments.Thedashedlinerepresentstheendeffectorspath.Thesolidlinewithgreensquaresrepresentsthepathoftheoperatorshand.Thevirtualrobotwasmanipulatedto grab the ball which is placed on a square. The datageneratedbythisexperimenthasshownthatthepositionerrors ranged from 13 to +13 mm and the orientationerrors ranged from 2 to2degree.Figure (c,d,e)showstheX,Y,Zdisplacementsof the endeffectorandhand,whiletherotationsofthemareshowninFigure (f,g,h).

Figure8.Noninvasivevisionbasedteleoperationsystem

8.Discussion

In the remote unstructured environment of the robotteleoperation, we assume that all the remote robot sitecomponents, including robotic arm, robot controller,camerasonendeffectorsandsomeothercameras,canbeinstalled on a mobile platform and enter thoseunstructured environments. The method shown here isproved on grabbing objects, picking up objects andpositioningaccuratelyduringgrabbingobjectsinthefineadjustment controlling mode. One advantage of thissystem is that it includes the operator into thedecisioncontrolloop.Itallowsarobottograb,moveandplacethe

objectwithoutanypriorknowledgelikestartinglocationand even destination location. There are some similartasks which require decision making when picking upobjectsandtargetsfrommultipleobjectslikepackingandcleaningsomeobjectswhichmaycontainsomedangerousitems. It is expected that this system can be used toachievethosemorecomplexposeswhenthe jointsoftherobotarelimited.Theholetaskshowshowtodeterminethe position of an extruded body and a target holerandomly.Assemblyanddisassemblymay includemorelimitedholetasks.Wemayneedanappropriategrabhook,biggerholeandgrooveunlessthissystem includesforcefeedback.


www.intechopen.com

(a) (b)

(c) (d)

(e) (f)

(g) (h)

Figure9.AnalysisoftheexperimentComparedwith the automatic capture (Kofman et al. [6]),this algorithmusesmanualpositioning.Consideringhandtremor,thisalgorithmincludesacoarseadjustmentandfineadjustmentfunction.Whenguidingtherobot,wecanusethecoarse adjustment to move the robot close to the targetquickly. When grabbing the target, we can use the fineadjustmenttopositiontherobotaccurately.Thatcanensure

thesafetyand theefficiencyof the teleoperation,andsolvetheproblemofinaccuracycausedbymanualoperation.This paper contributes to the guiding teleoperationsystem based on noncontact measurement. By usingtrackingbasedonKinect, robot teleoperationallows theoperator to control the robot in a more natural way.


Generally speaking, using the same hand motion thatnaturally would be used in a task can accomplish theoperation taskandwhat ismore, this trackingbasedonKinect is noncontact. Thus, compared with contactingelectromagneticdevices,devicesbasedonsensoranddatagloveswhichareusednormally,noncontactdevicesmaycause lesshindrance to thenaturalhumanlimbmotion.The method proposed here will allow the operator tofocusonthetaskinsteadofthinkingofhowtodecomposethecommandsintosomesimplecommandsthatthevoicerecognition teleoperation system can understand. ThismethodismorenaturalandintuitivethantheoperationinKofmanetal. [23].Thesystemcanbeused immediatelywithoutanyinitializationandthisnoncontactingcontrolsystemcanbeusedoutdoors.Becausethisalgorithmusesinfrareddistancemeasurementtogetarminformation,itcanignorethelightingeffectanddoesnotneedtoextractthe 3D coordinates by accurate image processing. Thatallowsthesystemtobeusedinmoresevereenvironments,like when it is too bright or too dark. In addition, thealgorithm of [23] reference 1 needs a bare hand torecognizethecolourofskin,otherwise,itcannotbeusedtoextract thehanddata.Comparedwith thatalgorithm,this algorithm does not require a bare hand and theoperatorcanweargloveswhenusingthesysteminacoldoutdoorworkingenvironment.Thatenlargesthefieldofapplicationofthesystem.

9.ConclusionAmethod of humanrobot interaction usingmarkerlessKinectbased tracking of the human hand for arobotmanipulatorteleoperation hasbeenpresented.Viatrackingofthethumbtip,indexfingertipandthepartofthe hand between the thumb and the index finger inrealtime,the3Dpositionandorientationofthehandarecomputed accurately and the robotmanipulator can becontrolledbyhandtoperformthetaskofpickingupandplacing.Tocompletethecomplextasks,multiKinectwillbeusedtoworktogetherinfuturework.10.References[1] YussofH,CapiG,NasuY,YamanoM,OhkaM.A

CORBABased Control Architecture for RealTimeTeleoperationTasks in aDevelopmentalHumanoidRobot. International Journal of Advanced RoboticSystems,8(2):2948,2011.

[2] MitsantisukC,KatsuraS,OhishiK.ForceControlofHumanRobot Interaction Using Twin DirectDriveMotor SystemBased onModal SpaceDesign. IEEETransactions on Industrial Electronics,57(4):13381392,2010.

[3] Hirche S, Buss M. HumanOriented Control forHaptic Teleoperation. Proceedings of the IEEE,100(3):623647,2012.

[4] Villaverde AF, Raimundez C, Barreiro A. PassiveInternetbasedCraneTeleoperationwithHapticAids.International Journal of Control Automation andSystems,10(1):7887,2012.

[5] WangZ,GiannopoulosE,SlaterM,PeerA,BussM.Handshake: Realistic Human Robot Interaction inHaptic Enhanced Virtual Reality.PresenceTeleoperators and Virtual Environments,20(4):371392,2011.

[6] Kofman, Jonathan, XianghaiWu, TimothyLuu, andSiddharth Verma. 2005. Teleoperation of a robotmanipulator using a visionbased humanrobotinterface.IEEETransactionsonIndustrialElectronics52(5):12061219.

[7] Lathuilire, Fabienne and Herv JeanYves. 2000.Visual hand posture tracking in a gripper guidingapplication.Proc.Int.Conf.RoboticsandAutomation(ICRA)16881694.

[8] GuanglongDu,PingZhang,LiyingYang,Yanbin Su.Robotteleoperationusingavisionbasedmanipulationmethod. Audio Language and Image Processing(ICALIP),2010InternationalConference.2010,945949.

[9] Peer A, Pongrac H, Buss M. Influence of VariedHumanMovementControlonTaskPerformanceandFeeling ofTelepresence.PresenceTeleoperators andVirtualEnvironments,19(5):463481,2010.

[10] Borghese, N. Alberto and Rigiroli Paolo. 2002.Tracking densely moving markers. IEEE FirstInternationalSymposiumon3DDataProcessingandTransmission,PadovaGiugno,682685.

[11] VermaSiddharth.2004.Visionbasedmarkerless3Dhumanarm tracking.M.A.Sc.Thesis,DepartmentofMechanical Engineering, University of Ottawa,Ottawa,Canada.

[12] Goncalves, Luis, DiBernardo Enrico, Ursella EnricoandPeronaPietro. 1995.Monocular tracking of thehumanarm in3D.Proceedingsof IEEE InternationalConferenceonComputerVision,ICCV95,764770.

[13] Kakadiaris, Ioannis A, Metaxas Dimitri and BajcsyRuzena.1994a.Activepartdecomposition,shapeandmotion estimation of articulated objects: aphysicsbased approach. Proceeding of IEEEComputer Society Conference on Computer VisionandPatternRecognition,980984.

[14] UedaEtsuko,YoshioMatsumoto,MasakazuImaiandTsukasaOgasawara.2001.Handposeestimation forvisionbasedhumaninterface.10thIEEEInternationalWorkshop on Robot and Human Communication(ROMAN2001)473478.

[15] RosalesRomerandSclaroffStan.2000.Inferringbodypose without tracking body parts. Proceedings ofIEEE Conference on Computer Vision and PatternRecognition2:721727.

[16] Koara, Kengo, Atsushi Nishikawa and FumioMiyazaki. 2001. Contour based hierarchical partdecomposition method for human body motion


www.intechopen.com

analysis from video sequence. In Human FriendlyMechatronics.Ed.ByE.Arai,T.Arai,andM.Takano,ElsevierScience.

[17] Mac Cormick, John and Isard, Michael. 2000.Partitioned sampling, articulated objects, andinterfacequality hand tracking. Proceeding ofEuropeanConferenceonComputerVision2:319.

[18] Triesch, Jochen and von der Malsburg, Christoph.2002.Asystemforpersonindependenthandposturerecognition against complex backgrounds. IEEETransactions on Pattern Analysis and MachineIntelligence23(12):14491453.

[19] Fong,Terrence,ContiFrancois,GrangeSebastienandBaur Charles. 2000. Novel interfaces for remotedriving: gesture, haptic and PDA. SPIETelemanipulator & Telepresence Technologies VII4195:300311.

[20] Hu,Chao,MaxQingHuMeng,PeterXiaopingLiu,andXiangWang.2003.Visualgesturerecognitionforhumanmachine interface of robot teleoperation.IEEE=RSJ International Conference on IntelligentRobotsandSystems,USA,15601565.

[21] Moy,MilynC.1999.GesturebasedinteractionwithaPetRobot.Proceedingsof6thNationalConferenceonArtificial Intelligence and 11th Conference onInnovative Applications of Artificial Intelligence,USA,628633.

[22] Ionescu,Bogdan,CoquinDidier,LambertPatrickandBuzuloiu Vasile. 2005. Dynamic and gesturerecognitionusingtheskeletonofthehand.JournalonAppliedSignalProcessing13:21012109.

[23] Kofman Jonathan, Verma Siddharth, Wu Xianghai.RobotManipulator Teleoperation by MarkerlessVisionBased HandArm Tracking. InternationalJournalofOptomechatronics,1:331357,2007.


intech-markerless kinect based hand tracking for robot teleoperation

Documents