Data-driven Finger Motion Synthesis for Gesturing Characters

Sophie Jörg∗

Carnegie Mellon UniversityClemson University

Jessica Hodgins†

Carnegie Mellon UniversityDisney Research, Pittsburgh

Alla Safonova‡

Disney Research, Pittsburgh

(a) (b) (c) (d)

Figure 1: Animations with synthesized finger motions: (a) ok gesture, (b)-(c) extracts from a conversation, (d) attention gesture.

Abstract

Capturing the body movements of actors to create animations formovies, games, and VR applications has become standard practice,but finger motions are usually added manually as a tedious post-processing step. In this paper, we present a surprisingly simplemethod to automate this step for gesturing and conversing char-acters. In a controlled environment, we carefully captured andpost-processed finger and body motions from multiple actors. Toaugment the body motions of virtual characters with plausible anddetailed finger movements, our method selects finger motion seg-ments from the resulting database taking into account the similarityof the arm motions and the smoothness of consecutive finger mo-tions. We investigate which parts of the arm motion best discrimi-nate gestures with leave-one-out cross-validation and use the resultas a metric to select appropriate finger motions. Our approach pro-vides good results for a number of examples with different gesturetypes and is validated in a perceptual experiment.

CR Categories: I.3.7 [Computer Graphics]: Three-DimensionalGraphics and Realism—Animation;

Keywords: finger motions, human animation, gesture synthesis,hand motions, virtual characters

Links: DL PDF

∗e-mail: sjoerg@clemson.edu†e-mail: jkh@cs.cmu.edu‡e-mail: alla.safonova5@gmail.com

1 Introduction

Hand and finger motions are omnipresent in our daily life. We useour hands to interact with objects and with each other. When com-municating, we use them to punctuate our speech, point to an ob-ject of interest, signal our opinion, and convey emotions. Com-munication – and therefore hand and finger motions – also playsan increasingly important role in digital applications. Examples ofsuch applications include virtual worlds, such as Second Life, dig-ital games, such as L.A. Noire, or any type of teaching, training,or advice application that uses embodied conversational agents. Ifwe want to create believable and compelling virtual characters, wetherefore need to generate convincing hand motions.

Motion capture has become a widely used technique for animatingrealistic and human-like characters for movies or games. However,the elaborate motions of the fingers with their high number of de-grees of freedom and small size are still not easy to capture. As aconsequence, hands are rarely captured in their full complexity. Ingeneral, two or three markers on fingertips (e.g., on the thumb, in-dex, and pinky) and a few on the back of the hand are used as a ref-erence [Kitagawa and Windsor 2008] and the fingers are keyframedmanually, which is a tedious, labor-intensive process.

In this paper, we propose a method to automatically add plausiblefinger motions to body motions (Figure 1). Our algorithm uses apreviously recorded database of body and finger motions to gener-ate finger movements that match an input body motion (Figure 2).We locate suitable finger motions in the database using a metricbased on the similarity of the arm movements and the smooth-ness of the reconstructed finger motions. Our key observation isthat plausible finger motions can be inferred from the wrist motion.Our approach preserves the naturalness and subtlety of motion cap-tured movements, without requiring time-consuming manual post-processing. As our method is intended as a post-processing step, itdoes not impact the motion capture session, saving valuable produc-tion time. We apply our algorithm to gesturing and conversing char-acters. It is not intended for object manipulations, such as grasping,where the fingers must be precisely positioned. We demonstrate ourapproach with several example databases, consisting of gestures,casual conversations, debates, and giving directions.

Our main contribution is the development of a data-driven methodto automatically add realistic and detailed finger motions to thebody motions of conversational characters. We explore the valid-ity of several parameters as predictors for the similarity of finger

http://doi.acm.org/10.1145/2366145.2366200http://portal.acm.org/ft_gateway.cfm?id=2366200&type=pdf

Figure 2: Our method automatically synthesizes finger movementsfor an input body motion based on a database of motions.

motions and find that a combination of wrist rotation and wrist po-sition leads to the best results. The rest of the paper is organized asfollows. After discussing related work in Section 2, we explain ourmethod in Section 3. In Section 4, we describe how we tested mul-tiple finger motion predictors with a leave-one-out cross-validationapplied to a set of controlled gestures. Our results, including ex-amples with different gesture types and a perceptual study, are pre-sented in Section 5. We conclude by discussing the advantages andlimitations of our approach and considering future work.

2 Related Work

Several approaches have been suggested to simplify the animationof finger motions for virtual characters. They can be divided intomethods to capture finger movements and algorithms to computethem. The latter consist primarily of data- and physics-based ap-proaches.

Capturing finger motions is a challenge. The high number of jointsin a very small space requires many small markers. Contrary tothe face, where the markers are roughly located on a vertical, two-dimensional plane, markers attached to fingers can point in any di-rection including the floor. The capture space therefore needs tobe very well covered. Frequent gaps in the data must be repairedwith manual post-processing. Glove-based solutions exist [Wangand Popović 2009; CyberGlove Systems 2012; Measurand 2012].However, these have drawbacks, such as lower accuracy, difficult ortime-consuming calibration, and drift [Kahlesz et al. 2004]. Othertechniques, such as combining optical motion capture with depthinformation from the Kinect [Zhao et al. 2012] or using computervision algorithms [Athitsos and Sclaroff 2003], are restricted tosmall, controlled areas and the capture of specific tasks [Wang et al.2011].

Majkowska et al. [2006] propose a method to motion capture bodymotions and hand motions separately and automatically assemblethem. This approach allows for a smaller capture space when onlyhand motions are recorded. Nevertheless, it requires the actor to re-peat the motions accurately, a difficult task for longer scenes with-out a specified choreography. Ye and Liu [2012], also observedthat detailed finger motion can be created based on the wrist move-ments and the motion of a handled object. Their approach, whichwas developed simultaneously to ours, focuses on the challenge ofmanipulation strategies and therefore complements our approach.

Most research on creating finger motions has focused on animat-ing specific tasks, such as playing a musical instrument [ElKouraand Singh 2003]. Manipulation tasks have been successfully simu-lated using physics-based approaches [Liu 2008; Liu 2009; Pollardand Zordan 2005], often using sensors to measure parameters or

forces [Kry and Pai 2006]. Neff and Seidel [2006] employ a sim-ilar method to animate relaxed hand shapes, deriving the parame-ters needed for the physics-based simulation from video recordings.Another solution for animating hands and finger motions are de-tailed, anatomically correct hand models [Tsang et al. 2005]. Thesemodels are usually time-consuming to animate.

Finger motions for conversational gestures represent an importantapplication area because gestures are considered an integral partof the act of speaking or producing an utterance [Kendon 2004].Even though gestures do not have a strict grammar, they do followrules that allow us to understand and interpret each other’s gestures[McNeill 1992]. Cassell et al. [2001] use a rule-based approach de-rived from linguistic and gesture studies research to synthesize ges-tures based on written text. Stone et al. [2004] animate the gesturesof a conversational character with a database of recorded speechand captured motions, fitting the gestures to its generated voice.Levine et al. [2010] train a probabilistic model to create gesturesbased on the prosody of the voice and a database of gestures. Neffet al. [2008] focus on creating different styles of speaker motionswith video or text as input. In contrast, our method focuses on thedetailed motion of the fingers. We do not take into account themeaning of the conversation or the rhythm in the audio track as weassume that those are adequately conveyed by the wrist motion.

The perception of finger animation has been studied by Jörg etal. [2010], who showed that even small errors in the synchroniza-tion of body and finger motions can be perceived and can changethe interpretation of an animation. Samadani et al. [2011] showthat an emotion (anger) can even be recognized on moving hand-like structures. Hoyet et al. [2012] investigate the perception offinger animations generated with different sets of markers and findthat finger motions reconstructed from a reduced set of eight mark-ers per hand are perceived to be similar to motions captured with aset of twenty markers for many types of motions but are perceivedto be different for some motions.

Our method searches for short segments of finger motions in adatabase and finds an optimal path through the resulting motiongraph. It therefore builds on the motion graph literature [Rose et al.1998; Kovar et al. 2002; Lee et al. 2002].

3 Method

The goal of our method is to automatically synthesize finger mo-tions for virtual conversational characters. With the term finger mo-tions, we denote the movements of the phalangeal joints of all fivedigits of both hands and the carpometacarpal joint of the thumb,while body motions designate the motions of the body joints of thevirtual character including the wrist positions and orientations butexcluding the finger movements. In its strict anatomical definition,the term finger does not include the thumb [Palastanga et al. 2006]but for simplicity we use the term to refer to all five digits.

The inputs to our system are the body motion of a virtual characterand a database of gesturing or conversing motions where both thefinger and body motions are present. As output, our approach gen-erates the movements of the finger joints of both hands (Figure 2).

The key observation behind our approach is that arm and finger mo-tions are highly correlated. When we see an animation of a char-acter without hand motion, we can usually infer what the hand mo-tions must have been. A skilled animator would be able to producea realistic rendition of the finger motion. To produce realistic fingermotions, our method therefore searches the database for body mo-tions that are similar to the body movements of the input motion.We then adapt the associated finger motions to fit the input motion.

16.2 23.8 27.0 32.8

32.6 37.4 38.3 38.7

10.6 12.2 13.8 15.9

32.7

27.3

start

10.9

end

0

0

0 34.9 32.5 42.5 17.8

Figure 3: Example graph for a short gesture. Time increases fromtop to bottom. Each node stands for a motion fragment, the numberrepresents the segment cost. One row shows the five motion frag-ments from the database with the lowest segment cost (most similarbody motions). Each edge represents a transition, so every possiblecombination of two consecutive fragments is connected by an edge.A few example transition costs are displayed. The red arrows andboxes depict the least-cost path.

Arm and finger motions are correlated for many gestures as weshow in Section 4. However, there is not a one-to-one mappingbetween them. Similar arm motions can be associated with severaldistinct finger motions. For example, rock and scissors from thewell-known game Rock-paper-scissors have the same arm motionbut different finger motion. Our method can predict plausible, re-alistic looking finger motions based on the arm motions alone, butcannot reproduce an exact match of the performed motion.

Our algorithm treats the right hand and the left hand independently.It consists of the following steps for each hand: (1) Segment theinput motion and the database into meaningful short segments.(2) For each segment of the input motion, search the database forthe k most similar body motion segments by computing a segmentcost. (3) Evaluate the quality of the finger motion transitions forall possible combinations of consecutive segments that were previ-ously found (transition cost). We can describe all possible result-ing motions in a weighted motion graph where each node repre-sents a segment with its associated cost and each edge correspondsto a transition with the transition cost as weight (see Figure 3).(4) Search this graph for the least-cost path. (5) Process the seg-ments to smooth the transitions. We describe each part of the algo-rithm below.

3.1 Segmentation

To effectively search for similar body motions, we must segmentour input motion and our database into meaningful fragments. Agesture consists of several phases: preparation, where the armmoves away from its rest position, pre-stroke hold, stroke, post-stroke hold, and retraction, where the arm moves back into the restposition [McNeill 1992; Kendon 2004]. Gestures can be combinedwithout returning to a rest pose. Only the stroke phase is obligatory,all other phases are optional. In the short gestures we recorded, wefound that the the pre-stroke hold was often missing.

Figure 4: Segmentation of a gesture of type “small”. The verticalblue lines indicate where the gesture is split into segments. Thefirst segment is 30 frames long (1.0s) and includes the preparationand stroke phases, the second segment is the post-stroke hold (25frames), and the retraction forms the third segment (46 frames).

We found that a simple approach, similar to the method describedby Levine et al. [2009], works well at separating motions into mean-ingful phases. We split the motion when the speed of the wrist jointcrosses a threshold close to zero, thus separating motion phases(high speed) and hold phases (low speed). We restrict segmentlength to be no less than 0.33 seconds and no more than 2 seconds.If segments are too long, an additional split is added at a suitablelocal minimum of the speed. These restrictions reduce spurioussegmentation due to noise and avoids overly long segments.

Out of our 64 single gestures from the gesture database, 49 are seg-mented into three phases (see Figure 4), where the first phase lastsfrom the rest position to the end of the stroke (quick wrist motion),the second phase is the post-stroke hold (slow) and the third oneis the retraction (quick). Out of the remaining gestures, ten haveshort post-stroke holds (our minimum segment length is 0.33s), sothe gestures are segmented into only two parts and five gestures aresegmented into four parts, e.g. with a long post-stroke hold split intwo segments.

3.2 Finding Similar Segments

For each segment of the input motion, we search the databasefor the k most similar segments. The candidate fragment in thedatabase must be within a factor of 1.5 of the length of the inputmotion to avoid creating unnaturally fast or slow motions.

To compare two segments SI (the segment from the input motion)and SD (the segment from the database), we compute the segmentcost cS . First, we adapt SD using dynamic time warping, so that ithas the same length as SI . Next, we compute the weighted squareddistance of the position and rotation coordinates of the wrist jointin the two segments according to the following formula:

cS =1

n(wP

n∑t=1

(PI(t)− PD(t))2 + wRn∑

t=1

(RI(t)−RD(t))2)

(1)

where wP =1 and wR=0.5 are weighting terms (see Section 4),t = 1, . . . , n is the frame number, PI(t) and PD(t) are the wristjoint position coordinates of the input motion and the fragment fromthe database, and RI(t) and RD(t) are the wrist joint rotation co-ordinates of the input motion and the fragment from the database,respectively. We use the position and rotation coordinates relativeto the root (hip) of the character. We use Euler angles for our com-putation but the method can be used with any angle representation.

3.3 Transition Cost

Not only do we require a good match to the arm motion segmentsbut we also need the transitions between different finger motionsegments to be smooth. We compute the transition cost cT for eachset of possible consecutive fragments by comparing the orientationsand angular velocities of the fingers at the last frame A of one frag-ment and the first frame B of the next fragment. We again usesquared differences:

cT =1

m(wJ

m∑i=1

(JA(i)−JB(i))2+wWm∑i=1

(WA(i)−WB(i))2)

(2)

where JA(i) and JB(i) are the rotations of the i-th degree of free-dom of the finger joints in frame A and frame B, respectively,while WA(i) and WB(i) are the corresponding angular velocities.Weighting terms, wJ=1 and wW =0.5, ensure that angular velocitiesand rotations have a similar influence. The finger skeleton consistsof m degrees of freedom, for our hand skeleton m=25.

3.4 Finding the Best Path

To select the final motion, we compute a weighted graph as shownin Figure 3. The start node of this graph is connected to the k seg-ments from the database that were most similar to the first inputmotion segment. Then, each segment is connected to the k seg-ments that were most similar to the second input motion segment.So, each row i represents the k best matches for input segment i andeach segment of one row (i) is connected to each segment of thefollowing row (i+1).

The graph is traversed from top to bottom. A weighted sum ofthe corresponding transition and segment costs is applied to eachconnection:

cost = wS ∗ cS + wT ∗ cT (3)

The weights wS=1 and wT =0.5 adjust the segment and transitioncosts so that they have the same influence on average, with positionsmeasured in cm and rotations in degrees. We find the best paththrough the weighted graph with Dijkstra’s algorithm.

3.5 Computing the Final Motion

Although the transition cost was taken into account when selectingthe fragments, there will be differences between the finger posesof consecutive fragments. Simple blending between each fragment(every 0.33 – 2 seconds) could result in a jittery motion. Therefore,for each transition, we compute the difference between the fingerrotations of the first frame of the new fragment and the last frameof the previous fragment and add this difference as an offset to thefinger rotations of the new fragment. Over time this offset can in-crease. We perform a linear blend only when the mean angulardifference or the maximum angular difference exceeds a threshold(5◦ and 20◦, respectively).

4 Analyzing the Similarity Function

Our algorithm relies on the hypothesis that it is possible to inferplausible hand motions from body motions, even if there might bemultiple solutions with substantially different valid finger motionsfor a specific body motion. We want to assess the validity of thishypothesis to determine which parts of the body are most appropri-ate for predicting finger motions. We test different parameters andcombinations of parameters, notably the positions and orientationsof the shoulders, elbows and wrists.

Figure 5: Variations of the ok gesture. The frames were taken fromthe hold phase of three ok gestures performed at different heights.

Figure 6: Variations of the big gesture. In the stroke phase, thehands were placed at different heights from abdomen to head andat different distances from close to far.

To assess the performance of these parameters, we created adatabase with known finger motion solutions. We recorded multipleshort samples of eight gesture types: attention, big, doubt shrug, ok,palm presentation (PP), small, turn, and wipe. These gesture typeshave characteristic finger motions but can have large variations intheir arm motions. For example, an ok-gesture can be performed atdifferent heights and in different directions but the thumb and theindex always form a circle and the other fingers are nearly straight(Figure 5). A good similarity function should, when given the bodymotion corresponding to an ok-gesture as input, find the ok-gesturesin a database that includes other gestures.

To decide which gesture types to capture, we annotated several min-utes of conversational motions with a method similar to Kipp et al.[2007] and chose gesture types that were present multiple times andhad characteristic finger motions. We recorded variations of eachgesture type, ensuring that different pacing, hand height, and dis-tances would be represented in each gesture type (Figure 5 and 6).

We used an equal number of samples from each gesture type, lead-ing to 64 gestures in the database (8 repetitions x 8 gesture types).The gestures were performed with one or two hands, dependingon what was natural. Each gesture was performed in a controlledand systematic way, starting and ending in a rest position with thearms hanging relaxed at the side of the body. We trimmed the be-ginning and ending of every gesture to remove rest poses beforeand after the actual gesture. We then segmented all gestures withthe method presented in Section 3.1, typically leading to three seg-ments: preparation and stroke, post-stroke hold, and retraction. Thedatabase consisted of 187 segments.

If we take one of the gestures out of our database and reconstructit using the remaining gestures, we can assume that the motion isplausible if gesture segments of the same type were selected forthe reconstruction and incorrect if other gestures were selected. Todetermine which body segments are good predictors for finger mo-tions, we used leave-one-out cross-validation. For each parame-ter, we repeat the following. We iterate over all segments in thedatabase and predict its gesture type based on the remaining ges-ture segments. The prediction itself is done by finding the k mostsimilar gesture motions in the database based solely on the compar-ison of the parameter we evaluate, using squared differences anddynamic time warping. We compute the percentage of correctlyclassified segments amongst the k closest suggested segments. Inour analysis k=5.

Figure 7: Percentage of correctly selected gesture types for someof the parameters we have considered.

Figure 7 plots the computed probabilities for some of the bodysegments we tested. When each parameter was considered on itsown, the wrist rotation performed best with 75%. We also includedweighted combinations of different parameters and concluded thata weighted combination of the position and orientation of the wristgives the best prediction result. With positions measured in cmand rotations in degrees, the best combination was found to beP +0.5 ∗R (80%), which explains the weights used in Equation 1.

A class confusion matrix can be seen in Figure 8. The gesture typeattention was the hardest to recognize. However, even for this ges-ture type, 55% of the suggestions were correct. In 18% of the casesa segment from an ok gesture and in 14% a segment from a smallgesture was suggested instead. Both are gestures with relativelysimilar arm motions to attention. In the full algorithm, the tran-sitions help to avoid segments with finger motions that are verydifferent from those of the surrounding segments. Therefore, thesuccess rate when a complete motion is reconstructed is likely to behigher.

5 Results

To demonstrate the flexibility of our approach, we motion capturedan actress and two actors with optical Vicon motion capture systemsconsisting of 13 to 18 cameras. In each capture, fifty-six markerswere attached to the actor’s body and 22 markers to each hand. Theactors were asked to perform a series of gestures and conversations.The resulting data was post-processed and a skeleton was fitted toit. For each hand and for the body, we recorded a series of move-ments rotating each joint across its full range of motion. We thencomputed three skeletons (one for each hand and one for the body)separately to increase their accuracy. The skeletons were then as-sembled into one. In the following sections, we present a series ofexamples computed with different gesture databases. The resultinganimations can be seen in the accompanying video.

5.1 Short Gestures

We synthesized the finger motions of several gestures from thelarge gesture database presented in Section 4. We chose one ofthe 64 gestures as an input motion and deleted its finger move-ments. The remaining 63 gestures formed the database that we usedto synthesize new finger motions. In the video, we show a compari-son between the ground truth motion capture and our reconstructedfinger motion for an ok and a doubt-shrug gesture. We also show anattention gesture as a failure case. In this example, the finger mo-tion of the gesture’s stroke phase was selected from an ok gestureinstead of an attention gesture.

gesture type of the closest segments

segm

entt

aken

outo

fthe

data

base

Figure 8: Class confusion matrix from the leave-one-out cross-validation. The numbers represent the percentages that a gesturewas found in the closest segments. Lines that do not sum up to 100are due to rounding errors. These results were obtained using acombination of wrist position and wrist rotation to find the closestsegments (P + 0.5 ∗R).

We created a small gesture database using a different actor andconsisting of seven types of motions, each captured twice in a verysimilar manner. Motion types were attention, count, no, ok, point,snap, and thumbs-up. We use the body motion of one of the at-tention gestures as input motion and reconstruct its finger motionusing the remaining 13 gestures as a database. The finger motionis synthesized correctly using the attention gesture that remained inthe database.

We also synthesize the finger motions of the attention gesture fromthe small gesture database using the large gesture database. Toaccount for differences in size between the actors, we scale the po-sition data based on the height of each character. The motion wasreconstructed appropriately as can be seen in the video.

5.2 Conversations, Debates, and Directions

We generated three databases consisting of longer scenes: conver-sations, debates, and directions. The databases were recorded withdifferent actors and the topics were chosen to lead to different ges-ture types.

For the conversation database, we recorded more than 13 minutesof motions of an actor conversing about such subjects as his lat-est holiday trip, a project he was working on, or a movie that herecently watched. To increase the naturalness of the motions, asecond person stood behind the cameras acting as a conversationalpartner. For motion synthesis, we used sequences of 2-4 minutes asinput motions, leaving a database of about 10 minutes. The synthe-sized finger motions look realistic and extracts can be seen in thevideo.

Our second database, debates, consists of 9 minutes of debates froma different actor. We chose debating because we hoped that it wouldlead to more energetic gestures. We listed a dozen of popular topicsfrom debating clubs and let the actor choose the subjects that hepreferred. Our five takes were each between 1.5 and 2 minutes inlength. To create examples, we excluded a take from the database,and synthesized its finger motions from the remaining database.

Third, for the directions database, we asked an actress to describehow to get to different places, such as the airport or her home. Giv-ing directions involves very specific gestures, for instance, for turn-ing or stopping. Four takes, each 2-3 minutes long, resulted in over9 minutes of data. We generated examples in the same way as forthe previous databases.

Figure 9: Means of the rated realism of short animated clips withdifferent finger motions. Animations without finger motions wererated significantly less realistic than animations with motion cap-tured finger motions or with finger motions synthesized with ourmethod. Error bars represent one standard error of the mean.

5.3 Perceptual Experiment

We tested the realism of our results in a perceptual experiment. Par-ticipants were shown short clips of animations with three types offinger motions: ”motion captured”, ”our method”, and ”no fingermotions”. The total stimuli consisted of 8 (segments) x 3 (motiontypes) x 3 (repetitions) = 72 clips. Five segments were chosen fromthe debates database and three from the large gesture database, in-cluding the incorrect reconstruction of the attention gesture shownin the video. We chose a camera perspective focusing on the char-acters hand space (thigh to neck) for most clips, because we did notwant to show the face as it is not animated.

The experiment was fast paced, with each clip lasting between 3 and5 seconds, as a complete lack of finger motions is usually obviousin longer sequences. The 72 clips were shown in random order.Between each clip participants had 5 seconds to respond.

The participants were asked to pay attention to the finger motionsand to rate the realism of each animation on a scale from 1 (very un-realistic) to 5 (very realistic). The 14 participants were students andpostdocs (7m, 7f, aged 18-35) who were recruited through posterson campus and announcements in classes. The experiment tookabout 20 minutes and participants were rewarded with $5. Partici-pants took part in five groups of two to three.

To analyze the results we averaged the three repetitions of eachsegment for each motion type and participant. A repeated mea-sures ANOVA (within effects motion type and segment) with aHuynh-Feldt correction determined a significant effect of motiontype: F(2,26)=25.784, p

References

ATHITSOS, V., AND SCLAROFF, S. 2003. Estimating 3d hand posefrom a cluttered image. In IEEE Computer Vision and PatternRecognition (CVPR), vol. 2, 432–439.

CASSELL, J., VILHJÁLMSSON, H. H., AND BICKMORE, T. 2001.Beat: the behavior expression animation toolkit. In ACM SIG-GRAPH 2001, 477–486.

CYBERGLOVE SYSTEMS, 2012.www.cyberglovesystems.com/products/cyberglove-ii/overview.

ELKOURA, G., AND SINGH, K. 2003. Handrix: animating thehuman hand. In Proceedings of the ACM SIGGRAPH/Eurogra-phics Symposium on Computer Animation, 110–119.

HOYET, L., RYALL, K., MCDONNELL, R., AND O’SULLIVAN,C. 2012. Sleight of hand: perception of finger motion fromreduced marker sets. In Proceedings of the ACM SIGGRAPHSymposium on Interactive 3D Graphics and Games, I3D ’12,79–86.

JÖRG, S., HODGINS, J., AND O’SULLIVAN, C. 2010. Theperception of finger motions. In Proceedings of the 7th ACMSymposium on Applied Perception in Graphics and Visualization(APGV 2010), 129–133.

KAHLESZ, F., ZACHMANN, G., AND KLEIN, R. 2004. Visual-fidelity dataglove calibration. In Computer Graphics Interna-tional, 403–410.

KENDON, A. 2004. Gesture – Visible Action as Utterance. Cam-bridge University Press, Cambridge, UK.

KIPP, M., NEFF, M., AND ALBRECHT, I. 2007. An annotationscheme for conversational gestures: How to economically cap-ture timing and form. In Journal on Language Resources andEvaluation, vol. 41, 325–339.

KOVAR, L., GLEICHER, M., AND PIGHIN, F. 2002. Motiongraphs. ACM Transactions on Graphics 21, 3, 473–482.

KRY, P. G., AND PAI, D. K. 2006. Interaction capture and synthe-sis. ACM Transactions on Graphics 25, 3, 872–880.

LEE, J., CHAI, J., REITSMA, P. S. A., HODGINS, J. K., ANDPOLLARD, N. S. 2002. Interactive control of avatars animatedwith human motion data. ACM Transactions on Graphics 21, 3,491–500.

LEVINE, S., THEOBALT, C., AND KOLTUN, V. 2009. Real-timeprosody-driven synthesis of body language. ACM Transactionson Graphics 28, 172:1–172:10.

LEVINE, S., KRÄHENBÜHL, P., THRUN, S., AND KOLTUN, V.2010. Gesture controllers. ACM Transactions on Graphics 29,4, 1–11.

LIU, C. K. 2008. Synthesis of interactive hand manipulation. InProceedings of the ACM SIGGRAPH/Eurographics Symposiumon Computer Animation, 163–171.

LIU, C. K. 2009. Dextrous manipulation from a grasping pose.ACM Transactions on Graphics 28, 3, 3:1–3:6.

MAJKOWSKA, A., ZORDAN, V. B., AND FALOUTSOS, P. 2006.Automatic splicing for hand and body animations. In Proceed-ings of the ACM SIGGRAPH/Eurographics Symposium on Com-puter Animation, 309–316.

MCNEILL, D. 1992. Hand and Mind: what gestures reveal aboutthought. The University of Chicago Press.

MEASURAND, 2012. www.shapehand.com.

NEFF, M., AND SEIDEL, H.-P. 2006. Modeling relaxed handshape for character animation. In Articulated Motion and De-formable Objects, vol. 4069 of Lecture Notes in Computer Sci-ence. Springer, 262–270.

NEFF, M., KIPP, M., ALBRECHT, I., AND SEIDEL, H.-P. 2008.Gesture modeling and animation based on a probabilistic re-creation of speaker style. ACM Transactions on Graphics 27,1, 1–24.

PALASTANGA, N., FIELD, D., AND SOAMES, R. 2006. Anatomyand Human Movement – Structure and Function, 5 ed. Butter-worth Heinemann Elsevier.

POLLARD, N. S., AND ZORDAN, V. B. 2005. Physically basedgrasping control from example. In Proceedings of the ACMSIGGRAPH/Eurographics Symposium on Computer Animation,311–318.

ROSE, C., BODENHEIMER, B., AND COHEN, M. F. 1998. Verbsand adverbs: Multidimensional motion interpolation using radialbasis functions. IEEE Computer Graphics and Applications 18,32–40.

SAMADANI, A.-A., DEHART, B. J., ROBINSON, K., KULIĆ, D.,KUBICAN, E., AND GORBET, R. 2011. A study of humanperformance in recognizing expressive hand movements. IEEEInternational Symposium on Robot and Human Interactive Com-munication, 93–100.

STONE, M., DECARLO, D., OH, I., RODRIGUEZ, C., STERE,A., LEES, A., AND BREGLER, C. 2004. Speaking with hands:creating animated conversational characters from recordings ofhuman performance. ACM Transactions on Graphics 23, 3, 506–513.

TSANG, W., SINGH, K., AND FIUME, E. 2005. Helpinghand: an anatomically accurate inverse dynamics solution forunconstrained hand motion. In Proceedings of the 2005 ACMSIGGRAPH/Eurographics Symposium on Computer Animation,319–328.

WANG, R. Y., AND POPOVIĆ, J. 2009. Real-time hand-trackingwith a color glove. ACM Transactions on Graphics 28, 3, 1–8.

WANG, R. Y., PARIS, S., AND POPOVIC, J. 2011. 6d hands:markerless hand-tracking for computer aided design. In ACMSymposium on User Interface Software and Technology, 549–558.

YE, Y., AND LIU, C. K. 2012. Synthesis of detailed hand manipu-lations using contact sampling. ACM Transactions on Graphics31, 4.

ZHAO, W., CHAI, J., AND XU, Y.-Q. 2012. Combiningmarker-based mocap and RGB-D camera for acquiring high-fidelity hand motion data. In Proceedings of the ACM SIG-GRAPH/Eurographics Symposium on Computer Animation.