2013 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) Wollongong, Australia, July 9-12, 2013

Human Skill Transfer System via N ovint Falcon

Tarinee Tonggoed and Siam Charoenseang

Abstract- This paper presents a skill transfer system of

hand movement via Novint Falcon. In the research, expert can

demonstrate his or her hand movement skill through the 3 DOF

device while the novice is able to practice how to perform that

trajectory on the same device by himself or herself. The

proposed skill transfer system composes of skill modeling, skill reproduction, skill playback, and skill evaluation. During skill

modeling, the Gaussian mixture model algorithm is selected to

model the hand's trajectory. The trajectory data can be

reproduced by applying a Gaussian mixture regression

algorithm in the reproduction process. Skill playback then

receives reproduced data to present the modeled trajectory of the device to the novice. After training, the novice will perform

the trajectory that he or she already practiced through the

device. Then, the skill evaluation process will determine the

similarities of spatial and temporal values between the modeled

trajectory and the captured novice's trajectory. The experimental results show that this proposed system is efficient

enough for modeling and reproducing the captured trajectory

in the human skill transfer from the expert to the novice.

I. INTRODUCTION

During the past decades, many research works have contributed on human's skill modeling and they also transferred some skills related to manipulation to the robots. This field of research also covers the robot programming by demonstration (PbD) or learning from demonstration (LID) and there are several approaches to apply the learning methods to the robots. Yang[l] and Calinon[2] have investigated the effectiveness of using Hidden Markov Model in skill modeling. Another interesting method for modeling skill is a Gaussian mixture model (GMM) which is used in[3] and [4] . Besides robot programming by demonstration, the skill transferring to human operator is also an attractive research field. N echyba and Yang sheng studied about skill transferring from an expert to a less-skilled operator[5] . By using Neural Network in the learning process, they focused on transferring human control strategy in a simulated inverted pendulum system from an expert user to a less-experienced. Kazuyuki HENMI presented a calligraphy transfer skill system using haptic virtual reality technology [6] . Jorge Solis proposed the idea of using the haptic system to exert the proper force to help user to perform tasks appropriately [7] . There are not many research works which cover both expert's skill modeling and transferring to novice with multiple DOF

Tarinee Tonggoed is with the Institute of Field Bobotics, King Mongkut' s University of Technology, Thonburi, Bangkok, 10140 Thailand, (corresponding author to provide phone: +(66) 0-2470-9715, 0-2470-9716; fax: +(66) 0-2470-9714; e-mail: 54501901@st.kmutt.ac.th).

Siam Chareonseang is with the Institute of Field Bobotics, King Mongkut's University of Technology, Thonburi, Bangkok, 10140 Thailand, (corresponding author to provide phone: +(66) 0-2470-9715, 0-2470-9716; fax: +(66) 0-2470-9714; e-mail: siam@fibo.kmutt.ac.th).

devices. Usually, they cover only I or 2 DOF skill transfer system.

In this paper, a development of a skill transfer system that can teach the non-expert with a skill related to the expert's hand trajectory using a 3-DOF Novint Falcon touch device is proposed. Gaussian mixture model is chosen for modeling the expert's manipulation skill. The component number of Gaussian mixture model is an important parameter for obtaining an efficient model. In this research, an estimation of the component number based on the changes of trajectory direction will be discussed. Furthermore, the regenerated data sent to the device is also reproduced by using Gaussian mixture regression. Then, the system will evaluate the temporal and spatial similarities between the novice's hand trajectory and the modeled expert's trajectory. The preliminary experimental results are collected and presented to show the performance of the proposed system. This paper is organized as follows. System Overview section covers details of the whole system. Experimental setup and results section show the experimental results of the proposed system. Finally, conclusions and further works are presented in the last section.

II. SYSTEM STRUCTURE

A. System Overview

The proposed system consists of an expert, a novice, a 3D Novint Falcon touch device [8], and a computer as shown in Fig.l. In this system, the expert plays the role of a teacher to transfer motion skill by holding a gripper of the Novint Falcon to perform the hand movement via that device. The Novint Falcon acts as both a sensor and an actuator. As a role of the sensor, the device will capture and send the hand trajectory of expert to the computer. In the other hand, the Novint Falcon will present touch feedback accordingly to the modeled movement to the novice during the training. The computer will record both spatial and temporal data during system training for skill capture and novice training for skill transfer. The computer will build the hand's trajectory model from the recorded data and evaluate the trajectory similarity of novice's movement and the captured path. In addition, the novice can use the Novint Falcon to practice the captured skill which is reproduced by the computer.

978-1-4673-5320-5/13/$31.00 ©2013 IEEE 483

Expert: DemonstrateHaod .:\Jovement Trajectory

Ncn-int Falcon ( Sensor and Acnu.tor)

Computer

Figue I System Overview

B. System Data Flow

In system training, the expert will demonstrate the hand movement trajectory through the Novint Falcon. The data gathering module then records that trajectory in the form of position (x, y, and z) and time (t). The recorded data will be sent to the trajectory modeling module which builds the captured trajectory's model by using the Gaussian mixture model algorithm. Next, the system will reproduce the captured trajectory from the model by the reproduction module. Time steps and Cartesian positions are the outputs of this module. After the reproduction process, the playbacked data will be sent to the robot controller module which is responsible to control the actuator's movement using PID control algorithm. Forces in x, y, and z axes are generated from that module and sent to the Novint Falcon module to perform its movements. This will guide the novice to learn and practice the modeled trajectory obtained from the expert. During the evaluation phase, the data gathering module will capture and send the obtained data which are time steps (t) and position (x, y, and z) of the novice's hand trajectory to the evaluation module. The module will compare the likeness between the expert's trajectory model and the novice's trajectory of using the Novint Falcon. The system data flow in Fig.2.

Figure 2 System Data Flow

III. LEARNING ALGORITHM AND ROBOT CONTROLLING

The learning algorithms applied in this research consist of Gaussian mixture model and Gaussian mixture regression. The first one is applied in skill modeling process. The estimation of Gaussian component number based on the direction changing will be discussed. In the reproduction process, the Gaussian mixture regression is the method for data recovering. In addition, a PID controller is implemented to control the Novint Falcon's movement.

A. Gaussian Mixture Model (GMM)

Data modeling from a mixture of Gaussian distribution K is the main idea of Gaussian mixture model. For example, if there are n data sequences and each data sequence has a length of T, data set contains N = nT data points. Let data set be a = {aj }J=l which aj = {at, as} and p( aj) is defined as the posterior probability. By using Bayes theorem, p( aj) is computed as in

(I ) [9]

p (k) are priors of each K Gaussian and p (aj I k) is a conditional probability density function as in

(2) [9]

From above equations, GMM parameters are {IT k' 11k, LIJ�=l . Where IT k' 11k and Lk are prior, mean, and covariance matrix of the kth Gaussian. For learning of GMM parameters, k-mean clustering technique is selected to initiate the GMM's parameters and Expectation­Maximization algorithm is used to train those parameters.

In this paper aj is a D-dimensional data set with D = 4. The

data sequences have n = 5 and each sequence's length is T=40.

B. Gaussian Mixture Regression (GMR)

Gaussian Mixture Regression is an algorithm for data recovering from the trained GMM. From the previous section, GMM is defined by prior(IT k), mean(l1k), and covariance (Lk) matrices. The condition expectation of as given at can be described as in

(3) [9]

Where, (4) [9]

In this research, at is the time step with length of 40 and step size is 1. Finally, the position in x, y, and z at each given time step can be recovered.

C. Estimation of Gaussian Component Number

The number of Gaussian component is the important initial parameter for updating in the expectation-maximization (EM) algorithm. The idea of estimating that number in this proposed paper is based on the changes of trajectory's direction. In the research, spatial and temporal data in 4 dimensions are collected. Before the data modeling process, the other important process is the data preprocessing for obtaining more clean and meaningful data. In this research, the changing of direction is contributed for good feature extraction. The idea of this algorithm is to calculate the angle between the spatial data which are the positions in x, y, and z axes at time step t and HI as shown in Fig.3. After the calculation of angle, the result of that process can be classified into 8 different groups .

484

Positior.U)

No

Po$ition (t+ 1)

3" LSJ2j" 0,2r.

5"\2JS7,r. ,,,

Figure 3 Direction Segmentation

Calculate Slope between data(t) and data(t+1)

ClassifY Slope into Group (8 groups)

Is data empty?

Figure 4 Flow Chart of Data Preprocessing

At the beginning, the one set of recorded data is read and

the moving average with window size of 4 is applied for

smoothing the raw data. After that, angle between time step or slope will be calculated and classified into each group.

This process will be repeated until there is no more recorded

data. The output of this process is the set of labeled data

which is a feature vector. The flow chart of this data

preprocessing can be shown in FigA.

From the data preprocessing, the labeled data sets are

obtained. The next process is the process for grouping data

of the same label into the same cluster. The algorithm of

grouping is shown in Fig.5. The last process is to find the

maximum value of cluster group for each data set and

average those values. In this process, the averaged Gaussian

component number can be obtained.

If Label at n iteralion = Label al n+I ileralion

Count Group Member = Count Group Member + I

Label Group umber to the Data as tcmpLabcl(startLabcl:(startLabcl+(Count Group Member .1))) = Group Number

" data empty?

Figure 5 Flow Chart of Data Clustering based on Direction Changing

E. PID controller

A PID controller is applied in the robot controller for this system. Since the robot's workspace is not large, the integral term of the controller does not effect much on this system. The control equation of PID controller can be demonstrated as in Equation 5.

d u(t) = Kp e(t) + K/ f e(t) dt + KD dt

e(t) (5)[ 10]

Where,

u(t) is Output signal to process

e(t) is Error e(t) = ret) - yet) ret) is Setpoint

yet) is Actual Output

For this system, r (t) is the target position which is obtained

from the reproduction process and yet) is the end effector

current position. Ziegler-Nichols method is applied for the

PID parameter tuning. The final gains are Kp = 10, Ki =

0.001, and Kd = 0.25.

IV. EXPERIMENTAL SETUP AND RESULTS

The experimental setup is shown in Fig.6 which illustrates

the expert or novice, a computer, and a Novint Falcon touch

device shown in Fig. 7 which is a 3-DOF parallel delta robot

with position resolution of 400 dpi in 4" x 4" x 4"

workspace[8] . One expert and seven novices were asked to

perform two main sets of experiments.

485

Figure 6 Experimental Setup

Figue 7 Novint Falcon [8]

In the research, the Novint Falcon is used as both an input and an output device. The hap tics device abstraction layer (HDAL) SDK [I I ] is implemented to manipulate the Novint Falcon's movement.

There are two trajectories that expert will have to demonstrate to the robot as shown in Fig.8 and Fig.9

a ) Pull Left b) Pull Right c) Pull Forward Figure 9 The Second Demonstrated Trajectory of Hand Movement

Three experimental sets were conducted to evaluate the effectiveness of this system. The first set is to test the efficiency of the proposed estimation method for Gaussian component number The second set is implemented to test the system performance of trajectory transferring from the expert to the robot. The performance in skill transferring from the robot to the novice is also investigated in the third experimental set. The system performance of trajectory transferring involves with recording time, training time, and the reproducing time which will be also measured. For the data set with a size of 3 x 40, the recording process takes about 4.78 seconds, the reproducing process using Gaussian mixture regression consumes about 3.6 milliseconds. Averaged training time will be also demonstrated in the experimental results of estimation Gaussian component number. For this research, the consumed times in recording, modeling, and reproduction processes are fast enough for the

implementation of skill transfer from the expert to the robot and the robot to the novice.

A. The Experimental Results of Estimation of Gaussian

Component Number

To evaluate this proposed algorithm, the incremental k-mean is selected to be compared with the estimation of component number. Bayesian information criterion (BIC) is selected as the criterion of stopping condition. The two 40-point trajectories are used in this research to evaluate the performance of this proposed algorithm. The experimental results are shown in Table I and 2.

TABLE I. RESULTS OF THE PROPOSED ALGORITHM AND INCREMENTAL K-MEAN ALGORITHM FOR GESTURE 1

Proposed Algorithm Incremental K-Mean Gaussian Process BIC Gaussian Process BIC

No No. Time Value No. Time Value

1 7 0.47 249.9 8 0.71 285.65

2 7 0.46 250.36 8 0.47 284.80

3 7 0.46 250.46 8 0.57 285.56

4 7 0.41 249.84 7 0.29 250.46

5 7 0.40 249.85 7 0.43 250.45

Avg. 7 0.44 250.08 7.6 0.50 271.38

TABLE II. RESULTS OF THE PROPOSED ALGORITHM AND INCREMENTAL K-MEAN ALGORITHM FOR GESTURE 2

Proposed Algorithm Incremental K-Mean Gaussian Process BIC Gaussian Process BIC

No No. Time Value No. Time Value

1 6 0.41 215.27 6 0.31 215.27

2 6 0.23 215.23 6 0.31 215.23

3 6 0.24 215.28 6 0.24 215.28

4 6 0.24 215.28 6 0.24 215.28

5 6 0.24 215.28 6 0.22 215.51

A\'£. 6 0.27 215.27 6 0.27 215.31

The experimental results in Table I show that the component numbers after applying the proposed algorithm in gesture lis equal to 7. The averaged modeling time is 0.44 seconds and the averaged of BIC value is 250.08. For applying the incremental k-mean algorithm, the averaged BIC value of the proposed algorithm is used as the stopping criterion of this method. The averaged component number is 7.6 but 8 is used in actual implementation. The averaged time consuming in the modeling process is 0.50 seconds and the BIC value is 271.38. Table 2 presents that the proposed algorithm took 0.27 seconds in the modeling process. The averaged BIC value is equal to 215.27 and the component number is 6. For the incremental k-mean algorithm, time consuming in the

486

modeling process took 0.27 seconds, and BIC value and component number are 215.31 and 6, respectively. These results indicate that the proposed algorithm can obtain similar performance on these specific data as the one obtained from the incremental k-mean algorithm.

B. The Experimental Results of System Performance of

Trajectory Transferring from the Expert to the Robot

In this research, log-likelihood is selected to evaluate the similarity between the trajectory of the Novint Falcon after being trained and the modeled trajectory obtained from the expert. The results of this evaluation show that an averaged log-likelihood of the Novint Falcon's trajectory after it performed first trajectory and second trajectory are -8.69 and -5.6, respectively. Based on those values and expert's decisions, it can be concluded that the Novnint Falcon can perform the alike movement as the expected trajectory which are created from the trained model. Fig. 10 presents that performance of using GMM and GMR is effective enough for the uses in trajectory modeling and reproduction processes.

-1

-2 06

0.5

-0.5

-1

-1.5

-2 2

-- Expected Trijutory -- Novint Falcon.'s Triljectory

04 0.2

-1 -0.2 -2

a) Novint Falcon's First Trajectory

__ ExpectedTriljectofV

06 0.4

0.2

-1 -0.2

b) Novint Falcon's Second Trajectory

Figure 10 Trajectories Performed by the Novint Falcon vs. the Expected Trajectory Model

C. The Experimental Results of System Performance of

Trajectory Transferring from the Robot to the Novice

The experiment's objective is to evaluate the effectiveness of the novice's trajectory training through the Novint Falcon. During the evaluation process, the novice is asked to practice to move his/her hand following the Novint Falcon's movement under the eye-closed and eye-opened conditions. After training under each condition, the novice performed the trajectory that he/she already has practiced on the Novint Falcon. The system will evaluate the resemblance of novice's trajectory with the model's one. The experimental results are shown in Fig.11 and Fig.12.

GeSIUI'e 1 � Novice's Log-likelihood froUl Ibe Tra ining Pbuse

0.00

-10.00 H�"""="'="""="""="'="""� -g -IS.OO t-:3:::::i='=-;""":::--�7"-"""''---� ·20 00 +--''---'''''/'-----',--''''''''7'''''-----='---� -25 00

cll)-JO.OO .3 -35.00 t------------,"0.00

-45.00

Novice no.

-Eye- losed audition -Eye-Opened oudition

-a-Expected Likelihood

Gesture 1: No,lh:e's Log.likelihood from Ihe Playb ack Phase

0.00 ,-=-=-=-==-=-� -S .OO

-g -IHIO +-------------=-­.,g -20.00 h.-----7"'r-�------+_­� ·25 .00 t-*-++--""'--:;:------;f-­db-30.OO

.3 -35.00

-40.00 t----\-f-----"""----t'-----�f----45_00 +-----"------"+-----=----SO.OO "------

N"'o-�-,-·

c-e

-no

-.

-----

__ Eye-Closed Condilion

___ Eye-Opened Conditioll

__ E;xpected Likelihood

Figure 11 Novice's Log-likelihood Plot for Gesture 1

G1!5t 1l1"1! 2 : �ovlc;e '5 Log_likelihood frOID the "rralnlng Phase

, , , , L ./Ao..-. ---\�/ '\.��

\ / V V

f--

'----Novice no.

.

----

,/'"

,

__ Eye-Closoo Condition ___ Eye-Openoo conditioll ....... Expected Likelihood

Ce�t\ln�2 ; N .;;", t cc' :!' LQg-likeliboooJ fl-Olll thc PI"y b lic k Pba:iOe

Novice no.

__ Eye-Closoo COrtdition ___ Eye-Opened Condition ........ Expected Likelihood

Figure 12 Novice's Log-likelihood Plot for Gesture 2

Fig.I I demonstrates that the log-likelihood plots obtained from the training and playing back with the eye-opened condition in first trajectory are about 57% better than the ones obtained from the learning and playing back with the

487

eye-closed condition. For the plots of the second trajectory shown in Fig. 12, the training with the eye-opened condition gives a better performance than the one with eye-closed condition upto 85%. Also, the play back of movement with eye-opened condition provides 57% better performance than another. This means the novices can learn the manipulation skill better under the eye-opened condition.

D. Conclusions and Further Works

The hand movement skill transfer from the expert to the novice through the Novint Falcon was presented. The proposed system covered human's trajectory modeling and reproduction by machine learning algorithms and demonstrated the real implementations of manipulation skill transfer from the expert to the device and the device to the novice. In the process of trajectory modeling, Gaussian mixture model is applied for building the captured trajectory's model. The estimation method of Gaussian component number based on the changes of trajectories' direction is proposed and compared with the incremental k­mean algorithm. The proposed method can provide similar performance as the well-known incremental k-mean algorithm. Gaussian mixture regression is a selected method in the reproduction process to regenerate the data accordingly to the captured model. The Novint Falcon is a 3-DOF parallel robot which plays the main roles of sensor and actuator in this proposed system. The expert demonstrates the hand trajectories through Novint Falcon and the system captures and builds their path models. The regenerated path data are presented through the Novint Falcon movement in the playback and novice training processes. The experimental results indicate that the novices can learn and playback with the systems under eye-opened condition better than the ones under eye-closed condition.

Several interesting points of work can be conducted to see whether the proposed system is robust enough to cope with some more sophisticated tasks which require more complex manipulation with multiple degrees of freedom. This leads in modification of robot to obtain more degrees of freedom and more accurate control. In addition, some augmented information could be implemented to provide graphical advised information in real time during the training and playing back processes. Also, this system can be enhanced for armlhand rehabilitations along with some force/tactile feedbacks via the proposed skill transfer system.

REFERENCES

[1] J. Yang, Y. Xu, and C. S. ChenZ, "Hidden Markov model approach to skill learning and its application to telerobotics," Robotics & Control

Systems vol. 10, 1993. [2] S. Calinon, F. D'halluin, E. L. Sauser, D. G. Caldwell, and A. G.

Billard, "Learning and Reproduction of Gestures by Imitation," IEEE

Robotics & Automation Magazine, vol. 17, 201 O. [3] S. Calinon and A. Billard, "Incremental learning of gestures by

imitation in a humanoid robot," in 2007 ACMIIEEE International

Conference on Human-Robot lnteraction, 2007.

488

[4] Y. Lin, S. Ren, M. Clevenger, and Y. Sun, "Learning Grasping Force from Demonstration," in 2012 lEEE lntemational Conference on

Robotics and Automation USA, 2012. [5] M. C. Nechyba and X. Yangsheng, "Human skill transfer: neural

networks as learners and teachers," in lntelligent Robots and Systems

95. 'Human Robot lnteraction and Cooperative Robots', Proceedings.

1995IEEEIRSJ International Conference on, 1995, pp. 314-319 vol.3. [6] K. HENMI and T. YOSHIKAWA, "Virtual Lessoin and Its

Application to Virtual Calligraphy System " in lnternational

Conference on Robotics & Automation Leuven, Belgium 1998. [7] J. Solis, C. A. Avizzano, and M. Bergamasco, "Teaching to Write

Japanese Characters using a Haptic Interface" in HAPTlC' 02, 2002. [8] S. Martin and N. Hillier, "Characterisation of the Novint Falcon

Haptic Device for Application as a Robot Manipulator," in Australasian Conference on Robotics and Automation (ACRA)

Sydney, Australia, 2009. [9] S. Calinon, Robot Programming by Demonstration: A Probabilistic

Approach: EPFL Press, 2009. 10] "PID controller," 2012. [ I I ] N. T. Incorporated, "Haptic Device Abstraction Layer (HDAL)" the

USA, 2008.