作者 : dipak kumar ghosh , samit ari 出處 :ieee 2011 報告者 : 邵建銘 ma1g0106
DESCRIPTION
A Static Hand Gesture Recognition Algorithm Using K-Mean Based Radial Basis Function Neural Network. 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106. Outline. INTRODUCTION METHODOLOGY EXPERIMENT RESULTS AND DISCUSSION CONCLUSION. INTRODUCTION. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/1.jpg)
1資訊工程系 智慧型系統實驗室 iLab南台科技大學
A Static Hand Gesture Recognition Algorithm Using K-
Mean Based Radial Basis Function Neural Network
作者 :Dipak Kumar Ghosh, Samit Ari出處 :IEEE 2011報告者 : 邵建銘 MA1G0106
![Page 2: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/2.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 2
Outline
• INTRODUCTION
• METHODOLOGY
• EXPERIMENT RESULTS AND
DISCUSSION
• CONCLUSION
![Page 3: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/3.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 3
INTRODUCTION• Generally, gestures can be classified into static
gestures, and dynamic gestures. • Static gestures are usually described in terms of
hand shapes or poses, and dynamic gestures are generally described according to hand movements.
• Static gestures can convey certain meanings and some time act as specific transitions state in dynamic gestures.
![Page 4: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/4.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 4
INTRODUCTION• For HCI and other application, the sensing
techniques which have used in static hand gesture recognition algorithm have been divided mostly into vision-based techniques and glove-based techniques.
• The glove-based techniques utilize sensors to measure the joint angles, the positions of the fingers and the position of the hand in real-time.
• In real time experiment, the images are captured by camera from different positions and angle, the extracted feature set which are input to the classifier is to be rotation invariant.
![Page 5: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/5.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 5
INTRODUCTION• In this paper, a k-mean based radial basis function neural
network (RBFNN) classifier is proposed for classification of static hand gesture images.
• Radial basis function neural network (RBFNN) is most popular neural network because – (i) its architecture is very simple– (ii) in hidden layer localised radial basis function are used to
nonlinear transform of feature vector from input space to hidden space;
– (iii) sensitivity of the hidden neuron is tuned by adjusting spread factor of basis function;
– (iv) this network is faster and free
![Page 6: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/6.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 6
METHODOLOGY• The proposed static hand gesture
recognition algorithm consists of three following stage: – preprocessing, – feature extraction – and classification.
![Page 7: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/7.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 7
METHODOLOGY
![Page 8: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/8.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 8
METHODOLOGY• Preprocessing
– Preprocessing of input gesture has been done through three operations: • segmentation, • rotation • morphological filter.
![Page 9: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/9.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 9
METHODOLOGY• Preprocessing
– Segmentation• The objective of gesture segmentation is to
extract the gesture region from the background of the image.
• The Otsu segmentation algorithm is applied in this work.
![Page 10: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/10.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 10
METHODOLOGY• Preprocessing
– Segmentation
Ω0 = {1, 2, . . . , T};
Ω1 = {T + 1, T + 2, . . . ,L}
![Page 11: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/11.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 11
METHODOLOGY• Preprocessing
– Segmentation
![Page 12: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/12.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 12
METHODOLOGY• Preprocessing
– Segmentation
Fig. 2: The original and segmented hand gesture of ‘A’ and ‘B’
![Page 13: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/13.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 13
METHODOLOGY• Preprocessing
– Rotation• The rotation is peformed by following steps:
– Step 1: Find the direction of 1st principal axes of segmented hand gesture.
– Step 2: Find the rotation angle between the 1st principal axes of the segmented hand gesture and vertical axes.
– Step 3: Rotate the segmented hand gesture,so that 1st principal axes of the segmented hand gesture coinside with vertical axes.
![Page 14: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/14.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 14
METHODOLOGY• Preprocessing
– Rotation
Fig. 3: Segmented and rotated hand gesture of‘A’and ‘B’
![Page 15: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/15.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 15
METHODOLOGY• Preprocessing
– Morphological filtering• A morphological filtering approach was
developed to obtain a smooth, closed, and complete contour of a gesture by using a sequence of dilation and erosion operations.
![Page 16: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/16.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 16
METHODOLOGY• Preprocessing
– Morphological filtering
![Page 17: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/17.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 17
METHODOLOGY• Preprocessing
– Morphological filtering
Fig. 4: Original, Segmented and Rotated and Filtered static hand gesture of‘A’and ‘B’
![Page 18: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/18.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 18
METHODOLOGY• Feature Extraction
– shape of the contour– contour tracking algorithm
![Page 19: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/19.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 19
METHODOLOGY• Feature Extraction
– The steps of the proposed contour tracking algorithm are given bellow:• Step 1: Scan the image from top to bottom
and left to right to find first contour pixel marked as P1(i, j).
![Page 20: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/20.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 20
METHODOLOGY• Feature Extraction
– The steps of the proposed contour tracking algorithm are given bellow:• Step 2: To scan clockwise direction searching
PJ sequences are PJ (i, j − 1), PJ (i − 1, j − 1), PJ (i − 1, j), PJ (i − 1, j + 1), PJ (i, j+1), PJ (i+1, j+1), PJ (i+1, j) and PJ (i+1, j−1) respectively.
![Page 21: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/21.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 21
METHODOLOGY• Feature Extraction
– The steps of the proposed contour tracking algorithm are given bellow:• Step 3: Scan clockwise until the next pixel
value PJ = 1.
![Page 22: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/22.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 22
METHODOLOGY• Feature Extraction
– The steps of the proposed contour tracking algorithm are given bellow:• Step 4: If PJ = 1, position of PJ != (i1, j1) and
position of PJ != (i2, j2), Store position of PJ into x and y arrays and set (i, j) = position of PJ . When J > 3 set (i1, j1) = position of PJ−1 and (i2, j2) = position of PJ−2.
![Page 23: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/23.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 23
METHODOLOGY• Feature Extraction
– The steps of the proposed contour tracking algorithm are given bellow:• Step 5: If step 4 become false then set x[J] =
x[J−1], y[J] = y[J − 1], (i, j) = position of PJ−1, (i1, j1) = position of PJ and (i2, j2) = position of PJ−2.
![Page 24: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/24.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 24
METHODOLOGY• Feature Extraction
– The steps of the proposed contour tracking algorithm are given bellow:• Step 6: Repeat steps 2 to 5 until position of
PJ = position of P1.
![Page 25: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/25.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 25
METHODOLOGY• Feature Extraction
– The LCS has the following important properties:• a) The LCS is not limited by shape complexity and it
is, therefore, appropriate for gestures which typically have convex and concave contours.
• b) The LCS has been used to robustly represent partial contours. Therefore, the representation of the visible part of the gesture will not be affected for a part of the gesture is obscured.
![Page 26: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/26.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 26
METHODOLOGY• Feature Extraction
– The LCS has the following important properties:• c) No derivative operation was involved during
computations; • d) The amplitudes of the samples of the localized
contour sequence proportionally increasing with w.
![Page 27: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/27.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 27
METHODOLOGY• Feature Extraction
Fig. 5: Computation of the samples of the LCS.
![Page 28: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/28.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 28
METHODOLOGY• Feature Extraction
![Page 29: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/29.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 29
METHODOLOGY• Classification
– In this paper, we proposed a k-means clustering based approach to determine the centers of the RBFNN.
![Page 30: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/30.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 30
METHODOLOGY• Classification
– Ourproposed approach is as follow:• Step 1: Set value of m = total number of
training classes and i = 1.• Step 2: Take Di ,the data of ith class.• Step 3: Set value of k = number of clusters
in Di.
![Page 31: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/31.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 31
METHODOLOGY• Classification
– RBFNN. Ourproposed approach is as follow:• Step 4: Assign initial centroid of each cluster
by selecting randomly k number of samples from data Di or first k number of samples from data Di.
• Step 5: Take each sample of data Di and compute its Euclidean distant from centroid of each of the cluster and assign to the cluster with the nearest centroid.
![Page 32: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/32.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 32
METHODOLOGY• Classification
– RBFNN. Ourproposed approach is as follow:• Step 6: Compute the centroid of each
cluster.• Step 7: Repeat step 5 to step 6 until centroid
of the each cluster don’t change.• Step 8: Store k number of centroid as
centers of RBFNN in ith iteration and increase i by 1.
![Page 33: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/33.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 33
METHODOLOGY• Classification
– RBFNN. Ourproposed approach is as follow:• Step 9: Repeat step 2 to step 8 until i <= m.
![Page 34: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/34.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 34
EXPERIMENT RESULTS AND DISCUSSION
Fig. 7: Danish/international sign language hand alphabets
![Page 35: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/35.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 35
EXPERIMENT RESULTS AND DISCUSSION
• The dataset is equally split into training and testing set.
![Page 36: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/36.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 36
EXPERIMENT RESULTS AND DISCUSSION
• The recognition rate can be defined as the ratio of the number of correctly classified samples to the total number of samples.
![Page 37: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/37.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 37
EXPERIMENT RESULTS AND DISCUSSION
Fig. 8: The variation of recognition efficiency with respect tonumber of hidden nodes.
![Page 38: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/38.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 38
EXPERIMENT RESULTS AND DISCUSSION
• With k-mean based RBFNN classifier, the proposed technique achieves 99.6% recognition accuracy when spread factor is 5.5.
![Page 39: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/39.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 39
EXPERIMENT RESULTS AND DISCUSSION
Fig. 9: The variation of recognition efficiency for differentspread factors.
![Page 40: 作者 : Dipak Kumar Ghosh , Samit Ari 出處 :IEEE 2011 報告者 : 邵建銘 MA1G0106](https://reader033.vdocuments.net/reader033/viewer/2022050805/5681458c550346895db27398/html5/thumbnails/40.jpg)
資訊工程系 智慧型系統實驗室 iLab南台科技大學 40
CONCLUSION• This work proposes a technique that
coincides the 1st principal component of the segmented hand gestures with vertical axes.
• K-mean based RBF neural network is also proposed as a classifier for recognition of hand alphabets from static hand gesture images.