3d hand tracking
DESCRIPTION
3D hand tracking using depth sensor/Kinect.TRANSCRIPT
04/13/2023 1
A Unified Framework for 3D Hand Tracking
Rudra PoudelJose A S FonsecaJian J ZhangHammadi Nait-Charif
ISVC 2013
04/13/2023Next Introduction 2
Overview1. Introduction2. Applications3. Related Work4. Skin Colour Detection5. Hand Region Segmentation6. Hand Pose Estimation7. 3D Hand Tracking8. Conclusions
04/13/2023Continue … 3
1. Introduction• Aim: detect 3D
position of hand joints• Task in details:
o Hand detection (skin colour, hand shape)
o Joints detectiono Estimate 3D positions of the
joints
• Problems: appearance similarity, occlusion, random motions, 27 degree-of-freedom/dimensions
04/13/2023Next Applications 4
1. Introduction
Overview of the system
04/13/2023 5
2. Applications
Next Related Work
• Human machine/computer interaction- TV remote controller
• Sign language recognition• Gamming• Health and Medicine• Many more …
04/13/2023Continue … 6
3. Related Work• Computer vision is
holy grail of Artificial Intelligent/Robotics
• 3D Hand tracking deals with three major problems- detection, recognition, tracking
• Research started two decades back
Starner et al. (PAMI, 1998)
04/13/2023Next Skin Color Detection 7
3. Related Work• Oikonomidis, I., Kyriazis, N., Argyros, A.A.:
Efficient model-based 3d tracking of hand articulations using kinect. In: BMVC. (2011): template matching so computationally very expensive
• Keskin, C., Kirac, F., Kara, Y., Akarun, L.: Real time hand pose estimation using depth sensors. In: ICCV Workshops. (2011) 1228–1234: needed extra occlusion handling technique, hasn’t utilizes temporal information
04/13/2023 8
4. Skin Colour Detection
Next Hand Region Segmentation
• Learned two skin and non-skin histograms using- 4,700 skin and 9,000 non-skin images
• Manually labelled pixels is nearly 1 billion
• Includes all lighting conditions and races
04/13/2023 9
5. Hand Region Segmentation
Next Hand Pose Estimation
• Kinect depth image • Segmented hand using skin colour, depth and hand at previous frame
04/13/2023Continue … 10
6. Hand Pose Estimation
Learned hand parts classifier using 450 thousands artificial images
Example of hand parts detection
04/13/2023Continue … 11
6. Hand Pose Estimation
• Regression of joint positions using hough voting technique
04/13/2023Continue … 12
6. Hand Pose Estimation
Comparison: our vs. state-of-the-art Keskin (2011)
04/13/2023 13
6. Hand Pose Estimation
Example of hand pose estimation
Next 3D Hand Tracking
04/13/2023 14
7. 3D Hand Tracking
Next Conclusion
• Tracking using- hand joints estimation at current frame (t), temporal information and kinematic constraints
• 3D hand tracking demo: http://www.youtube.com/watch?v=xqyfWWlAnVI
04/13/2023 15
8. Conclusion• Depth sensor Kinect: RGB and depth images• Skin colour detection• Hand segmentation using skin colour, depth and
hand position at previous frame• Hand joints regression• 3D hand tracking using joints detection, temporal
information and kinematic constraints
• Used techniques- Naive Bayes for skin detection, Random Forest for joints detection, Mean-shift for joint vote aggression, Markov Random Fields for tracking
Next Questions
04/13/2023 16
Questions ?
Thank you !!!
9th International Symposium on Visual Computing