A REAL-TIME HAND GESTURE RECOGNITION METHOD

SUBMITTED BY,

JAISON THOMAS

S7 ECE-A

ROLL NO : 48

GUIDED BY, SUNITHA S PILLAI ASST. PROFESSOR DEPT. OF ECE SJCET

2 Contents

• Introduction• Features of gesture recognition• Algorithms of hand gesture recognition• Different process of gesture recognition• Advantages• Applications• Conclusion• Future work

3 Introduction

Vision based hand gesture interface has been attracting more attentions due to no extra hardware requirement except camera, which is very suitable for emerging applications.

This method is not confined by aspect ratio of hand image and can deal with cluttered background.

Its also immune to camera movement in virtue of stable hand tracking.

4 What is Gesture ?

Non-verbal communication Gives message A gesture is a nonverbal

communication in which visible body communicates particular message.

Motion of body that contains information

5 Features of gesture recognition

Human computer interaction Gesture provides a way for computers to understand

human body language. Deals with the goal of interpreting human gestures via

mathematical algorithms. Enables humans to interface with the machine (HMI) and

interact naturally without any mechanical devices.

6 Few Hand Gestures

7 Our vision-based system

Wireless & Flexible No specialised hardwareSingle Camera Real-time

8 Algorithms of hand gesture recognition

1. 3D model-based algorithms2. Skeletal-based algorithms3. Appearance-based models

93D model-based algorithms

Describe hand movement and its shape. The software uses their relative position and interaction in

order to infer the gesture. There are some methods to obtain 3D model with 2D

appearance model. They are:

1.ISOSOM

2.PCA-ICA

10 Skeletal based algorithms

The skeletal version is effectively modelling the hand . This has fewer parameters than the volumetric version. It is easier to compute, making it suitable for real-time

gesture analysis systems.

11Appearance-based models

Technique is efficient but may be sensitive to different users and changes in scale and background.

The images represent typical input for appearance-based algorithms.

They are compared with different hand templates and if they match, the correspondent gesture is inferred.

12 Different process of gesture recognition

1. Hand detection2. Hand tracking3. Hand segmentation4. Gesture recognition

13 Hand detection

Hand detection is important for a gesture interface as it functions as a switch to turn on the interface.

Hand detection methods are sensitive to complicated background.

Hand detection uses extended Adaboost method.

14 Hand tracking

Texture or appearance based methods have been improved to be more robust for the non-rigid objects.

In this method, we use a multi-modal technique which combines optical flow and color cue to obtain stable hand tracking.

Flock of features method feasible in the articulated object tracking.

15 Hand segmentation

We use a single Gaussian model to describe hand colour in HSV colour space.

Histogram method is based on the assumption that no other exposed skin colour part of user in the certain area around the hand.

Wooden objects passing through the area, the histogram will deviate and segmentation results will be rapidly degraded. In that case our method can get better results.

16

Skin colour collect method

Hand segmentation results

17 Gesture recognition

Hand gestures using local oriental histogram feature distribution model, but background in experiments are quite simple and sleeve colour and texture are restricted.

Scale-space features detection have been widely applied in object recognition, image registering.

For planar hand shape, the scale-space feature detection can be used to detect blob and ridge structures, i.e. palm and finger structures.

In this method multi-scale feature detection with hand tracking and segmentation is used.

18

Blob and ridge detection of hand gestures

19 Advantages

Replace mouse and keyboard Pointing gestures Navigate in a virtual environment Pick up and manipulate virtual objects Interact with a 3D world No physical contact with computer Communicate at a distance

20 Applications Image controlling & Scaling To Control Mouse Sign Language Recognition Gaming Interface Robot Control Controlling Machines

21 Applications

Supermarkets Post Offices, Banks Allows control without having to

touch the device System Control and Image

Scaling

22 Conclusion

In this seminar we combines fast hand tracking, hand segmentation and multi-scale feature extraction to develop an accurate hand gesture recognition method.

This method has promising performance with various hand gesture posture under complicated backgrounds.

This take advantage of color and motion cues acquired during tracking to implement adaptive hand segmentation.

23 Future work

Current collaboration with Assistive Technology researchers and members of the Deaf community for continued design work should be considered for continued progress.

This system can be implemented in many application areas examples include accessing government websites whereby no video clip for deaf and mute is available or filling out forms online whereby no interpreter may be present to help.

24 References

Yikai Fang, Kongqiao Wang, Jian Cheng and Hanqing Lu, ‘Real-time hand gesture recognition method’ for National Lab of Automation, Chinese Academy of Sciences, Beijing (IEEE paper).

Y. Cui and J. Weng, “View-based hand segmentation and hand sequence recognition with complex backgrounds,” in Proceedings of 13th ICPR. Vienna, Austria, Aug. 1996, vol. 3, pp. 617– 621.

Mathias Kolsch, “Vision based hand gesture interfaces for wearable computing and virtual environments,” PHD Dissertation,UCSB, 2005.

25