one-shot learning gesture recognition

36
One-Shot Learning Gesture Recognition Students: Itay Hubara Amit Nishry Supervisor: Maayan Harel Gal-On

Upload: cruz

Post on 24-Feb-2016

86 views

Category:

Documents


0 download

DESCRIPTION

One-Shot Learning Gesture Recognition. Students: Itay Hubara Amit Nishry Supervisor: Maayan Harel Gal-On. Outline. Background. Gesture recognition is a strong upcoming field in computer vision - PowerPoint PPT Presentation

TRANSCRIPT

PowerPoint Presentation

One-Shot LearningGesture RecognitionStudents:Itay HubaraAmit Nishry

Supervisor:Maayan Harel Gal-On

Outline2BackgroundGesture recognition is a strong upcoming field in computer visionGesture recognition can be seen as a way for computers to begin to understand human body language3

3MotivationExisting Gesture recognition demand a long configuration and trainingDifferent Gestures are been solved using different approaches

44GoalsLearn and understand existing Gesture recognition algorithms.Compare different approachesDesign Gesture recognition algorithm which reduces training time

55DataThe Data is compose from several set each contains:

Gesture vocabulary (learning set) which contain only one sample per gesture.Test set which contain one or more gestures.

Each of the sets has different vocabulary features such as large/small gesture hand/legs movement etc.

6Data Train7Base gesture

Data Test8

Multiple base gesturesLarge movementsData - Test9

Multiple base gesturesSmall movementsChallengesOne shot learning - only one learning sample (unlike the common approach of multi class classification)Tests videos segmentationSame gesture can have different number of framesEach set has different features (small/big gestures)1010Outline11Reduced ProblemAssume that each of the test movies has only one gesture

Goal:finding features space and distance function which have good separation of the features space

1212Problem ApproachClassic machine learning problemSelect FeatureOne Shot Match using similarity function

13FeaturesMotion Energysubtracting consecutive framesSpace Quantization14

14FeaturesHarris Corner DetectorFind interest point in the difference image based on corner detectionSpace Time Interest PointsExtend Harris to the time domain

15

15FeaturesHarris Corner DetectorFind interest point in the image based on corner detection

16

16FeaturesSpace Time Interest Points17

17Features18

STIPHarris18FeaturesHead Relative Interest Points19

19Features20

Interest pointsHead Histogram20Distance FunctionsGood features space is defined not only by the features but also by the distance (similarity) function

Different features need different distance functions

2121Principal Motion Using PCAUsing principal component analysis (PCA), to find the main motion vectors.For test set - project feature onto each of train principals and evaluate similarity22

22Earth Moving Distance Given two sets of distribution, EMD will measure the minimum cost to shift dirt from one distribution to the other.23

23Perturbed Variations24

Given two sets of distribution and predefined value of permitted variations optimally perturbs the distribution to best fit each other.

Transportationproblem underpermitted variations constrain

24Perturbed Variations25

25Levenshtein DistanceMeasure the difference between two sequences.Consider lengths and classification.26

26Results27Results28Top 10Top 2028Results29Outline30Complete ProblemSeparate problems Basic Segmentation (equal/movement)Whole problem solving approachMoving WindowDynamic Time Warping (DTW)

3131Problem ApproachThree different method to solve the problem:Basic Segmentation (equal/movement)Moving WindowDynamic Time Warping (DTW)

3232Moving WindowMove a window along the test video.Assume each window frames has only one gesture Preform basic analysis as did before to and build the distance matrix33

33Moving Window34

34Moving WindowAfter Sorting the Distance matrix we extract labels and cuts35

Several other operation (such as smoothing) are done before extracting the final resultDynamic Time Warping Create a state machine from train data:Module standing positionForm standing position can move to start of base gesturesAssume we can move forward, or stay in the same sate.For a given gesture find the best path along the sate machine3636Dynamic Time Warping 37

Results3838Results3939Results4040Results41Top 10Top 2041Outline42ConclusionsEach approach receive better results in different feature and similarity function

Different algorithms has different strengths (segmentation\recognition)

Segmentation require standing position model.

43Conclusions44Pre-processing unsupervised algorithms help better representing the data.

There is still allot left to do on the field

Future WorkTry different models for the standing position to improve segmentation results

Try combing DTW for segmentation and PCA for recognition.

Use different unsupervised algorithms to better represent the data.

45ReferencesIvan Laptev, "On Space-Time Interest Points, 2005Hugo Jair Escalantea and Isabelle Guyonb, "Principal motion: PCA-based reconstruction of motion histograms

M.Harel, S.Manor, "The Perturbed Variation, NIPS 2012Elizaveta Levina, Peter Bickel Department of Statistics, The EarthMovers Distance is the Mallows Distance: Some Insights from Statistics.Ofir Pele,Michael Werman, Fast and Robust Earth Movers Distances.2008

4646