learning the space of time warping functions for activity recognition function-space of an activity...
Post on 21-Dec-2015
221 views
TRANSCRIPT
![Page 1: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/1.jpg)
Learning the space of time warping functions for Activity Recognition
Function-Space of an Activity
Ashok Veeraraghavan
Rama Chellappa
Amit K. Roy-Chowdhury
![Page 2: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/2.jpg)
Human Motion Analysis
[Johansson 1973]3D models of humans
with joint angles
Motion capture
[mocap.cs.cmu.edu]
Shape Sequences
[Veeraraghavan 2004]
![Page 3: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/3.jpg)
Human Action Recognition – Prior Work
Non-Imaging based methods : Motion capture, light displays etc
Point Light Displays
3D Motion capture
GPS Tracking
Anthropometry
View Invariance
Execution Rate
Modeling Point Trajectories
Local Affine Invariant Models
View invariant moments
3D models of human action
Modeling View-Transition
Gait - [Bobick et. al. 2003]
Activity - [Sheikh et. al. 2005]
Anthropometry Invariants
Dynamic Models
Shape Normalization
![Page 4: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/4.jpg)
Why is learning Execution rate important?
Sequence 1
Sequence 2
Average Sequence
Ghost Hands
Ghost Heads
Time Warped Average Sequence
StructuralInconsistencies
Structurally consistentaverage Sequence• Structural Inconsistencies. (2 heads, 4 arms etc.)
• Wrong match in recognition experiments.
• Algorithms attempt to explain temporal variation by modeling feature variation.
![Page 5: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/5.jpg)
Feature Space Time Warping Space
Model for an Activity
Possible Features: Silhouettes, 3D motion capture, shape of outline, body angles, location of joints in 3D/2D etc
Space of functions
Space of time warping functions
W3
Run
W1
Jog
W2 SitAa1(t): Jog
a2(t): Sit
![Page 6: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/6.jpg)
Modeling time-warping for ActivitiesModel For an Activity Nominal activity trajectory :
a(t); t ε (0,1)
W : Space of time warping functions.
Realizations of the activity
Also need to know how to sample candidate functions “f” from W.
Properties of functions in A1. All realizations of the activity starts
at time t=0 and ends at time t=1, i.e., f(0)=0 and f(1)=1.
2. The order of action units for each activity remains unaltered for all realizations i.e.,
3. We note that A is a convex set., i.e., if f1 and f2 are in A, then for α є (0,1)
f is also in A.
![Page 7: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/7.jpg)
Activity Specific time-warping space W W is a subset of A the space of time warping functions.
f(t) = t is a candidate function in W. This represents no time warping.
It is reasonable to assume that W is pointwise convex, i.e., for all f1,f2 Є W and α Є (0,1), f = α f1 + (1-α ) f2 is also in W.
Since the derivative is a linear operator, this means that if the rate of execution of some action can be speeded up by factors α1 and α2 then it can also be speeded up by any factor β in between α1 and α2. This is not just reasonable but in fact desirable.
![Page 8: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/8.jpg)
Activity Specific time-warping space W These properties imply that W
can be represented by a warping constraint window, given by two functions u(t) and l(t) where u(t) is the upper bounding function and l(t) is the lower bounding function.
Time warping functions f in the activity specific warping space W are such that
W
![Page 9: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/9.jpg)
From DTW to Activity Specific Warping Constraints
![Page 10: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/10.jpg)
Symmetric Representation of Activity Model
• Model parameters {a, W} non-unique.
• There exists an equivalence class of model parameters.
• Each equivalence class contains one member whose model parameters are symmetric.
• Learning and inference made only on symmetric representation of model in order to ensure uniqueness.
![Page 11: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/11.jpg)
Learning Symmetric Representation of Activity Model Learn the nominal activity trajectory a(t)
Learn the functional space of time-warps W.
Learning algorithm can be based on any chosen time alignment procedure.
We have used the Dynamic Time Warping (DTW) for time alignment.
EM based learning algorithm
Expectation
Maximization
![Page 12: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/12.jpg)
Learning the Model
![Page 13: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/13.jpg)
Pre-processing and Feature Extraction
1. Background Subtraction
2. Connected Component analysis
3. Extract Shape Feature. (Kendall’s statistical shape)
4. Shape Feature lives on a spherical shape-space.
5. Use appropriate distance measures like Procrustes distance for local distance computations.
[Veeraraghavan et. al.] CVPR 2004, PAMI 2005
![Page 14: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/14.jpg)
Results on USF Data 70 people, upto 10 sequences per person Variabilities: shoe type, surface, view point.
Figure: CMS curve for various algorithms on the USF database.
Baseline : [Sarkar 2005] DTW Shape, HMM Shape : [Veeraraghavan 2004] HMM Image : [Kale 2004]
![Page 15: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/15.jpg)
Activity Recognition on UMD Database
1. Database of 10 Activities and 10 Sequences per activity
2. 100 % Recognition and increase in discrimination in the similarity matrix compared to traditional DTW.
![Page 16: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/16.jpg)
Organize a large Database Hierarchically (Dendrogram)
1. USF Database. Total 1870 Sequences of 122 individuals.
2. No: of Leaves at every node = 3.
3. No: of Levels of Dendrogram = 3
![Page 17: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/17.jpg)
Conclusions Modeling, Learning and accounting for time
warping is important for activity recognition.
We proposed a convex activity specific function space for time warping functions and derived learning, recognition and clustering algorithms using this model.
Appropriate feature selection will enable view and anthropometry invariance.
![Page 20: Learning the space of time warping functions for Activity Recognition Function-Space of an Activity Ashok Veeraraghavan Rama Chellappa Amit K. Roy-Chowdhury](https://reader030.vdocuments.net/reader030/viewer/2022032522/56649d625503460f94a45392/html5/thumbnails/20.jpg)
Human Action Recognition – Prior Work
Non-Imaging based methods : Motion capture, light displays etc
[Murray 1964]
[Johansson1973]
[Cunado et. al. 1994]
Anthropometry
View Invariance
Execution Rate
[Seitz et. al. 1997]
[Yacoob et. al. 1998]
[Rao et. al. 2002]
[Parameswaran et. al. 2003]
[Syeda-Mahmood et. al. 2001]
[Bobick et. al. 2003]
[Sheikh et. al. 2005]
[Bissacco et. al. 2001]
[Gritai et. al. 2004][Kale et. al. 2004]
[Veeraraghavan et. al. 2004]