kb gait-recognition

1

Gait Recognition from Video

Mahfuzul Haque

2

Background

Biometric Recognition Biometric recognition or simply biometrics refers to the automatic

recognition of individuals based on their physiological and/or

behavioral characteristics.

Modes of Biometric Recognition Verification

Identification

Goal To confirm an individual’s identity based on “who he is” rather than

“what he possesses” or “what he remembers”.

3

Various Biometric Characteristics

DNA

Ear (Cartilegenous tissue of the pinna are distintive)

Face

Facial, hand and hand vein infrared thermo gram

Fingerprint

Gait

Hand and finger geometry

Iris

Keystrokes

Odor

Palm print

Retinal Scan

Signature

Voice

4

Why Gait is getting more attention?

Non-contact human identification

Can be captured from a great distance

Great importance in security

5

Applications

Human recognition/identification

Activity recognition

6

Research approaches in recent works

Model based

Extracted image feature mapped to a model

Computational cost high

Motion based

Motion pattern converted to a compact representation

Lower complexity

Simpler implementation

Mixed approach

Combination of above two approaches

7

Available Databases

USF Human ID database

(http://www.gaitchallenge.org)

Southampton HiD database

(http://www.gait.ecs.soton.ac.uk)

CMU Mobo data set

http://www.gaitchallenge.org/

http://www.gait.ecs.soton.ac.uk/

8

Recent works

9

1. An introduction to biometric recognition

Jain, A.K.; Ross, A.; Prabhakar, S.;

Circuits and Systems for Video Technology, IEEE Transactions on

Volume 14, Issue 1, Jan. 2004 Page(s):4 – 20

Comments:

This paper presents a brief overview of the field of biometrics and summarizes some of its advantages,

disadvantages, strengths, limitations and related privacy concerns. Following areas have been highlighted:

•Architecture of different types of biometric system and various modules.

•Modes of biometric system: identification and verification.

•Various biometric system errors

•Comparison of various biometrics

•Applications of biometric systems

•Advantages and disadvantages of biometrics

•Limitations of biometric systems

•Multimodal biometric systems (Fusion)

•Social acceptance and privacy issue

In general, it’s good paper presenting a concrete overview of biometric system and various biometric

characteristics.

10

2. Fusion of static and dynamic body biometrics for gait recognition

Liang Wang; Huazhong Ning; Tieniu Tan; Weiming Hu;


Volume 14, Issue 2, Feb. 2004 Page(s):149 - 158

Comments:

•Human recognition algorithm by combining static and dynamic body biometrics.

•Static features: body height, build

•Dynamic features: Joint angle trajectories of main limbs – how the static silhouette shape changes over time

•Both static and dynamic information may be independently used for recognition using the nearest exemplar

pattern classifier

Results:

Experiment on 20 subjects demonstrates the feasibility of the approach.

11

3. Quantifying and recognizing human movement patterns from monocular video Images-

part I: a new framework for modeling human motion

Green, R.D.; Ling Guan;


Volume 14, Issue 2, Feb. 2004 Page(s):179 – 190

Comments:

This paper presents a framework which forms a basis for the general biometric analysis of continuous human motion and demonstrated

through tracking and recognition of hundreds of skills.

Techniques

Computer vision-based framework

CHMR (Continuous Human Movement Recognition) framework

3D color clone-body-model is dynamically sized and texture mapped to each person for more robust tracking of edges and textured

regions during initialization phase.

Automatic initialization

Estimation of joint angles for the next frame using a forward smoothing particle filter

35 Dynemes: units of full-body movement skills defined.

CHMR uses multiple Hidden Markov Model (HMM) to infer human movement skill

Initialization phase required

Automated initialization assumes only one person is walking upright in front of a static background

Video->[Special Segmentation, feature extraction] -> motion vectors ->[Temporal Segmentation, hypothesis search]

Dyneme model: motion vector sequence -> dyneme

Skill model: dyneme sequence -> skill

Context model: skill pair or triplet

Activity model: Skill sequence -> activity

Results

Recognition was processed using the HMM Tool Kit (HTK )96.8% recognition accuracy on the training set and 95.5% recognition

accuracy on the independent test set.

No standard data set has been used

Tools and Terms

HTK – The HTK is a portable toolkit for building and manipulating hidden Markov models.

Gaussian prior

Future Work

Expanding dyneme model to improve discrimination

Expanding clone body model to include a complete hand-model

Use of multi-camera multimodal vision system to better disambiguate the body parts

12

4. Quantifying and recognizing human movement patterns from monocular video images-

part II: applications to biometrics

Green, R.D.; Ling Guan;



Comments:

Presented as a continuation to part I of this series.

•Using the CHMR framework introduced in part I following applications have been demonstrated

•Biometric authentication of gait, Anthropometric data, Human activities and movement disorder

Techniques:

•Body part dimensions are quantified using the CHMR body model

•Gait signatures are then evaluated using motion vectors, temporally segmented by gait dynemes and projected

into a gait space for an Eigengait-based biometric authentication

•Left-right asymmetry of gait is also evaluated.

•CHMR activity model is used to identify various activities

•Movement disorders were evaluated by studying patients of Parkinsonism

Results:

•Anthropometric signature: 92%

•Gait Signature: 88%

•Fusion of both: 94%

•Activity: 0% error

•Parkinson’s Disease (PD): 95%

Future Works:

•Extending the models for loose clothing and carried items

•Increasing tracking stability by enhancing body models to include more degrees of freedom

•Improving the accuracy by increasing sample size

13

6. The humanID gait challenge problem: data sets, performance, and analysis

Sarkar, S.; Phillips, P.J.; Liu, Z.; Vega, I.R.; Grother, P.; Bowyer, K.W.;

Pattern Analysis and Machine Intelligence, IEEE Transactions on


Comments:

The paper presents a basis infrastructure for identification of people by analysis of gait patterns from video.

•The problem HumanID Gait Challenge has been introduced

•The challenge problem consists of a baseline algorithm, a set of 12 experiments and a large data set. Data

collected at University of Southern Florida in 2001.

•The dataset consists of 1870 sequences from 122 subjects spanning five covariates (1.2GB)

•The covariates are:

1. Shoe type

2. Carrying or not carrying a briefcase

3. Walking surface

4. Camera angle

5. Time

•All materials available at http://www.gaitchallenge.org

Techniques:

Estimating silhouettes by background subtraction and performing recognition by temporal correlation of

silhouettes.

Results:

•78% on easiest experiment, 3% on the hardest. Result benchmarked on the CMU Mobo data which is a

commonly used dataset for which performance has been reported in numerous papers.

•Tools and Terms:

•Gaussian Mixture Model (GMM)

•Mahalanobis Distance

•Expectation Maximization (EM)

•Gallery

•Probes/Signature

In general, a milestone contribution in the field of gait recognition. The infrastructure and data set provided by this

work has been used by other researchers afterwards.

http://www.gaitchallenge.org/

14

7. A video database of moving faces and people

O'Toole, A.J.; Harms, J.; Snow, S.L.; Hurst, D.R.; Pappas, M.R.; Ayyad, J.H.; Abdi, H.;Pattern

Analysis and Machine Intelligence, IEEE Transactions onVolume 27, Issue 5, May 2005

Page(s):812 – 816

Comments:

This paper describes a database of static images and video clips of human faces and people that is useful for

various researches.

•This work was supported by a grant from the Human ID project of DARPA/DOD

•Complete data sets for 284 subjects.

•Students from UTD participated as subjects.

•Gait videos: parallel and perpendicular

•160 GB HDD

15

8. Matching shape sequences in video with applications in human movement analysis

Veeraraghavan A; Roy-Chowdhury, A.K.; Chellappa, R.;


Volume 27, Issue 12, Dec. 2005 Page(s):1896 – 1909

Comments:

•An approach for comparing two sequences of deforming shapes

•Two methods: parametric (AR-Auto Regressive, ARMA-Auto Regressive Moving Average) and non-parametric

(DTW – Dynamic Time Wrapping)

•Data set: USF Human ID and CMU

•Results/Findings: Role of shapes & kinematics in human movement analysis from video

•Kundall’s definition of shape is used for feature extraction

•Shape deformations of a person’s silhouette as a discriminating feature

16

10. Human Gait Recognition With Matrix Representation

Xu, D.; Yan, S.; Tao, D.; Zhang, L.; Li, X.; Zhang, H.-J.;


Volume 16, Issue 7, July 2006 Page(s):896 – 903

Comments:

Matrix representation based approach has been proposed for Human Gait recognition

Techniques:

•Binary silhouettes over one gait cycle are averaged

•Each gait video sequence, containing a number of gait cycles, is represented by a series of gray level averaged

images

•Pre-processing step: Each gait video sequence, containing a number of gait cycles, is represented by a series of

gray level averaged images. Then a matrix based unsupervised algorithm namely coupled subspace analysis

(CSA) is employed to remove noise and most representative information.

•Final step: a supervised algorithm namely discriminant analysis with tensor representation is applied to further

improve classification ability.

Results:

Demonstrates a much better gait recognition performance than USF HumanID gait database.

17

11. A Bayesian Framework for Extracting Human Gait Using Strong Prior Knowledge

Ziheng Zhou; Prugel-Bennett, A.; Damper, R.I.;


Volume 28, Issue 11, Nov. 2006 Page(s):1738 - 1752

Comments:

This paper proposes a Bayesian framework for extracting human gait.

Techniques:

•Depends on strong prior knowledge and learning

•A consistent Bayesian framework has been proposed for introducing strong prior knowledge into the system

•Model considers both static and dynamic (time invariant/variant) parameters.

•Model is easily modified to cater situations such walkers wearing clothing that obscures the limbs

•Hidden Markov model is used to detect the phases of images in walking cycle.

•Dataset: Southampton Human Identification at a distance(HiD) Database has been used

Results:

•Results comparable with baseline algorithm

•Not every result is better than the baseline algorithm

This paper handles the situations “such walkers wearing clothing that obscures the limbs” which are not

addressed by previous works.

18

12. Detection of Gait Characteristics for Scene Registration in Video Surveillance System

Havasi, L.; Szlvik, Z.; Szirnyi, T.;

Image Processing, IEEE Transactions on

Volume 16, Issue 2, Feb. 2007 Page(s):503 - 510

Comments:

•Presents a robust walk detection algorithm based on symmetry approach to extract gait characteristics from video

image sequence.

•Demonstrated application in image registration.

Techniques:

•Invariant and effective data representation in the Eigenwalk space, based on spline interpolation and a dimension-

reduction technique.

Results:

•Reliable detection rate

19

Future research direction…

Improving performance and accuracy

Naturalistic context

Recognition with older database

..

..

20

Thank you