Video Based Gait Analysis in Biometric person

authentication

Jani Rönkkönen

Gait in general

Shortly gait means the walking style of a person

Gait signature of each person is unique and thus can be used as a biometric

To form a gait signature many different components like cadence or frequency of walking can be used

Most common way of receiving gait information is by video cameras (others for example radar, pressure mat or motion sensors)

Current status

Quite a lot of published articles in recent yearsResearch is still mostly basic researchNo commercial solutions yet for authentication

purposes (May be some medical applications)Most promising areas are medical and surveillance

applicationsGeorgia Institute of Technology is developing a

method for recognizing people among crowd and estimate it could be commercialised in five years

Advantages

Can be used with low resolution video sequences

Target do not necessarily need to know about the surveillance

Sequences can be taken from long distance

Non intrusive

Gait is not very easily conceivable biometric (although can be altered purposely)

Disadvantages

The uniqueness of a persons gait signature is not proven with large datasets

Not yet clear which components of gait signature are most useful

A lot of data usually means high computational cost

Gait may be changed purposely

Conditions may affect the gait signature more than differences between subjects

Conditions

Walking speed• Affects cadence, stride lenght, frequency, pose

and hand swingsWalking surface• If surface is not smooth and obstacle free gait

pattern will no longer be repeatable an periodicPhysical Conditions• For example pregnancy, drunkness, fatigue or

physical injury

Conditions

Carrying a load• Carrying a load affects both the gait dynamics

and physical borders of a person

Clothes• Clothes alter the borders of a person and may

hide some movement (a dress for example)• footwear affect the gait dynamics (rubber boots vs

high heels)

Camera conditions

Camera angle• The gait pattern is very different if looked from

different anglesLightning conditions• Shadows cause error in border or silhouette

extractionContrast between clothing and backround• Too small contrast makes it harder to extract

borders or sihouette

History

G. Johansson used small light bulbs attached to a person to generate 2D motion patterns in 1973

He found out that people could recognize these to represent human movement

Later it was discovered that humans could also recognize the gender of the walker or even their friends identity from these patterns

A conclusion was drawn that gait could be used as a biometric

UMD database

University of Maryland (UMD) database• First dataset

– 25 persons and 4 camera angles• Second dataset

– 55 persons and 2 camera angles

CMU database

Carnegie Mellon University (CMU) database• 25 persons and 6 camera angles• Slow, fast and inclined walking style • A walk holding a ball

USF and USH databases

University of South Florida (USF) database• 71 persons and 2 camera angles• 2 different shoetypes and walking surfaces

(concrete and grass)• Walk holding a briefcaseUniversity of Southampton (USH) database• 28 persons and 1 camera angle• Uniform green backround to help extract clean

silhouettes

Features

Gait signature includes numerous components and even more features can be derived from these

Not yet clear which features are most usefulfor authentication purposes

Why not just simply use all there is?

Why not all features?

• Would be computationally costly, need much storage space and need complex algorithms

• Conditions affect different features differently– Some are more robust to changes

• Using more features may decrease performance– Bad ones only add unwanted noise

• All features are not always available

Methods

Another question is how to use our feature(s) of choice for authentication?

Typically there is three main steps in a gait recognition algorithm:

• Extracting the subject from the frames of video sequence (eg. silhouette)

• Extracting and modifying the wanted features (eg. PCA to simplify the data)

• Classification based on extracted features and somekind of decision (eg. KNN-classifier)

The width of outer contour

The basic biometric is the width of the outer contour of binarized silhouette of a walking person

• Retains physical structure and swings of the limbs during walking

• The pose information is lost• Smoothed and down sampled width vectors are

used directly• Also a velocity profile is extrated by calculating

the difference of subsequent vidth vectors

Overlay of width vectors

Results

UMD database• rank 1: 80% and rank 5: 91.2%• Velocity profile alone

– rank 1: 56% and rank 5: 83%CMU database• rank 1: over 95% and rank 5: 100%• Fast vs slow

– rank 1: 75% and rank 5: 87.5%

Results with USF database

G = Grass

C = Concrete

A, B = Footwear

R, L = Camera angle

Notes

Both structural and dynamical information is important fo recognition

Leg region is the most important

Difference in walking surface causes a lot of problems to the method

Walking speed is also an issue

Moments from silhouette

Silhouette is divided in 7 parts

For each part an ellipse is fitted

Features:• Centroid, Major axis, minor axis and Major axis

orientation• Height of the body• Another testset used gait spectral component

features received via fourier transform

Divided silhouette

Results

Tested on dataset consisting of 24 persons • Sequences were taken in 4 different days• Sequences of one day were compared against

other days• Rank 1: 30% - 47% and rank 10%: 53%-94%

With spectral component features• Rank 1: 31% - 82% and rank 10%: 70%-97%

Results

Also tested with CMU database• Results were almost perfect (only one mistake)

Third case was gender identification• Support vector machines• Best results using second degree polynomial

kernel 94% correctness

Notes

Most errors caused by clothing changes

Spectral component features were more robust

If only sequences taken in a same day were compared, spectral component features were slightly worse

Notably good result of gender classification

Body shape and gait from silhouette

The periodic dimensional changes in silhouette width are used to locate the key frames

Key frames are compared to corresponding ones in training data

Four subsequent comparison scores are amalgated and used for classification

Extracting the key frames

Results

CMU database• Rank 1: at least 92%

Slow vs fast• rank 1: 76% and rank 10%: 92%

Second testset contained 25 persons with sequences taken in different days

• rank 1: 45% and rank 10%: 77%

Notes

The main reason for failures (according to the author) were conditions that affected the quality of the silhouette

• Lightning conditions• Clothes• Hairstyles

Stride length and cadence

The method makes following assumptions:• Walking velocity is constant• Persons walks a straight line for 10-15 seconds• Camera is calibrated with respect to the ground

plane• Frame rate is greater than twice the walking

frequency

Stride lenght and cadence

The key is the periodicity of human walk

The width of a sihouette is used to calculate the period

From period a number of steps can be calculated

Also the distance walked is measured• Stride = distance/steps• Cadence = steps/time

Results

A database of 131 sequences from 17 persons

Notes

The method (according to author) is robust to changes in:

• Lightning conditions• Clothing• Tracking error

Also it is in principle wiev invariant, but the method used to calculate the frequency works best with fronto parallel sequences

Similarity plots

A sequence of images of a walking person is mapped to a similarity plot (SP)

Similarity plots

Each pixel in SP is a result of substracting two blobs• SP has dark main diagonal, because comparing a

blob to itself results to zero• Is symmetric along main dianonal• Is periodic, because from key poses A and C and

B and D are close to each other

Results

A dataset of 44 images of 6 persons from one camera angle

• 90% accuracy with rank 1

The second dataset consisted of 400 sequences of 7 peoples from 8 camera angles and 7 different days

• 65% accuracy with rank 1

Notes

The method is view dependent and performs best with fronto-parallel sequences

Changes in clothing and lightning affect performance

The author used binary blobs and gray level blobs with and without backround

• Best results with binary blobs, worst with gray level blobs with backround

Thigh and lower leg rotation

A sobel edge operator is used to obtain the leading edge of a walking person

The model

A model is then matched to the edge

Phase weighted magnitude

A phase weighted magnitude (PWM) is calculated from this with the help of fourier transform

Results

A dataset of 20 persons walkin and running

In addition to clean edged images, 25% grey scale random noise were added and also classification was done by decreasing the resolution (from 130*190 pixels to 65*95 pixels)

Best results were achieved with clean edged running sequences rank 1: 91.7% and worst with noisy walking sequences 60.8%

Notes

Reducing resolution did not reduce performance dramatically

Adding noise reduced to worse results

The reson for running sequences having slightly better identification accuracy was according to author the fact that there is more differeces between humans running styles than walking styles

Conclusions

Rather good results of identification can already be acquired with small datasets and fixed conditions

More robust methods are needed to achieve better accuracy in more general setup

A question still remains if gait is unique enough with larger datasets