Modeling the Model Athlete

Automatic Coaching of Rowing Technique

Simon Fothergill,

Fourth Year Ph.D. student,

Digital Technology Group, Computer Laboratory,

University of Cambridge

DTG Monday Meeting, 10th November 2008

Based on paper:Modelling the Model Athlete : Automatic Coaching of Rowing Technique; Simon Fothergill,

Rob Harle, Sean Holden; S+SSPR08; Orlando, Florida, USA, December 2008

Supplementary Sports coaching

• Feedback is vital

• Rowing technique is complex, precise and easy to capture

• Good coaches aren’t enough

• Sensor signals need interpreting

• Biomechanical rules are complex and require specific sensors, if they exist at all

Pattern Recognition

• StatisticalArbitrary features that summerise the data in some way. E.g. RGB values, number of X

• StructuralConsider constituent parts and how they are related. E.g. “contains”, “above”, “more red”

• Combination– Distance – Shape moments / smoothness

System overview

Population of

strokesStroke quality classifier

Good

Bad

Individual

aspect of

technique

Motion capture system

Lightweight markers

Preprocessing of motion

data

Feature extraction

Classification

stroke

Motion capture

• Bat system• Inertial sensors• Optical motion capture

– VICON– Nintendo Wii controllers

Preprocessing• Compensate for occlusions• Transform to the “erg co-ordinate system” defined by seat • Segment performance into strokes using handle trajectory extremities

Feature extraction

• Art

• Modified various algorithms until found “a good one” for a set of strokes where each stroke is obviously different in over-all quality.

Abstract features• Length

• Height

• Distance

• Shape moments (λ11, λ12, λ21, λ02, λ20)

• Speed moments: (μ 11, μ 12, μ 21, μ 02, μ20)

ψ(s)

Physical Performance features

• Wobble (lateral variance)

• Speed smoothness – μ-subtract,– LPF (3Hz) – dS/dt/dt, – ∑

• Shape smoothness – LPF (6Hz), – |dS/dt/dt|, – ++ > threshold (0.4ms-2)

Domain features

• Ratio (drive time : recovery time)

• Drive and recovery angles

System overview

Population of

strokesStroke quality classifier

Good

Bad

Individual

aspect of

technique

Motion capture system

Lightweight markers

Preprocessing of motion

data

Feature extraction

Classification

stroke

Machine learning

• Normalisation and Negation

– Each feature’s values are normalised to roughly between 0 and 1

– Highly negatively correlated features are negated

– Good strokes are scored as 1– Bad strokes are scored as 0

Machine learningClassification

Feature 0

Feature N

Weight 0

Weight N

Linear combination

Bias

Bias weight

Composite representation of motion

Method 1: Moore-Penrose

F w = s

(F-1 = Moore-Penrose pseudo-inverse of feature matrix)

Method 2: Gradient descent

Error function: Sum of the square of the differences

weights initialised to 0

750 iterations

0.001 learning rate

Machine learning• Validation of models

– Training repeated using populations formed by leaving out different sets of strokes

– Unseen strokes are then classified– Each stroke left out exactly once– Multiple performers (each performer left out)

• Sensitivity analysis– Threshold computed to

minimise misclassification– Features– Iterations

Empirical Validation• Population

– Six novice, male rowers in their mid-twenties– 60kg and 90kg – Very little or no rowing experience. – Not initially fatigued, comfortable rate, uncontrived manner.

• Scoring– Single expert (coach)– Score whole performances (95% representative)– Bad = Expert considers a significant floor in technique– Good = Expert considers a noticeable improvement

• Experimental method– Basic explanation– Give performance (~30 strokes)– Repeat to fatigue

• Identify fault• Teach correction• Give performance (~30 strokes) whilst coach helps to maintain improved technique (for

accumulating aspects)

Empirical Validation• For an Individual and specific aspect

– Training just that single aspects– Recognition of that single aspects with realistic combinations of different qualities for

different aspects

RowerCoached aspect

(chronological order)

Moore-Penrose training Gradient Descent training

Single aspect All aspects Single aspect All aspects

1Separate arms/legs 0 0 0 0

Overreaching 0 0 0 1

2 Separate arms/legs 0 - 3 -

3Separate arms/legs 0 0 0 0

Overreaching 0 0 2 1

4

Overreaching 0 0 0 0

Shins vertical 0 0 0 0

Early open back 0 0 2 1

5

Leaning back 0 3 0 6

Quick hands 0 4 0 7

Rushing slide 0 2 0 6

Early open back 3 0 5 0

6

Overreaching 0 0 0 0

Separate arms/legs 0 0 1 3

Quick hands 0 0 1 1

Empirical Validation

• Across Individuals

Rowers Aspect Moore-Penrose training Gradient Descent training

2 Quick hands 9 5

2 Early open back 33 29

3 Separate arms/legs 21 21

4 Overreaching 12 12

Discussion and Conclusions

• Useful featuresλ02, λ20 μ02 and μ20 used in at least 90% of the final feature sets for both algorithms.

• Comparison of techniques– For single athletes,

gradient descent not as fast

– For multiple athletes, gradient descent more reliable

• Encouragingly low misclassification

• Suggets inter-variation from different athletes > athlete’s intra-variation

Further Work

• Characterisation of the process– Population– Domain– Algorithms

• Reversing the models to allow prediction of optimal individual aspects of technique that can be merged to an optimal technique for an individual

References• Modelling the Model Athlete : Automatic Coaching of Rowing Technique; Simon Fothergill, Rob

Harle, Sean Holden; S+SSPR08; Orlando, Florida, USA, December 2008

– Ilg, Mezger & Giese. Estimation of Skill Levels in Sports Based on Hierarchical Spatio-Temporal Correspondences. DAGM 2003, LNCS 2781, pp. 523-531, 2003.

– Murphy, Vignes, Yuh, Okamura. Automatic Motion Recognition and Skill Evaluation for Dynamic Tasks. EuroHaptics 2003, 2003.

– Gordon. Automated Video Assessment of Human Performance. J. Greer (ed) Proceedings of AI-ED 95. pp. 541-546, 1995.

– Rosen, Solazzo, Hannaford & Sinanan. Objective Laparoscopic Skills Assessments of Surgical Residents Using Hidden Markov Models Based on Haptic Information and Tool/Tissue Interactions. The Ninth Conference on Medicine Meets Virtual Reality, 2001.

• Joint IAPR International Workshops on Structural and Syntactic Pattern Recognition and Statistical Techniques in Pattern Recognition (S+SSPR 2008) Orlando, Florida, USA, December 4-6, 2008 (http://ml.eecs.ucf.edu/ssspr/index.php)

• 19th International Conference of Pattern Recognition, ICPR 2008 (http://www.icpr2008.org/)

• Computer Laboratory, University of Cambridge (www.cl.cam.ac.uk)

Acknowledgements

• Professor Andy Hopper• Dr Sean Holden• Dr Rob Harle• Dr Joseph Newman• Brian Jones• Dr Mbou Eyole-Monono • The Digital Technology Group, Computer Laboratory• The Rainbow Group, Computer Laboratory

Thank you!

Questions?