towards a wearable snowboarding assistant · this thesis is the initial step to develop a wearable...

by Christian Guggenmos

Towards a Wearable

Snowboarding Assistant

Diploma Thesis at the Media Computing Group Prof. Dr. Jan Borchers Computer Science Department RWTH Aachen University

Thesis advisor:Prof. Dr. Jan Borchers

Second examiner:Prof. Dr.-Ing. Klaus Wehrle

Registration date: June 14th, 2007Submission date: Dec 20th, 2007

iii

Contents

Abstract xv

Überblick xvii

Acknowledgements xix

Conventions xxi

1 Introduction 1

1.1 A Wearable Snowboarding Assistant . . . . . 2

1.1.1 Scenario of Typical SnowboardingLessons . . . . . . . . . . . . . . . . . . 3

1.1.2 Scenario of Snowboarding Lessonswith the Snowboarding Assistant . . . . 5

1.1.3 Goals . . . . . . . . . . . . . . . . . . . 6

1.1.4 Requirements . . . . . . . . . . . . . . 7

1.2 Structure of the Thesis . . . . . . . . . . . . . 8

2 Sensor Technology 11

2.1 Accelerometer . . . . . . . . . . . . . . . . . . 11

iv Contents

2.2 Gyroscope . . . . . . . . . . . . . . . . . . . . 13

2.3 Force Sensitive Resistor (FSR) . . . . . . . . . 15

2.4 Bend Sensor . . . . . . . . . . . . . . . . . . . 16

2.5 Inertial Measurement Unit (IMU) . . . . . . . 17

3 Related Work 19

3.1 Context-Awareness . . . . . . . . . . . . . . . 19

3.1.1 Definition and Examples . . . . . . . . 19

3.1.2 Multi-Sensor Activity Context Detec-tion for Wearable Computing . . . . . 20

3.2 Health Care . . . . . . . . . . . . . . . . . . . 22

3.2.1 GaitShoe . . . . . . . . . . . . . . . . . 22

3.2.2 Biofeedback Wireless Wearable System 24

3.2.3 TactaPacks . . . . . . . . . . . . . . . . 26

3.3 Sports . . . . . . . . . . . . . . . . . . . . . . . 27

3.3.1 Wireless Force Sensing Body Protec-tors for Martial Arts . . . . . . . . . . 27

3.3.2 Towards Recognizing Tai Chi . . . . . 28

3.3.3 Audiofeedback for Karate Training . . 28

3.3.4 Combining Body Sensors and VisualSensors for Motion Training . . . . . . 29

3.3.5 Wearable Sensing System for Profes-sional Downhill Skiing . . . . . . . . . 31

3.4 Summary and Discussion . . . . . . . . . . . 32

3.4.1 Systems for Outdoor Use . . . . . . . 33

Contents v

3.4.2 Training Systems . . . . . . . . . . . . 34

3.4.3 Monitoring Systems . . . . . . . . . . 34

3.4.4 Comparison . . . . . . . . . . . . . . . 35

4 The Snowboarding Domain 37

4.1 Snowboarding Terms and Techniques . . . . 38

4.2 Literature and Interview Findings . . . . . . 41

4.2.1 Common Beginner Mistakes . . . . . 42

4.2.2 Further Aspects . . . . . . . . . . . . . 45

5 A Lab Prototype 49

5.1 Hardware Setup . . . . . . . . . . . . . . . . . 50

5.1.1 Sensor Interface . . . . . . . . . . . . . 50

5.1.2 Sensor Types and Locations . . . . . . 52

5.1.3 Additional Setup . . . . . . . . . . . . 55

5.2 Design for Mistake Detection . . . . . . . . . 55

5.2.1 Knee Bending . . . . . . . . . . . . . . 56

5.2.2 Upper Body Posture . . . . . . . . . . 57

5.2.3 Weight Distribution . . . . . . . . . . 58

5.2.4 Combining Basic Information to De-rive More Complex Mistakes . . . . . 63

5.2.5 Providing Feedback . . . . . . . . . . 65

5.3 Implementation . . . . . . . . . . . . . . . . . 65

5.3.1 Max/MSP Patches . . . . . . . . . . . 65

vi Contents

5.3.2 Discussion . . . . . . . . . . . . . . . . 71

5.4 Test and Findings . . . . . . . . . . . . . . . . 72

6 A Mobile Prototype for the Slope 75

6.1 Hardware Setup . . . . . . . . . . . . . . . . . 76

6.1.1 Sensor Interface . . . . . . . . . . . . . 76

6.1.2 Robust Casing . . . . . . . . . . . . . . 77

6.1.3 Sensor Attachment . . . . . . . . . . . 77

6.2 Software . . . . . . . . . . . . . . . . . . . . . 78

6.2.1 Wireless Communication . . . . . . . 78

6.2.2 Off-line Sensor Data Analysis . . . . . 80

6.3 Initial Self Tests on the Slope . . . . . . . . . . 81

6.3.1 Problems with the Hardware . . . . . 82

6.3.2 Successful Setup Features . . . . . . . 83

6.3.3 Summary . . . . . . . . . . . . . . . . 83

6.4 Improvements . . . . . . . . . . . . . . . . . . 85

6.4.1 Bend Sensors . . . . . . . . . . . . . . 85

6.4.2 Detecting Counter-Rotation . . . . . . 85

6.5 Final Setup . . . . . . . . . . . . . . . . . . . . 87

7 User Study and Data Analysis 91

7.1 Tests with Snowboard Beginners . . . . . . . 91

7.1.1 Test Subjects . . . . . . . . . . . . . . . 92

Contents vii

7.1.2 Test Procedure . . . . . . . . . . . . . 92

7.2 Approaches to Detect Mistakes . . . . . . . . 93

7.2.1 Weight Distribution . . . . . . . . . . 95

7.2.2 Knee Bending . . . . . . . . . . . . . . 102

7.2.3 Upper Body Posture . . . . . . . . . . 104

7.2.4 Counter-Rotation . . . . . . . . . . . . 105

7.3 Summary . . . . . . . . . . . . . . . . . . . . . 109

8 Summary and Future Work 113

8.1 Summary and Contributions . . . . . . . . . 113

8.2 Future Work . . . . . . . . . . . . . . . . . . . 115

A Interview Guideline 117

A.1 Interview Guideline (German) . . . . . . . . . 117

A.2 Interview Guideline (English) . . . . . . . . . 120

B MAX/MSP Patches 123

B.1 MyCubePatch.pat . . . . . . . . . . . . . . 123

B.2 MyEventReceiver.pat . . . . . . . . . . . 124

B.3 MyFeedbackGenerator.pat . . . . . . . . 125

C Smoothing Filters 127

C.1 Simple Moving Average (SMA) . . . . . . . . 127

C.2 Exponential Moving Average (EMA) . . . . . 128

viii Contents

Bibliography 129

Index 135

ix

List of Figures

1.1 Tennis lesson . . . . . . . . . . . . . . . . . . . 2

2.1 Working principle of an accelerometer . . . . 12

2.2 Gravitational acceleration on an accelerometer 13

2.3 Explanation of the Coriolis acceleration[Geen and Krakauer, 2003] . . . . . . . . . . . 14

2.4 Working principle of a MEMS gyroscope[Geen and Krakauer, 2003] . . . . . . . . . . . 14

2.5 Working principle of a force sensitive resistor(FSR) . . . . . . . . . . . . . . . . . . . . . . . 15

2.6 Working principle of a bend sensor . . . . . . 17

2.7 SHAKE SK6 and its axes . . . . . . . . . . . . 18

3.1 Overview of the wearable system introducedin [Kern and Schiele, 2003] . . . . . . . . . . . 21

3.2 Overview of the GaitShoe [Paradiso et al., 2004] 23

3.3 Outline of the Bio-WWS [Brunelli et al., 2006] 25

3.4 Outline and picture of a TactaPack [Lindemanet al., 2006] . . . . . . . . . . . . . . . . . . . . 26

x List of Figures

3.5 Overview of the motion training system in-troduced in [Kwon and Gross, 2005] . . . . . 30

3.6 Wearable system for professional skiing[Michahelles et al., 2005] . . . . . . . . . . . . 32

3.7 Sensor data viewer developed by Micha-helles et al. [2005] . . . . . . . . . . . . . . . . 33

4.1 Two snowboarders with reversed posture . . 38

4.2 Sketch of a snowboard and important terms . 39

4.3 Basic stance on a snowboard [Reil et al., 2003] 40

4.4 Stages of a frontside turn [Reil et al., 2003] . . 41

4.5 Snowboarder with incorrect body posture[Reil et al., 2003] . . . . . . . . . . . . . . . . . 43

4.6 Snowboarder with too much weight on theback foot [Reil et al., 2003] . . . . . . . . . . . 44

4.7 Snowboarder performing turn with counter-rotation . . . . . . . . . . . . . . . . . . . . . . 45

5.1 FSRs taped on a shoe insole . . . . . . . . . . 53

5.2 Bend sensors attached to the back of the kneevia velcro straps . . . . . . . . . . . . . . . . . 54

5.3 Hardware setup for the lab prototype . . . . 56

5.4 Measuring upper body tilt with an ac-celerometer . . . . . . . . . . . . . . . . . . . . 57

5.5 Weight distribution under the feet during turns 59

5.6 Placement and denomination of the FSRs un-der the feet . . . . . . . . . . . . . . . . . . . . 60

5.7 Max/MSP patch with iCube object . . . . . . 66

List of Figures xi

5.8 Data flow between Max/MSP patches . . . . 67

5.9 Different modules of MyCubePatch.pat . . 68

5.10 Mapping of sensor inputs according to stance 69

5.11 Outline of MyEventReceiver.pat . . . . . 69

5.12 Outline of MyFeedbackGenerator.pat . . 70

5.13 Visual feedback on mistakes . . . . . . . . . . 70

5.14 Testing the lab prototype . . . . . . . . . . . . 72

6.1 Robust Arduino casing . . . . . . . . . . . . . 77

6.2 Communication between Arduino BT andNokia N70 . . . . . . . . . . . . . . . . . . . . 79

6.3 Synchronization software for off-line analysis 81

6.4 Outline of a bend sensor attached to the backof the knee with a knee pad . . . . . . . . . . 82

6.5 Data plot of FSRs during alternating turns . . 84

6.6 Final version of the bend sensors . . . . . . . 85

6.7 Attachment of SHAKEs . . . . . . . . . . . . 87

6.8 SHAKE values in the lab . . . . . . . . . . . . 87

6.9 Final setup of the wireless prototype . . . . . 89

7.1 Outline of the user test runs . . . . . . . . . . 93

7.2 Difference of FSR values under the feet . . . 96

7.3 Different parameters for an exponentialmoving average filter . . . . . . . . . . . . . . 97

7.4 Discrete mapping for the toe–heel distribution 98

xii List of Figures

7.5 Detection of toe-heel distribution with frontand back foot of Subject 2 . . . . . . . . . . . 99

7.6 Front–back distribution (Subject 1) . . . . . . . 100

7.7 Front–back distribution of the author . . . . . . 101

7.8 Plot of bend sensors on knees . . . . . . . . . 102

7.9 Sensor values of the back knee (Subject 1) . . 103

7.10 Accelerometer values (Subject 1) . . . . . . . 104

7.11 Angular difference between upper andlower body of Subject 2 . . . . . . . . . . . . . 106

7.12 Plot showing counter-rotation (Subject 1) . . 107

7.13 Discrete mapping of toe–heel distribution andbody twists (Subject 1) . . . . . . . . . . . . . 108

B.1 Screenshot of MyCubePatch.pat . . . . . . 123

B.2 Screenshot of MyEventReceiver.pat . . . 124

B.3 Screenshot of MyFeedbackGenerator.pat 125

xiii

List of Tables

3.1 Comparison of related work . . . . . . . . . . 35

4.1 Overview of the interviewed snowboard in-structors . . . . . . . . . . . . . . . . . . . . . 42

5.1 Sensor attachment on the I-CubeX Digitizer . 68

7.1 Test subjects in the user study. . . . . . . . . . 92

7.2 Success of our approaches to derive sev-eral parameters from the test subjects’ sensorreadings . . . . . . . . . . . . . . . . . . . . . 109

xv

Abstract

Acquiring sport skills can be difficult, time-consuming, and frustrating, especiallyfor novices. We have initiated a project to investigate how wearable computing cansupport snowboarders in their learning process. Due to spatial separation duringriding exercises, a snowboard instructor usually cannot give feedback to his stu-dents on their mistakes immediately. Feedback is only possible when instructorand student are close to each other.Body-worn sensors on the students could detect wrong movements in real-timeand give direct feedback. This might increase the students’ awareness of their mis-takes and thus decrease learning time.This thesis is the initial step to develop a Wearable Snowboarding Assistant. By inter-viewing snowboard instructors and reviewing instructional literature we identifiedfour mistakes common for beginners. We selected suitable hardware componentsand developed approaches to detect these mistakes. In an iterative design process,we have developed a mobile wearable prototype robust enough to be taken on theslope. To show the feasibility of automatic mistake detection, we have conducted auser study with snowboard beginners and evaluated their sensors recordings.

xvi Abstract

xvii

Überblick

Das Erlernen einer Sportart kann mühsam, zeitaufwändig und frustrierend sein.Im Rahmen eines größeren Projektes untersuchen wir, wie mit Hilfe von WearableComputing der Lernprozess von Snowboardern unterstützt werden kann.Aufgrund der räumlichen Distanz zwischen Snowboardlehrer und Schüler, bekom-men Schüler während ihrer Fahrübungen häufig kein unmittelbares Feedback vonihrem Lehrer. Dies ist nur zwischen den Übungen möglich, wenn sich Lehrer undSchüler in Reichweite voneinander befinden.Sensoren die am Körper der Schüler angebracht sind, könnten fehlerhafte Bewe-gungsabläufe dagegen in Echtzeit erkennen und sofortiges Feedback geben. Wirsind der Ansicht, dass der Schüler dadurch fehlerhafte Bewegungen eher zurKenntnis nimmt und effektiver lernt.Diese Arbeit ist der erste Schritt in unserem Projekt zur Realisierung eines WearableSnowboarding Assistant. Aufgrund von Interviews mit Snowboardlehrern und derRecherche von entsprechender Literatur konnten vier Fehler identifiziert werden,die als typische Anfängerfehler gelten. Um diese Fehler mit Hilfe von Sensorenzu erkennen, mussten geeignete Hardware und Verfahren zur Fehlererkennungentwickelt werden.In einem iterativen Entwicklungsprozess entstand ein für die Piste geeigneter mo-biler und robuster Prototyp. Die Realisierbarkeit einer automatischen Fehlererken-nung wurde durch einen Benutzertest mit Snowboardanfängern und den darausgewonnen Sensordaten gezeigt.

xix

Acknowledgements

A lot of people contributed to this thesis and should not be forgotten here:

First of all, Prof. Dr. Jan Borchers for the opportunity to write this thesis at hischair. I really appreciated the good working atmosphere.

Daniel Spelmezan who offered me this interesting topic and had an open doorwhenever I needed advice.

Professor Wehrle, my second examiner, for offering his help on sensor relatedtopics.

Julie Wagner, Max Möllers and Jonathan Diel who were patient test users even onthe freezing cold slopes of SnowWorld.

Elaine Huang, Thorsten Karrer and Christian Mattar who found the time toproof-read parts of my thesis regardless of their tight time schedule.

Tim Stappen for his great comments on my initial drafts and for lending his nameto the scenario persona.

Elisabeth Bak and Marianne Mokosch for suggesting idiomatic English expressionsI never even heard of.

. . . and everyone else at the chair, for the pleasant time. Thank you!

Finally, I would like to thank my girlfriend Birgit Noack for her constant emotionaland professional support during the ups and downs of my thesis.You helped me a lot more than I can repay, thank you!

Last but not least, a big thank you to my parents Marianne and Gerhard Guggen-mos.

xxi

Conventions

Throughout this thesis the following conventions will beused:

The plural “we” will be used throughout this thesis insteadof the singular “I”, even when referring to work that wasprimarily done by the author.

The whole thesis is written in American English.

1

Chapter 1

Introduction

“We learn by example and by direct experiencebecause there are real limits to the adequacy of

verbal instruction.”

—Malcolm Gladwell, Blink: The Power of ThinkingWithout Thinking, 2005

For almost every kind of sport it is essential to first learn Learning sports incooperation with aninstructor

the very basics. This needs to be done properly so that fur-ther improvement is possible. Therefore, it is common toseek the assistance of a professional instructor. In severallessons the instructor teaches important aspects of the spe-cific sport, theoretically and practically. One of the mostvaluable and crucial attributes of an instructor is the su-pervision of his trainee and constructive feedback on hisperformance. The trainee’s progress highly depends on thequality of the instructor’s feedback.

A tennis instructor, for instance, observes the strokes of Immediate feedbackon tennis strokeshis trainee and immediately after each stroke the instruc-

tor can tell him how to improve his technique. The instruc-tor might even guide the trainee’s hand to demonstrate themovement (see Figure 1.1). In other sport areas direct feed-back might not be possible because of spatial separation oftrainer and trainee. A sprinter, for example, will not get anyfeedback on his performance during his training run. Thetrainer will give advice afterwards.

2 1 Introduction

Figure 1.1: Tennis lesson.

When learning how to snowboard a similar spatial sepa-Snowboard instructorand students arespatially separated

ration of instructor and student exists. The snowboard in-structor usually explains how to perform a movement anddemonstrates it. During demonstration he will move downthe slope, away from his students. Thereafter the studentstry to repeat the demonstrated movement as an exercise.The instructor observes every student from the distance.He can only give advice after the exercise, when the stu-dents are close enough to talk to him. Thus, the instruc-tor’s feedback is not given immediately on the students’mistakes.

1.1 A Wearable Snowboarding Assistant

Out of our experience with snowboarding we have initi-The wearableSnowboardingAssistant

ated a project at the Media Computing Group1 (RWTHAachen University) to investigate whether immediate feed-back during the exercises can help snowboard beginners intheir learning process. A wearable system with sensors onthe body, woven into the clothing or attached to the snow-board, could detect wrong movements and give real-timefeedback. We believe that making the snowboarders awareof their mistakes and giving hints on how to correct them

1http://hci.rwth-aachen.de

http://hci.rwth-aachen.dehttp://hci.rwth-aachen.de

1.1 A Wearable Snowboarding Assistant 3

during an exercise might increase learning speed. Through-out the whole thesis we will refer to such a system as theSnowboarding Assistant.

Based on our research in the application domain (see Chap-ter 4—“The Snowboarding Domain”) we provide two sce-narios. The first scenario illustrates the practices of today’stypical snowboarding lessons. The second one envisionshow lessons could be improved with the Snowboarding As-sistant.

1.1.1 Scenario of Typical Snowboarding Lessons

Every year the dutch family van Stappen comes to Tyrol Family on their twoweek winter vacationfor their two week winter vacation. Their fourteen-year-old

son Tim is very curious about trying out new things. Thistime he wants to try snowboarding. Although he has noexperience with board sports, like skateboarding or surfing,he is confident that it will be fun. The van Stappens registertheir son for snowboarding lessons, and Tim cannot wait toget on the slope.

After having received their snowboard equipment Tim and Three days oflessonseight other snowboard beginners meet their instructor Flo-

rian. He tells them that they will have lessons for two hoursin the morning for the next three days. After the course theywill be able to accomplish simple turns on the snowboard.

First the students need to get familiar with the snowboard First day: thesnowboard basicsbasics. They learn to strap on their bindings, how to fall

down and stand up and how to move with their snowboardon flat ground.

Before continuing with further exercises, Florian explains The basic stance ona snowboardand demonstrates his students the basic stance (see 4.1) on

the snowboard. The knees should be bent to be able to com-pensate bumps on the slope. The upper body should bealmost upright and the shoulders parallel to hip and snow-board. The weight should be distributed equally on rightand left foot.

After one hour the beginners learn to slide down a flat hill. Tim’s motivationdecreases

4 1 Introduction

Tim falls down a couple of times but his ambition to learnsnowboarding like he saw it on TV during the X-Games2

keeps him motivated. Florian teaches his students how toslide down the hill on the snowboard’s heel-side and toe-side edge. He also explains that the snowboard alwaysslides and turns on the side with the greatest pressure. Timhas problems keeping his upper body upright. He feelsa bit embarrassed in front of the other students becauseFlorian repeatedly reminds him to straighten up his body.Tim’s motivation is decreasing, but nevertheless he wantsto give his best.

On the next day the students learn how to traverse a hillSecond Day:students are not ableto realize instructionsimmediately

on the edge, i.e., without letting the snowboard drift away.They continue with several exercises which prepare themfor turns. At the end of the lesson, Florian explains anddemonstrates how to do turns. His students try to followhis example but they are not able to realize Florian’s in-structions immediately and sometimes struggle to make acomplete turn. Florian tells his group that they are going toimprove doing turns in the next lesson.

On the last day the students revise what they have learntDoing turns andresolving mistakes so far. When they try to do turns, Florian observes every-

one individually at a time and gives advice afterwards. Timperforms quite well, although sometimes he cannot accom-plish a turn. While his back is facing downhill he is afraidto shift his weight on the front foot because this accelerateshis snowboard. With his weight on the back foot, however,the snowboard does not turn easily. After every run Flo-rian reminds Tim to move his weight on the front foot, butwhile riding, Tim often forgets this. He is too focused tokeep his knees bent and his back upright and therefore doesthe same mistake again and again. Florian tells Tim that heis doing quite well. Yet, for improving his technique in thefuture he should try to resolve this last mistake.

2http:// expn.go.com/expn/index

http:// expn.go.com/expn/indexhttp:// expn.go.com/expn/index


1.1.2 Scenario of Snowboarding Lessons with theSnowboarding Assistant

Like in the previous scenario the van Stappens come to Ty-rol for their winter vacation.Tim’s parents register him for snowboarding lessons. Flo- Sensor enhanced

clothingrian, the snowboard instructor, asks Tim if he likes to rentthe ordinary equipment or the new sensor-enhanced cloth-ing that will analyze his movements and help him recog-nize his mistakes more easily. Tim is very interested in newtechnology, so he decides to give it a try. The clothing andboots, however, do not look special and Tim is a bit disap-pointed because he expected a futuristic suit. In addition tohis clothing, Tim receives knee pads and a helmet, whichalso contain sensors and moreover help to prevent injuries.

Thereafter, the snowboarding lessons begin and the stu-dents Florian teaches his students the basics of snowboard-ing. At first Tim feels a little uncomfortable on the snow-board. When he practices to slide down the slope on theheel-side edge he cannot control his speed. Florian tellsTim to bend his knees further to gain more control over hissnowboard. Tim does bend his knees, but not sufficiently.Therefore, Florian uses his mobile phone to enable the sen- Instant feedback

reminds beginner ofmistakes

sor system attached to Tim’s clothing, boots, helmet andknee pads. Florian tells Tim, that as long as his knees arenot bent enough he will notice a vibration at his knees. Af-ter that Tim practices again and notices the vibration fromhis knee pads. At first Tim is surprised and does not knowwhat to do. In the next run, however, he remembers whatFlorian has told him and bends his knees further until thevibration stops. Every time Tim does not bend his kneesenough the vibration starts again. After a while Tim re-members to always bend his knees and Florian turns offthe sensors.

The second day passed without Florian making use of theSnowboarding Assistant.

On the third day the beginners improve their turns. Tim Improving turns withthe SnowboardingAssistant

manages to accomplish turns but Florian is not satisfiedwith his technique. He tells Tim that he is doing a goodjob but that he should focus on improving his turning tech-

6 1 Introduction

nique. To initiate turns on steeper slopes Tim would needto shift his weight more towards the front foot. Florianenables the sensors in Tim’s boots to detect whether hisweight is too much on his back foot. When initiating aturn, Tim’s back foot now vibrates if his weight distribu-tion is not optimal. Tim notices the vibration immediatelyand leans more on his front foot. In fact, turning becomeseasier.

Florian is satisfied with his student and lends Tim the mo-System can be usedwithout instructor bile phone for the rest of the afternoon, so Tim can continue

to practice even without an instructor. The settings on thephone are already adjusted and Tim only needs to chooseon which of his mistakes he would like to focus.

1.1.3 Goals

As illustrated in the second scenario we envision two mainaspects of snowboarding lessons to be improved by theSnowboarding Assistant.

1. Helping snowboarders to be aware of their mistakesimmediately when they occur. This should decreaselearning time and thereby reduce frustration.

perceive their mistakes more easily during the exer-cise. This should increase learning speed, thereby re-ducing frustration.

2. Allowing beginners to exercise even without the pres-ence of an instructor by providing a system that en-sures the correct performance of movements. Thisshould not substitute a human instructor but supportbeginners beyond the lessons.

To reach these goals several steps towards a wearable Snow-boarding Assistant are necessary:

1. Building a wearable hardware platform with appro-priate sensors which operates on the slope.


2. Identifying and detecting common mistakes in snow-boarding.

3. Providing an adjustable interface to control the Snow-boarding Assistant .

4. Giving appropriate real-time feedback to the stu-dents, e.g., audio or tactile feedback.

5. Showing the benefit of real-time feedback for thelearning process.

1.1.4 Requirements

This thesis initiates the development of the SnowboardingAssistant and focuses on the first two steps. Therefore, thefollowing requirements need to be fulfilled:

Exploring the Application Domain. We need a deep in-sight into the basic terms and techniques of snow-boarding and teaching methods. This serves as astarting point for further development and providesthe basic knowledge to people working on the project.

Opportunities for Change. We have to identify disadvan-tages in the communication between instructor andstudent. The focus should lie on beginners problems.

Robust Hardware. A hardware platform that withstandsthe conditions on a slope needs to be assembled. Thehardware should not restrict the user’s freedom ofmovement. Thus, we need to select robust and un-obtrusive sensors to detect mistakes.

Algorithms. To give feedback to the student we must rec-ognize his mistakes via sensors. Therefore, we needto find appropriate algorithms to process the sensordata.

8 1 Introduction

1.2 Structure of the Thesis

According to the identified goals and requirements the re-mainder of the thesis is structured as follows:

Chapter 2—“Sensor Technology” gives an overview ofdifferent sensor types used in the development of the Snow-boarding Assistant.

Chapter 3—“Related Work” discusses projects whichdeal with context awareness, health care and sports.

Chapter 4—“The Snowboarding Domain” provides anoverview of the basic terms, concepts and techniques ofsnowboarding. Moreover, we identify common beginnermistakes and discuss further aspects of the teaching pro-cess as results of interviews with snowboard instructors.

Chapter 5—“A Lab Prototype” describes the necessarysteps to build a first prototype. Based on the results of theprevious chapter, we have developed a wired lab prototypeto stimulate further ideas. The chapter discusses choiceand placement of sensors and first algorithms to detect mis-takes.

Chapter 6—“A Mobile Prototype for the Slope” docu-ments the building of a wireless prototype. The chapterdiscusses the hardware setup as well as problems we facedon the slope.

Chapter 7—“User Study and Data Analysis” takes acloser look at the sensor data recorded on the slope. Afterconducting self-tests we recorded snowboard beginners’sensor data and discuss how the second prototype coulddetect common mistakes.

1.2 Structure of the Thesis 9

Chapter 8—“Summary and Future Work” sums up theresults of the previous chapters. As this thesis is only theinitial step in the Snowboarding Assistant project, the chapteroutlines the following steps.

Appendix A—“Interview Guideline” contains the origi-nal German version of the interview guideline we followedduring our interviews with snowboard instructors. We pro-vide an English translation as well.

Appendix B—“MAX/MSP Patches” includes screen-shots of the software we developed for the first prototypein Chapter 6—“A Mobile Prototype for the Slope”.

Appendix C—“Smoothing Filters” provides the formu-lae for the smoothing filters we used during the evaluationof the sensor data.

11

Chapter 2

Sensor Technology

“A sensor is a device that receives a stimulusand responds with an electrical signal.”

—[Fraden, 2003, p. 2]

In this chapter we give an overview of relevant sensorsused in wearable computing1. This overview is importantfor the projects discussed in Chapter 3—“Related Work”and for the decisions we have made for our own project.

For every sensor we discuss its measurand, working prin-ciple, and any special characteristics that need to be consid-ered for its application.

2.1 Accelerometer

Accelerometers measure acceleration, i.e., the rate of Accelerometersmeasure rate ofchange of velocity

change of velocity, along a designated axis. 2-D accelerome-ters combine two single axis accelerometers to measure ac-celeration on two orthogonal axes. Analogously, 3-D ac-celerometers combine three orthogonally arranged singleaxis accelerometers.

1 More in depth information on sensors can be found in [Fraden,2003].

12 2 Sensor Technology

An accelerometer can be imagined as a ball in a tube whichAccelerationmeasured throughforces on a tiny proofmass

is fixed with a spring at each end (see Figure 2.1) [van Laer-hoven et al., 2003]. When the tube is accelerated along itslongitudinal axis, the ball will lag behind the movement ofthe tube due to inertia. This causes the ball to change itsposition relative to the tube. The change of position is pro-portional to the acceleration and can be thought of as thesensor’s output. Many of today’s accelerometers are builtusing MEMS2 (Micro-Electro-Mechanical Systems) technol-ogy, which allows the ‘ball’ to be a tiny proof mass of lessthan 0.1 micrograms [Riedel, 1993].

!"#$%&'($(#)$%$*+)(,-)%!,$+(')-'"%(-)!#"%.(((

./01234(567(+68/93:87;?(.8/7

!+#)+,%+! *,! 4(+*();3(+-! &$+(/! $/*(.(*(#+! ,5! *-#!=#$%$&'#!/,023*#%!3+#%!DN+!$/*(.(*@! (+! /%3/($'! ()! /'$%(5@();!=-@!*-#!-#$%*!%$*#!(+!+344#)'@!-(;-#%+! 2,+*3%#! ()!#%;,),0(/! $22'(/$*(,)+6! "#$%$&'#! /,023*();!+/#)$%(,+! 3+#! $/*(.(*@:$=$%#! $22'(/$*(,)+! ()! ,*-#%!+#**();++! 2,+*3%#! $)4!;#+*3%#+6!!!%$!,"-$+)((A-(+!+#/*(,)!/'$%(5(#+!*=,!4#+(;)!/-,(/#+!*-$*!=#%#!0$4#! ),*! ,)'@! &@! *-#! $3*-,%+

)*+,( -.//(x

y

zSprings

Tube Ball

Figure 2.1: 3-D accelerometer Infusion Systemsa and work-ing principle. Based on [van Laerhoven et al., 2003].

ahttp://infusionsystems.com

An accelerometer is sensitive to gravity. Thus, its outputMeasurement issensitive to gravity is the sum of dynamic and static acceleration, i.e., accelera-

tion due to movement and due to gravity.3 When the sensoris at rest it measures exclusively the gravitational accelera-tion and can be used as a tilt sensor. The angle betweenthe sensor’s axis and the gravity vector can be computedwith basic geometry (Figure 2.2). Many wearable comput-ing projects use accelerometers to measure movements be-cause of their small composition and low price (see Chap-ter 3—“Related Work”).

2 For further information see http://www.memsnet.org.3 This is about 9.81 m

s2in Europe at sea level.

http://infusionsystems.comhttp://infusionsystems.comhttp://www.memsnet.org/mems/what-is.html

2.2 Gyroscope 13

x-axis sin Θ = cos(90◦ −Θ) =|!gx||!g|

⇔ Θ = arcsin |!gx||!g|

!g

!gx

90◦ −Θ

Θ

Figure 2.2: Gravitational force on a tilted accelerometer.The accelerometer measures only the projection ~gx of thegravity vector ~g on the x-axis. Hence, with |~g| = 9.81 m

s2the

angle Θ can be computed.

2.2 Gyroscope

Gyroscopes are sensitive to rotational speed, i.e., they mea- Gyroscopes measureangular velocitysure angular velocity relative to a designated axis. Simi-

lar to accelerometers, 2-D and 3-D gyroscopes combines or-thogonally arranged single axis gyroscopes.

Today’s MEMS gyroscopes measure the ‘Coriolis accelera- Coriolis accelerationoccurs duringangular movements

tion’, which can be explained as follows: Consider a personstanding at point 1© in Figure 2.3 on a rotating platformwith tangential velocity v1 relative to the (non-rotating)ground. If this person walks to point 2©, away from thecenter of rotation, its tangential velocity will increase to v2.The acceleration that causes this increase is the Coriolis ac-celeration. It is proportional to the angular velocity of therotation [Geen and Krakauer, 2003].

To measure the Coriolis acceleration, gyroscopes contain a Measuring theCoriolis accelerationwith a tiny mass

tiny mass which vibrates up and down in a fixed frame.When the mass is moving up (away from the center), itwill be accelerated towards the right. This will exert a forceon the frame to the left (Figure 2.4 (a)) as the mass is fixedwithin the frame. Vice versa when moving down (towards


1

2

!v1

!v2

direction of rotation

Figure 2.3: A person on a rotating platform. If she movesfrom point 1© to point 2© she will notice a tangential ac-celeration, the Coriolis acceleration. Taken from [Geen andKrakauer, 2003]

the center), the mass will exert a force on the frame to theright (Figure 2.4 (b)). This force is measured to indicate an-gular velocity as they are proportional to each other.

mass in fixed frame

(a) (b)

Figure 2.4: 2-D gyroscope ‘Spin2D’ (I-CubeX) and theworking principle of a MEMS gyroscope. A mass in a fixedframe is vibrating up and down. Because of the Coriolisacceleration the mass exerts a force on the frame (orangevector) proportional to the angular velocity of the rotatingplane.

Coriolis acceleration only occurs during rotational move-Gyroscopes are notsensitive to gravity ments. Thus, gyroscopes are insensitive to linear accelera-

tions and movements. In contrast to accelerometers, they

2.3 Force Sensitive Resistor (FSR) 15

are not affected by gravity.Gyroscopes can be used to monitor rotation of machines, Use in car safety

systemsairplanes, or vehicles. They are often used in vehicle safetysystems.4

2.3 Force Sensitive Resistor (FSR)

Despite its name, force sensitive resistor, an FSR’s electri- FSRs measurepressurecal resistance drops proportional to the amount of pressure

applied to its surface. Hence, its outcome depends on theamount of force applied as well as the area covered by theforce. FSRs are very thin (ca. 0.2 mm) and consist of differ-ent layers as depicted in Figure 2.5.! ! !

! "#$!%&'()*+',-&!./,0(!+&0!12+3/+',-&!4+*'5!6+'+3-)! 4+)(!7!! 8,'9!#/))(5'(0!13(:'*,:+3!%&'(*;+:(5!

!"#$%&'%()*(+&,%)*)-"#)&!"#$%&'%(&)#*+#,-.&/-"*0*12#

!

!

"#$%&!'&()*(+!,&)*)-#$)!."',/!0$&!0!

1#234&$!-5*%6!7*24!.89"/!:&;*%&!*-)!0!:&%$&0)&!*(!$&)*)-0(%&!&50;*#$@!!"#$!

*(-&$1$&-0-*#(02!%#(;&(*&(%&C!-5&!7#$%&!

;)@!$&)*)-0(%&!:0-0!*)!12#--&:!#(!0!2#+E2#+!

7#$40-@!!95&)&!:0-0!0$&!$&1$&)&(-0-*;&!#7!

#?$!-31*%02!:&;*%&)C!&*(+!-5&!$&)1#()&!#7!&;02?0-*#(!10$-!G!HID!.I@JK!LMD@N!44O!:*04&-&$!%*$%?20$!

0%-*;&!0$&0/@!P!)-0*(2&))!)-&&2!0%-?0-#$!>&$!

$&)*)-0(%&!.&@+@!)-0(:F#77!$&)*)-0(%&/@!

!!

",)/*(!=!$(5,5'+&:(!25?!"-*:(!

connectors

conductive film

active area (electrodes)

spacer

(a) (b)

Figure 2.5: (a) I-CubeXa TouchMicro. (b) Different layers ofan FSR.b

a http://infusionsystems.comb http://www.interlinkelectronics.com

The connectors on the bottom layer lead to an ‘active area’ Higher pressureresults in lowerresistance

that consists of interdigitating electrodes printed on a flexi-ble substrate. A conductive film is printed on the top layerseparated from the bottom layer by a plastic spacer. If nopressure is applied to the sensor’s surface, top and bottomlayers are not in contact. This results in a high resistance ofthe active area, as its interdigitating electrodes are not con-nected. The more pressure is applied the more of the activearea is pushed against the conductive film which leads to

4 For example, the Electronic Stability Control (ESC) to prevent skid-ding of cars (http://www.chooseesc.eu/)

http://infusionsystems.com/http://www.interlinkelectronics.com/force_sensors/technologies/index1.htmlhttp://www.chooseesc.eu/


a connection of the electrodes. Thus, the resistance of thesensor drops.5

The distance between the top and bottom layer can alsoFSRs need to bemounted on flatsurfaces

be decreased by flexing the sensor. Therefore, it should bemounted on a flat surface to eliminate mistakes through de-formation.FSRs are used in a wide range of application areas whereforce or pressure needs to be measured, e.g., in the automo-bile industry for measuring a tire’s pressure footprint6 orin the medical industry to analyze a patient’s force distri-bution under the feet.7

2.4 Bend Sensor

Bend sensors, also known as flex or flexion sensors, arelong, thin (ca. 0.1 mm, Figure 2.6(a)) sensors and changetheir electrical resistance proportionally to their flexion.Most bend sensors consist of a conductive ink printed onResistance drops

according to theflexion

a flexible substrate. The ink is very brittle, hence flexionof the sensor results in micro gaps within the ink (Fig-ure 2.6 (d),(e)). Higher flexion causes greater gaps and de-creases the conductance of the ink resulting in a higher re-sistance.8

The outcome of a bend sensor depends on both the flexionSensor readingdepends on bendangle and radius

angle as well as the flexion radius — a smaller flexion radiuswill cause greater gaps (Figure 2.6(d)). A bend sensor ofthis type only responds to one bending direction. Bend-ing it in the opposite direction does not cause any gapsand thus the sensor’s resistance remains unchanged (Fig-ure 2.6(c)). Bend sensors are often used in input glovesGloves for virtual

reality applications for virtual reality environments to determine the finger’sflexion, e.g., the CyberGlove R© II9 .

5http://www.electrade.com/html/produkte/sensorik fsr.htm6http://www.tekscan.com/industrial/tirescan-system.html7http://www.tekscan.com/medical/systems.html8http://www.flexpoint.com/technicalDataSheets/mechanicalDesignGuide.pdf9http://www.immersion.com

http://www.electrade.com/html/produkte/sensorik_fsr.htmhttp://www.tekscan.com/industrial/tirescan-system.htmlhttp://www.tekscan.com/medical/systems.htmlhttp://www.flexpoint.com/technicalDataSheets/mechanicalDesignGuide.pdfhttp://www.immersion.comhttp://www.electrade.com/html/produkte/sensorik_fsr.htmhttp://www.tekscan.com/industrial/tirescan-system.htmlhttp://www.tekscan.com/medical/systems.htmlhttp://www.flexpoint.com/technicalDataSheets/mechanicalDesignGuide.pdfhttp://www.immersion.com

2.5 Inertial Measurement Unit (IMU) 17

no gaps

conductive ink

gaps

(b)

(c)

(d)

(e)(a)

Figure 2.6: Different flexion states of a bend sensor.

2.5 Inertial Measurement Unit (IMU)

Inertial Measurement Units (IMU) measure orientation in IMUs combineaccelerometers andgyroscopes tocalculate absoluteorientation angles

3-D space based on an initial state, e.g., in Euler angles.IMUs usually consist of a 3-D accelerometer and a 3-D gy-roscope with their axes aligned parallel. As long as the IMUis not moving, its orientation can be inferred with the 3-Daccelerometer relative to the gravity vector. Yet, when theIMUs is moving, the measured acceleration is the sum ofdynamic and static acceleration, which is difficult to sep-arate. As gyroscopes are not sensitive to linear accelera-tions, they are used to keep track of changes in the orien-tation. Dedicated sensor fusion algorithms, which are usu-ally implemented on the IMU, combine the readings of theaccelerometer and the gyroscope to infer the absolute ori-entation relative to the initial state [Bachmann, 2004].

The absolute orientation in terms of ‘world coordinates’ The magnetic field ofthe earth serves asreference

cannot be derived with accelerometers and gyroscopesalone. Moreover, as the calculation of the current orienta-tion is always based on the previous one, any error in thesensor readings is accumulated through the whole calcula-tion. To account for these shortcomings, several IMUs in-corporate a 3-D magnetometer to measure earth’s magneticfield as a static reference. Thereby global coordinates canbe calculated.

There are several manufacturers that offer IMUs with built-in processing capabilities, thus minimizing errors with spe-cialized data fusion algorithms. Examples are the MTxfrom XSens,10 the InertiaCube3 from InterSense,11 and the

10http://www.xsens.com11http://www.isense.com

http://www.xsens.comhttp://www.isense.comhttp://www.xsens.comhttp://www.isense.com


SHAKE SK6.12 Figure 2.7 shows the SHAKE SK6 and theangles it calculates relative to a fixed coordinate system.The SHAKE SK6 can also be used as a digital compasswhich returns values between 0 to 360◦ in the x–y planeindependent of the SHAKE’s orientation.

x

y

z

yaw

roll

pitch

!"#$%&

Sensing Hardware Accessory for Kinaesthetic

Expression

Model SK6

User Manual Revision F

Figure 1

Figure 2.7: SHAKE SK6 IMU and its axes. It calculates ori-entation with respect to a fixed coordinate system.

12http://www.samh-engineering.com

http://www.samh-engineering.comhttp://www.samh-engineering.com

19

Chapter 3

Related Work

“Not to know what has been transacted informer times is to be always a child. If no use ismade of the labors of past ages, the world must

remain always in the infancy of knowledge.”

—Cicero (106 BC–43 BC)

The research projects presented in this chapter deal withbody-worn sensors to measure human movements. Theyare structured by application domain:First we present projects in the broad field of context aware-ness, i.e., being aware of the user’s surroundings. There-after we discuss work done in the health care sector, whichis another promising application area for wearable technol-ogy. To finish the discussion of related work we presentresearch projects in the application domain of sports.

3.1 Context-Awareness

3.1.1 Definition and Examples

Body-worn sensors are often used to identify the wearer’s Context is not clearlydefinedcontext. Context has been defined differently by several au-

thors. Abowd et al. define context as

20 3 Related Work

“[. . . ] any information that can be used to character-ize the situation of an entity. An entity is a person,place, or object that is considered relevant to the in-teraction between a user and an application, includ-ing the user and applications themselves.”

[Abowd et al., 1999, pp. 3–4]

This broad definition includes the user’s physical, social,emotional or informational state.1 Systems that take con-text into account when providing services to the user arecalled ‘context-aware’.

For instance, Schmidt et al. [1999a] have built a hardwareContext awaremobile phone platform that incorporates several sensors, among others,

a 2-D accelerometer, a light sensor, and a temperature sen-sors. With a combination of the sensor readings they in-fer different contexts of a mobile phone, e.g., whether thephone is in the user’s hand or in a bag. Accordingly thering tone profiles are automatically adjusted.

In recent years several projects have focused on ‘activityActivity recognitionwith wearablesensors

recognition’, i.e., recognizing the user’s activity, which is animportant part of the user’s context.2 Laerhoven and Cak-makci [2000] and Ravi et al. [2005] both try to recognize dif-ferent everyday activities of the user, like sitting, standingor walking, with only one accelerometer. Similarly Lesteret al. [2005] try to identify basic activities with one sensornode that incorporates different sensor types.

3.1.2 Multi-Sensor Activity Context Detection forWearable Computing

Kern and Schiele [2003] have built their own sensor hard-Activity recognitionfor real-worldapplications

ware on top of the Smart-Its hardware platform [Beigl et al.,2003] for activity recognition. Instead of building a lab pro-totype they have designed their system to be used in a real-world setting. As potential applications they envision the

1 For a more discrete classification of context see [Schmidt et al.,1999b].

2 Other aspects of context are, e.g., the user’s location, or his heart-rate, depending on the application.

3.1 Context-Awareness 21

domains of sports and manual work. Targeting these appli-cations they draw requirements on their hardware platformwhich apply to the Snowboarding Assistant as well. The most Robust hardware is

essentialimportant requirements are robust hardware and properfixation of the sensors at the desired location, as this heav-ily influences the quality of the sensor data. Furthermore,the user’s freedom of movement should not be restricted.

To fulfill these requirements, Kern and Schiele cover their Hardware setupsensors with shrink wrap and attach them with velcrostraps for tight fixation (Figure 3.1(c)). To be able to movearound freely, Kern and Schiele put the laptop that isconnected to the sensor hardware into a backpack (Fig-ure 3.1(a)). A PDA connected to the laptop starts and stopssensor recordings and is used for online data annotations.Data is later analyzed off-line to extract information.

(a) Recording Setup Mounted on a User (b) Recording Setup: Laptop with IPAQfor Online Annotation and 2 Smart–Its

Fig. 3. Recording Setup

5.1 Experimental Setup

All data is recorded on a laptop that the user carries in a backpack. The soleuser interface is a Compaq IPAQ that is attached to the laptop via serial line. Itallows to start/stop the recording application and to annotate the data onlinewith the current activity. Figure 3(a) shows the user with the mounted sensorswearing the backpack, holding the IPAQ in his hand.

For the desired number of sensors we need two complete sets of sensors withsix sensors each. Each set, consisting of a smart-it, an add-on board, and six3D acceleration sensor nodes is attached via a serial port to the laptop (see alsofigure 3(b)). Every sensor is sampled with approx. 92Hz.

Activities Our goal in this paper is to recognize everyday postures and activi-ties. First of all, this includes basic user postures and movements that allow toroughly classify the user’s activity. These are sitting, standing, and walking.

Apart from these basic postures and movements, it would also be interestingto know, what the user is currently occupied with. We hence included writingon a whiteboard and typing on a keyboard. The former indicates that the user isengaged in a discussion with others, while the latter indicates that the user isworking on his computer.

Finally, social interactions are very important and interesting information.We hence include shaking hands to determine, if the user is currently interacting

(a)

add–o

nboa

rdto

read

24an

alog

chan

nel

sw

ith

23D

acce

lera

tion

sen-

sor

nodes

atta

ched

(b)

Lef

t:A

singl

ese

nso

rboa

rdw

ith

4ch

annel

sof

acce

lera

tion

.R

ight:

the

senso

rw

ith

Vel

cro

stra

pco

vere

dby

shri

nk

wra

p

Fig

.2.A

cquis

itio

nP

latf

orm

boar

don

lyha

s5

anal

ogin

puts

am

ulti

plex

erw

asre

quir

ed.T

hem

icro

cont

rolle

ron

lyoff

ers

10bi

tre

solu

tion

inth

ean

alog

todi

gita

lco

nver

sion

.To

expe

rim

ent

and

asse

sth

eva

lue

ofa

high

erre

solu

tion

conv

ersi

onw

ein

clud

eda

16B

itan

alog

digi

talc

onve

rter

(AD

S832

0).

The

add-

onbo

ard

has

24an

alog

inpu

ts,s

etup

as6

grou

psof

4in

puts

.Eac

hgr

oup

has

one

conn

ecto

rth

atal

sooff

ers

pow

eran

dgr

ound

.E

ach

anal

ogin

put

isco

nnec

ted

toon

eof

the

inpu

tsof

one

ofth

e3

anal

ogm

ulti

plex

ers

(eac

hw

ith

8in

puts

and

one

outp

ut).

The

outp

utof

each

mul

tipl

exer

isco

nnec

ted

toan

anal

ogin

put

ofth

em

icro

cont

rolle

r.Fo

ron

eof

the

mul

tipl

exer

sth

eou

tput

isal

soco

nnec

ted

toth

eex

tern

alan

alog

digi

tal

conv

erte

r.T

heco

ntro

ls(f

orth

eco

nver

ter

and

the

mul

tipl

exer

)ar

eco

nnec

ted

toth

esm

art-

its

core

boar

d.Se

efig

ure

2(a)

.A

libra

ryis

real

ized

that

allo

ws

tore

adea

chof

the

exte

rnal

chan

nels

.Giv

enth

ere

adin

gti

me

and

the

tim

eto

swit

chbe

twee

nch

anne

lsa

sam

plin

gra

teof

abou

t10

0Hz

per

chan

nelc

anbe

achi

eved

.

3.5

3DA

ccel

erat

ion

Sen

sor

Nod

e

For

the

sens

orno

des

we

used

2A

DX

L311

mou

nted

on2

smal

lPC

Bs

whi

char

eat

tach

edto

each

othe

rin

a90

degr

eean

gle

toeff

ecti

vely

obta

in3D

acce

lera

tion

data

.See

figur

e2(

b).

The

base

PC

Bis

abou

t40

mm

by20

mm

and

cont

ains

allt

hesi

gnal

cond

itio

nco

mpo

nent

san

don

eof

the

acce

lero

met

ers.

The

boar

dm

ount

edup

righ

tis

abou

t20

mm

by10

mm

and

cont

ains

only

the

AD

XL3

11.T

heas

sem

bled

size

ofa

node

is40

mm

by20

mm

by10

mm

.We

did

delib

erat

ely

notre

duce

the

size

ofth

eno

des

inor

der

tobe

able

toha

veve

ryro

bust

scre

w-o

nco

nnec

tors

onth

ebo

ard.

We

also

incl

uded

rect

angu

lar

hole

sdi

rect

lyin

toth

eP

CB

toea

sefix

ing

ofst

raps

.To

incr

ease

the

robu

stne

ssth

eno

deca

nbe

cove

red

(aft

erfix

ing

the

stra

psan

dca

ble)

bysh

rink

wra

p.Se

efig

ure

2(b)

fora

pict

ure

ofa

com

plet

e,w

rapp

edse

nsor

boar

d.

(a)

(b)

(c)

Figure 3.1: The wearable system introduced in [Kern andSchiele, 2003]: (a) Person wearing the sensors and holdingthe PDA, (b) The system and its components, (c) Shrink-wrapped sensor with velcro strap

Unlike the previously mentioned projects that try to infer Sensor placementaccording to theapplication

the user’s activity with only one sensor [Ravi et al., 2005,Lester et al., 2005], Kern and Schiele have decided to at-tach several sensors at different locations specific for the in-tended activity.

In a first experiment, they aim at identifying activities such Initial experimentas sitting, standing, walking and hand-shaking. Based on

22 3 Related Work

these target activities they attach 3-D accelerometers to ma-jor joints of the human body at the following locations: an-kle, knee, hip, wrist, elbow and shoulder.

The experiment shows that simple activities, like standingFor complexactivities severalsensors are needed

or walking, can be recognized using only one sensor onthe leg. However, for more complex activities, like walk-ing downstairs, the combination of sensors at different lo-cations improves the recognition rate.

3.2 Health Care

The application of wearable computing technology in theMonitoring thepatient’s healthcondition

health care sector has been explored in several projects. Awide range of research projects focus on monitoring the pa-tient’s health condition [Anliker et al., Dec. 2004, Oliverand Flores-Mangas, 2006]. Most of them raise alarm in caseof dangerous changes in the measured parameters, espe-cially for elderly patients [Najafi et al., 2003, Degen et al.,2003].The following projects, however, analyze movements andpostures. We present them because they are more relatedto the concept of the Snowboarding Assistant.

3.2.1 GaitShoe

Usually gait can be analyzed using two different methods:Two methods for gaitanalysis Either in a motion laboratory with computer-based meth-

ods like optical tracking or in an office with a clinician ob-serving the patient. The first approach results in highly ac-curate data but is expensive. The second method, althoughbeing less expensive, yields highly subjective data depend-ing on the clinician.

Bamberg et al. [2007] propose the GaitShoe, a wearable sys-GaitShoe fills gapbetween traditionalmethods

tem that falls in between the two methods and combinestheir benefits. It provides accurate data and can be usedin the patient’s natural environment. The GaitShoe can beattached to any shoe to analyze the wearer’s gait. To mea-

3.2 Health Care 23

sure gait-relevant parameters Bamberg et al. have incorpo-rated several types of sensors. An overview is shown inFigure 3.2.

Figure 3.2: Overview of the GaitShoe and used sensors.Taken from [Paradiso et al., 2004].

The FSRs, PVDFs3 and bends sensors are collocated on oneinsole. The gyroscopes, accelerometers as well as the micro-controller, the power supply, and the antenna to transmitdata to a base station are placed on the back of the shoe(Figure 3.2).

The GaitShoe has been used simultaneously with a tradi- Comparison withtraditional methodtional gait analysis data acquisition system for compari-

3 Polyvinylidene fluoride (PVDF) strips — sensors that react on dy-namic pressure.

24 3 Related Work

son. The GaitShoe has been proven successful in distin-guishing the gait patterns of healthy persons and subjectswith Parkinson’s disease. In addition, determining theheel-strike and toe-off timing, i.e., when the foot touchesor leaves the ground, was highly successful using the FSRsand PVDFs. For the stride length and the pitch of the footthe GaitShoe integrates the values of the gyroscopes. Dueto the imprecisions of the gyroscopes and the compoundedeffect in the integration only fair result could be provided.

Gait is analyzed in real-time and gait analysis has been ex-Auditory feedback fortherapeutic purposes plored to be used for therapeutic purposes through audi-

tory feedback [Paradiso et al., 2004]. To provide rhythmiccues on how to walk, ambient music is played. Whenevera gait defect is detected the music becomes less melodic,encouraging the subject to return to a steady pace.

3.2.2 Biofeedback Wireless Wearable System

Farella et al. have contributed several projects to the wear-able computing community. In their recent research theyhave developed wireless sensor nodes to track human ges-tures [Barbieri et al., 2004] and detect human body postureswith a body area sensor network [Farella et al., 2006].

Based on their previous work, they have introduced theOptimizing balancethrough audiofeedback

‘Biofeedback Wireless Wearable System’ (Bio-WWS). Thissystem detects a human’s posture and gives audio feed-back to help optimizing balance, e.g., to support the reha-bilitation of patients that have lost their sense of balance[Brunelli et al., 2006].

Similar to gait analysis, current rehabilitation practices forRehabilitation withcumbersomemachines

balance monitoring are carried out with cumbersome andexpensive machines. These devices need to be controlledby an expert and cannot be operated by the patient alone.The Bio-WWS, however, is designed for autonomous andunobtrusive usage.

The current setup consists of a PDA, a Bluetooth headset,Hardware setup:PDA, headset andsensor nodes

three sensor nodes each with a 3-D accelerometer, and agateway. The sensor nodes attached to trunk, thigh and calf

3.2 Health Care 25

measure acceleration and forward the values wirelessly tothe gateway. The gateway collects the data of the differ-ent sensor nodes and sends them to the PDA via Bluetooth.The software for creating auditory feedback resides on thePDA and sends the created audio stream wirelessly to theheadphones via Bluetooth.

The range of acceleration values where the patient is in ‘Target Region’ and‘Limit Region’good balance is called the ‘Target Region’ (TR) (Figure 3.3).

The TR is subject specific and needs to be calibrated at thebeginning of each monitoring session. Therefore the patientnees to stand still for 10 seconds while the system samplesthe acceleration values from the attached sensors. Based onexperiments, Farella et al. set the TR to 1.5 times the stan-dard deviation of the samples collected during the calibra-tion process. Similarly they set a so-called ‘Limit Region’(LR) to 10 times the standard deviation of these samples.

frequency codes the value of the instantaneous acceleration while the volume increases with the distance from he Target Region (TR). The TR is a range of accelerations values which are safe for the user. The task of the user while using the audio-biofeedback is to remain inside the TR. When the user’s accelerations are inside the TR range the sound volume and frequency are fixed [19]. The user’s accelerations in the left-right direction are coded by Left/Right balance modulation of the sound (Figure 3B).

Figure 3. Audio biofeedback sound dynamics. A. Forward-Backward direction B. Left-Right direction

Every 50 ms, the biofeedback sound is updated according to the specific functions represented in Figure 3 and discussed in detail in [19]. The dynamics of the sound generation is determined by the TR and by the Limit Region (LR). TR reflects small ballistic-like movements typical of the postural control system [20]; it is subject-specific and is set in the first 10 seconds of each trial; LR is set consequently. Previous results and experimental sessions allowed us to set TR as 1.50 the standard deviation of the acceleration in the calibration time, and the LR as 10 times the standard deviation of the same value. TR (and consequently the LR) may be changed according to the specific use of Bio-WWS (e.g. smaller for applications in sport discipline or larger in severe damage of the postural system). On the PDA a GUI was implemented to support easy setting of parameters of the biofeedback algorithm.

At present, the sensors on the leg are included in the biofeedback code as quiet standing on/off sensors. Imminent developments of the system will include also the leg sensor nodes, to provide an audio biofeedback during movements (in particular during walking), and the 3D acceleration will be considered (including the vertical one).

5. DiscussionMobility, Usability and Costs The small form

factor both of the sensor nodes (size 20x20x18 mm) and the gateway (4x6x1.5cm, battery included) is suitable for easily wearing the system without

experiencing obtrusiveness. Moreover, for the range of applications here presented for WBAN a small number of nodes is needed. Palmtop computers are decreasing their size while increasing computing capabilities (e.g. HP IPAQ example size is 7,5 x 1,9 x 11,9 cm) and they can easily be worn in pocket or carried in bags as it happens for mobile phones.

From a cost perspective, the proposed solution is extremely interesting, considering that the prototype node has a cost of 35 , the gateway of 50 not including batteries. These costs can decrease significantly, since they are computed for small volume prototyping. Bluetooth headsets cost around 80 , but they can be easily replaced with cheapest wired one. For the application tested the WBAN, using low-cost headsets, has a cost ranging around 160(excluding the palmtop computer). Our system architecture is suitable for consumer market and widespread diffusion, e.g. as a potential partial substitute of expensive optical motion tracking systems (initial cost can easily reach $50,000), which do not allow outdoor acquisition and require high-cost maintenance during their lifetime.

Power consumption. Measurement of power consumption was performed in all the states of the system, setting sampling frequency at 60Hz (to obtain a useful bandwidth of 30Hz, adequate for capturing human movements). Communication speed was set at 32kbps from node to gateway on OOK transmission at 868MHz, and at 230kbps from gateway to PDA, transmitting via Bluetooth at 2,4GHz. Power consumption when all components are active and transceiver is continuously sending data corresponds to 200mW for the gateway and 45 mW for a single end-node. In idle state (only reception mode is active) the power lowers respectively at 25mW and 20 mW. When active, gateway battery (500mAh) and single node battery (100mAh) have a lifetime of 8 hours. Sleep mode corresponds to having all devices in the minor consumption state (1,5mW for gateway and 10 µW for end-nodes). For a 20% duty cycle the lifetime reaches 40 hours for the gateway and 38 hours for the node. This is a possible real situation, because typically it is not needed or it is not possible (e.g. the channel must be shared with other nodes) to send data continuously. The major limitation can be the palmtop computer lifetime. We tested the IPAQ transmitting a continuous data stream to a desktop PC through Bluetooth link. The evaluated duration of the 3,7 V battery (1000 Ah) in this worst case is of 12 hours, having that when on the PDA consumes 350mW. Thus we can conclude that WBAN lifetime is adequate for the applications we have in mind. System performance. The STMicroelectronics accelerometer sampling frequency used in our application is 560Hz, while its accuracy corresponds to

!"#$%%&'()*+#,+-.%+/#0"-.+1((023+4555+4(-%"(2-'#(23+6#(,%"%($%+#(+!%"72*'7%+6#890-'()+2(&+6#880('$2-'#(*+:#";*.#9*+?:@ABC+

ADEBFGDHGHADHIAB+JHAKAA+L+HAAB!!"""#

TR TR

LRLR

volu

me

freq

uenc

y

sensor node

Figure 3.3: The Bio-WWS: As soon as the patient leavesthe Target Region (TR) the audio feedback is modulated toguide the patient back to the TR. The maximum modula-tion is achieved at the borders of the ‘Limit Region’ (LR)[Brunelli et al., 2006].

Within the TR the audio stream is not modulated at all. The Working principle ofaudio feedbackmaximum modulation is achieved at the borders of the LR.

The more the patient leaves the TR the more the sound getsmodulated: volume and frequency modulation for forwardvs. backward leanings, left–right audio balance modulation

26 3 Related Work

for left vs. right leanings (Figure 3.3). This should guide thepatient back to the TR.

Farella et al. have conducted an evaluation of their systemwith healthy subjects who were required to close their eyes.With the audio feedback, the subjects left the TR less oftenthan without feedback.

3.2.3 TactaPacks

Lindeman et al. [2006] aim at supporting physical therapyRehabilitation forjoint replacementpatients

for joint replacement patients. They try to decrease injuryrisk by monitoring and warning patients when they are do-ing harmful motions that could result in injury.

For this purpose, Lindeman et al. have developed Tacta-Packs, small sized wearable boxes, consisting of a micropro-cessor with a Bluetooth unit for communicating to a hostcomputer, a 3-D accelerometer for sensing and a vibratorfor giving feedback (Figure 3.4).

Bluetooth Chipset

Battery 3-Axis Accel.

Microprocessor

Motor Driver

Tactor (Vibrator)

(a) (b)

Figure 3.4: TactaPack: (a) outline of its components, (b) pic-ture of the interior. Taken from [Lindeman et al., 2006].

During a training session the patient attaches several Tacta-Vibration asfeedback Packs to the limbs around the replaced joint. Each of the Tac-

taPacks autonomously measures the momentary tilt of itsaccelerometer relative to the gravity vector (cp. Figure 2.2).

3.3 Sports 27

Like the Bio-WWS, the TactaPacks need to be calibrated tostore the ‘safe region’ of accelerometer values. If the patientleaves this region the boxes begin to vibrate.

Vibration patterns and intensity as well as the delay, after Vibration intensitycan be adjustedwhich vibration starts, can be adjusted via a graphical user

interface on the computer. The sensor data, however, is pro-cessed by the microprocessor on the TactaPacks to preventcommunication delays.

3.3 Sports

The following projects cover a promising application ofwearable computing in the area of sport: monitoring mo-tions of athletes for objective analysis and to enhance train-ing practices.

3.3.1 Wireless Force Sensing Body Protectors forMartial Arts

Judging in Taekwondo competitions is a subjective task. Wearable device tojudge Taekwondocompetitionsobjectively

The judges cannot always tell if a punch or a kick was ex-ecuted powerful enough or if it hit the right body part tobe considered a valid score. To provide a more objectiveapproach, Chi et al. [2004] have built protectors with built-in force sensors, which are worn on the taekwondo com-petitor’s torso. The force sensors measure the impact of apunch or a kick. They send the readings in real-time over awireless connection to a base station, which is connected toa laptop.

Chi et al. have conducted experiments with experienced Automatic scoring isbased on forcemeasurements

Taekwondo competitors to gather sensor data for differentkicks and punches. From the experiments they have de-rived thresholds for the force readings to determine auto-matically whether a punch or a kick is valid. Chi et al. en-vision their system to be used together with human judgesensuring more objective results of Taekwondo competi-tions.

28 3 Related Work

3.3.2 Towards Recognizing Tai Chi

Kunze et al. [2006] have conducted a feasibility study to ex-plore the potential of body-worn sensors to automaticallyrecognize Tai Chi movements. As video analysis for suchmovements is tedious, time-consuming, expensive and er-ror prone, they argue in favor of a wearable solution to an-alyze trainees.

Kunze et al. use eight MT94 Inertial Measurement UnitsEight sensor unitsmounted on body (cp. 2.5—“Inertial Measurement Unit (IMU)”) sensor units

on different parts of the body. Discussions with Tai Chi ex-perts yield the following attachment locations for the sen-sor units: above the elbow, above the feet, above the knee(two sensors in each case), and one on neck and hip.

Kunze et al. have conducted an experiment with two TaiAmateurs andexperts can bedistinguished

Chi amateurs and two Tai Chi experts. Collecting sensordata over a sample window of 100 and calculating variousfeatures, e.g., root mean square, Kunze et al. try to recog-nize different Tai Chi movements. After having trained aK-Nearest-Neightbor clustering algorithm, they are able todistinguish three types of expertise with 76% accuracy andtwo different Tai Chi movements with 85%. These resultsshow that recognizing Tai Chi movements automatically isfeasible.

3.3.3 Audiofeedback for Karate Training

Takahata et al. [2004] try to improve a trainee’s understand-Movements hard toexplain with words ing of how to perform a certain karate punch. They ar-

gue that instructors can only vaguely deliver movementsby means of expressions and explanations.

To deliver feedback on the trainee’s performance they pro-Audio feedback onpunches vide real-time audio feedback. 2-D accelerometers on the

wrists measure twists and a microprocessor maps the ac-celerometers’ data to sound. For well performed punchesthe trainee gets clear sounds as feedback and therefore can

4 This is the predecessor of the MTx mentioned in Section 2.5—“Inertial Measurement Unit (IMU)” from XSens.

http://www.xsens.com/

3.3 Sports 29

check his performance on his own. In their tests Takahataet al. show that audio feedback increases the trainees’ mo-tivation. However, feedback on several aspects of a punchshould be avoided as the trainees can only focus on one.

3.3.4 Combining Body Sensors and Visual Sensorsfor Motion Training

Similarly to the previous project, Kwon and Gross [2005]propose a system to improve traditional training methodsin motion training, especially in martial arts. During atraining session, a trainer usually demonstrates a certainmovement and the trainees try to follow his demonstra-tions.

In the new training method the participants’ movements Body movementscaptured withcamera andbody-wornaccelerometers

are captured via a camera and body-worn accelerometers(Figure 3.5(a)) to create a motion data model in real-time.This data model has two purposes:

1. The trainer’s motion data is used to automatically cre-ate an instructive training video enriched with non-visible information, e.g., a circle around the handchanges its diameter according to the magnitude ofthe acceleration (Figure 3.5(b)).

2. The trainee’s data on the other hand is evaluated bythe system based on the trainer’s reference data us-ing Hidden Markov Models.5 Therefore, a trainee canstudy the trainer’s movements in detail with the in-structional video and gets feedback on the quality ofhis own movements.

When practicing a basic movement in martial arts, a traineefirst performs a certain posture, then executes the motion,e.g., one punch, and ends up in a posture again. Hence,Kwon and Gross divide the sensor data into postures and

5 Hidden Markov Model (HMM) are used in speech analysis to rec-ognize words and sentences and distinguish speakers. For more infor-mation see [Rabiner, 1990].

30 3 Related Work

May 21, 2005 16:35 ACE05 Proceedings: Trim Size: 295mm x 245mm ace05

ACE 2005, Valencia, Spain

Figure 10: Observations for body sensor trackingwith LED markers on the wrist body sensor.

maximum value of 100. During the tests, we found that thisminimum distance is better suited than mean or median tomeasure quality. The computed scores are displayed on themotion training video panel in real-time.

5. MOTION TRAINING VIDEOA motion training video is necessary for trainees and train-ers as a reference to follow and analyze motions. However,producing such a video usually takes a lot of time. First,it requires simultaneous video recording during the trainer’sperformance. Also, the captured videos should be edited forthe purpose of motion training such as selecting video framesand adding explanatory information. We provide a methodfor automatic generation of motion training videos. As soonas the input motion is detected, we save both the relevantvideo frames and the body sensor data. Then we generate avideo displaying body sensor data along the tracked sensorpositions, as illustrated in figure 11.

5.1 Body Sensor TrackingWe extract sensor positions from the captured images anduse the positions to generate visual feedback. We madevarious experiments to find suitable tracking solution forour purpose. First, the color band tracking highly dependson the training environment condition such as lighting andcolor. We also tested IR light sources, but they omit colorinformation which is required. We found that color LEDmarkers are most suitable for our purpose. Their brightnessprovides relatively robust tracking results in indoor train-ing environments. We developed a simple vision trackingalgorithm to find the pixel positions within a certain colorand brightness range. The number and position of LEDsare designed depending on the sensor position. In our tests,we attached four LEDs at the four sides of the wrist bend.This installation allows us to detect at least one point reli-ably even when the hand is rotated in different directions.Figure 10 illustrates four cases where one, two, three pointsare detected respectively. We use the center of the detectedpoint as the position of the body sensor.

5.2 Visual FeedbackVisual feedback helps trainers and trainees to explain andimprove their motion practice. During the user tests, weascertain the fact that visual feedback for body sensor datais absolutely needed. Users wanted to see how the bodysensor data is changing with the appearance of the posture.Especially for the trainees, visualizing motion path helpssignificantly to understand a dynamic gesture between twostatic postures. Thus we focus on visualizing body sensordata on the images along the motion path as illustrated infigure 11. We use the tracked sensor positions and design asimple template to display a moving circle along the path

Figure 11: Example video frames for visual feedbackwith the mean power of acceleration signal.

changing its size as a function of the magnitude of the ac-celeration. There are various design alternatives, of course,varying the shape and its transformation rules.

6. USER EXPERIMENTSWe conducted a set of user experiments to quantify the costsand benefits of combining visual and body sensor data formotion training. We expect our motion training system toprovide significant benefits over conventional motion train-ing. In our motion training system, visual sensor data isused as a feedback to the user allowing him to coarsely ad-just his motion to the reference motion. Conversely, thebody sensor data is utilized for adjusting required body partof the user precisely. In our experiment we measure thisbenefit. In addition, we quantify how our system evaluatespostures and gestures and how it detects human motions inreal-time. To this end, we use a martial art training scenario.Martial art training is specifically suited for our experimentbecause it includes highly complex, precise motions whichcontain both postures and gestures.

6.1 SubjectsFor this experiment, we used a trainer who is a master ofTaekwondo and six additional subjects as trainees, threemale and three female, all of them having no experience inmartial art training.

6.2 TaskWe designed the separate tasks for the trainer and for thetrainees. The task of the trainer was to produce the ref-erence motion data model for 10 sets of five motions each(punch, outside block, upper block, inside block, and downblock). This model was used for the trainee experiment lateron. Subsequently, the trainer was asked to perform 5 sets of10 outside-blocks for testing the motion evaluation methods.The rest time between each set was two hours, and in eachset he performed 10 times repeatedly without resting.

For the trainees, we designed two basic training conditions:posture training and gesture training. The task of posturetraining was to learn start and end postures of the five mo-tions. The gesture training serves for practicing individualgestures between a start posture and an end posture. Inposture training, the trainees were told to perform posturesof four motions three times each while watching a referenceimage without resting. We measured how long it takes tolearn to match their postures to the trainer’s average rolland pitch values. Among the five motions, we selected thepunch motion which is relatively easy for teaching novicesto use the system. The end posture of the punch is also

99

(a)

(b)

Figure 3.5: (a) The system and its components. (b) Se-quence of the visual feedback. The circle’s diameter in eachpicture reflect the current acceleration of the hand.

gestures, i.e., static and dynamic chunks of a movement.A single motion thus consists of three chunks: static–dynamic–static. Based on these chunks the trainee’s mo-tion is evaluated with respect to the trainer’s reference data.The system calculates a score for every motion chunk en-abling the trainee to systematically improve postures andgestures.

The system has been tested in an experimental Taekwondotraining with one trainer and six trainees who had no ex-perience with Taekwondo. The results have shown that thesystem helps beginners to learn simple postures and ges-tures.

3.3 Sports 31

3.3.5 Wearable Sensing System for ProfessionalDownhill Skiing

A project closely related to the Snowboarding Assistant is de- Video analysis is acommon practicescribed in [Michahelles et al., 2005]. This project aims to

support and enhance training practices of professional ski-ing athletes Usually the training runs of a skier are recordedwith a video camera during the day. Afterwards the skierand his coach analyze the videos to identify mistakes andopportunities for improvement.

Like [Kwon and Gross, 2005] in the previous section, Enriching videomaterial with sensordata

Michahelles et al. argue that making non-visible informa-tion (e.g., acceleration forces on the skier) visible throughbody-worn sensors might improve video analysis. Out oftheir own experience and from literature reviews they haveidentified ski-relevant features and according sensors:

• 3-D accelerometers at thigh, lower leg and torso tomeasure movements of the skier

• 3-D gyroscopes on the skies to measure rotation

• three FSRs under each foot to measure the weight bal-ance of the skier

• distance sensors attached at the boots to measure theedging angle of the skis relative to the slope

• a radar unit for measuring velocity (this sensor wasdropped for the final prototype due to its insufficientaccuracy)

Figure 3.6 shows a skier wearing the sensors. A laptop in Hardware setupthe backpack records the sensor data. The sensors on skiand boots are fixated with adhesive tape. The accelerome-ters on the body are attached via velcro straps. They shrink-wrapped the sensors to protect them from snow.

To analyze a skier’s run Michahelles et al. have written a Dedicated softwareto visualize andanalyze sensor data

dedicated software to view the sensor and video data ofthe run synchronously. Sensor values appear in differentvisualizations. Apart from raw data plots, they combine the

32 3 Related Work

8

entire framework. In SKI the user can associate video recordings of a ski withsensor data, and view those through different visualization representations (seeFigure 5a).

In particular, the framework provides rotating bars indicating rotation move-ments primarily for gyroscope data (see Figure 5b), a composition of all foot forcemeasurements (see Figure 5c), and, finally, an animated data plot showing theplain sensor values (see Figure 5d). All viewers can be played at adjustable speedsynchronous to the video recording. Furthermore, basic filter operations, suchas mean calculations or integration, allow simple data pre-processing. Analyzingfirst data we realized short random peaks in all of the sensor data. Nevertheless,mean calculations over a sliding window with 30 values width prove sufficient toeliminate those artefacts. Later versions of software should contain pre-definedfilter values, currently this is adjusted by hand.

5 Providing Experience and Meeting the Trainers

(a) (b) (c) (d)

Fig. 6. (a) sensor setup, (b) ski setup, (c) mounted accelerometers, (d) test-run

In order to provide more ‘realistic’ experience to the ski-experts we firstrecorded test-data ourselves: A student wore the sensors and several downhillruns were measured and video-taped on a race course. After that, having thesensor platform, software-analysis tool, and first data, we started off visitingseveral ski trainers in Switzerland. The following section gives an overview ofseveral meetings we have conducted. In each session we started with a coupleof slides describing our view about the benefits of wearable sensing for skiing,showed our video- and sensor-data with our software, and collected feedbackfrom the discussions.5.1 Meeting #1: Ryan Baumann, Interregio-Team Swiss WestRyan Baumann coaches 15 to 18 years skiing athletes who have been selectedfrom all over West of Switzerland. His skiing team has about ten members.

In the beginning, our meeting was centered a lot around today’s work prac-tices of trainers. The trainer records runs of his athletes to analyze very specificphases of their skiing technique. Later in the evening they sit together and thor-oughly analyze the recorded videos of the day. However, occlusion, snow sprayand perspective can limit the evidence from time to time.

During the meeting we realized that it does not make sense to ask trainerswhich sensors they think could be interesting for them: trainers just do not careabout sensors. Instead we continued learning more about the skiing technique.

8




(a) (b) (c) (d)


In order to provide more ‘realistic’ experience to the ski-experts we firstrecorded test-data ourselves: A student wore the sensors and several downhillruns were measured and video-taped on a race course. After that, having thesensor platform, software-analysis tool, and first data, we started off visitingseveral ski trainers in Switzerland. The following section gives an overview ofseveral meetings we have conducted. In each session we started with a coupleof slides describing our view about the benefits of wearable sensing for skiing,showed our video- and sensor-data with our software, and collected feedbackfrom the discussions.5.1 Meeting #1: Ryan Baumann, Interregio-Team Swiss WestRyan Baumann coaches 15 to 18 years skiing athletes who have been selectedfrom all over West of Switzerland. His skiing team has about ten members.

In the beginning, our meeting was centered a lot around today’s work prac-tices of trainers. The trainer records runs of his athletes to analyze very specificphases of their skiing technique. Later in the evening they sit together and thor-oughly analyze the recorded videos of the day. However, occlusion, snow sprayand perspective can limit the evidence from time to time.

During the meeting we realized that it does not make sense to ask trainerswhich sensors they think could be interesting for them: trainers just do not careabout sensors. Instead we continued learning more about the skiing technique.

8




(a) (b) (c) (d)


In order to provide more �

towards a wearable snowboarding assistant · this thesis is the initial step to develop a wearable...

Documents