Augmented Speed-Skate ExperienceApplied Movement Sonification

June 2009

Jelle Stienstra BSc.

Faculty of Industrial Design

Eindhoven, University of Technology

prof. dr. Kees Overbeeke

dr.ir. Stephan Wensveen

dr.ir. René Ahn

TVM Schaatsploeg B.V.

2

4

Preface

In the fall of 2007 I was inspired by Alan Murray to conduct a project

that was focused on extending the human senses.

I was very interested in empowering the extremely enabled people,

and push them to their limits by providing an extra sense modality.

With the TVM Schaatsploeg as committed client, the project started

to focus on speed-skating and empowerment through the sonifica-

tion of movement.

This book describes the two major phases of my project. The design

phase, in which speed-skating, biomechanics, psychology and elec-

tronics were researched and integrated through design; a research

platform was built enabling the coaching staff to gain insights in

the speed-skating technique. The second phase of the project was

concerned with field-lab research, in which this platform was used

to enable athletes to real-time adjust their technique supported by

movement sonification. Furthermore, the practical application of

movement sonification in speed-skating was researched.

During the one and a half year I worked on the project I was able to

pursue my academic and personal ambitions and explore my pas-

sions for design and life. After hard work, blood, sweat and tears I look

back on a satisfying and personally inspiring project which made me

grow as a designer and person.

This book presents my academic contribution to the augmented

speed-skate experience. I hope you will enjoy reading it as much as I

enjoyed the project.

Jelle Stienstra

6

Preface

Content

Summary

Part I

Introduction Background

Stakeholders

Concept Movement Sonification

Project Objective

ContextIntroduction in speed-skating

Biomechanics

Platform DesignRequirements

Overview

Electronics Design

Form-giving

Software Design

Summary

Part II

Research Outline Movement Sonification

Research Objectives

Auditory Information DesignParameter Mapping

Generating Movement Sonification

Pearl Movement Sonification

Alternative Movement Sonifications

ExperimentExperimental Set-up

Measurements

Results

Conclusion

RecommendationsRedesign

Movement Sonification

Acknowledgements

References

Reflections

Client Feedback

Appendices

4

7

8

11

12

13

16

18

21

22

24

27

28

29

30

34

38

42

45

46

50

52

53

54

55

56

59

60

62

64

69

70

71

72

74

76

78

80

83

8

Summary

The augmented speed-skate experience is a concept that enables

professional speed-skate athletes to improve their technique by

supportive real-time movement sonification providing an extra sense

modality. Movement sonification concerns the translation of move-

ment into auditive information.

The project is focused on integrating the physical technique with the

mental aspect of speed-skating. There is the perfect stroke; a sensa-

tion of flow at which the athlete is able to transfer all energy via the

ice into the forward direction, a sensation of a technique where the

whole body becomes one with the ice. The augmented speed-skate

experience targets to enable the athlete to discover and reproduce

this sensation.

The augmented speed-skate experience project consists of two

phases. In the first place the development of the platform which

measures movement data streams and computes those into auditory

streams. In the second place a research phase were experiments are

conducted to gain insights in the concept of movement sonification

applied in professional speed-skating.

Platform DesignBiomechanics as well as the context of professional speed-skating are

described to gain insights in the case study context. Together with the

client, the TVM Schaatsploeg, it was decided to focus on measuring

the forces transferred on the ice, separated over front and back of

each speed-skate to gain insights in the balance and whole speed-

skate stroke.

The augmented speed-skate experience platform consists of two

speed-skate devices, one for each speed-skate of the athlete, with a

continuous wireless connection to the central processor. The central

device is capable of processing the movement data into sonifica-

tion. This results in the real-time feedback loop as the RF headphones

deliver the sound-scape. Besides producing a sonification, the central

processor can record movement data and provide basic visual

insights in the speed-skate technique. Real-time sensor gauging and

software modifications can be made because of the modular soft-

ware design.

The augmented speed-skate experience measures the forces dis-

tributed in the vertical direction on the sagittal plane through the

front and back of each speed-skate. This leads to insight in power

distribution and balance of the athlete. Individual characteristics are

preserved. The electronics are encapsuled by a modular casing, that

enables tight fitting of the device on Viking bridges size 37 and 38.

The values of the TVM Schaatsploeg; ‘professionalism’, ‘fast’, ‘strong’

and ‘power’ are represented in the form-giving but more important,

it does not interfere with professional speed-skating allowing field-lab

studies.

ResearchWithin the research phase, movement sonification is implemented on

the augmented speed-skate experience platform. This auditory infor-

mation mapping is designed to accommodate auditory perception

through psychoacoustics and according to kinematic and dynamic

laws to establish intermodal convergence between proprioception

and hearing data.

In a total of five day-sessions covering approximately twelve hours of

qualitative field-lab studies; explorative insights in the interpretation of

data, recognizability, learning ability, reproducibility and experience

of movement sonification were acquired during the first experiment.

A second experiment explored the boundaries of movement sonifica-

tion through testing alternative auditory information mappings.

Results indicated that the primary auditory information mapping of-

fers a neutral, informative, motivating, un-coercive, robust and easy

to learn feedback. This, contrary to the slight variations of this auditory

information mapping.

The results following the exploration of the reproducibility of a specific

speed-skate stroke and a specific task concerning technique, indi-

cate that the movement sonification is recognized, learned and that

converging modalities are experienced. The sonification of move-

ment is perceived and interpreted as intended by the athlete. To fully

understand the value of the movement sonification, the intermodal

convergence should be experienced. Further research and platform

improvement is recommended.

10

IntroductionHere, the project “augmented speed-skate

experience” is introduced. Starting with the

background of this academic work, I pro-

ceed with a description of the partners in-

volved in the project, a quick synopsis of the

conceptual framework and finally the project

objectives.

12

Background

The augmented speed-skate experience project is a continuation of

the “Bat Biker” project by Industrial Design students Bart Friederichs

and Bram van der Vlist. Within the “Bat Biker” project, blind people

were enabled to perform off-road mountain biking through provided

auditive cues. (http://w3.id.tue.nl/nl/education/masters_program/design_re-

search_projects/bat_biker/)

Contrary to the “Bat Biker” project the focus of this project is on ex-

tremely enabled users, top athletes, instead of supporting the disa-

bled. In the early phases of the project the concept of movement

sonification was explored. While the “Bat Biker” provides representa-

tional audio cues I considered an opportunity for sonifying continuous

bodily movement resulting in a new sense modality; a sonification

enabling the user to define what is good and bad instead of leaving

that up to a system. This pointed towards sports involving repetitive

patterns such as pole vaulting, rowing, running and speed-skating to

establish a relative focused learning curve in understanding the sense

modality.

In November 2007, the TVM Schaatsploeg, a professional and com-

mercial speed-skating team was approached. Together we defined

an important physical movement to be measured. This led towards

the development of a measuring device and platform providing

specified sonification which matches the athletes senses and pro-

vides a rich auditive information mapping. An elaborative description

can be found throughout the following chapters.

Stakeholders

The project is accomplished, supervised and supported by a group

of stakeholders. Various interests in the project by the TVM Schaat-

sploeg, the University and myself are described below.

TVM SchaatsploegSince November 2007, TVM Schaatsploeg, a the world-class profes-

sional speed-skate team is involved in the augmented speed-skate

experience project as the client.

World-class athletes such as Sven Kramer, Ireen Wust, Paulien

van Deutekom, Carl Verheijen, Erben Wennemars and Renate

Groenewold represent the TVM Schaatsploeg. Their staff, including

Gerard Rietjens (Exercise Physiologist), Gerard Kemkers (Head Coach)

and Geert Kuiper (Assistant Coach), is closely involved and commit-

ted to the project. They provide their knowledge, equipment and the

necessary facilities to accomplish the project.

The client is extremely interested in the development of a measur-

ing platform to gain insights in the technique of speed-skating and

eventually improve their training methods. To them it is important to

recognize individuality of each athlete and to come close to the ac-

tual professional setting without adjustment to the existing equipment

and speed-skate environment.

14

Faculty of Industrial Design, Eindhoven, University of TechnologyThe Designing Quality in Interaction capacity group has been involved

throughout the whole project through involved staff and supervisors

such as Alan Murray, dr.ir. Stephan Wensveen and prof.dr. Kees Over-

beeke. Their interest lies within the overall design process methodology,

holistic integration of design disciplines, professional client collabora-

tion, ethical considerations, exploring the field of wearable senses, the

aesthetics of interaction and the integration of perceptual-motor skills

and bodily interaction in interaction design.

Dr. ir. René Ahn has been involved in the project since the early begin-

ning. Representing the Designed Intelligence capacity group, René is

interested in mapping the real-time transition of physical movement

data into coherent auditive feedback to gain insights in the conver-

gence of human senses.

Jelle StienstraPersonally, I am interested in developing concepts that create op-

portunities for the user to explore, learn, relate, distinguish and improve

their unique abilities in their context. I envision transformation through a

holistic highly dynamic design process involving validation in context.

My objectives are to enable the user to explore and better understand

its own capabilities, explore my rational as well as emotional designer

skills and to deliver a professional and complete project with excel-

lence.

16

Concept Movement Sonification

This chapter introduces the concept of movement sonification which

is elaborated further on in relation to the chosen context of speed-

skating. For now, a global description and related requirements for

design applications are given. Movement sonification concerns the

translation of movement into auditive information.

Sonification has been applied to a number of explorative and high-

dimensional analysis tasks such as navigation, aural supervision of

health measurements, fossil classification and software debugging

by offering an auditory display (Kramer, Ed 1994; Henkelmann 2007).

Sonification can also be used to enhance the performance of hu-

man perception in the field of motor control and motor learning.

Auditive bio-feedback can support motor perception and control in

sports, medical therapy and rehabilitation by offering supportive or

new channels of proprioception (Shea et al. 2001; Effenberg et al.

2001,2007; Hermann et al. 2006; Henkelmann 2007).

There are several methods of movement sonification which are de-

scribed later. To provide movement sonification enabling the athlete

to judge its movement it is chosen to use the continuous param-

eter mapping sonification method. In this method a sound-scape is

provided to support the athlete’s experience through continuously

offering movement data that cannot easily be cognitively evaluated

through a non-aural sense modality. Mechanosensational proprio-

ceptive movement data is translated into exteroceptive hearing

data according to kinematic and dynamic laws and thereby provide

sense modality convergence between proprioception and hearing

data.

Repetitive patterns enable the development of intermodal conver-

gence that helps the athlete to define its movement technique. It is

important to find a sport discipline that is restricted to relative easy

distinguishable and repetitive technical movement patterns. Move-

ment sonification requires spatial and time critical convergence. For

design applications this requires a movement data sample rate of

100 Hz and a sonification of movement within a maximum of 250 ms

from the event to reaction or optimally without delay.

18

Project Objectives

Through the use of movement sonification,

the augmented speed-skate experience

intends to enable the athlete to discover and

reproduce the sensation of a perfect speed-

skating technique.

The augmented speed-skate experience

project has two core objectives. The first ob-

jective is to build a solid platform that can be

used by the speed-skater and the coaching

staff for analysis of the speed-skate tech-

nique. The second objective is to implement

movement sonification through the use of this

platform.

In order to provide a solid platform for move-

ment sonification, the platform design is

focused on the following areas;

1. The development of a fully-working pro-

totype, concerning steady measurement of

specific physical parameters that cannot be

sensed by the athletes sensory system, to be

used in field-lab experiments.

2. A wireless data transmission protocol and

processing protocol which preserve the rich

physical parameters within the boundaries of

a limited delay of data processing to provide

the real-time feedback loop.

3. The form-giving of the prototype that

incorporates the sportive image of the cli-

ent, the physical limitations of hardware, the

preservation of aerodynamic and balancing

properties of the speed-skate.

4. A graphical user interface which provides

basic insights into the athletes technique to

the coaching staff and a modular design

to enable real-time gauging and software

modifications required for tuning the move-

ment sonification.

These areas are discussed within Part I of this

report after an introduction into speed-skat-

ing and its biomechanics.

Implementing movement sonification is ac-

complished by an iterative auditory infor-

mation mapping. This design is based on

research in applied movement sonification

conducted within the second phase of the

project as discussed in Part II.

The designed platform offers opportunities

for further research in movement sonifica-

tion, which can be very interesting and

meaningful for both the chosen application

of enhancing the speed-skate experience

as well as for other projects that are based

on the principles of movement sonification.

The platform rises many questions such as;

should the auditive feedback be a real-time

representation or summarize a whole set of

identifiers? Should the feedback provide the

athlete with pre-interpreted judgments or

provide the raw data leaving the interpreta-

tion to the athlete? Do novice users require

a different type of feedback compared to

professionals? Does the athlete get depend-

ant on such a system? If so, how can this be

prevented? Should we not only support the

somatosensory system but also direct actual

actuators? In what time frame can systems

like these be learned? What are the learning

phases towards becoming an actual sense

modality? And more general, what is the

current situation and what is the impact of

coaching? How is the empty data interpret-

ed and how should the sound/actuator to-

wards the required interaction be designed?

Since so many questions arise, it is decided to

focus on an optimal sonification guided by

strict design decisions founded on research.

Two main questions are answered on the ba-

sis of one final auditory information design;

1. Can movement that is sonified be recog-

nized, learned and reproduced?

2. Can the athlete un-coercively interpret

the sonification and experience converging

modalities?

Several other questions are answered and

insights are created through qualitative ex-

periments. This is described more elaborate in

Part II.

20

ContextA returning movement pattern required for

movement sonification and the opportunity

to collaborate with a world-class professional

team, enabling field-lab studies, made the

project focus on speed-skating as the use

case and context for implementing move-

ment sonification.

Speed-skating is a physical sport with multi-

ple bodily-movement data measuring and

sonification opportunities. Biomechanics help

to describe these movements and distinguish

relevant measurements. Before biomechan-

ics are introduced, the context of profes-

sional speed-skating is discussed.

22

Introduction in speed-skating

Speed-skating is what we call a typical Dutch sport. Even though nat-

ural ice on the Dutch rivers is rarely found these days, the sport can

be considered a national pride, especially in the northern regions.

The glory of gliding over the rivers as a way of transport has grown

into a serious sport discipline; worldwide venues, an international

competition, several disciplines such as marathon, short track and

long track, and Olympic recognition indicate an interest in this sport.

Basically, speed-skating is all about speed. Being the fastest over a

set distance using speed-skates to cross the ice.

Over the past decades speed-skating is grown into a professional

sport including outstanding training facilities, global events and com-

mercial teams. With the introduction of the klap-skate and improved

training methods, records keep being broken in contrast to for exam-

ple athletics where the limits seem to be reached.

Klap-skates are just one example of the many improvements made

over the past years. Suits, training methods and materials are upgrad-

ed yearly to have the cutting edge advantage over the competition.

Serious research is done, at places such as the biomechanics depart-

ment at the Vrije Universiteit Amsterdam, to keep improving. TVM

Schaatsploeg is the professional stakeholder in the project as stated

before. Their interest in the development of new technologies that

help improve is one that is shared among other commercial teams

and national associations all over the world.

Recent developments focus on creating a better understand-

ing in the biomechanics of speed-skating by basically measuring

all there is to track. Initiatives by NOC*NSF (Nederlands Olympisch

Comité*Nederlandse Sport Federatie) and commercial teams led to

measurement installations such as the LPM (local position measure-

ment system) and the Innosports measure-skate which is currently be-

ing developed (www.innosport.nl), resulting in training methodologies

and material improvements such as the klap-skate.

My augmented speed-skate experience is focused on the long-track

circuit of speed-skating and its several distance disciplines. This in-

volves not only the quality of ice (friction) on the 400 meter track, air-

pressure and other weather related conditions, but more importantly

the athlete, its physical build, condition, technique, materials and the

mental aspect.

The project is focused on integrating the physical technique with

the mental aspect. There is the perfect stroke; a sensation of flow

at which the athlete is able to transfer all energy via the ice into the

forward direction, a sensation of a technique where the whole body

becomes one with the ice. The augmented speed-skate experience

targets to enable the athlete to discover and reproduce this sensa-

tion.

24

Biomechanics

In most ways of locomotion such as running

and walking, propulsive forces are exerted

on the surface in a direction opposite to the

direction of locomotion. In this case speed

can not exceed the maximal velocity of leg

extension and backward swung and much

energy is lost in moving back the leg. Due to

the peculiar properties of ice; the coefficient

of friction between ice and blade in the glid-

ing direction is very low. Ice is soft enough to

allow the edge of the sharpened blade to

be pressed into the ice to exert a propulsive

force against the surface perpendicular to

the gliding direction of the speed-skate. This

implies that the speed of locomotion ex-

ceeds the leg extension velocity. The push-off

technique enables the athlete to develop

and maintain speed-skate velocities that ex-

ceed running velocities by two-fold (Houdijk,

2001).

Speed-skating strokes can be divided into

three phases: the gliding phase, in which the

skate is placed on the ice and the speed-

skater is gliding forward; the push-off phase,

Platforms that are currently used to gain

insights in speed-skating technique are dis-

rupting professional speed-skating because

of the size and assemblage of measuring

instruments. Limiting size, weight and more

importantly avoiding adjustment to athletes

personal equipment such as the suit, speed-

skate shoe and blades is required to prevent

actual race circumstance interference.

During speed-skating, the external power

output produced by the athlete is predomi-

nantly used to overcome the air and ice fric-

tional forces. Air friction is mainly influenced

by body build, velocity, clothing, shielding

drag posture ice condition, wind and alti-

tude. Friction should be prevented, therefor

the augmented speed-skate experience

needs to be designed as small and light as

possible.

All technique used by the speed-skater is

focused on gaining and maintaining speed

with the least amount of energy. Different

kinds of movements can be measured such

as the bending of the knee, forward acceler-

ation, angle of the hips and shank, etc. In the

end all those small movements contribute to

the transmission of power. Together with the

TVM Schaatsploeg it was decided to focus

on measuring the forces transferred on the

ice, separated over front and back of each

speed-skate, to gain insights in the balance

and whole speed-skate stroke.

The forces along the frontal axis of the

speed-skate are less then 1% of the force

in the vertical axis of the speed-skate. It is

assumed that the force vector along the

sagittal axis can continuously be directed to

the central mass of the speed –skater’s body.

No excessive inversion or eversion of the foot

occurs during the push-off, the push-off force

vector almost coincides with the sagittal

axis (Houdijk, 2001). This enables to focus the

measurements of the front and back of the

speed-skate on the vertical forces on the

sagittal axis only.

in which the gliding sideward push-off occurs,

and a repositioning phase, in which the leg

is brought in the starting position for the fol-

lowing stroke (de Koning et al. 1991). Theo-

retically, the athlete should prevent to have

both speed-skates on the ice as the power

distributed through the push-off speed-skate

will be lost in the leaning speed-skate. For the

athlete it is difficult to perceive this front-back

and left-right distribution of power during a

stroke.

The scientific history in speed-skate research

has only just started to develop. Therefor

measuring platforms are required to gain

new insights in optimization of speed-skating

technique. The novelty of research within

speed-skating produces generalized re-

search outcomes. Generalized research

tends to overlook the individuality of the top

athletes that distinguish them from good ath-

letes. It is important to preserve this individual

richness.

26

Platform DesignTo provide movement sonification to the

athlete, the augmented speed-skate experi-

ence’s platform is built. The platform enables

the measurement and processing towards

sonification of the athletes movement data

in a field-lab setting.

This chapter starts with a synopses of the re-

quirements that evolved from the concept of

movement sonification and biomechanics of

speed-skating. Further on, the final platform

is described from several perspectives such

as the required electronics and software

design but also the aesthetic values, techni-

cal requirements and small scale production

design decisions represented in the form-

giving of the device. A design summary and

specifications of the platform closes this part

of the book.

28

OverviewRequirements

The augmented speed-skate experience should measure the forces

distributed in the vertical direction on the sagittal plane through the

front and back of each speed-skate. This should lead to insight in

power distribution and balance of the athlete. Furthermore, indi-

vidual characteristics should be preserved and acceleration should

be measured as reference for speed-skate technique analysis. All

these measurements should be gathered by wireless communication.

The central processor should compute the sonification of movement

and return the auditive information to the athlete. The wireless data

transmission and processing protocol should cover at least 100 meter

over the speed-skate ring. Movement data should be gathered at

a rate of 100 Hz and be returned to the athlete sonified within 250ms

because of intermodal converge.

The graphical user interface should provide basic insights into the

athletes technique to the coaching staff, have the ability to store

sessions and the software should be modular to enable real-time

gauging and software modifications required for tuning the move-

ment sonification.

Form-givingThe form-giving of the prototype should incorporate the sportive im-

age of the client and the physical limitations such as the weight and

surface of the device should preserve aerodynamic and balancing

properties of the speed-skate. The measuring device should allow

assemblage on several sizes to accommodate multiple athletes, it

is important to avoid the athletes sensitive equipment such as the

speed-skate shoe and blade.

Measure devicesBroadcast the acquired acceleration and pressure

on the front and back of the speed-skate move-

ment data, created by the athlete, to the central

processor over 100 meter.

AthletePerceive auditory information about speed-skate

technique and experience intermodal convergence.

Central processorReceive and compute athlete’s movement data into

auditory data. Sonification as well as visualization of

data takes place here.

Coaching staffReview the visualizations of the movement data and

coach the athlete to improve speed-skate technique.

30

Electronics Design

The platform is designed to gather move-

ment data from the speed-skate. Both accel-

eration and pressure on the front and back

of the speed-skate are measured for each

side. This measured data streams are trans-

ferred from the speed-skate devices to a

central processor. The central processor proc-

esses the movement data into movement

visualizations and sonification. The sonified

movement is returned to the athlete via wire-

less RF (Radio Frequency) headphones. The

electronics involved are described below.

Data acquisitionTo measure pressure in the vertical direction

on the sagittal axis of the speed-skate two

FSR’s (Force Sensing Resistors) are used.

In the search for the most appropriate sen-

sors, various methods were considered; insole

pressure measurement was declined on the

basis of non-absolute pressure measurement

and in-shoe placement, various ways of pie-

zo-resistive strain gauge measurements were

declined because it would require bridge

weakening or rebuilding and some costly

and power consuming industrial solutions.

The Tekscan FSR sensors used in the augment-

ed speed-skate experience’s device are

usually limited to measure up to 450N which is

insufficient for the purpose of measuring the

power transferred by the athlete on to the

ice. A solution to this problem was found in

amplifying the analog signal at the device,

to extend the measuring range of absolute

values up to 2000N.

The sensors are placed between the speed-

skate shoe and the bridge, lifting the speed-

skate for 0,2mm. The back pressure sensor is

placed near the bolt at the back of the shoe.

The front pressure sensor is placed near the

hinge close to bolt to capture downward

pressure and eliminate pressures that are

being developed in other directions dur-

ing speed-skating. Because the sensors are

placed between the shoe and bridge which

are connected through bolts; static pres-

sure (caused by the forces of the bolt) and

dynamic pressure (caused by the athletes

movement) are distinguished by the software

later described.

Measuring acceleration in three directions is

done by an adjustable acceleration sensor

which can be set to sensitivities of 1.5, 2, 4

Pressure sensor located between speed-skate

shoe and bridge.

Pressure (A and B) measurement in the vertical

direction on the sagittal plane and accelera-

tion (C,D and E) measurement for a left (1)

and right (2) speed-skate.

32

and 6G. One problem with measuring ac-

celeration only is that it can only be used to

gather insight in acceleration data instead

of exact data as gravitation is measured as

well. Extracting this from the measured data

is infeasible without a gyroscope. Neverthe-

less, acceleration including the gravity noise

measured is reproducible as the exact move-

ment is reproduced.

CommunicationAs described earlier; movement sonification

is only possible with a high rate of movement

data samples. Sonification delay should be

prevented as much as possible. To fully cover

data transmission over the 400 meter speed-

skate track, a wireless connection should

cover 100 meters from the athlete to the cen-

tral placed data processor.

At the time the first prototype was built,

Bluetooth was the most appropriate way to

securely transfer the movement data over

the 100 meters. After the first prototype was

developed, the wireless communication did

not seem to work. Intensive debugging, in

cooperation with the GTD (Gemeenschap-

pelijke Technische Dienst) and experts avail-

able at the faculty, did not solve the issues. It

was decided to exchange the Bluetooth and

microcontroller for the zigbee protocol which

over time had upgraded its specifications up

to over 100 meter transmission range.

The augmented speed-skate experience

uses the zigbee protocol through Xbee

modules. Xbee modules do not require an

external data processor to translate the

analog signals. The modules used are con-

figured to convert the five analog inputs per

speed-skate (pressure in the front and back

and the acceleration in three directions) to

digital representations of 1024 steps and to

send them over the 100 meter to the central

processor at a sample rate of 100 Hz.

Board designA custom PCB (Printed Circuit Board) is de-

signed to compactly store all the earlier men-

tioned components as well as the necessary

resistors and capacitors. The electronic circuit

is designed in Eagle, a professional CADsoft

layout editor. All components chosen involve

SMD (Surface Mounted Device) packaging,

which in general means that it is rather small

and difficult to solder. Therefor some compo-

nents are assembled by using a solder-oven.

To power the augmented speed-skate expe-

rience’s device, a 3,7 Volt coin-cell battery is

used. Brought down to 3,3 Volt by a voltage

regulator to comply to the Xbee module

and chosen sensors, the battery can last for

approximate 40 minutes in the cold circum-

stances of a speed-skate arena. Just as the

other components, the battery holder is opti-

mally placed to use as less space as possible.

The battery holder is also easy accessible to

enable battery change as the battery func-

tions as on-off switch for the device.

Central processingOnce a battery is added to the device

located at the speed-skate, movement data

streams are transmitted to the central proc-

essor. The software and the visualization and

sonification of movement data is described

later.

FeedbackThe final step in completing the feedback

loop from gathering the athletes movement

data to providing a new sense modality, is

the return of the movement sonification’s

sound-scape to the athlete. A wireless RF

headphone is used to limit the delay. Con-

trary to Bluetooth headsets; RF headphones

do not require to decode the transmitted

digital audio signals which is a relative slow

process. Therefor RF headphones are much

more suitable.

A preliminary measuring session with Ireen

Wust at Heerenveen speed-skate venue

“Thialf” in June 2008 was conducted to

distinguish platform flaws. This session led to

platform redesign considering the wireless

protocol and some minor adjustments to the

sensor placement and data processing.

34

Form-giving

The form-giving of the augmented speed-skate experience’s device

is described from three perspectives; technical requirements defin-

ing the physical positioning and dimensions, aesthetic values of the

shape defined by the clients image and finally from a perspective of

how to produce, assemble and mount.

Technical limitationWith the purpose to measure pressure distributed through the speed-

skate it is an obvious solution to mount the device on the speed-skate

itself. Because it is not feasible to adjust the blade or shoe it is decid-

ed to provide a device mountable on the bridge between the shoe

and blade. The bridge itself is the least personal part of speed-skate

equipment in comparison with the tailor made shoes and blade.

The augmented speed-skate device is targeted to be as small and

light weighted as possible. The dimensions are mainly defined by the

physical design of electronics. It is decided to focus the platform on

the most common bridges; Viking bridges. Furthermore, it is decided

to fill the space beneath the speed-skate bridge, which has a width

of 20 mm, to limit external forces such as wind. Sturdiness of attach-

ment is accomplished by a robust polyurethane material with surface

strength 82 Shore D and tight fitting the dimensions of the speed-skate

bridge.

Aesthetic valuesThe device’s shape is based on the clients values. The following

values are used to describe the TVM Schaatsploeg: invincible, record

breaking, dynamic, craftsmanship, art, success, quality, knowledge,

hard work, toughness, effort, exertion, resilience, stamina, detail,

power, fast, physical, strong, recovery, robust, holistic, determination,

technique, energetic, skill, goals, proud, dedication, bravery, smooth,

professional and confident. The preliminary focus is on the terms ‘pro-

fessionalism’, ‘fast’, ‘strong’ and ‘power’. The other terms are used

subordinately but through a strong link between those associations

and the primary terms, subconsciously most of the terms come about

in the final design.

Fimo-clay modeling, on reproduced wooden speed-skate bridges, is

used to freely explore suspension and location of the casing on the

bridge using physical limitations of the printed circuit board. This ex-

ploration allowed me to distinguish the fast from the slow models and

the why; divergent long lines with the centre of visual gravity located

at the back or front do provoke an aesthetic perception of speed.

After exploring with the tangible media, fimo-clay, I used a drawing

pencil to sketch and add detail to the form-giving. Areas of interest

made it to the next level of the form-giving process involving flash-

drawing which enables fine line adjustments. Interesting proposals

were printed and attached to wooden bridge models. Each degree

of rotation or transition in the plane leads to different aesthetic value,

therefor this exploration was necessary within the process.

36

The mounting of the augmented speed-skate experience device fo-

cuses on a rails and clasping system to tight fit the bridge. On itself this

minimal shape is perceived as slow. Therefor it is decided to add an

extra highlighting shape element, making the slow shape subsidiary.

This form-giving is developed further with a more holistic approach

involving the context of use and three-dimensional prototyping.

To fully support the fast, powerful and professional attitude of the

device’s casing; the highlighted shape element got curved top ac-

cents to support the direction of speed. This slight bending provides

the required depth and curve needed at the stop line at the back of

the bridge to contribute to the perception of power and maintained

professionalism.

Produce and assembleStereolythography used for the first prototype could not stand

temperature changes, light, and humidity leading to brittleness and

shape transformation of the casing. Therefor it is decided to use

mould-casting to provide the required casings for the platform.

It is decided to develop a modular casing that would fit multiple sizes

of the Viking bridge. Size 37 and 38 are chosen and as illustrated they

differ somewhat. A solution is found in developing two parts; the first

part containing the electronics and tight fitting to the back curve of

the bridge, the second part that is specified for the separate bridge

sizes filling the gap for each bridge in the front and providing the solid

mount to the bridge rail.

The augmented speed-skate experience device’s casing is designed

in Rhinoceros and adjusted in Solidworks to deliver a stereolythogra-

phy model. Rubbed-up, primed, rubbed-up and again paint-brushed

models were used to develop a mould. A weather condition resistant

casing is produced through vacuum casting.

The illustrations provide details about the fitting and assemblage of

each separate part.

Base and size-depending part fit over each others’

grooves providing strength on bridge fit in horizon-

tal axis of the sagittal plane. Battery accessible

from the side.

Base part fitting the back of the bridge sizes 37 and

38.

Electronics are encapsuled in the base part, fit

tightly in grooves in the base and size-depending

part.

Size-depending part providing bridge fitting

strength in the sagittal axis.

Cover part fits on the base and size-depending

part of the casing and provide the professional-

ism’, ‘fast’, ‘strong’ and ‘power’ attitude support-

ing the image of the TVM Schaatsploeg..

38

Software Design

With the purpose of processing movement data into sonification, programming platform

MAX5 offers sublime opportunities to accomplish this task as it is created for sound design.

The software specifically developed for the augmented speed-skate experience is described

here from a data flow perspective; starting with the data acquisition, gauging, viewing,

recording and concluding with sonification opportunities in the software design. These back-

end objects are designed to support front-end functions which are described here first.

(front-end) FunctionsThe sonification software at the central processor is designed to provide the following func-

tions at the front-end of the graphical user interface; the central processor should be able

to connect to the augmented speed-skate experience’s devices, coaching staff should be

allowed to view the core and manipulated data streams of the athletes movement, speed-

skate sessions should be recorded and reviewed, individual settings and gauging should be

enabled to be modified during the sessions, the coaching staff should be able to talk to the

athlete and most importantly the central processor should provide real-time sonification of

the movement data.

InterfaceTo briefly give an idea of the software, the first interface screen is shown on the left. It shows

two core functions; recording and providing feedback. These core functions are filled with

related underlaying functions such as for recording; connect, record, data view, data settings

and the current athlete’s name. The software runs on presets, this means that the minimal

amount of information is shown and that the user does not need to do anything except when

adjustments are required.

MAX5This graphical design environment is originally developed for musical, audio and multimedia

purposes by Miller Puckette to enable interactive music creation on computers. This modular

software environment is used to provide the standalone central processing application for the

augmented speed-skate experience.

Device connectionTo establish a connection with the augmented speed-skate experience devices, two serial

ports associated with the Xbee modules are opened. By requesting data every 10 ms (100Hz),

two data streams enter the platform. To use these streams; data is separated over the ten

sensory inputs namely front and back pressure and acceleration in three directions for both

speed-skates.

Data acquisitionEven though the electronics circuit provides data acquisition through amplifying the pres-

sure sensors input sensitivity, the central processor does need to gauge the values to repre-

sent them as actual forces in Newton. The central processor allows the staff to manage the

dynamic and static forces by manipulating properties such as the scaling based on athlete

weight and removing static forces.

Data viewAs the raw data is managed by earlier mentioned gauging; clean streams of dynamic move-

ment data will remain. They can be viewed in graphs representing the amount of pressure on

the front and back of each speed-skate.

40

RecordingEach 10 ms all ten separate raw movement data streams are written

away in a text file which eventually contains of long columns of data.

Each recorded session is identified by the sessions time stamp and

athletes name and gauge settings of the specific session.

Reviewing the sessions allows the same flexibility of manipulating the

gauge settings as the recordings store raw movement data.

SonificationMovement streams can be approached throughout the whole back-

end of the software. The central processor delivers auditory feedback

and therefor manipulates the data streams into audio streams. To

accomplish this; a noise generator is used after which the streams

influence several band-pass filters adjusting the noise. More about this

is described later.

From the moment a movement data sample arrives till the moment

it manipulates the sound sent out to the athlete, data is processed

within 30 ms. Manipulation of multiple auditive dimensions take place

about 100 times per second related to the data sample rate. For de-

velopment purposes it is chosen to modularly design the filter objects

and its required scaling to manage the output.

42

Summary

The augmented speed-skate experience platform consists of two

speed-skate devices, one for each speed-skate of the athlete, with a

continuous wireless connection to the central processor. The central

device is capable of processing the movement data into sonifica-

tion. This results in the real-time feedback loop as the RF headphones

deliver the sound-scape. Besides processing a sonification, the

central processor can record movement data and provide basic

visual insights in the speed-skate technique. Real-time sensor gauging

and software modifications can be made because of the modular

software design.

The speed-skate devices are capable of measuring forces in the front

and back of the speed-skate distributed by the athlete in the vertical

direction on the sagittal plane and acceleration in three directions.

The electronics are encapsuled by a modular casing, that enables

tight fitting of the device on Viking bridges size 37 and 38.

The casing represents the values of the TVM Schaatsploeg; ‘profes-

sionalism’, ‘fast’, ‘strong’ and ‘power’ by its form-giving but more im-

portant, the speed-skate platform does not interfere with professional

speed-skating allowing field-lab studies.

Specifications

Bridge fitting

Device Dimensions

Device weight (inc. battery)

Assembly time

Battery

Battery replacement time

Battery uptime

Pressure sensor range

Acceleration sensor range

Wireless protocol

Sample rate

Data transfer range

Sonification platform

Sonification delay

Headphone

Viking size 37 and 38

36 x 124 x 32 mm

60 gram

10 minutes

3,7V, 200mAh

10 seconds

40 minutes

0 to 200 kg

up to 6G sensitivity

Zigbee (Xbee Pro)

60 to 100Hz

up to 120 meter

MAC OSX, MAX5

below 200 ms

Sennheiser RF Head-

phones

44

Research OutlineThis chapter elaborates on the concept of

movement sonification and how it is in-

tended to be applied within the augmented

speed-skate experience. Including the re-

search questions and objectives. This chapter

serves as the outline of this research.

46

Movement Sonification

Within the augmented speed-skate experience, sonification has

been chosen to mediate real-time kinematic and dynamic data

from the speed-skater that can be hardly perceived by the athletes

somatosensory system.

Sonification is the translation of any kind of data into non-speech

audio. Sonification has been applied to a number of explorative and

high-dimensional analysis tasks such as navigation, aural supervision

of health measurements, fossil classification and software debug-

ging by offering an auditory display (Kramer, Ed 1994; Henkelmann

2007). Sonification can also be used to enhance the performance of

human perception in the field of motor control and motor learning.

Auditive bio-feedback can support motor perception and control in

sports, medical therapy and rehabilitation by offering supportive or

new channels of proprioception (Shea et al. 2001; Effenberg et al.

2001,2007; Hermann et al. 2006; Henkelmann 2007).

Auditory perceptionOur life is surrounded by a rich acoustic landscape; birds and wind

blowing through trees increasing contextual awareness, the sound

of approaching footsteps and alarm clocks that direct our attention.

The richness of acoustic information in almost every human activity,

like the sound of a closing door, reveals a variety of information about

the interactants impact, material, stiffness, texture and energy (Gaver

1986; Hermann et al. 2006). Sounds have unique qualities caused by

the materials physical parameters resulting in the timbre. The kinetic

and dynamic auditory properties caused by the action result in dura-

tion and amplitude. The auditory perception is well suitable for per-

ceiving these time-critical structures about kinematic and dynamic

movement data (Effenberg 2005; Effenberg et al. 2005(2)).

Auditory perception can perceive multiple rows and dimensions of

sound created around the perceiver. This allows us to move freely

without losing track of the source, to concentrate on groups within

the sound or to focus attention on other tasks while still perceiving the

whole sound-scape. This is in contrast to the ability of our visual system

which is limited to a narrow scope of perception and demand for

focus. As hearing is simply different from vision, sonification may con-

vey new structures in data. Even though motor learning is dominated

visually, auditory perception offers unique subtle temporal resolution

as well as enormous integrative capacity (Effenberg et al. 2005(2),

Effenberg 2007).

All these auditory perceptive abilities make movement sonification

an appropriate method for providing supportive feedback on move-

ment for motor-learning. Movement sonification should be created

through focusing on the ecological approach of acoustic perception

and integration of multi-modal perception for optimal motor con-

trol. The more precise and concise the perceptual process can be

designed, the more efficiently the learning process can be arranged

(Effenberg 2005; Effenberg et al. 2005(2); Hermann et al. 2006).

Sonification techniques Sonification of data can be achieved through various techniques

such as Audification, Earcons, Auditory Icons, Audio Beacons, Model

Based Sonification or Parameter Mapping Sonification. Earcons,

Auditory Icons and Audio Beacons are discrete singular sound events

used to present information about the state of a situation or per-

formed action. Earcons allow the user to derive information from the

sound. These mostly symbolic relations should be learned. Auditory

Icons resemble physical attributes of the action and thereby reflect

the event’s trigger within the sound. Audio Beacons are considered

dynamic versions off Earcons and Auditory Icons (Gaver 1986; Kramer,

Ed 1994; Hermann et al. 2006, Henkelmann 2007).

Parameter Mapping Sonification constantly translates a data stream

into a parameter to be streamed auditorily. Audification is the sim-

plest case of Paramater Mapping in which data is directly (i.e. with-

out transformation) sonified. Model-based Sonification uses a dynam-

ic model to render sounds (Kramer, Ed 1994; Hermann et al. 2006).

Lacking a model of which speed-skate stroke movement is the best,

the high demand for individual differentiation among the athletes

and the richness of continuous movement data, guides the design to

continuous parameter mapping sonification. The challenge within this

auditory information design is the mapping.

48

When using movement sonification for generating supportive in-

formation, the system designer has to know about the workings of

the human perception system. Different senses should converge to

convey an intermodal mapping. The quality of sonification should be

designed to fit our auditive perception.

Converging perception streamsIntermodal mapping can best be realized according to natural kinet-

ic-acoustic relations to converge perceptional streams. Temporal co-

incidence and spatial convergence of both stimuli as well as similar

stimulus duration and intensity result in a structural equivalence. The

causation object as well as the emitting event of a generated sound

should be correctly identified by the perceptual system. Time-varying

similarities in the patterning of information might prove a more salient

feature for sensory integration then simultaneous onset or spatial con-

tiguity (Effenberg 2005; Effenberg 2007).

To achieve time-critical multisensory integration it is important to cre-

ate a latency free sonification. Therefore data should be obtained

and processed at a high rate, preferably 100 Hz with a maximal

processing delay of 250 ms. Limiting this delay will improve the multi-

modal convergence (Effenberg 2005; Hermann et al. 2006).

Aesthetics of InteractionBesides mapping the movement onto a coherent sonification, de-

veloping the quality of the sound is a challenge. The sound-scape

should be neutral though rich in information but most importantly the

sonification should be un-coercive. A common problem in auditory

displays and sonification is that systems are turned off because of an-

noying sounds (Kramer 1994; Henkelmann 2007). The richness of data

details generated by the athlete should be preserved in the richness

of sound parameters such as dynamics, loudness, tempo, pitch and

timbre. Interpreted pleasant sounds tend to persuade the user into

behavioural change. The sonifcation should be neutral, the interpre-

tation of movement technique is a responsibility of the athlete. The

auditory information design could take this responsibility if it contains

a model of the perfect technique.

PsychoacousticsHumans can hear periodic signals with a frequency between 20 Hz

and 20.000 Hz. Frequencies around 4500 Hz are perceived the loudest

when providing equal loudness over the whole range of frequencies.

Frequency is detected on a logarithmic scale therefor pitch-scaling

which is perceived linear is used. Perceived loudness is closely related

to the physical amplitude.

As for all of our other senses, our hearing is most sensitive to changes,

transition or transients. Auditory perception of multiple-rows of sounds

is susceptible to auditory masking, the covering of softener sounds

by stronger sounds. Sounds can be masked by other sounds that

are for example louder or consist of covering frequencies. Complex

tones are more difficult to mask than pure tones which also have the

disadvantage of exhausting the listener (Gaver 1986; Effenberg et al.

2005(2); Henkelmann 2007).

Though most of us are gifted with these same incredible senses, it is

important to understand we all differ in the way we use our auditive

perception and/or interpretation. Whether this is caused by back-

ground of the listener, as in damaged senses or perfectly trained

auditory perception for musicians etc.

50

Research Objectives

The augmented speed-skate experience

offers the athlete an alternative channel of

perception for perceiving the technique of

movement. Contrary to most movement

sonification applications, the augmented

speed-skate experience does sonify move-

ment data that can not be perceived with-

out the sonification.

The measured and sonified data is a result

of a complex set of movements. This causes

a complex process of achieving perceptual

converge, if at all. It is worthy exploring the

opportunities whether the idea of move-

ment sonification can be a successful tool in

movement technique improvement through

providing extra perceptual data.

The research conducted is a qualitative

exploration answering a limited set of fun-

damental questions on the subject stated

below. With this research I would also like to

provide insights in the practical appliance of

continuous movement sonification.

The core objective is to create an auditory

information mapping that offers a neutral,

informative, motivating, un-coercive, robust

and easy to learn feedback. But in the first

place it is important to discover whether this

kind of movement sonification could actually

be perceived and interpreted as intended.

The following research questions were

formulated within the scope of the project

to reflect the potential of the augmented

speed-skate experience and movement

sonification;

1. Can movement that is sonified be recog-

nized, learned and reproduced?

2. Can the athlete un-coercively interpret

the sonification and experience converging

modalities?

The auditory information mapping guided by

the earlier described notions on perception

and multimodal converge is designed. The

following chapters will elaborate on this fol-

lowed by a description on how the research

was conducted to answer the stated ques-

tions.

Besides an ultimate movement sonification,

later called the *pearl, several other auditory

information mappings were implemented

and tested to gain insight in practical appli-

ance. This exploration focuses on parameter

selection and feedback methodologies

within continuous movement sonification.

52

Auditory Information DesignThis chapter describes the * Pearl version of

the auditory information design that is used

for the core part of the experiment. Besides

this version of sonification other derived ver-

sions used within the experiment are de-

scribed. Before elaborately describing these

auditory information mappings the auditory

design process is introduced.

Parameter mapping

To conceive a sonification that allows the athlete to discover and reproduce the sensation of

a perfect speed-skating experience it is required to identify the appropriate physical move-

ment dimensions to measure. These have been selected throughout the stakeholder meet-

ings and biomechanics literature research resulting in a platform capable of measuring these

parameters. Within the auditory information design this is described as the data acquisition

focused on continuous measurement of pressure on the front and back and acceleration in

three directions for both feet.

The second layer of parameter mapping is the movement data interpretation. This consid-

ers data stream comparison, combination, deriving and scaling. The respectively 10 single

raw movement data streams provide an almost endless set of opportunities. Theoretically,

the measurement of acceleration in any direction would provide the acceleration, speed or

even jerk which is the change in acceleration. Combining acceleration streams can provide

the direction and speed of a speed-skate. When combining both speed-skates; the direction

and speed of the athlete can be derived. Describing a similar translation for the pressure pro-

vides new data streams such as pressure changes, balance per speed-skate, athlete balance

and change of balance, etc. Continuing with even more complex data streams representing

the forward speed over pressure or derived parameters such as the stroke frequency.

Together with the bottom layer of the parameter mapping which closes the feedback loop

through providing the actual sonification; the second layer is essential for the auditory infor-

mation design. With all these opportunities it is important to narrow down the parameters

used to provide a rich yet informative and un-coercive sound-scape. As the potential of

multiple complex auditive data-streaming is yet to be found; the * Pearl sonification mapping

is based on literature and my designer intuition. Alternatives are developed to explore the op-

portunities through qualitative experimentation. These mappings are described further on.

54

Generating Movement Sonification

As briefly mentioned before, the bottom layer of auditory information design is the sonifica-

tion. Continuous parameter mapping sonification is used for the augmented speed-skate

experience. The data streams provided by the second layer of the parameter mapping are

translated into auditive parameters.

It is chosen to use noise band-pass filtering instead of frequency modulation, which would

exhaust the athlete. The use of auditory icons or earcons existing of sound samples which

would include coercive triggers were rejected. A sound-scape is generated by providing a

pink noise stream and continuous manipulation. This pink noise has, contrary to white noise

which has a constant energy distribution over the whole spectrum, a logarithmic distribution

of energy in the sound accommodating the auditory perception.

Band-pass filtering provides the opportunity to manipulate the central frequency of the noise

resulting in pitch modulation. Besides the central frequency, intensity of sound is easily manip-

ulated providing loudness. Intensity highly interacts with the bandwidth of the filter providing

another parameter to play with within one stream of sound. The final manipulation parameter

is the spectral slope of the filter which can be used to generate speed perception. Pink noise

band-pass filtering has a very natural wind and wave like sound and it allows very complex

and rich information.

The modular software design enabled easy configuration of the ranges and transition of in-

tensity and pitch. The sound design is tuned in cooperation with the subject, Frouke Oonk, to

optimally accommodate her auditive perception. Personalizing these setting was required as

everyone has differently trained auditory perception.

Pearl Movement Sonification

The * Pearl movement sonification is intended to deliver a neutral, informative, motivating, un-

coercive, robust and easy to learn feedback, allowing the athlete to discover and reproduce

the sensation of a perfect speed-skating technique. Combining the biomechanics founda-

tion and relevance in balance and technique transmission of speed-skating, pshycho-acous-

tics, aesthetics of interaction, and converging streams required for continuous parameter

mapping led to the following auditory information mapping;

* Pearl mappingThe * Pearl mapping consists of two separate continuous auditory data streams, for

each side (left and right speed-skate) one creating a spatial converge between action

and reaction.

The amount of pressure delivered is sonified through the intensity and loudness of the

band-pass filter; ranging from the absence of sound while lacking pressure to the

intense loudness of the sound while put on full pressure. This is on a continuous scale

providing rich information.

Foot balance is mapped through adjusting the central frequency. Balancing on the

backside of each speed-skate translates in a low sound while balancing on the front

side of the foot translates in to a high sound.

A

B

C

Spectogram of a sound-scape showing pink-

noise manipulated by the central frequency (A),

loudness (B) and band-width (C) parameters.

50000

1

frequency (Hz)

am

plit

ud

e

Balance

Amount of Pressure

Balance

* Pearl Mapping, two auditory streams repre-

senting the balance in pitch and the amount of

pressure in the loudness of the sound-scape.

56

Alternative Movement Sonifications

The alternative auditory information mappings are described in

comparison to the * Pearl mapping as I considered it interesting to

explore the value of the initial * Pearl design. Therefor each alterna-

tive mapping uses the * Pearl as a foundation, enabling qualitative

comparison to some extent.

Alternative A.The * Pearl mapping provides the front and back balance per

speed-skate through pitch variation. This is mapped in a way

that the more pressure on the front, the higher the sound will

be. For alternative A, this is adjusted to a higher sound in case

the pressure balance is at the backside of the speed-skate. By

doing so, it is intended to gain insights in whether the balance

and pitch mapping is intuitively mapping the front balance to

high pitch.

Alternative B.This mapping combines the two separate balance streams

of the left and right speed-skate into one front-back balance

stream. The loudness based on the amount of pressure is still

divided over the two separate streams which now have the

same pitch. This mapping is intended to explore whether it is

possible to combine complex streams, though of similar con-

tent, without losing information richness.

Alternative E.Alternative E is not mainly concerned with the sonification of

pressure, but instead with sonification of sidewards movement

of the speed-skate. For each foot, sideward acceleration is

translated into a higher central pitch sound for greater ac-

celeration. The loudness is pressure dependent as used in the *

Pearl mapping.

This is the only attempt to incorporate the acceleration sensors

within the sonification. Even though acceleration is measured

in three directions, interfering gravity forces do not allow for

robust sonification. The measured data streams are a com-

bination of acceleration and gravity but since the angle of

gravity is changing during a speed-skate stroke it is impossible

to extract the acceleration or derived jerk and speed. This al-

ternative intents to explore whether this information is valuable

event though it is troubled but robust while performing repeti-

tive strokes.

Alternative F.To provoke attention on preventing both speed-skates on the

ice at the same time, alternative F is designed. The movement

sonification of the * Pearl mapping is used as the foundation.

At moments where only one of the speed-skates is on the ice

the sonification is removed. This leaves auditory feedback at

those moments where both skates glide providing information

on the balance and on which speed-skate the most pressure is

transmitted.

Alternative C.The auditory information mapping of alternative C is another

variation on the * Pearl mapping’s balance sonification. In-

stead of directly sonifying the balance into a pitch; the differ-

ence in balance over time is sonified.

Alternative D.While the * Pearl mapping distributes the front and back bal-

ance information over one auditory stream through pitch mod-

ulation, alternative D provides this through separate streams.

This results in four separate audio streams, all solely representing

a single pressure spot. The front pressures are sonified through

fixed high pitch with dynamic loudness based on the amount

of pressure, the back pressures through a fixed low pitch.

By doing so, front and back balance for each speed-skate is

provided. In comparison to the * Pearl mapping this sonifica-

tion intends to explore whether balance on itself (* Pearl map-

ping) or balance through separate stream perception (alter-

native D) is more informative.

Alternative G. On top of the * Pearl mapping, alternative G provides a third

auditory stream. This third stream, which has a stereo effected

spatial origin in the middle, sonifies the stroke frequency de-

rived from the stroke pressure pattern. The higher the frequen-

cy of strokes the higher the sound.

This alternative is intended to explore the scope of multiple

auditory streams containing varying information.

58

ExperimentThis chapter elaborates on the experiments

conducted to gain insights in applied move-

ment sonification in speed-skating. The ex-

perimental setups are described followed by

the qualitative as well as quantitative results

and conclusions.

60

Experimental set-up

As mentioned earlier the research consists of

two experiments. The first experiment de-

signed to fulfil the main objective of gaining

insights in the interpretation of data, recog-

nizability, learning ability, reproducibility and

experience. The second experiment explores

alternative auditory information mappings to

sketch the boundaries of sonification.

SessionsThe experiments were conducted over sev-

eral sessions at ‘Ijssportcentrum Eindhoven’

throughout February and March 2009. In

a total of five day-sessions covering ap-

proximately twelve hours of field-lab studies,

explorative insights in applied movement

sonification were acquired.

SubjectFrouke Oonk, former professional long-track

speed-skater and currently speed-skate

coach, participated as the only subject in

the experiments.

Setup Experiment IThe first experiment focused on answering whether the sonified movement can be recog-

nized, learned and reproduced. This experiment was also intended to figure out whether the

expectations about the sound design were met with the * Pearl mapping. In other words; is

the athlete able to un-coercively interpret and experience converging modalities by using

the movement sonification. To answer these questions it was important to allow the athlete to

get acquainted with the movement sonification. The subject freely explored the sonfication

for half an hour.

Speed-skating with movement sonification, using the * Pearl auditory information mapping,

was compared to speed-skating without any additional sound-scape. The augmented

speed-skate experience platform can either function with or without sonification while still

acquiring and storing movement data.

Two separate tasks were given to the participant. The first goal for the subject was to speed-

skate as steady as possible. With this, it was meant to speed-skate with one technique on the

straight part of the speed-skate track, attempting to repeat a specific stroke. The second task

given to the subject was to speed-skate the straight part of the track leaning on the backside

during the speed-skate stroke as much as possible. Both tasks were performed in four sessions

of four laps in a with-without-without-with order in which ‘without’ means speed-skating the

task without auditory information and ‘with’ speed-skate the task while perceiving the move-

ment sonification.

Data about the experiment were gathered in two ways. The first method for this was quantita-

tive: logging of the movement data throughout all sessions, to enable comparing the move-

ment data of ‘with’ and ‘without’. The second more, qualitative, was through informal evalu-

ations followed by a semi-structured interview about the subjects experience with movement

sonification as a whole. The questions formulated for this experiment and gathering qualita-

tive data about the * Pearl of movement sonification can be found in the appendix.

Setup Experiment IITo gather insights in movement sonification, alternative auditory mappings were designed.

The alternatives described ealrier allow for exploring the boundaries of movement sonification

to some extent. This exploration focused on parameter selection and feedback methodolo-

gies within continuous movement sonification.

Variations and their reasoning for explorative value in comparison to the * Pearl mapping are

described earlier. Each alternative movement sonification and the * Pearl mapping were ex-

perienced by the participant for about seven laps each (35” per lap), to allow some acclima-

tization to the new mapping and sound-scape. No specific tasks, other then just try to use the

sonification, were given. After each session a questionnaire was filled in.

A questionnaire extended with informal evaluations was designed to explore the opportuni-

ties of each mapping. The questionnaire contains questions to qualitatively gain insights in the

first impression, mapping understanding, experience, value for training but also some scales to

rank properties such as the information richness, motivational value, pleasantness, learnability,

complexity, control and coerciveness.

62

Measurements

During all sessions, movement data measurements were recorded with the platform. To pro-

vide understanding in the meaning and value of the data; an explanation on a data sample

is given on the right.

The data gathered for Experiment I can be analysed by comparing the raw data streams

representing the four separate pressure spots. I consider it more valuable to compare the

auditory data streams representing the balance per speed-skate in the pitch and the amount

of pressure in the loudness as mapped in the * Pearl auditory information design. This com-

parison is a more holistic view compared to separate data stream comparison which is more

precise but less informative on the balance required for comparison for the second task.

To compare the ‘with’ and ‘without’ on reproducibility of a specific speed-skate stroke or the

balance over a stroke; visualizations of the auditory display of a set of four following strokes on

the straight end of the track are compared. As speed-skate strokes differ in length (time) it is

decided to synchronize the start of the stroke to enable comparison.

DiscussionThe field-lab experiments took place at ‘Ijssportcentrum Eindhoven’. During the five separate

day-sessions, the weather and speed-skate crowd was of influence on the quality of the ice.

This should not be of influence to the measurements. However, illness of the only subject post-

poned some sessions but more importantly influenced the perception and mental element

during participation. Lack of condition caused by the illness de-motivated Frouke Oonk, on

two days, to some extend because it disabled her to fully experience the augmented speed-

skate experience.

Informal evaluations conducted straight after each measurement session turned out to be

richer then filling out the questionnaire form. The forms were filled out but the outspoken

enthusiasm of the subject during conversations was lost. This rich informal conversation data is

included in the qualitative data analysis and conclusion.

It is questionable whether the amount of data samples used for the first experiment is suf-

ficient. The platform acquired movement data of such a wide amount of different strokes

and speed-skate techniques even though measured on just one subject. The measured data

contains loads of useless movement patterns gathered during cooling down or in-session

data- and sonification-scaling. This led to comparison of data samples from selected areas

out of the sessions.

Speed-skate strokes pressure measured over time

by the Augmented Speed-skate Experience. On

top, abstractions of the athlete’s poses during the

stroke matching the blade position on the ice and

pressure visualization below. Three parts of the

stroke are highlighted; the gliding phase in which

the skate is placed on the ice (A) and the speed-

skater gliding forward and building up sideward

pressure (B); the push-off phase, in which the

gliding sideward push-off occurs and the speed-

skate will klap (C). As shown in the image, the pink

representing the pressure on the back and green

on the front, pressure shifts from the back to the

front at klapping.

0

60

pre

ssu

re (

kg)

time (s)0 20

60

pre

ssu

re (

kg)

A B C

Left speed-skate

Right speed-skate

64

Results

Here, the quantitative measurements conducted for the first experi-

ment as well as the more qualitative forms, formal and informal eval-

uations conducted for the second experiment and overall insights

in movement sonification are presented. All presented visual repre-

sentations of speed-skate stroke movements are available auditorily

on the DVD. The results of experiment II are presented as a summary

containing data and quotes from the informal evaluations and filled

in forms. These in relation to the alternative movement sonifications

to provide insights in the tested dimensions of auditory information

mapping.

Qualitative insight in movement sonificationThe * Pearl auditory information mapping was perceived as a com-

plete representation of the speed-skate technique during a stroke.

Frouke Oonk was able to continuously direct her speed-skate tech-

nique resulting in an sound-scape as intended. The sound-scape

enabled her to distinguish stroke rhythm, the front and back balance

of a speed-skate but also her global balance and the amount of

forces put onto the ice during the stroke. To her, this * Pearl mapping

was a un-coercive, rich informative sonification of the complete

speed-skate stroke progress. This sonification matched the movement

intuitively. Over time, more and more layers of information of the

speed-skate stroke progress came clear in the direct feedback loop

such as the switch between speed-skates, and the force progress in a

corner.

As a speed-skate coach, Frouke Oonk envisions many opportunities;

on one hand the coercive mappings can be useful for novice ath-

letes while on the other hand it satisfies the request for rich informa-

tive sonification for professional speed-skaters to learn more about

their technique. A danger indicated by Frouke Oonk is the possibility

of over-focusing; relaxation and listening to the body can be forgot-

ten.

To listeners who never used the augmented speed-skate experi-

ence, the sound-scape recordings of the sonified movement sound

like meaningless wind. To Frouke Oonk, on the other hand, this wind

caused by her movement provides a continuous rich information

flow.

Frouke Oonk enjoyed to process of exploring, discovering, practic-

ing and improving her technique. She discovered and reproduced

the sensation of a good stroke. “Ik heb in jaren niet meer zo lekker

geschaatst” (I did not speed-skate so well, for years), a clear indica-

tion of the motivational value of the augmented speed-skate experi-

ence. The movement sonification stimulated her to go faster but also

to consciously improve her technique caused by the provided new

sense modality.

About three to four laps, considering the sonification, were required

to complete the so-called cognitive phase of the motor-learning

process. This phase is described as the process of identifying and

developing a mental picture of the component parts required for the

skill. After eight hours of using the * Pearl mapping, signals indicated

that Frouke Oonk had reached the autonomous phase resulting from

the associative phase. Within the associative phase the * Pearl map-

ping as well as all alternative mappings forced the subject to speed-

skate and listen consciously, providing a smooth intermodal conver-

gence through exploring and practicing the speed-skate technique

using auditory information. The final phase which had been reached

involves little or no conscious thought or attention while performing.

This stage was identified while Frouke mentioned to not have used

the sonification consciously while perception of sonification blended

in with other sense modalities. She could still switch attention from

speed-skating to specific information provided in the sound-scape.

In early sessions when the movement sonification was not fine-tuned;

small annoying blurbs of sound occurred through the rich move-

ment sonification. At first they were annoying but after five laps these

sounds were ignored by Frouke’s perception. Slight masking of sound

took place through the convergence of the movement sonification

and the actual sounds generated by the movement of the athlete

and the surrounding. This was not an issue as the convergence of ac-

tion extends to the sound-scape, though at the shoot-start of a race

the loudness of the klap-skate overruled the sonification.

66

Experiment IAs shown in the figures on the left; the repetitive stroke performed us-

ing the movement sonification shows slightly less deviation compared

to the repetitive stroke performed without the extra sense modality.

Graphical representations of the speed-skate stroke sequence

performed during the second task indicate that the balance on the

back-side of the speed-skate during a stroke can be maintained

longer using the movement sonification compared to speed-skating

without auditory feedback. It seems that the movement sonification

enables the athlete to lean to the back for a longer period, resulting

in a less gradual transition towards the front of the speed-skate.

Experiment IIAlternative A, which switches the sonification mapping between the

front and back balance relation with the pitch, was extremely well

learned. The mapping was understood but combining the high pitch

with a balance to the back was not intuitive understood. Masking of

sound took place during the speed-skate strokes.

The exploration in combining complex though similar content streams

provided by the movement sonification of alternative B indicated

the requirement for a spatial relation between sound and movement

source as the mapping was hard to learn and lacking information

richness.

Alternative C, which provided a difference in balance through soni-

fication, was not understood as designed. Even though Frouke Oonk

was able to interact with the sonification, the sonification forced her

to pay attention to the unrolling of the balance. This auditory informa-

tion mapping coercive but not informative to the athlete.

Alternative D that provided separate streams of audio, contrary to

the balance stream of the * Pearl mapping, was learned and per-

ceived as valuable. However, separating the streams shifted the

attention from the balance over a speed-skate stroke to creating

forces on the separate spots.

Sounds from Mars as described by Frouke Oonk are the result of Alter-

native E’s auditory information mapping. Even though the sonification

of acceleration data formed a clear pattern, no correlation between

the movement and sonification was perceived.

Four consecutive speed-skate strokes of left speed-

skate by Frouke Oonk performing repetitive stroke

task ‘without’ movement sonification.

Repetitive stroke task performed ‘with’ movement

sonification five minutes before the ‘without’ ses-

sion.

With movement sonification, the repetitive task

is performed with less deviation between stroke

technique.

Comparison between ‘with’ (green) and ‘with-

out’ (purple) movement sonification. Four consecutive speed-skate strokes of left speed-

skate by Frouke Oonk performing lean-back task

‘without’ movement sonification.

Lean-back task performed ‘with’ movement sonifi-

cation five minutes after the ‘without’ session.

0

60

0

60

0

60

0

60

pre

ssu

re (

kg)

pre

ssu

re (

kg)

pre

ssu

re (

kg)

pre

ssu

re (

kg)

time (s)

time (s)0

0

0

time (s)

time (s)

time (s)

time (s)

0

0

0

0

ba

lan

ce

ba

lan

ce

ba

lan

ce

ba

lan

ce

ba

lan

ce

ba

lan

ce

1

9 9

1

1

9 9

68

Conclusion

Even though results of both experiments are based on limited sets of data samples and

thereby do not provide hard-evidence, it does seem that the * Pearl mapping offers neutral,

informative, motivating, un-coercive, robust and easy to learn feedback on the speed-skate

technique. This, contrary to the slight variations of this auditory information mapping. The

alternatives established intermodal converge to some extend but had coercive characteris-

tics and limited informative value. The * Pearl mapping can still be extended as alternative G

indicates opportunities for additional auditory streams.

Within eight hours of use, the athlete experienced the cognitive phase in exploring the

capabilities of the sound-scape in the first laps, the associative phase in which the athlete

consciously trained the skills and the third phase of motor-learning; the autonomous phase at

which the athlete used the sound-scape unconsciously and as part of the athlete’s sensory

system.

Results seem very promising and therefor further extensive research on reproducibility is ad-

vised. The augmented speed-skate experience platform can fulfil this role. The results follow-

ing the exploration of the reproducibility of a specific speed-skate stroke and a specific task

concerning technique, indicate that the movement sonification is recognized, learned and

that converging modalities are experienced. The sonification of movement is perceived and

interpreted as intended by the athlete.

The opportunity to explore and play with the speed-skate technique in many aspects added

to the experience of some kind of perfect-stroke reproduction, is very motivating to the ath-

lete. To fully understand the value of the movement sonification, the intermodal convergence

should be experienced.

Alternative F provoked attention while both speed-skates were on the

ice at the same time, another Mars sound-scape. Frouke Oonk saw

this as an added value for specific speed-skaters who need to learn

how to prevent this as this sonification is rather coercive and putting

attention on long gliding speed-skate strokes. Furthermore, this audi-

tory mapping did not provide control to play with the sonification.

The final alternative, alternative G, intended to explore the scope of

multiple auditory streams containing varying information. The infor-

mation streamed in this way was perceived clearly. On top of the

richness provided by the * Pearl mapping, Frouke Oonk was able to

explore and play with the additional auditory stream and she was

able to switch attention between the streams without losing the com-

plete overview of the speed-skate stroke.

The * Pearl mapping was judged the most information rich, un-coer-

cive and most easy to learn by the subject. As indicated earlier, audi-

tory information mappings could be rather coercive. These coercive

mappings can be perceived as most useful, motivating and informa-

tive but they actually direct the athlete towards a specific speed-

skating technique provoked by the sonification.

70

Redesign

Although the principle of Movement Sonification does not seem to require absolute values of

data streams, technical speed-skate analysis does. It is recommended to gauge the relative

though solid values towards absolute pressure values represented in Newton, this can be ac-

complished within the software.

Attempts to do so failed because of slight nonlinearity of the sensors. Compensating this

would not be an issue when the sensor placement causes less variability. Therefor I recom-

mend to upgrade the pressure sensor in a way that it is less sensitive to positioning between

the bridge and speed-skate shoe. Pressure should be sensed around the bold attaching the

shoe and bridge to prevent lever and movement of the sensor.

As mentioned before, measured acceleration is ‘spoiled’ by gravity. To acquire solid ac-

celeration data for speed-skate technique analysis purposes it is recommended to add a

gyroscope (gravitation filter) to the printed circuit board.

As a final phase of applying movement sonification in a marketable product; sonification

should not be provided by an external source such as the computer in the current platform. It

is recommended to prevent long-distance data transmission to limit data loss and sonification

delay. Once the design of an auditory information mapping is finalized for use, the device at-

tached to the speed-skate should process the movement sonification directly to the athlete.

RecommendationsThe platform that is build for this project offers

several research continuation opportunities.

In the first place it can provide insights in the

individual technique of top speed-skaters.

In the second place, movement sonification

has yet to be explored.

The following paragraphs sketch suggested

outlines for further exploration and research

in the field of Movement Sonification. At first

technical improvements on the platform are

recommended.

72

Movement Sonification (

The promising results from qualitative experiments conducted for this augmented speed-

skate experience project provide enough confidence to continue the research of applied

movement sonification in the field of professional speed-skating. Results gathered within my

project indicate the potential of movement sonification and the specific design approach

as described in the * Pearl mapping. It is recommended to conduct a long-term experiment

with multiple subjects to gather large amounts of data to profoundly ground the results on

the reproducibility, the learning phases and platform dependency. This should help to prove

whether movement sonification actually becomes an extended sense through intermodal

convergence.

It is interesting to further explore the information richness that can be contained in the audi-

tory information mapping. Results gained from alternative G indicate opportunities. Sonifica-

tions should be designed un-coercive and according to natural kinetic-acoustic relations

to converge perceptional streams. Sound masking should be prevented and sound design

should accommodate the individual auditory perception.

Combining the earlier mentioned acceleration data with the measured balance data should

provide insights in the ‘perfect stroke’. Once this perfect stroke is found beyond the individual

athletes perception of flow, auditory information mappings can be designed specifically

to persuade the athlete towards this rationalized judgement of good and bad. Personally, I

sport-ethically disagree with this model-based coercive approach as it will decrease the ath-

lete’s personal contribution in the performance.

74

Acknowledgments

First of all I thank the client, TVM Schaatsploeg, for its time, hospitality

and effort put into my project. Thanks Gerard Kemkers, Hans van Kuijk

and Geert Kuiper for sharing your knowledge and for your contribu-

tions during the creative sessions and client meetings. Thanks to the

supportive staff and athletes for a warm welcome. Further I would like

to thank Ireen Wust, for your enthusiastic participation in the prelimi-

nary field-test, Gerard Rietjens for your overall support, effort and trust

in the project and Frouke Oonk for participating in the field-lab stud-

ies conducted at ‘Ijssportcentrum Eindhoven’.

Thanks to my supervisors and supportive staff at the Faculty of Indus-

trial Design that inspired me over the past one and a half years. Alan

Murray, thank you for initiating and supervising the extend your senses

project in the fall of 2007. René Ahn and Stephan Wensveen your

supervision, support, inspiration, dedication, profound feedback and

pushing me to heights though putting my feet on the grounds over

the past years was of great value. Kees Overbeeke, your devotion

has been great, thanks for your faith and inspiration.

Dik Hermes and Berry Eggen, your contributions of sound design

methodology was of great value. Even though we never met, Alfred

Effenberg you inspiring me on the subject of movement sonification.

Thanks to the Department of Industrial Design for supporting my ambi-

tions. Sabine van Gent, Caroline Hummels, Esther Gielen and the

Board of Examiners I appreciate your outspoken confidence in the

project and enabling me the opportunities to reach my goals. Chet

Bangaru, Sjriek Alers, Geert van de Boomen and Peter Peters among

other staff of the Faculty of Industrial Design and Gemeenschap-

pelijke Technische Dienst of Eindhoven, University of Technology, your

technical support in developing the prototypes was generous. René

de Torbal, thanks for your support on legal and intellectual property

matter.

Friends, thanks for your love and distracting my mind from obsessively

working on the project. Jan, Jildou, Anne Maaike and Flávia, I am

grateful for your patience with me, your warmth, love and never-end-

ing confidence you have shown in me throughout the years.

76

References

Biomechanics

R.M, Bartlett, “Introduction to Sports Biome-

chanics”, Chapman and Hall, London, 1996

H. Houdijk, “The Klapskate, Shifting Gears in

Speed Skating”, Proefschrift Vrije Universiteit,

Amsterdam, 2001

G.J. van Ingen Schenau, “The Influence of Air

Friction in Speed Skating”, Journal of Biome-

chanics, Vol. 15. No. 6, Pages 449-458, 1982

G.J. van Ingen Schenau, G. de Groot and

R.W. de Boer, “The Control of Speed in Elite

Female Speed Skaters”, Journal of Biome-

chanics, Vol. 18, No. 2, Pages 91-96, 1985

J.J. de Koning, G. de Groot and G.J. van In-

gen Schenau, “Coordination of Leg Muscles

during Speed Skating”, Journal of Biome-

chonics, Vol. 24. No. 2, Pages 137-146, 1991

J.J. de Koning, G. de Groot and G.J. van

Ingen Schenau, “Ice Friction During Speed

Skating”, Journal of Biomechanics, Vol. 25,

No. 6, Pages: 565-571, 1992

J.J. de Koning, H. Houdijk, G. de Groot and

M.F. Bobbert, “From biomechanical theory

to application in top sports: the Klapskate

story”, Journal of Biomechanics Vol. 33, Pag-

es 1225-1229, 2000

Movement Sonification

A.O. Effenberg and H. Mechling, ” Bewegung

hörbar machen - Warum? Zur Perspektive

einer systematischen Umsetzung von Bewe-

gung in Klänge” Psychologie und Sport, 1,

29-38.,1998

A.O. Effenberg and H. Mechling, “The Contri-

bution of Motor-acoustic Information to Mo-

tor learning” In Proceedings of the 4th Annual

Congress of the ECSS, Rome, 780., 1999

A.O. Effenberg and H. Mechling, “Multimo-

dal Convergent Information Enhances Per-

ception Accuracy of Human Movement

Patterns” In Proceedings of the 6th Annual

Congress of the European College of Sport

Science (ECSS), 2001

A.O. Effenberg, “Movement Sonification: Ef-

fects on Perception and Action” Multimedia,

IEEE, Vol.12, Iss.2, April-June 2005, Pages: 53-

59, 2005

A.O. Effenberg, J. Melzer, A. Weber, and A.

Zinke, “MotionLab Sonify: A Framework for

the Sonification of Human Motion Data” In

9th International Conference on Information

Visualisation (IV’05), Pages 17–23, London,

UK. IEEE Press., 2005(2)

A.O. Effenberg, “Movement Sonification: Mo-

tion perception, behavioral effects and func-

tional data” In Proceedings of the 2nd Inter-

national Workshop on Interactive Sonification,

York, UK, February 3, 2007

W.W. Gaver, “Auditory Icons: Using Sound in

Computer Interfaces” In Human-Computer

Interaction, Volume 2, Issue 2, Pages 167-177,

1986

C. Henkelmann, “Improving the Aesthetic

Quality of Realtime Motion Data Sonication”

In Computer Graphics Technical Report CG-

2007-4, 2007

T. Hermann, O.Honer, H. Ritter, ”AcouMotion

- An Interactive Sonication System for Acous-

tic Motion Control” In GW 2005, pp. 312-323,

2006

G. Kramer, Ed., Auditory Display - Sonication,

Audication and Auditory Interfaces. Addi-

son-Wesley, 1994

J C. H. Shea, G. Wulf, J. H. Park, and B. Gaunt,

“Effects of an Auditory Model on the Learn-

ing of Relative and Absolute Timing” Journal

of Motor Behavior, 33:127–138, 2001

K B.N. Walker, G. Kramer, “Mappings and

Metaphors in Auditory Displays: An Experi-

mental Assessment” In ACM Transactions on

Applied Perception, Volume 2, Issue 4, Pages

407-412, 2005

78

Reflections

To conclude the project academically I

hereby briefly reflect on the following matter

in relation to the augmented speed-skate

experience project:

How did the project influence my designer

identity, explore my design process, expand

my skills, research, design integration and

complexity of interests.

Design identityThis academic project helped me in devel-

oping this vision on design. With my concepts

I attempt to create opportunities for the

user to explore, learn, relate, distinguish and

improve their unique abilities in their context.

I envision transformation through a holistic

highly dynamic design process involving

validation in context. To me it is important to

integrate the perceptual-motor skills within

the interaction.

A professional attitude, devotion, high qual-

ity prototyping and passion for design are

important to me. As I will always stay ambi-

tious I will need to surround myself with equal

minded to support me in the process of

achieving great things.

Design processMy Augmented Speed-skate Experience

project has always been about applying

movement sonification in some way. Even

though it took me about one year to reach

the point of being able to actually imple-

ment and research; movement sonification

has always been the drive. I consider my

design approach an example of the trans-

formative reflective design method with an

emphasis for vision (envisioning transforma-

tion in society) and multiple parallel design

dimensions integrated through several small

design cycles. To me it was very important

to combine academic research with actu-

ally building low-fidelity prototypes to experi-

ence. My design was guided by reflection,

vision and a good load of process anticipa-

tion. This fits me, though it requires loads of

energy. I can still learn how to spread my

workload or lower my ambitions.

CraftsmanshipIn the process of building the Augmented

Speed-skate Experience’s measuring devices

several crafts were executed. These skills

grew over the years and at this point in time

I consider to have reached my limitations

as far as I would like to make them grow.

Electronics and transforming the drawings

into three-dimensional casings, vacuum-

casted in polyurethane; are time consuming

processes which could better be executed

by experts. I do not plan to further develop

these skills as they exceed my ambitions as

designer. Nonetheless, I embrace these skills

in the process of creating custom high-fidelity

prototypes being independent of others.

ResearchMy research did not use validated meth-

odologies, though I do consider to have

gathered great insights in applied movement

sonification through field-lab experiments

and research through design. Defining a

research focus in an unexplored field was a

challenge as I would like to do it all. This was

not feasible, so the decision to create one

auditory information mapping and testing

whether the concept of movement sonifica-

tion works was a good solution. I found it very

interesting to design the experimental setups

to gain qualitative insights on recognizabil-

ity, learnability and reproducibility which

eventually would support the concept of

movement sonification as converging sense

modality and its value. I was very satisfied to

support my gut instinct and confidence in the

concept of movement sonification with the

experimental results.

CollaborationDuring the project I collaborated with the cli-

ent, coaching staff and quite some experts.

Communication was troubled by the non-dis-

closure agreement which did not allow me

to share details providing a complete story.

This led to collaborations at which I integrat-

ed the specific knowledge into one design

and research project because outsourcing

became complicated. For further projects I

envision a more open work atmosphere at

which it would be easier for me to seduce

collaborators towards a shared project out-

put.

The complexity of interests by the Faculty

of Industrial Design, myself and the TVM

Schaatploeg caused some trouble in defin-

ing focus. My ambitions to deliver quality, the

clients request for measuring platforms and

the Universities demand for a solid curriculum

stalled the process. Contractual negotiations

between the client and the University led by

myself clearly indicated these conflicts of

interests.

Opportunities to deliver all requirements were

investigated but not found at the Universities

resources. It is very important for me to clearly

define my own and the faculties capabilities

and project boundaries on forehand.

80

Client Feedback

At several occasions over the past years; the client, TVM Schaatsploeg, was closely involved

in the projects process. Their involvement consisted of providing materials, knowledge and

facilities but also participation in stakeholder meetings, creative sessions, field lab studies

and presentations. Throughout the project, they expressed an outspoken confidence in the

design.

The following formal feedback was published on June 27th 2008 by head coach, Gerard

Kemkers, of the TVM Schaatsploeg. This was in response to the introduction of the first proto-

type and finalizing of the design project conducted for the Faculty of Industrial Design.

De introductie van jezelf en van jouw produkt is enorm goed verlopen. De technische

staf ziet erg veel kansen in hetgeen je hebt geproduceerd en de schaatsers van de

TVM schaatsploeg waren behoorlijk onder de indruk (en dat gebeurt niet zo heel erg

vaak). Er is een enorm goede basis gelegd!

Wij zijn dan ook van mening dat je een zeer werkbaar model hebt gemaakt, waarbij

het grootste voordeel is dat de eigen schoenen en de eigen buizen gebruikt kunnen

worden. Natuurlijk zullen we aan de data-collectie (het meten zelf) mogelijk nog wat

moeten sleutelen, maar de conclusie kan met recht worden getrokken dat deze eerste

fase zeer goed is afgerond en door de TVM schaatsploeg zeer positief is ontvangen.

Als ik kijk naar het prototype wat je hebt gemaakt, dan hebben we een enorme stap

gemaakt door het meetinstrument te kunnen koppelen aan de bestaande schaats.

Aangezien iedere schaatser een eigen en specifieke combinatie van buizen, beugels

en schoenen heeft, gaat het nooit werken om met een standaard-meetschaats de

juiste gegevens te verzamelen. Het is jouw gelukt om een toevoeging aan de schaats

te maken, zodat iedereen op zijn eigen set schaatsen getest kan worden.

Om de praktische bruikbaarheid te verbeteren zou het een enorm winstpunt zijn als we

draadloos de gegevens kunnen gaan meten of dat we de gegevens ‘in de schaats’

kunnen opslaan, waarna na gebruik de geheugenkaart uitgelezen kan worden. In dat

geval zitten we heel dicht bij een ‘normale’ schaats en kunnen we zuivere en reële

metingen verrichten.

Schaatsen is wetenschappelijk gezien een erg jonge sport. Er zijn nauwelijks meetgegevens

en andere data over schaatsen bekend. Telkens weer wordt er gezocht naar mogelijkheden

om goede metingen te doen. Met jouw prototype lijken we een hele grote stap voorwaarts

gemaakt te hebben. Belangrijk is vooreerst dat we het prototype kunnen upgraden naar een

te gebruiken meetinstrument. Als ons dat echt lukt, dan dienen we een aantal in ons bezit te

hebben zodat we dit passend kunnen maken op verschillende maten beugels.

Vervolgens kunnen we data gaan verzamelen en van die verzamelde data leren. Waarom

schaatst hij hard in de bochten? Waarom een andere op de rechte stukken? Zien we over-

eenkomsten bij sporters die hard en goed schaatsen? Wat verandert er als iemand een

‘slechtere’ periode heeft? Enzovoort!

Uiteindelijk is het mijn unieke idee om een real time auditieve feedback naar de schaatsers

terug te sturen. Op basis van geluiden kunnen ze dan hun techniek aanpassen en optimalis-

eren. Laat staan dat het uiteindelijk de mogelijkheid geeft om schaatsenthousiasten op een

andere manier schaatsen te leren. Deze laatste toepassingsmogelijkheden zijn echter de

laatste stap in dit proces. Tevens voeg ik er graag aan toe dat dit idee vertrouwelijk behan-

deld dient te worden.

Ik ervaar het traject dat we tot nu toe doorlopen hebben als zeer succesvol. Het resultaat

mag er zijn en voldoet aan alle gestelde eisen, die wij vooraf hebben doorgesproken. Ik ben

erg nieuwsgierig naar de volgende stappen. Interessant was ook het feit dat het prototype

al de vormgeving van een eindproduct had. Strak en af! Niet een vierkant kastje, maar een

slank en goed passend meetinstrument. Bovendien doet het helemaal geen afbreuk aan het

ontwerp van de schaats/beugel en oog de combinatie snel. Daarbij heb je oog voor detail

gehad door een onderdeel van jouw instrument mooi in TVM-groen te spuiten.

Mijn dank voor de samenwerking tot nu toe! Succes met de presentatie van volgende week

en hopelijk kunnen we nog wat stappen voorwaarts maken met de ontwikkeling van de

‘meetschaats’!

Met vriendelijke groet,

mede namens de TVM schaatsploeg,

Gerard Kemkers

hoofdcoach

82

Appendices

Alternatives scheme

Images

Interview

84

Alternatives scheme

* Pearl Mapping

Alternative A

Alternative B

Alternative C

Alternative D

Alternative E

Alternative F

Alternative G

* Pearl Mapping, two auditory streams repre-

senting the balance in pitch and the amount

of pressure in the loudness of the sound-scape.

Pressure

Balance through pitch (back equals low)

Amount through loudness (no pressure equals silence)

Balance through pitch (back equals high)

Combining the left and right balance. Loudness equals * Pearl.

Change of pressure through pitch and loudness as applied in the * Pearl.

Four separate streams, fixed pitch, pressure through loudness.

Sideward acceleration through pitch while pressure.

Equals * Pearl if both speed-skates are on ice.

Added stream, stroke frequency through pitch.

86

88

90

Colophon

Text Jelle Stienstra BSc.

Prof. dr. Kees Overbeeke (correction)

Anne Maaike Stienstra BSc. (correction)

Photographs and illustrationsAll photos and illustrations with exception of the photos on pages 12,

13 and 22 by Jelle Stienstra

Eindhoven, University of Technology (p. 12)

http://w3.id.tue.nl/nl/education/masters_program/design_research_projects/

bat_biker/

Nederlandse Omroep Stichting (p. 13 and 22)

http://www.nos.nl

Faculty of Industrial Design Eindhoven, University of Technology

Den Dolech 2

Postbus 513, 5600 MB Eindhoven

Netherlands

Jelle Stienstraj.t.stienstra@madebyjelle.com

www.madebyjelle.com

92