interacting with an intelligent dancing figure: artistic experiments at the crossroads between art...

Leonardo

Interacting with an Intelligent Dancing Figure: Artistic Experiments at the Crossroadsbetween Art and Cognitive ScienceAuthor(s): Michel Bret, Marie-Hélène Tramus and Alain BerthozSource: Leonardo, Vol. 38, No. 1 (2005), pp. 46-53Published by: The MIT PressStable URL: http://www.jstor.org/stable/1577645 .

Accessed: 12/06/2014 23:45

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

The MIT Press and Leonardo are collaborating with JSTOR to digitize, preserve and extend access toLeonardo.

http://www.jstor.org

This content downloaded from 188.72.126.118 on Thu, 12 Jun 2014 23:45:47 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=mitpress

http://www.jstor.org/stable/1577645?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


GENERAL ARTICLE

Interacting with an Intelligent

Dancing Figure: Artistic

Experiments at the Crossroads

between Art and Cognitive Science

Michel Bret, Marie-Helene Tramus and Alain Berthoz

I nteractivity has introduced a certain type of sensorial awareness in the arts, especially when considered from the point of view of the spectator. Our hypothesis is that this sensorial aspect also may be envisaged from the point of view of the work of art itself, as the work has become endowed with perceptions of its own. This raises one of the most crucial ques- tions in contemporary digital arts: that of the relationships between natural and artificial "perception-movement-action" functions. One of our aims is to create art installations showing virtual actors who are endowed with artificial perceptions that enable them to react in an autonomous way to the cues given by a spectator, thus opening art and cognitive science to a whole new range of possibilities for the exploration of virtual life.

A SECOND INTERACTIVITY State of the Art Our purpose is set in the context of interactive arts in relation to artificial life and builds on research that we will present briefly and nonexhaustively in order to give a series of reference points.

Sparacino [1] distinguishes systems that are merely "reac- tive" (systems in which sensors transfer data from the audi- ence's actions to scripts that map predefined reactions) from "behavioral" systems (systems that apply the results of classi- cal artificial-intelligence research, such as, for example, "group behavior" theory, introduced by Reynolds in 1987 [2]) and, finally, from "autonomous systems," first introduced by Brooks [3] in the case of robotics, then developed by Maes [4,5] and first applied to the arts by Sims [6,7]

Blumberg [8,9] elaborated a general model for perception and action selection in real time. He then made a model of a dog that was capable of interacting with human beings, as well as with other virtual actors, in the behavioral mode in the meaning given above. With the "NeuroAnimator" [10], Ter- zopoulos offered a new approach in order to create anima-

Article Frontispiece. Michel Bret, Marie-Helene Tramus and Alain Berthoz, The Virtual Tightrope Walker, 2004. (? Michel Bret) Virtual tightrope walker interacting with a spectator.

ABSTRACT

The authors (a neurophysiolo- gist and two computer artists) give an account of a collabora- tion that took place within the framework of a study-cum- artistic experiment on virtual interactive figures at the bound- ary of art and cognitive science. This study, called "'Intelligent' Interactivity (Connectionism, Evolutionary Science and Artificial Life) in Digital Arts in Relation with the Physiology of the Perception of Action and Movement," was supported by .1hk r;-f-- I))rnrae

tions that are realistic in physical Lre ugnique uuu rrogram . . * * i, r r ~on Art and Cognition, an initia- terms by exploiting the features of of the French Ministry of

neural networks that are trained of- Research. fline in order to imitate the dy- namics of moving physical models. Cognitive Modeling Language (CML) [11] extends behavioral models by controlling what a virtual actor knows, how it acquires the knowledge and how it uses the knowledge in order to plan its actions.

According to Guillot and Meyer [12,13], the animat approach postulates that it is possible to study human cognition through a bottom-up approach that starts with minimal control architectures and simple environments and then gradually adds complexity to them. Evolutionary robotics applies the laws of genetics and natural selection to encode a robot's phenotype in its genotype; the robot is then submitted to an artificial process of natural selection by using genetic algorithms [14,15] and genetic programming [16].

Intensive research has been done on "cyberdance." Among the most eloquent examples is Cunningham's use of Calvert's [17,18] Life Forms program. Thalmann [19,20] realized per- formances by putting virtual actors on stage alongside real actors. In her show DanceSpace, Sparacino [21] generates, in real time, music and images from a real dancer's movements. She also applied this approach to the theater in TheaterSpace. Also in the realm of theater, Lambert-Wi's staging of Pasolini's Orgia used the Daedalus system, which creates artificial organisms from data sent by sensors recording the actor's stress and emo- tional levels [22].

Connectionism constitutes an advance in the sense that it allows the representation of complexity through the use of a network. We position ourselves in the trend inspired by connectionism, first explored by Van de Panne and Fiume [23]

Michel Bret (artist, educator), Departement Arts et Technologies de l'Image, University of Paris 8, 93256 Saint-Denis, France. E-mail: <[email protected]>.

Marie-Hel6ne Tramus (artist, educator), Departement Arts et Technologies de l'Image, University of Paris 8, 93256 Saint-Denis, France. E-mail: <[email protected]>.

Alain Berthoz (physiologist, educator), College de France (Laboratoire de la Physiologie de la Perception et de lAction), 75005 Paris, France. E-mail: <[email protected]>.

LEONARDO, Vol. 38, No. 1, pp. 46-53, 2005 47 ? 2005 ISAST



i: .. .,:

. .. ha



Fig. 1. Michel Bret, Marie-Helene Tramus and Alain Berthoz, The Virtual Tightrope Walker, 2004. (? Michel Bret) Stephanette Vendeville interacting with the virtual tightrope walker.

and Sims [24], by applying the results of connectionism to our animated figures, which had to adapt to an unknown and changing environment.

"Body-Thought" We chose to base our research on models drawn from cognitive science and biology, especially connectionism, genetics and the physiology of perception and action, in order to move towards what we suggest calling "second interactivity," in reference to "second cybernetics" [25], which deals with more complex and fuzzy relationships that are closer to intuitive human behavior, compared with previous cybernetics. Even though the work produces effects related to the meaning (effets de sens), these impressions are not primarily related to the play of language, concepts and symbols, but to an often unknown and undervalued form of thought that Tramus [26] calls "body-thought." The body this art deals with is not only the spectator's: The spectator also en- gages in a dialogue with a counterpart that the work calls into existence. Berthoz calls this counterpart the "doppelganger"

[27]. The doppelganger is characterized by the fact that it is a mirror-image, yet it keeps the separate identity of an autonomous being. This ambivalence is at the source of emotions that only art can provide.

Autonomy According to Varela, autonomy means in- ternal law related to self-generation, self- organization and the affirmation of identity. Autonomy is opposed to allon- omy, which is external law, or commands [28]. The connectionist approach offers a possible direction (but not the only one) for experimental interactive art, as it gives the virtual creature a certain degree of autonomy due to neural networks that generate unpredicted and nonex- plicitly programmed behavior.

Alongside a call for the reintegration of action and movement, which, according to Berthoz [29], are at the very heart of brain studies, our installations show a move in favor of a kind of digital art that also draws inspiration from physical sensations and movement. We thought that neural networks, which have the capacity

to self-configure, would be favorable for the development of experiments on the "body-brain-environment" interactions of a virtual creature.

"INTELLIGENT" INTERACTIVE INSTALLATIONS

We designed a virtual character to obey biomechanical laws and endowed it with reflex behavior patterns that help it maintain its balance on the ground. Fur- thermore, neural networks enable the virtual character to react to the spectator's movements in an "intelligent" way.

Two Installations with an Interactive Virtual Character In an installation called The Virtual Tightrope Walker, the spectator is invited to become a tightrope walker (Article Frontispiece). The image of the virtual tightrope walker is shown on a screen fac- ing the spectator, who is equipped with a movement sensor attached to his or her waist. This sensor sends information about the position and the orientation of the spectator to the computer, which

Fig. 2. Michel Bret, Marie-Helene Tramus and Alain Berthoz, The Virtual Tightrope Walker, 2004. (? Michel Bret)

48 Bret et al., Interacting with an Intelligent Dancing Figure



Fig. 3. Michel Bret, Marie-Helene Tramus and Alain Berthoz, Dance with Me, 2004. (? Michel Bret) Virtual dancer interacting with a spectator.

interprets the data in real time as a set of forces that influence the moving synthetic actor, controlled by neural networks. The tightrope walker is not a copy of the spectator, but rather an artificial being that is sensitive to the spectator's movements. If the spectator tries to un- balance the tightrope walker, she will attempt to regain her balance by devel- oping autonomous strategies in real time. These strategies are the result of a previous training phase. The duet between both "actors" then develops around a game of imbalance and balance (Figs 1- 2, Color Plate B No. 1).

In the installation Dance with Me, the spectator is now invited to interact in real time with a virtual dancer. Again the spectator interacts through the means of a movement sensor attached to her waist. The sensor's variations in speed are interpreted by the computer as forces in- fluencing a virtual body set in a gravity field and constrained by a floor it cannot fall through. When it faces a moving spectator, the virtual dancer improvises dance steps that are a compromise between previously learned choreography and balancing strategies, and the spectator's movements (Fig. 3). This artificial being, albeit elementary and far removed from the very complex forms of natural adaptation and anticipation, goes beyond simple retroactive loops in the way it includes certain features of live creatures. For example, generalization, a property of neural networks, endows the virtual dancer with a potentially unlimited array of unlearned, yet adapted, reactions. Its intelligence appears as a feature emerging from interactions between its elements (artificial neurons), the information it senses in its environment, and its struc- ture (the simulation of a human body endowed with certain behavior patterns).

Experiments with Spectators, Acrobats, Dancers: Interdependence vs. Autonomy Every time we showed the work and observed spectators react, experience taught us to gradually understand the is- sues of the relationship between both beings, in particular the delicate balance between autonomy and interdependence [30]. The response of the previously trained networks is modulated by the spectator's intervention via the sensor, so the data coming from the neural network and the data issuing from the sensor- related interaction module are mixed. Setting the right proportions between both of these sources is essential, as this is what allows the virtual and the real beings to act together in a complex balance of mutual interdependence and autonomy.

We filmed some of the moments of these various displays. The films show careful spectators seizing the balancing pole, looking at the tightrope walker and hesitantly starting to move. Some of them copy the acrobat's movements in order to become tightrope walkers themselves by reproducing the same gestures a split second later. This has produced beauti- ful scenes showing moments of harmony that surprised the spectators looking for their mirror images. This configuration differs from usual experiments involving synthetic clones, as the latter copy exactly the gestures of the movement-sensing de- vice to which they are enslaved. But the

spectator's mimetic quest for a duplicate is soon eclipsed by his own copycat behavior, which, on the contrary, upsets the virtual figure's balance and thus disrupts the whole choreographic harmony. The spectator makes new attempts, and the succession of these new attempts produces more movement similarities and

more disruptions of the figure's balance, thus establishing an original interrela- tion of gestures.

An experienced tightrope walker put her skills in practice to interact with the virtual tightrope walker. It was very moving to see the gestures of equipoise and imbalance of both performers together and to sense the similarity between them, for instance in the ways they enhanced their tread in order to regain their balance, leaning either backward or for- ward, and in the balancing moves of the torso, but also in the way the figure moved in search of its balance or when it lost its balance. It was mesmerizing to see the real tightrope walker laugh and exclaim in front of the virtual figure's ex- travagantly imbalanced poises. It was fas- cinating to witness the human tightrope walker scrutinizing the virtual one, as if she wanted to guess its "intentions."

Technical Description These installations, designed by Michel Bret [31], comprise four modules: the dynamic module, the behavioral module, the connectionist module and the interactive module.

The dynamic module calculates the movements of a body subjected to different forces, gravity, environments, biomechanical constraints and simulations of forces provided by the sensors and the behavioral module.

The behavioral module simulates re- balancing reflexes and forces the generation of voluntary movements provided by the connectionist module.

The connectionist module builds adaptive strategies in real time, because of a neural network whose inputs are connected to the interactive module and whose outputs are interpreted as movement schedules. The schedules are

Bret et al., Interacting with an Intelligent Dancing Figure 49



Fig. 4. Diagram showing how sensors are connected to the network inputs and how the network outputs are connected to the actuators.

vised networks, such as Kohonen's [33], in the future. Such networks should allow the virtual being to discover the regular features of its environment as well as to utilize the most adapted responses.

This interaction between the spectator and the artificial creature, which is endowed with a certain amount of autonomy and a certain capacity to invent gestures, creates an unprecedented kind of artistic event: While close to a real-life situation, it remains unpre- dictable and disposed to inspire improv- isation, inventiveness, imagination and wonder.

groups of torques, applied to the body's joints and sent to the behavioral module, which produces a movement. The network was instructed by a series of experiences that acted as a learning phase.

The interactive module manages in- terrelations between the model and the world. Its inputs are fed by external forces sent by the movement sensors. At the same time, the interactive module manages the relations between the model and itself, as some inputs proceed from in- ternal forces, such as the biomechanical constraints of the virtual body.

In this sense, schedules produced by the connectionist module are dynami- cally confronted by the particulars of the interaction that modify them. They can be interrupted at any given moment by another schedule that is better adapted to the present situation.

Teaching We programmed the back-propagation algorithm [32] (see appendix) for su- pervised learning on a layer network. A group of learning pairs is presented to the network, whose connection matrix previously has been randomly initialized. For each input, the network calculates an output that is generally different from the required output. The difference between

both outputs is used to correct the connection weights in order to minimize the error. Through a process of trial and error, the network configures itself and learns all the provided examples. If the series of examples is representative enough of the different situations the synthetic actor is to be confronted with, the network will be able to respond cor- rectly, even in the event of unlearned examples.

In practical terms, we connected the sensors to the network inputs and the network outputs to the actuators influenc- ing the muscular system (Fig. 4).

First, the tightrope walker was taught how to keep its balance on the rope, while the dancer was taught dance movements. Secondly, after these virtual characters had been trained, they were put in front of spectators, dancers or even acrobats, so that a kind of gesture invention composed of interdependence and autonomy could take place.

We also implemented a real-time ver- sion of this method by making the learning process and the interaction phases parallel. From this arose a more elabo- rate interaction in which the spectator could witness, control and even attempt to modify the effect of her actions on the virtual being's behavior. Finally, we are planning to use other kinds of unsuper-

| I Fig. 5. An example of the

13... . . phase plane. (? Michel Bret)

.....

..::

INTEGRATION IN THE MODEL OF PRINCIPLES AND NATURAL LAWS OF MOVEMENT

This section describes the integration of a few principles and natural laws of movement into the virtual body. They influence some of its autonomous behavior patterns, in a manner similar to how the brain defines movement strategies in order to reduce the number of control parameters.

About Neural Networks The virtual actor's network of neurons learns through a series of examples based on samples of natural movements recorded by the sensors. Replacing the one network controlling the legs with several networks that have learned different things raises the question of which network to choose. In a first phase, we chose the network with the smallest Eu- clidian distance between one of its learning inputs and the virtual actor. In a second phase, rather than a single most- adapted network, we chose a combi- nation of responses. In this way, we developed a multi-network environment that allowed combinations-our very first move towards an action selection model. Here, movement organization is based on a synergy directory, each synergy being a possible action. But not only is it necessary to have a library of easy-to- trigger available movements that are all compatible because they share the same reference frames or the same geometri- cal principles; it is also necessary to be capable of making choices among them. The formulation of this problem inau- gurates a series of reflections, and in fur- ther research we want to develop the study of other selection modes as models for action selection.

Thanks to input multiplexing, it was possible to control the degree of autonomy of the virtual figure. In the same way,




we simulated a "goal-directed action," using a method close to that of inverted kinematics, which is used in robotics.

The Role of the Head and the Gaze In The Virtual Tightrope Walker, the head movement control was managed by a behavioral program interacting with the dynamic model in a somewhat arbitrary mode. This resulted in quite unnatural movements, as the head and gaze in particular moved around without any defi- nite purpose.

In order to control these head movements, we decided to subject the figure to a behavior pattern whereby its gaze looks for the spectator. When the tightrope walker spins, its head orientation can change abruptly and find a new position. Biomechanical constraints will not let this position last continuously.

Automatic Correlations In order to control complex movements, the brain diminishes the range of different possible action controls it has to enact on the muscles by creating automatic correlations using certain parameters. For example, the three angles formed by the foot with the leg, the calf with the thigh and the thigh with the torso are in a relationship of linear dependence. If a 3D space is used to visualize the variations of the angles, the points whose coordi- nates are the values of these angles remain close to a "phase plane" in the event of a "natural" movement. In order to verify this tendency, a program analyzes the variations of the angles on a synthetic clone animated by a real actor wearing an exoskeleton that captures the angles of the joints. A graphic interface shows the plane that is closest to the scatter plot formed by a group of measures. It then appears that the corresponding points remain quite close to an ideal plane, whereas this correlation does not neces- sarily hold as far as other parts of the

E Fig. 6. An illustration of the law of the power of one-third. (? Michel Bret)

body are concerned. Even in the case it does verify, it does so not in a continu- ous way, but on segments of movements

(Fig. 5). Another type of automatic setting

managed by the brain concerns phase op- position, in which some secondary movements may be produced by inverted variations. For example, the angle between the arm and the rest of the body and the angle between the forearm and the upper arm are inversely related, and so they are considered to be "in phase op- position." This kinematic constraint allows one to control the movement with one single parameter, by changing only the amplitude between both these angles, which considerably simplifies control. In order to verify this law, we used the previous program to visualize these variations, and the results were convincing on quite a few intervals.

The One-Third Power Law Another example provided by the kinematic observation of natural movements that reveals the brain algorithms used in order to control movements is the one-third power law. This law brings together gesture kinematics and geometry: When a person draws an ellipse on a piece of paper with a simple gesture, there is a relation between the curve and

Fig. 7. Two moving characters who attempt to keep their gazes in line. (? Michel Bret)

the tangential speed of the movements of the hand. What is most striking is that this law not only has an effect on the way the movement is produced, but it also in- fluences the way movement is perceived, as a movement appears artificial when the law is not respected: The laws gov- erning natural movement production also model the laws of movement perception.

This relation is the following [34]: S = constant * R/3 with S = tangential speed and R = curve

radius Or the equivalent: A = constant * R2/3

with A = angular speed. We were able to verify this relation with

the same type of program that was em- ployed in the case of the previous laws

(Fig. 6).

Interaction Reference System Besides the visual reference systems such as verticals, horizontals or the body axis, other reference systems may be used by the brain. In this sense, the choreogra- pher Eshkol [35] has established a dance scripting system whereby it is possible to describe the movements of two dancers through the use of three reference systems: one in relation to the dancer's environment, one for the dancer's relationship with his or her own body, and lastly, one involving the dancer's relationship with the partner. This brings up the question of an interaction reference system between both dancers.

Another example is that of two dogs fighting: The reference systems related to them are mobile, but an interaction reference system could be defined whereby both previous reference systems are harnessed by certain constraints, for example, a constraint according to which each dog's gaze is riveted to the other's. This is why we chose to add a link between the reference system's movements and the virtual camera, not in an arbi-




trary way, but depending on the spectator's moves and on those of the synthetic creature. Furthermore, for the case of two moving figures, we created a relationship to try to preserve constant eye contact between the two. Figure 7 is an example of two moving characters whose gazes we attempt to keep in line.

CONCLUSION

If perceptions cease to be considered as forms of representation, but rather are viewed as simulated actions that are then

projected into the world, then could not these digital creations be equally envisaged no longer as shows (representations), but as "simulated actions that are then

projected onto the world"? Even though interactive art cannot

be reduced to simulation, it is based on simulation, in order to draw the spectator into the interactive interplay. This type of art cannot be reduced to a mere illusion-the illusion of simulated perceptions-although it contributes to elaborating a set of coherent perceptions for the spectator who is invited to play a part in the experience.

Artists generally try to move one step beyond the coherence of these perceptions by shuffling, disturbing, provoking a disorder of the senses (un dereglement des sens) in order to question and explore them, to reach their limits, to feel new emotions, to invent, to create.

The aims of scientists are different from those of artists: The latter are not interested in trying to assess the validity of the provided models, but rather con- sider them as a contribution to what could be called "aesthetic hypotheses." Sharing this adventure with scientists, however, enables artists to create new artistic experiments in hybridizing human and virtual beings as part of their quest for new sensations, new feelings, new ways of looking at the world, all stim- ulated by movement and action.

TECHNICAL APPENDIX The "back-propagation" method is a su- pervised learning technique, in which the "teacher" knows the theoretical answer (i.e. the one the network has to learn):

Take a network of neurons with an input layer, one or several hidden layers and an output layer.

Take xk as the vector standing for the input impulse number k (learning value).

Take ok as the vector standing for the cell's response for the output layer for input xk.

Take tk as the vector of the theoretical response (the one that is to be learned) for input xk.

A learning session is defined by the or- dered pairs (xl,tl) ... (xNk,tk).

Take w.i the weight of the connection of cell number i to hidden cell num-

berj. And finally take OUT=f(IN) the trans-

fer function of a cell. If the calculated response is differ-

ent from the theoretical response, the weights are modified so as to lessen the error made by the cell related to the answer. The calculation that takes the mis- take into account is the same for every layer, but the calculation of the error signal varies according to the layer in question.

In the case of an example k, oi, the calculated output and ti the theoretical output, the quadratic error must be min- imized:

Q= 0.5 * z (ti- )2

Gradient descent works by evolving wj. in the opposite direction from that of the gradient:

A wij =-n * 8Q/ 5w Where n is the learning constant in the

interval between 0.0 and 0.1 Take ei the input of neurone i, ai its out-

put and f its transfer function, it is shown that the correction of the weights of unit number i is:

A wi. = -n * d. * a.with di = -n *

aQ / aa * aai / aei that the error in the output layer is:

5i=f'(ei) * (oi-ai) (1) and that the error on the hidden layer is:

. =f'(ei) * d w (2) If the transfer function is the sigmoid

function: OUT = f(IN) = 1 / (1 + e-k'IN), its de-

rivative is: aOUT / aIN = OUT * (1 - OUT)

and the formulas (1) and (2) shown above become, for the output layer:

8i = ai * (1 - ai) * (oi- ai) (1') and for the hidden layer:

8i =ai* (1-ai) *d dk wi (2') In this recurrent method one starts cal-

culating the error signal for the output layer for formula (1), then going back into the hidden layers one uses the error signals of the cells of layer I to calculate that of each cell of layer i-1 with the formula (2). Hence the name: "back-propagation algorithm."

References

1. F. Sparacino, G. Davenport and A. Pentland, "Media in Performance: Interactive Spaces for Dance, Theater, Circus, and Museum Exhibits," IBM Systems Journal 39, Nos. 3 and 4, 479-510 (November 2000).

2. C.W. Reynolds, "Flocks, Herds and Schools: A Distributed Behavioral Model," Proceedings of SIGGRAPH 87, 14th International Conference on Com- puter Graphics and Interactive Techniques, Anaheim, CA, U.S.A., 27-31 July 1987. Published in Computer Graphics (Annual Conference Series) 21 (1987) pp. 25-34.

3. R.A. Brooks, "Intelligence without Reason," IJCAI- 91 1 (1991) pp. 569-595.

4. P. Maes, Situated Agents Can Have Goals, Designing Autonomous Agents: Theory and Practicefrom Biology to Engineering and Back (Cambridge, MA: MIT Press, 1990).

5. P. Maes, "Modeling Adaptative Autonomous Agent," Artificial Life 1 (1994) pp. 135-162.

6. K. Sims, "Artificial Evolution for Computer Graph- ics," Proceedings of SIGGRAPH 91, 18th International Conference on Computer Graphics and Interactive Tech- niques, Las Vegas, NV, U.S.A., 28July-2 August 1991. Published in Computer Graphics 25, No. 4 (Confer- ence Proceedings), 319-328 (1991).

7. K. Sims, "Evolving Virtual Creatures," Proceedings of SIGGRAPH 94, 21st International Conference on Computer Graphics and Interactive Techniques, Orlando, FL, U.S.A., 24-29 July 1994. Published in Computer Graphics (Annual Conference Series) 1994, pp. 15-22.

8. B.M. Blumberg and T. Galyean, "Multi-Level Direction of Autonomous Creatures for Real-Time Virtual Environments," Proceedings of SIGGRAPH 95, 22nd International Conference on Computer Graphics and Interactive Techniques, Los Angeles, CA, U.S.A., 6-11 August 1995. Published in Computer Graphics (Annual Conference Series) 1995, 47-54.

9. B. Blumberg et al., "Integrated Learning for In- teractixe Synthetic Characters," in Proceedings of the 29th Annual Conference on Computer Graphics and In- teractive Techniques, San Antonio, TX, U.S.A., 2002 (New York: ACM Press, 2002) pp. 417-426.

10. R. Grzeszczuk, D. Terzopotulos and G. Hinton, "NeuroAnimator: Fast Neural Network Emulation and Control of Physics-Based Models," Proceed- ings of SIGGRAPH 98, 25th International Confer- ence on Computer Graphics and Interactive Techniques, Orlando, FL, U.S.A., July 1998. Published in Com- puter Graphics (Annual Conference Series) 1999, pp. 29-38.

11. J. Funge, X. Tu and D. Terzopoulos, "Cognitive Modeling: Knowledge, Reasoning and Planning for Intelligent Characters," Proceedings of SIGGRAPH 99, 26th International Conference on Computer Graphics and Interactive Techniques, Los Angeles, CA, U.S.A., Au- gust 1999. Published in Computer Graphics (Anntial Conference Series) 1999, pp. 29-38.

12. A. Guillot andJ.A. Meyer, "La contribution de l'approche animat aux sciences cognitives," Cahiers Romans de Sciences Cognitives 1, No. 1, 1-26 (2003).

13. A. Drogoul,J.A. Meyer and Hermes, Intelligence artificielle situee, cerveau, corps, environnement (London: Hermes Science, 1999).

14.J.H. Holland, Adaptation in Natural and Artificial Systems (Ann Arbor, MI: University of Michigan Press, 1975).

15. D.E. Goldberg, Genetic Algorithms in Search, Opti- mization, and Machine Learning (Boston, MA: Addi- son Wesley, 1989).

16.J. Koza, Genetic Programming: On the Programming of Computers by the Means of Natural Selection (Cam- bridge, MA: MIT Press, 1992).

17. T. Calvert et al., "The Evolution of an Interface for Choreographers," in Proceedings of InterCHI 93, Conference on Human Factors in Computing Systems (April 1993) (Amsterdam: IOS Press, 1993) pp. 115- 122.

18. T.W. Calvert et al., "Composition of Multiple Fig- ure Sequences for Dance and Animation," The Vi- sual Computer 7, Nos. 2-3, 114-121 (1991).




19. S. Kshirsagar and N. Magnenat-Thalmann, "A Multilayer Personality Model," Proceedings of the 2nd International Symposium on Smart Graphics (June 2002) (New York: ACM Press, 2002) pp. 107-115.

20. S. Carion, G. Sannier and N. Magnenat-Thal- mann, "Virtual Humans in CyberDance," in Proceed- ings of Computer Graphics International, 22-26 June 1998 (Washington, D.C.: IEEE Computer Society, 1998) p. 142.

21. See Sparacino et al. [1].

22. First performance: Granit, Scene Nationale de Belfort, 1 April 2001, France.

23. M. Van de Panne and E. Fiume, "Sensor-Actuator Networks," Proceedings of SIGGRAPH 93, 20th Inter- national Conference on Computer Graphics and Inter- active Techniques, Anaheim, CA, U.S.A., 1-6 August 1993. Published in Computer Graphics (Annual Con- ference Series) 1993, pp. 335-342.

24. See Sims [6].

25. E. Couchot, M.H. Tramus and M. Bret, "La se- conde interactivite," in Arte e vida no seculo XXI, Or- ganizadora Diana Domingues (Brasil: Edition UNESP, 2003) pp. 27-38.

26. M.H. Tramus, "La question de la multisensoria- lite dans les arts numeriques interactifs," paper presented at ISEA 2000, available on-line at Actes ISEA 2000 <http://www.isea2000.com>.

27. A. Berthoz, La decision (Paris: Editions Odile Jacob, 2003).

28. F.J. Varela, Autonomie et Connaissance. Essai sur le 35. N. Eshkol, Angles andAngels (Tel-Avix, Islael: The vivant, edit (Paris: Le Seuil, 1989). Movement Notation Society, 1990).

29. A. Berthoz, Le sens du mouvement (Paris: Editions OdileJacob, 1997).

30. The Virtual Tightrope Walker (La funambule virtuelle) was exhibited at the following: Virtual World 2000, Paris, July 2000; Salon d'automne, Paris, November 2000; Les chemins du numirique, Reims, France, April 2001; Midia-Village, Paris, May 2001; Le cube s'ouvre a Issy les Moulineaux, exposition d'art numerique, Issy les Moulineaux, France, September 2001. Dance with Me (Danse avec moi) was exhibited at Arts-outsiders, Paris, September 2001 and le Salon d'automne, Paris, November 2001.

31. M. Bret, "Virtual Living Beings," inJean-Claude Heudin, ed., Lecture Notes in Artificial Intelligence, Vir- tual Worlds (New York and Heidelberg: Springer 2000) pp. 119-134.

32. M. Bret, "Experimentation dans les arts du con- nexionnisme," paper presented at ISEA 2000, available on-line at Actes ISEA 2000 <http://www. isea2000.com>.

33. T. Kohonen, Self-Organization and Associative Mem- ory (New York and Heidelberg: Springer, 1984).

34. P. Viviani and T. Flash, "Minimum-Jerk, Two- Thirds Power Law, and Isochrony: Converging Approaches to Movement Planning," Journal of Ex- perimental Psychology. Human Perception and Perfor- mance 21 (1995) pp. 32-53.

Manuscript received 2 August 2003.

Michel Bret is emeritus professor in the Arts et

Technologies de l'Image department of the Paris 8 University. He has written several 3D computer animation software programs and has realized numerous synthetic movies and artistic interactive installations since 1976.

Marie-Helene Tramus is a professor in the Arts et Technologies de l'Image department of the Paris 8 University. Since 1980 she has realized video art installations, synthetic movies and artistic interactive installations.

Alain Berthoz is a professor at the College de France, where he directs the Laboratory of Physiology of Perception and Action. He has written a great number of scientific papers in international journals, particularly on the

topics of body balance control and the perception of movment. He is the author of Le sens du mouvement (1997); The Brain's Sense of Movement (2000); and La decision (2003).




interacting with an intelligent dancing figure: artistic experiments at the crossroads between art...

Documents