Frank Eliassen - Lucas Luiz Provensi - Roman Vitenberg

Lille, 10.03.2010

• World Opera• Gilgamesh Production• Scenarios

– Location tracking– Motion recognition– Light Control– Multimedia sensor networks– Information services

• Challenges

Worldwide distributed operas◦ Singers that are

remotely located can interact as if they are in the same stage.

◦ Musicians distributed around the world can play together.

◦ Audience can see the opera from one of the stages or from other remote place.

• First World Opera Production (2012)

• 3 Stages: Tromsø – Stockholm – Copenhagen;

Based on The Epic Of Gilgamesh, an epic poem from old Mesopotamia.

Virtual StagesUruk

Gilgamesh

Cedar Forest

Humbaba

Land of the Dead

Enkidu

The number of multimedia flows/streams coming in and going out will be different for each stage

It can change at runtime as well At certain times, distributed actors may be

interacting with each other while the musicians at all stages keep playing ◦ Actor has a fixed assignment to a physical stage but

may travel between virtual stages ◦ A physical avatar to represent the actor at the

others stages, like a puppet with a display in the face that shows the video stream of the face of the actor

Musicians are very sensitive to audio quality but do not care too much about video quality

Remote musicians are sensitive to maestro acting. They are not very tolerant to delay in the maestro's video streaming

When acting against distributed peers, actors are not tolerant to delay and they need a good video quality and enough resolution to see face details

Actors also need to hear audio, e.g., when in a synchronized dialog◦ Embed a speaker into the avatar so that the actors have

the sensation that the voice is coming from that point

The maestro will be at the city stage, and he will represent the sun god and will be walking around the stage

The conductor location will be tracked to physically reproduce the movement of the maestro at the other stages.

Position of actors in a virtual world.

Wii Mote (Nintendo)

Composed by accelerometer and optical sensor. Can be extended with a gyroscope.

Project Natal (Microsoft)

Depth sensor consisting of an infrared projector combined with a monochrome CMOS sensor

Replicate musicians movements on virtual entities or phenomena.

Replicate actors movements on virtual avatars

Lighting sensors can provide information lighting condition to scene controller software.

Multimedia sensor networks◦ New sources of media content : views of different

angles and audio collected at different points.

Services offered to the audience at the stages and outside.◦ Information about the

ongoing act;◦ Access to alternative

media streams;◦ Shared comments

with other users.

Gilgamesh

Enkidu

Bull

Act 2Gilgamesh and Enkidu fights the Bull of Heaven

Questions?