Video Mediated CommunicationProducing a sense of presence between individuals

in a shared virtual reality

http://gt.kth.se

Claus J. KnudsenAmbjörn NaeveLeif Handberg

Division of Media Technology and Graphic ArtsThe Knowledge Management Research group

Department of Numerical Analysis and Computing ScienceThe Royal Institute of Technology

Stockholm, Sweden

http://kmr.nada.kth.se

Keynote address at the ISEC conference, Banff, Calgary, Canada, June 1, 2002

I-2-I

Alleviating absence by useof the "Telephonoscope".A mirror is envisage as thedisplay screen.

[From Albert Robida "Le Vingtième Siècle, 1880s]

iSpace project meeting between Royal Institute of Technology andStanford University

TelepresenceAs described by teleoperators, presence is thesensation of being at the remote worksite ratherthan at the operator`s control station.

Marvin Minsky (1980) in reference to teleoperation systems Teleoperators and Telerobotics, (Held & Durlach, 1992)Simulated environments, (Rheingold, 1991; Sheridan, 1992).

Definition of Presence at a Distance

Presenceat a distanceis defined asthe subjective experienceof being in one place or environment,even when one is physically situated in another.

Presence journal 2000

Definition of Presence in Virtual Environments

Presence is defined as the subjective experienceof being in one environment (there) when one isphysically located in another environment (here).

Measuring Presence in Virtual Environments,(Witmer Bob G. and Singer Michael J., 2000)

As applied to my own research,presence refers to the experiencingof both mediated bodies and spaces.

Two models of mediated communication

• traditional communication model

• telepresence view

Defleur & Ball-Rokeach, 1989; Schramm, 1974; Shannon & Weaver, 1962

Steuer (1993) after Krueger (1991)

BA

BA

Sharedvirtualreality

The main research question

How do we producea sense of presence and realityat a distancebetween individualsphysically separated in space?

Factors creating a sense of presence and reality

• sensory environment

• individual preconditions

• content characteristics

• vividness

• interactivity and control• company of others

• depth & breadth

• imagination• emotional state• associative context• suspension of disbelief

• plot and story• narration and dramaturgy• presentation and execution

[Enlund N.]

Experiments and user studies

• experiment with new methodsof creating human-to-human Interaction,including for musical and artistic performance.

• strong international collaboration

• produce “pre-competitive” results in theform of prototypes, demonstrators and userstudies .

The blue-screenbackgroundwas mappingout the personby use ofcolour keying.

Physicalbackground

Virtualspace withtwo non-physicalprojectedwalls

Physicalbackground

Mix ofaudio/videosources

Virtual marriage

As seen fromKTH

As seen fromGjøvik

ProjectorScreen

Camera ProjectorScreen

Camera

Video processing

Computer processing

Publishing

Fibre optics

Media Technology Provisorium KTH Stockholm Academic Forum

Internet

Synchronous virtual communicative spacestransparent technology for producing

a sense of presence at a distance

The audience ”point of view” at Academic Forum ”camera space”.

Synchronous virtual communicative spacesTransparent Technology for Producing

a Sense of Presence and Reality at a Distance

A studentwatching a videofrom the teacherscomputer at adistance.

Two studentslearning adance from ateacher at adistance.

Synchronous virtual communicative spacestransparent technology for producing

a sense of presence at a distance

Synchronous virtual communicative spacestransparent technology for producing

a sense of presence at a distance

Two studentsacting at a distance

Two musicansplaying with a singerat a distance

Telepresence Production Course

New Learning Modes in the Production ofPresence Distance Technique for Education

Interactive storytelling and presence production

New Learning Modes in the Production ofPresence Distance Technique for Education

Interactive storytelling and presence production

New Learning Modes in the Production ofPresence Distance Technique for EducationInteractive storytelling and presence production

- Games and competitions- Instruction- Role play and ceremonies- Multipoint cooperation- Interactive information package

Topics chosen for the exam projects:

New Learning Modes in the Production ofPresence Distance Technique for Education

• Storytelling• Interactivity• Technical level• Technology using skills

Evaluation criteria

The Cave ExperiencePeople and Technology in an ExperimentalPerformance Space connected to the Internet

Connecting Sweden on ISDN to Hungary on the Internet

Distance Learning Applicationsacross Multiple Platforms and NetworksA Media Laboratory Test Model

VideoFlyer

ISDN 384kY/C input 1Y/C input 2

Video composite outAudio line in

Audio outSCSI

RS232Mic input

Local microphone

Remote RS 232Y/C out

composite out

Monitor 2(Virtuel space-ISDN remote)

Video composite in

Monitor 3(ISDN remote)

Audio inputVideo composite in

Macintosh Video switchMacintoshmonitor 1

SCSIVideo composite out

Audio outVideo composite in

Y/C outAudio in left

Audio in right

Video testgenerator

Video Cassette Recorder

Video composite out 1Video composite out 2Audio inputAudio output 1Audio output 2Video composite in

H.320 3 x 2 B channels ISDN

Video input 1Video input 2Video input 3Video output

LAN/WAN/INTERNETVIDEO (MJPEG)

Distance Learning Applications acrossMultiple Platforms and NetworksA Media Laboratory Test Model

Continuous Presence – a virtual Meetingpoint

Interaction between musicans and audiencein a learning process on the Internet

The first performance for motivation

The concert based on the creativematerial library on the Internet

Unified Language ModelingVideo mediated communication

The KMR-group at CID

People:

• Ambjörn Naeve (senior researcher, head of the group)

• Mikael Nilsson (grad. student, mathematics didactics)

• Matthias Palmér (grad. student, computer science)

• Fredrik Paulsson (grad. student, MDI)

• Claus Knudsen (grad. student, media technology)

Knowledge Management Research areas

• Knowledge Manifolds

• Conceptual Modeling

• Conceptual Browsing

• New paradigms and tools for mathematics education

• E-learning frameworks / Semantic web technologies

• Component-based learning technologies

• Semantic Interoperability

• Presence production

A Knowledge Manifold

• is a structured information architecture that supports a number of different strategies for information hiding (encapsulation).

• can be regarded as a Knowledge Patchwork, with a number of linked Knowledge Patches, each with its own Knowledge Gardener.

• gives the users the opportunity to ask questions and search for certified human Knowledge Sources.

• can be used to design learner-centric, knowledge-pulling interactive learning environments that support Question Based Learning.

A Knowledge Manifold (cont.)

• has access to distributed archives of resource components.

• allows teachers to compose components and construct customized learning environments.

• makes use of conceptual modeling to support separation of content from context.

• contains a concept browser (Conzilla) that supports these principles and activites.

The different Knowledge Roles of a KM

• The knowledge cartographer

• The knowledge composer

• The knowledge librarian

• The knowledge coach

• The knowledge preacher

• The knowledge plumber

• The knowledge mentor

• constructs context-maps.

• fills the context-maps with content.

• composes content components into learning modules.

• cultivated questions.

• provides live answers.

• connects questions with appropriate preachers.

• provides motivation and supports self-reflection.

Design principles for Concept Browsers

• separate context (= relationships) from content.

• describe each context in terms of a context-map.

• assign an appropriate set of components as the content of a concept or a conceptual relationship.

• filter the content components through different aspects.

• label all resources with a standardized data description (metadata) scheme (LOM-IEEE).

• transform a content component which is a context-map into a context by contextualizing it.

Mathematical Knowledge Manifold work at KMR

• Virtual Mathematics Exploratorium (Conzilla)

• New ways to study geometrical constructions

• Interacting with mathematical formulas, using

• Shared 3D interactive learning environments

• Interactive geometry with PDB.

• Graphing Calculator (Ron Avitzur).• LiveGraphics3D (Martin Kraus).

• framework for archiving and accessing components

• CyberMath.

created with these (and other) techniques.

Virtual Mathematics Exploratorium 1

Virtual Mathematics Exploratorium 2

CyberMath: A Shared Virtual Environmentfor the Interactive Exploration of Mathematics

• teaching of both elementary, intermediate and advanced mathematics and geometry.

Goals: The CyberMath system should allow:

Means:• Making use of advanced VR technology (e.g. DIVE).

• global sharing of resources.

• the teacher to teach in a direct manner.

• students to work together in groups.

• teachers to present material that is hard to visualize using standard teaching tools.

CyberMath: The cylindrical exhibition hall

Presence Productionin a Distributed Shared Virtual Environmentfor Exploring Mathematics

CyberMath: The Teacher in Stockholm

STUDENTS / DIS, UPPSALA

ALL / CYBERMATH, VR

TEACHER / CID, STOCKHOLM

PERSON

AVATAR

CAMERA

COMPUTER SCREEN / VR REPRESENTATION

MONITOR / PRESENCE PRODUCTION

CyberMath: Experimental Lecture Connectivity

CyberMath: Students in Uppsala

CyberMath: The Virtual Museum Mode

CyberMath: Changing the Learning Mode

CyberMath: From Control to Chaos

Car

Vehicle

is a:Car

kind of

is a kind of

a

Unified Language Modeling

:Wheel Wheel

abstraction of

part of has

is a

a

is a part of ahas a

a kind of

Basic principles

Unified Language ModelingThe CyberMath lecture

Relevant web sites

http://kmr.nada.kth.se

http://www.gt.kth.se

http://www.r1.kth.se