Hear&ThereAn Augmented Reality System of

Linked Audio

Joey Rozier

Karrie Karahalios and Judith Donath

Sociable Media Group

MIT Media Lab

ICAD 2000 Conference - April 3, 2000

Overview

• Project Purpose and Goals

• Brief Scenario

• Basic Augmentation Details

• Hardware

• Authoring and Playback

• Security and Privacy

Purpose and Goals

• Augment Environment with Audio– Augmented Reality (or Mixed Reality)

• Develop Social Augmentations– Oral Histories– Focus on Authoring

• Activation Networks - Linking• Self-Contained and Portable• Limit Use of Visual Cues

Example Scenario

• Augmented Boston Common

• Can hear audio “left behind” (maybe history)

• Guided towards related audio (moo...)

• Author can add his/her own sound

• Others are unaware

Augmentation Details

• The SoundSpot– Basic unit of augmentation– Any type of audio (voice, music,

whatever)– “Placed” at a specific location

(longitude/latitude)– Has a specified size– Other audio properties (braids, etc.)– User walking “into” SoundSpot hears it

Augmentation Details

• Example

Augmentation Details

• Must Determine User’s Location

• GPS– Typical Accuracy: Order of 10m– Accuracy of Ours: sub-meter– Refresh Rate: 10Hz– Drawback: Bulk– Result: Media Lab Courtyard

Hardware

• GPS

• Compass

• PalmPilot

• Laptop

• Headphones

• Microphone

Authoring and Playback

• Focus on user authoring– Most other systems do not– Simple!

• Playback is still very important– New author options lead to new

playback issues

Authoring

• Author creates SoundSpots– In-Field– Using GUI

Playback: Basics

• User dons system

• Very simple start-up

• Moves through environment

• Hears sound tied to location

Playback: Spatialized Sound

• Each sound “appears” to be coming from center of a SoundSpot

• Java 3D API• Compass attached to headphones• Do not deal with environment effects

(enclosed spaces, etc.)• Future work will improve spatialized

sound

Playback: Two Modes

• Explorer– All SoundSpots activated– Can be very cluttered– Ultimate control with listener

• Activation Networks

Playback: Activation Networks

• For Linking and Organizing Sounds• Author creates Link Categories

– e.g. My Family, About Media Lab

• Author associates Link Categories with a SoundSpot

• User can activate Link Category when standing in SoundSpot

• StartSpots

Playback: Activation Networks Example

• StartSpot in Boston’s Public Garden– Link Categories

• SMG: SMG SoundSpots• Weeping Willow: interesting SoundSpots about

weeping willow trees• My Favorites: my favorite SoundSpots

– When entering StartSpot, activate “Weeping Willow”; now those “exist”

– Weeping Willow spots may have their own LinkCategories

Playback: Directing the User

• Palm Interface– Easy, intuitive– But uses visual information!

• Audio Beacons?

Security and Privacy

• Authentication for Authors

• Open System for Users

• Wireless Connection to Server?

Future Work

• Finish Implementation!

• More Portable System

• Temporal Information

• Other forms of audio augmentation

Contact and Contributors

• E-mail: jrozier@mit.edu

• http://smg.media.mit.edu

• Primary Contributors– Judith Donath– Karrie Karahalios– Birendro Roy and Tuan Phan