FilmEd

Collaborative Video Indexing, Annotation and Discussion over Broadband Networks

Jane Hunter, <jane@dstc.edu.au>

FilmEd Project Goals

• Education - Film/Media/Communications• Film/Media/Broadcasting Industry• Audiovisual Archives, Libraries, Museums• eScience – Biomedical, Tele-Microscopy• Remote Mentoring, eHumanities

DescriptionIndexing

AnnotationDiscussion

RetrievalSearch

Browsing

Collaborative

Synchronous

DigitalFilm/Video

Editobjective

subjective

GrangeNet POPBrisbane

GrangeNet POPSydney

GrangeNet POPCanberra

GrangeNet POPMelbourne

QUTInteractive Design

CRCDSTC / UQ

Media Studies

DSTC / UTSNew Media

AC3VisLab

ATP

ANUAPAC

AARNet

CSIROBoM

VPACI-cubed

RMIT

Uni Melb

ScreenSound

AARNetTrans-Pacific link

Federation SquareACMI

10 Gbps5 Gbps

GrangeNet

http://www.grangenet.net

Access Grid Nodes

• Videoconferencing over broadband• Large-scale, multiple groups• Interactive collaborative meetings• High quality video/audio• Low latency• Multiple cameras, mikes, projectors, echo

cancellation• Collaborative support software and

middleware• http://www.accessgrid.org/

Access Grid Session

Other Video Annotation Tools

• IBM’s Video Annotation Tool• Ricoh MovieTool• ZGDV’s VIDETO• COALA’s LogCreator• INA/INRIA’s System – ISWC 2003• Microsoft’s Research Annotation System (MRAS), DTVI

(Distributed Tutored Video Instruction)

- all non-collaborative, asynchronous indexing and annotation systems

- MRAS, DTVI only support low frame rates

Architecture

Students@ UTS

Film/Media Analyst@ Australian Centre

for Moving Image

Curator@ Screensound Australia

Students@ University of QLD

ApplicationServer

StreamingServer

Synchronous & CollaborativeIndexing, Annotation and Discussion

StreamingServer

AG

AG

AG

AG

AccessGridNodes

FilmEdApplication

VIC/RAT

Storage and Management

AV Library

AV Library

Screensound Australia

Australian Centre for Moving Image

IndexingDescription

Data

AnnotationDiscussion

Data

IndexingDescription

Data

AnnotationServer

objective

objective

subjective

Project Phases• Collaborative MPEG-2 indexing, search, browse & retrieval• Objective descriptions

Metadata Application Profile – XML/MPEG-7 + Ontologies Incl. automatic shot detection (Mediaware SDK) Map to RDF

• Subjective annotations Annotea Annotation Protocol

• Flexibility – ontologies from different domains• Synchronous Collaborative Discussion, Annotation, Editing

T.120 (e.g. NetMeeting) vs .NET Remoting• Storage (relational DB vs. native XML DB vs. RDF Store)• Rights Clearance/Management – MPEG-21

User-Interface – Components

Description &Indexing

Video Player Annotation &Discussion

HierarchicalSegmentation& Browsing

SegmentDescription

controlled vocabulary

Video Screen

Video Controls

Search

Browsing

Annotationtext & hyperlinks

audiovisual

Search

MPEG-7Ontology

Domain-specificOntology

XML (our application profile)MPEG-7 RDF

Web Server

Annotea Server

RDF Store

ABC

User-Interface

Semi-Automatic Indexing/Description(DC/MPEG-7)

Video Player(MPEG-1, MPEG-2)

Shared Annotation/Discussion (Vannotea)

Application Sharing – T.120

application netmeeting

Co

mp

ute

r A

application-sharingC

om

pu

ter

B

Screen/Window delta

netmeeting

Mouse/Keys

Pro:• Single framework

share any application

Cons:• Poor Quality (5fps) tailored for low bandwidth• Restrictive & inflexible

NetMeetingMaster

NetMeetingclients

Application Sharing – .NET Remoting + Multicasting of MPEG-2 UDP Streams (User Datagram Protocol)

Streaming-SERVER

Application-SERVER

CLIENT

CLIENTCLIENT -MASTER*

Mediator

Coordinator

Mediator

Receiver

MPEG-2

PauseEvent

Transmitter

Receiver

PlayerForm

PlayerForm

Mediator

Receiver

PlayerForm

RTP

RTP

Pause

Pause

PauseEvent

Display

Display

Display

RTP

* holds TokenUDP

UDP

UDP

Application Sharing – Multi-User

Streaming-SERVER

Application-SERVER

CLIENT -MASTER*

CLIENT -MASTER*

CLIENT -MASTER*

Mediator

Coordinator

Mediator

Receiver

MPEG-2

Event

Transmitter

Receiver

PlayerForm

PlayerForm

Mediator

Receiver

PlayerForm

* every client is in controlsimultaneously!

User-Interface – Access Grid Node

Video Conferencing+

Collaborative FilmEd Application

Next 6 months

• Usability studies – ACRA, UK Telemedicine• Annotate/share documents of all media types

– images (JPEG-2000), 3D objects, web pages, text, audio

• Audiovisual annotations – record & index meetings, audit of collaborative sessions

– VIRGIL – Virtual Meeting Archival

• Rights management/access controls on content and annotations

• Real-time capture, replay and annotation• Collaborative Editing

Applications

Community Grid projects – Telemedicine - Online medical

consultations– CyberSTEM – Networked Telemicroscopy– Indigenous Knowledge Management IKM

Project • Smithsonian NMAI, AIHEC, Tribal Colleges,

Indiana University

Other Applications

Dancing Beyond Boundaries –

collaborative live dance/music performances staged in multiple locations

Conducting Master Class

Michael Tilson Thomas, Musical Director for the New World Symphony (Miami) provides longDistance instruction to conducting student Donato Cabrera (New York) (arts.internet2.edu)

Acknowledgements

Ronald Schroeter, Douglas Kosovic, Michael Henderson

References

http://metadata.net/filmed/

http://metadata.net/IKM/