Tele-Action Objects Tele-Action Objects (TAO)(TAO)

Presented by Presented by

Kingsley AdeoyeKingsley Adeoye

April 3, 2008April 3, 2008

OutlineOutline IntroductionIntroduction MotivationsMotivations TAO BasicsTAO Basics Hypergraph StructureHypergraph Structure Knowledge StructureKnowledge Structure TAO Application and ManagementTAO Application and Management Active Management SystemActive Management System Current ApplicationsCurrent Applications TAO for MICETAO for MICE ConclusionConclusion

Tele-Action Object

What is a Tele-Action ObjectWhat is a Tele-Action Object

According to [1], According to [1], • ““A tele-action object is a multimedia object A tele-action object is a multimedia object

with an associated hypergraph structure and with an associated hypergraph structure and corresponding knowledge base structure”corresponding knowledge base structure”

HypergraphStructureHypergraph

Structure

KnowledgeStructure

KnowledgeStructure

MotivationsMotivations

Many multimedia applications Many multimedia applications require require • Effective presentation of objectsEffective presentation of objects• Efficient transfer of objectsEfficient transfer of objects

TAO approach provides:TAO approach provides:• Effective presentation algorithms Effective presentation algorithms • Simple but efficient transfer protocolsSimple but efficient transfer protocols• Automatic response to eventsAutomatic response to events

TAO BasicsTAO Basics

Hypergraph describes structure Hypergraph describes structure Knowledge describes actions Knowledge describes actions Tele-Action Objects can be:Tele-Action Objects can be:

• Simple piece of information or Simple piece of information or multimediamultimedia

• Complex network of multimediaComplex network of multimedia Object-oriented designObject-oriented design

Hypergraph StructureHypergraph Structure

Node typesNode types• Simple or compositeSimple or composite

Link typesLink types• Attachment Link: hierarchyAttachment Link: hierarchy• Annotation Link: commentaryAnnotation Link: commentary• Location Link: spatial distanceLocation Link: spatial distance• Synchronization: temporal distanceSynchronization: temporal distance• Reference Link: navigationReference Link: navigation

Hypergraph ExampleHypergraph Example

Hypergraph StructureHypergraph Structure Media TypesMedia Types

• Text is coded alphanumeric data.Text is coded alphanumeric data.• Graphics is formatted picture data.Graphics is formatted picture data.• Image is pixel formatted picture data.Image is pixel formatted picture data.• Moving-graphics also called animation, is the formatted Moving-graphics also called animation, is the formatted

data of a graphics sequence.data of a graphics sequence.• Moving-image is a sequence of image frames.Moving-image is a sequence of image frames.• Audio is formatted sound data.Audio is formatted sound data.• Video is a combination of synchronized moving-image Video is a combination of synchronized moving-image

and audio.and audio.• Form restricts user input, possibly with additional Form restricts user input, possibly with additional

formula to generate the content automatically.formula to generate the content automatically.• Live-demo is a program that can be run to provide an Live-demo is a program that can be run to provide an

interactive demo.interactive demo.• Composition is the media type of a composite node in Composition is the media type of a composite node in

TAO. TAO.

Hypergraph StructureHypergraph Structure

Bundled NodesBundled Nodes• Simplifies presentation and Simplifies presentation and

communication of multimedia objectscommunication of multimedia objects• Restricts partial access to bundled Restricts partial access to bundled

members maintaining integritymembers maintaining integrity• Any bundled node must be fully Any bundled node must be fully

referenced referenced For example, referencing to a bundled node For example, referencing to a bundled node

containing text and audio will reference both containing text and audio will reference both sub-nodessub-nodes

Hypergraph StructureHypergraph Structure

Hypergraph decoding algorithmsHypergraph decoding algorithms• Must consider network trafficMust consider network traffic• Hardware capabilitiesHardware capabilities• Best traversing sequence algorithmBest traversing sequence algorithm• Priority of multimedia application/filePriority of multimedia application/file• Link typesLink types• Node resources Node resources

MemoryMemory Execution timeExecution time

Knowledge StructureKnowledge Structure

Four levels of knowledgeFour levels of knowledge• SystemSystem

General actions for required by all TAOs; pre-fetch or General actions for required by all TAOs; pre-fetch or decoding privilegesdecoding privileges

• EnvironmentEnvironment Actions relating to set of circumstances for a group of Actions relating to set of circumstances for a group of

TAOsTAOs

• TemplateTemplate Provides predefined hypergraph/knowledge format for Provides predefined hypergraph/knowledge format for

frequently used TAOsfrequently used TAOs

• PrivatePrivate Important action information for a single TAO from userImportant action information for a single TAO from user

Knowledge ExampleKnowledge Example

Knowledge StructureKnowledge Structure

In Object Oriented DesignIn Object Oriented Design• Each level knowledge is implemented Each level knowledge is implemented

using inheritance and polymorphismusing inheritance and polymorphism• For example, system knowledge is the For example, system knowledge is the

main class therefore every class main class therefore every class inheriting this class will also have the inheriting this class will also have the same knowledge structuresame knowledge structure

• Also, single TAOs can return different Also, single TAOs can return different private knowledge when called using private knowledge when called using polymorphismpolymorphism

Knowledge StructureKnowledge Structure

Knowledge structures transforms Knowledge structures transforms simple static hypergraph objects into simple static hypergraph objects into dynamic objectsdynamic objects

Actions are pre-emptive and user-Actions are pre-emptive and user-configurableconfigurable

TAO Application and ManagementTAO Application and Management

TAO Application and ManagementTAO Application and Management

TAO Application and ManagementTAO Application and Management

Some actions may be associated with Some actions may be associated with the presentationthe presentation

TAO SpecificTAO Specific• Delete TAO after some timeDelete TAO after some time• Request responses from usersRequest responses from users

Environment LevelEnvironment Level• Require permission for some sub-TAOsRequire permission for some sub-TAOs

System LevelSystem Level• Enable pre-fetch of multimediaEnable pre-fetch of multimedia

Active Management SystemActive Management System Translates, executes and maintains tele-action objectsTranslates, executes and maintains tele-action objects Application domain allows users to develop and edit Application domain allows users to develop and edit

TAOsTAOs System domain translates and executes these TAOsSystem domain translates and executes these TAOs

Important AMS ComponentsImportant AMS Components

Local Knowledge BaseLocal Knowledge Base• Collection of ICsCollection of ICs

Transition between statesTransition between states Output messages to other ICsOutput messages to other ICs Perform some specific action sequencePerform some specific action sequence

Application Handler & Knowledge Application Handler & Knowledge GeneratorGenerator• Interface for TAO application toolInterface for TAO application tool• Formats Hypergraph and Knowledge dataFormats Hypergraph and Knowledge data

Important AMS ComponentsImportant AMS Components

InterpretorInterpretor• Receives formatted Knowledge dataReceives formatted Knowledge data• Translates them into corresponding ICsTranslates them into corresponding ICs

Event FilterEvent Filter• Retrieves important events from Retrieves important events from

environmentenvironment Smart EngineSmart Engine

• Maintains Active Index with current ICs Maintains Active Index with current ICs • Event driven system responding to internal Event driven system responding to internal

IC messages or external event filter IC messages or external event filter messagesmessages

Current ApplicationsCurrent Applications

Smart Multimedia MailSmart Multimedia Mail• Interface for email systemInterface for email system• Create actions for eventsCreate actions for events

Alarms, Statistics or even define template mailAlarms, Statistics or even define template mail

CASE Tools for Multimedia CASE Tools for Multimedia Software EngineeringSoftware Engineering

In traditional software engineering, In traditional software engineering, involves formal well-defined stepsinvolves formal well-defined steps• Requirements, Specifications, etc.Requirements, Specifications, etc.

According to [3], multimedia applications According to [3], multimedia applications cannot follow this traditional paradigm cannot follow this traditional paradigm • Minimal requirements are initially availableMinimal requirements are initially available• Applications are constantly evolvingApplications are constantly evolving

Multimedia IC Developer’s Environment Multimedia IC Developer’s Environment (MICE) offers promising solutions to this (MICE) offers promising solutions to this problemproblem

MICE Application DevelopmentMICE Application Development

Distributed IC ManagerDistributed IC Manager

ICBuilder

ICBuilder

TAOMLInterpreter

TAOMLInterpreter

TAOML HTMLTemplate

SR GrammarSR Grammar

Prototyping tool

Visual SpecificationSpecification

Builder

Working System

Specification tool

ConclusionConclusion

TAO-based multimedia systems offer TAO-based multimedia systems offer many advantages in efficiency and many advantages in efficiency and operationoperation

AMS allows user-friendly access to AMS allows user-friendly access to create, maintain and edit TAOscreate, maintain and edit TAOs

Opportunities for optimizing Opportunities for optimizing presentation and communication presentation and communication algorithms will continue to make algorithms will continue to make large network TAO-based systems large network TAO-based systems realizablerealizable

ReferencesReferences[1] [1] Hui-Jung Chang*, Tai-Yuan Hou+, Arding Hsu+, Shi-Kuo Chang, “Management and Applications of Hui-Jung Chang*, Tai-Yuan Hou+, Arding Hsu+, Shi-Kuo Chang, “Management and Applications of

Tele-Action Objects”, Department of Computer Science, University of Pittsburgh, Siemens Corporate Tele-Action Objects”, Department of Computer Science, University of Pittsburgh, Siemens Corporate Research, Inc.Research, Inc.

[2][2] H. J. Chang, T. Y. Hou, A. Hsu and S. K. Chang, "Tele-Action Objects for an Active Multimedia H. J. Chang, T. Y. Hou, A. Hsu and S. K. Chang, "Tele-Action Objects for an Active Multimedia System", Proc. of the Second Int'l IEEE Conference on Multimedia Computing and Systems, May System", Proc. of the Second Int'l IEEE Conference on Multimedia Computing and Systems, May 1995, Washington D.C. 1995, Washington D.C.

[3][3] Timothy Arndt, “The evolving role of software engineering in the production of multimedia Timothy Arndt, “The evolving role of software engineering in the production of multimedia applications”, Department of Computer and Information Scienceapplications”, Department of Computer and Information Science

QuestionsQuestions