smil: synchronized multimedia integration...

37
SMIL 1.0 — Synchronized Multimedia Integration Language Page 1 of 73 SMIL: Synchronized Multimedia Integration Language Lynda Hardman Multimedia and Human-Computer Interaction Group CWI, Amsterdam, The Netherlands Jacco van Ossenbruggen, Lloyd Rutledge: CWI Dick Bulterman, Jack Jansen, Sjoerd Mullender: Oratrix W3C SYMM working group Presentation Outline: Introduction — Problems with Multimedia on the WWW — Goals for a multimedia format Example Presentation SMIL basics — media elements, spatial layout, temporal layout, linking, alternate content, semantic annotations Relationship with other W3C recommendations such as XML SMIL Support SMIL 1.0 — Synchronized Multimedia Integration Language Page 2 of 73 The Problem: Multimedia Lots of Bits Images, audio and video are beyond Internet design specs Results in space/time constraints at: the server the network(s) the client Not All Bits are Equally Important Time between samples often more important than bits in sample, for example lip synchronization (but not always...) Content may be Distributed Across Network Need to synchronize presentation

Upload: others

Post on 14-Mar-2020

29 views

Category:

Documents


0 download

TRANSCRIPT

SMIL 1.0 — Synchronized Multimedia Integration Language Page 1 of 73

SMIL:Synchronized Multimedia Integration Language

Lynda Hardman

Multimedia and Human-Computer Interaction GroupCWI, Amsterdam, The Netherlands

Jacco van Ossenbruggen, Lloyd Rutledge: CWIDick Bulterman, Jack Jansen, Sjoerd Mullender: Oratrix

W3C SYMM working group

Presentation Outline:• Introduction

— Problems with Multimedia on the WWW— Goals for a multimedia format

• Example Presentation• SMIL basics

— media elements, spatial layout, temporal layout, linking, alternate content, semantic annotations• Relationship with other W3C recommendations such as XML• SMIL Support

SMIL 1.0 — Synchronized Multimedia Integration Language Page 2 of 73

The Problem: Multimedia

Lots of Bits• Images, audio and video are beyond Internet design specs

• Results in space/time constraints at:— the server

— the network(s)

— the client

Not All Bits are Equally Important• Time between samples often more important than bits in sample,

for example lip synchronization (but not always...)

Content may be Distributed Across Network• Need to synchronize presentation

SMIL 1.0 — Synchronized Multimedia Integration Language Page 3 of 73

Declarative: the key to Web integration

Choices:• Programmed: encode events that display and remove objects

— Lingo (Director) (using Shockwave)

— Java (Dynamic HTML)

• Declarative: specify a set of objects and their synchronization— VRML

— CWI’s CMIF

— INRIA’s Madeus

• Declarative for basic aspects, extra complexity using programmed.

SMIL 1.0 — Synchronized Multimedia Integration Language Page 4 of 73

Objectives for Web-based Multimedia

Add synchronization capabilities to the Web

Provide a declarative language for the Web

• interoperability

Format should be text based

• so can be edited by hand• preferably XML based

SMIL 1.0 — Synchronized Multimedia Integration Language Page 5 of 73

SMIL

Synchronized Multimedia Integration Language

SMIL is about timing...

combining Web resources...

not just graphics...

in an XML syntax

SMIL 1.0 — Synchronized Multimedia Integration Language Page 6 of 73

SMIL

Synchronized Multimedia Integration Language• pronounced smile• SMIL 1.0 became W3C recommendation on 15th June 1998

• SMIL Boston is public draft for next version, last released February 25 2000

SYMM group includes• CWI, Oratrix - GRiNS authoring environment and player• RealNetworks

- RealPlayer 7 (G2), time-focussed media type, 3rd party creation tools• Microsoft - Internet Explorer 5.5 supports HTML+SMIL

Focus• SMIL 1.0 - Multimedia “basics”, adaptive• SMIL Boston -“prettier” (animation, transitions),

integration with other W3C languages

SMIL 1.0 — Synchronized Multimedia Integration Language Page 7 of 73

SMIL as XML Markup

Integration language• Media elements referred to, not included

SMIL is XML• Defined with XML DTD• Can be hand-authored• Declarative language

- attribute/value pairs• Integrable with XML environments

Relationship with Other W3C Recommendations• Again, SMIL is XML• Basic layout isomorphic and replacable with CSS• Shares constructs with (X)HTML• SMIL Boston “Family” languages enable new SMIL-based XML formats

SMIL 1.0 — Synchronized Multimedia Integration Language Page 8 of 73

XML (eXtensible Markup Language)

xxxxx

xxxx=xxxxxxxx=

xxxxx

xxxxx

xxxx=xxxxxxxx=xxxx

xxxx=xxxxxxxx=

file

Foundation Syntax for all Documents

text content

smil

head

layout

region

body

par seq ref

Document Type Definitions (DTDs)

mediafile

regionsrc

dur=##s

SMIL 1.0 — Synchronized Multimedia Integration Language Page 9 of 73

An XML (SMIL) Document

<?xml version="1.0" encoding="ISO-8859-1"?><!DOCTYPE smil PUBLIC "-//W3C//DTD SMIL 1.0//EN" "http://www.w3.org/AudioVideo/Group/SMIL10.dtd"><smil> <head> <meta name="sync" content="soft"/> <layout> <root-layout id="SMIL-" width="492" height="810"/> <region id="address-region" width="50%" height="8%"/> <region id="image-region" top="8%" height="91%"/> </layout> </head> <body> <seq> <par> <text type="text/plain" region="address-region" src="Herengracht284.txt" dur="2s"/> <img region="image-region" src="http://www.amsterdam.nl/bmz/adam/pics/huizen/h284/h284.jpg"/> </par> <par> <text type="text/plain" region="address-region" src="Herengracht539.txt"/> <img region="image-region"

src="http://www.amsterdam.nl/bmz/adam/pics/h539.jpg" dur="2s"/> </par> </seq> </body></smil>

SMIL 1.0 — Synchronized Multimedia Integration Language Page 10 of 73

A Sample Presentation

The Network News

On demand on your screen

Formatted text,video and audio

SMIL 1.0 — Synchronized Multimedia Integration Language Page 11 of 73

Local anchor setup

Top Story:

Growth of the World-Wide Web

Amsterdam

Graph appears during spoken commentary

SMIL 1.0 — Synchronized Multimedia Integration Language Page 12 of 73

Remote Correspondent

Top Story:

Growth of the World-Wide Web

Amsterdam

Los Angeles

First video finishes, second video plays

SMIL 1.0 — Synchronized Multimedia Integration Language Page 13 of 73

Following a Link

Top Story:

Growth of the World-Wide Web

Amsterdam

Los AngelesAt any point during the videothe viewer can request extrainformation.

SMIL 1.0 — Synchronized Multimedia Integration Language Page 14 of 73

So what do we need to specify?

(part of) media item

Top Story:

Growth of the World-Wide Web

Amsterdam

Los AngelesI Content

II Spatial layout III Temporal layout

anchor

remote

anchor

remote

anchor

growth

A

remoteA

storyA

IV Links

source and destination

V Alternative contentbandwidthtaskuser characteristics

VI Semanticannotations

SMIL 1.0 — Synchronized Multimedia Integration Language Page 15 of 73

I Content

audio

audio

The top story tonight looks at the development of the World Wide Web. The growth of the Web has been exponential for the last few years and it is still unclear....

SMIL 1.0 — Synchronized Multimedia Integration Language Page 16 of 73

Spatial and Temporal Dimensions of Media Items

y

x

t

y

x

t

y

x

t

text:2D objects in linear sequence, aspect ratio not important

graphics:2D layout, aspect ratio probably important

y

x

t

?

video:2D layout plus time, aspect ratio probably important

audio:time only

?:0 or 2D layout, with or without time

SMIL 1.0 — Synchronized Multimedia Integration Language Page 17 of 73

Instance of Media Item

I Media item, or part

II Extent, position andZ-order

III Duration

IV Alternate Content

V Link end-points

VI Associated semantics

We will return to all these points at the end.

SMIL 1.0 — Synchronized Multimedia Integration Language Page 18 of 73

SMIL — Media Object Elements

SMIL 4.2.3. ref, text, textstream, img, audio, video and animation<ref src=”anything.???” ... />

<text src=”caption.html” ... />

<textstream src="stockticker.rtx" ... />

<img src=”graph.jpg” ... />

<audio src="http://www.w3c.org/SYMM/joe-audio.wav" ... />

<video src="rtsp://www.cwi.nl/SMIL/video.rm" ... />

<animation src=”cute.anim” ... />

The src attribute is a URI, locating the data.

The player does not derive the exact type of the media object from the name of the media object element. It uses, e.g., the "type" attribute, or the type information communicated by the server or the operating system.Names are for readability.

imagesvideo textaudio

SMIL 1.0 — Synchronized Multimedia Integration Language Page 19 of 73

Specification of part of media item

y

x

t

y

x

t

y

x

t

y

x

t

text — string image — area

video — (moving) area audio — phrase

SMIL 1.0 — Synchronized Multimedia Integration Language Page 20 of 73

SMIL — clips in time

Time and space treated independently.• Spatial clipping done via region mechanism, discussed later

• Time restricted to a single extent— a contiguous section of a continuous media object can be specified

SMIL 4.2.3. clip-begin and clip-end attributes<video src="the.news/mpeg/zoomin.mpv"

clip-begin="smpte=00:01:19:20" clip-end="smpte=00:01:38:40" ... />

— syntax of values is ...

t2t1duration

SMIL 1.0 — Synchronized Multimedia Integration Language Page 21 of 73

SMIL — clip begin/end syntax

Clip-time-value ::= Metric "=" ( Clock-val | Smpte-val )

Metric ::= Smpte-type | "npt"

Smpte-type ::= "smpte" | "smpte-30-drop" | "smpte-25"

Smpte-val ::= Hours ":" Minutes ":" Seconds

[":" Frames [ "." Subframes ]]

Hours ::= 2DIGIT

Minutes ::= 2DIGIT

Seconds ::= 2DIGIT

Frames ::= 2DIGIT

Subframes ::= 2DIGIT

Metric has three SMPTE options: smpte, smpte-30-drop, smpte-25,otherwise it is NPT (Normal Play Time), which uses a clock value.

Examples of clip-begin value:SMPTE timestamp: smpte=10:12:33:20NPT: npt=12:05:35.3

— syntax of a clock value is ...

SMIL 1.0 — Synchronized Multimedia Integration Language Page 22 of 73

SMIL 4.2.1. — Clock value syntax

Clock-val ::= Full-clock-val | Partial-clock-val | Timecount-val

Full-clock-val ::= Hours ":" Minutes ":" Seconds ("." Fraction)?

Partial-clock-val ::= Minutes ":" Seconds ("." Fraction)?

Timecount-val ::= Timecount ("." Fraction)?("h" | "min" | "s" | "ms")? ; default is "s"

Hours ::= 2DIGIT; any positive number

Minutes ::= 2DIGIT; range from 00 to 59

Seconds ::= 2DIGIT; range from 00 to 59

Fraction ::= DIGIT+

Timecount ::= DIGIT+

2DIGIT ::= DIGIT DIGIT

DIGIT ::= [0-9]

Examples of legal clock values:

Full clock value: 02:30:03 = 2 hours, 30 minutes and 3 seconds Partial clock value: 02:33 = 2 minutes and 33 seconds Timecount values: 3h , 45min , 30s, 5ms

SMIL 1.0 — Synchronized Multimedia Integration Language Page 23 of 73

II Spatial layout

Top Story:

Growth of the World-Wide Web

Amsterdam

SMIL 1.0 — Synchronized Multimedia Integration Language Page 24 of 73

Possible ways to specify layout

t

x

y

t

x

y

t

x

y

t

x

y w.r.t. x,y axes w.r.t. item

function of time regions

SMIL 1.0 — Synchronized Multimedia Integration Language Page 25 of 73

SMIL — Region

Each media object instance contains a region reference:• allows author to know where object will be played

<video src=”anchor.mpg” region=”V-main” />

SMIL 3.3.1. region is defined by:

<region id="V-main" top="5%" left="50%" height="100%" width="100%" z-index=”3” />

<region id="V-remote" top="10" left="100" height="200" width="200" z-index=”3” />

• An “id” for each region is required. Its value is an XML identifier.• Length values are percentage values or in pixels. The unit “px” may be omitted.

and ZHW

x,y

SMIL 1.0 — Synchronized Multimedia Integration Language Page 26 of 73

Z-index

The Z-index gives the stacking order (highest integer stacks on top).If elements A and B have the same value Z-index then:

• if B starts after A then B is stacked on top of A.

• if B and A start simultaneously and B is lexically after A

then B is stacked on top of A.

SMIL 1.0 — Synchronized Multimedia Integration Language Page 27 of 73

Why Have Height and Width?

Amsterdam

Los Angeles

Top Story:

Growth of the

World-Wide Web

. . . blah blah blah blah

blah blah blah blah! Blah blah blah blah blah blah blah blah blah bla blah blah blah blah bla blah blah blah blah bla

blah blah blah blah bla blah blah blah blah bla blah blah blah blah...

SMIL 1.0 — Synchronized Multimedia Integration Language Page 28 of 73

Constraining Layout

Amsterdam

Los Angeles

Top Story:

Growth of the

World-Wide Web

. . . blah blah blah blah

blah blah blah blah! B

SMIL 1.0 — Synchronized Multimedia Integration Language Page 29 of 73

The WebNews Layout

<layout>

<root-layout width="721" height="587" id="matise" />

<region id="T_title" left="2%" top="5%" width="40%"height="24%" z-index=2 />

<region id="V-remote" left="3%" top="44%" width="54%"height="40%" z-index=3 />

...

</layout>

Top Story: Growth of the

World-Wide Web

Amsterdam

Los Angeles

V-main

T-anchor

r_title

V-remote

root-layout

T-title

SMIL 1.0 — Synchronized Multimedia Integration Language Page 30 of 73

SMIL — Basic Layout

SMIL 3.3.2. root-layout defines a rectangular area. All other regions are defined with respect to it — there is no nested, or relative, layout.

The root-layout is the viewport, or window, to which the presentation is rendered visually.

A SMIL document can have at most one root-layout.

If a media object element has no explicit region attribute then a default layout is given by the player.

To obtain the default layout for all media object elements, an empty SMIL basic layout can be declared:

<layout type="text/smil-basic-layout"></layout>

If no root-layout is specified then the size of the window is calculated by the player to be as large as the largest media object element.

SMIL 1.0 — Synchronized Multimedia Integration Language Page 31 of 73

SMIL— clips in space

SMIL 3.3.1 fit attribute

fill

hidden (default) hidden (default)

meet

scroll

slice

aspect rationot preserved

media item

aspect ratio

not scaledmedia itemnot scaled

preserved

media itemnot scaled

aspect ratiopreserved

SMIL 1.0 — Synchronized Multimedia Integration Language Page 32 of 73

Relationship between SMIL layout and CSS2

SMIL basic layout is consistent with the visual rendering module in CSS2. It introduces the “fit” attribute. It is otherwise a subset.

SMIL basic layout applies only to media object elements.

SMIL media object elements refer to a region. CSS2 “region” elements refer to the media object elements.

SMIL 1.0 — Synchronized Multimedia Integration Language Page 33 of 73

III Temporal Layout

anchor

remote

anchor

remote

anchor

growth

A

remoteA

storyA

SMIL 1.0 — Synchronized Multimedia Integration Language Page 34 of 73

Which time?

Types of time:• media item time axis

— video divided in frames, audio sampled at 44kHz

• document time— image starts at certain time and ends at a later time

• run-time presentation— video data bits get caught up in network, so end time is delayed

duration

start time end time

start time end time

SMIL 1.0 — Synchronized Multimedia Integration Language Page 35 of 73

Duration of a media object element

Intrinsic• derived from content of media item

• intrinsic duration of discrete media, such as text or image, is zero.

Explicit• an explicit duration can be given

SMIL 4.2.1. dur attribute, value is a clock-value or “indefinite”<video src="zoomin.mpv" region=”V-main” dur=”4s” />

<video src="zoomin.mpv" region=”V-main” dur=”6.5s” />

— in this case, the audio track just stops and the last frame of the video remains

audio (or video) lasts 5.3 seconds

media object stops after 4 seconds

media object stops after 6.5 seconds

SMIL 1.0 — Synchronized Multimedia Integration Language Page 36 of 73

Duration of a Media Object Element ctd.

An object can have its duration extended by repeating the content.

SMIL 4.2.1. repeat attribute<video src="zoomin.mpv" region=”V-main” repeat=”3” />

<video src="zoomin.mpv" region=”V-main” repeat=”3” dur=”11s” />

<video src="zoomin.mpv" region=”V-main” repeat=”indefinite” />

Attribute value of repeat is an integer or “indefinite”.

media object stops after 15.9 seconds

media object stopsafter 11 seconds

media object stopswhen parent stops

SMIL 1.0 — Synchronized Multimedia Integration Language Page 37 of 73

Start time of elements—par

SMIL 4.2.1 par element groups elements which are played in parallel • Children of a par element are started at the same time

<par>

<text src="leader_title.html" region="m_title" dur=”5s” />

<video src="cnn.mpv” region="V-Main" />

<audio src="cnn.aiff" region="music" />

</par>

• The start time of a child of a par element is equal to the start time of the par element itself.

anchor

anchor

storyA

SMIL 1.0 — Synchronized Multimedia Integration Language Page 38 of 73

Start time of elements—seq

SMIL 4.2.2 seq element groups elements which are played sequentially• Children are played one after the other, based on the textual order

<seq>

<video src="logo.mpv” region="V-main" />

<video src="anchor.mpv” region="V-main" />

</seq>

• The start time of the first child of a seq element is the start time of the seq element itself.

• The start time of the next child is the end time of the previous child.

anchor

anchor

storyA

SMIL 1.0 — Synchronized Multimedia Integration Language Page 39 of 73

Par’s and seq’s can be nested

<seq><par>

<text src="leader_title.html" region="m_title" dur=”5s”/><video src="cnn.mpv” region="V-Main" /><audio src="cnn.aiff" region="music" />

</par><par>

<text src="story_title.html" region="m_title" dur=”2s” /><video src="anchor.mpv” region="V-Main" /><audio src="anchor.aiff" region="music" />

</par><seq>

anchor

anchor

storyA

SMIL 1.0 — Synchronized Multimedia Integration Language Page 40 of 73

Explicit start time in a par element

SMIL 4.2.1. begin attribute, delay-value<par>

<text src="leader_title.html" region="m_title" dur=”5s” />

<video src="cnn.mpv” region="V-Main" begin=”1.4s” />

<audio src="cnn.aiff" region="music" />

</par>

• Video is delayed until 1.4s after the start of the par element.

• The syntax of the delay-value is a clock-value (page 18).

anchor

anchor

storyA

SMIL 1.0 — Synchronized Multimedia Integration Language Page 41 of 73

Explicit start time in a seq element

SMIL 4.2.1. begin attribute, delay-value<seq>

<video src="logo.mpv” region="V-main" />

<video src="anchor.mpv” region="V-main" begin=”1.4s”/>

</seq>

• Video is delayed until 1.4s after the end of the previous element.

anchor

anchor

storyA

SMIL 1.0 — Synchronized Multimedia Integration Language Page 42 of 73

Start time relative to another element

SMIL 4.2.1. begin attribute, event-value<par>

<text src="leader_title.html" region="m_title" dur=”5s” />

<video id=”v1” src="cnn.mpv” region="V-Main" begin=”1.4s” />

<audio src="cnn.aiff" region="music" begin=”id(v1)(0.5s)” />

</par>

• Audio is delayed until 0.5s after the start of video element “v1”.

• Syntax of event-value is “id(Id-value)(begin|end|clock-value)”.

anchor

anchor

storyA

SMIL 1.0 — Synchronized Multimedia Integration Language Page 43 of 73

End time of media object element

SMIL 4.2.4.2. A media object element with an implicit or explicit duration and a start time has an end = begin + duration.

<video src="cnn.mpv” region="V-Main" begin=”4s” />

SMIL 4.2.1. end attribute. Syntax same as begin attribute.SMIL 4.2.4.2. A media object element with an explicit start time and an explicit end has a duration = end - begin.

<text src="title.html" region="m_title" begin=”4s” end=”8s” />

anchor

begin end

duration

storyA

begin end

duration

SMIL 1.0 — Synchronized Multimedia Integration Language Page 44 of 73

End time of media object element ctd.

A media object element with an explicit start time, a duration and an explicit end, has as end time the minimum of the end time and (begin+duration).

<seq>

<text src="title.html" region="m_title" begin=”4s” dur=”4s” end=”10s” />

<text src="title.html" region="m_title" begin=”14s” dur=”6s” end=”18s” />

</seq>

storyA

begin end begin end

dur dur

end time=8s end time=18s

SMIL 1.0 — Synchronized Multimedia Integration Language Page 45 of 73

End time of media object element ctd.

A media object element may also have an intrinsic duration.<video src="cnn.mpv” region="V-Main" begin=”3s” dur=”14s”

end=”12s” />

anchor

begin (3s) end (12s)

implicit durationdur

end time=12s

(17s)

SMIL 1.0 — Synchronized Multimedia Integration Language Page 46 of 73

End time of media object element ctd.

If the end time is before the begin time then the element is not played.

The end time of a media object element with indefinite duration is determined by the end time of its parent.

If the parent has no definite end time then the media object element will play indefinitely.

SMIL 1.0 — Synchronized Multimedia Integration Language Page 47 of 73

End time of seq element

A seq element ends when its last child ends or according to its end attribute.

If its last child has an indefinite end then the seq has an indefinite end.<seq>

<video src="logo.mpv” region="V-main" />

<video src="anchor.mpv” region="V-main" />

<text src="title.html" region="m_title" dur=”indefinite” />

</seq>

anchor

anchor

storyA

SMIL 1.0 — Synchronized Multimedia Integration Language Page 48 of 73

End time of par element—first

SMIL 4.2.1 endsync• par can end when the first element to finish ends

<par endsync=”first”>

<text src="leader_title.html" region="m_title" dur=”5s” />

<video id=”v1” src="cnn.mpv” region="V-Main" begin=”1.4s” />

<audio src="cnn.aiff" region="music" begin=”id(v1)(0.5s)” />

</par>

anchor

anchor

storyA

end time

SMIL 1.0 — Synchronized Multimedia Integration Language Page 49 of 73

End time of par element—id-ref

• par can end when the referenced element ends: id(Id-value)<par endsync=”id(v1)”>

<text src="leader_title.html" region="m_title" />

<video id=”v1” src="cnn.mpv” region="V-Main" begin=”1.4s” />

<audio src="cnn.aiff" region="music" begin=”id(v1)(0.5s)” />

</par>

anchor

anchor

storyA

end time

SMIL 1.0 — Synchronized Multimedia Integration Language Page 50 of 73

End time of par element—last

• par can end when the last element to finish ends (default)<par endsync=”last”>

<text src="leader_title.html" region="m_title" dur=”5s” />

<video id=”v1” src="cnn.mpv” region="V-Main" begin=”1.4s” />

<audio src="cnn.aiff" region="music" begin=”id(v1)(0.5s)” />

</par>

anchor

anchor

storyA

end time

SMIL 1.0 — Synchronized Multimedia Integration Language Page 51 of 73

Types of synchronization

Hard synchronization• Video delayed by network, audio stops and waits

— useful for separate lip-synchronized audio and video streams.

Soft synchronization• Video delayed, audio continues — useful for background music.

• Note: this behaviour is not specified in the standard.

anchor

anchor

anchor

anchor

SMIL 1.0 — Synchronized Multimedia Integration Language Page 52 of 73

IV Alternate content

... explosive growth of the WWW ...

text

audio

... explosieve groei van het WWW ...

... eksplozivni rast WWW ...

... explosive growth of the WWW ...

... crescita esplosiva della WWW ...

SMIL 1.0 — Synchronized Multimedia Integration Language Page 53 of 73

Adaptation issues

Adaptation for User• Task/environment/disabilities• Language• Previous knowledge

Adaptation for Environment• Delays: bandwidth, available CPU time• Available processing: media peripherals, additional browser features

W3C Web Accessibility Initiative (WAI)• Guidelines for accessible (text-based) Web documents• Meaningful values for attributes like alt, title, abstract and longdesc• Meaningful content of link triggers (a element)• How to apply these to a fixed timeline?

SMIL 1.0 — Synchronized Multimedia Integration Language Page 54 of 73

Switch

SMIL 4.3. switch

• At most one of the children of a switch element is played.

• The first acceptable element is chosen, so ordering should be best first.<switch>

<audio system-bitrate=”44000” src=”hi-res.aiff” />

<audio system-bitrate=”16000” src=”low-res.aiff” />

</switch>

• If no element is suitable then no child of the switch is played.A catch-all choice at the end of the switch (with no test attribute) could be used.

SMIL 1.0 — Synchronized Multimedia Integration Language Page 55 of 73

Test Attributes

Selecting Content Alternatives• system-bitrate — bandwidth of Web connection

- can switch media: video -> image -> text• type — mime type of media object• system-required — select if certain processing available

Selecting for User• system-language — what language the user understands best• system-captions — show content if user want closed captioning (subtitles)

- usually single content of switch (on or off)

• system-overdub-or-caption — choice between audio or text

Adaptive Visual Complexity• system-screen-size, system-screen-depth

Extension Attributes for Particular Domain• Won’t be recognized by all browsers

Potential examples — knowledge level, audience profile, length of time

SMIL 1.0 — Synchronized Multimedia Integration Language Page 56 of 73

Multiple test attributes

Switch on bitrate and language<switch>

<audio system-bitrate=”44000” system-language=”nl” src=nl-hi-res.aiff />

<audio system-bitrate=”44000” system-language=”en”src=uk-hi-res.aiff />

<audio system-bitrate=”16000” system-language=”nl”src=nl-low-res.aiff />

<audio system-bitrate=”16000” system-language=”en”src=uk-low-res.aiff />

</switch>

SMIL 4.2.1. par, seq and media object elements can also have test attributes<textstream system-captions=”on” src=”txstream.rtx” />

• Element is played, or not, independently of any other element.

SMIL 1.0 — Synchronized Multimedia Integration Language Page 57 of 73

V Linking

Top Story:

Growth of the World-Wide Web

Amsterdam

Los Angeles

SMIL 1.0 — Synchronized Multimedia Integration Language Page 58 of 73

Link from element to presentation

SMIL 4.5.1. <a> element — similar to HTML <a> element.• Source is unaffected and destination, href, is shown in new window.

<a show="new" href="archives-dcab.smi">

<video src="zoomin.mpv" region="V-Main" />

</a>

• Source may also pause while destination is shown,

• or destination may replace the source (default).

Top Story:

Growth of the World-Wide Web

Amsterdam

Los Angeles

SMIL 1.0 — Synchronized Multimedia Integration Language Page 59 of 73

Link from element to element

SMIL 4.5. Linking to SMIL fragments• Destination element within another SMIL document uses # connector.

<a show="new" href="time-time.smil#XVII">

<text src="archives-dcab.html" region="I-Main" dur=”indefinite” />

</a>

• Destination presentation starts as if the presentation had been fast-forwarded to the beginning of the element designated by the fragment.

SMIL 1.0 — Synchronized Multimedia Integration Language Page 60 of 73

Link from element to subpart of media object

SMIL 4.5.2. anchor elementThe anchor element allows the specification of temporal and spatial subparts of a media object element.

• Spatial subparts use the coords attribute (similar to HTML image maps).<video src="zoomin.mpv" region="V-Main" >

<anchor id=”mic” coords=”40%, 70%, 55%, 100%” />

</video>

• Order of coords is left-x, top-y, right-x, bottom-y.• Temporal subparts use the begin and end attributes.

<video src="zoomin.mpv" region="V-Main" >

<anchor id=”graph-ref” begin=”4.3s” end=”6.8s” />

</video>

fit=”slice”

Defined w.r.t. media object,not w.r.t. region

SMIL 1.0 — Synchronized Multimedia Integration Language Page 61 of 73

Links in embedded documents

SMIL 4.5. Handling of links in embedded documents• SMIL players may involve other (non-SMIL) applications or plug-ins,

for example an HTML renderer.<text src=”comment.html” region=”C-right” />

• An HTML browser may display SMIL documents embedded in an HTML page.

• A SMIL document may contain a media object element of data type SMIL.<ref src=”webnews.smi” region=”S-left” />

. . . blah blah blah blah blah [4] blah

blah blah blah blah! Blah blah blah --blah blah blah. Blah blah blah? Blah!Blah blah blah, blah [5], [6], [7]. (Blah

: anchor in containing SMIL document : anchor in embedded SMIL document

Live from Amsterdam

Dick Bulterman next story

: anchor in embedded HTML fragment

SMIL 1.0 — Synchronized Multimedia Integration Language Page 62 of 73

Anchors as source and destination of a link

• href needed if used as source, id needed if used as destinationSource document (image in SMIL, anchor and link defined in SMIL):

<img src="home-sweet.gif" region="I-Main">

<anchor href=”time-time.smil#gable-3” show=”new”coords=”35%, 5%, 40%, 95%” />

</img>

Destination document “time-time.smil” (image in SMIL):

<img src="XVII.tiff" region="house-right" >

<anchor id=”gable-3” coords=”30%, 0%, 70%, 100%” />

</img>

SMIL 1.0 — Synchronized Multimedia Integration Language Page 63 of 73

VI Semantic annotations

SMIL 3.4. meta element defines properties of a document

• The name attribute is the property and the content attribute gives the value.<meta name=”title” content=”Web News, 15th June 1998” />

<meta name=”base” content=”http://www.cwi.nl/SMIL/webnews/” />

• The list of properties (values of name attribute) are open-ended.

Attributes on par, seq and media object elementsSMIL 4.2.1. abstract, author, copyright, title (recommended)

Attributes on media object elementsSMIL 4.2.3. alt (contains alternative text, recommended), longdesc (supplement to alt, but longer and should include descriptions of anchors)

Attributes on region elementsSMIL 3.3.1 title (recommended)

SMIL 1.0 — Synchronized Multimedia Integration Language Page 64 of 73

High-Level Structure of Document

Partitioning in Sections

<smil>

<head>

<meta>

... information about the document ...

</meta>

<layout>

... layout definition ...

</layout>

</head>

<body>

... objects and temporal relations ...

... links and anchor objects ...

</body>

</smil>

SMIL 1.0 — Synchronized Multimedia Integration Language Page 65 of 73

Backwards Compatibility

SMIL 3.3.1. skip-content attribute allows documents in higher SMIL versions to be played by players of lower SMIL versions.

If the skip-content attribute is true (default value) then the content of the element is ignored in the SMIL 1.0 player.

SMIL 1.0 — Synchronized Multimedia Integration Language Page 66 of 73

SMIL’s Relationship with Other W3C Recommendations

SMIL Documents are XML Documents• SMIL syntax is defined by an XML DTD

Private Extensions must use Namespaces• Skip-content attribute allows content of non-SMIL elements to be played• System-required attribute states the subtree requires the named

implementation

SMIL Layout and CSS-2• SMIL basic layout is consistent with the visual rendering module in CSS-2

- it introduces the “fit” attribute- it is otherwise a subset.

• SMIL basic layout applies only to media object elements.• SMIL media object elements refer to a region

- CSS-2 “region” elements refer to the media object elements.

SMIL 1.0 — Synchronized Multimedia Integration Language Page 67 of 73

SMIL Boston

Much Much More• SMIL 1.0 spec is 30 pages, SMIL Boston spec is 300 pages

Animation• Values of SMIL constructs change over time• Enables more vibrant presentation• Incorporation with SVG

Timing Integration• Use of SMIL constructs in other document sets• Enables, for example, HTML+SMIL in Internet Explorer• Raises issues of semantic significance of hierarchy

Broadcasting/streaming• No preload or full download• Use of non-predictive events in timing• Need to maintain hard synchronization• Large potential use of SMIL

SMIL 1.0 — Synchronized Multimedia Integration Language Page 68 of 73

SMIL Boston Modules

SMIL is broken up into separate modules• Thus not all of SMIL Boston needs to be used in one instance

The SMIL Boston Modules are:• Animation• Content Control — selection, adaptation and optimization• Layout • Linking — navigation• Media Object — media content that is integrated into presentation• Metadata — machine-processible data about the presentation• Structure — base elements for high-level SMIL structure• Timing and Synchronization — 98 pages!!• Transition Effects — fades and wipes• Document Object Model (DOM) — XML data transfer and exchange

SMIL 1.0 — Synchronized Multimedia Integration Language Page 69 of 73

SMIL Boston Language Profiles

Multiple SMIL Profiles• Made by combining SMIL Boston modules

SMIL Boston Profile• “Maximum” SMIL• Describes how all SMIL Boston modules are combined into one format

HTML+SMIL• In Internet Explorer 5.5 — originally HTML+Time• Encodes “PowerPoint-like” presentations• Textflow-based display that changes with time and interaction

SMIL Basic Profile• “Minimum” SMIL• For low-end devices like mobile phones and palmtops• Ensures a class of SMIL presentations that can be seen on these devices

Baseline Media Formats• Non-normative selected media formats for each media type

SMIL 1.0 — Synchronized Multimedia Integration Language Page 70 of 73

Existing Tools

RealPlayer 7 (G2)• Primary driving force

- millions downloaded

• Real-Media specific player made broader with SMIL incorporation• 3rd party template-focussed authoring tools

GRiNS• Commercial spinoff from CMIFed research tool• High-end authoring tool for complex presentations• Authors presentations using RealPlayer 7 formats• GRiNS player

Internet Explorer• HTML+SMIL profile

QuickTime 4.1• Recently announced SMIL support

SMIL 1.0 — Synchronized Multimedia Integration Language Page 71 of 73

SMIL Applications

Infotainment Accessibility

Conceptual Art

SMIL 1.0 — Synchronized Multimedia Integration Language Page 72 of 73

Summary

SMIL — media object element revisited<video id=”vid1” region=”R_video”

src=”rtsp://www.w3.org/CoolStuff.rm” clip-begin="smpte=00:01:19:20" clip-end="smpte=00:01:38:40"begin=”3s” dur=”22s”end=”21s”alt=”Video of Joe chatting to Tim”longdesc=”Joe and Tim are in a meeting room. Joe is on the

left and Tim is on the right”title=”Joe greets Tim”system-bitrate=”28800”>

<anchor id="joe" begin="0s" end="5s" coords="0%,0%,50%,50%"href="http://www.w3.org/" />

<anchor id="tim" begin="5s" end="10s" coords="50%,50%,50%,50%"href="http://www.w3.org/Tim" />

</video>

SMIL 1.0 — Synchronized Multimedia Integration Language Page 73 of 73

More information

• http://www.cwi.nl/SMIL

• http://www.w3.org/TR/REC-SMIL• http://www.w3.org/TR/SMIL-Boston

• http://www.w3.org/AudioVideo/• http://www.w3c.org/Press/1998/SMIL-REC

• http://www.justsmil.com/

• http://www.oratrix.com/• http://www.real.com/player