html5 multimedia accessibility

HTML5 multimediaAccessibilityBruce Lawson

Anne van Kesteren annevk at opera.comWed Feb 28 05:47:56 PST 2007

Hi,Opera has some internal expiremental builds with an implementation of a <video> element. The element exposes a simple API (for the moment) much like the Audio() object: play() pause() Stop()

The idea is that it works like <object> except that it has special <video> semantics much like <img> has image semantics. In markup you could prolly use it as follows:

<figure> <video src=news-snippet.ogg> ... </video> <legend>HTML5 in BBC News</legend> </figure>

I attached a proposal for the element and as you can see there are still some open issues. The element and its API are of course open for debate. We're not enforcing this upon the world ;-)

Cheers,http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2007-February/009702.html

<object width="425" height="344"><param name="movie"

value="http://www.example.com/v/9sEI1AUFJKw&hl=en&fs=1&"></param>

<param name="allowFullScreen" value="true"></param>

<param name="allowscriptaccess" value="always"></param>

<embed src="http://www.example.com/v/9sEI1AUFJKw&hl=en&fs=1&" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="425" height="344"></embed></object>

<video src=pudding.webm controls autoplay poster=poster.jpg width=320 height=240> <a href=video.webm>Download movie</a></video>

<audio src=bieber.ogg controls autoplay> <a href=bieber.ogg>Download horrid pap</a></audio>

<video src=pudding.webm loop>

<audio src=bieber.ogg preload><audio src=bieber.ogg preload=auto><audio src=bieber.ogg preload=none><audio src=bieber.ogg preload=metadata>

video as native object...why is it important?

● keyboard accessibility built-in● “play nice” with rest of the page● Simple API for controls

"In addition to giving video an HTML element, we must also agree on a baseline video format that will be universally supported, just like the GIF, JPEG and PNG image format are universally supported. It's important that the video format we choose can be supported by a wide range of devices and that it's royalty-free (RF). RF is a well-established principle for W3C standards."

http://people.opera.com/howcome/2007/video/

video formats

webM Ogg/ Theora mp4/ h264

Opera yes yes

Chrome yes yes Nope (Chrome.soon)

Firefox Yes (FF4) yes

Safari yes

IE9 Yes (if installed) yes

audio formats

mp3 Ogg/ Vobis wav

Opera yes yes

Chrome yes yes

Firefox yes yes

Safari yes yes

IE9 yes yes

Giving everybody video

<video controls autoplay poster=… width=… height=…><source src=pudding.mp4 type=video/mp4><source src=pudding.webm type=video/webm><source src=pudding.ogv type=video/ogg><!-- fallback content -->

</video>

<video controls poster="…" width="…" height="…"><source src="movie.mp4" type="video/mp4" /><source src="movie.webm" type="video/webm" /><source src="movie.ogv" type="video/ogg" /><object width="…" height="…" type="application/x-

shockwave-flash" data="player.swf"><param name="movie" value="player.swf" /><param name="flashvars" value=" … file=movie.mp4" /><!-- fallback content -->

</object></video>

still include fallback for old browsershttp://camendesign.com/code/video_for_everybody

powerful (simple) API

www.w3.org/TR/html5/video.html#media-elements

controlling <video> with JavaScript

var v = document.getElementByTagName('video')[0];

v.play();v.pause();v.volume = … ;v.currentTime = … ;…

events fired by <video>var v = document.getElementById('player');v.addEventListener('loadeddata', function() { … }, true)v.addEventListener('play', function() { … }, true)v.addEventListener('pause', function() { … }, true)v.addEventListener('timeupdate', function() { … }, true)v.addEventListener('ended', function() { … }, true)…

<video> accessibility

WebM release does not support subtitles

http://code.google.com/p/webm/issues/detail?id=11Egg image Kacper "Kangel" Aniołek http://commons.wikimedia.org/wiki/File:Egg.jpg

WHATWG / W3C RFC will release guidance on subtitles and other overlays in HTML5 <video> in the near future. WebM intends to follow that guidance.

<track> element

<video controls poster=… width=… height=…><source src=movie.webm type=video/webm><track src=subtitles-en.vtt kind=subtitles srclang=en><track src=subtitles-de.vtt kind=subtitles srclang=de><!-- fallback content -->

</video>

http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#the-track-element

● 1 Adobe Encore

● 2 AQTitle (.aqt)

● 3 ARIB STD-B24

● 4 AYA (.aya)

● 5 CA (.ca)

● 6 CHK (.chk)

● 7 CIN (.cin)

● 8 CIP (.cip)

● 9 DKS (.dks)

● 10 DVD Studio Pro

● 11 EBU STL (.ebu)

● 12 FAB Subtitler ★

● 13 Gloss Subtitle (.gsub)

● 14 JACOSub (.jss)

● 15 Kate

● 16 L32 (.l32)

● 17 MacSUB

● 36 Subsonic

● 37 smilText

● 38 SubStation Alpha (.ssa)

● 39 SubViewer (.sub*)

● 40 TIT (.tit)

● 41 TitleVision TV2003 text format

● 42 TTML

● 43 Turbotitler

● 44 ULT (.ult)

● 45 Universal Subtitle Format (.usf)

● 46 VPlayer

● 47 WinCaps (.w32)

● 48 XombieSub

● 49 ZeroG (.zeg)

● 50 CVD

● 51 DVB subtitles

● 52 Philips DVD subtitling format

● 53 VobSub (.sub and .idx), XSUB (DivX subtitles)

● 55 CMML, SMIL,

● 18 MicroDVD (.sub*)

● 19 MPEG-4 Timed Text (.ttxt)

● 20 MPL (.mpl)

● 21 MPSub (.sub*)

● 22 Ogg Writ

● 23 Phoenix Subtitle (.pjs)

● 24 PowerDivX (.psb)

● 25 PU2000 (.rac, .pac)

● 26 RealText (.rt)

● 27 SAMI (.smi)

● 28 Scantitling format 890 (.890)

● 29 Screen Poliscript

● 30 Softel SwiftXIF (.xif)

● 31 Sonic Solutions DVD text format

● 32 Spruce STL

● 33 ST4, ST7 (.st4, .st7)

● 34 Structured Subtitle Format (.ssf)

● 35 SubRip (.srt) ★

webVTT

WEBVTT FILE

101:23:45,678 --> 01:23:46,789Hello world!

201:23:48,910 --> 01:23:49,101HelloWorld!

WebVTT formatting

Supports positioning of text

Supports <b> and <i>

Colouring individual speakers

Support vertical text

Supports RTL

Supports ruby annotations

<track> polyfillhttp://www.delphiki.com/html5/playr/

Synchronising media elementsEach media element can have a MediaController. A MediaController is an object that coordinates the playback of multiple media elements, for instance so that a sign-language interpreter track can be overlaid on a video track, with the two being kept in sync....

Media elements with a MediaController are said to be slaved to their controller. The MediaController modifies the playback rate and the playback volume of each of the media elements slaved to it, and ensures that when any of its slaved media elements unexpectedly stall, the others are stopped at the same time.

When a media element is slaved to a MediaController, its playback rate is fixed to that of the other tracks in the same MediaController, and any looping is disabled.

http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#synchronising-multiple-media-elements

http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_API

getUserMedia API(previously known as "HTML5 <device>")

<video autoplay></video><script>var video = document.querySelector(video);

If (navigator.getUserMedia) {navigator.getUserMedia('video',successCallback, errorCallback);

function successCallback(stream) { video.src = stream; }...</script>

my.opera.com/core/blog/2011/03/23/webcam-orientation-preview