streaming of svg animations on the web
Post on 29-Nov-2014
3.138 Views
Preview:
DESCRIPTION
TRANSCRIPT
Cyril Concolato
Streaming
SVG Animations
on the Web
Cyril Concolato
HTML5 Dev Conf / The Graphical Web
22-23 Oct 2013
Cyril Concolato
Context and Motivations
Animating using web technologies is getting popular
• Lots of content being produced
« SVG 2 » and « Web Animations 1.0 » being drafted
• Opportunities to improve the technologies
My goal: Enable treating web animations like video
• Why?
• What kind of animations?
• How?
23/10/2013 Streaming SVG animations on the Web 2
Cyril Concolato
Web Animation Use Cases
Animations in User Interfaces
Animations for Data Visualization
Timeline-based Animations
• Graphics animations (cartoons, …)
• Synchronized audio & graphics (karaoke, ads …)
• Synchronized video & graphics (annotations, subtitles)
23/10/2013 Streaming SVG animations on the Web 3
Popcorn.js SVG Wow! Cartoon
Cyril Concolato
Timeline-based Animations Possibilities
Playback control using a clock
• Play(0)/Play(T)
• Pause/Resume
• Speed-up/slow down
Accurate synchronization
• Between animations or
between animation and media
• Dependent clocks and timestamps
23/10/2013 Streaming SVG animations on the Web 4
Similar to video
but not allowed in HTML5 <video>
Cyril Concolato
An SVG video-like JavaScript Player
Principles
Using HTML 5 <video> syntax but for SVG
content
23/10/2013 Streaming SVG animations on the Web
1
2
3
<video controls width="640" height="480">
<source src="myAnimatedFile.svg" type="image/svg+xml" >
</video>
5
Demo 1
Demo 2
1
2
3
4
<video controls width="640" height="480">
<source src="file.mp4" type="video/mp4" >
<track src="annotations.svg" kind="graphics" type="image/svg+xml">
</video>
Cyril Concolato
An SVG video-like JavaScript Player
Status
Simple Implementation
• GUI based on Video.js
• Replacing <video> by <svg> at run-time
• Using XHR to fetch SVG data and insert in the page
• Uses get/setCurrentTime() and un/pauseAnimations()
Limitations
• Bugs!
• Same origin restrictions
• Incremental/progressive loading not working
• Cannot reproduce HTML network-related events (can
play, can play through)
23/10/2013 Streaming SVG animations on the Web 6
Cyril Concolato
An SVG video-like JavaScript player:
Next Step
Network-aware Controlled Playback • Load SVG data just before it’s needed (and let garbage collection
do the rest)
How to achieve controlled playback of SVG ? • Apply streaming concepts
23/10/2013 Streaming SVG animations on the Web 7
Cyril Concolato
Applying streaming concepts to SVG
Streaming “Controlled continuous delivery and consumption of data units”
What’s needed to stream SVG content? • Fragmentation of SVG documents into streaming units
─ Needs guidelines for SVG authoring
• Assign timing to each unit ─ Mapping between byte-offset/time-position of the SVG document
─ Needs a streaming instruction format
• Seek information ─ Needs explicit processing behavior
SVG Streaming draft • http://dev.w3.org/SVG/modules/streaming/
• Feedback welcome
Example: Streaming SVG Cartoons
23/10/2013 Streaming SVG animations on the Web 8
Cyril Concolato
Cartoons on the Web – Today
Largely based on Vector Graphics • Bezier curves filled with solid color or simple gradients and/or stroke
• Sometimes using raster images (e.g. background or texture)
Mostly frame-based content • Sometimes with additional interpolation
• Typically synchronized with a sound track
• Good candidate for streaming
Currently delivered as Flash or video streams • Plugins, coding artefacts or bandwidth problems …
23/10/2013 Streaming SVG animations on the Web 9
Source: www.cartoonnetwork.com
Cyril Concolato
Cartoons on the Web – Next Steps
Could be better integrated in the web platform
• Using of vector graphics primitives for scalability and new primitives for advanced rendering
─ E.g. coons patches, hatches (SVG 2), diffusion curves, …
• Using other web technologies
─ Styling, language translation, …
Attempts to convert Flash to HTML5/CSS/SVG/Canvas
• Many JS-based approaches
─ Google Swiffy
─ Mozilla Shumway
─ Adobe CreateJS
─ PixelPlant
• Some declarative approaches
─ MP4Box (100% SVG)
23/10/2013 Streaming SVG animations on the Web 10
Cyril Concolato
Frame-based SVG Cartoons using MP4Box <svg viewBox="0 0 768 576" ...>
<rect width="100%" height="100%" fill="lightblue"/>
<g id="frame_1" display="none">
<animate attributeName="display" to="inline" begin="0" end="0.16666667"/>
<defs>
<g id="Shape1"><path fill="rgb(111,90,41)" stroke="none" d="M 524.0 -109.0 ..."/>...</g>
<g id="Shape2"><path fill="rgb(111,90,41)" stroke="none" d="M 524.0 -109.0 ..."/>...</g>
<g id="Shape3">...</g>
</defs>
<use xlink:href="#Shape1" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/>
<use xlink:href="#Shape2" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/>
<use xlink:href="#Shape3" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/>
</g>
<g id="frame_2" display="none">
<animate attributeName="display" to="inline" begin="0.16666667" end="0.25"/>
<defs>
<g id="Shape4"><path .../></g>
</defs>
<use xlink:href="#Shape1" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/>
<use xlink:href="#Shape4" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/>
<use xlink:href="#Shape2" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/>
</g>...
</svg>
23/10/2013 Streaming SVG animations on the Web 11
Cyril Concolato
Fragmentation of SVG document
into streaming units
Identify stream header • Static throughout the streaming session
• Required to initialize the parser
Make sure SVG data is properly ordered • Use only backward references
• Document order == time order
Identify units • Assign timestamps
• No overlap in position / possible overlap in time (like WebVTT cues)
Make self-discardable units • <discard>, JavaScript, scoped stylesheets
Mark self-contained units • Random Access Point
23/10/2013 Streaming SVG animations on the Web 12
Cyril Concolato
Streaming Instructions Example
<stream timescale="24" file="cuisine.swf.svg">
<header range-end="204"/>
<unit time="0" range-end="353" rap="1" />
<unit time="2" range-end="38403" rap="0" />
<unit time="3" range-end="39591" rap="0" />
<unit time="5" range-end="41923" rap="0" />
<unit time="6" range-end="45825" rap="0" />
<unit time="7" range-end="51276" rap="0" />
<unit time="10" range-end="52596" rap="1" />
<unit time="12" range-end="55261" rap="0" />
…
</stream>
23/10/2013 Streaming SVG animations on the Web 13
Cyril Concolato
How to generate Streaming Instructions ?
Post-authoring
• Requires byte positions to be final
Hard to determine without author’s instructions
• Where does the frame start/end?
• What’s the timestamp of the unit?
• Where are the random access points?
Possible automatic generation from Flash
Possible with additional elements/attributes
23/10/2013 Streaming SVG animations on the Web 14
Cyril Concolato
Flash-to-SVG streaming
23/10/2013 Streaming SVG animations on the Web 15
SWF
file Converter
SVG
file Packager MP4
file
Extractor
SVG
file
Streaming
Instructions
file
Converter &
Packager
Cyril Concolato
Generating Streaming Instructions
for SVG content converted from Flash
23/10/2013 Streaming SVG animations on the Web 16
MP4Box -add flash.swf:fmt=svg file_with_svg_track.mp4
MP4Box -six 1 file_with_svg_track.mp4
MP4Box -info file_with_svg_track.mp4
Cyril Concolato
SVG Streaming Demo
• Playing an SVG stream
• Viewing Random Access Points and others
• Seeking into an SVG unit not yet downloaded
• Concatenating SVG content (splicing)
• Live and pseudo-live streaming of SVG content
─ Web-based Vector Graphics TV channel
23/10/2013 Streaming SVG animations on the Web 17
Web Server
(byte-range support)
Browser
+
JS Player
SVG
+
Streaming
Instructions
Demo 3
Cyril Concolato
SVG streams and Media Containers
Why? What for? • Packaging audio/video/graphics together
─ To download only one resource
─ To facilitate synchronization of tracks
• Possible use with HTML5 native <video> elements ─ SVG in-band graphical track
• Possible use with HTML 5 Media Source extensions ─ Adaptive streaming of SVG
Formats • ISO/IEC-14496:12 (royalty-free) defines how to store
timed text in MP4
• Could be done for other formats (WebM, Ogg, …)
23/10/2013 Streaming SVG animations on the Web 18
Flash MP4Box
MP4
SVG
MP4
SVG Video
Audio
Cyril Concolato
SVG streams and WebVTT
WebVTT allows metadata tracks and raw cues
23/10/2013 Streaming SVG animations on the Web 19
Flash MP4Box
SVG
WebVTT
WebVTT
SVG
MP4Box –webvtt-raw 1 file_with_svg.mp4
MP4Box –webvtt-raw 1:embedded file_with_svg.mp4
1
2
3
4
<video controls width="640" height="480">
<source src="file.mp4" type="video/mp4" >
<track src="annotations.vtt" kind="metadata">
</video>
Cyril Concolato
SVG embedded in WebVTT
23/10/2013 Streaming SVG animations on the Web 20
WEBVTT Metadata track generated by GPAC MP4Box kind: metadata inBandMetadataTrackDispatchType: image/svg+xml metadata-header: <svg viewBox="0 0 384 288" ...> <rect width="100%" height="100%" fill="lightblue"/> 00:00:00.000 --> 00:00:00.083 isRAP:true <g display="none"> <animate ... /> <defs>...</defs> <use .../> </g> 00:00:00.083 --> 00:00:00.025 isRAP:false <g>...</g>
Cyril Concolato
SVG referenced in WebVTT
23/10/2013 Streaming SVG animations on the Web 21
WEBVTT Metadata track generated by GPAC MP4Box
kind: metadata
inBandMetadataTrackDispatchType: image/svg+xml
baseMediaFile: flash7.svg
text-header-length: 207
00:00:00.000 --> 00:00:00.083 mediaOffset:207 dataLength:311 isRAP:true
00:00:00.083 --> 00:00:00.166 mediaOffset:518 dataLength:320 isRAP:false
00:00:00.166 --> 00:00:00.250 mediaOffset:838 dataLength:207 isRAP:false
...
Cyril Concolato
Summary & Conclusion
Presented • Guidelines for authoring streamable SVG content
• Tools for generating streaming instructions ─ from Flash, will be extended to support SVG natively
• JavaScript-based players for streamable SVG content
Outcome • Provided that your SVG content is well structured and annotated,
you can stream SVG content ─ In different modes: Controlled Progressive Download, Live, On Demand,
Playlists, Adaptive, …
─ Out of band graphics tracks (currently with JS shim)
─ In band tracks in media containers (in browsers in the future)
Future work • Implement adaptive streaming (MP4 based)
• Implement media synchronization (WebVTT)
• Work with CSS-based or JS-based animations
23/10/2013 Streaming SVG animations on the Web 22
Cyril Concolato
Thank you
Questions?
23/10/2013 Streaming SVG animations on the Web 23
Cyril Concolato
Further reading
SVG Related
• HTML 5 Video and SVG Graphics Synchronization
• Carriage of SVG and WebVTT in MP4 (Blog, Text
Track CG)
Streaming related
• Live streaming over HTTP of video and subtitles (Part
1 & Part 2)
• WebVTT streaming
23/10/2013 Streaming SVG animations on the Web 24
top related