silverlight multimedia

Multimediain

Silverlight

Carl Franklin Franklins.Net

carl@franklins.net

• Media Features in Silverlight 3.0 / 4.0• Encoding• MediaElement Overview• Third-Party Projects• Cool Demos!• Let us repair to the pub

Multimedia Overview

• Supported Resolutions:– All – up to 720/1080p

• Codecs:– H.264 Native– MP4/F4V– WMV – SMPTE VC-1 Standard– Third-party codecs

• GPU Acceleration• Olympics, Sunday Night Football, DNC, Presidential

Inauguration.

True HD Video

• Adapts bitrate/quality as bandwidth/cpu changes• HTTP-based• Quick start-up time• Minimal buffering• Uninterrupted viewing• Requires IIS 7.0 Media Services• Live Smooth Streaming (in Silverlight 4)

– PVR-style access– Navigation, scrubbing

IIS Media Services : http://www.iis.net/mediaSmooth Streaming Demo – Watch it in actionhttp://shrinkster.com/1d0g

Smooth Streaming

• One-to-many video broadcasting• Silverlight 3.0

– QUMU Plugin (http://www.qumu.com)– Free on Codeplex– Polycom Video Media Center 1000 uses it

• Silverlight 4.0– Built-in. No plugin necessary.

Multicast Support

• Write once. Run on Windows, Mac, and Linux• Store and play offline video out-of-browser• DRM for stream and content protection

– Silverlight DRM– Windows Media DRM– PlayReady

• Advertising Support– Linear pre-rolls– Post-rolls– Interstitials– Non-linear tickers– Bugs– Transparent overlays

General Silverlight Features

• Expression Encoder– Convert Videos to Silverlight-compatible WMV– Create HTML/Javascript Template– Copy to web server– Rock and roll

Encoding Videos for Silverlight

• Expression Encoder Live Encoding Feature– Select Video Source– Select Push or Pull Modes– Publish– Rock and Roll– Small potatoes – doesn’t scale.

Live Encoding

• Real geeks use hardware– InLet Technologies Spinnaker– http://shrinkster.com/1d0r– Olympics :

• 20 of these @ 20K each• IIS Server Farm

– Camera -> Inlet -> IIS7 -> CDN• CDN = Content Delivery Network

– Akamai, Limewire» Big $$

Hardware Encoding

MediaElement

Plays video or audio• Streamed or downloaded media• Video: WMV, H.264• Audio: AAC, MP3 and WMA• Add Media File to your project

– Set Build Action property (of media file) to Resource<MediaElement Source="bear.wmv" />

• Scrubbing Media– Works on Downloaded or Buffering Media– Position method is writable

• Position not valid until MediaOpened Fires• Downloaded media it simply changes the

location• Buffering media it may cause re-buffering• NaturalDuration gives you the length of the

media– Though may be invalid in some streams

(e.g. Live)

Using MediaElement

thePlayer.Position = thePlayer.Position.Add(TimeSpan.FromSeconds(5));

• Markers– Media must contain markers to work– MarkerReached Event Fired when a marker is

reached• Sends a TimeMarker object as the event

argument

Using MediaElement

void thePlayer_MarkerReached(object sender, TimelineMarkerRoutedEventArgs e) { TimelineMarker m = e.Marker; theStatus.Text = string.Format("{0}:{1}:{2}", m.Type, m.Text, m.Time);}

• Silver Bay Labs Development Guide Videos– Table of Contents on Left– Video on Right– Important Code in Textbox below video– http://shrinkster.com/1d0j

Good Use of Markers

• Play Position– No event for Position Change– Use Binding to Position Property Instead

Using MediaElement

<TextBlock Text=“{Binding ElementName=MediaElement1,

Path=Position}” />

• Video Capture• Media Security

What’s New in Silverlight 4.0?

• Webcam and Microphone Support– AsyncCaptureImage

• Captures a single frame, sends bitmap to callback

– VideoBrush.SetSource• Sets the source of the VideoBrush using a video

capture source

Webcam Sample Apps online:http://shrinkster.com/1d0o

Video Capture

void AsyncCaptureImage(Action<WriteableBitmap> Callback)

void SetSource(CaptureSource source)

• VideoSink– Base Class for creating video capture classes– Override OnSample, convert data to your format– CaptureSource property provides a capturesource

associated with your VideoSink• CaptureSource

– Use AsyncCaptureImage to capture a frame• VideoCaptureDevice

– Provides metadata for a video capture device, such as a webcam.

Webcam Sample Apps online:http://shrinkster.com/1d0o

Video Capture Classes

if (CaptureDeviceConfiguration.AllowedDeviceAccess || CaptureDeviceConfiguration.RequestDeviceAccess()){ VideoCaptureDevice vcd = CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice(); AudioCaptureDevice acd = CaptureDeviceConfiguration.GetDefaultAudioCaptureDevice(); if (null != vcd && null != acd) { cs = new CaptureSource(); cs.VideoCaptureDevice = vcd; cs.AudioCaptureDevice = acd; cs.Start();  VideoBrush videoBrush = new VideoBrush(); videoBrush.Stretch = Stretch.Uniform; videoBrush.SetSource(cs); TO_FILL.Fill = videoBrush; } else MessageBox.Show("Error initializing Webcam or Microphone.");}

Video Capture Sample

• Protection for Content Owners– Protected content is only viewed through a secure

connection• Supported Scenarios:

– Rental, Subscription, Purchase• Read Output Protection policies inside PlayReady

Licenses• Offline Content Protection for H.264 media through

Silverlight DRM– Two methods

• Windows Media DRM (WMDRM10)• PlayReady (AES)

Content Security

• Silverlight Media Player• Silverlight Media Framework• Silverlight Hyper-Video• Augmented Reality

Third Party Goodies and the Future

• Lightweight, enough features for most video streams• Used by Channel 9 • Developed by Tim Heuer of Microsoft and MVP Joel

Nuebeck• http://shrinkster.com/1d0p

Silverlight Media Player

• Came out of Vertigo Software’s work with Olympics and Sunday Night Football

• Does all the heavy lifting• Rich multimedia control

– DVR, Rewind, Alternate Language Tracks, In-stream Data Feeds, Analytics Tracking, Player Branding, Markers.

• http://smf.codeplex.com/

Silverlight Media Framework

• 11-18-09– SMF Released at PDC

• 1-15-10– SMF v1.1 released

• Progressive Download Added• SMF v2.0 under development

SMF Timeline

• Timeline Markers• Progressive Download Support• Smooth Streaming Support• IIS In-Stream Data Access Framework• Out-of-Stream Data Access Framework

SMF Player 1.1 Features

• Playlists• Chapters• Manifest Marker Support• Javascript API• Closed Captioning• W3C Timed Text 1.0 DFXP Support• UI Automation• Media Plug-In Model• Logging Plug-IO

SMF 2.0 Additional Features

• Hypertext-like features for Video– Jesse Liberty – http://shrinkster.com/1d0q– Click or gesture while watching a video– Video pauses and you get some other information

• Usually another video• Could be any kind of media

– Uses the MEF framework• http://slhvp.com• Video Demo: http://shrinkster.com/1d0s

Silverlight Hyper Video Project

• Augmented Reality– Adding to (or augmenting) video with

superimposed or embedded graphics.• SLAR Toolkit

– http://slartoolkit.codeplex.com/– Silverlight 4.0 Only– Webcam support via CaptureSource– Multiple Marker Detection, Custom markers– Uses the Matrix3DEx library by Rene Schulte

• Adds 3D features to Silverlight• http://matrix3dex.codeplex.com/

SLARToolkit – Silverlight AugmentedReality Toolkit

• Sharepoint Video Player– Embed video streams into your Sharepoint

website.– http://svp.codeplex.com/

• xLite Player– http://xliteplayer.codeplex.com/

Other Players

• Scott Guthrie’s Silverlight 4 Demos from PDC– http://shrinkster.com/1d0f

• Silverlight 4 Demos– http://shrinkster.com/1d0o

• Sunday Night Football– http://www.vertigo.com/snf

• March Madness– http://mmod.ncaa.com/video?player=hq

Demos!