design in motion: video production workflow
TRANSCRIPT
![Page 1: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/1.jpg)
![Page 2: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/2.jpg)
Design in Motion:Video Production WorkflowDES05Ben WaggonerTechnical Evangelist,Core Media Processing TechnologiesMicrosoft Corporation
![Page 3: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/3.jpg)
Agenda
Video and audio in WPF/EBest practices for processing video and audioBest practices for Windows Media encoding for WPF/EUsing the Expression Media Encoder
![Page 4: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/4.jpg)
Video and audio in WPF/E
It's just Windows Media!Huge library of existing content already compatibleExisting URLs from Windows Media Services still workAlready supported by many hosting companiesLots of tools available for different markets
Video codec;Windows Media Video 7/8/9 and 9 Advanced Profile
Audio codec:Windows Media Audio "Standard"
Not Pro, Voice, or Lossless in this version
![Page 5: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/5.jpg)
HD Video Playback
video
FLV v. WMV
![Page 6: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/6.jpg)
What does WMV give content creators?
Higher quality video and audio Fewer bits needed for same quality
Much faster encoding3x faster for optimum quality on 4-core workstation15x faster with Tarari card for hardware acceleration
Much better TCO for video hostingMany more streams off a 2K3 Server than Flash serverCheaper per GB charges from hosting providers
Mature live streaming/encoding tools
![Page 7: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/7.jpg)
Best Practices for Video Processing
CaptureGetting the content in higher quality
PreprocessingOptimizing the content for the codec itself
![Page 8: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/8.jpg)
Best Practices: Capture
Capture native bitstream or uncompressedDV, DVCPRO HD as native bitstreamOther formats as uncompressedDon’t use DV bridge for analog-digital conversion
DV codec throws out a lot of image data, especially color
Access earliest generation dub as possibleMost VHS tapes didn’t get posted on VHS!Ripping from DVD also sub-optimal
Best to grab digital export from NLE
![Page 9: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/9.jpg)
Preprocessing
Everything we do to the video and audio between the source and the codecThe biggest single factor in good-looking WPF/E video
The best the codec can do is give back what you gave it
For challenging projets, I spend ~80% of my keyboard-and-mouse time on preprocessing
Once the codec settings are nailed, you're good to go.
![Page 10: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/10.jpg)
Unprocessed frame
![Page 11: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/11.jpg)
Same Frame Preprocessed
![Page 12: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/12.jpg)
Unprocessed @ 800 Kbps
![Page 13: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/13.jpg)
Preprocessed @ 800 Kbps
![Page 14: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/14.jpg)
Deinterlacing and Inverse Telecine
WPF/E uses progressive videoFor 60i/50i, deinterlace to progressiveFor video telecined from film, use inverse telecine to restore 24pIf it’s progressive, leave it alone!<example slide>
![Page 15: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/15.jpg)
Crop
Video monitors don’t go to edgesComputers doCrop edge blankingAsymmetric okay to cut out noiseFor low rez, crop to safe areaCrop letterboxing as wellIf going to 16x16 resolution, can crop to final aspect ratioFor mixed video/computer playback, matte out noise in blanking
![Page 16: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/16.jpg)
The Safe Area
![Page 17: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/17.jpg)
Image adjustment
Brightness and ContrastComputers have broader range — use itBlacks should look identical to RGB=0,0,0 in browser
Noise ReductionMake image easier for compressorRemove grainSmooth sharp edges
![Page 18: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/18.jpg)
Levels example
![Page 19: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/19.jpg)
Levels in Action
After The Settings
![Page 20: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/20.jpg)
![Page 21: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/21.jpg)
Best Practices: Audio Normalization
Make loudest point of audio as loud as system beep-3 dB / 90%Not compressor/limiter - no change in dynamic range
![Page 22: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/22.jpg)
Best Practices: Encoding
Progressive versus StreamingData RatesFrame SizeFrame RateAudio SettingsAdvanced Registry Key settings
![Page 23: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/23.jpg)
Progressive v. Streaming
Two legitimate modes of delivering web videoNeed to know which one will be used before encoding
Progressive should use 2-pass VBRStreaming should use 2-pass CBR
![Page 24: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/24.jpg)
Progressive Download
Off a web serverE.g., IIS in Windows 2003 Server
Most efficient with Variable Bit Rate (VBR) encodingRandom access in part of stream already downloadedGreat for movie trailers
![Page 25: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/25.jpg)
Streaming
Files fromWindows Media ServicesIncluded in Windows 2003 Server
Needs Constant Bit Rate (CBR) encodingOffers random access throughout streamReduces need for local caching (big for mobile devices)Great for movies
![Page 26: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/26.jpg)
Data Rates
We measure rates in power-of-ten numbersK=1000, not 1024M=1000000, not 1048576
StreamingNeed more sustained bandwidth to user more than data rate
ProgressiveA slower connection means user waits longer for playbackBut can use high bitrates for short clips
![Page 27: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/27.jpg)
Frame Size
WPF/E wants square pixel videoSo need to convert to aspect ratioExample
DV is 720x4804:3 content becomes 640:48016:9 content becomes 832x480
Depends somewhat on data rate200-500 Kbps: 320x240 or 432x2401-2 Mbps: 640x480 or 832x240
![Page 28: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/28.jpg)
Encoding modes
1-pass versus 2-pass1-pass reads the source in and encodes out at the same time2-pass does analysis pass first, calculating optimal bit-budget2-pass is always better if you've got the time
And it’s not 2x slower – the analysis pass runs quite a bit faster
CBR versus VBRConstant Bitrate
Optimum for streaming – quality varies, bitrate constant
Variable BitrateOptimum for progressive download – varies bitrate, quality constant
![Page 29: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/29.jpg)
Frame Rates
Use frame rate that’s an integer division of source59.94 to: 59.94, 29.97, 15, 10, 7.5, 6, 550 to: 50, 25, 12.5, 10, 524 to: 24, 12, 8, 6
For anything over 200 Kbps, can use full frame rate of source
![Page 30: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/30.jpg)
Expression Media Encoder
New compression tool targeting .WMV for WPF/EBunded with Expression Media ProductAccessible to both designers and video professionalsOffline and Live Encoding
But there's no secret sauce – you're welcome to use any WMV encoding tool with WPF/E
![Page 31: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/31.jpg)
Using Expression Media Encoder
demo
![Page 32: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/32.jpg)
Advanced Compression Settings
The codec has a bunch of registry key setting to further optimize encodingI fear regedit – you can use “WMV9 PowerToy”
Lots more options than I’ll discuss todayLink to full documentation at end
Tools will be integrating these controls later this year
![Page 33: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/33.jpg)
WMV9 PowerToy
http://citizeninsomniac.com/WMV/#WMV9PowerToy
![Page 34: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/34.jpg)
Number of Threads
Current codec can use up to four threads for encoding
Big speed improvement with multiple cores
Auto picks number of threads availableSees HyperThreading (Pentium 4) as 2 processors
But HT is less of a gain with compression than additional core
Want to have at least 64 pixels high per thread<128 height: 1-thread<256 height (so 320x240): 2-threads256+ height: 4-threads
![Page 35: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/35.jpg)
Motion Vector Search Range
Motion Estimation copies moving elements between frames
Much more efficient to encode “motion vector” than second copy of element
Using a larger Motion Search rangeProvides better efficiency with faster motionSlows encodingCan reduce efficiency with slower motion
![Page 36: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/36.jpg)
MV Range options
Available Options:Macroblock-adaptive128 pixels horizontally, 64 pixels vertically (default)256 pixels horizontally, 128 pixels vertically1024 pixels horizontally, 256 pixels vertically2056 pixels horizontally, 512 pixels vertically
Each progressive option slowest encoding by ~2xMacroblock-adaptive slightly faster than 1024x256
Use Macroblock Adaptive for non-realtime encoding
![Page 37: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/37.jpg)
MV Range – Reference Frame
![Page 38: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/38.jpg)
MV Range – Search Frame
![Page 39: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/39.jpg)
Chroma Search
Finds where color changes but B&W channel doesn’tQuality improvements 5-20% (most with colorful content)
Crowd shots, motion graphics helped in particular
Modes:Macroblock-adaptive with nearest integer chroma (fast, good)Macroblock-adaptive with true chroma (speed/quality mix)Luma only (default)Luma with nearest-integer chromaLuma with true chroma (best, slowest)
![Page 40: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/40.jpg)
B-Frames
Number of bi-directional (B-Frames) between I-Frames and P-FramesB-Frames based off previous and next frameB-Frames typically fewer bits for same qualityNo frames are based on B-frame
Don’t insert I-frame for flashes if they land on a B-frame
So dynamically insert B-frames when there are flashes!Big win for gunfights and discos
1 B-frame optimal for most web encoding
![Page 41: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/41.jpg)
B-Frames Example
![Page 42: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/42.jpg)
Loop Filter
Smooths out hard edge blocking artifacts“In-loop” - used as prediction for future frames
Only blurs content where an artifact would appearMuch less noticeable at higher resolutionsSoftness better if alternative is artifacts
Increased CPU decoder requirements somewhat
![Page 43: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/43.jpg)
Overlap Filter
Softens edges where motion vectors overlapSoftens image somewhatBut reduces artifacts
RecommendationOn if you see artifactsOtherwise Off
Also increases decoder complexity some
![Page 44: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/44.jpg)
Median Filter
Denoises image before calculating motion vectorsKeeps grain/noise from “swirling”Helps low-bitrate encodesBut can leave “trails”
RecommendationOnly use with grainy source, and QA carefully
![Page 45: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/45.jpg)
Denoise filter
Simple noise reduction filterHelps when the source is noisyNot as good as offline noise reducersRecommendation
Good for realtime encoding in low lightBut better to not have noisy source…
![Page 46: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/46.jpg)
Noise Edge Removal
Blanks Line 21, other noise around edge of frameHandy for real-time encoding of analog sourcesOnly use if source has edge noise
![Page 47: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/47.jpg)
Lookahead
Specifies up to 16 frames forward for analysisOnly needed in 1-pass encoding
Used to detect scene changes, flash framesBig improvement for 1-pass CBR encodesCan dynamically insert B-frames with flashes
Default is Off (Does not look ahead)Adds to latency for live encoding
~½ second for Lookahead=15 @ 29.97 fps
Slightly slows encoding as Lookahead increases
![Page 48: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/48.jpg)
DQuant
“Differential Quantization”Different macroblocks are compressed different amounts
Compress gradients less, to reduce banding
RecommendationI and P frames
Using on B-frames wastes bits that will be discarded
Lots of levels available“Regular” good default with most content
![Page 49: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/49.jpg)
Perceptual Option
Like DQuant, optimizes bits based on imageUsed in addition to DQuantAdaptive Dead Zone 1 good default for film content
Experiment carefully with other kinds of sources
![Page 50: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/50.jpg)
Optimum Settings: On demand
Highest quality encoding and reasonable performance
Up to 4x slower than default settings
Encoder complexity = 4/80 (second slowest)Motion Search Level = macroblock adaptive true chromaMotion Search Range = macroblock adaptiveMotion Match Method = Adaptive SAD/HadamardDQuant = RegularPerceptual Option = Adaptive Dead Zone 1 for filmForce LoopFilter = OnNumBFrames = 1At least 96 pixels high per thread (so 2 max for 320x240)
![Page 51: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/51.jpg)
Optimum Settings: Webcasting
Optimized for reliable real-time encodingRemember dual-duals are almost 4x faster than single core
Motion Search Range = Biggest that doesn’t drop framesMotion Search Level = Luma + integer chromaNumBFrames = 1Force LoopFilter = 1 (on) Dquant Option = RegularPerceptual Option = Adaptive Dead Zone 1 for filmLookahead = 16 (Max – most important single option)
![Page 52: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/52.jpg)
Designing Windows Applications with Expression Blend
2-day instructor-led training in over 20 US cities
Tools and Technologies UsedExpression Blend and Expression Design
NET Framework 3.0, Windows Presentation Foundation, XAML
Visual Studio 2005
Target AudienceUser interface designers with advanced experience with Flash or Director. Understanding of Visual Studio 2005 or programming skills is not required.
PrerequisitesDoes not require prior experience using Expression Blend or Windows Presentation Foundation.
We recommend:
Building an understanding of .NET Framework 3.0
Download Expression Blend http://www.microsoft.com/expression
Get inspired! www.seewindowsvista.com
Learn More: http://tinyurl.com/3y6jg7
![Page 53: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/53.jpg)
Q&A and Contact
My blog:http://on10.net/blogs/benwagg/
My class at Stanfordwww.digitalmediaacademy.org/courses/video-compression-training.html
![Page 54: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/54.jpg)
Please fill out your eval
evaluation
![Page 55: Design in Motion: Video Production Workflow](https://reader030.vdocuments.net/reader030/viewer/2022032504/55c4aaccbb61ebc7488b46b9/html5/thumbnails/55.jpg)
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions,
it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.