December 3, 2013 Sam Siewert

CS A490 Digital Media and Interactive

Systems

Lecture 13 – Final Exam Review

Major Concepts Real-Time Services (Best Effort, Hard, Soft and Isochronal Real-Time) – Difference Between Hard and Soft Real-Time

POSIX Threading, NPTL – FIFO Scheduler and Priority Pre-emptive Run-to-Completion – Create, Detach, and Join

Static Vs. Dynamic Priority (RM Vs. EDF/LLF) – Harmonics – Advantages of Dynamic for General Utility

RT Implementation Challenges – Shared Resources – Blocking – Inversion, Deadlock, Livelock

Sam Siewert 2

Major Concepts Audio and Video Encoding

– Audio PCM Sampling and Encoding – Video Encoding (Lossy and Lossless)

Color Spaces, Models, and Pixel Encoding Frame Sequence

Elementary, Program, and Transport Streams MPEG Encoding Order of Operations

– RGB->YUV (Lossy) – Block Level DCT (8x8) – Quantization (Lossy) – RLE (Run Length Encoding) and Huffman Encoding – Macro Block GoP (Group of Pictures), I-frame, B-frame, P-frame

MPEG PS/TS Packets (188 Byte) and Multi-plexing Broadband (RF Cable/OTA) and Baseband Carriers Sam Siewert 3

History of Interactive Systems Command Line Interfaces (CLI), Still in Use Today – ASCII and EBCDIC Terminals – Canonical Mode (Buffer until Return) – Non-Canonical Mode (Capture and Transmit

Each Character with Echo or No-Echo, e.g. for password)

Mouse and Bit-mapped Window Manager (GUI) – Xerox PARC (Palo Alto Research Center) –

1970, – Mac OS with Toolbox (1979) – Jef Raskin – X-Windows (MIT) and Windows 3.x, NT,

Win7, Win8 (Microsoft)

Sam Siewert 4

http://en.wikipedia.org/wiki/Jef_Raskin http://en.wikipedia.org/wiki/Apple_Macintosh

http://en.wikipedia.org/wiki/X_Window_System

http://en.wikipedia.org/wiki/PARC_(company)

Concepts

Long Tail Digital Media – Pareto Rule – 80% of Consumers Consume 20% of Available Content

(Head) – Remaining 20% of Consumers Consume Remaining 80% of Unique

Content Selections – Common Wisdom: Invest Most in Head (20%) of Content – Broadcast Follows Pareto Rule – On-Demand Has Shown Increased Economic Value in the Tail and Very

Large Content Libraries

On-Demand, Time-Shifted (E.g. DVR), Location-Shifted (E.g. Mobile or Web), and Narrowcasting (Very Specific Markets) Steganography (How it works in concept)

Sam Siewert 5

Concepts

Digital Rights Management – Encryption of Digital Media in Transport – Decryption Only by Authenticated End-Users – Key Management and Conditional Access – Triple DES (Data Encryption Standard) – Rijndael (“Rhine Doll”) AES-128/256 CBC (Cyphper Block Chaining)

Content Protection, Revenue Protection (Digital Media Forensics) – Watermarking – Original Master Source (Studio, Head-end) – Fingerprinting – Addition of Client Information (Set-Top Box, Network) – Watermarks/Fingerprints – Hidden in Encoding, Must Not Be Eliminated by

Encoding Digital Media Rarely Distributed in Resolution Anywhere Near Original Masters – E.g. Digital Cinema – 2K, 4K, 6K Resolution – HD – 1080p MPEG-2 or MPEG-4 Lossy Encoding

Sam Siewert 6

NVCC Compiles CUDA Code, GCC Compiles C Code

Linked Together for Hybrid SPMD Executable

Sam Siewert 7

Host/Device SPMD

Sections of Serial and Parallel Co-Proc Execution

Sam Siewert 8

Hiding Latency – Dedicated Write-Back I/O

Schedule Reads Ahead of Processing

Requires Nthreads = 2 + Ncores

Synchronize Frame Ready/Write-backs Balance Stage Read/Write-Back Latency to Processing 1.5 to 2x Threads for SMT (Hyper-threading)

Sam Siewert 9

Wait Process F1 Process F3 Process F5 …

Wait Process F2 Process F4 Process F6

Read F1 Read F2 Read F3 Read F4 Read F5 Read F6 Read F7 Read F8

Start-up

Wait … WB F1 WB F2 WB F3 WB F4 WB F5 WB F6

Dual-Core Concurrent Processing Completion

Top 10 Concepts 1. Flynn’s Taxonomy for Processor Scaling

– MIMD – Multi-Core, Multi-Thread (Native POSIX Threading) – SIMD – Vector Processing Instructions (E.g. Many Word Multiply and Accumulate) – SPMD – Single Program, Multiple Data Streams (Cell BBE and GPU, GP-GPU)

2. Real-Time Services (Hard Vs. Soft)

3. Dynamic Vs. Static Thread Dispatch and Execution Priority

4. MPEG Transport with 188 Byte Packets Over Internet Encapsulation, Audio ES, Video ES, and

Program Specific Information (PSI) are Multi-plexed (e.g. required for Lip-sync)

5. Discrete Cosine Transform for Intraframe (Macroblock) Compression in Frequency Domain

6. Digital Video Encoding Order of Operators (Pixel, Macroblock, Frame to Frame)

7. Lossy Vs. Lossless Video Encoding Steps

8. Digital Rights Management and Digital Forensics

9. Digital Media Delivery – Broadband (E.g. OTA), Baseband (E.g. Internet), Broadcast, On-Demand, Multi-cast, Narrowcast, Time-Shift, Location-Shift

10. Parallel Processing Speed-up (Amdahl’s Law)

Sam Siewert 10