MediaGrid Processing Framework

2009 February 19

Jason Danielson

Confidential

2

Agenda

MediaGrid Processing Framework♦ Objectives♦ How does it work?♦ Applicable Applications♦ Results♦ Benefits

Confidential

Objectives of MediaGrid Processing

Faster media processing for high-demand environments♦ CPU/Bandwidth/Storage intensive processes such as

transcoding, quality assurance, rendering, watermarking, etc.

Streamlined workflow practices♦ No need to move media from storage, process it and then move it

back – can all be achieved in place in the active storage

Simplified systems infrastructure through integrated system♦ No need for discreet storage and processing farms

3

Confidential

MediaGrid System Components

ContentServers♦ Basic storage unit containing disk drives and Ethernet ports ♦ Three configurations optimized for bandwidth, performance, capacity

ContentDirectors♦ Present a single virtual file system♦ Provide clients with pointers to files♦ 2 or more for redundancy

Ethernet switching infrastructure♦ Simplifies networking infrastructure

Confidential

5

MediaGrid Processing - How does it work?

Application Controller•Application server that connects to MediaGrid

•Establishes environment on ContentServer for application process

•Uses MediaGrid API for communication

Grid Resource Manager•Software running on ContentDirectors

•Manages grid processing on MediaGrid

•Responds to requests from Application Controller

Application Process•Linux process in ContentServer that executes the application job

Grid Application Loader•Software running on ContentServers

•Sets up application process and launches applications

•Responds to requests from Application Controller

Sub-Grid•Subset of ContentServers used by a given application

Confidential

6

MediaGrid Processing - How does it work?

Application Server requests pool of resources from the Grid Resource Manager which assigns available resources to the job

Application server launches processes on the assigned ContentServers & is notified when the job is complete

Grid Resource Manager handles errors such as failed ContentServers by reassigning new ContentServers to a job and informing the Application Server

Confidential

Leverage the MediaGrid API

MediaGrid File System Interface♦ Session and buffer management♦ File operations e.g. read, write

MediaGrid Application Processing Interface♦ Sub Grid management♦ Application Load♦ Application execution and management

7

Confidential

Applications Criteria

Applications can be easily ported, if:♦ Based on Linux kernel version 2.6 or later♦ Distributed architecture on 32-bit x86 platform♦ Application has 2 basic parts:

▪ Application Controller (AC)▪ Application Process (AP)

Applications that can parcel out jobs can take advantage of many CPUs acting in parallel on single clips♦ But no need to parallel process to gain benefit of multiple CPUs

8

Confidential

Transcode Performance

Confidential

10

Benefits of Processing and Storing within the same Clustered File System

Use available processing power of the active storage to process stored content

Active storage can be balanced for performance, bandwidth, and capacity requirements

Performance scales linearly as the active storage system scales

Minimal to no traffic on house network

Reduces islands of storage

Thank You