Building a Shared Pipeline for Content DistributionAWS Media and Entertainment Cloud Symposium

23rd March 2016

Jim Taylor & Sam Bashton

Who are we? - ITV/Jim

● Jim Taylor - Technical Product Owner, VOD Distribution● jim.taylor@itv.com

ITV is the biggest commercial television network in the UK. We love telly, we’re passionate about content and most importantly we understand our audiences.

As a growing international company, we’re commercially minded and have a global focus. We’re all about pushing the boundaries and being innovative.

Who are we? - Sam Bashton

VOD Distribution - Push model

ITV

Partner 1Push content

Partner 2Push content

Scale issues with:● Bandwidth● Processing● Unique delivery methods● Redelivery requests

...but same content to each partner!

Partner 3

...

Business problem

How can we continuously distribute large volumes of high quality video to several destinations, without spiralling distribution costs?

● Partners need high quality mezzanine so they can transcode to multi-platform● ITV has about 200 hours/week of Catch Up ≈ 6TB/week● 1Gbps link per partner = high cost/long lead time● Push model unsustainable as distribution points increase

Solution - Cloud delivery - pull model

Use AWS to deliver once using a templated delivery and offer partners a fixed menu via a pull model

ITV AWSITV

Partner 1 AWS

Partner 1

Upload once

Downloadcontent

Notifications

Notifications

Partner 2

Partner 2 AWS

Notifications

Downloadcontent

Notifications

Solution - Package

Shared:Series/Episode Images 16:9 & 4:3HD (XDCam50) or SD (IMX30) asset

Private:Metadata file containing● Tech metadata, asset locations● Licence details● Editorial metadata

Technology used

ITV AWS

ITV

Partner AWS

Partner

Signiant Flight

SNS

Amazon S3AWS CloudFormationAmazon LambdaELK stack on EC2

Splunk

Partnership - ITV & Bashton

Partner Setup: AWS

● Each partner has their own AWS account● Files in the bucket use ‘requester pays’ billing - partners pay for data transferred

outside of AWS● Partners notified via SNS when new content is available

Partner setup: SNS

● ‘Simple Notification Service’● Push notifications

○ Webhooks○ SQS (Simple Queue Service)○ Lambda

Lambda

● Code without a server*● Trigger simple tasks● Used as part of shared pipeline

○ Receives notifications when videos are uploaded○ Looks up partners, sends SNS notifications to those that are relevant

Test environments

● Creating resources via web console fine for trying things out● Proof of concept environment created in this way● In production, we need to build multiple copies of the environment● Changing everything manually quickly becomes tiresome

CloudFormation

● CloudFormation to the rescue!● Infrastructure as Code● Specify what an environment looks like in a JSON text file

○ Networks, subnets○ Firewall rules○ S3 Buckets○ Instances, volumes○ Access control

● Commit to source control (GIT)

CloudFormation: Rich documentation

● Every change has a commit message● History of every setting can be viewed back to the beginning of the project

○ With comments about every change to say why they were made

CloudFormation: Testing

● Change a template● Apply to test environment● Run the same change against staging, production● Confidence that every change will work, and that all tests are valid

● Lambda - code without servers*● Code still needs to be tested● For software deployed to instances we use Jenkins for CI

○ Automated Testing○ Deployment and ‘promotion’ between environments (test->stage->prod)

● What to use with Lambda?

Software Testing

● Jenkins● Code committed to git repo● Jenkins checks out code from git● Runs automated tests● Deploys to environment● Deploys to ‘next’ environment (test->stage) at the click of a button

Jenkins + Lambda

Future plans

ITV AWSITV

Partner 1 AWS

Partner 1

Upload once

Downloadcontent

Notifications/metadata

Notifications

Partner 2

Partner 2 AWS

Notifications

Downloadcontent

Notifications/metadata

Metadata API

Added ourselves as a Shared Pipeline partner!

Add access services

Cloud transcoding & stitching

● Deliver a ‘reference’ test system for partners - easier to look at a working example than documentation● Issues using “requester pays” in egress - make this clearer● Avoid logging overload

What would we do differently?

● Delivery time of project - 5 months● Reduced time to deliver a new partner from 3-6 months to 1-2 months

○ ...providing they use our system● Reduced cost to deliver a new partner by up to 75%

○ ...providing they use our system● Can start testing within a week

Success? - Yes!