multi cloud strategien mit cloud management …...» microservices, vms » media workflow...

Ulrich VoigtHead of Design

[email protected]

MULTI‐CLOUD STRATEGIEN MIT CLOUD MANAGEMENT PLATTFORMEN

INTEGRATED SYSTEMS IN THE CLOUD

INTEGRATED SYSTEMS IN THE CLOUD

The future for broadcast systems will be the integration of best‐of‐breed components in the cloud. But this requires new methods, tools and skills.

Evolution of the „platform“ of broadcast systems

IBC SHOWCASE – LIVE EXAMPLE

INCREASING GRANULARITY

Single Unit Coarse‐grained Fine‐grained

Monolithic SOA Microservices

THE DEATH STAR

https://www.appcentrica.com/the‐rise‐of‐microservices/

AlphaGo

DIGITAL TRANSFORMATION

Rise of the internet

Mobile Internet (UMTS)

2002

Facebook

Youtube

Smart Phone

Netflix Streaming (US only)

WhatsApp

Instagram

Oculus Rift Prototype

2004 2005 2007 2009 2010 2011

Siri

Snap

Smart Home (partly earlier)

2014

Smart speaker

2015 20181950s 1970s 1990s

Rise of the Television

Colour TV broadly adopted

HDTV introduced

DVB‐T / Analogue Switch off

DTV rollout worldwide DVB‐T2

Raise of IoT

2000s 2003

File‐basedworkflows

Feature Phones

UHDSDIover IP

STOP THE TIDE!?!

TECHNOLOGY ADOPTION

Adop

tion

Time

Ignore Ignore Ignore “No“ “No“I said“No“

dammit“Oh No“ “Oh %*&!“

Enterprise IT

Rest of World

By Simon Wardley http://enterpriseitadoption.com

TECHNOLOGY ADOPTION

Adop

tion

Time

Ignore Ignore Ignore “No“ “No“I said“No“

dammit“Oh No“ “Oh %*&!“

By Simon Wardley http://enterpriseitadoption.com

Enterprise IT

Rest of World

In our industry we have to develop the tools for solution design and system integration with cloud‐

native technology.

SYSTEM INTEGRATION TODAY

SYSTEM INTEGRATION TOMORROW

SYSTEM INTEGRATION TOMORROW

ST 2110

„IT RUNS IN THE CLOUD“ – CAN MEAN...

» Software You can install it in any virtualized environment You can install it in a very specific virtualized environment It is delivered as a VM image (or similar) It is delivered as an Amazon AMI image (or similar) It can be packed as a (Docker) container It is delivered as (Docker) container (for Swarm, Kubernetes, ...) You can select it in the marketplace of a cloud provider and it installs itself then

» Software as a Service, SaaS You can book it in the marketplace of a cloud provider and it will then be operated for you You can book it with the manufacturer and it will then be operated for you It uses native functions of a cloud (its DB, its serverless functions, etc.)

SYSTEM INTEGRATION IN THE CLOUD

With which cloud providers do the individual components run? What is operated for me (aaS), what do I have to operate myself? What are the individual service levels? How is billing done? And of course data protection...

So you need answers for Where are my essences? (today: Central Storage) How to build hybrid systems (on‐premise, cloud) How do I make the interfaces and workflow integration? (today: ESB/Orchestrator) How do I do my monitoring? What can I monitor with aaS components? (today: Nagios & Co) How do I do a central user administration? (today: Active Directory) How do I do my scaling? (today: Power button) How much is this going to cost me? (today: purchase plus maintenance contract)

Contribution / Ingest Central Systems Transmission / Playout

Central Ingest (CMX) Orchestration Compliance Rec. Linear Playout

News Ingest (NMX) Asset Management Inhouse IPTV OTT

Radio Ingest Quality Check KVM OOH

Upload Platform Transcoding AV‐Core Infra. Delivery external / web

Subtitle Creation Storage IT‐Core Infra. In‐/Outgest

File Delivery Archive HSM BIT Network Various AV‐Glue

EXAMPLE MEDIA SYSTEM LANDSCAPE

News & TV Studios News Production Post Production Radio Production

Vision Mixers, Studio Cameras Newsroom System Central Post Production Radio News Prod.

Audiomixing & BCS News PAM Audio Post Pro. Music Stations

Various AV‐Glue Graphics & Studio Automation Censorship, Editing Audio Craft Editing

CR Furniture News Editing Color Grading Traffic Watch

Contribution / Ingest Central Systems Transmission / Playout

Central Ingest (CMX) Orchestration Compliance Rec. Linear Playout

News Ingest (NMX) Asset Management Inhouse IPTV OTT

Radio Ingest Quality Check KVM OOH

Upload Platform Transcoding AV‐Core Infra. Delivery external / web

Subtitle Creation Storage IT‐Core Infra. In‐/Outgest

File Delivery Archive HSM BIT Network Various AV‐Glue

News & TV Studios News Production Post Production Radio Production

Vision Mixers, Studio Cameras Newsroom System Central Post Production Radio News Prod.

Audiomixing & BCS News PAM Audio Post Pro. Music Stations

Various AV‐Glue Graphics & Studio Automation Censorship, Editing Audio Craft Editing

CR Furniture News Editing Color Grading Traffic Watch

EXAMPLE MEDIA SYSTEM LANDSCAPE

AWSAWS

IBM Cloud(own/SaaS)IBM Cloud(own/SaaS)

TelestreamCloud(SaaS)

TelestreamCloud(SaaS)

Diva Cloud (Private Cloud)Diva Cloud

(Private Cloud)

iTX Flex(SaaS in Azure)

iTX Flex(SaaS in Azure)

Adobe PR (any)

Adobe PR (any)

Avid(Azure/SaaS)

Avid(Azure/SaaS)

CLOUD INTEGRATION DIFFERS FROM LEGACY INTEGRATION

» Platform provider» Operational responsibility

(self‐managed … *aaS)» Service level» Billing» Application integration» Storage model

(central … distributed)» Monitoring» User management» Security» Scaling method (auto?)» Cost insight» Lift&shift or cloud‐native» …

CLOUD MANAGEMENT PLATFORMS

CLOUD MANAGEMENT PLATTFORMEN (CMP)

“CMPs provide a means for a cloud service customer to manage the deployment and operation of applications and associated datasets across multiple cloud service infrastructures, including both on‐premises cloud infrastructure and public cloud service provider infrastructure. In other words, CMPs provide management capabilities for hybrid cloud environments.”

2017, Cloud Standards Customer Council

» Gartner defines the minimum requirements for CMP offerings as: “…products that incorporate self‐service interfaces, provision system images, enable metering and billing, and provide for some degree of workload optimization through established policies.”

» IT analysts say an average business uses a combination of five or six different cloud environments, typically a mix of private local and public clouds.

» Hybrid cloud environments are expected to have an average annual growth rate of 34.3% over the 2016‐2022 period.

CLOUD MANAGEMENT PLATTFORMEN (CMP)

http://www.gartner.com/it‐glossary/cloud‐management‐platforms | http://assets.rightscale.com/uploads/pdfs/RightScale‐2017‐State‐of‐the‐Cloud‐Report.pdf | https://www.infoholicresearch.com/worldwide‐hybrid‐cloud‐computing‐market‐to‐grow‐at‐a‐cagr‐of‐34‐3‐during‐the‐period‐2016‐2022‐to‐aggregate‐241‐13‐billion‐by‐2022/

REFERENCE ARCHITECTURE FOR HYBRID CLOUD MANAGEMENT

2017, Cloud

Stand

ards Customer Cou

ncil

CLOUD MANAGEMENT PLATTFORM FOR MEDIA

» Cloud Infrastructure Automation» Multi‐Cloud, on‐premise, hybrid» Microservices, VMs

» Media Workflow Orchestration» API connections, define application

integration processes, file transfers, storage » User Management

» SSO, SAML, Oauth, AD» Monitoring & Measurement

» Dashboards, process‐related metrics, logs» Cost Control

» Transparency, process‐related, budgets, timely

» Security» Certificate Management, Vulnerability

Scanning, …

QVEST.CLOUD

DEMO: „BEHIND THE SCENES“ CLOUD INFRASTRUCTURE AUTOMATION

EINE WICHTIGE KOMPONENTE DABEI

VIRTUAL MACHINES AND CONTAINER

App A App B

Bins/Libs Bins/Libs

Guest OS Guest OS

Hypervisor

Host OS

Server

Host OS

Server

Docker Engine

App A App B

Bins/Libs Bins/Libs

WE ARE GOING TO DEPLOY A PLAYOUT AUTOMATION

» Containerized application» 7 microservices» In AWS (would work the same for

other public clouds or on‐prem Vsphere)

AutomationDB

AutomationAPI

Metadata/ Mediainfo

Transcoder

Filetransfer

Transcoder Queue DB

Import Workflow

WHAT HAPPENED SO FAR: PROVIDE IMAGE INTO REGISTRY

Image Publishing

Git RepoSource Code

CIBuild & Test

Pipelines

Image RegistryReplicated ImagesVulnerability Scanner

Container Singnage

Image RegistrySigned Images

New Commit/Release checked in

Automated Tests & Image Build Processes

Pushing or pulling Image

Image Backup & Replication

PushingImage Deployment

1

2

3

6

4

5 7

@Site@Provider

App A

WE ARE THEN GOING TO DEPLOY THE SOLUTION

Image RegistryReplicated Images

@Site

Catalog

Deployment

Container Orchestration

Virtual Network (Service Mesh)

App A

App B

App C

Solution

App B

App AApp A

1

3

2

4

Solution gets composed from multiple apps according to version, environment, parameters

User selects Solution from Catalog

Images are pulled from registry and deployed in selected configuration

Deployment includes creation of virtual network and storages

ProductionIntegrationTest

IN REAL LIFE THIS WOULD GO TO MULTIPLE ENVIRONMENTS


@Site

Catalog

Deployment


1

3

2

5




From identical images, solution can be deployed to different system environments


App A

App B

App C

Solution

App B

App AApp A

4Deployment includes creation of virtual network and storages


THE DEPLOYED APPS WILL CONNECT TO LOGS AND METRICS


@Site

Catalog

Deployment


Load

Bal

ance

r

Das

hboa

rd/

Mon

itorin

gLo

gfile

M

anag

emen

tA

PI

Met

rics

Col

lect

ion

Log

Col

lect

ion

1

3

2

5




From identical images, solution can be deployed to different system environments


App A

App B

App C

Solution

App B

App AApp A

4Deployment includes creation of virtual network and storages


SOLUTION SCALING


@Site


Load

Bal

ance

r

Das

hboa

rd/

Mon

itorin

gLo

gfile

M

anag

emen

tA

PI

Met

rics

Col

lect

ion

Log

Col

lect

ion


App A

App B

App C

Solution

App B

App AApp AApp CApp CApp C

Scaling Rules

1The scaling engine manages the policies for auto-scaling

Apps instances scale up and down according to policies, load, time etc.

2Apps instances scale up and down according to policies, load, time etc.


USING A CMP THE PLACEMENT CAN BE MULTI‐CLOUD


@Site

Scaling Rules



App A

App B

App C

Solution

App B

App AApp A

Load

Bal

ance

r

Das

hboa

rd/

Mon

itorin

gLo

gfile

M

anag

emen

tA

PI

Met

rics

Col

lect

ion

Log

Col

lect

ion


HYBRID MEDIA WORKFLOW


@Site

Scaling Rules



App A App C

Solution

App AApp A

Load

Bal

ance

r

Das

hboa

rd/

Mon

itorin

gLo

gfile

M

anag

emen

tA

PI

Met

rics

Col

lect

ion

Log

Col

lect

ion

App X

App BApp B

Media Workflow

Orchestration

BENEFITS OF AUTOMATED DEPLOYMENT

» Separation of configuration and environment/version: Simple deployment in production, reproduction of errors, rollback

» "Infrastructure as Code"» Detection of failures/errors, automated restart» Granular scaling» Reduction of error propagation» Initial rollout only to a small part of users» Integrated monitoring and log file management » Significantly simplified operation/support» Short release cycles

QVEST.CLOUD: CLOUD MANAGEMENT PLATFORM FOR MEDIA

One‐stop cloud management platform for media and broadcast

» Integrates on‐prem, cloud and hybrid» Integrates new and existing systems» Centralises management » Enriches existing systems» Consolidates contracts» Application independent

Radically simplifies the cloud, hybrid and on‐prem integration and management of best‐of‐breed solutions

THANK YOU FOR YOUR ATTENTION

multi cloud strategien mit cloud management …...» microservices, vms » media workflow...

Documents