optimization and efficient utilization of adaptive …...over the top (ott) video to multiscreen...

OPTIMIZATIONANDEFFICIENTUTILIZATIONOFADAPTIVEBITRATE(ABR)VIDEOUSINGBIGDATAANDPREDICTIVEANALYTICS

SRIDHARKUNISETTYJEFFREYTYREROBERTMYERS

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 2

TABLEOFCONTENTSINTRODUCTION.............................................................................................3IPVIDEOANDABRENABLINGSOLUTIONS....................................................3ABRinIPVideoNetworks...............................................................................................3

UnifiedMultiscreenAnalyticsandOperationalDashboards..........................................5

C-ABR(CloudAssistedABR)............................................................................................6

M-ABR(MulticastAssistedABR).....................................................................................7

BIGDATA&PREDICTIVEANALYTICS..............................................................9Overview........................................................................................................................9

ApplicabilitytoABREnablingSolutions........................................................................10

IdentifyingandAddressingOperationalIssues............................................................12

UsingPredictiveAnalyticsinM-ABR............................................................................14

AugmentingDecisionMakinginC-ABR........................................................................18

CONCLUSION...............................................................................................19ABBREVIATIONS...........................................................................................20RELATEDREADINGS.....................................................................................21REFERENCES................................................................................................22

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 3

INTRODUCTIONAdaptiveBitrate(ABR)protocolshavebecomethede-factotechnologyfordeliveringOvertheTop(OTT)videotomultiscreendevicessuchassmartphones,tablets,etc.ABRstreamingisalsousedbyoperatorstodeliverlinearcontentoverIPtothehome’ssecond-screendevices,andit’smakingitswaytotheprimaryTVscreenintheformofnextgenerationIPset-topboxes(STBs)andCustomerOwnedandMaintained(COAM)devices.WhiletheintentistoenableallTVsubscriberdevicestobeABR-based,withthatcomestheneedfornewreal-timedataprocessinganddatarepositoryplatformsthatcanstoreandprocessnetworkanalyticsandviewershipdatafrommillionsoftheseHTTP-baseddevices.

Actionableknowledgederivedfromanalyzingthevastamountofdataofhomedevicesprovidesvaluableinformationthatnotonlyallowsustooptimizethenetworkbyaddressingtheoperationalissues,butalsoenablessavingvaluablenetworkbandwidthbypredictingthebestusageofresources.HarnessingBigDatatechnologiesinreal-timeenablesafeedbackloop,usingthingssuchasmulticastchannelmaps,thatcanbechangeddynamicallyatagranularlevel–ataspecifictimeofthedayinaspecificgeographicalarea–tobetterutilizethenetworkbandwidthandreduceoperationalcomplexity.Inaddition,historicaldataanalysisusedinconjunctioninEPGmetadatacanbeusedtopredictseveralimportantfactorssuchasusersurfingpatterns,channelpopularity,andchannelchangeprobabilityatgranularlevel.Thisknowledgecanbeappliedinreal-timetooptimizeABRdeliveryanddirectlyimpactoperatornetworkbandwidthsavingswhilemaintainingsubscriberQualityofExperience(QoE).

ThispaperprovidesanoverviewintohowharnessingBigDataenablesinsightintouserbehaviorandcorrespondingvideoconsumption,andhowPredictiveAnalytics,amongotherthings,enablesidentificationofthebestresourcesforABRvideodeliverytooptimizeoperatornetworkbandwidthusage.

RecentresearcheffortsintooptimizingIPVideodeliveryusingABRledtopromisingsolutionssuchasM-ABR(MulticastassistedABR)andC-ABR(CloudassistedABR).TheideasdiscussedinthispaperaredirectlyapplicableintheoptimizationandeffectiveutilizationofM-ABRandC-ABR.

IPVIDEOANDABRENABLINGSOLUTIONSABRinIPVideoNetworksSincetheearlydeploymentofIP-basedvideonetworks,varioustechnologieshaveemergedtohelpcopewiththevariabilityassociatedwithdeliveringvideoovernon-

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 4

deterministic,besteffortnetworking.HTTPprogressivedownloadhashistoricallybeenapopularmeansofvideodeliveryovertheInternet.Yetinmanagedenvironments,PayTVserviceoperatorshavetraditionallyusedMPEG-2TSasthetransportmechanismforvideooverIPnetworks.

Today,however,anewertechnologycalledadaptivestreaminghasemerged.Adaptivestreamingpromisestoenablevideostobedeliveredoverunmanagednetworkswithaveryhighqualityofexperience,andisthusapplicabletobothInternetvideoenvironmentsandmanagedvideonetworksthatareseekingtoextendthedeliveryofpremiumcontenttodevicesotherthanthetelevisionset.HTTPadaptivestreaminghasproventobethetechnologyofchoiceformanytypesofvideodelivery—bothoverthepublicInternetandwhathastraditionallybeenmanagedvideonetworkenvironments.

Unlikepreviousstreamingtechnologiessuchasprogressivedownload,adaptivestreamingintroducestheabilitytodynamicallyreacttochangesinnetworkconditionsbyswitchingtoavideoencodedatadifferentbitrate.Thisabilitytoadaptinrealtimemoreaccuratelyreflectsthedynamicconditionsoftoday’snetworks,content,anddevices.Withusersstreamingmorepremiumlongformcontent,itisnaturaltoexpectthattherewillbefluctuationsintheamountofbandwidthavailableduringatwo-hourmovie,forexample.AdaptivestreamingisrecognitionofthisfactandenablesviewerstowatchthispremiumcontentwithasuperiorQoE.

Adaptivestreamingworksbyleveragingthesamecontentencodedinvariousbitrates—inarangethatreflectstheexpectedqualityofthecontentitself,thenetworkperformance,andthescreenresolutiondesired.Forexample,avideocouldbeencodedinbitratesrangingfrom300Kbps(YouTube-qualityonlinevideo)upto6Mbpsorhigher(highqualitystreamingcontenttotheTV).Atypicalvideoisencodedinasmanyaseightdifferentprofiles,dependingontherangeofdevicesandtheirdisplaysizesdesiredtobesupported.Eachofthesefilesarethenfurthersegmentedor“chunked”intoshortsegments(typicallytwosecondslong)thatareeachpreciselytime-stamped.

Asthevideoisdelivered,theHTTPclientmaintainsacommunicationchannelwiththeadaptivebitrateserver.Theclientdownloadsthesechunksasindividualfiles,whicharebufferedbytheclient,decoded,andplayedoutasacontinuouspresentationofvideoandaudiototheviewer.Duringtheviewingsession,theclientplayermonitorstherateatwhichthebufferisfillingandcantherebyinfertheperformanceofthenetwork.

Ifthereisdegradationinnetworkperformance,theclientcanrequestthatchunksbedeliveredfromoneofthelowerbitratefiles.Thisisallseamlesstotheviewersinceeachsourcefileischunkedandtime-stampedinthesame,verypreciseintervals–sothereisnovisibleinterruptionorhesitationwhenswitchingtoadifferentbitrate.Likewise,iftheplayerdetectsanimprovementinperformance,itcanrequestHTTPfilesegmentsfromoneofthehigherbitrates.

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 5

AdaptiveBitrate(ABR)protocolshavebecomethemainstayofmultiscreendevicesliketablets,smartphones,gamingdevices,andsmartTVsforaccessingOTTvideocontent.Becauseoftheirexplosivepopularity,itishighlydesirableforanoperatortoprovideexistingservicestotheseIPVideo,multiscreendevices.TheABRprotocolshavebeenoptimizedtodeliverawiderangeofbitratesforvaryingscreensizesfromthumbnailsto4Kultra-HD,operatingoverIPNetworkconnectionswithfluctuatingbandwidthandservicelevels.

Figure1–ABREcosysteminIPVideoNetworks

AdaptiveBitratestreamingisalreadycommonlyusedtodaybypayTVoperatorstodelivervideocontentoverIPtothehometosecond-screensubscriberdevicessuchasPCs,laptops,tablets,andmobilephones.ABRstreamingisnowmakingitswaytotheprimaryTVscreeninthehomeintheformofnextgenerationIPSTBs.WeexpectthatpayTVoperatorswillcontinueandacceleratetheirmigrationstrategiestoHTTP,ABR,andotherIPVideotechnologiesintermsofmultiscreenstrategiesthatcanunifythefullrangeofsubscriberexperienceandbehavior.

UnifiedMultiscreenAnalyticsandOperationalDashboardsOperational,predictive,andmonetizationaspectsareallsupportedusingaBigDataAnalyticsplatform.

Understandingtheoperationalissuesbecomesessential,especiallyfornewtechnologies,sothatwecanaddressthoseissuestoimproveefficiency,reducecost,

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 6

andprovidebetteruserexperience.Actionableinsightderivedfromcollecting,processing,andanalyzingthehugeamountofdatafromthemillionsofhomegatewaysprovidesseverallevelsofvaluableinformationthatallowsustobettermanagethenetworkbyunderstandingtheoperationalissues.Theprocesseddataisexposedasdashboardsandchartstoprovideavisualviewintothedifferentoperationalandusageaspectsofthesystem.

Forexample,wecancaptureclientdatastatisticsacrossallscreensinthehome,suchasvideostarttime,videostartfailure,exitsbeforevideostarts,numberofsuccessfulplaybacks,andclientbufferingorstallingoccurrenceshelps.ThiscanbeusedtoextractsubscriberviewershippatternsandtoleranceforQualityofService(QoS)levelsandcorrectiveactionscanbetakenbyoperatorstoimproveoverallsubscriberQoE.

Byunifyingdatacollectionandprocessing,insteadofdoingitinsilos,fromallcomponents–fromtheheadendtotheendclients-wecannotonlyprovideaunifiedoperationaldashboardthatallowscorrelationswithdifferentaspectsofthesystemacrosstheboard.

C-ABR(CloudAssistedABR)Akeyprobleminserviceoperator’stransitiontoleveragingIPvideonetworkingisthatABRclientsbydesignactindependently,tryingtousethemaximumnetworkbandwidthavailable.InconventionalABRvideodelivery,theABRclientdeterminesthebitratedecisionsbasedonitsowninterpretationofnetworkconditions.Eachclientmanagesitsownrequirementsformaintainingvideo/audioplayoutwithnoinsightintosystem-widerequirements,leadingtoademonstrationofmanyinappropriatebehaviorsattheindividualclientlevelintermsofinconsistentcontentpresentationtotheviewer.TheABRclient’s“greedy”behaviorleadstosignificantunfairness,instability,andinefficienciesrelativetothelargerpopulationofABRclients[2].TheseABRclienttraitsareoftennotinlinewithanoperatorlookingtoofferatrue“managed”ABRvideoservicewiththeassociatedQoE.

Byaddingsomecloudbaseddecisionmakingintothesolution(asopposedtoisolated,client-basedcontrolinthetypicalIPvideodeploymentusingABRstreaming),theoperatorcanregaincontroltoprovideafirstratevideoservicewithbetterQoEwhileretainingthekeyunderlyingbenefitsofadaptiveprotocols.WerefersuchasolutionasC-ABR(Cloud-assistedABR).InaC-ABRsolution,thekeyABRdecisionmakingastowhichbitratetosendtoaclientiscontrolledfromtheserversideinthecloud.Thesystemlevelintelligenceintheserverunderstandsthestateforeveryclient;theavailablebandwidthforeachclient;anda“reasonable”visualqualityofthevideoforagivensizeofdisplayandattributesofvideoetc.Basedonthatintelligencetheservercontrolswhatbitrateeachclientgets.Thisincreasesnetworkutilizationandprovidessignificantlybetterfairnessbetweenclients.

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 7

ForC-ABRtobeeffectiveandmakeaninformeddecision,thekeyistounderstandthestateofeachclient,thenetwork,contentthatisbeingconsumed,etc.Thisiswhereanalyticscomeintopicture,byhelpingC-ABRtomakeaninformeddecision.Dataanalyticscanbeusedtooptimallymanagemediadeliveryacrossalargeanddiversepopulationofmultiscreen,suchasABRclientsdeployedformanydifferenttypesofTVservices:Linear,VOD,NetworkDVR,etc.

C-ABRcannotonlyassistinprovidingefficientABRvideostreamdeliveryoverIPVideoNetworks,itcanbeafundamentalcornerstoneintheoperator’stoolsetinsupportingastrategicmigrationtoWeb-basedanalyticsplatforms.ABRclienttelemetrydatacanprovideoperationalmonitoringandbereadilyintegratedintooperatordataloggingsystems.Integrationwithcustomerbackofficesystemspresentsopportunityforlongtermtrending,periodicdataanalyticsandmonetization.ThetechnologiesandideasdescribedfurtherbelowcanassistintheoverallplanningforleveragingCloud/NFV(NetworkFunctionsVirtualization)systemsaspartofaunified,IPVideoservicedeploymentstrategy.

M-ABR(MulticastAssistedABR)M-ABRisasolutionthathelpsinconservingthebandwidthwhilefacilitatingABRstreamingatthesametime.

Figure2illustratesthemajorcomponentsofaMulticast-assistedABRsolutionbasedonCableLabsspecifications[1].

Figure2–MajorComponentsinaM-ABRSystem

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 8

ThemajorcomponentsinthebaselineM-ABRarchitectureare:

1. MulticastController2. MulticastServer3. EMC(EmbeddedMulticastClientakaMulticastClient)

TheMulticastServerpullscontentfromtheCDNasanormalABRclientwoulddo,andsendsthecontentasamulticasttransportoveranIProutednetwork.

TheMulticastControllerisresponsiblefordefiningthechannelmaps,mappingofspecificABRdeliveredstreams,fortheMulticastServerstomulticastdeliveryoftheABRcontentandprovidethesechannelmapstoedgedevicessotheyknowwhatservicesareavailableandhowtoconnecttothem.TheMulticastControllerdetermineswhatchannelsaremulticastbyinstructingM-ABRServerstosetupandsubsequentlyteardownmulticaststreamsasABRstreamviewershipchanges.

TheEMCisanapplicationrunninginthegateway(M-ABREdgeDevice)thatmanagesallaspectsofthemulticastinputsreceivedfromthemulticastserverandtheirconversiontoHTTPunicastABRstreamsdeliveredoveralocalnetwork(e.g.subscriberhomenetwork)toABRclientsforplayout.

ThefollowingdescribestheprocessthatoccurswhenanABRplayerclientmakesarequestforcontentthroughthegateway.WhenEMCreceivesanABRvideosegmentrequestsfromABRplayerclientswhichhavebeenproxiedtoit:

1. EMCchecksiftheABRclientURLrequestmatchesastreamthatiscurrentlybeingmulticastandinthegateway’slocalcache.Ifnot,EMCrequeststhevideosegmentviaHTTPunicastandreturnsittoABRclient.Inthiscapacity,EMCactsasatraditionalCDNtransparentwebcache.

2. IfthevideosegmentrequestedbytheABRclientispresentinthelocalcache,EMCreturnsittotheABRclientimmediately.AvideosegmentcouldalreadybepresentinthelocalcacheduetoapreviousABRclientrequestorbasedonascheduledoperatorrequesttopre-cachecontentbasedonprojectedprogramviewershipsuchasforlivesportsevents.

3. Ifthemulticaststreamhasnotbeenjoined,theEMCjoinsthemulticaststreamtoreceivevideosegmentsviamulticast,thenconvertsthesefromNACK-OrientedReliableMulticast(NORM)toanABRformat(e.g.HLS,DASHISO),andstorestheseasABRsegmentsinthegateway’slocalcache.

4. AllsubsequentABRsegmentrequestsaredeliveredfromthegatewaycachebasedonstep#1above.

5. EMCperiodicallychecksforupdatestotheconfigurationandmulticaststreamchannelmap.

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 9

Duetothebandwidthlimitationsandotherconsiderations,itisnotpracticaltodelivermulticasttrafficforallthechannels[3][4].M-ABRsystemachievesefficientuseofnetworkbandwidthbymulticastingonlythepopularchannels.Thisiswhereanalyticscomeintopicture,byidentifyingthepopularchannelsandotherinformation.

BIGDATA&PREDICTIVEANALYTICSOverviewTheadventoftheInternetandtheresultingdatathatneedstobemanaged,stored,andanalyzed,overcheapcommodityhardwarethatcaneasilyscalehorizontally,ledtotheemergenceofmoderndayBigDatatechnologies.TermstypicallyusedtocharacterizeBigDataarevolume,velocity,andvariety.Volumereferstothesheeramountofdatathatisbeingcreatedcontinuously.Velocityisthespeedwithwhichthisdatacanbeprocessedandanalyzedforameaningfuluse.Varietyisthedifferenttypesandformatsofdatathatiscollected.

ThecoretenetsofanyBigDatasystemareitsabilitytocollectavarietyoflargeamountsofdata,useparallelprocessingtoanalyzeandprocessthatinformationtoderivemeaningfulinsight,andtoprovideaccesstothederivedinformationthroughdashboards,reports,andAPIs[5].Allthiscanbeachievedatnearlinearscalability-asthedatasizeincreases,onecanthrowmorehardwareatitandseeprocessingcompleteinthesameamountoftime–andatthesametimebeingtoleranttofailuresonanyofthecomputernodesprocessingthatdata.

BigDatarelatedtechnologieshaveevolvedovertheyearsandmostofthepopularones,suchasHadoop,MapReduce,YARN,SpringXD,andSparkareavailableasopensourcemakingiteasierforadoption.WhileHadoopisgoodathandlingBigData,itprimarilydealswithbatch-orienteddataprocessingandstorage.TraditionalHadooptechnologiesarenotwellsuitedforreal-timedataprocessingneeds.Inrecentyears,theopensourceSparkframeworkhasbecomethedefactochoiceforprocessingandanalyzingbigdatainreal-time.SparkalsomakesthelifeofadatascientisteasierbyprovidingAPIsforstatistics,machine-learning,graphprocessing,etc.–allinasingleframeworkwhileprovidinglinearscalability.SparkprovidesstrongintegrationwithHadoopecosystemandinmanycasesbothofthemareusedtogether–Sparkforreal-timedataprocessing&Hadoopforbatchprocessingandtostorethedata.ASparkcluster,runningonmultiplenodes,providestheAnalysis&ProcessingLayertheabilitytohorizontallyscaleasthedatatrafficincreases.

AtypicalAnalyticsSystemisbroadlyclassifiedintothreedifferentlayers:

• DataIngestLayer:Thislayerfacilitatescollectionofmulti-formattedinformationfromavarietyofdatasourcesinreal-time,doingparsingandpre-processingas

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 10

neededandpassingittotheAnalysis&ProcessingLayer.Optionally,acopyoftherawdataisstoredinHadoopforlong-termstorageneeds

• Analysis&ProcessingLayer:Thisiswhereallthereal-timeprocessingandanalyticshappen.Itconsistsofdifferentprocessesallrunningasynchronouslyinparallel.Onesetofprocessesoperateontherawdata,processesthem,andstorestheresults.Anothersetofprocessesperformthenextlevelofprocessingbydoingsummarizations,correlations,deeperanalysis,etc.

• DataOutputLayer:Theresultantdataisexposedthroughmultiplemethods–dashboardsandchartsthatprovideavisualviewintothedifferentoperationalandusageaspectsofthesystemandRESTAPIsthatcanbeprogrammaticallyconsumedbyvariousservicestoactontheanalyzeddata

WhileBigDataprovidesaninfrastructuretocollectthedataandprovideascalableplatformtoprocessandstorethedata,predictiveanalyticsprovidesawaytoleverageallofthatinformationandgaintangiblenewinsightsbyrecognizingpatternsindatatoprojectprobability.Predictiveanalyticsencompassesavarietyoftechniquessuchaspredictivemodeling,machinelearning,anddataminingthatanalyzecurrentandhistoricaldatatomakepredictionsaboutfutureorotherwiseunknownevents.

Inbusiness,predictivemodelsexploitpatternsfoundinhistoricalandtransactionaldatatoidentifyrisksandopportunities.Organizationsusepredictiveanalyticsinavarietyofdifferentwaysbyapplyingmachinelearning(ML)andartificialintelligence(AI)algorithmstooptimizebusinessprocessesanduncovernewstatisticalpatterns.Predictivemodelsoftenperformcalculationsduringlivetransactions,forexample,creditcardfrauddetection.

Predictivemodelsaremodelsoftherelationbetweenthespecificperformanceofaunitinasample(referredtoasthe“trainingdata”)andoneormoreattributesoftheunit.Theobjectiveofthemodelistoassessthelikelihoodthatasimilarunitinadifferentsamplewillexhibitthespecificperformance.Modelscapturerelationshipsamongmanyfactorswhichhelpinguidingdecisionmaking.

BigDatatechnologiessuchasSparkaremakingpredictiveanalyticssystemsmoreaccessibleandeasiertoimplement.

ApplicabilitytoABREnablingSolutionsAwell-instrumentedABRecosystemmirrorsthewebworldintermsofvolume,velocity, and variety of data that needs the processing power of a Big Datasystem in terms of collecting the data, processing, and storing. A typicaloperator has a few million subscriber homes, each of them having a homegateway,alongwithoneormoreSTBsandsecond-screendevicesconnectedtothe home gateway. Each of thesemillions of home-based subscriber devices

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 11

are constantly producing events. For example, we have a system that isgeneratingmillionsofdataeventseveryminute. Inaddition to thedevices inthesubscriber’shome,thecomponentsinthevideodeliverysidealsoproducehugeamountofusefuldata.

Analyzing this data provides insights into the ABR ecosystem generating afeedback loop as shown in Figure 3 to help in the automation for fixing theoperational issues andmaking key decisions in terms of network bandwidthsavings,QoS,andmore.

AsmentionedintheC-ABRandM-ABRsections,analyticsplayaroleintheirdecisionmaking.Subscribers’homegatewaysareinstrumentedtocollectuserinteractionandcontentconsumptioninformationtoderiveinsightintouserbehaviorwithrespecttovideoconsumption.AnalyzinghistoricaldatainconjunctioninEPGmetadatacanbeusedtopredictseveralimportantfactorssuchasusersurfingpatterns,channelpopularity,andchannelchangeprobabilityatgranularlevel(ataspecifictimeofthedayinaspecificgeographicalarea).ThisknowledgecanbeappliedtooptimizeABRdeliveryanddirectlyimpactoperatornetworkbandwidthsavingswhilemaintainingsubscriberQoE.

Figure3–AnalyticsProvidingaFeedbackLoop

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 12

IdentifyingandAddressingOperationalIssuesDataiscollectedconstantlyfrommillionsofdevices(homegateways,COAMdevices)insubscriber’shomeandalsofromthecomponentsinthecontentdeliveryside,suchasMulticastServer.Thedatathatiscollectedandprocessedprovidesactionablevaluetotheoperator.

AnalyzingthisdataprovidesinsightsintotheABRsolutionsforunderstandingoperationalissuesandgeneratingafeedbacklooptohelpintheautomationforfixingtheoperationalissuesthatweredetected.Inorderforthistobeeffective,thecollecteddataneedstobeanalyzedinreal-time.Forexample,ifwedetectthatthenetworklatencyisbeyondaparticularthreshold,thenthisinformationcanbeusedforcorrectiveactionsimmediately.

Someofthegenerichigh-levelgoalsinclude:

• Understandingusageofdifferentcomponents,services,andnetwork• Understandingandclassifyingnetworkandsystemlimitationsbyanalyzing

trendsofsystemusagewithrespecttoscalability,capacityplanning,andbandwidthmanagement

• Creatingdashboardstoprovidebusinessintelligence,andAPIsfordownstreamusage

• ExaminingtheperformanceandworkloadoftheCMTS/CCAP

Whilethegenericgoalscoverallgenericaspectsofvarioussolutions,eachparticularsolutionwouldhaveitsownspecificgoals.Forexample,thegoalsfortheoperationalaspectsoftheM-ABR[3]solutioninclude:

• AnsweringkeyquestionsabouttheperformanceofmulticastoverDOCSIS,includingthetelemetryofanypacketlossdetectedbytheEMC

• EvaluatingtheworkloadwithintheEMCtoperformthefunctionsofthe“transparentproxycache”

• ExaminingtheperformanceoftheCMTS/CCAPformulticastdeliveryandimplicationsrelatedtoitssupportforIGMP

NotethatwhiletheabovementionedgoalsspecificallymentionscablenetworkrelatedtechnologiessuchasCCAP/CMTSandDOCSIS,similargoalsarevalidforPassiveOpticalNetwork(PON)networktechnologiessuchasOLT(OpticalLineTerminal).

InordertotakespecificactionstooptimizethenetworkandprovideabetterQoS,wewouldneedtoanswerspecificquestions.Forexample,inthecaseofM-ABR[3],theoperationaldatawouldhelpinanalyzingandansweringquestionssuchas:

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 13

• Howoftendoesafilesegmentexperienceanypacketloss?• Isthepacketlosscharacterizedbyfactorssuchastimeofday,downstream

capacity,CMTStype/configuration,aspectsoftheHFCaccessnetwork,IGMPtrafficmanagement,etc.?

• HowdoeschannelsurfingaffecttheIGMPbehaviorofjoinsandleavesandwhendoesitbecomemoreofaburdenandlessofabenefit?

• Howquicklycaninitialtuningtoanewchannelresultinmovingfromunicastvideosegmentstomulticast(cached)videosegments?

• HowlongshouldtheEMCremainonamulticaststreambeforeitdeterminesthereisnobenefittocachingstreamsegments?

• Whatisthetimingofacachesegmenttoclientrequest?• Whatisthebehaviorofclientchangeinqualityusingalternate,non-multicast

variantplaylists?

Theoperationaldataisalsoexposedinavisualformthroughdashboardscharts.Thisvisualrepresentationallowstheoperationtogetaquickoverviewoftheoperationsaspectsofthesystem.Forexample,thechartsinFigures4and5provideinsightsintoanM-ABRsystem[3].Figure4showsthenumberofclientsessionsthatexistonapermulticastchannel(stream)basisoveraspecifiedtimeperiod,termedasthePopularityofaChannel.Itcanhelptodeterminewhichchannelsaremorepopularandwhen.ThisdatacanbeusedtooptimizingthenetworkandQoSforthepopularchannelsandtodeterminewhatchannelsaremoreappropriateformulticast.Figure5showsthepeakandtheaveragetimethatittookforthemulticastclienttoreceiveasegmentfromthemulticaststream.Thisvalueisdependentonotherfactorssuchasdurationofthesegment,theavailablebandwidthandthemulticastratepercentage.Ifthisvaluebecomesgreaterthanthesegmentdurationitcouldindicateanissuesomewhereinthenetworkorinthemulticastserver.

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 14

Figure4–ChannelUtilization

Figure5–NetworkLatency

UsingPredictiveAnalyticsinM-ABRM-ABRsystemimprovesefficiencyandsavesbandwidth&networkcostsinIPVideodeliverytomultiscreendevices,bystreamingsomechannelsviamulticastinsteadofunicast.

Duetocapacitylimitsandconstraints[4](e.g.,DOCSISnetworkconstraintsattheprovider),thenumberofchannelsthatcanbemulticastarelimited(e.g.thetop20

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 15

popularchannelscanbemulticastoutoftheavailable500+channels).So,thereisaneedtoidentifythechannelsthatarethebestcandidatestomulticast.Popularchannels(i.e.,morenumberofpeoplewatching)arethebestcandidatesastheyresultinmorenetworkbandwidthsaving.Channelpopularitycanchangedynamicallywithtimeandgeographicregion(akazone).

Oneofthestraight-forwardmechanismstoidentifypopularchannelsisthroughreal-timemonitoringofchannelusage.Thisapproachhasdrawbacks:

1. Itiscostlyandresourceintensive(requiresfrequentheartbeatorequivalenttechniques)tomonitorchannelusageinreal-time.

2. Itbecomescounterproductiveinthecaseofflapping–channelsareswappedin-and-outveryfrequentlyinshortdurations,thusincreasingtheproblemratherthansolvingit.

Regardingthefirstdrawback,collectingandprocessingheartbeats(channelusagestatistics)veryfrequently(say,every10seconds)frommillionsofsubscribergatewaysiscostlyandresourceintensive.Predictiveanalyticscanbeusedtopredictthebestchannelstomulticastthatchangeintimeandgeography(zone)i.e.,foreachtimeperiodofaday,predictthemulticastchannelmapataneighborhood/zonelevelgranularity.Thissolutioncancomplement(orreplace)thetraditionalreal-timechannelmonitoringmechanismtosaveoncostsandresources.Forexample,ifweknowthatCNNisgoingtoapopularchannelataparticulartime,thenwecanaddCNNautomaticallytothemulticastgroupattheappropriatetimeandthereal-timechannelmonitoringmechanismcanskipCNNandmonitorotherchannelstherebyreducingthenumberofchannelsitmonitors

Theseconddrawback-channelflapping–canalsobeaddressedusingpredictiveanalytics.Inthegroupofchannelsthatareselectedtobemulticast,thechannelsareorderedbypopularity(thefirstchannelinthemostpopularandthelastonetheleastpopularinthatgroup).Say,thegroupsizeis10foratop10list.Thechannelsatthetopportion(saytop80%)arerelativelystable.Astimeprogresses,thepositionofachannelinthetopmaychange,butitwillstillremaininthetop10list.However,thechannelsatthebottomportion(saybottom20%)willvaryfrequentlyduetoviewerschanging/flippingchannels.Thischanging/flippingchannelscanresultinachannel(inthebottomportionofpopularchannelslist)todropoutofthetop10listandbebackintothetop10listwithinafewseconds.Typically,whenachannelisflapped-out,theM-ABRsystemwillstopmulticastingthatchannelanditwillthenbeavailableonlyviaunicast.Whenachannelisflapped-in,theM-ABRsystemwillstartmulticastingit.Ifthisin-and-outhappensonachannelwithinafewseconds,thenitcreatesaproblemwheremulticastisnotefficientlyusedandswitchingbetweentheprotocolscreatesadditionaloverheadandpotentialunicastbursts.

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 16

Figure6providesanoverviewonanalyticscollectingdata,generatingchannelmapsandfeedingthisinformationtothemulticastcontroller,whichusesthisinformationtodecideonthelistofchannelsthatwouldbedeliveredusingmulticast.

Figure6–PredictiveAnalyticsinM-ABR

Theanalyticssystemkeepscollectingusageandtelemetrydatafromallthesubscriber’sgateways.Thisdataisstoredandovertimewegetaconsiderableamountofhistoricaldatawhichisanalyzedtofinduserbehavior,usagepatternsandcorrelations.Notethatforprivacyreasons,PersonallyIdentifiableInformation(PII)areanonymized.

Avarietyoftechniquesareusedtomakedifferenttypesofpredictionstakingdifferent.Eachofthosetechniquestakesdifferentdatapointsintoconsiderationtomakepredictions.

Theuseofexternaldata,suchasEPGmetadataaugmentsthepredictioncapabilitiesbyallowingustopeekintomoregranularlevelofusagepatterns.TheEPGmetadataprovidesdetailssuchasgenre,rating,cast,title,channel,timestamp,etc.Thisdatahelpstobuildaprogrampopularitymodelandallowsustoderiveinsightsintopopularityofaprogram(andthecorrespondingTVchannel)andthesubscriber’s

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 17

likes/dislikes.Forexample,ifweseethatasubscriber(enduser)regularlyistuningintoCBSchannelat7PMonMondays,fromtheEPGmetadatawecanfindthattheprogramduringthattimeisBigBangTheoryandderivethattheviewerlikesthatprogramandcanpredictthathe/sheislikelytowatchthatprogramwhenittelecastsatafuturedate/timeandpredictagivenprogram’susageinfuture.

Understandingusagepatternsprovidesinformationtopredictchannelusage.Thehistoricaldataisanalyzedtofindpatternssuchas:

• Bingewatchingvs.randomvs.followingaschedule(timeofaday)• Favoriteprograms&channels• Frequentchannelsurfingvs.steadywatchingvs.surfingduringcommercials/ads• Favoritetypes-sportsvs.newsvs.realityshows• Automated/scheduledrecordings.

Channelchangepredictionfindstheprobabilitywhenachannelwillbechanged(usersleaving&usersjoining)usinghistoricaldatawithinputfromprogrampopularityandusagepatterns.Itcouldpredicteventssuchas:

• Channeljoin(theprobabilitythataviewertunestoachannelatagiventimeofday)

• Channelleave(theprobabilitythataviewerleavestoachannelatagiventimeofday)

• Channelflippingduringanad(SCTE-35markersisonewaytoknowwhenanadoccurs)

• Whetherchannelchangecoincideswiththechangeintheprogram(usuallyaroundthehalfhourorhourtimeboundary)

• Viewershipofachannelinthenextfewminutes• ProgramchangebasedonEPGmetadata

Theviewershipandchannelsurfingpatternsofindividualusersareaggregatedtopredictaninformeddecisionaboutachannelchangeatanygeographicalzoneandtimeframe.Thisallowsustoaddressflappingi.e.,wecanpredictifachannelneedstobekeptinthepopularlist(e.g.,top10list)eventhoughtheviewershipdropstemporarilyorvice-versa.

Basedonalltheabovementionedpredictionmodels,withoutusinganyheartbeatkindofmechanismstheanalyticssystemcanbuildchannelmaps–foragivenzoneandforagiventimeoftheday–asshowninFigure7andprovidethatinformationtothemulticastcontrollerwhichusesthisdatatodecideonthechannelstobemulticast.

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 18

Figure7–ExampleShowingMulticastChannelMaps:Time&ZoneView

AugmentingDecisionMakinginC-ABRInaC-ABRsolution,thekeyABRdecisionofdecidingwhichbitratetosendtoaclientiscontrolledfromthenetworksideinthecloud,asopposedtoatypicalABRsolutionwhereeachclientdecideshowmuchbandwidthisavailableindependentofallotherclients.Inparallel,subscriberQoEcanbemaintainedbymaximizingnetworkdeliverycapacitybasedonthedynamicmeasurementofvideoqualityonapervideostreambasis.Inordertodomakeanetwork-wideoptimizedbandwidthcapacityandsubscriberQoEdecisions,collectionofdatainreal-timefromalltheend-clients(e.g.,subscriber’sCOAMdevices)isrequired.

TheBigDatainfrastructureisusedtocaptureclientdatastatisticsacrossallscreensinthehome,suchasvideostarttime,videostartfailure,exitsbeforevideostarts,numberofsuccessfulplaybacks,andclientbufferingorstallingoccurrences.ThisdataisusedtoextractsubscriberviewershippatternsandtoleranceforQoSlevels.BandwidthmeasurementscanprovideanalyticssuchastheWi-Fibandwidthachievedinahomebyclienttype(Mobile,STB,Gateway,SmartTV,etc.),whichcanbeusedfortailoringmediadeliverytosuchclientaswellasbeingleveragedforlongertermserviceassessment.

Usingthisdata,theanalyticssystemwouldaugmentthedecisionmakingfortheC-ABR,byunderstandingthestateforeveryclient;theavailablebandwidthforeachclient;anda“reasonable”visualqualityofthevideoforagivensizeofdisplayandattributesofvideoetc.Basedonthatintelligence,C-ABRcanenforceanetwork-widebandwidthcapacitypolicythatallocatesbandwidthbyABRclientorclass,withprecisionpotentiallydowntothespecificABRstreambitratethateachclientgetsoveragivendurationoftime.

Theneteffectincreasesoverallnetworkbandwidthandresourceutilizationwhileprovidingsignificantlybetterfairnessbetweenclients.Asacloud-basedsolution,C-ABR

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 19

candynamicallyscaleprocessingrequirementsaccordingtonetworkcapacitybeingconsumedbyABRclientsyetwithintheguidelinesofnetworkcapacityprovisionedbytheserviceoperator.

UnmanagedABRclientdeploymentshavebeenextensivelystudiedandreportedinearlierresearchpapers[2].

CONCLUSIONAsoperatorsmigratetoIPVideodelivery,theyshouldlookintoadoptingpromisingsolutionssuchasM-ABR(MulticastAssistedABR)andC-ABR(CloudAssistedABR)inordertooptimizethenetworkandimproveQoSandQoE,whilestillenablingABRstreamingtoTVsubscriberhomedevices.

ThispaperprovidesanoverviewintohowharnessingBigDataenablesinsightintouserbehaviorandcorrespondingvideoconsumptionandhowPredictiveAnalyticscananalyzehistoricaldatatopredictseveralimportantfactorssuchasusersurfingpatterns,channelpopularity,andchannelchangeprobabilityatgranularlevel(ataspecifictimeofthedayinaspecificgeographicalarea).ThisknowledgecanbeappliedtooptimizeABRdeliveryanddirectlyimpactoperatornetworkbandwidthsavingswhilemaintainingsubscriberQoE.

Also,understandingtheoperationalissuesbecomesessentialforthesenewtechnologiessothatwecanaddressthoseissuestoimproveefficiencyandreducecost.BigDataprovidestoolsandtechnologiestocollect,processandstoredatathatareconstantlybeingproducedfromthemillionsofhome-basedsubscriberdevices.Analyzingthisdataandprovidingafeedbackloopintotheecosystemhelpsintheautomationofaddressingtheoperationalissues.

WhilethispaperhasshownafewpossibilitiesintermsofhelpingindecisionmakingandaddressingoperationsissuesbyusingBigDataandAnalytics,operatorscanthinkofmanyotherpossibilitiesthatarespecifictotheirsolutions.Forexample,Analyticscanhelppredicttrafficvolumewhichallowsustoplanandmanagethecapacityofresourcesinthecloud(CDNcapacity,MulticastServers,VODServers,andNetworkResources/CMTSServiceGroups).

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 20

ABBREVIATIONSABR AdaptiveBitrateC-ABR CloudassistedABRCCAP ConvergedCableAccessPlatformCMTS CableModemTerminationSystemsCOAM CustomerOwnedandMaintainedDOCSIS DataOverCableServiceInterfaceSpecificationEPG ElectronicProgrammingGuideIP InternetProtocolM-ABR MulticastassistedABROTT OverthetopSABR SmartABRSTB SetTopBoxQoE QualityofExperienceQoS QualityofService

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 21

RELATEDREADINGS• ABRDeliveryArchitecturesandVirtualization–thispaperdiscussestwo

emergingtrendsinvideoprocessingdelivery,namely,migrationofvariousvideoprocessingfunctionstothenetworkcloudtoleverageadvancesinvirtualizationanddynamicpackagingtechniquesforadaptivebitrate(ABR)deliveryofvideo.

• EffectiveUtilizationofMulticastABRUsingBigDataandReal-timeAnalytics–ThispaperprovidesanoverviewintohowBigDataandreal-timeanalyticscanenableinsightintovideoconsumptionandnetworkoperationalaspectstomakeM-ABRmoreeffective.Byleveragingactionableinsight,serviceproviderscanbettermanagetheirnetworks,whileincreasingbandwidthutilizationandreducingoperationalcomplexity.

• SmartABR:TheFutureofManagedIPVideoServices–ThispapercomparesindetailtheSABRsystemtotraditionalunmanagedABRdeliveryaswellasasystemwithenhancedCMTSQoS.WithSABR,operatorscansignificantlyincreasetheirIPVideocapacitywhilegracefullyhandlingcongestionandprovidinganimproveduserexperience.

MEETONEOFOUREXPERTS:SridharKunisettySridharKunisetty,DistinguishedEngineer,has20yearsofexperienceinvarioustechnicalandmanagementrolesatARRIS,Motorola,Oracle,CommerceOne,andiPass&Connectbeam.HisexpertiseisinAnalytics,WebServices,Cloud,andDatabasestechnologies.Sridharisaleadinventoronseveralpatents.Whenhegetsabreak,Sridharenjoysbiking&runningandhasparticipatedinseveralmarathons.SridharholdsanMSinComputerScience&EngineeringfromUniversityofFlorida,GainesvilleandaBachelor'sDegreeinComputerScience&EngineeringfromNationalInstituteofTechnology(NIT),Warangal,India.

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 22

REFERENCES(1) IPMulticastAdaptiveBitrateArchitectureTechnicalReport;CableLabsdocumentOC-TR-IP-MULTI-ARCH-V01-141112,Nov2014

(2) SMARTABR:ThefutureofmanagedIPVideoservices;JohnUlm,AjayLuthra,PraveenMoorthy,MarkSchmidt,andHaifengXu;2013CableShowTechnicalSession

(3) EffectiveutilizationofM-ABR(MulticastassistedABR)usingBigDataandReal-timeAnalytics;SridharKunisetty,JeffreyTyre,andRobertMyers;INTX2016SpringTechnicalForum

(4) ChallengesdeliveringMultiscreenLinearTVServices:MulticastassistedABRtotheRescue;JohnUlm,SCTECable-TecExpo,Fall2014

(5) TVINSIGHTS-ApplicationofBigDatatoTelevision”;BhavanGandhi,AlfonsoMartinez-Smith,&DougKuhlman,IBC2015