white paper_latency matters.pdf

7/27/2019 white paper_latency matters.pdf

http://slidepdf.com/reader/full/white-paperlatency-matterspdf 1/13

www.o3bnetworks.com 1

Contents

ThispaperispresentedbyO3bNetworkstoprovideclarityandunderstandingofacommonlymisunderstoodfacetofdatacommunicationsknownaslatency.ItisourgoalthatthereadertakesawayaclearunderstandingoflatencyandgainsinsightintothesignificantadvantagethatO3bNetworksbringstothesatellitecommunicationsmarketforthedevelopingworld.

Executive Summary 2

What Causes Latency? 2

Transmission Rate and Bandwidth 4

Latency and TCP/IP 6

Satellite Link Latencies 7

Applications 10

Summary 10

Appendix 11

What is latency? 11

Networking 101 11

What is Network Latencyand Why Does It Matter?

by O3b Networks




Executive Summary

Internetdataispackagedandtransportedinsmallpiecesofdata.Theflowofthesesmallpiecesofdatadirectlyaffectsauser’sinternetexperience.

Whendatapacketsarriveinasmoothandtimelymannertheuserseesacontinuousflowofdata;ifdatapacketsarrivewithlargeandvariabledelaysbetweenpacketstheuser’sexperienceisdegraded.

Latencyandoverallthroughputisdominatedbytwofactors,thelengthoftheroutethatthepacketshavetotakebetweensenderandreceiverandtheinteractionbetweentheTCPreliabilityandcongestioncontrolprotocols.

ThispaperwilladdressthesourcesofdelaystointernetdataanddescribewhyO3bNetworkssatelliteconstellationhaslowerpathlengthandlatencythantraditionalgeosynchronoussatellitesandinsomecasesfiberbasednetworks.

What Causes Latency?

Regardlessofthespeedoftheprocessorortheefficiencyofthesoftware,ittakesafiniteamountoftimetomanipulateandpresentdata.Whethertheapplicationisawebpageshowingthelatestnewsoralivecamerashotshowingatrafficjam,therearemanywaysinwhichanapplicationcanbeaffectedbylatency.Fourkeycausesoflatencyare:propagationdelay,serialization,dataprotocols,routingandswitching,andqueuingandbuffing.

Propagation Delay

Propagationdelayistheprimarysourceoflatency.Itisafunctionofhowlongittakesinformationtotravelatthespeedoflightinthecommunicationsmediafromsourcetodestination.Infreespace,thespeedoflightisapproximately3x105km/sec.Thespeedoflightislowerinothermediasuchascopperwireorfiberopticcable.Theamountofslowingcausedbythistypeoftransmissioniscalledthevelocityfactor(VF).

Mostpeoplearesurprisedtolearnthatcopperwireandfiber-opticcableshavesimilarvelocityfactors.Fiberopticcablestypicallymeasurearound

70%ofthespeedoflightwhereascoppercablevariesfrom40%to80%dependingontheconstruct.CoaxialcableiscommonlyusedandmanytypeshaveaVFof66%.

Satellitecommunicationlinksuseelectromagneticwavestopropagateinformationthroughtheatmosphereandspace.Theinformationisconvertedfromelectricalsignalstoradiosignalsbythetransmitterandtheantenna.Oncetheseradiosignalsleavetheantenna,theytravelapproximatelyatthespeedoflightforfreespace.

Let’scalculatehowlongitwilltakeanemailtotravelfromNewYorktoLondonassumingthatwearetheonlyuseronaprivatecommunicationschannel.

What is Network Latency and Why Does It Matter?




Ignoringtheactualroutestakenbyunderseacablesduetotheocean’sfloor,let’sassumethepathfromNewYorktoLondonisthegreatcircledistanceof5458km.

Propagation delay = distance/speed:

The email sent using a copper link: 5458 / 197863.022 = 23.58 msThe email sent using a fiber-optic link: 5458 / 209854.720 = 26.01 msThe email sent using a radio link: 5458 / 299792.458 = 18.21 ms

Thesearethelatenciescausedonlybypropagationdelaysinthetransmissionmedium.Ifyouweretheonlyonesendingonesingledatabitandyouhadunlimitedbandwidthavailable,thespeedofthepacketwouldstillbedelayedbythepropagationdelay.

Thisdelayhappenswithoutregardfortheamountofdatabeingtransmitted,thetransmissionrate,theprotocolbeingusedoranylinkimpairment.

Serialization Delay

Serializationistheconversionofbytes(8bits)ofdatastoredinacomputer’smemoryintoaserialbitstreamtobetransmittedoverthecommunicationsmedia.Serializationtakesafiniteamountoftimeandiscalculatedasfollows:

Serializationdelay=packetsizeinbits/transmissionrateinbitspersecond

For example:

- Serialization of a 1500 byte packet used on a 56K modem link willtake 214 milliseconds

- Serialization of the same 1500 byte packet on a 100 Mbps LAN willtake 120 microseconds

Serializationcanrepresentasignificantdelayonlinksthatoperatealowertransmissionrates,butformostlinksthisdelayisatinyfractionoftheoveralllatencywhencomparedtotheothercontributors.

Voiceandvideodatastreamsgenerallyusesmallpacketsizes(~20msofdata)tominimizetheimpactofserializationdelay.

Data Protocols and Latency

Datacommunicationsprotocolsatvariouslayersintheprotocolstackusehandshakessynchronizethetransmitterandreceiver,forthetransmitterandreceivertoupdateeachotheronlinkstatusandtocorrectforerrosintransmission.Thesehandshakestaketimetopropagateacrossthelinkandcanaddtothelatencyofthetransmissionofinformationfromsourcetodestination.





Routing and Switching Latencies

InIPnetworkssuchastheInternet,IPpacketsareforwardedfromsourcetodestinationthroughaseriesofIProutersorswitchesthatcontinuouslyupdatetheirdecisionaboutwhichnextrouteristhebestonetogetthepackettoitsdestination.Arouterorcircuitoutageorcongestiononalinkalongthepathcanchangetheroutingpathwhichinturncanaffectthelatency.

HighperformanceIProutersandswitchesaddapproximately200microsecondsoflatencytothelinkduetopacketprocessing.IfweassumethattheaverageIPbackbonerouterspacingis800km,the200microsecondsofrouting/switchingdelayisequivalenttotheamountoflatencyinducedby40kmoffiber;routing/switchinglatencycontributestoonly5%oftheendtoenddelayfortheaverageinternetlink.

Queuing and Buffer Management

Anotherissuewhichoccurswithinthetransportlayersiscalled“queuinglatency”.ThisreferstotheamountoftimeanIPpacketspendssittinginaqueueawaitingtransmissionduetoover-utilizationoftheoutgoinglinkaftertherouting/switchingdelayhasbeenaccountedfor.Thiscanadduptoanadditional20msoflatency.

Transmission Rate and Bandwidth

TransmissionRateisatermusedtodescribethenumberofbitswhichcanbeextractedfromthemedium.Transmissionrateiscommonlymeasuredasthe

numberofbitsmeasuredoveraperiodofonesecond.The“maximumtransmissionrate”describesthefundamentallimitationofanetworkmedium:

IfthemediumisacopperLocalAreaNetwork,maximumtransmissionratesarecommonly10,100,or1000Megabitspersecond.Theseratesareprimarilylimitedbythepropertiesofthecopperwiresandthecapabilitiesofthenetworkinterfacecardarealsoafactor.

Fiber-opticthetransmissionratesrangefromaround50Mbpsupto100Gbps.Unlikecoppernetworks,theprimaryfactorlimitingfiber-optic

transmissionratesistheelectronicswhichoperatesateachendofthefiber.

Wirelesslocalareanetworks(LANs)andsatellitelinksusemodems(modulator/demodulator)toconvertdigitalbitsintoananalogmodulatedwaveformatthetransmitterendofalink,andthenatthereceiveendademodulatorwillthenconverttheanalogsignalbackintodigitalbits.Thelimitingfactorintransmittinginformationoverradio-basedchannelsisthebandwidthofthechannelthatisavailabletoaparticularsignalandthenoisethatispresentthatwillcorruptthesignalwaveform.





Radio Channel Bandwidth and Noise

Signalstransmittedusingradiowavesoccupyradiospectrum.Radiospectrumisnotanunlimitedresourceandmustbeshared.Topreventradiointerferencebetweenuserstheuseofradiospectrumiscontrolledbynearlyeverygovernmentontheplanet.Theamountofradiospectrumoccupiedbyanygivenradiosignaliscalleditsbandwidth.

Thenatureofradiospectrumuseisbeyondthispaperbutit’simportanttounderstandthatgenerallytheoccupiedradiospectrumofamodemsignalwillincreasewiththedatarate:

- Higher modem data rates cause the modem to occupy moreradio bandwidth

- Lower modem data rates will let the modem occupy lessradio bandwidth

Sinceradiospectrumisalimitedresource,theoccupiedradiobandwidthisanimportantlimitingfactorinwirelessandsatellitelinks.

Noiseintheradiochannelwillperturbtheanalogsignalwaveformandcancausethedemodulatoratthereceivertochangeadigitaloneintoazeroorviceversus.Theeffectofnoisecanbeovercomebyincreasingthepowerlevelofthetransmittedsignal,orbyaddingafewextraerrorcorrectingbitstothedatathatisbeingtransmitted.Theseerrorcorrectingbitshelpthereceivercorrectbiterrors.However,theerrorcorrectionbitsincreasethebandwidththatisrequired.

Data Bandwidth

Indatatransmission,thedatabandwidthissynonymoustothetransmissionratebeingused.Bandwidthisimportantbecauseitdefinesthemaximumcapacityofadatalink.

- A 10 Mbps copper LAN cannot sustain traffic flowing at a higher ratethan 10 megabits every second.

- A satellite link using modems operating at a 600 Mbps rate cannotflow any more than 600 megabits every second.

It’sveryimportanttounderstandthatdatabandwidthisamaximumdataflowobtainableoveragiventransportationsegmentoveragivenperiodoftime.





Latency and TCP/IP

ThefinaldetailrequiredforunderstandingwhylatencyisimportantisanunderstandingofwhatisgoingonattheTransportprotocollayer.The

Transportlayersitsbetweentheapplication-specificApplicationlayerandtheIPlayerthatisusedforpacketrouting.TheTransportlayerperformsanumberoffunctionsincludingsequencingthepacketsintotheirproperorder.

There are two types of Transport layer protocols:

- connectionless- connection-oriented

Connectionlessdataprotocolssimplypushthedatapacketstotheenddestinationwithoutregardforpacketlossorpacketorder.Connectionlessprotocolsareoftenusedforreal-timecommunicationssuchasvoiceor

interactivevideowhereretransmissionoflostpacketsisnotusefulbecausetheywillarrivetoolatetobeplayedout.IntheInternet,themostcommonconnectionlessTransportlayerprotocolistheUserDatagramProtocol(UDP).

Connection-orientedprotocolsaremorecomplex.Theseprotocolsrelyontheestablishmentofa“connection”whichrequireshandshakesandinteractionbetweenthesourceanddestinationcomputers.Connection-orientedTransportprotocolsprovideerrorfreesequenceddeliveryofpackets.Themostcommonconnection-orientedTransportprotocolusedintheInternetistheTransmissionControlProtocol(TCP).IfpacketsarriveoutofordertheTCPlayerputsthembackinorder.Ifpacketsaremissingorcorruptedin

transmission,TCPusesanacknowledgement(destinationbacktosourcefeedback)mechanismtoaskforthemissingorcorruptedpackettoberetransmitted.TheTCPprotocolalsomeasurestheperformanceofthelinkintermsofavailablebandwidth,delayanderrorrate.Usingawindowingtechnique,itadjuststhetransmissionrateofdatapacketstomatchtheabilityofthechannelbetweensourceanddestinationtoacceptthemanddeliverthemcorrectly.AstheTCPprotocolatthesourceanddestinationdeterminethatthechannelisoperatingefficientlywithlowdelay,itwillopenthewindowtoallowmorepacketstobeoutstanding(inflightbetweenthesourceanddestinationandnotyetacknowledged).Thishastheeffectofincreasingthetransmissionrate.Aserrors,congestion,ordelayincreases,

theTCPprotocolwillclosedownthewindowtoslowdownthetransmissionrate.Becauseofthehighdelaysingeosynchronoussatellitelinks,theTCPprotocolactstoreducetheoverallthroughputofthelink.Tocompensateforthis,satellitelinkequipmentvendorshavedevelopedTCPspoofingmechanismsthatfaketheTCPhandshakebyterminatingtheTCPlinklocallytothetransmittingsiteandtrytoconvincethetransmittingsitethatthelinkhaslowerdelaythanitreallyhas.Thisspoofingworksifthelinkisrelativelyuncongestedandhasaverylowerrorrate,whichisnotnecessarilythecaseforsatellitelinks.





Satellite Link Latencies

Satellitelinkscanintroducelargerlatenciesthanmostterrestrialnetworkingsegmentsduetolongdistancesfromthegroundstationstothesatellite.The

followingtableshowsthelatencycausedbypropagationdelaysfromtwotypesofsatelliteconfigurations,1)theO3bNetworksMEOorbitconstellationatanaltitudeof8063Kilometers,2)ageosynchronoussatelliteat35,786Kilometers.

Altitudeiskeydeterminateinlatencybecausethehigherthesatellitethegreaterthelatency.

Forsatelliteswhichoperateasabent-pipe,thepropagationdelaysaredoubledsincethesignalhastotravelbothuptothesatelliteandbackdowntotheearthbeforeitreachesthenextsegmentofthenetwork.

Thefollowingtablecomparestwotypesofsatellitesusingagroundstationin

Lagos,Nigeria.


Latency Calculations – London Internet Exchange (LINX) to Lagos

GatewaytoSatellitekm

CustomertoSatellitekm

GatewaytoSatellitemsec

CustomertoSatellitemsec

Total SatRTTmsec

Fiberto LINXkm

FibreRTTmsec

Total SatRTTmsec

O3b w/8 SatellitesSpainGWY Lagos

10126 8135 33.8 27.1 121.7 2144 10.7 132.5

GeosyndchronousSatUKTeleport Lagos

38,469 35,847 128.2 119.5 495.4 135 0.7 496.1

Fiber

LondonLagoskm

TotalFiberRTT

8,000 80.0

Satellite Latency Calculations

TheO3bNetworksMEOsatelliteusingtoateleportinSpainwillexperienceroundtriptime(RTT)ofapproximately122msec.Ifweaddintheaverage

InternetlatencyfromtheSpainteleporttotheLondonInternetExchange(LINX)(~10msec.),weendupwithanoveralllatencyfromLagostoaEuropeaninternetsiteof132.5msec.O3b’s132.5msec.roundtriplatencyisclosetofiber’s80msec.RTT

Bycomparison,thesameLagoscustomersiteusingageosynchronoussatellitetoaUKTeleportandthenontoLINXwillhavelatenciesof496msec.

ToactuallyretrieveawebpagefromanInternetserveratLINX,willtakesomeadditionaltimeforthedatarequestpacketsandtheir

acknowledgments:





This data request time includes:

- Therequestpacketfromtheusertothewebserver- Thewebserveracknowledgingtherequest- Thewebserverpushingtorequesteddatatotheuser

- Thedataarrivingattheuser’scomputer

Geosynchronoussatelliteusersmustwaitalmost750msec.beforetheystartgettingdata,whereasthelowerlatencyO3bsatellitelinkwillreceiveitnearly4xsooner.

Anotherproblemwithhighlatencylinksistheintroductionofalargeamountofinter-packetjitterduetotheTCPprotocolflowcontroloperatinglessefficientlyoverlongerdelaypaths.ThefollowingplotsweremadeusingaTCP/IPpacketcaptureutility.Theseplotsshowthepacketbitrateonthey-axisandtimeofdayonthex-axis.Thedatabeingtransmittedwas

theun-cachedweb-pagere-loadofthecontentfromtheCNNwebpage(http://www.cnn.com).

Theonlyconditionchangedduringwasthedelaybetweenpackets–thetransmissionrateremainedthesame.




TheaddednetworklatencyanditaffectontheflowofTCPdataspreadthewebpageloadovertime.

• The 50 ms latency link took 3 seconds• The 150 ms latency link took 5 seconds

• The 300 ms latency link took 11 seconds• The 600 ms latency link tool 17 seconds

Thespreadingofnetworkdataovertimereduceswhat’scalledthe“EffectiveBandwidth”ofalink.Packetsarestillbeingtransportedatthesamebitratebutduetolatencyitistakingmuchmoretimeforalloftheweb-pagepacketstoarrive.

It’sthis“spreadingovertime”behaviorofhighlatencynetworkswhichbecomesnoticeabletotheuserandcreatestheimpressionthatalinkisnotoperatingatahighspeed.

O3brecentlyconductedanotherdemonstrationofreal-worldeffectsoflatencyusingthetimetoloadawebpage.Thisisaverycommonactivityandclearlyshowsusersthatlatencydirectlyaffectsthewayauserobtainsdatafromtheinternet.

ThefollowingplotsshowtheeffectsoflatencyonthetimetoloadtheWallStreetJournalwebpage(http://www.wsj.com):

Web Page Load Time: wsj.com to Florida

Itisclearthatwebpageloadtimesdramaticallyincreasewhenlatencyincreases.Inthiscasedoublingthelatencynearlydoubledthewebpageloadtimes

Whenlookingatthebasiclatencynumbers,it’seasytoseethattheO3b

Satelliteconstellationwillofferusersanoticeablybetterinternetexperiencewithmoreimmediatefeedbackandquickeraccesstodata.





Applications

Whatapplicationscannowberunoverlowerorbitsatellitesthatwerepossibleafewyearsago?

Manyapplications,suchasbasicwebsitedownloads,Internetvideostreamingthatoperatedsluggishly(orwereveryslowtostartup)overgeosynchronoussatellitelinksnowhavemuchfasterresponseandworkmuchbetteroverO3b’slowlatencyconnections.Videoconferencingandregularvoicecallsovergeosynchronoussatelliteswereverystiltedandnon-interactive.Conversationswere“half-duplex”andonepartyhadtowaitfortheotherpartytofinishtalkingbeforetheycouldstartup.Youcouldnotbreakintoaconversationandinterrupt.VoicecallsandvideoconferencingaremuchmorenaturalandinteractiveoverO3b’slinks.Interactivegamingplayerswhowereontheendofageosynchronoussatellitelinkwereoftenkilledbeforetheycouldgeta

shotoff.Nowtheycanbefullparticipantsintheaction.Manybusinessapplicationssuchasremotedatabaseaccessandinteractivetransactionprocessingcontainsoftwaretimersthatdonotadjustforlongdelaysovergeosynchronoussatellites.Examplesincludemanybankingapplicationsandairlinereservationandschedulingapplications.Nowtheseapplicationswillworkforusersthatareinremoteunderservedlocations.

Summary

WehavedescribedthestructureofIP-basedpacketswitchednetworks,

thefunctionsofthevariousprotocollayers,andthecausesoflatencyinpacketswitcheddatanetworks,suchastheInternet.Latencyandoverallthroughputisdominatedbytwofactors,thelengthoftheroutethatthepacketshavetotakebetweensenderandreceiverandtheinteractionbetweentheTCPreliabilityandcongestioncontrolprotocolsandthispathlength.O3bNetworkssatelliteconstellationinamuchlowerMEOorbithassignificantlylowerpathlengthandthereforesignificantlylowerlatencythantraditionalgeosynchronoussatellites.O3b’snetworklatencyandthroughputapproximatethatoffiber-basedterrestrialnetworksandpermitusersinremotelocationsthatarenotservedbyfibertorunmanyapplicationsthatonlyworkoverfiber.





Appendix:

What is latency?

Definition of Latency from Wikipedia:Latencyisatimedelaybetweenthemomentsomethingisinitiated,andthemomentoneofitseffectsbeginsorbecomesdetectable.Thewordderivesfromthefactthatduringtheperiodoflatencytheeffectsofanactionarelatent,meaning“potential”or“notyetobserved”.

Mostpeopleunderstandthatittakestimeforwebpagestoloadandforemailstogetfromyouroutboxtothedestinationinboxandyes,thisisaformoflatency.Butinordertounderstandwhythishappensweneedtothinkaboutlatencyatalowerlevel:

Latencyisatimedelayimpartedbyeachelementinvolvedinthetransmissionofdata.

Theremainderofthispaperwilldiscussthespecificmechanismsinvolvedandhowlatencylimitstheperformanceofadatacommunicationslink.

Networking 101

It’simportantforthereadertounderstandtheverybasicelementsofnetworkingtoproperlygraspthelatencyissue.

Earlynetworkingengineersanticipatedtheneedbeabletohandlethousandstomillionsofusersononecohesivenetwork,andthustheTCP/IPnetworkingmodelwasdeveloped.

ThekeydesignfeatureoftheTCP/IPnetworkingmodelistheconceptofencapsulationwhichistheideaoftakingdataandwrappingitinacommoncontainerforshipping.ThecontainerthatwasdevelopediscalledtheIPDatagram,alsoknownasan“IPPacket”.

TheIPPacketisaverysimplething:aheader,followedbydata.

TheHeadercontainsinformationusedforroutingthepackettothe

destination.Thedatacanbeanyinformationwhichneedstobetransportedsuchasasnippetofstreamingmusicoraportionofemailtraffic.TheexactconstructofthedataportionofanIPPacketisdefinedbythedataprotocolthatisbeingcarried.Dataprotocolswillbediscussedlater.

Tounderstandexactlywherelatencyoccurs,it’svaluabletoknowhowthismostbasicunitofnetworkingdataisbuiltandtransported.ForthisweturntotheOSIModel:





The OSI Model

TheOSImodelwascreatedtodescribetheprocessofturningyourapplicationdataintosomethingthatcanbetransportedontheinternet.

TheupperlayersoftheOSImodeldescribethingsthathappenwithintheapplicationsthatarerunningonthecomputer.Theseapplicationsarewebbrowsers,emailprograms,etc.

Thelowerlayersarewhereinformationtoandfromapplicationsareturnedintodatafortransportonanetwork.Thisiswheredataencapsulationoccursandourbasicnetworkingdataelement–theIPDatagramor“packet”isbuilt.

Thefollowingdiagramshowstheencapsulationprocessinwhat’sknownastheTCP/IPStack.ThepreciseworkingsoftheTCP/IPstackcanbedifferentbetweenvariouscomputeroperatingsystems.Thesedifferencesmayseem

trivialaslongastheprotocolsareimplementedproperlybutwhenseekingtheabsolutehighestlevelsofperformanceit’simportanttoknowthatthenetworkstackimplementationcanbeasignificantcauseofnetworkingperformancevariability.


Application LayerMessageformat,Human-MachineInterfaces

Presentation Layer

Codinginto1sand0s;encryption,compression

Session LayerAuthentication,permissions,sessionrestoration

Transport LayerEnd-to-enderrorcontrol

Network LayerNetworkaddressing;routingorswitching

Data Link LayerErrordetection,flowcontrolonphysicallink

Physical LayerBitstream:physicalmedium,methodofrepresentingbits

T r a n s p o

r t s e r v i c e

U p p e r L a y e r s

7

65

4

3

2

1



www o3bnetworks com 13

UpperLayers

UpperLayers

Upper Layer Message

UpperLayer

Headers

UpperLayer

Headers

TCP/UDPHeader

Upper Layer(Application) Data


TCP/UDP Message

UpperLayer

Headers

TCP/UDPHeader

IPHeader


IP Datagram

UpperLayer

Headers

TCP/UDPHeader

IPHeader

Layer 2Header

Layer 2Header


Layer 2 Frame

1 0 0 1 0 1 1 0 1 1 0 0 1 0 1 1

TCP/UDP

IP

Layer 2

Layer 1

TCP/UDP

IP

Layer 2

Layer 1

The TCP/IP Stack Thetransportofnetworkdataisathreestepprocess:

1. Data from a source application is passed down through the stack. Duringthis process the application data is wrapped into IP Datagrams which arecommonly called “packets”. Packets are then transmitted by the sendingcomputer in the network

2. Packets are passed along the network (purple line) until they reach thedestination computer

3. Packets are received from the network by the destination computer andare passed up through the stack. During this process the application datais extracted and the passed along to the destination application.

TheadditionalencapsulationatLayer2iscalledframing.ThisisthestagewheretheIPDatagramisturnedintobitswhichareappropriateforaparticulartypeofnetwork.

Layer1isthephysicalnetworkmediumconnection.Thislayerhandlestheconversionofthelayer2bitsintoelectrical,optical,orradiosignalsthatcanbetransported.Thenetworkinterface,oftencalledtheNICorNetworkInterfaceCard,canbefiber-optic,copperwire,orawirelessradiointerface.


white paper_latency matters.pdf

Documents