webrtc - · pdf filewebrtc: real-time communica2ons on the web dan burne9, phd ... –...
Post on 12-Feb-2018
220 Views
Preview:
TRANSCRIPT
0
WebRTC:Real-TimeCommunica2onsontheWeb
DanBurne9,PhDh9p://standardsplay.comh9p://allthingsrtc.org
AlanJohnston,PhDRowanUniversity
WebRTC
IETF100SingaporeNovember2017
WebRTC:AJointStandardsEffort• InternetEngineeringTaskForce(IETF)andWorldWideWeb
ConsorFum(W3C)areworkingtogetheronWebRTC• IETF
– Protocols–“bitsonwire”– MainprotocolsarealreadyRFCs,butmanyextensionsin
progress– RTCWEB(Real-TimeCommunicaFonsontheWeb)Working
Groupisthemainfocus,butotherWGsinvolvedaswell• MMUSICWG(ICEandSDPextensions)• TRAMWG(STUNandTURNextensions)• AVTCOREWG(RTPextensions)• RMCATWG(RTPMediaCongesFonAvoidance)
• W3C– APIs–usedbyJavaScriptcodeinHTML5– WEBRTCWGandMediaCaptureTFaremaingroups
1IETF100SingaporeNovember2017
WebRTCProtocols
IP
UDP
WebSocket
SRTP SDP
TURNSTUN
ICE
TCPTLS
TransportLayer
NetworkLayer
ApplicaFonLayer
DTLS
HTTP
UDPDTLS
SCTP
2IETF100SingaporeNovember2017
TheBrowserRTCFuncFon• NewReal-TimeCommunicaFon(RTC)FuncFonbuilt-intobrowsers
• Contains– Audioandvideocodecs– AbilitytonegoFatepeer-
to-peerconnecFons– EchocancellaFon,packet
lossconcealment• InChrome,Firefox,Microso^EDGE,andSafaritoday
3
HTTPorWebSockets
On-the-wireprotocols
RTCAPIsOtherAPIs
NaFveOSServices
WebServer
JavaScript/HTML/CSS
WebBrowser
BrowserRTC
FuncFon
(Signaling)
(MediaorData)
HTTPorWebSockets
SignalingServer
IETF100SingaporeNovember2017
BenefitsofWebRTC
ForDeveloper • Streamlineddevelopment–
oneplaaorm• NATtraversalonlyuses
expensiverelayswhennootherchoice
• Advancedvoiceandvideocodecswithoutlicensing
ForUser• Nodownloadorinstall–
easytouse• Allcommunicaton
encrypted–private• Reliablesession
establishment–“justworks”• Excellentvoiceandvideo
quality• Manymorechoicesforreal-
FmecommunicaFon
4IETF100SingaporeNovember2017
WebRTCTriangle
• BothbrowsersrunningthesamewebapplicaFonfromwebserver
• PeerConnecFonestablishedbetweenthemwiththehelpofthewebserver
WebServer(ApplicaFon)
BrowserM(RunningHTML5ApplicaFon
fromWebServer)
BrowserL(RunningHTML5ApplicaFon
fromWebServer)
PeerConnecFon(Audio,Video,and/orData)
5IETF100SingaporeNovember2017
WebRTCTriangle
• BothbrowsersrunningthesamewebapplicaFonfromwebserver
• PeerConnecFonestablishedbetweenthemwiththehelpofthewebserver("Signaling")
WebServer(ApplicaFon&Signaling)
BrowserM(RunningHTML5ApplicaFon
fromWebServer)
BrowserL(RunningHTML5ApplicaFon
fromWebServer)
PeerConnecFon(Audio,Video,and/orData)
6IETF100SingaporeNovember2017
"Signaling"
WebRTCTrapezoid
• SimilartoSIPTrapezoid• WebServerscommunicateusingSIPorJingleorproprietary
WebServerA(ApplicaFonA)
PeerConnecFon(Audioand/orVideo)
WebServerB(ApplicaFonB)
SIPorJingle
BrowserM(RunningHTML5ApplicaFon
fromWebServerA)
BrowserT(RunningHTML5ApplicaFon
fromWebServerB)
7IETF100SingaporeNovember2017
NATTraversalandSTUN
Peer-to-PeerMediawithWebRTC
WebServer
Internet
Router
BrowserL
HomeWiFiRouter
BrowserT
BrowserM
CoffeeShopWiFiRouter
BrowserD
9
Mediacangodirectlybetweenbrowsers!
IETF100SingaporeNovember2017
WebServer
Internet
RouterwithNAT
BrowserL
HomeWiFiwithNAT
BrowserT
BrowserM
MostbrowsersarebehindNATsontheInternet,whichcomplicatestheestablishmentofpeer-to-peermediasessions.
CoffeeShopWiFiwith
NAT
BrowserD
NATComplicatesPeer-to-PeerMedia
10IETF100SingaporeNovember2017
Internet
HomeWiFiwithNAT
BrowserT192.168.0.6
BrowserM192.168.0.5
“Outside”PublicIPAddress203.0.113.4
“Inside”PrivateIPAddresses192.168.x.x
NAT–NetworkAddressTranslator
11
ANATmapsan“inside”addresstoan“outside”addressallowingmulFplehoststosharethesameIPaddress.
IETF100SingaporeNovember2017
WebServer
Internet
RouterwithNAT
BrowserL
HomeWiFiwithNAT
203.0.113.4
BrowserT
BrowserM192.168.0.5
CoffeeShopWiFiwith
NAT
BrowserD
TURNServer198.51.100.2
BrowsersendsSTUNtestpackettoSTUNservertolearnitspublicIPaddress(addressoftheNAT).
STUNServer198.51.100.9
STUNSessionTraversalUFliFesforNAT
12IETF100SingaporeNovember2017
NoauthenFcaFonofSTUN!
WebServer
Internet
RouterwithNAT
BrowserL
HomeWiFiwithNAT
BrowserT
BrowserM
CoffeeShopWiFiwith
NAT
BrowserD
TURNServerasaMediaRelay
Insomecases,holepunchingfails,andaTURNMediaRelayonthepublicInternetmustbeused.
STUNServer
TURN–TraversalUsingRelayaroundNAT
13IETF100SingaporeNovember2017
What'sNewforWebDevelopers
TwoAPIpieces
• Localmediacapture– Camera,Microphone,<video>element,screen/window
• PeerconnecFon– Mediaanddatabetweentwoclientbrowsers(orarbitrarydevicesusingnaFveAPI)
IETF100SingaporeNovember2017 15
WebRTCusageinbrief
Set Up Peer Connection
Obtain Local Media
Exchange Offer/Answer
Attach Media or Data
Getmoremedia
Allmediaadded
PeerConnecFonestablished
Apachmoremediaordata
Readyforcall
IETF100SingaporeNovember2017 16
WebRTCusageinbrief
• getUserMedia() – Audioand/orvideo– Constraints– Userpermissions
• Browsermustaskbeforeallowingapagetoaccessmicrophoneorcamera
• MediaStream • MediaStreamTrack
Set Up Peer Connection
Obtain Local Media
Exchange Offer/Answer
Attach Media or Data
Getmoremedia
Allmediaadded
PeerConnecFonestablished
Apachmoremediaordata
Readyforcall
IETF100SingaporeNovember2017 17
WebRTCusageinbrief
• RTCPeerConnection – Directmedia– Betweentwopeers– ICEprocessing– SDPprocessing– DTMFsupport– Datachannels– IdenFtyverificaFon– StaFsFcsreporFng
Set Up Peer Connection
Obtain Local Media
Exchange Offer/Answer
Attach Media or Data
Getmoremedia
Allmediaadded
PeerConnecFonestablished
Apachmoremediaordata
Readyforcall
IETF100SingaporeNovember2017 18
WebRTCusageinbrief
• addTrack() – Doesn'tchangemediastate!
• removeTrack() – Dipo!
• createDataChannel() – Dependsontransport
Set Up Peer Connection
Obtain Local Media
Exchange Offer/Answer
Attach Media or Data
Getmoremedia
Allmediaadded
PeerConnecFonestablished
Apachmoremediaordata
Readyforcall
IETF100SingaporeNovember2017 19
WebRTCusageinbrief
• createOffer(), createAnswer()
• setLocalDescription(), setRemoteDescription()
• Offer/answerexchangeneededforthistowork
Set Up Peer Connection
Obtain Local Media
Exchange Offer/Answer
Attach Media or Data
Getmoremedia
Allmediaadded
PeerConnecFonestablished
Apachmoremediaordata
Readyforcall
IETF100SingaporeNovember2017 20
WebRTCusageinbrief
SetUpPeerConnecFon
ObtainLocalMedia
ExchangeSession
DescripFons
ApachMediaorData
Getmoremedia
Apachmoremediaordata
SetUpSignalingChannel
21IETF100SingaporeNovember2017
LocalMedia
Mediacaptureanduse
• navigator. mediaDevices.getUserMedia() – Requestcamera,microphoneaccess– Permissioncheckrequiredforhpppage
• <video>.srcObject = <mediastream>– Directassignment– Worksfor<audio>aswell
• new MediaStream() – Canbuildfromtracksinotherstreams
IETF100SingaporeNovember2017 23
BrowserMediaModel:SourcesandSinks
• Sourcescanbe:– staFc:files,RTSP,<canvas>,etc– dynamic:
• local:webcam,microphone-needgetUserMediatoaccess
• remote:RTCPeerConnecFonorstreamingmedia
• Sinksare<img>,<video>,and<audio>tags– Theycanbesized,whichcancausescalingdependingontheconstraintsonthesource
• RTCPeerConnecFoncanbebothasourceandasink
24IETF100SingaporeNovember2017
BrowserPromptsforPermission
25IETF100SingaporeNovember2017
Local:TracksandStreams
• AMediaStreamTrack – isahandletoonereal-Fmesourceofmediaofonetype(audio/video/depth)
• AMediaStream – representsacollecFonofzeroormoreMediaStreamTracks,ofthesameordifferentmediatypes
– indicatesthatthecollectedtracksaretobekeptsynchronizedtothebestabilityofthebrowser
IETF100SingaporeNovember2017 26
PeerConnecFons
RTCPeerConnecFonAPIs
• Trackstuff– addTrack(),removeTrack()– onaddtrack,onremovetrack
• Offer/answerstuff– createOffer(),createAnswer()– setLocalDescripFon(),setRemoteDescripFon()
• Muchmore
IETF100SingaporeNovember2017 28
SDPOffer/Answer
SessionDescripFonProtocol(SDP)
• UsedbybrowserandWebRTCAPIstodescribemediasession– RTPmediaflows,candidatetransportaddresses,codecinformaFon,mediakeyinginformaFon
• SDPiswidelyusedinSIPVoIPandvideosystems• BrowseruseofSDPisbasedonOffer/Answer– DefinedinJSEP-JavaScriptSessionEstablishmentProtocol
– NegoFaFonproceedsaccordingtoOffer/AnswerStateMachineinnextslides
30IETF100SingaporeNovember2017
SDPOffer/AnswerinRTCPeerConnecFon
• PeerConnecFonAPIstreattheprotocol-levelconfig(SDP)asablob
• createOffer(),createAnswer() – generateSDP
• setLocalDescription(),setRemoteDescription() – tellthebrowserwhichSDPtouse
31IETF100SingaporeNovember2017
Offer/AnswerStateMachine
32
stable
have-remote-offer
have-local-offer
IETF100SingaporeNovember2017
createOffer()
Offer/AnswerStateMachine
33
stable
have-remote-offer
have-local-offer
setLocalDescripFon(localOffer)
IETF100SingaporeNovember2017
createOffer()
Offer/AnswerStateMachine
34
stable
have-remote-offer
have-local-offer
setLocalDescripFon(localOffer)
setRemoteDescripFon(remoteOffer)
IETF100SingaporeNovember2017
SDPoffersentoversignalingchanneltootherbrowser
createOffer()
createAnswer()
Offer/AnswerStateMachine
35
stable
have-remote-offer
have-local-offer
setLocalDescripFon(localAnswer)
setLocalDescripFon(localOffer)
setRemoteDescripFon(remoteOffer)
IETF100SingaporeNovember2017
SDPoffersentoversignalingchanneltootherbrowser
createOffer()
createAnswer()
Offer/AnswerStateMachine
36
stable
have-remote-offer
have-local-offer
setLocalDescripFon(localAnswer)
setRemoteDescripFon(remoteAnswer)
setLocalDescripFon(localOffer)
setRemoteDescripFon(remoteOffer)
IETF100SingaporeNovember2017
SDPanswersentoversignalingchanneltootherbrowser
SDPoffersentoversignalingchanneltootherbrowser
createOffer()
createAnswer()
SDPExtensionsinWebRTC• AnumberofnewSDPextensionsdevelopedforWebRTC.Theyinclude:– BUNDLE–awaytosignalthatasetofm=medialines(e.g.audioandvideo)shouldbemulFplexedoverthesametransportaddressandport• a=group:BUNDLE audio video
– MSID–awaytosignaltheMediaStreamIDusedinJavaScriptinSDP• a=msid
– Source-SpecificApributes–properFesofasource• a=ssrc
37IETF100SingaporeNovember2017
SDPOffer/AnswerUsesSignaling
(notstandardized)
WebRTCSignalingApproaches• Signalingisrequiredforexchangeofcandidatetransport
addressesandSDPblobs• ManyopFons–choiceisuptowebdeveloper
39IETF100SingaporeNovember2017
Example:WebRTCSignalingusingSIP
40
WebServer
SRTPMedia
SIPProxy/RegistrarServer
BrowserM(runningJavaScriptSIPUA)
BrowserT(runningJavaScriptSIPUA)
HTTP(HTML5/CSS/JavaScript)
WebSocket(SIP)
WebSocket(SIP)
HTTP(HTML5/CSS/JavaScript)
• BrowserrunsaSIPUserAgentbyrunningJavaScriptfromWebServer• SRTPmediaconnecFonusesWebRTCAPIs• DetailsinRFC7118thatdefinesSIPtransportoverWebSockets
IETF100SingaporeNovember2017
DataChannel
DataChannelProtocols
• Datachannelprovidesanon-mediachannelbetweenbrowsers• StreamControlTransportProtocol(SCTP)providesreliability,
congesFoncontrol,andmessagedelivery• MulFplexedoversameportsasRTPmedia
42
DTLS
DataChannelData
ProvidescongesFonandflowcontrol
Providessecurity(confidenFality)
ProvidestransportthroughNAT(a^erICEholepunching)
Internet
UDP
DTLS
SCTP
IETF100SingaporeNovember2017
Datachannel
• RTCPeerConnecFon.createDataChannel()– send()methodforasyncmessagesending– onmessagehandlerforasyncmessagereceipt
• BidirecFonalbydefault• CansendstringsorArrayBuffertypes
IETF100SingaporeNovember2017 43
Otherdetails
MediaCodecs
45
• Audiomandatory-to-implement:– Opus(RFC6716):NarrowbandtowidebandInternetaudiocodecforspeechandmusic
– G.711(RFC3551):PCMaudioencodingforPSTNinterworkingandbackwardscompaFbilitywithVoIPsystems
– TelephoneEvents(RFC4733):TransportofDualToneMulFFrequency(DTMF)tones
• Videomandatory-to-implement:– H.264(RFC6184):Commonvideocodec(requireslicensing)
– VP8(RFC6386):Opensourcevideocodec
IETF100SingaporeNovember2017
Security
• MediaisalwaysencryptedinWebRTC• SecureRTP(SRTP)isusedtoassureconfidenFalityandauthenFcaFonofRTPandRTCPpackets
46IETF100SingaporeNovember2017
Privacy
• SRTP,DTLSguaranteeintegrityofcontentbutnottheendpoint
• NewIdenFtyProxyproposedinWebRTC– dra^-iea-rtcweb-security– Allowsuseofthird-partyidenFtyservice(e.g.,FacebookConnect)
– BrowsersignsSRTPkeysusingmaterialfromidenFtyservice,verifiedatotherendusingidenFtyservice
IETF100SingaporeNovember2017 47
NewLow-levelControls
V
A
HowitReallyWorks
IETF100SingaporeNovember2017 49
AV
AA
V
AV
V
AV
Audiom-line
Audiom-line
Videom-line
Videom-line
Stream1 Stream1
Stream2 Stream2
SDPm-line:bidirecFonalSRTPstream
Browser1 Browser2
Senders/Receivers
• WebRTC1.0recentlyadded:– RTCRtpSenders–AhandletoanoutboundRTPstream(one-halfofanm-line)
– RTCRtpReceivers–AhandletoaninboundRTPstream(theotherhalfofanm-line)
• ForeverytracksentorreceivedoveraPeerConnecFonthereisanassociatedsenderorreceiver.ThesegivedirectaccesstoandcontrolovertherelevantRTPstreams.
IETF100SingaporeNovember2017 50
V
A
HowitReallyWorks
IETF100SingaporeNovember2017 51
AV
AA
V
AV
V
AV
Audiom-line
Audiom-line
Videom-line
Videom-line
Stream1 Stream1
Stream2 Stream2
Senders
Senders
Receivers
Receivers
SDPm-line:bidirecFonalSRTPstream
Transceivers
• Butthatwasn'tenough,because– attheSDPleveleverysenderispairedwithareceiver,evenifonlyonehasatrackthatisacFvelyusingit
• EntertheRTCRtpTransceiver.Ateachendpoint – Thereispreciselyonetransceiverforeachm-line.– Thereisasenderandareceiverforeachtransceiver.– Themidofthem-lineisthemid(mediaid)ofthetransceiver.ItisuniqueperPeerConnecFon.
IETF100SingaporeNovember2017 52
V
A
HowitReallyWorks
IETF100SingaporeNovember2017 53
AV
AA
V
AV
V
AV
Audiom-line
Audiom-line
Videom-line
Videom-line
Stream1 Stream1
Stream2 Stream2
Senders
Senders
Receivers
Receivers
Foreachside,eachofthesehasaTransceiver
SDPm-line:bidirecFonalSRTPstream
StatusofWebRTCAPIs• JavaScriptAPIsarebeingstandardizedbyW3C• TwomainspecificaFons– “WebRTC1.0:Real-FmeCommunicaFonBetweenBrowsers”(akaPeerConnecFon)• CandidateRecommendaFon:hpp://www.w3.org/TR/webrtc• Coreisstable,justcleaningupedgecasesnow
– “MediaCaptureandStreams”(akagetUserMedia)• CandidateRecommendaFon:
– hpps://www.w3.org/TR/mediacapture-streams/• Ohsoclose...
• NeedimplementaFonexperienceatthispoint54IETF100SingaporeNovember2017
StatusofWebRTCProtocols
• Tworeferencestocheckforthis:– PrimaryisCullenJennings'WebRTCDependenciesdra^dra^-jennings-rtcweb-deps
– RTCWEBdocumentscanbefollowedasusualhpps://tools.iea.org/wg/rtcweb/
55IETF100SingaporeNovember2017
ToolsandServices• TohelpWebRTCprogramming – Libraries,codesnippets
• EasyRTC,SimpleWebRTC.js,CoTurn(hpps://github.com/coturn/coturn)
– SIPsignaling• sipML5,JsSIP,reSIProcate
• TohelpWebRTCdeployment– Hostedsignalingservices
• PubNub,FireBase– HostedSTUNandTURNservers
• XirSys,Twilio– HostedSFUand/ornetworkopFmizaFon
• Jitsi,SwitchRTC,Agora.io
IETF100SingaporeNovember2017 56
Higher-levelAPIs
• Alternate(higher-level)APIs– Twilio,TokBox– APIDaze– PeerJS,RTCMulFConnecFon
IETF100SingaporeNovember2017 57
Greatonlineresources
• InformaFonalsites– hpp://webrtc.org– http://html5rocks.com/en/tutorials/webrtc/basics– http://webrtchacks.com– hpps://webrtcstandards.info
• Games/demos/apps– hpp://www.cubeslam.com– hpp://shinydemos.com/facekat– hpp://sharefest.me(github.com/Peer5/Sharefest)
IETF100SingaporeNovember2017 58
Whataboutdeployments?• Well-knownservices
– FacebookChat– AmazonMayday(andnowChime)– GoogleHangouts,Duo
• Plaaormembeddings– EveryUCplaaormtoday– Andmosttelecomproviders
• Free(mium)commservices– Crowdcast.io– GoToMeeFngFree(previouslyHu.p)– vLine– Talky.io– Appear.in– Gruveo– Room– Rabbit– GetARoom.io– UberConference
IETF100SingaporeNovember2017 59
top related