delivering great webrtc on mobile devices
Post on 21-Oct-2014
354 views
DESCRIPTION
Gain in-depth insights about the challenges of implementing WebRTC on iOS and Android mobile devices for both an HTML5 and a native experience. Learn best practices to overcome them.TRANSCRIPT
![Page 1: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/1.jpg)
![Page 2: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/2.jpg)
Delivering Great WebRTC on Mobile Devices
Amir Zmora
![Page 3: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/3.jpg)
Workshop Presenters • Saraj Mudigonda
– ImaginaBon Technologies
• Thomas Sheffler – Weemo
![Page 4: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/4.jpg)
WebRTC on Mobile -‐ IntégraBon, Challenges & SoluBons Saraj Mudigonda ImaginaBon Technologies [email protected]
![Page 5: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/5.jpg)
Agenda • Background • IntegraBon OpBons • V.VoIP Building Blocks • V.VoIP Mobile ApplicaBons • Challenges & SoluBons • Summary
![Page 6: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/6.jpg)
Background Pre-WebRTC
§ Limited tools for developers § Real-time voice/video conferencing algorithms were
limited to only few proprietary native and enterprise applications
§ Phone applications and web applications were two independent identities
PC
Modem
Mobile IP Phone
§ Abundant Tools/APIs available for developers § Easy access to complex real-time voice/video algorithms § Enabling real-time communications on numerous platforms
with different operating systems, sizes, functions and on browsers with no-plugins …. But it does not come free, it brings along multiple challenges
Post-WebRTC
![Page 7: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/7.jpg)
IntegraBon OpBons Over-‐The-‐Top ApplicaBons
Pre-‐Loaded ApplicaBons
Embedded ApplicaBons
NATIVE APPLICATIONS
Web Browser-‐Based ApplicaBons
• Best-‐effort services offered by non-‐operator
• Can be downloaded from Apple Store/Google Play
• Bundled with the plaTorm • Cannot be deleted or
disabled
• Operator driven • Deeply integrated
applicaBons • Part of the NaBve Dialer
![Page 8: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/8.jpg)
V.VoIP Building Blocks V.VoIP Applica4on
WebRTC
Media Engine Audio Coding
Module Video
Conferencing Module
Voice/Video QoS
Audio Pre-‐Processing Module –
AEC, NC, AGC
ICE
STUN
TURN
Security
TLS
SRTP
DTLS
Audio/Video Capture/Rendering
Signaling & User Interface
Network Interface
![Page 9: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/9.jpg)
Mobile V.VoIP ApplicaBons SIP Signaling Voice + Video App
SIP + TradiBonal Voice + Video Codecs Enterprise Telephony
IR.92/IR.94 + AMR Video +Voice over LTE
RCS 5.0/5.1 Rich CommunicaBon Suite
![Page 10: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/10.jpg)
Challenges I. Device Related
1. Ba]ery Life 2. Latency Variability 3. PlaTorm Diversity
II. Network Related 4. Bandwidth Variability 5. Latency Variability 6. Packet Losses
III. Ecosystem Related 7. Network Handoff 8. Interoperability
![Page 11: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/11.jpg)
1. Ba]ery Life • Challenge
– Improved ba]ery life for mobile video/voice conferencing • PotenBal SoluBons
– OpBmize for target mobile applicaBon processors • MIPS intensive algorithms such as echo cancellaBon and codecs should be opBmized for the target
processors thus reducing the overall mw/MHz required – Leverage hardware accelerators wherever available
• Not all plaTorms expose APIs for hardware integraBon – Diminishing returns for frame rate
• Significant difference in perceivable video quality between 10fps and 15fps, but not so much between 15fps and 30fps to jusBfy a 2x increase in ba]ery consumpBon
– Principle of good enough • ResoluBon based on form factor of the devices. No need for HD video telephony burning up CPU and
your hands on a 4 inch screen. • If user really wants highest possible quality, give him a sehng for it with an appropriate warning J
![Page 12: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/12.jpg)
0
10
20
30
40
50
60
70
80
90
100
Voice NB Call Voice WB Call Voice + SW Video Voice + HW Video
CPU USAGE %
1.1 Example CPU Usage %
HD Video
VGA 30fps on Single-‐Core CPU
Hardware accelerators not only offloads CPU and reduces power consumpBon but also enables HD resoluBon Video
![Page 13: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/13.jpg)
2. Latency Variability Propaga4on + Queuing Delay TX Processing Delay RX Processing Delay
Network Interface Delay
Algorithmic Delay –
Encode/AEC
Audio/Video Capture Delay
Network Interface Delay
Algorithmic Delay –
Decode/AJB
Audio/Video Rendering Delay
![Page 14: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/14.jpg)
2.1 Latency Variability • Challenge
– Reduced latency for superior user experience and to avoid talking over each other
• PotenBal SoluBons – Reduced algorithmic delay for voice/video encode/decode and DSP algorithms such as echo cancellaBon
– OpBmized integraBon with audio, video and network interfaces
– Dedicated bearer for real-‐Bme applicaBons -‐ voice and video conferencing
![Page 15: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/15.jpg)
2.2 Example Latency Measurements
0
50
100
150
200
250
300
350
400
1 2 3
Audio Interface Delay
Network Interface Delay
Algorithmic Delay
Handset# Handset# Handset#
• Audio and network interface delay is very device specific and contributes to major chunk of the latency on the client
• Algorithmic delay is dependent on the opBmizaBon of the voice processing algorithms
![Page 16: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/16.jpg)
3. PlaTorm Diversity • Challenge
– Maintain same user-‐experience across different plaTorms • PotenBal SoluBons
– Adapt to different form-‐factors – Auto tuning of AcousBc Echo CancellaBon (AEC) across the devices. AEC characterisBcs are heavily dependent on the plaTorm industrial design (mic and speaker locaBon)
– OpBmizaBon of the DSP algorithms for target processor – OpBmizaBon/integraBon with the plaTorm audio and network interfaces
![Page 17: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/17.jpg)
3.1 Example Audio CharacterisBcs
Mobile Device
Flat Delay (msec)
ERL
Non-‐Linearity
Jiaer (msec)
Record Event Play Event
Handset#1 330 -‐3dB No non-‐linearity 15 8
Handset#2 250 -‐4dB Non-‐Linear 20 5
Handset#3 225 -‐40dB Highly Non-‐Linear 20 5
Legend: • Flat Delay -‐ Delay between the speech sample played ay the APP (sent to audio interface for play back) to the Bme the echo of the sample is received by the APP (data capture provided by the audio interface).
• Non-‐Linearity -‐ Because of the post/pre-‐processing components used in the audio path, the enBre frequency range of the played and echoed speech is not modified uniformly and this nonlinear distorBon impacts the echo cancellaBon
• ERL -‐ Echo return loss -‐ specifies the loss/gain encountered by the signal in the acousBc environment • Ji]er -‐ variaBons in the data capture/play out intervals
![Page 18: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/18.jpg)
4. Bandwidth Variability • Challenge
– Adapt quickly and efficiently to varying bandwidth condiBons • PotenBal SoluBons
– Video AdaptaBon Parameters • Frames Per Second • Video ResoluBon • Bit Rates
– Voice AdaptaBon Parameters • Data Rate AdaptaBon • Codec Change
– Diminishing returns for bandwidth • Marginal to no improvement in quality beyond a certain bandwidth for a given resoluBon and frame rate • Don't use up available bandwidth if it is not required. A judicious choice is a must in order to eliminate blockiness
– Bandwidth control • Maintaining a given bandwidth and the right split of codec bitrate and FEC in the presence of variable loss • A constant bitrate configuraBon of the codec is essenBal. Variable bandwidth can play havoc with pacing/ bandwidth
adaptaBon
![Page 19: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/19.jpg)
4.1 Example Bandwidth AdaptaBon
0
10000
20000
30000
40000
50000
60000
70000
80000
1 1110
2219
3328
4437
5546
6655
7764
8873
9982
11091
12200
13309
14418
15527
16636
17745
18854
19963
21072
22181
23290
24399
25508
26617
27726
28835
29944
31053
32162
33271
34380
35489
36598
37707
38816
39925
41034
42143
43252
44361
45470
46579
47688
48797
49906
51015
52124
53233
54342
55451
56560
57669
58778
59887
60996
Represents Allo]ed Bandwidth in bps
![Page 20: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/20.jpg)
6. Packet Losses • Challenge
– MiBgate packet loss to provide a smooth voice and video experience
• packet loss is very common in wireless environments parBcularly on Wi-‐Fi • MiBgaBon Approach
– AdapBve ji]er buffer and packet loss concealment to miBgate packet losses
– A suitable FEC strategy to combat loss and maintain user experience with minimal latency
– Balancing FEC protecBon, quality, latency especially at low bandwidths
![Page 21: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/21.jpg)
6.1 Example Packet Loss & MiBgaBon
• Packet loss miBgaBon algorithm should help maintain good voice call quality for random and small bursts
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
0% 3% 5% 10% 20% 30% 40% 50%
Voice Qua
lity (PESQ)
Packet Error Rate (PER)
App#1
App#2
App#3
Dropped Call
Good Voice Quality
![Page 22: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/22.jpg)
7. Network Handoff • Challenge
– Provide seamless call handoff • MiBgate dropped calls when moving from one access network to the other
• PotenBal SoluBons – Implement make before break/call anchoring algorithms that can seamlessly handoff when moving from one network to the other
– Some of the 3GPP standards support VCC (Voice Call ConBnuity) for seamless handoff between circuit-‐switched networks and packet networks
![Page 23: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/23.jpg)
7.1 Typical Network Scenario
Dropped calls when changing networks
Home On-‐the-‐road Enterprise
3G
![Page 24: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/24.jpg)
8. Interoperability • Challenge
– Interoperability with legacy systems • PotenBal SoluBons
– Because of the open-‐source nature of WebRTC it is possible to integrate addiBonal codecs and algorithms to WebRTC codebase for interoperability
– Transcoding might be required in the infrastructure to support some legacy systems
![Page 25: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/25.jpg)
Summary • PPA (Performance, Power, AdaptaBon) – the key metrics for opBmal voice/
video experience on mobile devices • Performance
– For improved performance all the parameters discussed have to be tuned carefully for device and networks
• Power – OpBmize for target apps processor, leverage hardware accelerators wherever
available – Find the sweet spot -‐ adopt principle of good enough and diminishing returns for
frame rate • AdaptaBon
– Adapt quickly and efficiently to varying bandwidth condiBons, network packet losses and form-‐factors
![Page 26: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/26.jpg)
WebRTC on Mobile -‐ IntégraBon, Challenges & SoluBons Saraj Mudigonda ImaginaBon Technologies [email protected]
![Page 27: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/27.jpg)
Developing Mobile ApplicaBons using WebRTC Thomas Sheffler Weemo
![Page 28: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/28.jpg)
Outline • A ConstellaBon of OpBons
– browser versus naBve Apps • Mobile ConsideraBons
– WebRTC is hard. Mobile is harder.
• Case study across plaTorms using an SDK – iOS, Android, Javascript
![Page 29: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/29.jpg)
Browser vs NaBve • HTML5+Javascript+WebRTC
– the promise: write once, run everywhere! – Chrome leads, Firefox and Opera closely follow
• NaBve App on iOS and Android – deliver a naBve app experience – what does WebRTC interoperability mean?
![Page 30: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/30.jpg)
WebRTC PlaTorm OpBons • Android
– Browser: Chrome, Firefox, Opera (Mar 2014) • Note: WebRTC is not in the Android WebView
– NaBve: SDKs • Android device market is incredibly fragmented
• iOS – Browser: really none – NaBve: SDKs
• iOS plaTorm is relaBvely well managed • 80% iOS7 (ArsTechnica)
NaBve App
WebView
![Page 31: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/31.jpg)
OpBons Summary
Android iOS Browser Chrome, Firefox, Opera n/a Webview n/a n/a NaBve Yes* Yes*
*use a WebRTC compaBble SDK
![Page 32: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/32.jpg)
Browser Interoperability
h]p://iswebrtcreadyyet.com/
![Page 33: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/33.jpg)
What is WebRTC? • A Suite of Technologies standardized by W3C
– Codecs: VP8, Opus – Transport: RTP, encrypBon – SpecificaBons for describing endpoints (SDP) – Methods for endpoints to connect (ICE, STUN, TURN) – Javascript bindings for
• accessing your camera, microphone • obtaining SDP entry of yourself • rendering media streams described by an SDP entry
![Page 34: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/34.jpg)
What WebRTC is not • A way to idenBfy users
– authenBcaBon • A way to locate other users
– phonebook, user database • A way to iniBate calls to other users
– signaling • A way to see the online status of other users
– presence
![Page 35: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/35.jpg)
What is WebRTC-‐compaBble? • NaBve Apps may be compaBble if
– they can interoperate with a WebRTC browser
• NaBve Interoperability – VP8, Opus – RTP, encrypBon – SDP, ICE, STUN, TURN
• May or may not present bindings similar to Javascript
![Page 36: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/36.jpg)
Compiling WebRTC yourself • Ref: h]p://ninjaneBc.com/how-‐to-‐get-‐started-‐with-‐webrtc-‐and-‐ios-‐without-‐wasBng-‐10-‐hours-‐of-‐your-‐life/
• “The current HEAD has audio and video” [May 2014] • Demo App uses ICE servers apprtc.appspot.com
• My summary: this is really hard
![Page 37: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/37.jpg)
Back-‐end technolgies • Website versus Mobile Backend
– Website development across plaTorms (desktop, iOS, Android) is understood (Rails, Django, Express, etc)
– Mobile App backend development may be unfamiliar • Parse
• Why do you need one? – IdenBfy users (authenBcaBon, roster) – Keep your API keys secret – Protect access to your realBme resources
• TURN relays cost real $$
![Page 38: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/38.jpg)
When Worlds Collide • If you’re coming from the telephony world
– this is not SIP, it’s not a so�phone – webservices bring their own problems – security, cross-‐site request forging, Oauth2, Javascript code is “in the clear”, mobile authenBcaBon
• If you’re coming from Webservices – RTC is not request/response, it’s not ReST – get up to speed on Offer/Accept, SDP, RTP, ICE candidates
![Page 39: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/39.jpg)
IdenBfy your Use Case • Is two-‐way video the applicaBon? • Is it FaceBme? • Is it customer assistance?
– video is a feature, not the focus – does it get out of the way – one-‐way video?
• Is it a mulBway call? (meeBng)
• The way screen real estate and compute resources is used is VERY different from the desktop.
![Page 40: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/40.jpg)
These are very different
• WVGA or HD • 250 kbps – 1 mbps
• Video IS the focus
• 240x180 might be enough • 100 kbps
• Video floats while App conBnues
![Page 41: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/41.jpg)
Is Tablet the same as Mobile? • Tablets do not roam so o�en: more tethered to a basestaBon
• May not have a cellular network • Screensize? Camera?
– Samsung Galaxy Tab 2 10” front camera is 0.3MP • Ba]ery life? CPU? • Shaky camera effect
![Page 42: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/42.jpg)
ICE in One Minute
![Page 43: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/43.jpg)
ICE Candidates
RT
TURN
UDP
![Page 44: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/44.jpg)
ICE Candidates
RT
STUN
UDP
UDP
![Page 45: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/45.jpg)
ICE Candidates
RT
UDP
![Page 46: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/46.jpg)
Cellular network consideraBons • The cellular network does not behave like a consumer home network – do not expect peer-‐to-‐peer to work on 4G
• Handoffs between cells affects IP addresses – sudden changes in network connecBvity [RFC5944: mobilility]
• QuesBon: how does your WebRTC applicaBon behave in the context of mobile networks?
![Page 47: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/47.jpg)
Handoffs from WiFi to Cellular • Mobile apps like FaceBme set consumer expectaBons
• WebRTC technologies do not really address changing network topologies
![Page 48: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/48.jpg)
App Development • How is an App different from a Desktop?
– Apps are sandboxed: a restricted execuBon environment – no shared memory or libraries
• Screensharing – Mobile environments sandbox apps for security – Apps can sample their own pixels, not the pixels of other apps
![Page 49: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/49.jpg)
Power State TransiBons • Foreground/Background
– Can a user depart the video engagement to go look at something else?
– Can the App or Page receive noBficaBons while in the background?
• Browser-‐based implementaBon – background window is not acBve
![Page 50: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/50.jpg)
Use WebRTC-‐compaBble SDK • Abstracts WebRTC endpoints. • AuthenBcates users. Presents calls. • Built on WebRTC technologies
– VP8, Opus – RTP, encrypBon – SDP, ICE, STUN, TURN
![Page 51: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/51.jpg)
Examples using SDK • iOS, Android, Javascript
– iniBalize package – authenBcate – make and receive calls – connect to UI elements
![Page 52: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/52.jpg)
iOS
![Page 53: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/53.jpg)
IniBalize the SDK !static NSString *APP_ID = @"3b04d9add47c”;!!@implementation AppDelegate!- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions!{! ViewController* mainController = ! (ViewController*) self.window.rootViewController;! ! NSError *e;! Weemo *w = [Weemo WeemoWithAppID:APP_ID andDelegate:mainController error:&e];! return YES;!}!!
![Page 54: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/54.jpg)
Mobile AuthenBcaBon
Your ApplicaBon
Auth Controller
Web Service
Cloud-‐based Video Service
Web Service
SDK
Video Service
token
token
requ
est token
(proxy) (uid,client,profile)
token
Auth
token
![Page 55: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/55.jpg)
AuthenBcate static NSString *AUTH_HOST = @"http://192.168.1.233:8000";!!- (IBAction)login:(id)sender {!! dispatch_async(dispatch_get_main_queue(), ^{! self.auth = [[WeemoAuthSimpleClient alloc] init];! [self.auth authWithHost:AUTH_HOST uid:@"testuser1" success:^(NSString *token) {!! [[Weemo instance] authenticateWithToken:token andType:USERTYPE_INTERNAL];!! } failure:^(NSError *e) {! NSLog(@"Login failure:%@", e);! }];! });!}!!
![Page 56: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/56.jpg)
Call, Hangup - (IBAction)call_button:(id)sender {! NSLog(@"call button");! [[Weemo instance] createCall:otherUsername];!}!!- (IBAction)hangup_button:(id)sender {! NSLog(@"hangup button");! ! WeemoCall *call = [[Weemo instance] activeCall];! if (call) {! [call hangup];! }!}!!
![Page 57: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/57.jpg)
Connect call to UI #pragma mark Weemo Delegate!!- (void) weemoCallCreated:(WeemoCall *)call {! NSLog(@"weemoCallCreated:%@", call);! ! [call setViewVideoIn:self.v_videoIn];! [call setViewVideoOut:self.v_videoOut];! ! call.delegate = self;! !}!
UIView v_videoIn
UIView v_videoOut
![Page 58: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/58.jpg)
iOS Summary • We learned how to
– iniBalize package – authenBcate – make and receive calls – connect to UI elements
![Page 59: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/59.jpg)
Android • What is different?
– event bus instead of delegates – decorators to register event listeners
![Page 60: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/60.jpg)
EventBus
• Based on a common component in Google Guava • Our lightweight implementaBon • Decouples event responsibility from class hierarchy
![Page 61: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/61.jpg)
IniBalizaBon public class FullscreenActivity extends Activity {! @Override! public void onStart() {! super.onStart();! Weemo.eventBus().register(this);! Weemo.initialize("APIKEY", this);! }!! @WeemoEventListener! public void onConnected(final ConnectedEvent event) { ... }! ! @WeemoEventListener! public void onAuthenticated(final AuthenticatedEvent event) { ... }!!
![Page 62: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/62.jpg)
Summary • WebRTC in Chrome has taken many man-‐years
– there are more things to think about for mobile
• Leverage a NaBve SDK for your Mobile ApplicaBon
![Page 63: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/63.jpg)
Thank You Please remember to complete an evaluaBon of today’s sessions
![Page 64: Delivering Great WebRTC on Mobile Devices](https://reader031.vdocuments.net/reader031/viewer/2022020720/54461505afaf9fdb2a8b477d/html5/thumbnails/64.jpg)
QuesBons (hidden)