ORTC LibReal-time Communication LibraryIntroduction | August 2014

ORTC Lib - Introduction

What is ORTC Lib?A C++ library and mobile wrappers based on ORTC (Object Real-time Communication) API Public Draft.

API Public Draft: http://ortc.org (upper right hand side)Codebase: https://github.com/openpeer/ortc-lib

1

ORTC - Other related projectsOther ORTC projects dependent on ORTC Lib:

Open Peer: P2P communications projecthttp://openpeer.org

ORTC Node: Node.js implementation (will require ORTC Lib server hardening)https://github.com/openpeer/ortc-node

Other related proposed projects:JavaScript Shims● Upshim (WebRTC 1.0 ORTC)● Downshim (ORTC WebRTC 1.0)

ORTC Project Maintainer

Who are the maintainers?

Hookflash Inc. is the primary sponsor of the ORTC Open Source projects.This Canadian company, made up industry veterans, offers a PaaS for Real-time Communications. Hookflash is also the creator and custodian of Open Peer, an Open Source project for P2P communications.

More on Hookflash: http://hookflash.com

ORTC Lib - IntroductionWhat can it be used for? Enabling real-time communications (Voice, Video, Data Channel) for mobile and web via C++ & JavaScript libraries. The JS libs are out of scope for this project.

What are the goals?The primary goal is to build an open source ORTC client reference implementation that will support all of the ORTC API objects, methods & attributes, using WebRTC open source components for maximum interoperability.

Platforms:Under development: C++, iOS, AndroidUnder consideration: Mac, Windows, Windows Phone

ORTC Lib - Value Statement

Why should I use ORTC Lib?

The libs have been built to the ORTC spec as it has evolved. This is the open source ORTC C++ library that is maintained by the specification founders, editors and authors.

ORTC Lib - Chrome DependenciesA word about Chrome DependenciesThe maintainers wanted to provide an implementation of ORTC providing maximum flexibility for developers.

Signaling: As signaling on the wire is not defined as part of WebRTC 1.0, the the maintainers removed all “on the wire” signalling from the library, (e.g. SDP offer/answer and Libjingle).

ORTC is a WebRTC wire compatible "do as I say" API, that allows the developer to provide their own signalling in Javascript. As a result, a built-in SDP parser is not needed and the PeerConnection ICE implementation was replaced with a custom ICE implementation.

ORTC Lib - Chrome Compatibility

Is ORTC Lib compatible with Chrome?Yes, it is interoperable with Chrome “on the wire” protocols.

ORTC Lib uses Chrome's WebRTC real-time communication engine internally, minus the library formerly known as “Libjingle” (XMPP/Peer Connection, ICE and TURN). Thus RTP, SRTP, and codecs are all compatible, as they are common to Chrome.

ORTC Lib - License

What is the license model?

ORTC Lib is utilizes the FreeBSD open source model (see slide 21 for details).

ORTC Lib - Coverage

Is it compatible with the ORTC Public Draft?

Not yet.

The public draft specification was completed and released on Aug. 20, 2014. As a result, the codebase has not yet achieved parity with the ORTC API Public Draft.

ORTC Lib - Work Completed● Removed Chrome dependencies:

○ WebRTC renderer○ JSEP/SDP ○ PeerConnection○ Libjingle

● Codecs○ VP8 Support (Non SVC)○ ISAC Support○ Opus Support

● Security○ OpenSSL

● NAT Traversal○ Custom ICE/STUN/TURN implementation

(RFC Standard)○ IPv6○ Extended support for Trickle ICE○ Extended support for Continuous

Gathering Mode○ Multiple Network Interfaces (multi-homed)○ ICE Consent Freshness

● DNS○ IDN (Internationalized Domain Names)○ SRV○ A / AAAA

● Mobile○ Reduced energy consumption○ Application Backgrounding○ Reduced footprint

● Assembly optimizations○ YUV conversion○ ...and too much more to list here

● Media○ RTP / RTCP Multiplexing○ A/V Multiplexing○ Enhanced audio routing○ Video rotation control○ Facial recognition (iOS)

● Logging○ Remote○ Local

● Bug fixes

ORTC Lib - Outstanding Work ItemsWhat’s left to do?

● General “tidying up” of the codebase.● RTCRTPSender / Receiver (+ support for "encoding params")● RTCDtlsTransport

○ DTLS demux○ DTLS/SRTP key management

● Data Channel (e.g. SCTP/DTLS/UDP, proprietary RTP data channel, etc.)● ICE

○ Wrappers for ICE Listener/ICE Transport○ Support for "continuous" gather vs "complete" mode○ ICE Transport controller (bandwidth and ICE freezing)○ Custom gathering options (for mobility)

● Media○ MediaStreamTrack○ Advanced VP8 - temporal scalability and simulcast○ H.264 support - RFC 6184 RTP payload transport

● DTMF Sender● Stats● IdP

ORTC Lib - H.264 Work PlanThe goal is to support the RFC 6184 RTP payload format based on the existing Chromium patch and Cisco’s Open H.264 encoder/decoder.

However, the Open H.264 source will NOT be pulled into the codebase (though the build process will support compiling it, if desired).

Initial goal is to support Constrained Baseline Profile, and to interoperate with with Mozilla Firefox.

H.264/SVC (e.g. temporal scalability) and the RFC 6190 RTP payload format will not be supported initially.

ORTC Lib - Extending

Should I contribute or fork?

We encourage contributions & extensions to this library versus forking. By potentially creating multiple incompatible implementations of the ORTC API, forking could cause confusion among developers as well as in the market, adversely impacting the viability of the ORTC initiative. Once forked, recreating a unified code base will be time consuming and difficult.

ORTC Lib - Standards Compliance

How will standards compliance be handled?

The implementers of ORTC Lib intend to produce an implementation faithful to the WebRTC protocol specifications as well as the ORTC API specification. In order to keep the ORTC toolset close to the intent of the specifications, issues discovered will be brought back to the IETF RTCWEB WG and ORTC CG for discussion and resolution.

The ORTC API is designed to support all features of the WebRTC 1.0 API as well as to be backward-compatible with the WebRTC 1.0 API via “shims”.

ORTC Lib - Contributing

How do I contribute?

Want to help? Excellent!

Send an email to contribute@ortc.org with your ideas and we will respond with next steps.

ORTC Lib - Sponsoring

How do I sponsor work?

If you would like to contract the maintainers to do work that would be contributed back to the open source project under BSD license, please email your project ideas to sponsor@ortc.org

ORTC Lib - ThanksThis Open Source work would not have been possible without valuable contributions from the community. The maintainers would like to thank those who contributed for their effort.

Google Inc.- WebRTC contributions

Microsoft Open Tech Inc. & Microsoft Inc.- Demo and Node.js contributions

ORTC Lib - LinksCodebase: https://github.com/openpeer/ortc-libDev Forum: https://groups.google.com/a/ortc.org/forum/#!forum/discuss-ortc

W3C CG: http://www.w3.org/community/ortc/ Current API Specification: http://ortc.org/

Contribute: contribute@ortc.org Sponsor: sponsor@ortc.org

ORTC Deep Dive

ORTC Lib - License DetailsLicense:Copyright (c) 2014, Hookflash Inc.

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the

documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED

TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR

CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,

PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND

ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE

OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

The views and conclusions contained in the software and documentation are those of the authors and should not be interpreted as representing

official policies, either expressed or implied, of the FreeBSD Project.

Library License Usage

WebRTC BSD RTP, Codecs

libvpx BSD VP8

libyuv BSD YUV scaling and conversion

opus BSD Audio codec

usrsctp BSD SCTP

libsrtp BSD SRTP

op-services-cpp BSD Open Peer "services" submodule (ICE, TURN, STUN)

OpenSSL BSD DTLS [replaceable]

curl MIT/X HTTP [optional, HTTP usage outside browser context]

CryptoPP Public Domain Crypto random, Buffering, Hex/base64 transforms [replaceable]

UDNS LGPL DNS lookup (e.g. TURN server) [replaceable]

zsLib BSD Asynchronous interface delegate API, timer, sockets, logging, JSON

boost BSL Smart/weak pointer, Time, Threading, numerical parsing [replaceable]

ORTC Lib - 3rd Party Libraries

ORTC Lib - Chrome Sync Policy

Chrome Code Syncing

While ORTC Lib uses Chromium's WebRTC engine, a fork of the WebRTC engine was performed. Updates from Chromium are synchronized periodically (and delta changes from Chromium are re-applied).

Pull requests are not performed back to Chromium simply due to the amount of manual labour that is required to perform such duties.

Chrome WebRTC Architecture

ORTC Lib - Architecture

Cache SettingsDNS

UDNS

3rd party library

STUN

zsLib

boost

HTTP/HTTPS

curl

replaceable

ICE TURN

DTMF Sender

ICE Transport

Logger

SCTPDTLS Transport

RUDP Security

RSA (public key)

Diffie Hellman

Message Layer

Security

CryptoPP

ICE Transport Controller

ICE Listener

RTP Sender

RTP Receiver

WebRTCOpenSSL

RTP Encoding Params

IdPTrack

(capture/render)

ortc-lib

mandatory componentoptional

Backgrounding

(mobile)

op-services-cpp

usrsctp

ORTC Lib - WebRTC Architecture

Video Engine

Voice Engine

Audio Coding

webrtc library

webrtc media engine

Audio Processing

Audio Device

Audio Mixer

Video Coding

Video Processing

Video Capture

Video Renderer

ICU JsonCPP libc++ libjpeglibsrtp

libvpx libyuv

openssl/nss

opus

sql-lite usrsctp

DTLS Transport

ICE Negotiation

JSEP/SDP Engine

Peer Connection

Media Stream/Track

RTP/RTCP

WebRTC media componentWebRTC library component

removed3rd party library

moved

isac