cinema columbia internet extensible multimedia architecture

CINEMAColumbia InterNet Extensible

Multimedia Architecture

Kundan [email protected]

Agenda• Basic IP telephony• Infrastructure and

components• Additional services• Current and future work

Joint work with Wenyu Jiang, Jonathan Lennox, Sankaran Narayanan, Henning Schulzrinne, Xiaotao Wu and Min Yan

“Internet Real Time Lab”

Basic IP TelephonyBasic IP TelephonyAudio Packet TransferAudio Packet Transfer

• Digitization (e.g., sampling at 8kHz, 16 bits per sample, i.e, 128 kb/s or 320 bytes per 20 ms)

• Real-time compression/encoding (e.g., G.729A at 8 kb/s)• Transport to remote IP address and port number over

UDP (Why not TCP?)• Processing on receiver side is the reverse

…

Basic IP TelephonyBasic IP TelephonySampling, Quantization, EncodingSampling, Quantization, Encoding

+127

+0

-127

10101111…01101101

Sample at twice the highest voice frequency 2 x 4000=8000

Round off samples to one of 256 levels (introduces noise)

Encode each quantized sample into 8 bit code word

PCM: 8000 x 8 bits = 64 kb/s

Other techniques (differential coding, linear prediction) 2.4 kb/s to 64 kb/s

Basic IP TelephonyBasic IP TelephonyProblems with UDPProblems with UDP

Unreliable UDP Unreliable UDP a) Packet lossb) Out-of-order (very rarely)c) Jitter (delay variation)

1 2 3 5 7 6

1 2 3 4 5 6 7

timeline

Sender

Receiver

(a)(b)

Basic IP Telephony Basic IP Telephony Receive bufferReceive buffer

• Sequence number: to detect packet loss; Just ignore the loss!

• Receive buffer: to absorb jitter

1 2 3 5 7 6

1 2 3 4 5 6 7

Sender

Receiver

8 9 0 2 3 4

8 9 0 1 2 3 4

1 2 71 2 5 7 8 9 0 2 3

1 2 3 5 6 7 8 9 0 2

Basic IP Telephony Basic IP Telephony Timestamp Vs sequence numberTimestamp Vs sequence number

1 2 3 4

1 2 3 4

Sender

Receiver

5 6 7

5 6 7Silence …

t1 t2 t3 t4 t5 t6 t7 t8 t9

Playout time vs packet loss detection

• Silence suppression• Variable length packets

Basic IP Telephony Basic IP Telephony Real-time Transport Protocol - RTPReal-time Transport Protocol - RTP

Encoded Audio

RTP Header

UDP header

IP header

msg

sendto(…, msg, …)recvfrom(…, msg, …)

Sequence number

Optional contributors’ list (CSrc)

Source identifier (SSrc)

Timestamp (proportional to sampling time)

Payload typeCC MV P X

Basic IP Telephony Basic IP Telephony RTP based conferenceRTP based conference

Mixer TranscoderPCMU

PCMU

G.729

G.729

PCMU

PCMU

An example RTP conference (without multicast)

Basic IP Telephony Basic IP Telephony Why do we need signaling?Why do we need signaling?

• What is the IP address of Alice’s host?• What audio encoding can it support?

Bob’s host Alice’s host128.59.19.194

Register asAlice=>128.59.19.194Query for Alice

128.59.19.194

Basic IP Telephony Basic IP Telephony Session Initiation Protocol - SIPSession Initiation Protocol - SIP

• Address based on email ([email protected])

Columbia.eduCisco.com home.com

office.com

Alice

Bob

1. DNS SRV for SIP home.compc1.home.compc1.home.com 129.59.19.140

2. INVITE [email protected]. INVITE [email protected]

(proxy mode)

(2)(3) m2.home.com

Basic IP Telephony Basic IP Telephony SIP Message FormatSIP Message Format

INVITE sip:[email protected] SIP/2.0From: “Bob” <[email protected]>To: “Alice” <[email protected]>Subject: do you know SIP?...

SIP/2.0 200 OKFrom: “Bob” <[email protected]>To: “Alice” <[email protected]>Subject: do you know SIP?...

Request

Response

Basic IP Telephony Basic IP Telephony Session Description Protocol - SDPSession Description Protocol - SDP

AliceBob

INVITE Alice I can support PCMU and G.729Send me audio at 202.16.49.27:6780

OK; I can support PCMUSend me audio at 128.59.19.194:8000

202.16.49.27 128.59.19.194

To port 6780

To port 8000RTP

RTP

Basic IP Telephony Basic IP Telephony SDP Message FormatSDP Message Format

INVITE sip:[email protected] SIP/2.0...v=0o=bob 26172 27162 IN IP4 202.16.49.27s=SIP callc=IN IP4 202.16.49.27t=0 0m=audio 6780 RTP/AVP 0 8 5m=video 6790 RTP/AVP 31

Request

Response

SIP/2.0 200 OK...c=IN IP4 128.59.19.194t=0 0m=audio 8000 RTP/AVP 0 8m=video 0 RTP/AVP 31

Basic IP Telephony Basic IP Telephony Example scenarioExample scenario

Cisco.com

Home.com

Office.com

Alice

Bob

(1)

(2)

(3)

(4)

1. Invite [email protected]. Moved to [email protected]. Invite [email protected]. Invite [email protected]

pc5

Basic IP TelephonyBasic IP TelephonyVoIP GatewaysVoIP Gateways

External line

7043

7040

7042

PBX

Corporate/Campus

InternetLAN

8154

8151

8152

8153

PBX

Another campus

LAN

(Plain Old) TelephoneNetwork

Gateway

Test-bed ArchitectureTest-bed Architecture

SIP proxy,redirectserver

SQLdatabase

sipd

Proxy, Redirect, Registration server.• Authentication• Programmable (SIP- CGI)

OpenSource SQL database: MySQL

http://www.mysql.com

User information:• Contact location• Profile (e.g., password)• Aliases• Address book

System information• Configuration



SQLdatabase

sipd

Web based configuration

Web server

User profile can be modified using web browser. • Creating new user (admin/normal)• Changing profile and contact information (“follow me” service).• Web CGI scripts• Both sipd and web scripts use the database



SQLdatabase

sipd

e*phone

sipc

Software SIP user agents

Hardware Internet (SIP)

phones


Web server

IP phones connected to the departmental LAN. Users are identified by id, e.g., “[email protected]”

Software (sipc) for desktop. Allows audio, video, chat, white board, device control, instant message, presence and desktop sharing.



SQLdatabase

sipd

e*phone

sipc



Phones128.59.19.233


Web server

Phones register themselves with sipd when powered up.

Sipd stores the contact information in the database table:[email protected] => [email protected]

There can be multiple contacts. All registered phones ring, and the first to pick up is connected.

Registration can also be altered from the web interface



SQLdatabase

sipd

e*phone

sipc



Phones128.59.19.233


Web server

(2) Phone rings, the user picks up the call and can talk to the caller.

(1) When somebody calls [email protected], sipd gets the INVITE message and “proxies” the call to the current location.

Another IP phone

Based on user profile, sipd may ask for caller authentication.

e*phone

sipc



phones



SQLdatabase

sipdT1/E1 RTP/SIP

Telephone

SIP/PSTN Gateway

Department PBX Web based

configuration

Web server

Cisco 2600 router with SIP/PSTN gateway connects the departmental LAN with the PBX.

Departmental PBX (Nortel Meridian) connects both internal and external lines to the gateway.

Telephoneswitch

Internal T1

External T1

(Extension:713x)

Dial “8” to reach outside line

e*phone

sipc



phones



SQLdatabase

sipdT1/E1 RTP/SIP

Telephone

SIP/PSTN Gateway


configuration

Web server

(2) The gateway forwards all PSTN calls to sipd; sip:[email protected]

(1) When PBX receives a call for 9397132, it forwards the call to extension 7132. 7130-7139 is assigned to the gateway.

Telephoneswitch

Internal T1(Extension:713x)

Dials 9397132

128.59.19.141

(3) Sipd looks into the dialplan, finds a mapping 7132=>[email protected] and forwards the call to the current location of “hgs”.

e*phone

sipc



phones



SQLdatabase

sipdT1/E1 RTP/SIP

Telephone

SIP/PSTN Gateway128.59.19.28


configuration

Web server(2) Sipd authenticates the caller

and checks permissions. Sipd maps the number 5551212 to [email protected], adding the prefix “8” and the gateway address.

(3) PBX forwards the call to external line.

Telephoneswitch

128.59.19.141

(1) The IP user dials “sip:[email protected]”.

Dial “8” to reach outside lineExternal T1

(4) PSTN user receives the call.

e*phone

sipc



phones



SQLdatabase

sipdT1/E1 RTP/SIP

Telephone

SIP/PSTN Gateway


configuration

Web server

Can use no-DID (direct inward dialing) mode for more numbers.

Telephone number mappings and privileges modifiable from the web

Telephoneswitch

Internal T1

External T1

Additional ServicesAdditional Services

• Advantage: cost savings + new services• “Think of receiving your voicemail messages in an email

that you can later play out in a conference to show it to others”

• Easy integration of email, web, instant messaging, etc.• Open architecture vs Close architecture (traditional

telephones)

ServicesServicesProgrammable serversProgrammable servers

• “Forward my call to office phone during day time and to my mobile in the evening” or “do not allow foo@bar to call me”

• Server side scripts – similar to HTTP• SIP-CGI vs HTTP-CGI• SIP Java Servlet• Call Processing Language (CPL)

e*phone

sipc



phones



SQLdatabase

sipdT1/E1 RTP/SIP

Telephone

SIP/PSTN Gateway


configuration

Web server

Programmable server: SIP-CGI and Call Processing Language (CPL)

Telephoneswitch

Scripts can be uploaded by clients also.

ServicesServicesManagement - SNMPManagement - SNMP

• Constantly monitor the SIP server from remote• Report any problem (e.g., unauthorized registration request)• Use Simple Network Management Protocol (SNMP) –

client server• SIP Management Information Base (MIB) – describe

variables and and parameters for monitoring and control• SIP server implements MIB, can be controlled remotely by

an SNMP client.

e*phone

sipc



phones



SQLdatabase

sipdT1/E1 RTP/SIP

Telephone

SIP/PSTN Gateway


configuration

Web server

SNMP agent for SIP MIB. Allows remote monitoring and control of the SIP server. (e.g., prompt when an unauthorized registration is attempted)

Telephoneswitch

SNMP(Network Management)

ServicesServicesInteroperability with H.323Interoperability with H.323

• H.323 – Proposed by ITU-T• Based on previous H.324/H.320 architecture for video

conferencing• Collection of other specifications (for signaling, control,

transport, etc.)• Defines everything, from supported codecs to controlling

remote camera. (vs SIP, define only what is needed)• NetMeeting – most commonly known client• http://www.packetizer.com - good introduction on H.323

(biased comparison with SIP!!)

e*phone

sipc



phones



SQLdatabase

sipdT1/E1 RTP/SIP

Telephone

SIP/PSTN Gateway


configuration

Web server

Telephoneswitch


SIPH.323convertor

NetMeetingsiph323

H.323

Allows an H323 client (Netmeeting) to use the services of our SIP infrastructure.

ServicesServicesUnified messagingUnified messaging

• Single messaging system – email, voicemail, fax/video, etc.• Voice mail and answering machine – first step• Streaming media for storage and retrieval – Real-Time

Streaming Protocol • RTSP – similar to HTTP, request-response, uses RTP• RealPlayer, QuickTime – popular clients• SIP for signaling of IP telephony calls• Access from telephone using a gateway

e*phone

sipc



phones



SQLdatabase

sipdT1/E1 RTP/SIP

Telephone

SIP/PSTN Gateway


configuration

Web server

Telephoneswitch


SIPH.323convertor

NetMeetingsiph323

H.323

Provides a voice mail and answering machine service to all the registered users. Has web interface for accessing voice mails.

rtspd

SIP/RTSPUnified

messaging

RTSP media server

sipum

Quicktime

RTSP clients

RTSPGeneric media server for playback and recording of messages. Can work with existing RTSP client, Apple’s QuickTime.

ServicesServicesConferencingConferencing

• Multiparty conferencing using SIP and RTP• Models – centralized, decentralized, multicast• Audio mixing, you don’t get back your own audio• Decode-add-encode• Video replication• Centralized SIP control• Centralized RTP mixer

e*phone

sipc



phones



SQLdatabase

sipdT1/E1 RTP/SIP

Telephone

SIP/PSTN Gateway


configuration

Web server

Telephoneswitch


SIPH.323convertor

NetMeetingsiph323

H.323

rtspd

SIP/RTSPUnified

messaging

RTSP media server

sipum

Quicktime

RTSP clients

RTSPCentralized conferencing server for audio and video. Users can join from IP as well as PSTN.

SIP conference

server

sipconf

e*phone

sipc



phones



SQLdatabase

sipdT1/E1 RTP/SIP

Telephone

SIP/PSTN Gateway


configuration

Web server

Telephoneswitch


SIPH.323convertor

NetMeetingsiph323

H.323

rtspd

SIP/RTSPUnified

messaging

RTSP media server

sipum

Quicktime

RTSP clients

RTSP

Netmeeting user dials “sip:[email protected]”

SIP conference

server

sipconf

SIP user dials “sip:[email protected]”

PSTN user dials 1-212-9397139

Sipd maps 7139=> [email protected]

128.59.19.196

ServicesServices(What more?)(What more?)

Okay, SIP can do so many thing… but can it turn on my lamp?

Answer is yes!!

ServicesServicesSIP ExtensionsSIP Extensions

• SIP is just a session signaling protocol.• Can send message to a logical address

(URI) and initiate any session• Extensions to do new things• Instant messaging; presence; event

notification• Call Transfer• Device control (identify device as URI)

e*phone

sipc



phones



SQLdatabase

sipdT1/E1 RTP/SIP

Telephone

SIP/PSTN Gateway


configuration

Web server

Telephoneswitch


SIPH.323convertor

NetMeetingsiph323

H.323

rtspd

SIP/RTSPUnified

messaging

RTSP media server

sipum

Quicktime

RTSP clients

RTSP

SIP conference

server

sipconf

Device gateway allows control of X10 devices. We can control a “lamp” using SIP DO message.

Device GW

X 10

e*phone

sipc



phones

Complete ArchitectureComplete Architecture


SQLdatabase

sipdT1/E1 RTP/SIP

Telephone

SIP/PSTN Gateway


configuration

Web server

Telephoneswitch


SIPH.323convertor

NetMeetingsiph323

H.323

rtspd

SIP/RTSPUnified

messaging

RTSP media server

sipum

Quicktime

RTSP clients

RTSP

SIP conference

server

sipconf

Device GW

X 10

e*phone

sipc



phones

(In)Complete Architecture(In)Complete Architecture


SQLdatabase

sipdT1/E1 RTP/SIP

Telephone

SIP/PSTN Gateway


configuration

Web server

Telephoneswitch


SIPH.323convertor

NetMeetingsiph323

H.323

rtspd

SIP/RTSPUnified

messaging

RTSP media server

sipum

Quicktime

RTSP clients

RTSP

SIP conference

server

sipconf

Device GW

X 10

SIP based Voice XML browser to allow access to voice mail and other services from touch tone phones

Performance measurement and scalability for very large call/session volumes (for sipd, sipconf and rtspd).

Accounting and billing services

Communication portal:Address book, calendar, message board, file sharing, conferencing, etc.

SummarySummaryInternet Telephony is not only telephony on Internet Telephony is not only telephony on

Internet !Internet !

• Basic IP telephony (PC-to-PC)• VoIP gateway (PC-to-phone and phone-to-PC)• Additional services:

• Multimedia• Web and email integration• Instant messaging• Presence and event notification• Conferencing• Collaborative work environment

• Media streaming• Unified messaging

SummarySummaryWorld of ProtocolsWorld of Protocols

• Session Initiation Protocol (SIP) – signaling of Internet sessions

• Real-time Transport Protocol (RTP) – transport of real-time multimedia

• Real Time Streaming Protocol (RTSP) – signaling for multimedia streaming, both playing and recording

• Session Announcement Protocol (SAP) – announcing Internet radio session information

• Multimedia encoding, G.711, GSM, ADPCM, G.729, H.261, H.263, MPEG.

SummarySummaryOur IP telephony test-bedOur IP telephony test-bed

• For implementing new services• For test and performance measurement• For replacing the plain old PBX

SummarySummaryMore InformationMore Information

More information on the system designhttp://www.cs.columbia.edu/~kns10/publication/cinematr.pdfhttp://www.cs.columbia.edu/~kns10/publication/cinematr.pdf

More information on the software components:http://www.cs.columbia.edu/~hgs/softwarehttp://www.cs.columbia.edu/~hgs/software

Questions asked during Questions asked during the talksthe talks

Some of the questions that (I think) were raised and answered by the speaker during the talk.1.Why not use IP address instead of SSRC in RTP?2.What are the authentication mechanisms used in SIP?3.Why is it not LGPL licensed? Are there other LGPL’ed SIP implementations? Does Columbia want to make money out of this?4.What is the capacity of the system? How many users, calls can it handle?5.Actual quality of audio, if we have tested between two IP phones on the real Internet? 6.Do you support any kind of encryption?

Additional slidesAdditional slides

ComponentsComponentsVoIP: Call from POTS to IPVoIP: Call from POTS to IP

1. Convert telephone number to a SIP address (user@domain)2. ENUM: Use DNS lookup

If you dialed 9397042Convert to +12129397042Do DNS lookup for 2.4.0.7.9.3.9.2.1.2.1.arpa.inResult will be a SIP URL, e.g., sip:[email protected]

3. Private mapping within a domain by SIP server7030 - sip:[email protected] - sip:[email protected]

ComponentsComponentsVoIP: Call from IP to POTSVoIP: Call from IP to POTS

1. If you know gateway’s IP address, dial sip:number@gatewayip2. TRIP: Telephone routing over IP – to discover a gateway that

can route your number correctly

Gateway1 +12129397??? freeGateway2 +1408??????? 2¢/minGateway2 +1718??????? 2¢/minGateway3 +1?????????? 4¢/min

ComponentsComponentsInteractive Voice Response - IVRInteractive Voice Response - IVR

PSTN

End userEnd user

IVRIVR1 platform platform• Voice and telephony functions (ASR2, TTS3, DTMF4)• Service logic (application specific)

• Receives incoming PSTN5 call• Responds back with prompts• Accepts user input (DTMF or speech)• Takes action based on user input

(Usually the service logic is programmed for the specific

application, say weather report)

[1] Interactive voice response[2] Automated speech recognition[3] Text to speech [4] Dual tone multi-frequency (touch tone)[5] Public switched telephone network

1-212-8545224

ComponentsComponentsIVR – on internetIVR – on internet

PSTN

End userEnd user

IVR platformIVR platform• Voice and telephony functions (ASR, TTS, DTMF)• Service logic (application specific)

End userEnd userVoice gatewayVoice gateway• Voice and telephony functions

Internet

Web serverWeb server• Service logic

ComponentsComponentsIVR – on InternetIVR – on Internet

PSTN

End userEnd user

Internet

Voice gatewayVoice gateway

Web serverWeb server• Service logic (CGI, servlet, JSP)

• Voice and telephony functions• VoiceXML browser

End userEnd userVXMLVXML HTMLHTML

DB

Multimedia

Audio/grammar

Scripts

Web server

ComponentsComponentsVoiceXML ExampleVoiceXML Example

<form><field name=“drink”> <prompt>Would you like Coffee, Tea, Milk or Nothing.</prompt> <option value=“coffee”>coffee</option> <option value=“tea”>tea</option> <option value=“milk”>milk</option> <option value=“nothing”>nothing</option></field><block> <submit next=“http://…/bartender.cgi” namelist=“drink”/></block></form>

ComponentsComponentsMiscellaneous Miscellaneous

1. Media Streaming2. Multiparty conferencing3. Voice mail and answering machine4. Unified messaging5. Event notification and presence6. Collaborative work environment

cinema columbia internet extensible multimedia architecture

Documents