cinema columbia internet extensible multimedia architecture kundan singh [email protected]...
TRANSCRIPT
CINEMAColumbia InterNet Extensible
Multimedia Architecture
Kundan Singh
Agenda• Basic IP telephony• Infrastructure and
components• Additional services• Current and future work
Joint work with Wenyu Jiang, Jonathan Lennox, Sankaran Narayanan, Henning Schulzrinne, Xiaotao Wu and Min Yan
“Internet Real Time Lab”
Basic IP TelephonyBasic IP TelephonyAudio Packet TransferAudio Packet Transfer
• Digitization (e.g., sampling at 8kHz, 16 bits per sample, i.e, 128 kb/s or 320 bytes per 20 ms)
• Real-time compression/encoding (e.g., G.729A at 8 kb/s)• Transport to remote IP address and port number over
UDP (Why not TCP?)• Processing on receiver side is the reverse
…
Basic IP TelephonyBasic IP TelephonySampling, Quantization, EncodingSampling, Quantization, Encoding
+127
+0
-127
10101111…01101101
Sample at twice the highest voice frequency 2 x 4000=8000
Round off samples to one of 256 levels (introduces noise)
Encode each quantized sample into 8 bit code word
PCM: 8000 x 8 bits = 64 kb/s
Other techniques (differential coding, linear prediction) 2.4 kb/s to 64 kb/s
Basic IP TelephonyBasic IP TelephonyProblems with UDPProblems with UDP
Unreliable UDP Unreliable UDP a) Packet lossb) Out-of-order (very rarely)c) Jitter (delay variation)
1 2 3 5 7 6
1 2 3 4 5 6 7
timeline
Sender
Receiver
(a)(b)
Basic IP Telephony Basic IP Telephony Receive bufferReceive buffer
• Sequence number: to detect packet loss; Just ignore the loss!
• Receive buffer: to absorb jitter
1 2 3 5 7 6
1 2 3 4 5 6 7
Sender
Receiver
8 9 0 2 3 4
8 9 0 1 2 3 4
1 2 7
1 2 5 7 8 9 0 2 3
1 2 3 5 6 7 8 9 0 2
Basic IP Telephony Basic IP Telephony Timestamp Vs sequence numberTimestamp Vs sequence number
1 2 3 4
1 2 3 4
Sender
Receiver
5 6 7
5 6 7Silence …
t1 t2 t3 t4 t5 t6 t7 t8 t9
Playout time vs packet loss detection
• Silence suppression• Variable length packets
Basic IP Telephony Basic IP Telephony Real-time Transport Protocol - RTPReal-time Transport Protocol - RTP
Encoded Audio
RTP Header
UDP header
IP header
msg
sendto(…, msg, …)recvfrom(…, msg, …)
Sequence number
Optional contributors’ list (CSrc)
Source identifier (SSrc)
Timestamp (proportional to sampling time)
Payload typeCC MV P X
Basic IP Telephony Basic IP Telephony RTP based conferenceRTP based conference
Mixer Transcoder
PCMU
PCMU
G.729
G.729
PCMU
PCMU
An example RTP conference (without multicast)
Basic IP Telephony Basic IP Telephony Why do we need signaling?Why do we need signaling?
• What is the IP address of Alice’s host?• What audio encoding can it support?
Bob’s host Alice’s host128.59.19.194
Register asAlice=>128.59.19.194Query for Alice
128.59.19.194
Basic IP Telephony Basic IP Telephony Session Initiation Protocol - SIPSession Initiation Protocol - SIP
• Address based on email ([email protected])
Columbia.edu
Cisco.com home.com
office.com
Alice
Bob
1. DNS SRV for SIP home.compc1.home.compc1.home.com 129.59.19.140
2. INVITE [email protected]. INVITE [email protected]
(proxy mode)
(2)(3) m2.home.com
Basic IP Telephony Basic IP Telephony SIP Message FormatSIP Message Format
INVITE sip:[email protected] SIP/2.0From: “Bob” <[email protected]>To: “Alice” <[email protected]>Subject: do you know SIP?...
SIP/2.0 200 OKFrom: “Bob” <[email protected]>To: “Alice” <[email protected]>Subject: do you know SIP?...
Request
Response
Basic IP Telephony Basic IP Telephony Session Description Protocol - SDPSession Description Protocol - SDP
AliceBob
INVITE Alice I can support PCMU and G.729Send me audio at 202.16.49.27:6780
OK; I can support PCMUSend me audio at 128.59.19.194:8000
202.16.49.27 128.59.19.194
To port 6780
To port 8000RTP
RTP
Basic IP Telephony Basic IP Telephony SDP Message FormatSDP Message Format
INVITE sip:[email protected] SIP/2.0...v=0o=bob 26172 27162 IN IP4 202.16.49.27s=SIP callc=IN IP4 202.16.49.27t=0 0m=audio 6780 RTP/AVP 0 8 5m=video 6790 RTP/AVP 31
Request
Response
SIP/2.0 200 OK...c=IN IP4 128.59.19.194t=0 0m=audio 8000 RTP/AVP 0 8m=video 0 RTP/AVP 31
Basic IP Telephony Basic IP Telephony Example scenarioExample scenario
Cisco.com
Home.com
Office.com
Alice
Bob
(1)
(2)
(3)
(4)
1. Invite [email protected]. Moved to [email protected]. Invite [email protected]. Invite [email protected]
pc5
Basic IP TelephonyBasic IP TelephonyVoIP GatewaysVoIP Gateways
External line
7043
7040
7042
PBX
Corporate/Campus
InternetLAN
8154
8151
8152
8153
PBX
Another campus
LAN
(Plain Old) TelephoneNetwork
Gateway
Test-bed ArchitectureTest-bed Architecture
SIP proxy,redirectserver
SQLdatabase
sipd
Proxy, Redirect, Registration server.• Authentication• Programmable (SIP- CGI)
OpenSource SQL database: MySQL
http://www.mysql.com
User information:• Contact location• Profile (e.g., password)• Aliases• Address book
System information• Configuration
Test-bed ArchitectureTest-bed Architecture
SIP proxy,redirectserver
SQLdatabase
sipd
Web based configuration
Web server
User profile can be modified using web browser. • Creating new user (admin/normal)• Changing profile and contact information (“follow me” service).• Web CGI scripts• Both sipd and web scripts use the database
Test-bed ArchitectureTest-bed Architecture
SIP proxy,redirectserver
SQLdatabase
sipd
e*phone
sipc
Software SIP user agents
Hardware Internet (SIP)
phones
Web based configuration
Web server
IP phones connected to the departmental LAN. Users are identified by id, e.g., “[email protected]”
Software (sipc) for desktop. Allows audio, video, chat, white board, device control, instant message, presence and desktop sharing.
Test-bed ArchitectureTest-bed Architecture
SIP proxy,redirectserver
SQLdatabase
sipd
e*phone
sipc
Software SIP user agents
Hardware Internet (SIP)
Phones128.59.19.233
Web based configuration
Web server
Phones register themselves with sipd when powered up.
Sipd stores the contact information in the database table:[email protected] => [email protected]
There can be multiple contacts. All registered phones ring, and the first to pick up is connected.
Registration can also be altered from the web interface
Test-bed ArchitectureTest-bed Architecture
SIP proxy,redirectserver
SQLdatabase
sipd
e*phone
sipc
Software SIP user agents
Hardware Internet (SIP)
Phones128.59.19.233
Web based configuration
Web server
(2) Phone rings, the user picks up the call and can talk to the caller.
(1) When somebody calls [email protected], sipd gets the INVITE message and “proxies” the call to the current location.
Another IP phone
Based on user profile, sipd may ask for caller authentication.
e*phone
sipc
Software SIP user agents
Hardware Internet (SIP)
phones
Test-bed ArchitectureTest-bed Architecture
SIP proxy,redirectserver
SQLdatabase
sipdT1/E1 RTP/SIP
Telephone
SIP/PSTN Gateway
Department PBX Web based
configuration
Web server
Cisco 2600 router with SIP/PSTN gateway connects the departmental LAN with the PBX.
Departmental PBX (Nortel Meridian) connects both internal and external lines to the gateway.
Telephoneswitch
Internal T1
External T1
(Extension:713x)
Dial “8” to reach outside line
e*phone
sipc
Software SIP user agents
Hardware Internet (SIP)
phones
Test-bed ArchitectureTest-bed Architecture
SIP proxy,redirectserver
SQLdatabase
sipdT1/E1 RTP/SIP
Telephone
SIP/PSTN Gateway
Department PBX Web based
configuration
Web server
(2) The gateway forwards all PSTN calls to sipd; sip:[email protected]
(1) When PBX receives a call for 9397132, it forwards the call to extension 7132. 7130-7139 is assigned to the gateway.
Telephoneswitch
Internal T1
(Extension:713x)
Dials 9397132
128.59.19.141
(3) Sipd looks into the dialplan, finds a mapping 7132=>[email protected] and forwards the call to the current location of “hgs”.
e*phone
sipc
Software SIP user agents
Hardware Internet (SIP)
phones
Test-bed ArchitectureTest-bed Architecture
SIP proxy,redirectserver
SQLdatabase
sipdT1/E1 RTP/SIP
Telephone
SIP/PSTN Gateway128.59.19.28
Department PBX Web based
configuration
Web server(2) Sipd authenticates the caller
and checks permissions. Sipd maps the number 5551212 to [email protected], adding the prefix “8” and the gateway address.
(3) PBX forwards the call to external line.
Telephoneswitch
128.59.19.141
(1) The IP user dials “sip:[email protected]”.
Dial “8” to reach outside lineExternal T1
(4) PSTN user receives the call.
e*phone
sipc
Software SIP user agents
Hardware Internet (SIP)
phones
Test-bed ArchitectureTest-bed Architecture
SIP proxy,redirectserver
SQLdatabase
sipdT1/E1 RTP/SIP
Telephone
SIP/PSTN Gateway
Department PBX Web based
configuration
Web server
Can use no-DID (direct inward dialing) mode for more numbers.
Telephone number mappings and privileges modifiable from the web
Telephoneswitch
Internal T1
External T1
Additional ServicesAdditional Services
• Advantage: cost savings + new services• “Think of receiving your voicemail messages in an email
that you can later play out in a conference to show it to others”
• Easy integration of email, web, instant messaging, etc.• Open architecture vs Close architecture (traditional
telephones)
ServicesServicesProgrammable serversProgrammable servers
• “Forward my call to office phone during day time and to my mobile in the evening” or “do not allow foo@bar to call me”
• Server side scripts – similar to HTTP• SIP-CGI vs HTTP-CGI• SIP Java Servlet• Call Processing Language (CPL)
e*phone
sipc
Software SIP user agents
Hardware Internet (SIP)
phones
Test-bed ArchitectureTest-bed Architecture
SIP proxy,redirectserver
SQLdatabase
sipdT1/E1 RTP/SIP
Telephone
SIP/PSTN Gateway
Department PBX Web based
configuration
Web server
Programmable server: SIP-CGI and Call Processing Language (CPL)
Telephoneswitch
Scripts can be uploaded by clients also.
ServicesServicesManagement - SNMPManagement - SNMP
• Constantly monitor the SIP server from remote• Report any problem (e.g., unauthorized registration request)• Use Simple Network Management Protocol (SNMP) –
client server• SIP Management Information Base (MIB) – describe
variables and and parameters for monitoring and control• SIP server implements MIB, can be controlled remotely by
an SNMP client.
e*phone
sipc
Software SIP user agents
Hardware Internet (SIP)
phones
Test-bed ArchitectureTest-bed Architecture
SIP proxy,redirectserver
SQLdatabase
sipdT1/E1 RTP/SIP
Telephone
SIP/PSTN Gateway
Department PBX Web based
configuration
Web server
SNMP agent for SIP MIB. Allows remote monitoring and control of the SIP server. (e.g., prompt when an unauthorized registration is attempted)
Telephoneswitch
SNMP(Network Management)
ServicesServicesInteroperability with H.323Interoperability with H.323
• H.323 – Proposed by ITU-T• Based on previous H.324/H.320 architecture for video
conferencing• Collection of other specifications (for signaling, control,
transport, etc.)• Defines everything, from supported codecs to controlling
remote camera. (vs SIP, define only what is needed)• NetMeeting – most commonly known client• http://www.packetizer.com - good introduction on H.323
(biased comparison with SIP!!)
e*phone
sipc
Software SIP user agents
Hardware Internet (SIP)
phones
Test-bed ArchitectureTest-bed Architecture
SIP proxy,redirectserver
SQLdatabase
sipdT1/E1 RTP/SIP
Telephone
SIP/PSTN Gateway
Department PBX Web based
configuration
Web server
Telephoneswitch
SNMP(Network Management)
SIPH.323convertor
NetMeetingsiph323
H.323
Allows an H323 client (Netmeeting) to use the services of our SIP infrastructure.
ServicesServicesUnified messagingUnified messaging
• Single messaging system – email, voicemail, fax/video, etc.• Voice mail and answering machine – first step• Streaming media for storage and retrieval – Real-Time
Streaming Protocol • RTSP – similar to HTTP, request-response, uses RTP• RealPlayer, QuickTime – popular clients• SIP for signaling of IP telephony calls• Access from telephone using a gateway
e*phone
sipc
Software SIP user agents
Hardware Internet (SIP)
phones
Test-bed ArchitectureTest-bed Architecture
SIP proxy,redirectserver
SQLdatabase
sipdT1/E1 RTP/SIP
Telephone
SIP/PSTN Gateway
Department PBX Web based
configuration
Web server
Telephoneswitch
SNMP(Network Management)
SIPH.323convertor
NetMeetingsiph323
H.323
Provides a voice mail and answering machine service to all the registered users. Has web interface for accessing voice mails.
rtspd
SIP/RTSPUnified
messaging
RTSP media server
sipum
Quicktime
RTSP clients
RTSPGeneric media server for playback and recording of messages. Can work with existing RTSP client, Apple’s QuickTime.
ServicesServicesConferencingConferencing
• Multiparty conferencing using SIP and RTP• Models – centralized, decentralized, multicast• Audio mixing, you don’t get back your own audio• Decode-add-encode• Video replication• Centralized SIP control• Centralized RTP mixer
e*phone
sipc
Software SIP user agents
Hardware Internet (SIP)
phones
Test-bed ArchitectureTest-bed Architecture
SIP proxy,redirectserver
SQLdatabase
sipdT1/E1 RTP/SIP
Telephone
SIP/PSTN Gateway
Department PBX Web based
configuration
Web server
Telephoneswitch
SNMP(Network Management)
SIPH.323convertor
NetMeetingsiph323
H.323
rtspd
SIP/RTSPUnified
messaging
RTSP media server
sipum
Quicktime
RTSP clients
RTSP
Centralized conferencing server for audio and video. Users can join from IP as well as PSTN.
SIP conference
server
sipconf
e*phone
sipc
Software SIP user agents
Hardware Internet (SIP)
phones
Test-bed ArchitectureTest-bed Architecture
SIP proxy,redirectserver
SQLdatabase
sipdT1/E1 RTP/SIP
Telephone
SIP/PSTN Gateway
Department PBX Web based
configuration
Web server
Telephoneswitch
SNMP(Network Management)
SIPH.323convertor
NetMeetingsiph323
H.323
rtspd
SIP/RTSPUnified
messaging
RTSP media server
sipum
Quicktime
RTSP clients
RTSP
Netmeeting user dials “sip:[email protected]”
SIP conference
server
sipconf
SIP user dials “sip:[email protected]”
PSTN user dials 1-212-9397139
Sipd maps 7139=> [email protected]
128.59.19.196
ServicesServices(What more?)(What more?)
Okay, SIP can do so many thing… but can it turn on my lamp?
Answer is yes!!
ServicesServicesSIP ExtensionsSIP Extensions
• SIP is just a session signaling protocol.• Can send message to a logical address
(URI) and initiate any session• Extensions to do new things• Instant messaging; presence; event
notification• Call Transfer• Device control (identify device as URI)
e*phone
sipc
Software SIP user agents
Hardware Internet (SIP)
phones
Test-bed ArchitectureTest-bed Architecture
SIP proxy,redirectserver
SQLdatabase
sipdT1/E1 RTP/SIP
Telephone
SIP/PSTN Gateway
Department PBX Web based
configuration
Web server
Telephoneswitch
SNMP(Network Management)
SIPH.323convertor
NetMeetingsiph323
H.323
rtspd
SIP/RTSPUnified
messaging
RTSP media server
sipum
Quicktime
RTSP clients
RTSP
SIP conference
server
sipconf
Device gateway allows control of X10 devices. We can control a “lamp” using SIP DO message.
Device GW
X 10
e*phone
sipc
Software SIP user agents
Hardware Internet (SIP)
phones
Complete ArchitectureComplete Architecture
SIP proxy,redirectserver
SQLdatabase
sipdT1/E1 RTP/SIP
Telephone
SIP/PSTN Gateway
Department PBX Web based
configuration
Web server
Telephoneswitch
SNMP(Network Management)
SIPH.323convertor
NetMeetingsiph323
H.323
rtspd
SIP/RTSPUnified
messaging
RTSP media server
sipum
Quicktime
RTSP clients
RTSP
SIP conference
server
sipconf
Device GW
X 10
e*phone
sipc
Software SIP user agents
Hardware Internet (SIP)
phones
(In)Complete Architecture(In)Complete Architecture
SIP proxy,redirectserver
SQLdatabase
sipdT1/E1 RTP/SIP
Telephone
SIP/PSTN Gateway
Department PBX Web based
configuration
Web server
Telephoneswitch
SNMP(Network Management)
SIPH.323convertor
NetMeetingsiph323
H.323
rtspd
SIP/RTSPUnified
messaging
RTSP media server
sipum
Quicktime
RTSP clients
RTSP
SIP conference
server
sipconf
Device GW
X 10
SIP based Voice XML browser to allow access to voice mail and other services from touch tone phones
Performance measurement and scalability for very large call/session volumes (for sipd, sipconf and rtspd).
Accounting and billing services
Communication portal:Address book, calendar, message board, file sharing, conferencing, etc.
SummarySummaryInternet Telephony is not only telephony on Internet Telephony is not only telephony on
Internet !Internet !
• Basic IP telephony (PC-to-PC)• VoIP gateway (PC-to-phone and phone-to-PC)• Additional services:
• Multimedia• Web and email integration• Instant messaging• Presence and event notification• Conferencing• Collaborative work environment
• Media streaming• Unified messaging
SummarySummaryWorld of ProtocolsWorld of Protocols
• Session Initiation Protocol (SIP) – signaling of Internet sessions
• Real-time Transport Protocol (RTP) – transport of real-time multimedia
• Real Time Streaming Protocol (RTSP) – signaling for multimedia streaming, both playing and recording
• Session Announcement Protocol (SAP) – announcing Internet radio session information
• Multimedia encoding, G.711, GSM, ADPCM, G.729, H.261, H.263, MPEG.
SummarySummaryOur IP telephony test-bedOur IP telephony test-bed
• For implementing new services• For test and performance measurement• For replacing the plain old PBX
SummarySummaryMore InformationMore Information
More information on the system designhttp://www.cs.columbia.edu/~kns10/publication/cinematr.pdfhttp://www.cs.columbia.edu/~kns10/publication/cinematr.pdf
More information on the software components:http://www.cs.columbia.edu/~hgs/softwarehttp://www.cs.columbia.edu/~hgs/software
Questions asked during Questions asked during the talksthe talks
Some of the questions that (I think) were raised and answered by the speaker during the talk.1.Why not use IP address instead of SSRC in RTP?2.What are the authentication mechanisms used in SIP?3.Why is it not LGPL licensed? Are there other LGPL’ed SIP implementations? Does Columbia want to make money out of this?4.What is the capacity of the system? How many users, calls can it handle?5.Actual quality of audio, if we have tested between two IP phones on the real Internet? 6.Do you support any kind of encryption?
ComponentsComponentsVoIP: Call from POTS to IPVoIP: Call from POTS to IP
1. Convert telephone number to a SIP address (user@domain)2. ENUM: Use DNS lookup
If you dialed 9397042Convert to +12129397042Do DNS lookup for 2.4.0.7.9.3.9.2.1.2.1.arpa.inResult will be a SIP URL, e.g., sip:[email protected]
3. Private mapping within a domain by SIP server7030 - sip:[email protected] - sip:[email protected]
ComponentsComponentsVoIP: Call from IP to POTSVoIP: Call from IP to POTS
1. If you know gateway’s IP address, dial sip:number@gatewayip2. TRIP: Telephone routing over IP – to discover a gateway that
can route your number correctly
Gateway1 +12129397??? freeGateway2 +1408??????? 2¢/minGateway2 +1718??????? 2¢/minGateway3 +1?????????? 4¢/min
ComponentsComponentsInteractive Voice Response - IVRInteractive Voice Response - IVR
PSTN
End userEnd user
IVRIVR1 platform platform• Voice and telephony functions (ASR2, TTS3, DTMF4)• Service logic (application specific)
• Receives incoming PSTN5 call• Responds back with prompts• Accepts user input (DTMF or speech)• Takes action based on user input
(Usually the service logic is programmed for the specific
application, say weather report)
[1] Interactive voice response[2] Automated speech recognition[3] Text to speech [4] Dual tone multi-frequency (touch tone)[5] Public switched telephone network
1-212-8545224
ComponentsComponentsIVR – on internetIVR – on internet
PSTN
End userEnd user
IVR platformIVR platform• Voice and telephony functions (ASR, TTS, DTMF)• Service logic (application specific)
End userEnd userVoice gatewayVoice gateway• Voice and telephony functions
Internet
Web serverWeb server
• Service logic
ComponentsComponentsIVR – on InternetIVR – on Internet
PSTN
End userEnd user
Internet
Voice gatewayVoice gateway
Web serverWeb server
• Service logic (CGI, servlet, JSP)
• Voice and telephony functions• VoiceXML browser
End userEnd userVXMLVXML HTMLHTML
DB
Multimedia
Audio/grammar
Scripts
Web server
ComponentsComponentsVoiceXML ExampleVoiceXML Example
<form><field name=“drink”> <prompt>Would you like Coffee, Tea, Milk or Nothing.</prompt> <option value=“coffee”>coffee</option> <option value=“tea”>tea</option> <option value=“milk”>milk</option> <option value=“nothing”>nothing</option></field><block> <submit next=“http://…/bartender.cgi” namelist=“drink”/></block></form>