csci4211: application layer

CSci4211: Application Layer 1

Objectives• Understand

– Service requirements applications placed on network infrastructure

– Protocols distributed applications use to implement applications • Conceptual + implementation aspects of network

application protocols– client server paradigm– peer-to-peer paradigm

• Learn about protocols by examining popular application-level protocols– World Wide Web– Electronic Mail– P2P File Sharing

• Application Infrastructure Services: DNS

2: Application Layer2


Some network apps• e-mail• web• instant messaging• remote login• P2P file sharing• multi-user network

games• streaming stored

video clips

• social networks• voice over IP• real-time video

conferencing• grid computing• cloud computing and

services

2: Application Layer

Creating a network appwrite programs that

– run on (different) end systems

– communicate over network– e.g., web server software

communicates with browser software

No need to write software for network-core devices– Network-core devices do

not run user applications – applications on end

systems allows for rapid app development, propagation

applicationtransportnetworkdata linkphysical



4


Applications and Application-Layer Protocols

Application: communicating, distributed processes– running in network hosts in

“user space”– exchange messages to

implement app– e.g., email, file transfer, the

WebApplication-layer protocols

– one “piece” of an app– define messages exchanged

by apps and actions taken– user services provided by

lower layer protocols





App-layer protocol defines• Types of messages

exchanged, – e.g., request, response

• Message syntax:– what fields in messages

& how fields are delineated

• Message semantics – meaning of information

in fields• Rules for when and how

processes send & respond to messages

Public-domain protocols:

• defined in RFCs• allows for

interoperability• e.g., HTTP, SMTP,

BitTorrentProprietary protocols:• e.g., Skype,

ppstream

6


Application architectures• Client-server

– Including data centers / cloud computing• Peer-to-peer (P2P)• Hybrid of client-server and P2P

7


Application Structure

Programming Paradigms:• Client-Server Model: Asymmetric

– Server: offers service via well defined “interface”– Client: request service– Example: Web

• Peer-to-Peer: Symmetric – Each process is an equal– Example: telephone, p2p file sharing (e.g., Kazaar)

Internet application distributed in nature! - Set of communicating application-level processes (usually on different hosts) provide/implement services

Both require transport of “request/reply”, sharing of data!

2: Application Layer 9

Client-server architectureserver:

– always-on host– permanent IP address– server farms for scaling

clients:– communicate with server– may be intermittently

connected– may have dynamic IP

addresses– do not communicate

directly with each other

client/server

Google Data Centers• Estimated cost of data center: $600M• Google spent $2.4B in 2007 on new

data centers• Each data center uses 50-100

megawatts of power

10


Pure P2P architecture• no always-on server• arbitrary end systems

directly communicate• peers are

intermittently connected and change IP addresses

Highly scalable but difficult to manage

peer-peer

11


Peer-to-Peer Paradigm

Difficulty in implementing “pure” peer-to-peer model?• How to locate your peer?

– Centralized “directory service:” i.e., white pages• Napters

– Unstructured: e.g., “broadcast” your query: namely, ask your friends/neighbors, who may in turn ask their friends/neighbors,

• Freenet – Structured: Distributed hashing table (DHT)

• How do we implement peer-to-peer model?• Is email peer-to-peer or client-server application?

• How do we implement peer-to-peer using client-server model?


Hybrid of client-server and P2P

Skype– voice-over-IP P2P application– centralized server: finding address of remote party: – client-client connection: direct (not through server)

Instant messaging– chatting between two users is P2P– centralized service: client presence

detection/location• user registers its IP address with central

server when it comes online• user contacts central server to find IP

addresses of buddies

13


Network Applications: some jargon

• A process is a program that is running within a host.

• Within the same host, two processes communicate with interprocess communication defined by the OS.

• Processes running in different hosts communicate with an application-layer protocol

• A user agent is an interface between the user and the network application.– Web:browser– E-mail: mail reader– streaming audio/video:

media player


Processes communicatingProcess: program

running within a host.• within same host, two

processes communicate using inter-process communication (defined by OS).

• processes in different hosts communicate by exchanging messages

Client process: process that initiates communication

Server process: process that waits to be contacted

Note: applications with P2P architectures have client processes & server processes

15


Sockets• process sends/receives

messages to/from its socket

• socket analogous to door– sending process shoves

message out door– sending process relies on

transport infrastructure on other side of door which brings message to socket at receiving process

process

TCP withbuffers,variables

socket

host orserver

process

TCP withbuffers,variables

socket

host orserver

Internet

controlledby OS

controlled byapp developer

16


Application Programming Interface

API: application programming interface

• defines interface between application and transport layer

• socket: Internet API– two processes

communicate by sending data into socket, reading data out of socket

Q: how does a process “identify” the other process with which it wants to communicate?– IP address of host

running other process– “port number” - allows

receiving host to determine to which local process the message should be delivered API: (1) choice of transport protocol; (2) ability to fix a few

parameters (lots more on this later)


Addressing Processes• to receive messages,

process must have identifier

• host device has unique 32-bit IP address

• Exercise: use ipconfig from command prompt to get your IP address (Windows)

• Q: does IP address of host on which process runs suffice for identifying the process?– A: No, many processes

can be running on same

• Identifier includes both IP address and port numbers associated with process on host.

• Example port numbers:– HTTP server: 80– Mail server: 25

18


What transport service does an app need?

Data loss• some apps (e.g., audio)

can tolerate some loss• other apps (e.g., file

transfer, telnet) require 100% reliable data transfer

Timing• some apps (e.g.,

Internet telephony, interactive games) require low delay to be “effective”

Throughput some apps (e.g., multimedia)

require minimum amount of throughput to be “effective”

other apps (“elastic apps”) make use of whatever throughput they get

Security Encryption, data integrity, …

19


Transport service requirements of common apps

Application

file transfere-mail

Web documentsreal-time audio/video

stored audio/videointeractive games

financial apps

Data loss

no lossno lossloss-tolerantloss-tolerant

loss-tolerantloss-tolerantno loss

Bandwidth

elasticelasticelasticaudio: 5Kb-1Mbvideo:10Kb-5Mbsame as above few Kbps upelastic

Time Sensitive

nononoyes, 100’s msec

yes, few secsyes, 100’s msecyes and no


Network Transport Services

• Connection-Oriented, Reliable Service– Mimic “dedicated link”– Messages delivered in correct order, without errors– Transport service aware of connection in progress

• Stateful, some “state” information must be maintained– Require explicit connection setup and teardown

• Connectionless, Unreliable Service – Messages treated as independent– Messages may be lost, or delivered out of order– No connection setup or teardown, “stateless”

end host to end host communication services


Internet Transport Protocols

TCP service:• connection-oriented: setup

required between client, server

• reliable transport between sender and receiver

• flow control: sender won’t overwhelm receiver

• congestion control: throttle sender when network overloaded

UDP service:• unreliable data

transfer between sender and receiver

• does not provide: connection setup, reliability, flow control, congestion control

Q:Why UDP?


Internet apps: their protocols and transport protocols

Application

e-mailremote terminal access

Web file transfer

streaming multimedia

remote file serverInternet telephony

Applicationlayer protocol

smtp [RFC 821]telnet [RFC 854]http [RFC 2068]ftp [RFC 959]proprietary(e.g. RealNetworks)NSFproprietary(e.g., Vocaltec)

Underlyingtransport protocol

TCPTCPTCPTCPTCP or UDP

TCP or UDPtypically UDP


Client-Server Paradigm RecapTypical network app has two

pieces: client and server applicationtransportnetworkdata linkphysical


Client:• initiates contact with server

(“speaks first”)• typically requests service from

server, • for Web, client is implemented

in browser; for e-mail, in mail reader

Server:• provides requested service

to client• e.g., Web server sends

requested Web page, mail server delivers e-mail

request

reply


Client-Server: The Web Example

• Web page:– consists of “objects”– addressed by a URL

• Most Web pages consist of:– base HTML page, and– several referenced

objects.• URL has two

components: host name and path name:

• User agent for Web is called a browser:– MS Internet Explorer– Netscape Communicator

• Server for Web is called Web server:– Apache (public domain)– MS Internet Information

Server

www.someSchool.edu/someDept/pic.gif

some jargon


The Web: the HTTP protocolHTTP: hypertext transfer

protocol• Web’s application layer

protocol• client/server model

– client: browser that requests, receives, “displays” Web objects

– server: Web server sends objects in response to requests

• http1.0: RFC 1945• http1.1: RFC 2068

PC runningExplorer

Server running

NCSA Webserver

Mac runningNavigator

http request

http request

http response

http response


The HTTP protocol: morehttp: TCP transport

service:• client initiates TCP

connection (creates socket) to server, port 80

• server accepts TCP connection from client

• http messages (application-layer protocol messages) exchanged between browser (http client) and Web server (http server)

• TCP connection closed

http is “stateless”• server maintains no

information about past client requests

Protocols that maintain “state” are complex!

• past history (state) must be maintained

• if server/client crashes, their views of “state” may be inconsistent, must be reconciled

aside


HTTP ExampleSuppose user enters URL www.someSchool.edu/someDepartment/home.index

1a. http client initiates TCP connection to http server (process) at

www.someSchool.edu. Port 80 is default for http server.

2. http client sends http request message (containing URL) into TCP connection socket

1b. http server at host www.someSchool.edu waiting for TCP connection at port 80. “accepts” connection, notifying client

3. http server receives request message, forms response message containing requested object (someDepartment/home.index), sends message into sockettime

(contains text, references to 10 jpeg images)


HTTP Example (cont.)

5. http client receives response message containing html file, displays html. Parsing html file, finds 10 referenced jpeg objects

6. Steps 1-5 repeated for each of 10 jpeg objects

4. http server closes TCP connection.

time


Non-persistent and persistent connectionsNon-persistent• HTTP/1.0• server parses request,

responds, and closes TCP connection

• 2 RTTs to fetch each object

• Each object transfer suffers from slow start

Persistent• default for HTTP/1.1• on same TCP connection:

server, parses request, responds, parses new request,..

• Client sends requests for all referenced objects as soon as it receives base HTML.

• Fewer RTTs and less slow start.

But most 1.0 browsers useparallel TCP connections.


http message format: request• two types of http messages: request, response• http request message:

– ASCII (human-readable format)

GET /somedir/page.html HTTP/1.0 User-agent: Mozilla/4.0 Accept: text/html, image/gif,image/jpeg Accept-language:fr

(extra carriage return, line feed)

request line(GET, POST,

HEAD commands)header

linesCarriage return,

line feed indicates end of message


http request message: general format


http message format: response

HTTP/1.0 200 OK Date: Thu, 06 Aug 1998 12:00:15 GMT Server: Apache/1.3.0 (Unix) Last-Modified: Mon, 22 Jun 1998 …... Content-Length: 6821 Content-Type: text/html data data data data data ...

status line(protocol

status codestatus phrase)

header lines

data, e.g., requestedhtml file


http response status codes

200 OK– request succeeded, requested object later in this message

301 Moved Permanently– requested object moved, new location specified later in this

message (Location:)400 Bad Request

– request message not understood by server404 Not Found

– requested document not found on this server505 HTTP Version Not Supported

In first line in server->client response message.A few sample codes:


Trying out http (client side) for yourself

1. Telnet to your favorite Web server:Opens TCP connection to port 80(default http server port) at www.eurecom.fr.Anything typed in sent to port 80 at www.eurecom.fr

telnet www.eurecom.fr 80

2. Type in a GET http request:GET /~ross/index.html HTTP/1.0 By typing this in (hit carriage

return twice), you sendthis minimal (but complete) GET request to http server

3. Look at response message sent by http server!


Web and HTTP Summary

GET /index.html HTTP/1.0 HTTP/1.0200 Document followsContent-type: text/htmlContent-length: 2090 -- blank line --HTML text of the Web page

Client Server

Transaction-oriented (request/reply), use TCP, port 80


User-server interaction: authentication

Authentication goal: control access to server documents

• stateless: client must present authorization in each request

• authorization: typically name, password– authorization: header line in

request– if no authorization presented,

server refuses access, sendsWWW authenticate: header line in response

client serverusual http request

msg401: authorization req.

WWW authenticate:

usual http request msg

+ Authorization:lineusual http response msg


+ Authorization:lineusual http response msg

timeBrowser caches name & password sothat user does not have to repeatedly enter it.


User-server interaction: cookies• server sends “cookie” to

client in response mstSet-cookie: 1678453

• client presents cookie in later requestscookie: 1678453

• server matches presented-cookie with server-stored info– authentication– remembering user

preferences, previous choices

client serverusual http request

msgusual http response +

Set-cookie: #


cookie: #usual http response msg


cookie: #usual http response msg

cookie-speccificaction

cookie-specificaction


Electronic MailThree major components: • user agents • mail servers • simple mail transfer protocol:

smtp

User Agent• a.k.a. “mail reader”• composing, editing, reading

mail messages• e.g., Eudora, Outlook, pine,

Netscape Messenger• outgoing, incoming

messages stored on server

user mailbox

outgoing message queue

mailserver

useragent

useragent

useragent

mailserver

useragent

useragent

mailserver

useragent

SMTP

SMTP

SMTP


Electronic Mail: mail serversMail Servers • mailbox contains incoming

messages (yet to be read) for user

• message queue of outgoing (to be sent) mail messages

• smtp protocol between mail servers to send email messages– client: sending mail server– “server”: receiving mail

server

mailserver

useragent

useragent

useragent

mailserver

useragent

useragent

mailserver

useragent

SMTP

SMTP

SMTP


Electronic Mail:SMTP [RFC 821]

• uses tcp to reliably transfer email msg from client to server, port 25

• direct transfer: sending server to receiving server• three phases of transfer

– handshaking (greeting)– transfer of messages– closure

• command/response interaction– commands: ASCII text– response: status code and phrase

• messages must be in 7-bit ASCII


Sample SMTP Interaction S: 220 hamburger.edu C: HELO crepes.fr S: 250 Hello crepes.fr, pleased to meet you C: MAIL FROM: <[email protected]> S: 250 [email protected]... Sender ok C: RCPT TO: <[email protected]> S: 250 [email protected] ... Recipient ok C: DATA S: 354 Enter mail, end with "." on a line by itself C: Do you like ketchup? C: How about pickles? C: . S: 250 Message accepted for delivery C: QUIT S: 221 hamburger.edu closing connection


• telnet servername 25• see 220 reply from server• enter HELO, MAIL FROM, RCPT TO, DATA, QUIT

commands above lets you send email without using email

client (reader)

Try SMTP interaction yourself


SMTP: final words• smtp uses persistent

connections• smtp requires that

message (header & body) be in 7-bit ascii

• certain character strings are not permitted in message (e.g., CRLF.CRLF). Thus message has to be encoded (usually into either base-64 or quoted printable)

• smtp server uses CRLF.CRLF to determine end of message

Comparison with http• http: pull• email: push• both have ASCII

command/response interaction, status codes

• http: each object is encapsulated in its own response message

• smtp: multiple objects message sent in a multipart message


Mail message formatsmtp: protocol for

exchanging email msgsRFC 822: standard for text

message format:• header lines, e.g.,

– To:– From:– Subject:different from smtp

commands!• body

– the “message”, ASCII characters only

header

body

blankline


Message format: multimedia extensions• MIME: multimedia mail extension, RFC 2045, 2056• additional lines in msg header declare MIME content

type

From: [email protected] To: [email protected] Subject: Picture of yummy crepe. MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Type: image/jpeg

base64 encoded data ..... ......................... ......base64 encoded data

multimedia datatype, subtype,

parameter declaration

method usedto encode data

MIME version

encoded data


MIME typesContent-Type: type/subtype; parameters

Text• example subtypes: plain,

html

Image• example subtypes: jpeg,

gif

Audio• example subtypes: basic

(8-bit mu-law encoded), 32kadpcm (32 kbps coding)

Video• example subtypes: mpeg,

quicktime

Application• other data that must be

processed by reader before “viewable”

• example subtypes: msword, octet-stream


Multipart TypeFrom: [email protected] To: [email protected] Subject: Picture of yummy crepe. MIME-Version: 1.0 Content-Type: multipart/mixed; boundary=98766789 --98766789Content-Transfer-Encoding: quoted-printableContent-Type: text/plain

Dear Bob, Please find a picture of a crepe.--98766789Content-Transfer-Encoding: base64Content-Type: image/jpeg

base64 encoded data ..... ......................... ......base64 encoded data --98766789--


Mail access protocols

• SMTP: delivery/storage to receiver’s server• Mail access protocol: retrieval from server

– POP: Post Office Protocol [RFC 1939]• authorization (agent <-->server) and download

– IMAP: Internet Mail Access Protocol [RFC 1730]• more features (more complex)• manipulation of stored msgs on server

– HTTP: Hotmail , Yahoo! Mail, etc.

useragent

sender’s mail server

useragent

SMTP SMTP POP3 orIMAP

receiver’s mail server


POP3 protocolauthorization phase• client commands:

– user: declare username– pass: password

• server responses– +OK– -ERR

transaction phase, client:• list: list message numbers• retr: retrieve message by

number• dele: delete• quit

C: list S: 1 498 S: 2 912 S: . C: retr 1 S: <message 1 contents> S: . C: dele 1 C: retr 2 S: <message 1 contents> S: . C: dele 2 C: quit S: +OK POP3 server signing off

S: +OK POP3 server ready C: user alice S: +OK C: pass hungry S: +OK user successfully logged on


Email SummaryAlice

Messagetransfer agent(MTA)

Messageuser agent(MUA)

outgoing mail queue

Bob Messagetransfer agent(MTA)

Messageuser agent(MUA)

user mailbox

client

server

SMTP over TCP(RFC 821)

port 25POP3 (RFC 1225)/ IMAP (RFC 1064) for accessing mail

SMTP


Application Layer• World Wide Web• Electronic Mail • Domain Name System• P2P File Sharing

Readings: Chapter 2: section 2.1-2.6


Internet: Naming and Addressing

• Names, addresses and routes:According to Shoch (1979)– name: identifies what you want– address: identifies where it is– route: identifies a way to get there

• Internet names and addressesExample Organization

MAC address flat, permanentIP address 128.101.35.34 2-level

Host name afer.cs.umn.edu hierarchical


IP addresses• Two-level hierarchy: network id. + host id.

• (or rather 3-level, subnetwork id.)– 32 bits long usually written in dotted decimal notation

e.g., 128.101.35.34• No two hosts have the same IP address

• host’s IP address may change, e.g., dial-in hosts– a host may have multiple IP addresses– IP address identifies host interface

• Mapping of IP address to MAC (physical) IP done using IP ARP (this is called address resolution)

• one-to-one mapping• Mapping between IP address and host name done

using Domain Name Servers (DNS)• many-to-many mapping


Internet Domain Names• Hierarchical: anywhere

from two to possibly infinity

• Examples: afer.cs.umn.edu, lupus.fokus.gmd.de– edu, de: organization type

or country (a “domain”)– umn, fokus: organization

administering the “sub-domain”

– cs, fokus: organization administering the host

– afer, lupus: host name (have IP address)

. (root)

. com . edu. uk

yahoo.comumn.edu

cs.umn.eduitlabs.umn.edu

afer.cs.umn.eduwww.yahoo.com


Domain Name Resolution and DNS

DNS: Domain Name System:• distributed database

implemented in hierarchy of many name servers

• application-layer protocol host, routers, name servers to communicate to resolve names (address/name translation)– note: core Internet function

implemented as application-layer protocol

– complexity at network’s “edge”

• hierarchy of redundant servers with time-limited cache

• 13 root servers, each knowing the global top-level domains (e.g., edu, gov, com) , refer queries to them

• each server knows the 13 root servers

• each domain has at least 2 servers (often widely distributed) for fault distributed

• DNS has info about other resources, e.g., mail servers


DNS name servers• no server has all name-

to-IP address mappingslocal name servers:

– each ISP, company has local (default) name server

– host DNS query first goes to local name server

authoritative name server:– for a host: stores that host’s

IP address, name– can perform name/address

translation for that host’s name

Why not centralize DNS?• single point of failure• traffic volume• distant centralized

database• maintenance

doesn’t scale!


DNS: Root name servers• contacted by local

name server that can not resolve name

• root name server:– contacts

authoritative name server if name mapping not known

– gets mapping– returns mapping to

local name server• ~ dozen root name

servers worldwide


Simple DNS example

host homeboy.aol.com wants IP address of afer.cs.umn.edu

1. Contacts its local DNS server, dns.aol.com

2. dns.aol.com contacts root name server, if necessary

3. root name server contacts authoritative name server, dns.umn.edu, if necessary

requesting hosthomeboy.aol.com

afer.cs.umn.com

root name server

authorititive name serverdns.umn.edu

local name serverdns.aol.com

1

23

45

6


DNS exampleRoot name server:• may not know

authoritative name server

• may know intermediate name server: who to contact to find authoritative name server


afer.cs.umn.edu

root name server


1

23

4 5

6

authoritative name serverdns.cs.umn.edu

intermediate name serverdns.umn.edu.

7

8


DNS: iterated queriesrecursive query:• puts burden of

name resolution on contacted name server

• heavy load?

iterated query:• contacted server

replies with name of server to contact

• “I don’t know this name, but ask this server”


afer.cs.umass.edu

root name server


1

23

4

5 6

authoritative name serverdns.cs.umn.edu

intermediate name serverdns.umn.edu

7

8

iterated query


DNS: caching and updating records• once (any) name server learns mapping, it

caches mapping– cache entries timeout (disappear) after some time

• update/notify mechanisms under design by IETF– RFC 2136– http://www.ietf.org/html.charters/dnsind-charter.html


DNS recordsDNS: distributed db storing resource records (RR)

• Type=NS– name is domain (e.g.

foo.com)– value is IP address of

authoritative name server for this domain

RR format: (name, value, type,ttl)

• Type=A– name is hostname– value is IP address

• Type=CNAME– name is an alias name for

some “canonical” (the real) name

– value is canonical name

• Type=MX– value is hostname of

mailserver associated with name


DNS protocol, messagesDNS protocol : query and reply messages, both with same message format

msg header• identification: 16 bit #

for query, reply to query uses same #

• flags:– query or reply– recursion desired – recursion available– reply is authoritative


DNS protocol, messages

Name, type fields for a query

RRs in reponseto query

records forauthoritative servers

additional “helpful”info that may be used


DNS Protocol• Query/Reply: use UDP

• Transfer of DNS Records between authoritative and replicated servers: use TCP


P2P File Sharing

Example• Alice runs P2P client

application on her notebook computer

• Intermittently connects to Internet; gets new IP address for each connection

• Asks for “Hey Jude”• Application displays

other peers that have copy of Hey Jude.

• Alice chooses one of the peers, Bob.

• File is copied from Bob’s PC to Alice’s notebook: HTTP

• While Alice downloads, other users uploading from Alice.

• Alice’s peer is both a Web client and a transient Web server.

All peers are servers = highly scalable!


P2P: Centralized Directoryoriginal “Napster” design1) when peer connects, it

informs central server:– IP address– content

2) Alice queries for “Hey Jude”

3) Alice requests file from Bob

centralizeddirectory server

peers

Alice

Bob

1

1

1

12

3


P2P: problems with centralized directory

• Single point of failure• Performance

bottleneck• Copyright

infringement

file transfer is decentralized, but locating content is highly centralized

BitTorrent• Files are shared by many users (as

chunks: around 256KB)• Active participation: peers download

and upload chunks• A torrent is a group of peers that

contain chunks of a file.• Each torrent has a tracker that keeps

track of participating peers


Torrent Setup


Tracker

Alice

p2p_1

p2p_2

p2p_3

p2p_1, p2p3

Register

chunks

chunks 12

3

Trading chunks• What does Alice know?

– Subset of chunks she have.– Which chunks her neighbors have.

• Which chunks she requests first form neighbors?– Use rarest first (chunks with least repeated copies).

• Which requests should Alice respond to?– Priority is given to neighbors supplying her data at the

highest rate.– Utilize unchoked and optimistically unchocked peers.– Tit-for-tat



Query Flooding: Gnutella

• fully distributed– no central server

• public domain protocol

• many Gnutella clients implementing protocol

overlay network: graph• edge between peer X

and Y if there’s a TCP connection

• all active peers and edges is overlay net

• Edge is not a physical link

• Given peer will typically be connected with < 10 overlay neighbors


Gnutella: protocol

Query

QueryHit

Query

Query

QueryHit

Query

Query

QueryHit

File transfer:HTTP Query message

sent over existing TCPconnections peers forwardQuery message QueryHit sent over reversepath

Scalability:limited scopeflooding


Gnutella: Peer Joining1. Joining peer X must find some other peer in

Gnutella network: use list of candidate peers2. X sequentially attempts to make TCP with

peers on list until connection setup with Y3. X sends Ping message to Y; Y forwards Ping

message. 4. All peers receiving Ping message respond

with Pong message5. X receives many Pong messages. It can then

setup additional TCP connections

Distributed Hash Table (DHT)• DHT = distributed P2P database• Database has (key, value) pairs;

– key: ss number; value: human name– key: content type; value: IP address

• Peers query DB with key– DB returns values that match the key

• Peers can also insert (key, value) peers

DHT Identifiers• Assign integer identifier to each peer in

range [0,2n-1].– Each identifier can be represented by n bits.

• Require each key to be an integer in same range.

• To get integer keys, hash original key.– eg, key = h(“Led Zeppelin IV”)– This is why they call it a distributed “hash” table

How to assign keys to peers?• Central issue:

– Assigning (key, value) pairs to peers.• Rule: assign key to the peer that has

the closest ID.• Convention in lecture: closest is the

immediate successor of the key.• Ex: n=4; peers: 1,3,4,5,8,10,12,14;

– key = 13, then successor peer = 14– key = 15, then successor peer = 1

1

3

4

5

810

12

15

Circular DHT (1)

• Each peer only aware of immediate successor and predecessor.

• “Overlay network”

Circle DHT (2)

0001

0011

0100

0101

10001010

1100

1111

Who’s resp for key 1110 ?

I am

O(N) messageson avg to resolvequery, when thereare N peers

1110

1110

1110

1110

1110

1110

Define closestas closestsuccessor

Circular DHT with Shortcuts

• Each peer keeps track of IP addresses of predecessor, successor, short cuts.

• Reduced from 6 to 2 messages.• Possible to design shortcuts so O(log N) neighbors,

O(log N) messages in query

1

3

4

5

810

12

15

Who’s resp for key 1110?

Peer Churn

• Peer 5 abruptly leaves• Peer 4 detects; makes 8 its immediate successor;

asks 8 who its immediate successor is; makes 8’s immediate successor its second successor.

• What if peer 13 wants to join?

1

3

4

5

810

12

15

•To handle peer churn, require each peer to know the IP address of its two successors. • Each peer periodically pings its two successors to see if they are still alive.


P2P Case study: Skype• inherently P2P: pairs

of users communicate.• proprietary

application-layer protocol (inferred via reverse engineering)

• hierarchical overlay with SNs

• Index maps usernames to IP addresses; distributed over SNs

Skype clients (SC)

Supernode (SN)

Skype login server


Peers as relays• Problem when both

Alice and Bob are behind “NATs”. – NAT prevents an outside

peer from initiating a call to insider peer

• Solution:– Using Alice’s and Bob’s

SNs, Relay is chosen– Each peer initiates

session with relay. – Peers can now

communicate through NATs via relay


Exploiting Heterogeneity: KaZaA

• Each peer is either a group leader or assigned to a group leader.– TCP connection between

peer and its group leader.– TCP connections between

some pairs of group leaders.

• Group leader tracks the content in all its children.

ord inary peer

group-leader peer

neighoring re la tionshipsin overlay network


KaZaA: Querying• Each file has a hash and a descriptor• Client sends keyword query to its group leader• Group leader responds with matches:

– For each match: metadata, hash, IP address• If group leader forwards query to other group

leaders, they respond with matches• Client then selects files for downloading

– HTTP requests using hash as identifier sent to peers holding desired file


KaZaA Tricks• Limitations on simultaneous uploads• Request queuing• Incentive priorities• Parallel downloading

For more info: J. Liang, R. Kumar, K. Ross, “Understanding KaZaA,”(available via cis.poly.edu/~ross)


Summary• Application Service Requirements:

– reliability, bandwidth, delay• Client-server vs. Peer-to-Peer Paradigm• Application Protocols and Their Implementation:

– specific formats: header, data; – control vs. data messages– stateful vs. stateless– centralized vs. decentralized

• Specific Protocols:– http– smtp, pop3– dns

csci4211: application layer

Documents