lecture 3: application layer1 מבוא לרשתות תקשורת : רמת היישום דואר...
Post on 20-Dec-2015
228 views
TRANSCRIPT
Lecture 3: Application Layer 1
מבוא לרשתות תקשורת:רמת היישום דואר אלקטרוני
( DNS)ומערכת השמות (chapter 2 – application)
פרופ' אמיר הרצברג
Computer Networking: A Top Down Approach Featuring the Internet, 2nd edition. Jim Kurose, Keith RossAddison-Wesley, July 2002.
Based on foils by Kurose & Ross ©, see: http://www.aw.com/kurose-ross/
My site: http://AmirHerzberg.com
Lecture 3: Application Layer 2
Application Layer (ch. 2): outline
2.1 Principles of app layer protocols
2.2 Web and HTTP HTTP Overview Connections:
Persistent, pipeline Requests/responses Authorization &
cookies Caching and proxies
2.3 FTP 2.4 Electronic Mail
SMTP, POP3, IMAP 2.5 DNS 2.9 Content
distribution Content distribution
networks P2P file sharing
Lecture 3: Application Layer 3
Network Applications & Application-layer protocolsNetwork Application:
application involving processes in different hosts e.g., e-mail, Web, FTP runs in end systems communicates using…
Application-layer protocols Communication protocol
for network applications Use transport services
(TCP, UDP) Define messages and their
processing (actions) Standard, public or
proprietary
application
transportnetworkdata linkphysical
application
transportnetworkdata linkphysical
application
transportnetworkdata linkphysical
Lecture 3: Application Layer 4
App-layer protocol defines Rules for when and how
processes communicate Type of communication,
e.g., request & response messages
Syntax of messages: what fields, & how fields are delineated
Semantics of the fields, i.e., meaning of information in fields
Public-domain application layer protocols:
defined in RFCs allows for
interoperability e.g., HTTP, SMTPProprietary protocols: e.g., KaZaA
Based on (proprietary) FastTrack protocol
Allowing them to disable `clone` clients
Lecture 3: Application Layer 5
Electronic MailThree major components: Mail user agents (mail client) Mail servers
Some w/ mailboxes Protocols: SMTP (Simple Mail
Transfer Protocol), POP3,…
User Agent (UA) a.k.a. “mail reader/client” e.g., Outlook, Eudora, web-
mail (e.g. Hotmail) Compose mail, send to server Pick-up mail from mailbox in
server
user mailbox
outgoing message queue
mailserver
useragent
useragent
useragent
mailserver
useragent
useragent
mailserver
useragentPOP3
SMTP
SMTP
SMTP
SMTP
SMTP
Lecture 3: Application Layer 6
Mail Pick-Up protocols
Mail Pick-Up (access) protocol: retrieval from server POP3: Post Office Protocol ver. 3 [RFC 1939]
• Authentication (agent <-->server) and download IMAP: Internet Mail Access Protocol [RFC 1730]
• More features, e.g. manage messages in folders• More complex; less common
HTTP: Hotmail , Yahoo! Mail, etc.• Authentication usually by password, cookies
useragent
sender’s mail server
useragent
SMTP(or ___) SMTP Pick-up
protocol
receiver’s mail server
Lecture 3: Application Layer 7
Electronic Mail: Mail ServersMail Servers Message queue of outgoing (to
be sent) mail messages Mailboxes of incoming mail Use SMTP: standard protocol to
send, receive email messages Focus: interoperability and
connectivity Initiated by (user agent or
mail-server) sending mail• `mail transfer client`
To destination or `relay`• `mail transfer server`
Other (proprietary) protocols exist, e.g. Exchange, Notes
Receive mail from User Agent: often also via SMTP…
mailserver
useragent
useragent
useragent
mailserver
useragent
useragent
mailserver
useragent
SMTP
SMTP
SMTP
Lecture 3: Application Layer
Alice'sMUA
MSA
Bob'sMUA
MDA
Internet
AlDomaina.com
Domainb.com
MTAA
MTAB
Different Types of Mail ServersUser Agent
submits (sends) via Mail Submit Agent (MSA)
User’s mailbox kept on Mail Delivery Agent (MDA)
Relaying by Mail Transfer Agents (MTA)
Relaying is tricky…
Lecture 3: Application Layer
Alice'sMUA
MSA
Bob'sMUA
MDA
Internet
AlDomaina.com
Domainb.com
MTAA
MTAB
Mail Relaying Services Basic relaying:
within domain… MUA to MSA MSA to MTA MTA to MDA
Sending Domain’s MTA to Receiving Domain’s MTA
Open Relay: not in sender, receiver domains
Long ago: many open relays (connectivity)
Abused by spammers
So… now `forbidden`
`Legitimate` intermediary services: forwarding, mailing lists,,,
Open Relay MTA
Lecture 3: Application Layer
Alice'sMUA
MSA
Bob'sMUA
MDA
Internet
Carl'sMUA
AlDomaina.com
Domainb.com
MTAA
MTAB
MTAX
Standard Anti-Spam Mechanisms Spam: `unsolicited bulk` Ads/other `often
undesirable` mail Without clear label
A big problem 80% of mail…
No `open relays` Relay in domain Ok
Block port 25 Prevent spam
directly from computer, exit only via MSA/MTA
Authenticate senders
Esp: from outside Quotas So: is email sender
ID authenticated? Not…
© Gila Kaplan)with permission(
Lecture 3: Application Layer 13
The Simple Mail Transfer Protocol (SMTP) [RFC 2821] Uses TCP to reliably transfer email message Initiated by sending agent, called SMTP-client SMTP-server listens on port 25, receives email
MUA to MSA, MSA/MTA to MTA, MTA to MDA three phases of transfer
handshaking (greeting) transfer of messages closure
command/response interaction commands: ASCII text response: status code and phrase
Lecture 3: Application Layer 14
SMTP Phases Handshaking (greeting)
TCP Connection Setup: SMTP-server rejects if overloaded Server hello: sends 220 B.com Server may delay hello (until ready) Client hello: sends Or EHLO (extended hello, supports options)
Transfer of one or more messages: MAIL FROM: sender email (for error report) RCPT TO: recipient that Receiver-SMTP should deliver to Data: RFC 822 message (with headers), then terminator
Closure SMTP closure: C: QUIT, S: 221 TCP connection closure (initiated by SMTP-client)
Lecture 3: Application Layer 15
SMTP Envelope SMTP delivers each message by the sequence:
MAIL FROM: [email protected] RCPT TO: [email protected]
• Repeat for more recipients DATA
Mail envelope: MAIL FROM:, RCPT TO: addresses SMTP uses these fields to route / reject messages
Accept only mail to known users More on this later
SMTP passes mail envelope with the message To the mail delivery system / protocol (e.g. POP3) MDA may copy to message headers (often not)
Distinct from “similar” RFC822 message headers
RCPT TO: [email protected]
RCPT TO: [email protected]
MAIL FROM: [email protected]
Lecture 3: Application Layer 16
Mail message format (RFC 822/2822)SMTP: protocol for
exchanging email msgsRFC 822/2822: standards
for text message format: header lines, e.g.,
To: From: Subject:different from SMTP
commands! Blank line body
the “message”, printable ASCII characters only
How to send non-ASCII?
header
body
blankline
Lecture 3: Application Layer 17
MIME: multimedia mail extension How to send non-ASCII data, e.g. picture, program? Encode: map data bytes to (more) printable-ASCII bytes
E.g.: Hex encoding (x’A3’”A3”); usually use base64 (more compact)
Identify in header: MIME version, encoding, data type, subtype Types: Text, Image, Audio, Video, Application, Multipart
From: [email protected] To: [email protected] Subject: Picture of yummy crepe. MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Type: image/jpeg
base64 encoded data ..... ......................... ......base64 encoded data
multimedia datatype, subtype,
parameter declaration
method usedto encode data
MIME version
encoded data
Multipart: multiple parts (with separators defined in header)
Lecture 3: Application Layer 18
Envelope vs. Message [from, to] S: 220 B.com C: HELO A.com S: 250 OK C: MAIL FROM: <[email protected]> S: 250 OK C: RCPT TO: [email protected] S: 250 OK C: DATA S: 354 Enter mail, end with "." on a line by itself C: from: [email protected] C: to: [email protected], [email protected] C: C: what a wonderful message C: . S: 250 OK C: QUIT S: 221 b.com closing connection
RFC822/2822Email Message Syntax
Lecture 3: Application Layer 19
Envelope RCPT TO: vs. Message to: S: 220 B.com C: HELO A.com S: 250 OK C: MAIL FROM: <[email protected]> S: 250 OK C: RCPT TO: [email protected] S: 250 OK C: DATA S: 354 Enter mail, end with "." on a line by itself C: from: [email protected] C: to: [email protected] C: C: what a wonderful message C: . S: 250 OK C: QUIT S: 221 b.com closing connection
Forwarding services, Mailing-lists, …
Lecture 3: Application Layer 20
Envelope RCPT TO: for Message bcc: S: 220 B.com C: HELO A.com S: 250 OK C: MAIL FROM: <[email protected]> S: 250 OK C: RCPT TO: [email protected] S: 250 OK C: RCPT TO: [email protected] C: DATA S: 354 Enter mail, end with "." on a line by itself C: from: [email protected] C: to: [email protected] C: C: I _still_ did not receive the goods you promised C: . S: 250 OK C: QUIT S: 221 b.com closing connection
Notice also multiple RCPT TO)one per recipient, all in same domain(
Lecture 3: Application Layer 21
SMTP – Basic Entities S: 220 B.com C: HELO A.com S: 250 OK C: MAIL FROM: <[email protected]> S: 250 OK C: RCPT TO: [email protected] S: 250 OK C: DATA S: 354 Enter mail, end with "." on a line by itself C: from: [email protected] C: to: [email protected] C: C: what a wonderful message C: . S: 250 OK C: QUIT S: 221 b.com closing connection
HELO name HELO name
Message originatorMessage originator
Message RecipientMessage Recipient
Envelope RecipientEnvelope Recipient
Envelope originatorEnvelope originator
Server greeting nameServer greeting name
Lecture 3: Application Layer 22
SMTP – Basic Entities (cont’)
S: 220 B.com C: HELO A.com S: 250 OK C: MAIL FROM: <[email protected]> S: 250 OK C: RCPT TO: [email protected] S: 250 OK C: DATA S: 354 Enter mail, end with "." on a line by itself C: from: [email protected] C: to: [email protected] C: C: what a wonderful message C: . S: 250 OK …
HELO nameHELO name
Message originatorMessage originator
Message RecipientMessage Recipient
Envelope RecipientEnvelope Recipient
Envelope originatorEnvelope originator
Server greeting nameServer greeting name
TCP connection parms: SMTP-Client (33.4.5.6) connect to SMTP-Server-IP (35.6.7.8)
Lecture 3: Application Layer 23
Email Sender Identities SMTP-Client identifies only relaying agent (MTA)
Often unrelated to email sender identity MUA displays only the From: header (usually)
RFC822 From: asked for message to be sent Most MSAs allow use of any From: identity
To allow transparent mobility But, spoofing email becomes trivial… Partial alternate solution: use Reply-To: header
Several other sender-related identifiers: Sender: “entered message into email stream” Resent-from, resent-sender:
• RFC2822: identifies party (time, etc.) resending message• Specifically not intended for use for forwarding (by user or by
service)• But used, e.g. by sendmail, mailing-lists…
Bounce address to report errors (next topic)
Lecture 3: Application Layer 24
Some Email Originator Identities
… C: MAIL FROM: <[email protected]> S: 250 OK C: RCPT TO: [email protected] S: 250 OK C: DATA S: 354 Enter mail, end with "." on a line by itself C: from: [email protected] C: sender: [email protected] C: reply-to: [email protected] C: to: [email protected] C: C: what a wonderful message C: . S: 250 OK …
Bounce-To address
Lecture 3: Application Layer 25
SMTP Error Handling Reliability: Key goal of SMTP If agent cannot deliver before sending OK:
Recipient not known, sender not allowed, queue full, etc.
Signal to Sender-SMTP by error code Permanent (5xx) or transient (4xx) error codes
On error in transfer to next mail server: Transient error (4xx): retransmit after reasonable delay May try alternate route (use secondary MX record)
If agent cannot deliver (after sending OK): Send Delivery Status Notifications (DSN) to bounce
address DSN has empty MAIL FROM field
Lecture 3: Application Layer 26
Bounce Address: MAIL FROM or Return-Path?
Bounce address: where to send DSN Usually (agent is server): MAIL FROM:
address Bounce from recipient’s MUA:
No envelope, MAIL FROM: Bounce address copied to Return-Path: header
Lecture 3: Application Layer 27
SMTP Traceback Delivery Status Notifications (DSN) depend on
bounce address (MAIL FROM / Return-Path:) What if bounce error is corrupted?
Possibly intentionally (to hide traces – e.g. spoofed email) Solution: Traceback message route Origin selects, puts unique Message-ID: Each SMTP-receiver <rec> adds header to
messagereceived: from <src> by <rec> for <dest>;
<date> <src>, <rec> : domain names and IP addresses
SMTP servers maintain logs to confirm / deny received: records Traceback conflict one of the pair lies
Lecture 3: Application Layer 28
Relaying in SMTP Usually: Relaying is Inside sender / receiver
domain Receiving Relay: recipient in the domain
Authorized-sender Relay: from authenticated domain user, or from authenticated MTA / MSA in domain
Or: Direct from sender domain to receiver domain From outgoing MTA in sender domain (e.g. a.com)
To incoming MTA in recipient’s domain (e.g. b.com)
Traceable Relaying: Inside (sender/recipient) domain, or direct (sender to receiver domain)
Allows responding to traceback conflict
Lecture 3: Application Layer 29
Relaying and Traceback Conflict Assume: Authorized-sender or receiving
relaying Direct from sender domain to receiver domain
Or: Inside sender / receiver domain
Consider traceback (e.g. of spoofed email)
No conflict using log files and traceback information, identify “real” source of email
Traceback conflict: If among servers within domain: detect corruption!
If sender domain denies: may refuse future connections
Caution: Can also be corruption of“our” server or a spoofed connection
Lecture 3: Application Layer
Email Relaying Services Relaying within
domain is fine MUA to MSA MSA to MTA MTA to MDA
Sender-Receiver relaying is fine From sending MTA To receiving MTA
Open Relay: Not in sender and
receiver domains
AliceMUAAliceMUA
MSAMSA
BobMUABobMUA
MDAMDA
Domaina.com
Domainb.com
MTAA
MTAA
MTAB
MTAB
Open Relay MTA
Open Relay MTA
Lecture 3: Application Layer 31
Open Relay Considered Harmful Open Relay: MTA that relays from and to outside
its domain Years Ago: Open relaying was critical for
connectivity (e.g. before DNS were adopted) Today: Often abused to send bad email (e.g.
spam) Upon traceback, Open Relay claims it got email from xxx Is xxx bad? Or is the Open Relay lying and faulty?
Most MTAs are not Open Relays Relay only inside domain or from sender’s domain
Many MTAs refuse email from Open Relays
Lecture 3: Application Layer 32
Chapter 2 outline
2.1 Principles of app layer protocols clients and servers app requirements –
choosing transport srvc 2.6 Socket
programming 2.2 Web and HTTP
HTTP Overview Connections:
Persistent, pipeline Requests/responses Authorization &
cookies Caching and proxies
2.3 FTP 2.4 Electronic Mail
SMTP, POP3, IMAP 2.5 DNS 2.8 Building a Web
server 2.9 Content
distribution Content distribution
networks P2P file sharing
Lecture 3: Application Layer 33
DNS: Domain Name System
People: many identifiers: Name, ID#, e-mail
Internet hosts, routers: IP address (32 bit) - used
for addressing datagrams• One per network adapter
(may have two) Domain names: biu.ac.il Fully Qualified Domain
Name (FQDN) or “name”, e.g., gaia.cs.umass.edu
Names : used by humans
Q: map between IP addresses and names ?
Domain Name System: distributed names database
implemented in hierarchy of many name servers
application-layer protocol host, routers, name servers communicate to resolve names (address/name translation) Application-layer
protocol for Basic Internet function
complexity at network’s “edge”
Used to distribute other data as well
Lecture 3: Application Layer 34
DNS
Why not centralize DNS? single point of failure traffic volume Proximity (close to
client) Maintenance and
security
doesn’t scale!
DNS services Hostname to IP
address translation Host aliasing
Canonical and alias names
Mail server aliasing Load distribution
Replicated Web servers: set of IP addresses for one canonical name
Lecture 3: Application Layer 35
TLD and Authoritative Servers Top-level domain (TLD) servers: responsible
for com, org, net, edu, etc, and all top-level country domains uk, fr, ca, jp. Network solutions maintains servers for com TLD Educause for edu TLD
Authoritative DNS servers: organization’s DNS servers, providing authoritative hostname to IP mappings for organization’s servers (e.g., Web and mail). Maintained by organization or service provider
Lecture 3: Application Layer 36
Root DNS Servers
com DNS servers org DNS servers edu DNS servers
poly.eduDNS servers
umass.eduDNS servers
yahoo.comDNS servers
amazon.comDNS servers
pbs.orgDNS servers
Distributed, Hierarchical Database
Client wants IP for www.amazon.com; 1st approx: Query root server to find com DNS server Query com DNS server (TLD) to get
amazon.com DNS server Query amazon.com DNS server to get IP
address for www.amazon.com
Lecture 3: Application Layer 37
Local Name Server
Queries processed by a Local Name Server, not by DNS client
Does not strictly belong to hierarchy Each ISP (residential ISP, company,
university) has (at least) one. Also called “default name server”
When a host makes a DNS query, query is sent to its local DNS server Acts as a proxy, forwards query into
hierarchy.
Lecture 3: Application Layer 38
DNS: Root name servers contacted by local name server that can not resolve name root name servers:
if name mapping not known: get from authoritative name server
return mapping to local name server
b USC-ISI Marina del Rey, CAl ICANN Marina del Rey, CA
e NASA Mt View, CAf Internet Software C. Palo Alto, CA
i NORDUnet Stockholm
k RIPE London
m WIDE Tokyo
a NSI Herndon, VAc PSInet Herndon, VAd U Maryland College Park, MDg DISA Vienna, VAh ARL Aberdeen, MDj NSI )TBD( Herndon, VA
13 root name servers worldwide
Lecture 3: Application Layer 39
requesting hostcis.poly.edu
gaia.cs.umass.edu
root DNS server
local DNS serverdns.poly.edu
1
23
4
5
6
authoritative DNS serverdns.cs.umass.edu
78
TLD DNS server
Example
Host at cis.poly.edu wants IP address for gaia.cs.umass.edu
Lecture 3: Application Layer 40
requesting hostcis.poly.edu
gaia.cs.umass.edu
root DNS server
local DNS serverdns.poly.edu
1
2
45
6
authoritative DNS serverdns.cs.umass.edu
7
8
TLD DNS server
3
Recursive queries
recursive query: puts burden of
name resolution on contacted name server
heavy load?
iterated query: contacted server
replies with name of server to contact
“I don’t know this name, but ask this server”
Lecture 3: Application Layer 41
DNS: caching and updating records DNS mappings are kept and sent in resource records
(RR) DNS servers cache mappings (RRs)
cache entries timeout (disappear) after ttl seconds The ttl value is specified in each DNS record
Several different types of RR Used for domain name resolution And for other purposes… where the distribution
among DNS servers is useful• E.g. distribute black list of spam mail servers• Own domain (e.g. easy.10$.com), run DNS server,
define arbitrary DNS `subtree` under it: policy.easy.10$.com,…
Lecture 3: Application Layer 42
DNS records
DNS: distributed db storing resource records (RR)
Type=NS name is domain (e.g. foo.com) value is IP address of
authoritative name server for this domain
Type=TXT value is some text
RR format: (name, value, type, ttl)
Type=A name is hostname value is IP address
Type=CNAME name is alias name for (real)
“canonical” name value www.ibm.com is really servereast.backup2.ibm.com
Type=MX value is name of mail server associated
with name
Other types… Problem: support by servers
Lecture 3: Application Layer 43
Example: e-mailDNSServer
SenderMTA
RecipientMTA
Sender Recipient
Send mail
MX requestb.com
MX responsemail.b.com
Deliver mail
Helo sender.com 250 OkMAIL FROM: [email protected] 250 OKRCPT TO: [email protected] 250 OKDate: Sun, 25 Jul 2004 17:30Subject: Hello messageHello world
SRC IP=xx.xx.xx.xx
rDNS requestxx.xx.xx.xx
rDNS responsesender.com
Lecture 3: Application Layer 44
DNS protocol, messagesDNS protocol : query and reply messages, both with same message formatUses UDP (except for large replies) [Why?]
Components: Header:
identification: 16 bits (match reply to query)
• Why is this needed?
4 flags: query/reply, recursion desired, recursion available, authoritative reply
Number of questions and reply records
Lecture 3: Application Layer 45
Chapter 2 outline
2.1 Principles of app layer protocols clients and servers app requirements –
choosing transport srvc 2.6 Socket
programming 2.2 Web and HTTP
HTTP Overview Connections:
Persistent, pipeline Requests/responses Authorization &
cookies Caching and proxies
2.3 FTP 2.4 Electronic Mail
SMTP, POP3, IMAP 2.5 DNS 2.9 Content
distribution Content distribution
networks P2P file sharing
Lecture 3: Application Layer 46
Content Distribution Problem Clients are spread over the Net Using one centralized server is bad…
One server cannot serve all clients Shipping data across Net: overhead, congestion
Solutions: Distribute data as DNS records
• Used e.g. for black lists of spammers Use HTTP: data kept by proxies Load Balancing: many servers with same IP
• Hardware switch selects server randomly• Preserve TCP connections to same server
DNS gives IP address from pool or by location• Both load balancing and by proximity (close to client)
Lecture 3: Application Layer 47
Content distribution networks (CDNs) CDN company provides
content distribution services
The content providers are the CDN customers.
Content replication CDN company installs
hundreds of CDN servers throughout Internet in lower-tier ISPs, close
to users CDN replicates its
customers’ content in CDN servers. When provider updates content, CDN updates servers
origin server in North America
CDN distribution node
CDN serverin S. America CDN server
in Europe
CDN serverin Asia
Lecture 3: Application Layer 48
CDN example
origin server www.foo.com distributes HTML Replaces: http://www.foo.com/sports/ruth.gif
with
http://www.cdn.com/www.foo.com/sports/ruth.gif
HTTP request for
www.foo.com/sports/sports.html
DNS query for www.cdn.com
HTTP request for
www.cdn.com/www.foo.com/sports/ruth.gif
1
2
3
Origin server
CDNs authoritative DNS server
NearbyCDN server CDN company
cdn.com distributes gif files uses its authoritative
DNS server to route redirect requests Also for load balancing
Lecture 3: Application Layer 49
Application architectures
Client-server Peer-to-peer (P2P) Hybrid of client-server and P2P
Lecture 3: Application Layer 50
Client-server architectureserver:
always-on host permanent IP address server farms for scaling Sends responses to
requests
clients: Initiate communication with
server (sends requests) may be intermittently
connected may have static or dynamic
IP addresses do not communicate
directly with each other
Lecture 3: Application Layer 51
Pure P2P architecture
no always-on server arbitrary end systems
directly communicate peers are intermittently
connected and change IP addresses
example: Gnutella
Highly scalable
But difficult to manage
Lecture 3: Application Layer 52
Hybrid P2P: centralized directory
original “Napster” design
1) when peer connects, it informs central server: IP address content
2) Alice queries for “Hey Jude”, receives Bob IP address
3) Alice requests file from Bob
centralizeddirectory server
peers
Alice
Bob
1
1
1
12
3
Lecture 3: Application Layer 53
Hybrid of client-server and P2PClients register, find peer via management
server(s)Napster:
File transfer P2P File search centralized:
• Peers register content at central server• Peers query same central server to locate content
Instant messaging: Chatting between two users is P2P Presence detection/location centralized:
• User registers its IP address with central server when it comes online
• User contacts central server to find IP addresses of buddies