mail protocol, postfix and mail security · n postfix n qmail n ms exchange n cc:mail n lotus notes...
TRANSCRIPT
Mail Protocol, Postfix and Mail security
How Email Appears to Works
2
How Email Appears to Work
How Email Really Works
3
How Email Really Works
Message Format• Envelop
– Routing information for the "postman"
• Message Header – Sender– Recipients (simple, lists, copies, blind copies) – Other fields of control (date, subject)
• Message Body – Free text – Structured document (i.e.: MIME)
Message Format From: GZ Kabir <[email protected]>
To: Diep Kong < [email protected] >
Cc: Moin < [email protected] >
Subject: How Internet mail works
Hi Kong,I'm going to be running a course on ...
ü Format was originally defined by RFC 822 in 1982 ü Now superseded by RFC 2822ü Message consists of
ü Header lines ü A blank line ü Body lines
Message Format • Embedded MUA uses inter-process call to send to MTA
• Freestanding MUA uses SMTP to send mail
• Headers added by the MUA before sending
From: GZ Kabir <[email protected]>
To: Diep Kong < [email protected] >
Cc: Moin < [email protected] >
Subject: How Internet mail works
Hi Kong,I'm going to be running a course on ...
• Headers added by MTAs
Mail Delivery Agent (MDA) / Mail Transfer Agent (MTA)
From: GZ Kabir <[email protected]> To: Diep Kong < [email protected] > Cc: Moin < [email protected] > Subject: How Internet mail works
…..
A message in Transit • A message is transmitted with an envelope:
– MAIL FROM:<[email protected]>RCPT TO:<[email protected]>
• The envelope is separate from the RFC 2822 message • Envelope (RFC 2821) fields need not be the same as the
header (RFC 2822) fields • MTAs are (mainly) concerned with envelopes
– Just like the Post Office... • Error (“bounce”) messages have null senders or Postmaster’s
– MAIL FROM:<>
An SMTP Session telnet mail7i.protonmail.ch 25 220 mail7i.protonmail.ch ESMTP Postfix ...
EHLO mail-pg0-f54.google.com 250- mail7i.protonmail.ch ... 250-SIZE 10485760 250-PIPELINING250 HELP
MAIL FROM:<[email protected]> 250 OK RCPT TO:<[email protected]> 250 Accepted DATA 354 Enter message, ending with “.”
(continued >>>>)
Received: from ...From: ... To: ...etc...250 OK id=10sPdr-00034H-00 quit221 mail-pg0-f54.google.com closing conn... SMTP return codes 2xx OK3xx send more data 4xx temporary failure 5xx permanent failure
DNS Resolution and Transfer Process To find the recipient's IP address and mailbox, the MTA must drill down through the DNS system, which consists of a set of servers distributed across the Internet beginning with the root name servers
root servers refer requests for a given domain to the root nameservers that handle requests for that tld– MTA can bypass this step because it has already knows which domain nameservers handle requests
for these .tlds e.g. bofh.imasks the appropriate DNS server which Mail Exchange (MX) servers have knowledge of the subdomain or local host in the email address
DNS server responds with an MX record: a prioritized list of MX servers for this domain
To the DNS server, the server that accepts messages is an MX server. When is transferring messages, it is called an MTA.
MTA contacts the MX servers on the MX record in order of priority until it finds the designated host for that address domain
sending MTA asks if the host accepts messages for the recipient's username at that domain (i.e., [email protected]) and transfers the message
Firewalls, Spam and Virus Filters • An email encountering a firewall may
be tested by spam and virus filters before it is allowed to pass inside the firewall
• filters test to see if the message qualifies as spam or malware
• If the message contains malware, the file is usually quarantined and the sender is notified
• If the message is identified as spam, it will probably be deleted without notifying the sender.
16
Firewalls, Spam and Virus Filters
An email encountering a firewallmay be tested by spam andvirus filters before it is allowedto pass inside the firewall
filters test to see if themessage qualifies as spam ormalware
If the message containsmalware, the file is usuallyquarantined and the sender isnotified
If the message is identified asspam, it will probably bedeleted without notifying thesender.
Troubleshooting Email Issues • transient failures
– If a transient error occurs, the MTA will hang onto the message, periodically retrying the delivery until it either succeeds or fails, or until the MTA decides that the transient issue is really a permanent condition.
• permanent failures – If the MTA cannot deliver the message (it has received a fatal error
message or failed to complete the transfer after repeated attempts), it bounces the message back to the sender. If the sender is a mailing list, the bounce may be handled by automated bounce-handling software.
Concept of Mail Protocols
Component of Email system❏ Mail Transport Agent/Message Transfer Agent (MTA)❏ Mail User Agent (MUA)
❏ Mail Delivery Agent(MDA)
MTAThe actual mail transfer is done through message transfer agents (MTAs). To send mail, a system must have the client MTA, and to receive mail, a system must have a server MTA. The formal protocol that defines the MTA client and server in the Internet is called Simple Mail Transfer Protocol (SMTP)
SMTP uses commands and responses to transfer mail between an MTA client and MTA server.
SMTP Mail transfer Flow
MTA connection setup
MTA Connection Setup..Contd..Sender opens TCP connection with receiver □ Once connected, receiver identifies itself□ 220 service ready□ Sender identifies itself□ HELO <domain>□ Receiver accepts sender’s identification□ 250OK□ If mail service not available, step 2 above becomes: □ 421 service not available
SMTP Exchange of command response pair
Message TransferSender may send one or more messages to receiver
Each message transfer has the following phases:• One MAIL command, identifies originator
•Gives reverse path to use for error reporting•Receiver returns 250 OK or appropriatefail/error message
• One or more RCPT commands, identifies recipients for the message
• Each recipient identified by a separate RCPT• Separate reply for each recipient (250 OK etc.)
• One DATA command transfers message text• End of message indicated by line containing just period (.)
MTA connection termination
MTA connection terminationSender sends QUIT and waits for reply
□ Then initiate TCP close operation
□ Receiver initiates TCP close after sending reply to QUIT
OptimizationIf message is sent to multiple users on a given host,it is sent only once.
❖ Delivery to users handled by destination host
If Multiple messages are ready for given host,a single TCP connection can be used.
❖ Saves overhead of setting up and termination of connection
Possible Errors❖ Host unreachable❖ Host out of operation❖ TCP connection fail during transfer❖ Faulty destination address
User errorTarget user address has changed Redirect if possibleInform user if not
Sender re-queue the mail - will retry till a configurable period of time
SMTP protocol reliability❖ TCP connection is used to transfer mail from sender to receiver over TCP
connection❖ Attempts to provide reliable service❖ No guarantee to recover lost messages❖ No end-to-end ack to sender❖ Error indication report not guaranteed
SMTP receiver❖ Accepts arriving message❖ Places in user mailbox or copies to outbound message queue for forwarding❖ Receiver must
● verify local mail destination● deal with errors❏ Transmission❏ Lack of disk space
SMTP status codes(DSN-Delivery status Notification)Leading digits Indicates catagories
2XX-Positive Completion Reply(Successful)
3XX-Positive Intermediate Reply(Redirection)
4XX-Transient negative completion reply(Client error)
5XX-Permanent negative completion reply(Server error)
SMTP status codes..Cont.
Problems with SMTP
No inherent securityn Authenticationn Encryption
Only uses NVT (Network Virtual Terminal) 7-bit ASCII format
E-mails can be forged…..HELO mail.rose.eduMAIL FROM: [email protected] TO: [email protected]: Dr. Art ZennerTo: Professor RichardsSubject: CIT 2243
Professor Richards,By department decree all students in your “Introduction to
Unix” class are hereby to be granted automatic A’s.Thank you,Dr. Art Zenner.
QUIT
Extensions to SMTPMIME – Multipurpose Internet Mail Extensionsn Transforms non-ASCII data to NVT (Network Virtual Terminal) ASCII
dataw Text w Applicationw Imagew Audiow Video
MIME and Base64 Encoding
If the internet is the information highway, then the path for email is a narrow tunnel n Only very small vehicles can pass trough
Then how do you send a big truck through a small ravine? n You have to break it down to smaller pieces
and transport the pieces through the ravine, and reassemble the truck
MIME and Base64 Encoding
The same happens when you send a file attachment via email. This is known as encodingn the binary data (256 bits) is transformed to
ASCII text (128 bitsn allowing it to fit through the tunnel
On the recipient's end, the data is decoded and the original file is rebuilt.
Mail Transfer Agents (MTA)MTAs do the actual mail transfersMTAs are not meant to be directly accessed by users.Other MTA’s are:n Postfixn Qmailn MS Exchangen CC:Mailn Lotus Notesn ….etc.
Problems with simple SMTP
The first one relates to message length. Some older implementations cannot handle messages exceeding 64KB.Another problem relates to timeouts. If the Client and server have different timeouts, one of them may give up while the other is still busy, unexpectedly terminating the connection.Infinite mail storms can be triggered. For example, If host 1 holds mailing list A and host 2 holds mailing list B and each list contains an entry for the other one, then a message sent to either list could generate a never ending amount of email traffic unless somebody checks for it.
ESMTP (RFC 2821)
To get around the problems with simple SMTP,extended SMTP has been defined in RFC2821.Clients wanting to use it should send an EHLOmessage instead of HELO initially. If this isrejected, then the server is a regular SMTPserver, and the client should proceed in theusual way. If the EHLO is accepted, then newcommands and parameters are allowed.
POP3Mail access protocol:
POST OFFICE PROTOCOL [RFC 1939]
Simple and limited functionality,Consists of client software and Server software, Server performs user authorization
IMAP: Internet mail access protocol [Version 4]
POP3Retrieves messages from a mail server
Typically, messages are downloaded to your mail client, and deleted from the server
Designed for use with dial-up connections when people were intermittently connected
Listens on Port 110 (with Secure POP generally on port 995)
POP3 Connection Establishment
POP3 protocol sessionroot@amberit ~]# telnet mail.amberit.com.bd 110 Trying 206.71.88.102...Connected to mail.amberit.com.bd. Escape character is '^]'.+OK Dovecot ready. user [email protected]+OKpass letmein+OK Logged in. list+OK 1 messages:1 482.retr 1+OK 482 octets<snip>Message Headers</snip>This is my short message quit+OK Loggingout.
Basic POP3 commands□ USER <name> - identifies the user□ PASS <password> - authentication for user□ STAT - lists all messages in the mailbox of user□ LIST <msg no.> - lists the content of a message □ RETR <msg no.> - retrieves a particular message □ DELE <msg no.> - Deletes a particular message□ NOOP□ RSET□ QUIT
□ Replies□ +OK□ –ERR
IMAP Protocol session[root@amberit ~]# telnet mail.amberit.com.bd 143 Trying 206.71.88.102...
Connected to mail.amberit.com.bd Escape character is '^]'.* OK Dovecot ready.A1 LOGIN [email protected] A1 OK Logged in.A2 SELECTInbox* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)*
IMAP Protocol session..cont.OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.* 1 EXISTS* 0 RECENT* OK [UIDVALIDITY 1225333589] UIDs valid* OK [UIDNEXT 2] Predicted next UIDA2 OK [READ-WRITE] Select completed. A3 FETCH 1 BODY[HEADER]* 1 FETCH (BODY[HEADER] {454}<snip> Message Header Delivered</snip> A3 OK Fetch completed.A4 LOGOUT* BYE Logging outA4 OK Logout completed. Connection closed by foreign host.
Webmail
Webmail
Webmail (or web-based email) is any email client implemented as a web application running on a web server
• Online in nature• IMAP connections mostly• Secured HTTP (HTTPS) must• Can be hosted, e.g.: RoundCube, SquirrelMail, Horde, Zimbra, Rainloop, Kite,
MailPile,iRedMail, etc.• Webmail Service providers : Gmail, Outlook, AOL, Yahoo, etc.• Privacy and Security Concern• Easy to configure, easy to host.
Common Threat Landscape• Eavesdropping
• Spamming and Phishing• Spoofing• Malicious Email Attachments
• Replying and Forwarding Issues• CC & BCC Issues
Postfix
Short History• Originally developed in the late 90s at IBM by Wietse
Venema, author of security software (SATAN, TCPwrappers, ...), as ”IBM Secure Mailer”
• Place under an Open Source license, and renamed ”Postfix”
• Intended as a replacement for then insecure mail systems, such as Sendmail
Design Goals• Safety
• Robustness • Performance • Modularity
• Compatibility
Safety• Postfix makes it very hard to lose mails – many checks to
ensure that mail has been written to disk or delivered • Back off mechanisms in case of repeated failure
Security• Collection of daemons working together Doesn't use
environment for communication • Very paranoid about input checking, all allocation is dynamic
(avoiding buffer overflows) • chroot support out of the box for almost all processes &
daemons • No data is ever exchanged directly between processes – all is
done via IPC, and files on disk • Conservative resource usage
Performance • Designed to be fast from the ground up
• Also behaves well with neighbors, doesn't flood them with mail, and instead uses a throughput adaptation
• Will not block delivery for a message if one recipient domain fails
Modular• One program, one function
All programs controlled from ”master.cf” • Many small programs working together, with limited
privileges
• Compatible with Sendmail's /etc/aliases and .forward conventions
Features• Virtual domains – domains and users are completely
independent of system (UNIX) users • Aliases – sendmail compatible
Rewriting – senders, recipients, globallyRBL support (Realtime Blackhole Lists) support Content filtering using pipes, SMTP or milter
• Support for arbitrary mail manipulation with policy services (custom programs talking to postfix)
More Features• Restriction classes Conditional filtering
• Sender or recipient address verification (test email addresses before accepting mail from them)
• TLS support
Core Concepts : maps• In postfix, everything is looked up in a map (table)
• Maps can be in many formats or use many data sources: – hash/btree– regexp/PCRE – CIDR– NIS – LDAP, *SQL (user defined queries)
ArchitectureArchitecture
Basic Postfix Configuration• Two primary configuration files
– main.cf• Main configuration file where all the subsystems are configured (smtp, smtpd,
cleanup, routing, ...)
– master.cf• File controlling how the ”master” process of postfix will launch all the necessary
postfix daemons to perform mail routing, on-demand
Other Configuration Details• Reside in ”maps” mentioned earlier
• Tables containing values and conditions, referred to from main.cf, controlling all aspects such as: – Virtual and local domains – Routing rules– Access control– Rewriting
– ...
Configuration: postconf command• postconf – used to view and edit configuration parameters
• For changing the configuration, it is usually done vi editing ”main.cf” directly
Some Basic main.cfsmtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = group1.group01.net
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = group1.group01.net, localhost.group01.net, , localhost
relayhost = group1.group01.net
mynetworks = 127.0.0.0/8 61.45.254.0/24 [2001:df0:a:4::]/64 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
Some Basic main.cf# TLS parameterssmtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
Some Basic main.cf## in the file /etc/mailname
group1.group01.net
## in the file /etc/aliasespostmaster: rootsysadm: apnic