Email Security: Part 1 – How email works:
In Part 1 of the Email Security Blog series, we discuss how email works. Read through the process, a description of different mail protocols, and some key terminology. The second part of the series will cover how the InQuest Email Security capability is installed, while the final part will cover the features to include detection or prevention for ransomware, VIP impersonation, phishing, password-protected attachments, invoice fraud, crypto scams, brand impersonation, and other forms of ever-evolving social engineering.
The moment an email is sent, a message is routed from server to server via the Simple Mail Transfer Protocol until it makes its way from the client to the email recipient’s email server. This is described quite well by Namecheap
Sending an email is just like sending a letter to a friend. Let’s say [email protected] sends an email to their friend [email protected]. The email gets sent first to an outgoing mail server (SMTP) whose job, just like that of the post office, is to transport emails. The SMTP checks the postage address to figure out where to send the mail. Unfortunately, the SMTP doesn’t understand how to read the domain name (just like the mailman consults a map since he doesn’t know every street name by memory). The SMTP needs a computer-friendly IP address to locate and deliver the message to the recipient. To find the IP, the SMTP contacts the DNS server to translate the recipient’s email address, [email protected], to an IP address like “54.88.225.116”. Once the associated IP has been found, it checks if that domain has any mail exchange servers (MX) which detail any information about where the message should be sent. This is just like checking if the recipient uses a mailbox or a PO box to receive their mail in real life.
The SMTP has all the necessary information to send the message from its server onto the email recipient’s MTA server.
The MTA decides where to put the mail and whether the recipient uses a client that works via POP or via the IMAP protocol. The recipient will then receive a new email notification, and the mail will wait in the mailbox until it is fetched.
So there it is, email transmission works in virtually the same way as sending real mail. Once an email is sent, the mail server puts it in an envelope (the SMTP protocol connection). Let’s take a look at how these work.
Once an email is written and the send button is clicked, the message is sent to the Mail Transfer Agent (MTA). This communication is done via the Simple Mail Transfer Protocol (SMTP).
The SMTP queries the Domain Name System (DNS) to find the address of the recipient. This is done with the help of a Mail eXchanger (MX) record. The MX record is a resource record that specifies the mail server of a domain name. Once located, the SMTP server will send the message to that server.
The next step involves transferring the message between mail servers. The message is now in the recipient’s mail server (MTA). The receiving server will store the message and make it available to the recipient, accessing it via the web, POP, or IMAP protocols.
Email protocols
Mail Transfer Agents (MTAs) communicate with each other over the Internet using SMTP protocol (SMTP servers). The recipient’s MTA then forwards the email to the incoming mail server (MDA, mail delivery agent), tasked with storing the mail until the user accepts it. To retrieve email on an MDA, a supporting protocol must be used. There are two main protocols, POP3 and IMAP. You might recognize these two acronyms since incoming mail servers are called POP servers, or IMAP servers, depending on which protocol they use.
Pop vs. IMAP
POP stands for Post Office Protocol. This piece of software is used for retrieving email. POP3 gives an email user access to their emails stored in their user account on that server. You don’t need to stay online for the emails to come through. You just need to leave a copy of an email on the server to access it.
POP does have some drawbacks; specifically, information transmitted through POP travels one way. This means that once an email is downloaded to a client, the client takes charge of sorting through the different status flags (e.g., sent, deleted, or answered). This was fine when the Internet was young, before smartphones, tablets, and the like. People accessed their mail from a single location. Nowadays, it’s more likely that you access emails from many places; thunderbird at home, the mail app on your cell phone, or a web interface when you are at work, for example. With POP, you would have to sort through the information over each different device — assuming you’ve saved a copy of each email on the server.
IMAP (Internet Message Access Protocol) is a bit smarter about how it coordinates emails. IMAP clients have two-way communication with their servers. The IMAP protocol saves a copy of every message on the server so that, unlike with POP, multiple clients can access them. It’s completely synchronized. With IMAP, when you check an email on your tablet, it will be marked as read when you check your inbox on your phone. This happens because the status of the email is updated with all other clients during the server interaction.
IMAP is just like when your mail is categorized and stored at the post office for you and redelivers it when you are at home, at work, or pick it up in person. You can keep a properly marked archive on your home client as well as on your mail server. IMAP has an offline mode where any changes are synced with the server the next time you’re online. You may configure IMAP mail servers to fetch mail from POP inboxes, too, which works well if you’re seeking to consolidate. Of course, given that IMAP works with the “cloud” best, servers get involved, and storage can be problematic. Thankfully, storage space and bandwidth isn’t as pricey as it once was, but this will truly be a change-off for a few humans.
How email is received
Let’s now take a look at how email is received. No surprises here — we’ll revert straight back to our mail carrier analogy. How would an envelope be delivered to the recipient on the front of the envelope? The postal service finds the most logical route to the recipient.
The electronic version of events is handled similarly:
The mail server locates the recipient’s server, but since the recipient’s server won’t accept every mail that comes its way, it asks who sent the email.
The sending server gives the recipient server information on who the sender is by querying the envelope. Acknowledging the email is from a legitimate source (not spam, etc.), the recipient server says, “sure, I understand that InQuest exists, and from that sending address.”
Satisfied the sender address is correct, the recipient server asks for the receiver’. This is how envelope data is treated. The sending server will now forward the contents of the email contained in the envelope. Once the email has been received, the recipient server gives the mail server a receipt.
Types of MUA
Retrieving email is tasked by a software program called a Mail User Agent (MUA). There are two types of MUA, and these are classed depending on how emails are accessed, via installed software (email client) or through in browser (webmail).
When an MUA is installed on a user’s system, it’s called an email client. To use an email client, MUA such as Microsoft Outlook, Mozilla Thunderbird, and Lotus Notes allows users to the MUA program to their computer. This program is used to download and store email messages to their computers. With a client MUA, emails can be read and written offline.
When email is accessed online, it’s called webmail. Web-based MUAs such as Yahoo, Gmail, and Hotmail store messages on their mail servers and can only be accessed through a web page. The main advantage of webmail is sending and receiving mail from a web browser, anywhere. The main disadvantage of using a web-based application is the need to be connected to the Internet to use it.
Terminology
Client and server components
A Fresh Cloud describes the different components involved in an email transmission from sender to recipient.
MUA (Mail User Agent)
Client application that allows receiving and sending emails. It can be a desktop application such as Microsoft Outlook/Thunderbird/… or web-based such as Gmail.
MSA (Mail Submission Agent)
A server program that receives mail from an MUA, checks for any errors, and transfers it (with SMTP) to the MTA hosted on the same server.
MTA (Mail Transfer Agent)
A server application that receives mail from the MSA or another MTA will find (through name servers and the DNS) the MX record from the recipient domain’s DNS zone to know how to transfer the mail. It then transmits the mail (with SMTP) to another MTA (known as SMTP relaying) or, if the recipient’s server has been reached, to the MDA.
Examples of MTAs are Postfix, Exim, Sendmail, qmail …
MDA (Mail Delivery Agent)
A server program that receives mail from the server’s MTA, and stores it into the mailbox. MDA is also known as LDA (Local Delivery Agent).
An example is Dovecot, which is mainly a POP3 and IMAP server allowing an MUA to retrieve mail, but also includes an MDA which takes mail from an MTA and delivers it to the server’s mailbox.
Mailbox: maildir/mbox
The server’s mail storage. Maildir is a way of storing email messages. It is usually preferable over mbox.
SMTP
Protocol used by MUAs to send emails to an MSA. The recommended SMTP port for sending mail (from an MUA to an MSA) is the port 587, which uses TLS encryption.
IMAP/POP3
Protocols used by MUAs to retrieve emails from a server mailbox. POP3 deletes the email messages from the server after they have been downloaded. IMAP is usually preferable as it maintains all email messages on the server, permitting management of a mailbox by multiple email clients.
MX (Mail Exchanger) record
A Mail Exchanger (MX) record in the DNS specifies which server is responsible for accepting email addresses on behalf of a domain. The host name from the MX record must map to one or more address record (A or AAAA) in the DNS, and must not point to any CNAME records.
Email authentification
Email authentication is a technique used to improve email deliverability (reduce the risk of emails ending up in the recipients’ spam folders). It allows antispam filters to better verify the identity of the sender, and prevent spoofing and phishing scams.
Terms to know
MAIL FROM address
The email address to which bounce messages are delivered. This address is included in the SMTP envelope (data part of the SMTP protocol).
It is also known as RFC5321.MailFrom, envelope sender address, envelope from address, Return Path address, Bounce address, …
Display From address
The email address that is displayed to the end user within their email client (MUA). This address is contained in the header (meta data) of the email. The email client (MUA) doesn’t have access to the SMTP envelope (and thus can’t see the aforementioned MAIL FROM address). The Display From address can be different than the MAIL FROM address.
The MAIL FROM address is used by the SMTP. The Display From address is used by the MUA to display the address in the Form field.
It is also known as RFC5322.From, header From address.
There are 3 main protocols allowing email authentication:
DKIM – DomainKeys Identified Mail
The sender creates an MD5 hash value of some elements of the email (e.g. the email header). The sender then uses a private key (only known by him) to encrypt that MD5 hash. The encrypted string is inserted into the mail, known as the DKIM signature. The sender stores a public key in a DNS record.
The receiver finds the public key from the DNS for that domain. The receiver then uses this public key to decrypt the DKIM signature from the email back to the original MD5 hash. The receiver generates a new MD5 hash from the elements of the email signed by DKIM, and compares it with the original MD5 hash. If they match the receiver knows that:
- The sending email domain is the owner of that domain (it is mathematically almost impossible to spoof a correct DKIM signature that decrypts to the original MD5 hash using the public key).
- The elements of the email signed by DKIM were not changed in transit (otherwise the original MD5 hash and the receiver’s generated MD5 hash would not match).
The shortcoming of DKIM is that it can only protect mail that has been signed, but it doesn’t provide a mechanism to prove that an unsigned message should have been signed.
SPF – Sender Policy Framework
SPF enables the sender domain to publicly state which MTA servers (IPs) may send emails on its behalf.
The receiver server checks if SPF exists in the DNS for the domain in the MAIL FROM address (also called envelope sender address). If SPF exists, the receiver checks if the IP address of the sending server matches in the list of IPs in the SPF.
The shortcoming of SPF is that it validates the originating server only looking at the domain in the MAIL FROM address, not the mail header From address. The MAIL FROM address is the email address that receiving servers use to notify the sending server of delivery problems; it is also called envelope sender, envelope from, bounce address or Return Path address. The problem with this limitation is that the From address is what recipients see in their email clients.
DMARC – Domain-based Message Authentication, Reporting & Conformance
DMARC is built on top of SPF and DKIM to address the shortcomings of these two authentication standards. DMARC allows to enforce DKIM or SPF validation and confirm that the Display From address is authentic.
The sender adds a DMARC policy in the DNS.
The receiver
- looks up the DMARC policy in DNS
- performs DKIM signature validation and/or SPF validation;
- performs domain alignment checks;
- applies DMARC policy.
Domain alignment consists of verifying that the From address (address displayed to the final recipient) matches with:
- for SPF, the envelope sender (MAIL FROM) domain;
- for DKIM, the DKIM d= domain (the signing domain included in the DKIM signature header alongside the encrypted hash).
Free Email Hygiene Analysis
Solid email security begins with proper email hygiene. There are a variety of email hygiene technologies and wrapping one’s head around them all is challenging. Try our complimentary Email Hygiene Analysis and receive an instant report about your company’s security posture including a simple rating with iterative guidance, as well as a comparison against the Fortune 500. Try it today!
