[Dr. WoW Season 2] [No 9] Multiple Approaches to Email Control---Mail Filtering Mechanism

Latest reply: Jun 10, 2018 06:12:39 981 1 2 1

The popularity of instant messaging apps, such as WhatsApp, is resulting in the decline of email use. However, email remains indispensable in enterprises, many of which use their own mail servers.

Email facilitates business, but it brings about some issues. For example, unsolicited spam mails may compromise productivity and confidentiality. Therefore, enterprises need to control incoming and outgoing email.

To resolve these issues, the NGFW provides the email filtering mechanism, including spam filtering and email content filtering. Now, let's start with the spam filtering function.

Spam Filtering

Spam mails are unsolicited emails sent to users. They include advertisements, promotional materials, fraud messages, and even viruses. Not only do they consume network bandwidth and mailbox space, spam mails also bring security risks. The NGFW functions as an enterprise's gateway and can block spam mails.

Using the Real-time Blackhole List (RBL) technology, the NGFW filters out spam by querying the RBL server. This allows the NGFW to determine whether the SMTP server's IP address of the email sender is blacklisted. The RBL server maintains a real-time blacklist of SMTP servers that have sent spam. After obtaining the result returned by the RBL server, the NGFW takes the appropriate action.

As shown in the following figure, the spam filtering process involves the NGFW, DNS server, and RBL server.

20180601104743812001.png

 

The sender initiates an SMTP connection to the enterprise's mail server through the sender's mail server.

The NGFW parses the IP address of the sender's mail server in the SMTP request, reverses this IP address, and combines it with the RBL server name into a new "domain name" to initiate a query request to the DNS server. For example, if the IP address of the sender's mail server is 10.11.12.13 and the RBL server is sbl.spamhaus.org, the NGFW initiates the query request with "13.12.11.10.sbl.spamhaus.org".

The DNS server obtains the RBL server name from the query packet, parses the IP address of the RBL server, and forwards the query request to the RBL server.

The RBL server checks the IP address (10.11.12.13) and returns the query result to the DNS server. If the IP address is in the RBL list, the RBL server returns a reply code. Otherwise, the RBL server returns NXDOMAIN.

The DNS server forwards the query result to the NGFW.

The NGFW processes the SMTP request based on the query result.

  • If the RBL server returns a reply code, the email is regarded as a spam mail. The NGFW forwards the SMTP connection request and generates a log or blocks the SMTP connection based on the specified action.
  • If the RBL server returns NXDOMAIN, the NGFW forwards the SMTP connection request.
  • If the RBL query times out, the NGFW forwards the SMTP connection request.

This is a complete query process. After the query, the NGFW caches the RBL query results to improve efficiency and reduce query traffic. For subsequent queries, the NGFW parses the IP address of the sender's mail server and then searches the cache. If the IP address is not found in the cache, the NGFW initiates an RBL query.

When deploying spam filtering on the NGFW, pay attention to the DNS server, RBL server, and reply code settings.

DNS Server

Standard DNS query packets are used in the RBL query process. The NGFW initiates a query to the DNS server, and the DNS server returns the query result. Therefore, the success of a query depends on the operating status of the DNS server. The DNS server must meet the following conditions to ensure a successful RBL query:

l  The DNS server must be open to queries from the NGFW.

Some DNS servers (such as root servers) are not open to queries. Other DNS servers, for security purposes, respond only to query requests from specified clients (for example, DNS servers of one telecom operator may not be open to users of another operator). Therefore, only DNS servers that are open to queries from the NGFW can be used.

l  The DNS server must support the recursive query mode.

In recursive query mode, the DNS server sends to other DNS servers a query request for which it cannot find a record. When it obtains a result, the DNS server forwards the result to the NGFW. RBL query has the same requirements on DNS servers.

Iterative query is another query mode of the DNS server. If DNS server 1 cannot parse the query request, it returns only the IP address of DNS server 2 (not a reply code) to the NGFW, and the NGFW sends the query request to DNS server 2. This process continues until a record is found. If the reply code is configured improperly on the NGFW ("any reply code" is used), a legitimate email will be treated as a spam mail, resulting in a false positive.

l  The DNS server must not be hijacked.

DNS hijacking is a practice in which an ISP configures the DNS server to return the IP address of a value-added site instead of the NXDOMAIN, thereby redirecting users to the value-added site. In this scenario, the query result received by the NGFW is the IP address of the redirected site. This results in a false positive and legitimate emails being treated as spam mails.

To determine whether a DNS server meets the preceding requirements, we can use the nslookup command (on Windows) or the dig command (on Linux). The commands allow us to test the DNS server and then determine the operating status of the DNS server based on the command output. For details, refer to the NGFW product documentation.

RBL Server

After the DNS server is configured, we need to determine which RBL server to use. Different RBL service providers use different criteria and levels of strictness for matching spam mail. Some providers offer multiple RBL servers for IP addresses of different types.

Enterprise administrators should select an appropriate RBL service provider based on the location of the enterprise and the source of frequently received spam mails. Here are two mainstream RBL service providers as an example:

China Anti-Spam Alliance (CASA)    http://www.anti-spam.org.cn/

RBL

Query Set

Reply Code

Remarks

CBL

cbl.anti-spam.org.cn

127.0.8.2

CBL is a list of IP addresses that send spam in China. CBL is best suited for use in China.

CBL+

cblplus.anti-spam.org.cn

127.0.8.2

The combination of CBL and a list of dynamic spam email IP addresses in China.

CBL-

cblless.anti-spam.org.cn

127.0.8.2

Similar to CBL+, but without the whitelisted Chinese email service providers.

 

Spamhaus   http://www.spamhaus.org/

RBL

Query Set

Reply Code

Remarks

SBL

sbl.spamhaus.org

127.0.0.2-3

Verified spam sources and a real-time blacklist of IP addresses that have sent spam.

XBL

xbl.spamhaus.org

127.0.0.4-7

A real-time blacklist of IP addresses that send spam due to security problems (such as zombie hosts and hosts infected by Trojans).

PBL

pbl.spamhaus.org

127.0.0.10-11

A list of IP addresses (including dynamic addresses) that should not send emails.

ZEN

zen.spamhaus.org

127.0.0.2-11

This is a combination of SBL, XBL, and PBL, and is recommended.

 

The query set in the preceding tables refers to the name of the RBL server. This name should be configured on the NGFW. Reply codes must also be configured on the NGFW. In the following section, we will describe the reply codes.

Reply Code

The reply code is the result returned by the RBL server during the RBL query process. This reply code is usually in the form of an IP address and indicates that the IP address queried by the NGFW is included in the RBL. Mail servers corresponding to the IP addresses included in the RBL are deemed by the RBL service provider to be responsible for sending spam.

To ensure that the NGFW can correctly process spam sent from IP addresses included in the RBL, the reply codes must be correctly configured on the NGFW. If a specified reply code is different from that returned by the RBL server, the NGFW considers the email legitimate and forwards the SMTP connection request.

Two options are available when you configure reply codes on the NGFW:

l  Obtain all reply codes from the RBL service provider and specify one or more so that an email is considered a spam mail only if one of the specified query codes is returned.

l  Use "any reply code" instead of a specific reply code. The NGFW considers any mail server as a source of spam if the reply is an IP address. However, false positives may occur if the DNS server is hijacked.

It is recommended to configure specific reply codes correctly on the NGFW rather than using "any reply code." This guards against the impact of a hijacked DNS server.

False positives and false negatives may occur due to the limitations in RBL technologies. To address this issue, the NGFW provides local whitelist and blacklist.

Whitelist

Whitelist is used to resolve false positives. If the IP address of the sender's mail server matches a whitelist entry, the NGFW permits the email and records a log without any further check.

Enterprise administrators can view the email filtering logs, obtain the IP addresses of the mail servers that are falsely blocked, and add them to the whitelist. The administrators can also add the IP addresses of trusted mail servers to the whitelist.

Blacklist

Blacklist is used to resolve false negatives. If the IP address of the sender's mail server matches a blacklist entry, the NGFW blocks the email and records a log.

An email might be forwarded by more than one mail server. Each server that forwards the email is added to a Received field in the email header, based on which we can trace the source of a spam mail and add it to the blacklist.

The basic expression format of the Received statement is: From Server A by Server B. In this expression, Server A is the mail server that sends emails, and Server B is the mail server that receives emails. Usually, Server A in the last Received statement is the mail server that sends the spam mail. Here is an example of the email header of an email received in Microsoft Outlook:

20180601104744818002.png

 

According to the blacklist filtering mechanism, we need to determine the mail server that initiates SMTP connection to the NGFW. That is, we need trace the forwarding path of the received email. As shown in the preceding figure, the email is sent from m15-39.126.com (220.181.15.39) to the enterprise's mail server xxx.huawei.com. To ensure the NGFW blocks this email, blacklist 220.181.15.39.

The RBL, blacklist, and whitelist are filtering mechanisms based on IP addresses. Now we are going to learn about email content filtering.

Email Content Filtering

Email content filtering consists of email address check (including anonymous email check) and email attachment control. Email address check filters emails based on the email addresses of the email sender and recipient. Anonymous email check blocks emails with an empty sender address. And email attachment control filters emails based on the number of attachments carried by an email and the size of each attachment.

Email Address Check

The NGFW detects key SMTP, POP3, and IMAP commands as a proxy, extracts email addresses of the senders and recipients, and takes actions based on the email addresses. The NGFW provides the following actions:

l  Allow: allow the email through.

l  Block: block the email. For IMAP, only the alert action is supported. Specifically, the NGFW allows the emails through and records logs, but cannot block the emails.

SMTP is used for sending email, either from a client to a mail server or from one mail server to another. In contrast, POP3 and IMAP are used for receiving emails. Accordingly, the NGFW implements email address check based on the control direction (specifically, the email protocol used for sending or receiving). The control of email sending is based on SMTP, while the control of email receiving is based on POP3 and IMAP.

Let's look at two typical application scenarios. In the first scenario, an enterprise deploys a mail server on its intranet. As shown in the following figure, enterprise users send emails through this mail server. The mail server also receives emails from the mail server on the Internet.

20180601104745891003.png

 

In this scenario, incoming and outgoing SMTP emails are filtered by sender email address.

To improve filtering accuracy, the recipient's email address can also be checked. If an email matches the check rules for both the sender and recipient's email addresses and the actions in the rules are different (one is allow and the other is block), the NGFW implements the stricter action (block).

In the second scenario, shown in the following figure, the enterprise does not have its own mail server and uses a third-party email service provider to send and receive emails.

20180601104745950004.png

 

In this scenario, incoming POP3/IMAP emails are filtered by sender email address.

This is also true if the enterprise deploys a mail server on its intranet. This method cannot conserve mail server resources or bandwidth, and is not as effective in terms of defense as the method used in the first scenario.

20180601104746382005.png

 

Similar to URLs, email addresses are character strings. The NGFW provides four matching modes: prefix matching, suffix matching, exact matching, and keyword matching. The following table describes these matching modes.

Matching Mode

Results

Example

Prefix matching

Match all email addresses that start with the specified character string.

Use "username@" to match all email addresses from the username, such as username@263.net and username@gmail.com.

Suffix matching

Match all email addresses that end with the specified character string.

Use "@gmail.com" to match all Gmail emails.

Suffix matching filters emails from specified domains (companies).

Exact matching

Match the specified email address.

Use "username@gmail.com" to match username@gmail.com.

Keyword matching

Match all email addresses that contain the specified character string.

Use "username@gmail" to match all email addresses that contain "username@gmail", such as username@gmail.com and username@gmail.net.

 

The effectiveness of email address check depends on the accuracy of email addresses. Note that SMTP servers do not verify the sender's email address when forwarding an email. The sender's address might be a fake. This is a drawback of SMTP. Therefore, false positives may occur in email address check.

When using the email address check function, enterprise administrators must specify who (which email addresses) can send or receive emails and to whom they can send emails.

Anonymous Email Check

Anonymous emails are those without sender addresses. As mentioned earlier, SMTP servers do not check the sender's email addresses. Consequently, senders can use fake addresses or even provide no email address.

Anonymous emails are usually sent with questionable intentions and might contain useless or harmful information. For enterprises, anonymous emails are generally unrelated to the business and should be blocked.

The NGFW provides the anonymous email check function, which is similar to email address check. If the sender address is empty, the action can be allow, block, or alert. However, IMAP does not support the block action and supports only the alert action.

Email Attachment Control

If not controlled, attachments not only waste bandwidth but also risk information leaks. The NGFW provides the email attachment control function to control the number of attachments and the size of each attachment.

Enterprise administrators can set the maximum number of attachments and the maximum size of each attachment in outgoing and incoming emails. If the size of any attachment exceeds the limit, the email will be processed according to the specified action. The action can be alert or block (only alert for IMAP).

The email attachment control is a coarse-grained filtering method. The NGFW also supports fine-grained email filtering by subject, body, and attachment name. We will introduce this function in the subsequent content security feature.

The following figure shows the processing order of different modules of email filtering and the corresponding actions.

20180601104747172006.png

 

The email filtering mechanism is summarized as follows:

l  The NGFW provides RBL technology to prevent spam mails, and provides local whitelist and blacklist to prevent false positives and negatives. These email filtering mechanisms are based on IP address and apply to SMTP.

l  The NGFW also provides email address check, anonymous email check, and email attachment control functions, which apply to SMTP, POP3, and IMAP.

In the next article, we will introduce the configuration of email filtering and provide some typical configuration examples.


To view the list of all Dr. WoW technical posts, click here.

  • x
  • convention:

wissal
MVE Created Jun 10, 2018 06:12:39 Helpful(0) Helpful(0)

Thanks for sharing, it will be helpful for users
  • x
  • convention:

Telecommunications%20engineer%2C%20currently%20senior%20project%20manager%20at%20an%20operator%2C%20partner%20of%20Huawei%2C%20in%20the%20radio%20access%20network%20department%2C%20for%2020%20years%20I%20managed%20several%20types%20of%20projects%2C%20for%20the%20different%20nodes%20of%20the%20network.

Comment

Reply
You need to log in to reply to the post Login | Register

Notice Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " Privacy."
If the attachment button is not available, update the Adobe Flash Player to the latest version!
Login and enjoy all the member benefits

Login and enjoy all the member benefits

Login