best practices for commtouch outbound spam … practices for commtouch outbound spam protection...

www.commtouch.com blog.commtouch.com (US) 650 864 2000 [email protected] Confidential © Commtouch 2011 (International) +972 9 863 6888

Best Practices for Commtouch

Outbound Spam Protection

configuration

Contents

General .................................................................................................................................................... 3

Overview ................................................................................................................................................. 3

Commtouch Advanced Security daemon (ctasd) configuration best practices ...................................... 4

SenderID ................................................................................................................................................ 4

Email classification ................................................................................................................................ 5

SenderID Counters ................................................................................................................................ 6

SenderIDWindow and SenderIDWindowSize ........................................................................................ 7

SenderID Thresholds ............................................................................................................................. 7

ctasd headers ........................................................................................................................................ 9

Recommended actions and policy enforcement ideas ........................................................................... 10

Blocking spam...................................................................................................................................... 10

Blocking a spammer ............................................................................................................................ 10

Anti-Abuse process automation .......................................................................................................... 12

Summary ................................................................................................................................................. 13


Best Practices for the Commtouch Outbound Spam

Protection configuration




General

This document assumes that the reader is familiar with the Commtouch Outbound Spam Protection (OSP)

solution and Commtouch Advanced Security daemon (ctasd). For more information on Commtouch OSP

and ctasd please refer to “Outbound Spam Protection – Product Overview” and “ctasd - outbound

integration manual”.

Overview

There are a few key differences between incoming and outgoing email traffic.

Incoming email originates from various sources that target mailboxes on your infrastructure. Email

senders may or may not have legitimate intentions and your primary goal, as an email service provider,

is to protect your clients from unwanted and malicious content while at the same time delivering all

legitimate email to the destination mailbox. Nowadays, most people are well aware of the spam issue.

They recognize your efforts at protecting their mailboxes, can understand the challenges of accurate

detection, and tolerate an occasional false negative or false positive.

When it comes to outbound email the situation is different. Your clients send email out using your

infrastructure and they often pay for it. They expect their emails to be delivered promptly and, especially

those who pay for their email service, have zero tolerance for outbound false positives or delivery issues.

At the same time they are not concerned if spam or other malicious emails are sent out from your

servers and it becomes your responsibility to maintain the good reputation of your mail servers.

Although, email service providers have policies and service agreements pertaining to email use, they

often have limited tools to monitor and enforce these policies.

Commtouch’s Outbound Spam Protection (OSP) was designed to deal specifically with the outbound

spam issue. Using Commtouch’s patented Recurrent Pattern Detection (RPD) technology and adjusted for

the specific requirements of outbound spam, the solution is able to determine in real-time if an email is

spam and as well as identifying the sender. The primary advantages of OSP include:

Detecting both “local” and “global” spam

Real-time detection

Spammer identification

Reduced False Positives

Service Provider control




Simply put, you get full, real-time visibility into your outbound email traffic and have the ability to take

actions to enforce your outbound email policy. Better yet, the entire process can be automated to save on

your resources and react to outbound email abuse in real time.

The following sections describe Commtouch’s best practices regarding configuration of the Commtouch

Advanced Security daemon (ctasd) and our recommendations and ideas to help you with implementation

and policy enforcement.

Commtouch Advanced Security daemon (ctasd)

configuration best practices

ctasd configuration is done using a configuration file. By default, the configuration file name is ctasd.conf

and it is located in the /etc/ctasd/ directory. A full description of ctasd.conf is available in the “ctasd -

outbound integration manual”. Here we will concentrate on several important configuration parameters

and help you better understand how to configure ctasd for optimal operation in your environment.

Note: In order to get full outbound detection functionality an “Outbound” license key is required. If an

“Inbound” license is used, the ctasd will still work and detect spam, but you may experience lower

outbound detection accuracy and certain outbound features will not work.

The first thing to do when you open the ctasd.conf file is to switch ctasd into “Outbound mode”. It is done

by setting “OutboundEnabled = 1” at the top of the configuration file. All other outbound detection

specific configuration is done in the [Outbound] section of the configuration file.

SENDERID

ctasd calculates email and spam thresholds based on a SenderID that it takes from email headers.

Therefore it is important to correctly identify and specify SenderID in the SenderIDHeaderName

parameter. By default, ctasd uses the From: header, however this field can be easily spoofed and we do

not recommend using it unless there is no other alternative. Good candidates for the

SenderIDHeaderName are headers or x-headers that allow you to identify the original sender without

doubt. For example, X-Authenticated-User, X-Auth-User, X-POP-User, or X-UserID headers are great

candidates. In some cases the authenticated user information is available as part of the Received header,

or elsewhere in the message headers. To extract a SenderID in those scenarios you can use a regular

expression and specify it using the SenderIDHeaderRegEx configuration parameter. For example, if your

Received header looks like this:

Received: from client ([216.163.188.12])(authenticated user [email protected]) by mailserver

(using TLSv1/SSLv3 with cipher AES256-SHA (256 bits))




You can specify SenderIDHeaderRegEx = \(authenticated user (.*)\)in order to extract the

“[email protected]” username of an authenticated user.

Sometimes, as in a dedicated server hosting environment, it is enough to know the abusing server rather

than a user account. In this case you can set SenderIDHeaderName = Received and ctasd will

automatically calculate the originating server IP address and use this as the SenderID.

Note: If you allow forwarding of email from external accounts, the SenderID will be that of the

originating server that is outside of your environment.

Usually, service providers have complex systems that may use different methods of user identification or

authentication, resulting in two or more potential SenderID headers. To address this you can combine the

use of SenderIDHeaderName and SenderIDHeaderRegEx. If both are specified, ctasd will look for the

SenderIDHEaderRegEx and then, if not found, for SenderIDHeaderName.

Please keep in mind that complex regular expressions are not recommended because ctasd processes all

outgoing messages and having complex regular expressions can result in higher CPU use and may affect

performance.

EMAIL CLASSIFICATION

It is important to understand ctasd email classification categories and how they help to identify

potentially “bad” senders.

ctasd classifies messages into five categories as described in the table below:

Classification Explanation What does it say about sender?

Confirmed Spam messages containing known and

verified spam patterns. Confirmed spam can

be safely blocked.

Senders of recurrent confirmed spam

emails are usually compromised or

spammer’s accounts.

Bulk Bulk contains dubious mass mailing and

spam messages.

Accounts sending Bulk email can be used

for direct marketing or other dubious

activity.

Suspect When Local RPD identifies suspicious email

distribution from a particular sender, it

marks messages as suspected. Suspected

message samples can be sent to

Commtouch’s SpamLab for an immediate

analysis and re-classification

In this context “Suspected” merely means

that a sender is sending higher volume of

email than expected. Accounts sending a

lot of “suspected’ email, are candidates for

white/blue listing.

Unknown This classification represents all good Regular account




messages.

Non-Spam Messages that are confirmed, without

doubt, as coming from a trusted source. This

classification is very rarely used.

A sender of “Non-Spam” email is a known

trusted account.

SENDERID COUNTERS

In order to monitor sender activity ctasd manages email counters for each SenderID. There are 7 counters

in total and you can enable or disable them according to your needs. ctasd counts SenderID statistics as

follows:

Suspected counter – counts Suspected messages

Bulk counter – counts Bulk messages

Confirmed Spam counter – counts Confirmed Spam messages

Spam counter –combines Bulk and Confirmed Spam counters

Viruses – viruses are counted only if you purchase and enable the Zero-Hour AV service from

Commtouch

Total amount of email –counts all email messages – good and bad – sent by a SenderID

Recipients per message – shows the amount of recipients the message is addressed to

All but the Recipients counters are calculated based on a sliding time window (see SenderIDWindow and

SenderIDWindowSize paragraph below).

By default, only Suspected, Spam, and Total Messages counters are enabled. For better granularity we

recommend enabling at least Suspected, Bulk, Confirmed, and Total messages counters.

Note: During OSP evaluation you may want to enable all counters in order to review all aspects of the

system.

You control SenderID counters using the CounterMask parameter. This is a bit-wise flag that defines what

counters are enabled. To enable the recommended minimum of counters use CounterMask = 27 and to

enable all counters use CounterMask = 127. If you do not plan on using the Zero-Hour AV you can exclude

Virus counters and set CounterMask = 63.




SENDERIDWINDOW AND SENDERIDWINDOWSIZE

SenderID counters are calculated based on a sliding time window. By default, the time window is 5

minutes, which means that the numbers in the counters represent statistics from 5 minutes prior to the

moment a message was scanned by ctasd.

The time window is specified using two parameters:

SenderIDWindowSize – the size of each time window.

SenderIDWindows – the number of time windows to maintain SenderID statistics.

In general, the default SenderID time window size is good for any environment. However, in some cases

you may need to modify it in order to better identify spamming accounts. Usually, spammers’ tactic is to

send many spam messages in a short amount of time before their (or a compromised) account is

detected and blocked by a service provider. However, some spammers adopt a different scheme and

send low volumes of spam from a large amount of compromised accounts in order to slip “under the

radar”. As long as they send more than a couple of spam messages in a five minutes period, you are

covered and ctasd will catch the spamming SenderID. That being said, if they send one or two spam

emails every 10 minutes, ctasd will reset the spam counters and miss the “bad” SenderID. Al you need to

do in such situation is increase the SenderIDWindow size to 10 and the “slow spammers” will be caught

by ctasd.

Note: When you increase the SenderIDWindow or SenderIDWindowSize parameters take into account

that it will also increase the system memory use by ctasd.

SENDERID THRESHOLDS

You can specify up to three thresholds for each counter and ctasd will report when a sender crosses a

threshold. By default, all thresholds are disabled, and only the first threshold parameter is listed in the

ctasd.conf file. You can add up to two additional threshold parameters for each counter; however, it is

rarely required. For example, the first threshold for Confirmed Spam counter is specified using

ConfirmedThreshold1 configuration parameter. You can add ConfirmedThreshold2 and

ConfirmedThreshold3 parameters to set three thresholds for the Confirmed Spam counter.

The below tables explains thresholds and their potential use for each counter.

Threshold Recommended use case

SuspectedThreshold1 Suspected threshold is used for specifying when to submit suspected messages to

Commtouch Spam Lab for additional analysis and reclassification. We recommend

setting initial SuspectedThreshold1 value to 3, which means if a sender sent three

suspicious messages within 5 minutes, Commtouch will take a closer look at those




messages. This value can be adjusted based on the email traffic trends in your

environment. If you set the value too high, suspected email samples may never

be sent to Commtouch. If the value is too low ctasd will submit too many samples

of good email, thus creating “noise” in the system.

BulkThreshold1 Bulk threshold can be used to identify bulk and spam senders. Our

recommendation is to set the first bulk threshold to 1 so you will see SenderID

that sends bulk spam right away. You may want to specify BulkThreshold2 and

BulkThreshold3 and use them in your policy enforcement logic. For example: a)

notify abuse team when the first threshold is crossed (they can decide whether to

add the SenderID to blue or white list); b) block email from the SenderID when

the second threshold is crossed; c) disable SenderID account when the third

threshold is crossed

ConfirmedThreshold1 ConfirmedThreshold1 should be set to 1, because usually we want to know about

confirmed spam senders right away. It is recommended to block confirmed spam

before it leaves your system. However, if your customers are extremely sensitive

to outbound false positives (FP), you can avoid FP by design by setting the first or

the second confirmed threshold to 2 and blocking from the second confirmed

spam message. By doing this you make sure that the first copy of a message is

always sent out. You can use the third confirmed threshold to specify when to

block the account. In our experience, confirmed spam senders are usually

compromised or spammers’ accounts and it is safe to block confirmed spam

and/or disable confirmed spam senders.

SpamThreshold1 Spam threshold combines the above Bulk and Confirmed thresholds. You can set

and use this threshold to add even more granularity and flexibility to your policy

enforcement logic. For example, your Bulk threshold is set to 2 and your

Confirmed threshold is set to 1. You can set the spam threshold to 2 and take

actions when spam, bulk, or both cross this threshold.

VirusThreshold1 Virus threshold can help you identify malicious senders. Usually, there is less

tolerance to virus senders and this threshold can be set to 1.

TotalThreshold1 Total threshold shows you the total amount of email – good and bad –sent by a

SenderID. You can use this counter to enforce outbound traffic rate limitation.

Although, you may be enforcing daily or hourly rate limits today, the total

thresholds allow you to set more granular rate limitation based on the SenderID

time window (5 minutes by default) Your customers will be able to send more

email overall, but their traffic will be spread evenly and you can prevent bursts of

bulk mail going out.

RecipientsThreshold1 Recipient threshold can be used to limit size of email distribution. Most likely, you

already have that kind of limitation in place on your mail server. You can use this




threshold in conjunction with all other thresholds to build a robust and flexible

policy enforcement system.

We recommend enabling and using at least SuspectedThreshold1, BulkThreshold1, ConfirmedThreshold1,

and TotalThreshold1.

CTASD HEADERS

ctasd returns the following information for each outbound email it scanned:

Note: This is a full list of headers. Some of them may not be visible, according to your “CounterMask”

configuration.

X-CTCH-Spam: (Confirmed/Bulk/Suspect/Unknown/NonSpam) - spam categories

X-CTCH-VOD: (Virus/High/Medium/Unknown/NonVirus) – virus categories

X-CTCH-RefID: str=0001.0A010207.4F2C16ED.007B,ss=4,re=0.000,fgs=32 – message tracking ID

X-CTCH-SenderID: [email protected] – ID of the message sender

X-CTCH-SenderID-Flags: 8192 – Threshold flag (see ctasd integration manual for more details)

X-CTCH-SenderID-TotalMessages: 5 – total amount of messages sent in the specified time period

X-CTCH-SenderID-TotalSpam: 1 - amount of Spam (Confirmed + Bulk) sent in the specified time period

X-CTCH-SenderID-TotalSuspected: 0 - amount of suspicious messages sent in the specified time period

X-CTCH-SenderID-TotalConfirmed: 1 - amount of confirmed spam messages sent in the specified time

period

X-CTCH-SenderID-TotalBulk: 0 - amount of bulk messages sent in the specified time period

X-CTCH-SenderID-TotalVirus: 0 - amount of infected messages sent in the specified time period

X-CTCH-SenderID-TotalRecipients: 0 – amount of recipients the message was sent to

A combination of headers can be used to create outbound detection policies. For example, to avoid false

positives by design, you may decide not to block the first spam message from a sender, but block all

messages that have X-CTCH-Spam: Confirmed and X-CTCH-SendrID-TotalConfirmed: 2 headers. To block or

deactivate a spamming account you can use a combination of X-CTCH-SenderID: <sender> and X-CTCH-

SenderID-Flags: <value> to determine that a spam threshold has been crossed.

mailto:[email protected]




Recommended actions and policy enforcement ideas

BLOCKING SPAM

Every scanned message receives one of the five spam classifications: Confirmed, Bulk, Suspect, Unknown,

Non-Spam. The classifications are independent from SenderID statistics and can be used for immediate

action.

Confirmed spam category has virtually zero false positives and therefore you can safely block confirmed

spam messages, thus significantly reducing the amount of spam leaving your network.

Bulk spam category may contain direct marketing emails and locally distributed newsletters as well as

spam. One option is to review your bulk email traffic and white- or blue-list all legitimate senders, and

start blocking bulk messages after that. Another option is to take a more aggressive approach, which

could be appropriate if your email policy/service agreement prohibits bulk mailing, start blocking bulk

messages and then respond to user complaints on a case-by-case basis. If you provide bulk email services

for a fee, this may be an opportunity to prevent abuse of your mail service and generate additional

income by converting complaining customers to bulk email service users.

Suspected messages are not considered spam and therefore should not be blocked. Commtouch outbound

system has a mechanism that converts suspected spam to bulk or confirmed spam categories, and ctasd

will block those emails according to your Confirmed/Bulk spam blocking settings.

Unknown or Non-Spam messages are good email that should not be blocked.

Note: In some cases you may want to consider issuing a bounce back message to the sender when you

block spam. It can serve two purposes: first, a legitimate sender will know that his message did not go

through, and second, spammers will know that there is an anti-abuse system at work and may decide

to stop abusing your system because of an extra-effort required to send spam.

Blocking spam is a traditional way of dealing with the problem, but in case of outgoing email it does not

address the source of the issue and becomes a never ending process. To really resolve the outbound

spam problem you need to block the spammer. The next section discusses various options for blocking the

spammer.

BLOCKING A SPAMMER

Beside message classification, ctasd reports a SenderID for every message. By using a combination of

message classification, thresholds, and sender identification information, you can automatically take care

of abusive accounts. You can block spamming or compromised accounts in real-time and make it very

challenging for spammers to abuse your infrastructure; therefore you can address the source of the




outbound spam problem. You can take the outbound protection system to the next level by combining

ctasd results with you provisioning system and automating certain actions.

Note: Actions that you apply will usually depend on the existing anti-abuse policy, but you may

consider adjusting your policies in order to take full advantage of the Commtouch Outbound

Protection solution.

Let’s review a few possible scenarios that you can employ to detect and block spamming accounts.

NEW ACCOUNTS POLICY

When you identify a new, recently registered account is sending confirmed or bulk spam messages, you

may consider deactivating that account right away. Most likely, the account was created with the purpose

of sending spam or bulk email. If you don’t want to be too aggressive with the new registrations, you can

implement a mechanism that sends a warning message to the account with a confirmation link. You can

block the account if you do not receive the confirmation, or if the account continues sending spam.

Note: You can use the same policy with old, inactive accounts that “wake up” and start sending spam.

EXISTING ACCOUNTS POLICY

Exiting accounts that send spam are usually compromised. You can employ either one or a combination of

the following tactics:

Limit the amount of emails when SenderID crosses the first Confirmed, Spam, or Bulk threshold. You

can use the TotalMessages threshold to enforce rate-limiting (for example, allow sending only one

message every 5 minutes).

Temporarily restrict email sending when SenderID crosses the first Confirmed or Spam threshold.

Optionally, send a notification to the account holder.

Change password on an account when SenderID crosses the first Confirmed or Spam threshold, and

notify the account holder about the incident.

Suspend an account when SenderID crosses the first/second Confirmed or Spam threshold. Notify the

account holder about the suspension of his/her account.

Deactivate an account when SenderID repeatedly sends spam and other measures did not help.




BULK SENDERS

Your service agreement or outbound email policy may not allow regular accounts to send bulk or

commercial emails directly. If that’s the case, you may require customers to use your dedicated

infrastructure or third party mass-mailing services.

In order to identify bulk senders you can set multiple TotalMessages, Bulk, and Suspected thresholds and

take various actions when those thresholds are crossed. For example, you can send a

warning/explanatory notice and provide your customer with available mass-mailing options.

If you allow legitimate mass-mailing, you can identify bulk senders and add them to a white or blue lists

to prevent false detection of bulk email as spam. (For more details about white and blue lists see the

“Commtouch outbound integration manual”)

ANTI-ABUSE PROCESS AUTOMATION

One of the big advantages of using Commtouch Outbound Protection system is that it allows you to

automate many anti-abuse tasks and reallocate valuable resources that otherwise have to deal with

system and user issues caused by outbound spam. Below we provide a few automation ideas that you

can use to automate certain anti-abuse processes.

EMAIL NOTIFICATIONS: you can send email notifications and educate or warn your customers according

to their email sending behavior. You can interact with customers by asking them to confirm receipt of

your message, or reactivate email delivery/remove limitations by adding a confirmation/activation link

to the notification message.

WEB TRAFFIC REDIRECTION: In some cases, when you suspend an account or change the password,

email notifications won’t work. In that case Internet service or Web mail providers can redirect inactive

accounts to a web page where they explain the reason for account deactivation, and possible actions that

can help the customer resolving the spam issue. Like with email notifications, you can make this an

interactive experience and allow the customer to reactivate the account, or perform other actions that will

prevent a call to your helpdesk.

SMS NOTIFICATIONS: Another option to handle spamming accounts is sending SMS with a new password

to the account holder when his account is caught sending spam. It serves several purposes. A customer

knows that his account is getting abused, and he has immediate access to his account using the new

temporary password. If the account continues to send spam after the password change, it is safe to

assume that the account sender intentionally abuses your system and you can disable his/her account.

Some of the suggested options can be implemented using built-in features or APIs of your existing

provisioning system, while some may require scripting and integration. In any case, we firmly believe

that investing in anti-abuse automation with the help of Commtouch Outbound Spam Protection system

will help you to improve your reputation, customer service, and minimize operation costs.




Summary

The Commtouch Outbound Spam Protection solution enables service providers to identify and block

outbound spam caused by compromised user accounts, malicious users, and zombie computers. It is

designed to block locally-generated spam unique to each service provider in real-time, as well as to

provide the identity of the spammer to the service provider abuse teams. It also enables service providers

to automate many anti-abuse tasks and improve the quality of the outbound traffic while saving on

resources.

For more information contact your technical account manager at Commtouch.

This document is proprietary and confidential. Commtouch Software Ltd. All rights reserved. The use, copying and handling of this document is subject to the restrictions set forth in the NDA between Commtouch and the recipient of this document (or the employer or other entity through which recipient has lawfully obtained this document).

Commtouch Software Ltd. Recurrent Pattern Detection, RPD, Zero-Hour and GlobalView are trademarks, and Commtouch, Authentium, Command Antivirus and Command Anti-malware are registered trademarks, of Commtouch. U.S. Patent No. 6,330,590 is owned by Commtouch.

best practices for commtouch outbound spam … practices for commtouch outbound spam protection...

Documents