How to manage and customize MailCleaner 2020.01

This is the third part of a guide on how to install, configure and manage the MailCleaner antispam server.

In case you missed the other two parts:

How to install MailCleaner 2020.01

How to configure MailCleaner 2020.01

Now we are going to take a look at the most common management tasks, as well as how to perform some more advanced configuration.

Tracing messages

The most common thing I do in MailCleaner is tracing an e-mail message to find out how it’s been processed. It’s great for just looking at the latest messages arrived and also excellent for finding a specific delivery that should’ve gotten through, but didn’t, for example.

If you go to “Management > Tracing”, select your domain and just hit “Refresh”, you will see something like this:

Values in status fields vary between:

• Arrival status: Accepted or Rejected (and Relayed, if you use outbound)
• Spam status: spam or “ - ” (not spam)
• Content status (virus): Detected, Cleaned or “ - ” (nothing detected)
• Deliver status: Completed, Quarantined or Whitelisted

You can search by date, recipient and sender address.

If you hover over one “Rejected” status, you can see why it was refused:

And If you hover over the “Spam status” field (of any message, not just spam) you can see the Spamc (SpamAssassin) score for that particular message, as well as which rules had a match:

You can expand any message to see the trace info, containing the result of every stage the message went through:

Newsletter

Since version 2017, MailCleaner has a complementary spam filter tailored to detect newsletters, bulletins, journals, weekly reports, etc. That’s why you see “Newsl” in addition to “Spamc” when looking at the Spam analysis score for a particular message.

E-mail messages identified as Newsletter are by default quarantined as well. Each user has the ability to “accept” a particular newsletter, which will automatically allow its future occurrences. This is done by logging to the quarantine web interface at https://your_mailcleaner_name.

Quarantines

Spam quarantine

As admin you can see every e-mail that’s been retained as spam. Go to “Management > Spam quarantines”:

You (or the respective user) can release any message to be delivered to its intended mailbox. There’s also a “filter” icon to “Request a filter adjustment”, but all that does is send an e-mail to the admin, indicating that the message is a false positive.

The quarantine is interesting, because you get to search by subject too (in addition to recipient and sender), and you can click a particular message to get a preview:

The html content is stripped from the original body, so it looks fuzzy, but you get the idea.

And, better yet, you can expand the respective tab to see the full “Header” or a detailed “Rule score”:

Content quarantine

The content quarantine follows the same idea, but there you will find messages that were blocked because they contain virus or other known malicious code. You can find it in “Management > Content quarantine”.

End user quarantine

The quarantine web interface available to end users is similar to the Spam quarantine GUI from the /admin interface we just saw, but even more friendly and simple.

And it has a “Configuration” tab where users can define their preferences, including the ability to add an address to his/her own individual whitelist:

Note: users can only see quarantined mail and change settings for his own account (e-mail address), as previously explained. If someone has multiple SMTP addresses or needs to control the quarantine of mail-enabled groups, for example, you would have to create a MailCleaner user with the “hotline” role for such person (more bellow, on Delegating).

Logs!

You will find every log you need in “Monitoring > Logs”. The nice thing is that you don’t have to access a shell prompt and start hunting log files. You can view them in the web interface or download them. And the web log preview is functional enough, with paging and search.

The most useful is the Incoming MTA log, where you can spot most of reception issues or check if there is someone trying to brute-force a SMTP password, or another funny business:

Delegating administrative access

You can create other MailCleaner users in addition to the default admin. Access “Configuration > Accesses” and click “New administrator +”.

There are three roles:

• hotline
  Can manage quarantines (release e-mail, etc.) and tracing.
• manager
  hotline + change configuration of domains (that you specify).
• administrator
  Just like the admin user, all privileges.

Creating your own rules

You can change the behavior of Spamc by creating a custom configuration file in /usr/mailcleaner/share/spamassassin on you MailCleaner server. This will allow you to:

• Write your own rules to detect and block e-mail messages that contain some specific characteristics;
• Change the applied score for default (already existing) rules;
• Alter configuration options, like the “required score” limit, after which a message is considered spam.

You have to create a file with the .cf extension in the provided path. E.g.: /usr/mailcleaner/share/spamassassin/Custom.cf

I recommend that you use a SCP or SFTP file explorer such as WinSCP, FileZilla or even Nautilus (that’s the native file manager on Ubuntu) to create and edit this file:

After you create/edit this file, you have to restart the “Filtering service” in “Monitoring > Status”.

I also recommend running spamassassin --lint before restarting the service, to make sure your syntax is OK.

Syntax

SpamAssassin rules are written using basically three statements:

• rule type, the most common: header, body and rawbody
• score : to define how many points a message gets if the rule matches
• describe : to briefly explain what the rule means

And there is the meta statement, to combine rules.

The header command needs a parameter to define which header field we are trying to evaluate, like From, Subject, etc.
What differentiates body from rawbody is that “body” does not include the html code and “rawbody” does. So if you want to search the message body for a specific <a> tag, for example, you have to specify rawbody.

Then, the evaluation is usually done using regex. I recommend Regex 101 https://regex101.com to write and test your regex syntax.

So here are a few examples:

rawbody   GOOGLE_APIS   /https?:\/\/storage\.googleapis\.com/i
score     GOOGLE_APIS   1.5
describe  GOOGLE_APIS   Has a hyperlink that points to Google APIs
header    DOMAIN_NUM    From =~ /\@[\w-+=\.]*\d{2,}[\w-+=\.]*/
score     DOMAIN_NUM    1.0
describe  DOMAIN_NUM    Domain name has 2 or more numbers
header    LONG_SUBJECT  Subject =~ /.{120,}/
score     LONG_SUBJECT  0.8
describe  LONG_SUBJECT  Subject line is very long

Note: your regex must be enclosed in slashes to be treated as regex.

Required score

You might want to change the score limit after which a message is treated as Spam. This can be done by adding the following option to your custom.cf file:

required_score 4.0

The default is 5.0. But I usually find it a little high and many spam messages get through with a score between 4.0 and 5.0. Another approach is to increase or decrease the score on default SpamAssassin rules to improve chances of catching spam and reducing false positives, considering your scenario.

A good strategy is to monitor your messages in Tracing and try to see which rules match more often in the spam messages you usually get, and then declare these rules in your custom.cf file with a higher score, like this:
score NAME_OF_RULE 2.0 (just the score statement is enough)

Sample Custom.cf

Here I leave a sample for you, already with some goodies in it:

And some references:

MailCleaner Support

Apache Software Foundation

Use MailCleaner for outbound e-mail

There are a few reasons why you might want to relay your outbound mail through MailCleaner. In a common scenario, this is not needed, because your mail server can perfectly send your mail to it’s destination, either directly, using MX records, or by an external relay (if you have or subscribe to one).

But you may find use for this if you want to:

• Sign your e-mail with DKIM (if your mail server doesn’t natively support it, like Microsoft Exchange);
• Have your traffic logs in one place (e.g.: see everything in MailCleaner’s Tracing)
• Scan your outgoing mail for viruses

Authorizing relay

All you need to do is: go to “Configuration > SMTP > Connection control” and add the IP address(es) of your internal mail server to “Allow external relaying for these hosts”. Then click “Submit” and restart the MTA service.

Public DNS records and EHLO/HELO name

If you’re sending e-mail messages from MailCleaner directly to their destinations, you sure need to take care of SPF records for your domains. And you need to ask your ISP to configure the right PTR record for the public IP you’re using for your MailCleaner, so a reverse DNS lookup on that IP would resolve to the hostname you’ve chosen to use with your MailCleaner server.

Also, you probably want to define the exact name (your hostname) that you want MailCleaner to use while talking to other mail servers, I mean, the name presented when sending a EHLO smtp command. This hostname is included in the “SMTP banner” and must match the reverse DNS value (PTR) of your public IP. This is done on Configuration > Base system > DNS settings:

Send mail to an external relay

If you already use an external relay (smarthost), either for IP reputation or other reasons, you can tell MailCleaner to also relay through it (mail server -> MailCleaner -> external relay).

However, MailCleaner doesn’t support authenticating against the relay (smarthost), which can be a deal breaker for you. Your relay service would need to have the flexibility of allowing an IP address (your IP) as a trusted source, for example.

If you need this feature, it’s available in the domain properties, on the “Outgoing relay” section. You have to check “Use a smarthost to relay to”, save your settings, then fill in the smarthost server and port and save again.

E-mail messages sent by MailCleaner will appear in Tracing too, with a status “Relayed”. You can expand the trace info to make sure the message is being treated the way you expect.

DKIM

DKIM signatures are a nice feature and, if you don’t have it in your domain yet, you should consider configuring it right away. Coupled with SPF, it helps people know that the e-mail messages they’re getting are really coming from you. If you want MailCleaner to sign your outbound messages using DKIM, it’s quite simple.

To implement DKIM you need:

• A DKIM compatible service in your mail infrastructure that can sign the header of e-mail messages;
• A simple and friendly name to use as “Selector”;
• A public-private RSA key pair;
• A new TXT record in your public DNS zone;

But MailCleaner takes care of almost all of that for you. In “Configuration > Domains”, select your domain and change the section to “Outgoing relay”. Then:

1. Change “DKIM signing” to: This domain (yourdomain.com)
2. Fill in a “Selector” name of your choice (lower case letters, no special characters, please). Suggested name: default
3. Click in “Generate new private key”
4. Click “Submit” to save it

After you save, this nice box appears with your public key and the correct syntax for creating the proper DNS record in your domain:

Now you have to access your public DNS service, and in your domain/zone:

1. Add a new record of type TXT
2. Type the name of the record, that should be: yourselector._domainkey
   (e.g.: default._domainkey)
3. In the value of this record, paste the DKIM record contents that MailCleaner has generated for you (that starts with v=DKIM;). Do not copy the quotes. In our example above, the value would be:

v=DKIM1;k=rsa;p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC3ehTi2ePCRPCsD9mH8+WUqHJCwZEyHnHjA+eJy7dOS/xLSE8JFbYk+hGjzQCxwJnTC8SCA88vqiiBcRtAG+x5YHvHRDka7NibLwjvI+Qd2sK1Gu3u5dK2rproN3YLG3MHTrqJ6uuw7G12d5+ltHBpnAZeq4wEErnjL7gjFVjUVwIDAQAB

Be careful not to include any line breaks or quotes. Save it and let the DNS record propagate. All set!

You can test the DKIM record using these online tools:

• https://mxtoolbox.com/dkim.aspx
• https://www.mail-tester.com/spf-dkim-check

And you can test the whole singing and validation process using this great tool:

https://dkimvalidator.com

Altering the SMTP banner

You might want to change the welcome message that the incoming MTA presents to connecting servers. This could be a good idea, because the less information about your infrastructure you offer publicly, the better (like your antispam software and version).

The default banner is: 220 <yourdomain.com> ESMTP MailCleaner (Community Edition <current version>) <the current date and time zone>

In reality, the exact banner value on Exim configuration in MailCleaner is (edition and version is resolved during service startup):

$smtp_active_hostname ESMTP MailCleaner (Community Edition 2020.01) $tod_full

If you want to change that, modify the file /usr/mailcleaner/etc/exim/exim_stage1.conf_template and replace the string __SMTP_BANNER__ , but make sure to keep the variables $smtp_active_hostname and $tod_full (and their respective positions):

Your smtp_banner option could look like this, for example:

smtp_banner = $smtp_active_hostname ESMTP Your_Company $tod_full

After saving this file, restart the “Incoming MTA” service. Keep in mind that this value can be replaced (back to default) during product updates. There is also a value for this in MailCleaner’s database, but it seems to be left out (it’s not taken into consideration and doesn’t define anything). Let’s hope the developers create an option for this in the GUI.

Prevent virus-detected messages from being delivered

For some reason, the virus scan engine gives you only the option “Drop known viruses silently”. Then when a messages has a virus in the attachments, for example, MailCleaner deletes the virus and delivers the message to its intended recipient with a prepended “Attention Virus” in the subject.

If you also think this is not desirable, because these are likely scam messages and users should not get such e-mail in any way, here’s how to change this behavior:

1. Access your MailCleaner server using SSH, or through the console
2. Edit the file MailScanner.conf_template:
   nano /usr/mailcleaner/etc/mailscanner/MailScanner.conf_template
3. Then locate Deliver Cleaned Messages and change it to no.
   The file it’s quite big, so you might want to use Ctrl + W to search.
4. Save it (Ctrl + X) and restart the MailScanner service:
   /usr/mailcleaner/etc/init.d/mailscanner restart

Reference: https://forum.mailcleaner.org/viewtopic.php?f=12&t=2296&p=9965#p10116

Loading Whitelist entries in the DB

This is just a quick tip. You may already have a list of trusted domains somewhere that you want to add to your whitelist. This could be tedious because MailCleaner doesn’t have a import/export feature for whitelist entries yet.

The table that holds these records is called wwlists. You can access the MySQL database, like we’ve seen before, using:

/usr/mailcleaner/bin/mc_mysql -m
use mc_config;

The syntax for inserting into this table is:

INSERT INTO wwlists (sender, recipient, type, expiracy, status, comments)
values
('*@trusted.com', '@yourdomain.com', 'white', '0000–00–00', 1, '-'),
('*@another.net', '@yourdomain.com', 'white', '0000–00–00', 1, '-'),
('*@keepgoin.ca', '@yourdomain.com', 'white', '0000–00–00', 1, '-');

Remember that the whitelist is per domain configuration. I recommend that you first add a few addresses manually using the admin web interface, and run a SELECT * FROM wwlists limit 5; for example, to have an idea of how this works.

Note: there’s also a script in /usr/mailcleaner/tools/insert_bulk_wwlists.sh if you want to take a look.

Using mysqldump

If you want to use mysqldump for some of this, here is a sample of how to use it with MailCleaner (maybe you’re migrating from another server, or want a backup):

/opt/mysql5/bin/mysqldump -S /var/mailcleaner/run/mysql_master/mysqld.sock -umailcleaner -pYourMCdbPassword mc_config wwlists > /root/dump_wwlists.txt

Note that mysqldump includes “DROP table” statements in the dump file, so it’s dangerous to just use this file later without verifying/modifying the code.

Clustering

MailCleaner has the ability to form clusters, with a master-slave approach. It’s not a cluster in a sense that two servers will share the same IP address for high availability or something like this. It’s more like a synchronization of files and database content to provide a central place for management, plus extend the number of MX servers you can have for mail reception, which provides redundancy. And you can have your servers in different locations (given that you take care of the necessary traffic between them).

You can have one master server and a number of slave servers. After you form a cluster, the configuration can only be changed in the master server. And the end user interface for accessing the spam quarantines will only be available at the master server too.

We’re not going to cover clustering here. I’ll just point you in the right direction. You need to have a significant mail income, different sites, or a tight availability requirement in order to demand a cluster.

The setup of a MailCleaner cluster is done using the following script in you MailCleaner server:

/usr/mailcleaner/scripts/configuration/slaves.pl

Here are some references:

MailCleaner Support

Topic about cluster nodes in different locations: https://forum.mailcleaner.org/viewtopic.php?f=2&t=2927&p=12243#p12229

Security concerns

Servers connected to the Internet are never safe, as you know. Hacker groups are always trying their luck exploiting vulnerabilities on web servers, mail servers and alike.

Recently a security flaw was found on Exim, that could allow an attacker to execute code remotely and take control of the target server (or at least take it down). The breach has already been patched and you can find out more here and check the bulletin here: CVE-2019–16928. What draws attention in this type of case is how specific and obscure the code fault is, in despite of how careful the development team might be, and how helpless your server is when exploited this way.

So you need to make sure you:

1. Keep you server up to date, fully patched. MailCleaner does that automatically, but is a good idea to check from time to time. The admin home page shows latest news on updates and in the “Monitoring > Status” you can see the patch level.
2. Couple your MailCleaner installation (and any other published server) with a good firewall solution, that can block connections based on “GeoIP” and “whois” (ASNs), for example, among other things.
3. Monitor the Incoming MTA log and search for things like “incorrect authentication” or simply “error” to spot problems and malicious connections. Run offending IPs against tools like:
   - MXToolBox (IP to ASN and subnet list)
   - DNS Checker (IP location)
   - DNS Checker (ASN Whois)
   - UltraTools (ASN quick info, including country)
   and then block connections originated from that whole country, if you’re not expecting to receive e-mail from it, or at least block that particular AS number, for example. It’s a good idea to block connections from top spammer countries (or some of them, depending on your needs, of course).

Conclusion

We’ve reached the end! As you can see, MailCleaner is a complete antispam solution and I’m happy to share all this information with you. The VMs available are a bit outdated right now and Debian 8 Jessie is in “End of life”, so we’re waiting for new versions and when they release something I’ll try to update this content. We have news that a new release is on its way, as per this post, but it’ll take some time due to the complexity of the product and the fact that this is a Community edition. Also, if you like the software, please consider donating, or trying the Enterprise edition, and post your opinion on some websites like Capterra, Source Forge and SpiceWorks. If you have any doubt or suggestion, please leave a comment. Let me know if I forgot something.

Cheers,

Victor.