• Homepage
• For Business
• For Personal
• Domain Names
• Support

nolisting - The move to Trap25

nolisting - an obscure technical geek term that means nothing to normal people. It’s actually a very effective way to combat spam. Well, actually it’s a very effective way to combat spammer methodology.

Spammers are impatient people. Their margins, such as they are, depend on getting their message on as many machines in a short a time as possible. This means speed. Blast the email out to as many people as possible as quickly as possible.

SMTP, back when it was created was designated to be an unreliable protocol. Many things could happen to an email on its winding way from sender to recipient. The RFC on SMTP actually outlines the requirement for sending SMTP servers to have a retry / resend ability. Also, it is widely accepted to have more than 1 receiving email server in case one of the servers is down or unresponsive. This is why you may have seen a domain have multiple MX (email) records:

eg.

domain.com
MX 5 spamgate.3dpixel.net
MX 5 spamfence.3dpixel.net

This means that email sent to the fictional domain.com from a legitimate email sender will have 2 servers to choose from. The ‘5′ means that the servers are of equal weighting and statistically will receive an equal amount of email to domain.com. You can of course mismatch the priorities to weight email to one particular server. In MX terms, a lower number means a higher priority. In this case MX 0 would be higher priorit and MX 10 would be lower.

Spammers tend to go for the highest priority mailserver available and do not conform to email SMTP standards. Legitimate sending servers will check the highest priority mail server and send to that. If it fails, it will retry the next highest priority server and so forth. Spammers tend to use a ‘fire and forget’ strategy that sends an email out once and only once.

Nolisting relies on this assumption, and rightly so as I’ll show below.

We looked at the statistics of spamgate.3dpixel.net and spamfence.3dpixel.net over the last several months and found that ‘Clean’ email passing through the servers was virtually identical conforming to the equal MX weighting of the 2 servers. However, Spam being processed was 5 times higher on spamgate.3dpixel.net than spamfence.3dpixel.net. That server was simply receiving (and deleting as it happens) more spam. It’s first in the list, that’s why on an MX check.

To combat this, we’ve implemented nolisting. Nolisting means that we’ve added a higher priority (MX 0) mailserver in front of our 2 spamgate servers. This server however immediately sends a ‘TCP Reset’ to the sending server - basically a ‘we don’t accept email here’ message. Legitimate servers will immediately drop to the MX5 servers and deliver email as normal. The spammers who sent a fire and forget spam email will have their email rejected at an SMTP network level and will not retry. Simple eh?

We’ve seen the results already, whilst ‘Clean’ email is still at 50/50 on the 2 MX5 servers, spam processing is now roughly equal. This ‘overflow’ is no longer present. Less spam processing time!

domain.com
MX 0 primary.3dpixel.net
MX 5 spamgate.3dpixel.net
MX 5 spamfence.3dpixel.net

Posted in Web Hosting | No Comments »

syslog-ng - The move to Trap25

I’m not sure if any of you know Linux or Unix for that matter, but there is a centralised logging system called syslog. It’s one of the oldest parts of *nix and is by modern standards, horrendously out of date. It logs system data and events to flat files… only flat files.

As part of Trap25 we need to move to a database system so our customers can see what is going on in terms of email blocking, quarantines and delivery. This means pushing log files to mysql. With the old syslog this is impossible.

I recently stumbled across syslog-ng which besides being a better replacement for syslog, can output logs directly into mysql, on local and remote servers. This is what we will be using, after some trials, on Trap25.

Posted in Web Hosting | No Comments »

Spam Filter Updates - The move to Trap25

We’ve recently rebranded our Spamgate system to Trap25. As a standalone product we can move to a more commericial model whilst also keeping the bare engine available for our hosting customers.

One major issue we faced on Spamgate was the fact that the mail relay accepted email for a domain, spam checked it and then relayed to the remote server and asked if the user existed before sending, or indeed bouncing the email. This is very expensive in terms of processor time as essentially the servers are scanning and processing defunct email.

As part of the upgrades to Spamgate to turn it into Trap25 we had to solve this issue. Our own coding has scanned each of our Plesk platforms and compiled a list of all email mailnames, redirects, groups, aliases… everything and added them to a specific type of flat file database called a .cdb file. This, combined with a recompile of our qmail systems to add the appropriate patches allows us to block email at the network level after a check of this database.

End users will see no benefit to this system as all it serves to do is reduce to spamassassin load on our scanning servers allowing us to scale them further.

Refer to: Trap25 for more details of this new project if you’re interested.

Tags: Trap 25, Trap25
Posted in Web Hosting | No Comments »

Blog Archives