Technologies behind Spam Filter Software

January 8th, 2009

How to make a spam filter work?

Creating an effective spam filter is undoubtedly a difficult task. The main problem is that spammers are constantly devising new ways to avoid the solutions that are put forth. Every single technique can be overcame, it is just the matter of time. No wonder that spam filter software vendors have been implementing multiple techniques to be working simultaneously.

What are the technologies used in spam filter software?

Almost each vendor has its own technological approach. Some of them may be completely useless, some may work only when used with others. I will give brief explanation of the most general spam filtering technologies.

Heuristics

These are the spam filter solutions that use computer-driven methods to fight spam. The system tries to detect patterns in the messages, which are associated with spam. Spam filter software that uses these can be very effective, and the effectiveness should improve with time. Main drawback is that it takes time for the system to adapt and when used alone, it may lead to disputable results.

Checksum Database

Spam filter using these methods assigns a unique identifier to each spam message it comes across. This way a database of such identifiers is created and each new email is compared with the records. Main drawback is that there are already many spam tactics developed to fool spam filter software using this method, and it needs a huge network to work at an optimal level.

 

Three more coming up soon.

 

Spam filtering backfires

December 12th, 2008

I have recently came across an interesting post about the side effects of using a spam filter.

Have you ever had your legit mail blocked, or wasted hours looking for missing mails in your spam folder? According to the article, one of the reasons for such situations may be inclusion of, let’s call it, keyword of high risk.

One of the examples mentioned was a reference to ‘young adult’ – a common classification of books for adolescents. It is easy to understand why this phrase is treated as a potential spam. Nevertheless, the simple fact, that a single word may be a decisive factor whether my email is delivered or not, remains.

We need spam filters without a shadow of doubt, but these need to be one step ahead, not behind the spammers.

Am I exaggerating? Of course I am. I just don’t want to settle down with the idea, that concludes the mentioned post, namely that the spam filtering crusade has chosen lesser of two evils.

If a spam filter stops one or two private messages I can live with that, but I when I use email for business communication, 99% of success may not be enough.