The British Caving Association

Hi Cookie, I have a number of questions about spam filtering. Obviously, asking you to provide a tutorial would be a bit cheeky, so if you know of a web page that will answer these questions, this would be a great help. All of these questions refer specifically to the BCA server and CPanel...

1. If SpamAssassin is disabled on an account, does the server do any spam filtering all, or does it still reject messages where the sender is blacklisted?
2. If the answer to the above is "yes" then what additional filtering itakes place when SpamAssassin is enabled?
3. If the SpamAssassin config page (click on the SpamAssassin icon in CPanel) is set to Auto Reject, based on a score, what is the difference between setting the score there, and setting it in an account-level filter (i.e. by clicking on Account Level Filtering in CPanel)?
4. Should we be enabling DKIM and SPF (click on Email Authentication in CPanel), or are the "complex" issues with these that are best avoided . Specifically, because I send my mail via a third-party (Gradwell, or BT, depending), but with Sender set to bcra.org.uk, so do I need to do anything special?
5. If I try to enable DKIM I get a warning: In order to ensure that SPF or DKIM takes effect, you must confirm that this server is an authoritative nameserver for hidden-earth.org.uk. If you need help, contact your hosting provider. But there is no indication of how I do "confirm" that. The nameservers for hidden-earth are set to ns1.memset.com which is presumably correct, but how do I "confirm" that?
6. Some time ago, you suggested that I change BCRA's name servers to Memset instead of ns.hosteurope.com, which is where they are now (set at 123reg.co.uk) but I cannot remember why I didnt do it. Im sure I came up with a reason, but what was it? Can you remember? Anyway, if I do that, now, I still, presumably, need to "confirm" it (as above).
7. Obviously the type of spam people are targeted with varies depending on who you are. The nature of the spam Im receiving (and I assume, other people with BCRA and Hidden-Earth addresses) suggests that it could be minimised if we were able to write some custom rules. The first is that any message containing an unencrypted ZIP attachment should be rejected. (Ive seen this on other mail servers). The second is that any message that was also sent to a non-existent user should be rejected. (I gather that's how Cloudmark works). But I guess that both of these rules are simply too intensive of manpower for any of us, here, to think about

Ok... managed to answer a couple of those questions myself

3. If the SpamAssassin config page (click on the SpamAssassin icon in CPanel) is set to Auto Reject, based on a score, what is the difference between setting the score there, and setting it in an account-level filter (i.e. by clicking on Account Level Filtering in CPanel)?

Answer: if you do the former, a rule is written, which appears at the bottom of the list at "Account Level Filtering".

any message containing an unencrypted ZIP attachment should be rejected

Ive now noticed that one of the filter rules allows you to do a match to Body text - didnt spot that before . So a rule such as Body Contains Content-Type: application/x-zip-compressed; will do the job! Of course, that's a nuisance for anyone wanting to send a zip file, but they just need to re-name it. Im also now using the rule: Any Header Matches Regexp mailer.*(PHP|sourceforge|[a-z]{4,6}[^ ][0-9]{2}|[A-Z][a-z]* v[0-9].[0-9]) but there's a slight puzzle: the regexp doesnt seem to like "^" or "$", so I cant say "begins with" or "ends with", which is what I want to. Ideas anyone?

David Gibson wrote:Ive now noticed that one of the filter rules allows you to do a match to Body text - didnt spot that before . So a rule such as Body Contains Content-Type: application/x-zip-compressed; will do the job!

Ive modified that rule; and the one looking for the specified regex in headers. No point in continually updating this forum though. Ask me for details if youre interested.

A while ago I 'seeded' various web sites with a set of fictitious email addresses that are invisible to humans. This was to investigate how quickly bots harvest such addresses. Its interesting: I dont think harvesting email addresses from web sites is that common, although it does happen. I have now implemented a mail filter rule that discards all incoming messages that contain one of these fictitious addresses. It can safely do this because any message containing such an address must be a spam, because the address can only have been obtained by dubious means.

I did a similar experiment, seeding my own outgoing messages with such addresses. This was to see if the recipient's computer was compromised. No feedback on this yet, although I do believe that some of the people in my circle of email contacts are the source of some of the spam attacks Im seeing. Incidentally, its interesting how the spam reduces at weekends, and over bank holidays. Im assuming this is because people do not use their (compromised) PCs so often at these times.

David Gibson wrote: Im also now using the rule: Any Header Matches Regexp mailer.*(PHP|sourceforge|[a-z]{4,6}[^ ][0-9]{2}|[A-Z][a-z]* v[0-9].[0-9]) but there's a slight puzzle: the regexp doesnt seem to like "^" or "$", so I cant say "begins with" or "ends with", which is what I want to. Ideas anyone?

answering my own posting again. Its obvious: just use \n

Another useful tip for anyone writing detailed regexps for spam filtering...

I wanted a rule that discarded any message where the To: address list contained three or more occurances of the same (specified) string (e.g. for spammers using lists of similar addresses. My problem was that the To: list may be split over several lines, and a 'dot' matches any character other than new-line. You cannot alter the 'mode' of the regexp because that is fixed by the software that processes the user-specified regexp, but you can switch different modes on and off in sub-expressions, so you just need to encapsulate your regexp in (?s:<your regexp>). So, for example, to discard any email where the string "d.gibson" appears three or more times you would write
?s: means change the mode to single-line mode for the duration of this sub-expression. .*? means match any character 0 or more times, but as few times as possible and (3,} means match the expression 3 or more times.

Any thoughts, Cookie, on...

David Gibson wrote:

1. Should we be enabling DKIM and SPF (click on Email Authentication in CPanel), or are the "complex" issues with these that are best avoided . Specifically, because I send my mail via a third-party (Gradwell, or BT, depending), but with Sender set to bcra.org.uk, so do I need to do anything special?
2. If I try to enable DKIM I get a warning: In order to ensure that SPF or DKIM takes effect, you must confirm that this server is an authoritative nameserver for hidden-earth.org.uk. If you need help, contact your hosting provider. But there is no indication of how I do "confirm" that. The nameservers for hidden-earth are set to ns1.memset.com which is presumably correct, but how do I "confirm" that?
3. Some time ago, you suggested that I change BCRA's name servers to Memset instead of ns.hosteurope.com, which is where they are now (set at 123reg.co.uk) but I cannot remember why I didnt do it. Im sure I came up with a reason, but what was it? Can you remember? Anyway, if I do that, now, I still, presumably, need to "confirm" it (as above).