The British Caving Association

BCA's IT team has recently implemented several features to enhance the reputation of email passing through its servers. This was done to reduce the amount of spam which, in turn, enhances the reputation of the servers and prevents large ISPs from periodically blocking BCA email. These measures, known as SPF, DKIM and DMARC, were put in place primarily for BCA's own domain name but they also affect everyone who buys web services from BCA and uses BCA's servers to send or receive email.

This posting is not an official message from the IT team, rather it is a list of my own observations, compiled whilst I was making the necessary changes to my own mail system in order to catch up with BCA's swiftly implemented policy. I may have got some things wrong, but this is a forum and you can correct me.

A good starting point would be to read Matt Wire's blog. You can also run a web search for spf syntax, dkim syntax and dmarc syntax.

Thanks to Matt Wire and Dave Cooke for various discussions.

SPF, DKIM and DMARC
There are three processes that BCA now applies to email - SPF, DKIM and DMARC. For each process there are three scenarios to consider - incoming mail, outgoing mail, and remote mail (i.e. what happens to your outgoing mail when it reaches a remote mail server). These notes are written from the point-of-view of domains hosted on the BCA server(s).

SPF
An SPF record is a public record, stored with your DNS settings, that says which mail servers are allowed to send email on your behalf. You can enable SPF in your cpanel. If your DNS is external to BCA then you will need to edit your DNS entries manually. The precise details will depend on your DNS hosting company. In my case, I would create a DNS ENTRY called @ with TYPE = "TXT/SPF" and DESTINATION/TARGET = something like which describes britiac2 and britiac3 as designated senders for your domain. You may wish or need to add other domains.

i) Incoming mail: If you have enabled SPF and an incoming message fails your SPF test then your mail will be rejected to the sender

ii) Outgoing mail: SPF is not applicable to outgoing email

iii) Remote mail: If your DNS contains an SPF record and this is tested by a remote server then your email might be rejected, or it may be lost or it may be passed on, but perhaps with an additional header warning of the test failure.

• One server that is known to incorrectly generate SPF fails is btinternet.com. BT's email system involves forwarding incoming mail from one server to another, but the forwarding machine incorrectly tries to verify itself as a sender on your domain. Fortunately, all it does - at the moment - is to add a header warning of the SPF failure. However, if you do want to prevent the test failing, you can add BT's internal forwarder to your SPF, viz: +ip4:65.20.0.12
• David Gibson reports that Exeter University's internal mail forwarder makes the same error as BT, and needs: +a:owa.exeter.ac.uk/16 It seems that this behaviour might be widespread.
• If you send mail via a third-party SMTP server, and you have an SPF record, then you will need to add that server as a designated sender in your SPF record. eg: ?include:btinternet.com
• If you use a BCA or BCRA email address and send via a third-party server then BCA's (and BCRA’s) policy is not to alter the SPF but to ask you to send via their own SMTP server.

DKIM
A DKIM record is a public record, stored with your DNS settings, and is the public-key part of a cryptographic signature. Your outgoing emails are signed using the corresponding private-key. If, when tested, the signature (contained in one of the email headers) is incompatible with the email body and certain (specified) headers then the email must have been tampered with.

You can enable DKIM in your cpanel. If your DNS is external to BCA then you will need to edit your DNS entries manually. The precise details will depend on your DNS hosting company. In my case, I would create a DNS ENTRY called default._domainkey with TYPE = "TXT" and DESTINATION/TARGET beginning something like but see below for more information on setting DKIM records.

i) Incoming mail: All incoming mail, regardless of your cpanel setting, is tested for a valid DKIM signature; but it is currently accepted anyway, regardless of the test result (but is subject to DMARC testing later). This BCA policy may change, so please check here or ask the IT team if you are in any doubt. One reason for accepting signature failures is that Mailman notoriously breaks DKIM signatures.

ii) Outgoing mail: if you have enabled DKIM then a signature is added to your outgoing mail.

iii) Remote mail: If your outgoing mail has a DKIM signature it is likely to be tested by a mail server somewhere along the route to its destination. If your DNS contains a DKIM record then your mail should pass the test. If your DKIM record is missing, or the mail is a forgery then the test will fail and your email might be rejected, or it might be lost or it might be passed on, but perhaps with an additional header warning of the test failure.

• Warning: messages sent to a Mailman server might result in a corrupted DKIM signature, or the server might remove the signature, or it might replace it with the server's own signature. In the first of these cases, your mail might not arrive at its intended destination. If you require to use DKIM and your BCA signature is getting broken by Mailman, one recourse is to request the list manager to select a "Munge From" setting for the reflected messages.

DMARC
A DMARC record is a public record, stored with your DNS settings, which tells a mail server what to do if your email fails an SPF or DKIM test. DMARC is not yet supported by cpanel servers (Jan 2017, version 60). For now, you have to use the zone edit tools to create your DNS entry. If your DNS is external to BCA then you will need to edit your DNS entries manually. The precise details will depend on your DNS hosting company. In my case, I would create a DNS ENTRY called _dmarc.caves.org.uk and with TYPE = "TXT". Matt Wire suggests a DESTINATION/TARGET something like where postmaster@example.org is your reporting address. The above setting will send a forensic report when either SPF or DKIM checks fail. Alternatively, you may wish to implement a less stringent setting to begin with. See the advice "Deploy Slowly" at https://support.google.com/a/answer/2466563?hl=en

i) Incoming mail: Your DMARC settings explain what to do if an email fails an SPF or DKIM test. This may include notifying the sender of the test failure.

ii) Outgoing mail: Your DMARC setting is not applicable to outgoing email

iii) Remote mail: Your DMARC settings explain what to do if an email fails an SPF or DKIM test. This may include notifying the sender of the test failure.

• If you have a DMARC entry in your DNS, but you do not have both an SPF record and a DKIM record then an error will result.

Generating DKIM signatures
If your DNS is hosted at BCA you just need to enable DKIM in your cpanel. If your DNS is external to BCA then you will need to paste the public key, displayed to you in cpanel, into the DNS settings at your DNS hosting company. However, there are a number of things that can go wrong.

• If you disable DKIM and then re-enable it, a new key will be generated, but cpanel might not display it. (Is this a bug?). In this situation, you will need to ask the IT team to retrieve the new public key from /var/named/yourdomain.db
• If you have a sub-domain, cpanel will not display the (different) key required for this so, as above, you must retrieve it from /var/named/yourdomain.db. As an example, the name for the default key for a subdomain called list would be default._domainkey.list
• If you generate mails automatically, via code on your web site, then they will not have a DKIM signature unless you specifically route the mail via BCA's SMTP server. This is possible with some applications, (see Matt Wire's Blog) but if you are 'merely' using PHP's mail() function then you will need to write code to generate the DKIM mail headers. David Gibson reports using code from http://php-dkim.sourceforge.net. The salient point is that you must generate a public and private key pair, manually, for incorporation into your code. The sourceforge code suggests you run the Unix utility openssl to do this. At the time, I didnt have SSH access to my server, so I wrote a PHP routine instead, viz:

  Code: Select all

  <?
  // Create a key pair.  David Gibson, 1-Mar-2017.  Based on example at php.net
  // Im trying this because I dont have SSH access on my current server
  
  $config = array( "private_key_bits" => 4096);
  $resource = openssl_pkey_new($config);         // Create the private and public key
  openssl_pkey_export($resource, $privKey);      // Extract the private key
  $pubKey = openssl_pkey_get_details($resource); // Extract the public key 
  $pubKey = $pubKey["key"];
  echo "<pre>$privKey\n\n\n$pubKey";
  ?>

  You need to incorporate the private key into your PHP code, whilst the public key goes in your DKIM record. The private key is used to sign each email as PHP generates it.

Some more information...

• If you use server-generated emails and you have to manually incorporate your private key into some PHP code (as explained above) then you should make sure that the PHP code is stored outside your document root so that a server-config error cannot result in it being available to snoopers.
• If you use server-generated emails you may find it helpful (e.g. for debugging) to use a different signature to your default mail signature, generated by cpanel. DKIM allows you to specify a "selector" (an identifier) for the signature. So you might have a DNS entry named, for example, myphpmail._domainkey
• The default scope of the DKIM signature, used by cpanel, encompasses a wide range of headers. This gives rise to a problem because some of these headers are altered by Mailman, leading to the problems outlined above. David Gibson reports that this "fault" seems to be widespread - not only in Mailman but in other mail reflectors used as announcement lists by large companies. To circumvent this, Matt Wire has written code that runs on the BCA server to over-ride cpanel and produce a less widely-scoped signature, based on

  Code: Select all

  h=Content-Type:From:Subject:MIME-Version:Date:To:Message-ID:CC;

  For reference, David Gibson's experimental php-dkim signature is currently the minimalist h=From:To:Subject; although clearly Date could easily be added. I dont think its possible to add Message-ID because that's added later by the mail transport.