Setting up Sendmail with TLS & Auth support under FreeBSD

Posted by Benjamin Close on November 13, 2008 under FreeBSD, OpenSource | 6 Comments to Read

In order to setup a secure mail transport agent (MTA) that helps eliminate some spam and also allows roaming client support, some sort of authentication mechanisim must be added when setting up a MTA. In my case I’m using FreeBSD 4.9 with sendmail as my MTA. Setting Up Sendmail

The first step in setting up TLS/Auth support is to install the required additional libraries. For SSL support (required by TLS) the following ports must be installed:

• security/cyrus-sasl
• security/cyrus-sasl-saslauthd//

These must be installed before sendmail is recompiled.

Sendmail in FreeBSD by default is not compiled with TLS/Auth support. In order to allow it to work with these features it must be recompiled. This is actually quite simple as long as the FreeBSD machine has the FreeBSD source collection (aka /usr/src). First you need to edit ///etc/make.conf// (//cp /etc/defaults/make.conf// if it doesn’t already exist. Edit the file so the following lines exist/are uncommented.

  # with SASLv2:
  SENDMAIL_CFLAGS=-I/usr/local/include -DSASL=2
  SENDMAIL_LDFLAGS=-L/usr/local/lib
  SENDMAIL_LDADD=-lsasl2

Next you need to recompile sendmail. Due to the way sendmail exists in the ports collections you must compile some of the libraries first. The following lines show the procedure for recompiling the libraries and sendmail.

1. Compile the smutil library

  cd /usr/src/lib/libsmutil
  make clean
  make obj
  make

1. Compile the smlibrary

  cd /usr/src/lib/libsm
  make clean
  make obj
  make

1. Compile Sendmail

  cd /usr/src/usr.sbin/sendmail
  make clean
  make obj
  make
  make install

At this point sendmail with TLS/Auth support is installed. Now sendmail must be configured to work with these features. Setting Up TLS Support

TLS stands for Transport Layer Security. It’s a bit like SSL (Socket Layer Security) in that it provides encryption between two points. The difference is that TLS provides it only in the data, SSL provides encryption of the headers as well.

Using TLS is a good idea. It provides encyrption for authentication purposes and also Trusted mail headers. Ie A Mail server can create a TLS connection between itself and another TLS server and this is reported in the mail headers and the mail headers are thus deemed ‘accurate’.

To setup TLS support you need to first generate a public/private key pair for use with the mail server. This is outside the scope of this document. Then you need to setup sendmail with the following options:

  define(`CERT_DIR', `MAIL_SETTINGS_DIR`'certs')
  define(`confCACERT_PATH', `CERT_DIR')
  define(`confCACERT', `CERT_DIR/CAcert.pem')
  define(`confSERVER_CERT', `CERT_DIR/MYcert.pem')
  define(`confSERVER_KEY', `CERT_DIR/MYkey.pem')
  define(`confCLIENT_CERT', `CERT_DIR/cert.pem')
  define(`confCLIENT_KEY', `CERT_DIR/MYkey.pem')

You must set both the client and the server key so that incomming mail can be encrypted and so that outgoing mail can be encrypted.

Once you have set this up, restart sendmail and test it (See below) Setting Up Auth Support

In order to use authentication support you must first add a few options to the sendmail configuration file. Authentication is used to allow relaying from domains that are not listed as relay domains provided authentication exists. Ie: as long as the authentication is successful, then the mail server is affectivly an open relay on that connection. This is great for roaming laptop users who want to send mail but are often in a different domain or on dialup.

To set this up add the following to your //sendmail.mc// file:

  define(`confAUTH_MECHANISMS',`GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN')
  TRUST_AUTH_MECH(`GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN');

This tells sendmail to support the listed authentication methods. There is also a ‘PLAIN’ option but this should be avoided as the password is sent in plain text. Even of TLS/SSL this is not a good idea.

With that option in place you now have to tell sendmail what it is allowed to relay based on authentication. To allow relays to anywhere provided authentication works add the following to the access file.

  From: * OK
  To: * RELAY

Make sure that you rebuild the access database (run make) then restart sendmail and authentication should be enabled so test it! Extra Sendmail Configuration Options

It is possible and highly recommended that sendmail be setup to force TLS for authentication purposes. Otherwise sendmail will quite happily accept plain text passwords across an unencrypted data stream. This is just a big security problem. Hence adding the following line to sendmail forces the use to use TLS/SSL before authentication is possible:

  define(`confAUTH_OPTIONS',`p,y')

Testing it all

After everything has been setup, it is extreemly important to test that the results are as expected. There are a number of tests you will want to run. These include the following: Test Test Implementation TLS Test

The following shows a telnet to the mail server, type the text in bold and look for the result in italics/bold

  $ telnet localhost 25
  220 mail.example.net ESMTP Sendmail 8.11.1/8.11.1; Sat, 19 May 2001 08:04:04 -0400
  EHLO localhost
  250-mail.example.net Hello IDENT:jose@[127.0.0.1], pleased to meet you
  250-ENHANCEDSTATUSCODES
  250-EXPN
  250-VERB
  250-8BITMIME
  250-SIZE
  250-DSN
  250-ONEX
  250-ETRN
  250-XUSR
  250-STARTTLS
  250-DELIVERBY
  250 HELP
  Auth Test

The following shows a telnet to the mail server, type the text in bold and look for the result in italics/bold

  $ telnet localhost 25
  220 mail.example.net ESMTP Sendmail 8.11.1/8.11.1; Sat, 19 May 2001 08:04:04 -0400
  EHLO localhost
  250-mail.example.net Hello IDENT:jose@[127.0.0.1], pleased to meet you
  250-ENHANCEDSTATUSCODES
  250-EXPN
  250-VERB
  250-8BITMIME
  250-SIZE
  250-DSN
  250-ONEX
  250-ETRN
  250-AUTH DIGEST-MD5 CRAM-MD5
  250-XUSR
  250-STARTTLS
  250-DELIVERBY
  250 HELP

You might find that the authentication methods supported don’t match what you selected. This is due to various libraries not supporting those methods of authentication.

Open Relay Test

http://www.relaycheck.com telnet relay-test.mail-abuse.org Authentication Test On the system it should work Off the system it should require authentication

  telnet server 25
  helo server
  mail from: user@server
  rcpt to: someotheruser@someotherserver

This should complain about relaying being denied without authorisation if not on the local machine

Authentication Test

With SSL & Authentications Enabled You should be able to email anywhere.