====== OpenBSD SMTPd ======

>  **(!) NOTE:**
> //My words about maturity and documentation are rendered void by the development that has been happening since 4.6-release. By following -current, you will have access to better documentation, added features and a more stable smtpd.\\
Please read http://www.openbsd.org/faq/current.html for information on following -current.//


As of version 4.6 of [[http://www.openbsd.org/|OpenBSD]], a new alternative MTA is available. 
The new smtpd daemon is named according to the simple naming scheme of most
other OpenBSD packages; it's name is simply 'smtpd'.

Assuming this will be ported to other *nixes like so many other OpenBSD projects,
this will probably in time also be know under the name OpenSMTPd.

What fascinated me about this new MTA, and made me want to try it out, is its
relatively simple, and very much OpenBSD-like configuration syntax. Stealing
the first example config from the man-page of [[http://www.openbsd.org/cgi-bin/man.cgi?query=smtpd.conf&sektion=5&arch=i386&apropos=0&manpath=OpenBSD+Current|smtpd.conf]], it may look somewhat similar to:

  listen on lo0
  accept for local deliver to mbox
  accept for domain "example.org" relay via "mx1.example.org"
  accept for domain "example.net" relay

This kind of syntax should be really familiar to people having touched pf.conf, snmpd.conf,
ntpd.conf and other OpenBSD BNF-based config files.

The documentation is at the moment slightly less-than-optimal in my personal view. smtpd is currently
still getting new features added, and code is still maturing, so it is not unexpected, and documentation
will most likely improve as the project matures. I am willing to accept the lack of early documentation, 
and dive in nonetheless.

A note of caution: the version that shipped along with OpenBSD 4.6 is deemed secure, but the project is quite new,
and there is bound to be flaws exploitable for DoS on smtp. I would probably not put this MTA as my
frontline SMTP on a production environment quite yet.

This ingress was written 24. oct 2009. If you are reading this at  a later time, do assume that
claims about documentation and flaws have been rendered void.

===== My use of smtpd =====

Currently I am running smtpd on my gateway system, where I also do packet filtering, nat and rdr.

The following is my uses of smtpd:

  * In the home network, all port-25 traffic is transparently redirected to smtpd
    * No authentication internally is required, but all outgoing mail is relayed via a relay-host that requires auth.
    * This allows visitors to use any pre-existing MTA setup in their mail clients, I will simply "override" their settings, as long as they do not require TLS Auth.
  * Externally, smtpd is listening to SMTPS with authentication required.
    * I am running multiple mutt setups on multiple computers outside my home network. I wanted to have one MTA that I could configure for all those setups, without having to fiddle too much.

===== The setup =====

==== Remove the old mailer ====

First, we need to change from the default MTA, sendmail, to our new shiny smtpd. Start off by killing sendmail:

  pkill sendmail

Next, change /etc/mailer.conf to reflect smtpd in stead of sendmail:

  mv /etc/mailer.conf /etc/mailer.conf.sendmail
  cat > /etc/mailer.conf << EOF
  # Replacing sendmail with smtpd:
  sendmail        /usr/sbin/smtpctl
  send-mail       /usr/sbin/smtpctl
  mailq           /usr/sbin/smtpctl
  makemap         /usr/libexec/smtpd/makemap
  newaliases      /usr/libexec/smtpd/makemap
  EOF

Rebuild the aliases database:

  newaliases

Append the following to rc.conf.local to disable sendmail, and enable smtpd, at boot:

  echo "sendmail_flags=NO" >> /etc/rc.conf.local
  echo "smtpd_flags=" >> /etc/rc.conf.local

==== First config, trying things out: ====

The following sets smtpd up to listen on standard SMTP port 25 on the local loopback interface and the interface rl1,
which is an internal interface. All mail created lcoally, destined to "localhost", is to be delivered locally, 
in mbox format. All mail coming in from the internal IP-range is to be relayed via a relayhost.
No authentication is used; not for incoming, and not for relaying.

<code>
# /etc/mail/smtpd.conf simple example
int_if = "rl1"
int_net = "10.0.3.0/24"
ext_if = "rl0"

listen on lo0
listen on $int_if

map "aliases" { source db "/etc/mail/aliases.db" }
accept for local deliver to mbox
accept from $int_net for all relay via smtp.example.com
</code>  

Testing this out, start smtpd with the dead simple command:

  smtpd

==== Adding authentication for relayhost ====

To add support for SMTP authentication when talking with a relay host, things get a little, but not much,
more involved. 

SMTP relay username/password configuration is not stored in smtpd.conf. Instead, we need to add a secrets file.
Create the file /etc/mail/secrets:

  touch /etc/mail/secrets
  chown root:_smtpd /etc/mail/secrets
  chmod 640 /etc/mail/secrets

The format of this file allows us to store username/password pairs for multiple SMTP hosts that we
want smtpd to talk authenticated TLS/SSL/SMTPS with. The format is to have definitions line by line:

  smtp.host.name   username:password

So, in a setup where we only talk to smtp.example.com, with username "theuser" and password "53cre37":

  echo "smtp.example.com  theuser:53cre37" > /etc/mail/secrets

After creating the file, with content, we need to make a data map of it:

  makemap /etc/mail/secrets

This map needs to be added to our configuration, along with a statement that makes the relay
use authentication:

<code>
# /etc/mail/smtpd.conf simple relay authentication example
int_if = "rl1"
int_net = "10.0.3.0/24"
ext_if = "rl0"

listen on lo0
listen on $int_if

map aliases { source db "/etc/mail/aliases.db" }
map secrets { source db "/etc/mail/secrets.db" }

accept for local deliver to mbox
accept from $int_net for all relay via smtp.example.com tls enable auth
</code>  

For sake of clarity, the available options to "relay via" at the moment are:

  relay via host [port port] [ [tls | smtps | ssl] [certificate name] [enable auth] ]

==== Adding SMTPS with authentication ====

To be able to do authentication of clients connecting, at least a certificate for the interface
where the clients will be connecting is required. In my case, the interface that I want
to connect to using SMTPS and Auth is rl0.

  mkdir -p /etc/mail/certs
  openssl genrsa -out /etc/mail/certs/rl0.key 4096
  openssl req -new -x509 -key /etc/mail/certs/rl0.key -out /etc/mail/certs/rl0.crt -days 365  

Before adding configuration to use authentication, note the following detail: All authenticated users
will be treated as local! This means that all "accept" or "reject" lines in your config needs to
account for that.

<code>
# /etc/mail/smtpd.conf simple relay authentication example
int_if = "rl1"
int_net = "10.0.3.0/24"
ext_if = "rl0"

listen on lo0
listen on $int_if
listen on $ext_if smtps enable auth

map aliases { source db "/etc/mail/aliases.db" }
map secrets { source db "/etc/mail/secrets.db" }

accept for local deliver to mbox
accept from $int_net for all relay via smtp.example.com tls enable auth
accept for all relay via smtp.example.com tls enable auth
</code>  

As far as I can tell, there is one significant limitation to SMTP Auth in smtpd currently:
users are authenticated against the local user database. This means that all users that
are to be authenticating their SMTP connections, have to have some form of local authentication
on the OpenBSD box serving SMTP. Yes, these accounts may be non-login-accounts, and yes,
external authentication mechanisms may be plugged into the OpenBSD auth system. But I personally would
really like to be able to have SMTP user accounts separate from user accounts on my SMTP gateway.

> //**Update**//: Different authentication backends are in the works!
> Posted the same day I wrote the article, gilles@ comments that mapping to db's and other backends is to come: http://undeadly.org/cgi?action=article&sid=20091013122256&pid=2

==== Adding SMTPS without authentication ====

I have a single host where I do not have the option of running TLS, SSL or SMTPS authentication (Mutt setup with limited SMTP support, and no option of installing an MTA helper). So, in addition to accepting SMTPS with authentication, I need to allow
this single host to send mail without auth. But, due to firewall restrictions, I need to run on a different port because of firewalling/packetfiltering between my system, and the computer in question. I am still using SMTPS for the heck of it; my mutt does support smtps/tls/ssl, it just does not understand auth...

So, expanding on the previous example, with 172.17.18.19 being my strange client:

<code>
# /etc/mail/smtpd.conf simple relay authentication example
int_if = "rl1"
int_net = "10.0.3.0/24"
ext_if = "rl0"

listen on lo0
listen on $int_if
listen on $ext_if port 587 smtps enable auth

map aliases { source db "/etc/mail/aliases.db" }
map secrets { source db "/etc/mail/secrets.db" }

accept for local deliver to mbox
accept from $int_net for all relay via smtp.example.com tls enable auth
accept for all relay via smtp.example.com tls enable auth
accept from 172.17.18.19 for all relay via smtp.example.com tls enable auth
</code>  

So, the trick here is that authenticated users are treated as local users, while non-authenticated connections are treated as remote ones, and will have to match an "accept from" rule to pass.

To this config set, I have also thrown in a change to what port we are listening on on the external interface. SMTPS by default listens on port 465, here I have changed it to 587.

==== Different relays for different recipients ====

I want to use different relayhosts depending on where I deliver mail to, specifically, I want to deliver mails destined for GMail users directly to GMail. This additions showcases the fact that rules are selected on a first-match basis, so order does count when one or more of your rules contain "for all"...


<code>
# /etc/mail/smtpd.conf simple relay authentication example
int_if = "rl1"
int_net = "10.0.3.0/24"
ext_if = "rl0"

listen on lo0
listen on $int_if
listen on $ext_if port 587 smtps enable auth

map aliases { source db "/etc/mail/aliases.db" }
map secrets { source db "/etc/mail/secrets.db" }

accept for local deliver to mbox
accept from $int_net for all relay via smtp.example.com tls enable auth
accept for all relay via smtp.gmail.com tls enable auth
accept for all relay via smtp.example.com tls enable auth
accept from 172.17.18.19 for gmail.com relay via smtp.gmail.com tls enable auth
accept from 172.17.18.19 for all relay via smtp.example.com tls enable auth
</code>  

By adding a new relay-host that uses SMTP auth, /etc/mail/secrets needs to be updated,
and makemap needs to be run.

  echo "smtp.gmail.com  gmail.username:passw0rd" >> /etc/mail/secrets

  makemap /etc/mail/secrets

When working with different relays for different recipients, I also discovered some peculiarity.
The documentation tells that wildcards are supported. Make sure that you check the following if you
want to do wildcards:

Quotes are required on wildcards. Follow the syntax:

  accept for domain example.com     # <- This is valid, as it involves no wildcard.
  accept for domain *.example.com   # <- This is INvalid, wildcards require quotes
  accept for domain "*.example.com"  # <- This is valid syntax with wildcards.

... so it is probably best to get in the habit of adding quotes to all domains...

Check the scope of your wildcard. If you are running "mail.com", you probably do not
want your wildcard to glob "gmail.com" and "hotmail.com". So, add that dot. But when you
add that dot, your wildcard no longer matches the "stem". So, to match all "mail.com" adresses
without globbing too much, you need two "accept .. for" rules:

  accept ... for domain "*.mail.com" ...
  accept ... for domain "mail.com" ...

Also make sure you have your relaying chains set up for all __from__ definitions you have...

==== Multiple external IP addresses ====

My setup has several IP adresses on the external interface, yet I only want to listen
for connections on one of them. Selecting a single IP means we can no longer listen
on the interface, as smtpd will listen on all IP's by default. In stead we need to listen
to connections on the specific IP. For this to work with SMTPS/SSL/TLS, a certificate
valid for that IP, as well as a key file for the cert, needs to exist in
/etc/mail/certs. Assuming the global IP I want to listen for connections on is 172.20.0.12:

  openssl genrsa -out /etc/mail/certs/172.20.0.12.key 4096
  openssl req -new -x509 -key /etc/mail/certs/172.20.0.12.key -out /etc/mail/certs/172.20.0.12.crt -days 365  

Next, we change from listening on an interface, to listening on an IP. In this config, I have dropped
using the macros (you could still stuff the IP in a macro):

<code>
# /etc/mail/smtpd.conf simple relay authentication example

listen on lo0
listen on rl1
listen on 172.20.0.12 port 587 smtps enable auth

map aliases { source db "/etc/mail/aliases.db" }
map secrets { source db "/etc/mail/secrets.db" }

accept for local deliver to mbox
accept from 10.0.3.0/24 for all relay via smtp.example.com tls enable auth
accept for all relay via smtp.gmail.com tls enable auth
accept for all relay via smtp.example.com tls enable auth
accept from 172.17.18.19 for gmail.com relay via smtp.gmail.com tls enable auth
accept from 172.17.18.19 for all relay via smtp.example.com tls enable auth
</code>  

The trick to this, is that smtpd will not select a single IP from an interface, so you have
to configure the listening for the IP directly. By doing so, you have to have both .crt and .key
certificate data reflecting the IP in your /etc/mail/certs. There is an alternative to this autoselection,
but I think the documentation is really flaky on this.

Create your key, with a name you select, and the suffix .key, Then generate the certificate, with the
same name and suffix .crt. Now add the name of the certificate, with no path and no suffix, to
the "listen on ..." line. I created the files:

  * /etc/mail/certs/smtp.example.com.key
  * /etc/mail/certs/smtp.example.com.crt

Then I modified the listen-statement to:

  listen on 172.20.0.12 port 587 smtps certificate smtp.example.com enable auth

If Gilles@ or any of the guys working on the code reads this: please update the documentation, 
neither starttls(8), smtpd(8) or smtpd.conf(8) informs the user that smtpd does not handle full patchs
to the "certificate" option, and that certs need to live in /etc/mail/certs, regardless of wether the
cert is auto-located or manually defined.

==== Transparent redirection of SMTP ====

As mentioned early, I intended to do transparent redirection of internally generated mail.
This is naturally handled by pf. The configurations I am splaying now are closer to reality
than the previous ones, so it is probably also appropriate to add some info about my home setup:

=== The setup ===

  * My connection to the world is handled by an OpenBSD installation with pf and NAT.
  * My external connection has several IP-adresses on the interface rl0. For sake of example, let us call these: 
    * 172.16.81.241
    * 172.16.81.242
    * 172.16.81.243
    * 172.16.81.244
    * 172.16.81.245
  * The first address is used for NAT of my internal "home" network
  * The following addresses are binat'ed to my "DMZ" network
  * I am running VLAN tagging to separate my networks:
    * VLAN id 1 is used for netboxes, no user equipment. IP-range 10.0.1.0/24
    * VLAN id 2 is used for DMZ, IP-range 10.0.2.0/24
    * VLAN id 3 is used for my internal net, IP-range 10.0.3.0/24
    * All VLAN's are mapped directly numeric to interface names.
  * The internal network is fully nat'ed.
  * The OpenBSD box holds the .1 IP-adress inn all VLANs.
  * My troublesome external client has the IP 172.16.17.18

=== The smtpd.conf ===

<code>
listen on lo0
listen on vlan1
listen on vlan1 smtps certificate home.defcon.no
listen on 172.16.81.241 port 465 smtps certificate home.defcon.no enable auth

map "aliases" { source db "/etc/mail/aliases.db" }
map secrets { source db "/etc/mail/secrets.db" }

accept for local deliver to mbox

# I really would have liked the "from local" keyword.
# That one is available in CVS, but I am not running CVS -current :(
accept for domain "rt.defcon.no" relay via rt.home.defcon.no
accept for domain "*.defcon.no" relay via mail.defcon.no
accept for domain "defcon.no" relay via mail.defcon.no

# For the internal networks
accept from 10.0.0.0/22 for domain "rt.defcon.no" relay via rt.home.defcon.no
accept from 10.0.0.0/22 for domain "*.defcon.no" relay via mail.defcon.no
accept from 10.0.0.0/22 for domain "defcon.no" relay via mail.defcon.no
accept from 10.0.0.0/22 for domain gmail.com relay via smtp.gmail.com ssl enable auth
accept from 10.0.0.0/22 for all relay via smtp.hig.no tls enable auth

# For troublehost
accept from 172.16.17.18 for domain "rt.defcon.no" relay via rt.home.defcon.no
accept from 172.16.17.18 for domain "*.defcon.no" relay via mail.defcon.no
accept from 172.16.17.18 for domain "defcon.no" relay via mail.defcon.no
accept from 172.16.17.18 for domain gmail.com relay via smtp.gmail.com ssl enable auth
accept from 172.16.17.18 for all relay via smtp.hig.no tls enable auth

# All other (local) mail, including authenticated.
accept for all relay via smtp.hig.no tls enable auth
</code>

=== The pf.conf ===

<code>
# The internal network is NAT'ed
nat on rl0 from 10.0.3.0/24 -> 172.16.81.241
# ...while the 'DMZ' is binat'ed
binat on rl0 from 10.0.2.2 to any -> 172.16.81.242
binat on rl0 from 10.0.2.3 to any -> 172.16.81.243
binat on rl0 from 10.0.2.4 to any -> 172.16.81.244
binat on rl0 from 10.0.2.5 to any -> 172.16.81.245

# This is the transparent redirect! I am running smtpd on my
# VLAN1 IP so that the box it self is not trapped by rdr ;)
# I am also transparently redirecting SSL/SMTPS on both "standard" ports..
rdr pass on vlan2 proto tcp to any port 25 -> 10.0.1.1 port 25
rdr pass on vlan2 proto tcp to any port 465 -> 10.0.1.1 port 465
rdr pass on vlan2 proto tcp to any port 587 -> 10.0.1.1 port 465
rdr pass on vlan3 proto tcp to any port 25 -> 10.0.1.1 port 25
rdr pass on vlan3 proto tcp to any port 465 -> 10.0.1.1 port 465
rdr pass on vlan3 proto tcp to any port 587 -> 10.0.1.1 port 465

# Some filter rules for good measure... Nothe, this is absolutely not what
# is running in prod, but it is a minimum that works ;)

# Block all unknown traffic
block in log
# Permit all "known" traffic, unless we block it again later. And keep state!
pass in log on vlan2 keep state
pass in log on vlan3 keep state

# No, DMZ is not allowed to communicate with internal computers,
# unless asked to do so...
block in log on vlan2 inet to 10.0.3.0/24

# Allow rdr'ed and binat'ed traffic in to the DMZ
pass in log on rl0 inet from any to 10.0.2.2 keep state
pass in log on rl0 inet from any to 10.0.2.3 keep state
pass in log on rl0 inet from any to 10.0.2.4 keep state
pass in log on rl0 inet from any to 10.0.2.5 keep state

# Permit incoming SMTPS to gw.home.defcon.no from the world.
pass  in log on rl0 inet proto tcp from any to 172.16.81.241 port 465
pass  in log on rl0 inet proto tcp from any to 172.16.81.241 port 587

# But block any unsolicited SMTP
block in log on rl0 inet proto tcp from any to 10.0.2.0/24 port 25
</code>

So, that was a bit more pf.conf listing that strictly needed for the smtpd
example, so here are the relevant parts separated:

<code>
rdr pass on vlan2 proto tcp to any port 25 -> 10.0.1.1 port 25
rdr pass on vlan2 proto tcp to any port 465 -> 10.0.1.1 port 465
rdr pass on vlan2 proto tcp to any port 587 -> 10.0.1.1 port 465
rdr pass on vlan3 proto tcp to any port 25 -> 10.0.1.1 port 25
rdr pass on vlan3 proto tcp to any port 465 -> 10.0.1.1 port 465
rdr pass on vlan3 proto tcp to any port 587 -> 10.0.1.1 port 465
pass  in log on rl0 inet proto tcp from any to 172.16.81.241 port 465
pass  in log on rl0 inet proto tcp from any to 172.16.81.241 port 587
</code>

===== Final words =====

The new smtpd in OpenBSD is, as you hopefull have seen through my examples, really very
simple to set up and configure. It has tolerated my stress-testing really well, and shows
great promise.

The version that ships with OpenBSD 4.6 is probably not something I personally would put
into production for a large-scale system quite yet, but work is progressing with good
updates in CVS, and I am expecting this to be "large-scale capable" by the time 4.7 is
out. But to achieve main-stream adoption, the documentation really needs more content
and more examples.

For a smaller setup, like a small to medium company, or home use, smtpd is really quite
complete, and a lot less pain to get running than most other MTA's.

In this walk-through, I have not touched on virtual domains at all, and that is
probably for the better, as the implementation if virtual domains in 4.6 (release) and
CVS (current) is quite different.

My next step is to re-introduce my old spamd setup, and get spamd and smtpd to "talk nice"
with each other, while still maintaining separate logic for internal and external use.

I would really love to see a sensible way to handle authentication for non-local user accounts.
This may really already be supported somehow, and if that is the case, my knowledge of
BSD Auth is apparently not aboce "newb" level...
> //**Update**//: Different authentication backends are in the works!
> Check the comment under the heading [[#Adding SMTPS with authentication]].

> **(!) NOTE:**\\
> //My words about maturity and documentation are rendered void by the
> development that has been happening since 4.6-release. By following -current,
> you will have access to better documentation, added features and a more stable smtpd.
> Please read http://www.openbsd.org/faq/current.html for information on following -current.//