Chapter 11. Network Services: Postfix, Apache, NFS, Samba, Squid, LDAP, SIP, XMPP, TURN

Network services are the programs that users interact with directly in their daily work. They are the tip of the information system iceberg, and this chapter focuses on them; the hidden parts they rely on are the infrastructure we already described. They usually require the encryption technology described in Section 10.2, “X.509 certificates”.

11.1. Mail Server

The Falcot Corp administrators selected Postfix for the electronic mail server, due to its reliability and its ease of configuration. Indeed, its design enforces that each task is implemented in a process with the minimum set of required permissions, which is a great mitigation measure against security problems.

ALTERNATIVE The Exim4 server

Debian uses Exim4 as the default email server (which is why the initial installation includes Exim4). The configuration is provided by a separate package, exim4-config, and automatically customized based on the answers to a set of Debconf questions very similar to the questions asked by the postfix package.

The configuration can be either in one single file (/etc/exim4/exim4.conf.template) or split across a number of configuration snippets stored under /etc/exim4/conf.d/. In both cases, the files are used by update-exim4.conf as templates to generate /var/lib/exim4/config.autogenerated. The latter is the file used by Exim4. Thanks to this mechanism, values obtained through Exim's debconf configuration — which are stored in /etc/exim4/update-exim4.conf.conf — can be injected in Exim's configuration file, even when the administrator or another package has altered the default Exim configuration.

The Exim4 configuration file syntax has its peculiarities and its learning curve; however, once these peculiarities are understood, Exim4 is a very complete and powerful email server, as evidenced by the tens of pages of documentation.

→ https://www.exim.org/docs.html

11.1.1. Installing Postfix

The postfix package includes the main SMTP daemon. Other packages (such as postfix-ldap and postfix-pgsql) add extra functionality to Postfix, including access to mapping databases. You should only install them if you know that you need them.

Several Debconf questions are asked during the installation of the package. The answers allow generating a first version of the /etc/postfix/main.cf configuration file.

The first question deals with the type of setup. Only two of the proposed answers are relevant in case of an Internet-connected server, “Internet site” and “Internet with smarthost”. The former is appropriate for a server that receives incoming email and sends outgoing email directly to its recipients, and is therefore well-adapted to the Falcot Corp case. The latter is appropriate for a server receiving incoming email normally, but that sends outgoing email through an intermediate SMTP server — the “smarthost” — rather than directly to the recipient's server. This is mostly useful for individuals with a dynamic IP address, since many email servers reject messages coming straight from such an IP address. In this case, the smarthost will usually be the ISP's SMTP server, which is always configured to accept email coming from the ISP's customers and forward it appropriately. This setup (with a smarthost) is also relevant for servers that are not permanently connected to the internet, since it avoids having to manage a queue of undeliverable messages that need to be retried later.

The second question deals with the full name of the machine, used to generate email addresses from a local user name; the full name of the machine ends up as the part after the at-sign (“@”). In the case of Falcot, the answer should be mail.falcot.com. This is the only question asked by default, but the configuration it leads to is not complete enough for the needs of Falcot, which is why the administrators run dpkg-reconfigure postfix so as to be able to customize more parameters.

One of the extra questions asks for all the domain names related to this machine. The default list includes its full name as well as a few synonyms for localhost, but the main falcot.com domain needs to be added by hand. More generally, this question should usually be answered with all the domain names for which this machine should serve as an MX server; in other words, all the domain names for which the DNS says that this machine will accept email. This information ends up in the mydestination variable of the main Postfix configuration file — /etc/postfix/main.cf.

Figure 11.1. Role of the DNS MX record while sending a mail

In some cases, the installation can also ask what networks should be allowed to send email via the machine. In its default configuration, Postfix only accepts emails coming from the machine itself; the local network will usually be added. The Falcot Corp administrators added 192.168.0.0/16 to the default answer. If the question is not asked, the relevant variable in the configuration file is mynetworks, as seen in the example below.

Local email can also be delivered through procmail. This tool allows users to sort their incoming email according to rules stored in their ~/.procmailrc file. Both Postfix and Exim4 suggest procmail by default, but there are alternatives like maildrop or Sieve filters.

After this first step, the administrators got the following configuration file; it will be used as a starting point for adding some extra functionality in the next sections.

Example 11.1. Initial /etc/postfix/main.cf file

# See /usr/share/postfix/main.cf.dist for a commented, more complete version


# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname

smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# See http://www.postfix.org/COMPATIBILITY_README.html -- default to 2 on
# fresh installs.
compatibility_level = 2



# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_security_level=may

smtp_tls_CApath=/etc/ssl/certs
smtp_tls_security_level=may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache


smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = mail.falcot.com
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
mydestination = mail.falcot.com, falcot.com, localhost.localdomain, localhost
relayhost = 
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 192.168.0.0/16
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
default_transport = smtp
relay_transport = smtp
inet_protocols = all
myorigin = /etc/mailname

SECURITY Snake oil SSL certificates

The snake oil certificates, like the snake oil “medicine” sold by unscrupulous quacks in old times, have absolutely no value: you cannot rely on them to authenticate the server since they are automatically generated self-signed certificates. However, they are useful to improve the privacy of the exchanges.

In general they should only be used for testing purposes, and normal service must use real certificates. The Let's encrypt initiative offers free and trusted SSL/TLS certificates, which can be generated using the certbot package as described in Section 11.2.2, “Adding support for SSL” and then used in postfix like this:

smtpd_tls_cert_file = /etc/letsencrypt/live/DOMAIN/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/DOMAIN/privkey.pem
smtpd_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtpd_tls_CApath = /etc/ssl/certs
smtp_tls_CApath = /etc/ssl/certs

A different way to generate your own certificates using your own certificate authority is described in Section 10.2.2, “Public Key Infrastructure: easy-rsa”.

11.1.2. Configuring Virtual Domains

The mail server can receive mails addressed to other domains besides the main domain; these are then known as “virtual“ domains. In most cases where this happens, the emails are not ultimately destined to local users. Postfix provides two interesting features for handling virtual domains.

11.1.2.1. Virtual Alias Domains

A virtual alias domain only contains aliases, i.e. addresses that only forward emails to other addresses.

Such a domain is enabled by adding its name to the virtual_alias_domains variable, and referencing an address mapping file in the virtual_alias_maps variable.

virtual_alias_domains = falcotsbrand.com
virtual_alias_maps = hash:/etc/postfix/virtual

The /etc/postfix/virtual file describes a mapping with a rather straightforward syntax: each line contains two fields separated by whitespace; the first field is the alias name, the second field is a list of email addresses where it redirects. The special @domain.com syntax covers all remaining aliases in a domain.

[email protected]  [email protected]
[email protected]    [email protected], [email protected]
# The alias below is generic and covers all addresses within
# the falcotsbrand.com domain not otherwise covered by this file.
# These addresses forward email to the same user name in the
# falcot.com domain.
@falcotsbrand.com           @falcot.com

After changing /etc/postfix/virtual the postfix table /etc/postfix/virtual.db needs to be updated using sudo postmap /etc/postfix/virtual.

11.1.2.2. Virtual Mailbox Domains

Messages addressed to a virtual mailbox domain are stored in mailboxes not assigned to a local system user.

Enabling a virtual mailbox domain requires naming this domain in the virtual_mailbox_domains variable, and referencing a mailbox mapping file in virtual_mailbox_maps. The virtual_mailbox_base parameter contains the directory under which the mailboxes will be stored.

virtual_mailbox_domains = falcot.org
virtual_mailbox_maps = hash:/etc/postfix/vmailbox
virtual_mailbox_base = /var/mail/vhosts

The virtual_uid_maps parameter (respectively virtual_gid_maps) references the file containing the mapping between the email address and the system user (respectively group) that “owns” the corresponding mailbox. To get all mailboxes owned by the same owner/group, the static:5000 syntax assigns a fixed UID/GID (of value 5000 here).

Again, the syntax of the /etc/postfix/vmailbox file is quite straightforward: two fields separated with whitespace. The first field is an email address within one of the virtual domains, and the second field is the location of the associated mailbox (relative to the directory specified in virtual_mailbox_base). If the mailbox name ends with a slash (/), the emails will be stored in the maildir format; otherwise, the traditional mbox format will be used. The maildir format uses a whole directory to store a mailbox, each individual message being stored in a separate file. In the mbox format, on the other hand, the whole mailbox is stored in one file, and each line starting with “From ” (From followed by a space) signals the start of a new message.

# Jean's email is stored as maildir, with
# one file per email in a dedicated directory
[email protected] falcot.org/jean/
# Sophie's email is stored in a traditional "mbox" file,
# with all mails concatenated into one single file
[email protected] falcot.org/sophie

11.1.3. Restrictions for Receiving and Sending

The growing number of unsolicited bulk emails (spam) requires being increasingly strict when deciding which emails a server should accept. This section presents some of the strategies included in Postfix.

If the reject-rules are too strict, it may happen that even legitimate email traffic gets locked out. It is therefor a good habit to test restrictions and prevent the permanent rejection of requests during this time using the soft_bounce = yes directive. By prepending a reject-type directive with warn_if_reject only a log message will be recorded instead of rejecting the request.

CULTURE The spam problem

“Spam” is a generic term used to designate all the unsolicited commercial emails (also known as UCEs) that flood our electronic mailboxes; the unscrupulous individuals sending them are known as spammers. They care little about the nuisance they cause, since sending an email costs very little, and only a very small percentage of recipients need to be attracted by the offers for the spamming operation to make more money than it costs. The process is mostly automated, and any email address made public (for instance, on a web forum, or on the archives of a mailing list, or on a blog, and so on) will be likely discovered by the spammers' robots, and subjected to a never-ending stream of unsolicited messages. Also every contact found at a compromised system is targeted.

All system administrators try to face this nuisance with spam filters, but of course spammers keep adjusting to try to work around these filters. Some even rent networks of machines compromised by a worm from various crime syndicates. Recent statistics estimate that up to 95% of all emails circulating on the Internet are spam!

11.1.3.1. IP-Based Access Restrictions

The smtpd_client_restrictions directive controls which machines are allowed to communicate with the email server.

When a variable contains a list of rules, as in the example below, these rules are evaluated in order, from the first to the last. Each rule can accept the message, reject it, or leave the decision to a following rule. As a consequence, order matters, and simply switching two rules can lead to a widely different behavior.

Example 11.2. Restrictions Based on Client Address

smtpd_client_restrictions =
    permit_mynetworks,
    warn_if_reject reject_unknown_client_hostname,
    check_client_access hash:/etc/postfix/access_clientip,
    reject_rhsbl_reverse_client dbl.spamhaus.org,
    reject_rhsbl_reverse_client rhsbl.sorbs.net,
    reject_rbl_client zen.spamhaus.org,
    reject_rbl_client dnsbl.sorbs.net

The permit_mynetworks directive, used as the first rule, accepts all emails coming from a machine in the local network (as defined by the mynetworks configuration variable).

The second directive would normally reject emails coming from machines without a completely valid DNS configuration. Such a valid configuration means that the IP address can be resolved to a name, and that this name, in turn, resolves to the IP address. This restriction is often too strict, since many email servers do not have a reverse DNS for their IP address. This explains why the Falcot administrators prepended the warn_if_reject modifier to the reject_unknown_client directive: this modifier turns the rejection into a simple warning recorded in the logs. The administrators can then keep an eye on the number of messages that would be rejected if the rule were actually enforced, and make an informed decision later if they wish to enable such enforcement.

TIP access tables

The restriction criteria include administrator-modifiable tables listing combinations of senders, IP addresses, and allowed or forbidden hostnames. These tables can be created using a copy of the /usr/share/doc/postfix/examples/access file shipped with the postfix-doc package and explained in access(5). This model is self-documented in its comments, which means each table describes its own syntax.

The /etc/postfix/access_clientip table lists IP addresses and networks; /etc/postfix/access_helo lists domain names; /etc/postfix/access_sender contains sender email addresses. All these files need to be turned into hash-tables (a format optimized for fast access) after each change, with the sudo postmap /etc/postfix/file command.

The check_client_access directive allows the administrator to set up a blacklist and a whitelist of email servers, stored in the /etc/postfix/access_clientip file. Servers in the whitelist are considered as trusted, and the emails coming from there therefore do not go through the following filtering rules.

The last four rules reject any message coming from a server listed in one of the indicated blacklists. RBL is an acronym for Remote Black List, and RHSBL stands for Right-Hand Side Black List. The difference is that the former lists IP addresses, whereas the latter lists domain names. There are several such services. They list domains and IP addresses with poor reputation, badly configured servers that spammers use to relay their emails, as well as unexpected mail relays such as machines infected with worms or viruses.

11.1.3.2. Checking the Validity of the `EHLO` or `HELO` Commands

Each SMTP exchange starts with a HELO (or EHLO) command, followed by the name of the sending email server. Checking the validity of this name can be interesting. To fully enforce the restrictions listed in smtpd_helo_restrictions the smtpd_helo_required option needs to be enabled. Otherwise clients could skip the restrictions by not sending any HELO/EHLO command.

Example 11.3. Restrictions on the name announced in EHLO

smtpd_helo_required = yes
smtpd_helo_restrictions =
    permit_mynetworks,
    reject_invalid_helo_hostname,
    reject_non_fqdn_helo_hostname,
    warn_if_reject reject_unknown_helo_hostname,
    check_helo_access hash:/etc/postfix/access_helo,
    reject_rhsbl_helo multi.surbl.org

The first permit_mynetworks directive allows all machines on the local network to introduce themselves freely. This is important, because some email programs do not respect this part of the SMTP protocol adequately enough, and they can introduce themselves with nonsensical names.

The reject_invalid_helo_hostname rule rejects emails when the EHLO announce lists a syntactically incorrect hostname. The reject_non_fqdn_helo_hostname rule rejects messages when the announced hostname is not a fully-qualified domain name (including a domain name as well as a host name). The reject_unknown_helo_hostname rule rejects messages if the announced name does not exist in the DNS. Since this last rule unfortunately leads to a lot of rejections, the administrators turned its effect to a simple warning with the warn_if_reject modifier as a first step; they may decide to remove this modifier at a later stage, after auditing the results of this rule.

The reject_rhsbl_helo allows to specify a black list to check the hostname against an RHSBL.

Using permit_mynetworks as the first rule has an interesting side effect: the following rules only apply to hosts outside the local network. This allows blacklisting all hosts that announce themselves as part of the falcot.com network, for instance by adding a falcot.com REJECT You are not in our network! line to the /etc/postfix/access_helo file.

11.1.3.3. Accepting or Refusing Mails Based on the Announced Sender

Every message has a sender, announced by the MAIL FROM command of the SMTP protocol; again, this information can be validated in several different ways.

Example 11.4. Sender checks

smtpd_sender_restrictions =
    check_sender_access hash:/etc/postfix/access_sender,
    reject_unknown_sender_domain,
    reject_unlisted_sender,
    reject_non_fqdn_sender,
    reject_rhsbl_sender rhsbl.sorbs.net

The /etc/postfix/access_sender table maps some special treatment to some senders. This usually means listing some senders into a white list or a black list.

The reject_unknown_sender_domain rule requires a valid sender domain, since it is needed for a valid address. The reject_unlisted_sender rule rejects local senders if the address does not exist; this prevents emails being sent from an invalid address in the falcot.com domain, and messages emanating from [email protected] are only accepted if such an address really exists.

Finally, the reject_non_fqdn_sender rule rejects emails purporting to come from addresses without a fully-qualified domain name. In practice, this means rejecting emails coming from user@machine: the address must be announced as either [email protected] or [email protected].

The reject_rhsbl_sender rule reject senders based on a (domain-based) RHSBL service.

11.1.3.4. Accepting or Refusing Mails Based on the Recipient

Each email has at least one recipient, announced with the RCPT TO command in the SMTP protocol. These addresses also warrant validation, even if that may be less relevant than the checks made on the sender address.

Example 11.5. Recipient checks

smtpd_recipient_restrictions =
    permit_mynetworks,
    reject_unauth_destination,
    reject_unlisted_recipient,
    reject_non_fqdn_recipient,
    permit

reject_unauth_destination is the basic rule that requires outside messages to be addressed to us; messages sent to an address not served by this server are rejected. Without this rule, a server becomes an open relay that allows spammers to send unsolicited emails; this rule is therefore mandatory, and it will be best included near the beginning of the list, so that no other rules may authorize the message before its destination has been checked.

The reject_unlisted_recipient rule rejects messages sent to non-existing local users, which makes sense. Finally, the reject_non_fqdn_recipient rule rejects non-fully-qualified addresses; this makes it impossible to send an email to jean or jean@machine, and requires using the full address instead, such as [email protected] or [email protected].

The permit directive at the end is not necessary. But it can be useful at the end of a restriction list to make the default policy explicit.

11.1.3.5. Restrictions Associated with the `DATA` Command

The DATA command of SMTP is emitted before the contents of the message. It doesn't provide any information per se, apart from announcing what comes next. It can still be subjected to checks.

Example 11.6. DATA checks

smtpd_data_restrictions = reject_unauth_pipelining

The reject_unauth_pipelining directives causes the message to be rejected if the sending party sends a command before the reply to the previous command has been sent. This guards against a common optimization used by spammer robots, since they usually don't care a fig about replies and only focus on sending as many emails as possible in as short a time as possible.

11.1.3.6. Applying Restrictions

Although the above commands validate information at various stages of the SMTP exchange, Postfix sends the actual rejection as a reply to the RCPT TO command by default.

This means that even if the message is rejected due to an invalid EHLO command, Postfix knows the sender and the recipient when announcing the rejection. It can then log a more explicit message than it could if the transaction had been interrupted from the start. In addition, a number of SMTP clients do not expect failures on the early SMTP commands, and these clients will be less disturbed by this late rejection.

A final advantage to this choice is that the rules can accumulate information during the various stages of the SMTP exchange; this allows defining more fine-grained permissions, such as rejecting a non-local connection if it announces itself with a local sender.

The default behavior is controlled by the smtpd_delay_reject rule.

11.1.3.7. Filtering Based on the Message Contents

The validation and restriction system would not be complete without a way to apply checks to the message contents. Postfix differentiates the checks applying to the email headers from those applying to the email body.

Example 11.7. Enabling content-based filters

header_checks = regexp:/etc/postfix/header_checks
body_checks = regexp:/etc/postfix/body_checks

Both files contain a list of regular expressions (commonly known as regexps or regexes) and associated actions to be triggered when the email headers (or body) match the expression.

Example 11.8. Example /etc/postfix/header_checks file

/^X-Mailer: GOTO Sarbacane/ REJECT I fight spam (GOTO Sarbacane)
/^Subject: *Your email contains VIRUSES/ DISCARD virus notification

BACK TO BASICS Regular expression

The regular expression term (shortened to regexp or regex) references a generic notation for expressing a description of the contents and/or structure of a string of characters. Certain special characters allow defining alternatives (for instance, foo|bar matches either “foo” or “bar”), sets of allowed characters (for instance, [0-9] means "any digit", and . — a dot — means "any character"), quantification (s? matches either s or the empty string, in other words 0 or 1 occurrence of s; s+ matches one or more consecutive s characters; and so on). Parentheses allow grouping search results.

The precise syntax of these expressions varies across the tools using them, but the basic features are similar.

→ https://en.wikipedia.org/wiki/Regular_expression

The first one checks the header mentioning the email software; if GOTO Sarbacane (a bulk email software) is found, the message is rejected. The second expression controls the message subject; if it mentions a virus notification, we can decide not to reject the message but to discard it immediately instead.

Using these filters is a double-edged sword, because it is easy to make the rules too generic and to lose legitimate emails as a consequence. In these cases, not only the messages will be lost, but their senders will get unwanted (and annoying) error messages.

11.1.4. Setting Up greylisting

“Greylisting” is a filtering technique according to which a message is initially rejected with a temporary error code, and only accepted after a further delivery attempt with some delay. This filtering is particularly efficient against spam sent by the many machines infected by worms and viruses, since this software rarely acts as a full SMTP agent (by checking the error code and retrying failed messages later), especially since many of the harvested addresses are really invalid and retrying would only mean losing time.

Postfix doesn't provide greylisting natively, but there is a feature by which the decision to accept or reject a given message can be delegated to an external program. The postgrey package contains just such a program, designed to interface with this access policy delegation service.

Once postgrey is installed, it runs as a daemon and listens on port 10023. Postfix can then be configured to use it, by adding the check_policy_service parameter as an extra restriction:

smtpd_recipient_restrictions =
    permit_mynetworks,
    [...]
    check_policy_service inet:127.0.0.1:10023

Each time Postfix reaches this rule in the rule set, it will connect to the postgrey daemon and send it information concerning the relevant message. On its side, Postgrey considers the IP address/sender/recipient triplet and checks in its database whether that same triplet has been seen recently. If so, Postgrey replies that the message should be accepted; if not, the reply indicates that the message should be temporarily rejected, and the triplet gets recorded in the database.

The main disadvantage of greylisting is that legitimate messages get delayed, which is not always acceptable. It also increases the burden on servers that send many legitimate emails.

IN PRACTICE Shortcomings of greylisting

Theoretically, greylisting should only delay the first mail from a given sender to a given recipient, and the typical delay is in the order of minutes. Reality, however, can differ slightly. Some large ISPs use clusters of SMTP servers, and when a message is initially rejected, the server that retries the transmission may not be the same as the initial one. When that happens, the second server gets a temporary error message due to greylisting too, and so on; it may take several hours until transmission is attempted by a server that has already been involved, since SMTP servers usually increase the delay between retries at each failure.

As a consequence, the incoming IP address may vary in time even for a single sender. But it goes further: even the sender address can change. For instance, many mailing-list servers encode extra information in the sender address so as to be able to handle error messages (known as bounces). Each new message sent to a mailing-list may then need to go through greylisting, which means it has to be stored (temporarily) on the sender's server. For very large mailing-lists (with tens of thousands of subscribers), this can soon become a problem.

To mitigate these drawbacks, Postgrey manages a whitelist of such sites, and messages emanating from them are immediately accepted without going through greylisting. This list can easily be adapted to local needs, since it is stored in the /etc/postgrey/whitelist_clients file.

GOING FURTHER Selective greylisting with milter-greylist

The drawbacks of greylisting can be mitigated by only using greylisting on the subset of clients that are already considered as probable sources of spam (because they are listed in a DNS blacklist). This is not possible with postgrey but milter-greylist can be used in such a way.

In that scenario, since DNS blacklists never triggers a definitive rejection, it becomes reasonable to use aggressive blacklists, including those listing all dynamic IP addresses from ISP clients (such as pbl.spamhaus.org or dul.dnsbl.sorbs.net).

Since milter-greylist uses Sendmail's milter interface, the postfix side of its configuration is limited to “smtpd_milters = unix:/var/run/milter-greylist/milter-greylist.sock”. The greylist.conf(5) manual page documents /etc/milter-greylist/greylist.conf and the numerous ways to configure milter-greylist. You will also have to edit /etc/default/milter-greylist to actually enable the service.

11.1.5. Customizing Filters Based On the Recipient

Section 11.1.3, “Restrictions for Receiving and Sending” and Section 11.1.4, “Setting Up greylisting” reviewed many of the possible restrictions. They all have their use in limiting the amount of received spam, but they also all have their drawbacks. It is therefore more and more common to customize the set of filters depending on the recipient. At Falcot Corp, greylisting is interesting for most users, but it hinders the work of some users who need low latency in their emails (such as the technical support service). Similarly, the commercial service sometimes has problems receiving emails from some Asian providers who may be listed in blacklists; this service asked for a non-filtered address so as to be able to correspond.

Postfix provides such a customization of filters with a “restriction class” concept. The classes are declared in the smtpd_restriction_classes parameter, and defined the same way as smtpd_recipient_restrictions. The check_recipient_access directive then defines a table mapping a given recipient to the appropriate set of restrictions.

Example 11.9. Defining restriction classes in main.cf

smtpd_restriction_classes = greylisting, aggressive, permissive

greylisting = check_policy_service inet:127.0.0.1:10023
aggressive =
        reject_rbl_client sbl-xbl.spamhaus.org,
        check_policy_service inet:127.0.0.1:10023
permissive = permit

smtpd_recipient_restrictions =
        permit_mynetworks,
        reject_unauth_destination,
        check_recipient_access hash:/etc/postfix/recipient_access

Example 11.10. The /etc/postfix/recipient_access file

# Unfiltered addresses
[email protected]  permissive
[email protected]     permissive
[email protected]  permissive

# Aggressive filtering for some privileged users
[email protected]         aggressive

# Special rule for the mailing-list manager
[email protected]       reject_unverified_sender

# Greylisting by default
falcot.com             greylisting

11.1.6. Integrating an Antivirus Filter

The many viruses circulating as attachments to emails make it important to set up an antivirus solution at the entry point of the company network, since despite an awareness campaign, some users will still open attachments from obviously shady messages.

SECURITY Controversial Discussion of Anti-Virus Software

The usage of virus scanners, or so called antivirus software, is controversial. There is usually a gap between the release of some piece of malware and the addition of detection rules to the antivirus database. During this gap, there is no software-based protection. Further, the usage often requires to run additional software, for example, to uncompress archives and scan all kinds of executables, which drastically increases the exploit potential of the antivirus software itself. Usage of such software solutions can therefor never replace awareness campaigns and simple behavioral rules (never open unsolicited sent attachments, etc.).

The Falcot administrators selected clamav from the homonymous package.

The task of interfacing between antivirus and the email server goes to clamav-milter. A milter (short for mail filter) is a filtering program specially designed to interface with email servers. A milter uses a standard application programming interface (API) that provides much better performance than filters external to the email servers. Milters were initially introduced by Sendmail, but Postfix soon followed suit.

Once the clamav-milter package is installed, the milter should be reconfigured to run on a TCP port rather than on the default named socket. This can be achieved with dpkg-reconfigure clamav-milter. When prompted for the “Communication interface with Sendmail”, answer “inet:[email protected]”.

The standard ClamAV configuration fits most situations, but some important parameters can still be customized with dpkg-reconfigure clamav-base.

The last step involves telling Postfix to use the recently-configured filter. This is a simple matter of adding the following directive to /etc/postfix/main.cf:

# Virus check with clamav-milter
smtpd_milters = inet:[127.0.0.1]:10002

If the antivirus causes problems, this line can be commented out, and systemctl reload postfix should be run so that this change is taken into account.

All messages handled by Postfix now go through the antivirus filter.

11.1.7. Fighting Spam with SPF, DKIM and DMARC

The high number of unsolicited email sent every day led to the creation of several standards, which aim at validating that the sending host of an email is authorized and that the email has not been tampered with. The following systems are all DNS-based and require the administrators to not only have control over the mail server, but over the DNS for the domain in question too.

CAUTION Controversial Discussion

Like any other tool, the following standards have limits and real effects if put to use. They can (and should) lead to emails being rejected or even just discarded. If that happens to some legitimate emails (sometimes sent from a misconfigured SMTP server), it usually causes anger and a lack of understanding by the user. Therefore these rules are often applied as a "soft fail" or a "soft reject", which usually means that failing the checks only leads to adding a (header) mark to the affected email. There are people who think that this makes these standards "broken by design". Decide for yourself and be careful about how strict you choose to apply these standards.

11.1.7.1. Integrating the Sender Policy Framework (SPF)

The Sender Policy Framework (SPF) is used to validate if a certain mail server is allowed to send emails for a given domain. It is mostly configured through DNS. The syntax for the entry to make is explained in detail at:

→ http://www.open-spf.org/SPF_Record_Syntax

→ https://tools.ietf.org/html/rfc7208

→ https://en.wikipedia.org/wiki/Sender_Policy_Framework

The following is a sample DNS entry which states that all the domain's Mail Exchange Resource Records (MX-RRs) are allowed to send email for the current domain, and all others are prohibited. The DNS entry does not need to be given a name. But to use the include directive it must have one.

Name: example.org
Type: TXT
TTL:  3600
Data: v=spf1 a mx -all

Let's take a quick look at the falcot.org entry.

# host -t TXT falcot.org
falcot.org descriptive text "v=spf1 ip4:199.127.61.96 +a +mx +ip4:206.221.184.234 +ip4:209.222.96.251 ~all"

It states that the IP of the sender must match the A record for the sending domain, or must be listed as one of the Mail Exchange Resource Records for the current domain, or must be one of the three mentioned IP4 addresses. All other hosts should be marked as not being allowed to send email for the sender domain. The latter is called a "soft fail" and is intended to mark the email accordingly, but still accept it.

The postfix mail server can check the SPF record for incoming emails using the postfix-policyd-spf-python package, a policy agent written in Python. The file /usr/share/doc/postfix-policyd-spf-python/README.Debian describes the necessary steps to integrate the agent into postfix, so we won't repeat it here.

The configuration is done in the file /etc/postfix-policyd-spf-python/policyd-spf.conf, which is fully documented in policyd-spf.conf(5) and /usr/share/doc/postfix-policyd-spf-python/policyd-spf.conf.commented.gz. The main configuration parameters are HELO_reject and Mail_From_reject, which configure if emails should be rejected (Fail) or accepted with a header being appended (False), if checks fail. The latter is often useful, when the message is further processed by a spam filter.

If the result is intended to be used by opendmarc (Section 11.1.7.3, “Integrating Domain-based Message Authentication, Reporting and Conformance (DMARC)”), then Header_Type must be set to AR.

Note that spamassassin contains a plugin to check the SPF record.

11.1.7.2. Integrating DomainKeys (DKIM) Signing and Checking

The Domain Keys Identified Mail (DKIM) standard is a sender authentication system. The mail transport agent, here postfix, adds a digital signature associated with the domain name to the header of outgoing emails. The receiving party can validate the message body and header fields by checking the signature against a public key, which is retrieved from the senders DNS records.

→ http://dkim.org/

The necessary tools are shipped with the opendkim and opendkim-tools packages.

First the private key must be created using the command opendkim-genkey -s SELECTOR -d DOMAIN. SELECTOR must be a unique name for the key. It can be as simple as "mail" or the date of creation, if you plan to rotate keys.

Example 11.11. Create a private key for signing E-Mails from falcot.com

# opendkim-genkey -s mail -d falcot.com -D /etc/dkimkeys
# chown opendkim.opendkim /etc/dkimkeys/mail.*

This will create the files /etc/dkimkeys/mail.private and /etc/dkimkeys/mail.txt and set the appropriate ownership. The first file contains the private key, and the latter the public key that needs to be added to the DNS:

Name: mail._domainkey
Type: TXT
TTL:  3600
Data: "v=DKIM1; h=sha256; k=rsa; s=email; p=[...]"

The opendkim package in Debian defaults to a keysize of 2048 bit. Unfortunately some DNS servers can only handle text entries with a maximum length of 255 characters, which is exceeded by the chosen default keysize. In this case use the option -b 1024 to chose a smaller keysize. If opendkim-testkey succeeds, the entry has been successfully set up. The syntax of the entry is explained here:

→ https://tools.ietf.org/html/rfc6376

→ https://en.wikipedia.org/wiki/DKIM

To configure opendkim, SOCKET and RUNDIR must be chosen in /etc/default/opendkim. Please note that SOCKET must be accessible from postfix in its chrooted environment. The further configuration is done in /etc/opendkim.conf. The following is a configuration excerpt, which makes sure that the Domain "falcot.com" and all subdomains (SubDomain) are signed by the Selector "mail" and the single private key (KeyFile) /etc/dkimkeys/mail.private. The "relaxed" Canonicalization for both the header and the body tolerates mild modification (by a mailing list software, for example). The filter runs both in signing ("s") and verification ("v") Mode. If a signature fails to validate (On-BadSignature), the mail should be quarantined ("q").

[...]
Domain                  falcot.com
KeyFile                 /etc/dkimkeys/mail.private
Selector                mail

[...]
Canonicalization        relaxed/relaxed
Mode                    sv
On-BadSignature         q
SubDomains              yes

[...]
Socket                  inet:12345@localhost

[...]
UserID                  opendkim

It is also possible to use multiple selectors/keys (KeyTable), domains (SigningTable) and to specify internal or trusted hosts (InternalHosts, ExternalIgnoreList), which may send mail through the server as one of the signing domains without credentials.

The following directives in /etc/postfix/main.cf make postfix use the filter:

milter_default_action = accept
non_smtpd_milters = inet:localhost:12345
smtpd_milters = inet:localhost:12345

To differentiate signing and verification it is sometimes more useful to add the directives to the services in /etc/postfix/master.cf instead.

More information is available in the /usr/share/doc/opendkim/ directory and the manual pages opendkim(8) and opendkim.conf(5).

Note that spamassassin contains a plugin to check the DKIM record.

11.1.7.3. Integrating Domain-based Message Authentication, Reporting and Conformance (DMARC)

The Domain-based Message Authentication, Reporting and Conformance (DMARC) standard can be used to define a DNS TXT entry with the name _dmarc and the action that should be taken when emails that contain your domain as the sending host fail to validate using DKIM and SPF.

→ https://dmarc.org/overview/

Let's have a look at the entries of two large providers:

# host -t TXT _dmarc.gmail.com
_dmarc.gmail.com descriptive text "v=DMARC1; p=none; sp=quarantine; rua=mailto:[email protected]"
# host -t TXT _dmarc.yahoo.com
_dmarc.yahoo.com descriptive text "v=DMARC1; p=reject; pct=100; rua=mailto:[email protected]; ruf=mailto:[email protected];"

Yahoo has a strict policy to reject all emails pretending to be sent from a Yahoo account but missing or failing DKIM and SPF checks. Google Mail (Gmail) propagates a very relaxed policy, in which such messages from the main domain should still be accepted (p=none). For subdomains they should be marked as spam (sp=quarantine). The addresses given in the rua key can be used to send aggregated DMARC reports to. The full syntax is explained here:

→ https://tools.ietf.org/html/rfc7489

→ https://en.wikipedia.org/wiki/DMARC

The postfix mail server can use this information too. The opendmarc package contains the necessary milter. Similar to opendkim SOCKET and RUNDIR must be chosen in /etc/default/opendmarc (for Unix sockets you must make sure that they are inside the postfix chroot to be found). The configuration file /etc/opendmarc.conf contains detailed comments and is also explained in opendmarc.conf(5). By default, emails failing the DMARC validation are not rejected but flagged, by adding an appropriate header field. To change this, use RejectFailures true.

The milter is then added to smtpd_milters and non_smtpd_milters. If we configured the opendkim and opendmarc milters to run on ports 12345 and 54321, the entry in /etc/postfix/main.cf looks like this:

non_smtpd_milters = inet:localhost:12345,inet:localhost:54321
smtpd_milters = inet:localhost:12345,inet:localhost:54321

The milter can also be selectively applied to a service in /etc/postfix/master.cf instead.

11.1.8. Authenticated SMTP

Being able to send emails requires an SMTP server to be reachable; it also requires said SMTP server to send emails through it. For roaming users, this may need regularly changing the configuration of the SMTP client, since Falcot's SMTP server rejects messages coming from IP addresses apparently not belonging to the company. Two solutions exist: either the roaming user installs an SMTP server on their computer, or they still use the company server with some means of authenticating as an employee. The former solution is not recommended since the computer won't be permanently connected, and it won't be able to retry sending messages in case of problems; we will focus on the latter solution.

SMTP authentication in Postfix relies on SASL (Simple Authentication and Security Layer). It requires installing the libsasl2-modules and sasl2-bin packages, then registering a password in the SASL database for each user that needs authenticating on the SMTP server. This is done with the saslpasswd2 command, which takes several parameters. The -u option defines the authentication domain, which must match the smtpd_sasl_local_domain parameter in the Postfix configuration. The -c option allows creating a user, and -f allows specifying the file to use if the SASL database needs to be stored at a different location than the default (/etc/sasldb2).

# saslpasswd2 -u `postconf -h myhostname` -f /var/spool/postfix/etc/sasldb2 -c jean
[... type jean's password twice ...]

Note that the SASL database was created in Postfix's directory. In order to ensure consistency, we also turn /etc/sasldb2 into a symbolic link pointing at the database used by Postfix, with the ln -sf /var/spool/postfix/etc/sasldb2 /etc/sasldb2 command.

Now we need to configure Postfix to use SASL. First the postfix user needs to be added to the sasl group, so that it can access the SASL account database. A few new parameters are also needed to enable SASL, and the smtpd_recipient_restrictions parameter needs to be configured to allow SASL-authenticated clients to send emails freely.

Example 11.12. Enabling SASL in /etc/postfix/main.cf

# Enable SASL authentication
smtpd_sasl_auth_enable = yes
# Define the SASL authentication domain to use
smtpd_sasl_local_domain = $myhostname
[...]
# Adding permit_sasl_authenticated before reject_unauth_destination
# allows relaying mail sent by SASL-authenticated users
smtpd_recipient_restrictions =
    permit_sasl_authenticated,
    permit_mynetworks,
    reject_unauth_destination,
[...]

It is usually a good idea to not send passwords over an unencrypted connection. Postfix allows to use different configurations for each port (service) it runs on. All these can be configured with different rules and directives in the /etc/postfix/master.cf file. To turn off authentication at all for port 25 (smtpd service) add the following directive:

smtp      inet  n       -       y       -       -       smtpd
    [..]
    -o smtpd_sasl_auth_enable=no
    [..]

If for some reason clients use an outdated AUTH command (some very old mail clients do), interoperability with them can be enabled using the broken_sasl_auth_clients directive.

EXTRA Authenticated SMTP client

Most email clients are able to authenticate to an SMTP server before sending outgoing messages, and using that feature is a simple matter of configuring the appropriate parameters. If the client in use does not provide that feature, the workaround is to use a local Postfix server and configure it to relay email via the remote SMTP server. In this case, the local Postfix itself will be the client that authenticates with SASL. Here are the required parameters:

smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
relay_host = [mail.falcot.com]

The /etc/postfix/sasl_passwd file needs to contain the username and password to use for authenticating on the mail.falcot.com server. Here is an example:

[mail.falcot.com]   joe:LyinIsji

As for all Postfix maps, this file must be turned into /etc/postfix/sasl_passwd.db with the postmap command.