Qmail Patch Explanations and Recommendations

Only patch if you really need to and only if you understand exactly what the patch does.

Contents

  1. Introduction
  2. How to apply
  3. Minimum Patches
  4. Small Site Patches
  5. Big Sites

Introduction

I'm a qmail enthusiast. I've even written a book about it (buy a copy!). One of the things that any Qmail administrator ought to know is that (for a variety of historical reasons) Qmail has a lot of available patches. There are good reasons why you might need each patch, but in general you should try to use as few as possible. It is the general consensus that the more patches you use, the more likely there are to be bugs that crop up and conflicts between patches. The general rule for patching qmail is highlighted across the top of this page. Ignore this rule it at your peril.

That said, I have found a collection of patches that work for me. I have installed them for reasons I explain below, which are not necessarily reasons that matter to everyone. This is a list of the patches I use (and some others I have run across), with descriptions and reasons for using them. Most of these patches were pulled from qmail.org, which has unfortunately vanished.

Some people feel that qmail has certain shortcomings (like non-conformance to RFCs) and either like complaining or have found a solution, or have developed patches to fix the problem. Trust me, the issues have been hashed over again and again and again on the qmail mailing list, and the current state of things seems to keep the most people happy. In most cases, things are the way they are on purpose (please feel free to search the qmail list archives for the explanation of any particular detail!).

These patches all work on netqmail, which you should be using anyway. While vanilla qmail is as cool and unbreachable as ever, netqmail is a convenient packaging of some of the patches that have cropped up as being very important. It is not officially the same thing as qmail, but is a convenience packaging of qmail. For more information, go here.

One final note, some of these patches conflict (or seem to), and resolving them takes a little bit of knowledge of C. If and when I get a chance, I'll put up an über patch collecting and reconciling the patches that I use. For any others, you're on your own.

Apply these patches with the following commands:

cd /path/to/netqmail/
patch -p1 < /path/to/patch

Minimum Patches

(Recommended for: everybody who refuses to use netqmail) These patches all fix known bugs in qmail. They are also almost all simple patches, and unlikely to conflict with other patches.

Small Site Patches

These are patches that I have installed on my small qmail installation and I recommend them. They are not necessary, but they are quite useful. In some cases, the feature the patch is for requires a little bit more setup, and perhaps some more software. This is generally minimal, but very worth it.
RFC 2821
DNS-related Patches
Common Spam Relay-attempt Rejection
Spammers frequently rely on bizarre manipulations of the standard user@host.tld email address format in order to trick mail servers into forwarding their spam. Tricks like user@remoteaddress@thismailserver, for example, or user%remoteaddress@thismailserver (aka "the percent hack"). Unfortunately, qmail doesn't reject all of these attempts out of hand, but instead accepts them and generates a bounce message. This behavior is technically valid, but is unwise: it can be used to create bounce-spam. It's important to state: this does NOT make qmail a relay... but it can be used as a bounce-spam source (though the content of the bounce is not entirely dictated by the sender). Some automated relay testing software assumes that if the message is accepted then it will be delivered/relayed instead of bounced (or black-holed), and as a result will provide an inaccurate diagnosis: that your server is an open-relay. Such relay testers are incorrect. However, as a result, you may get onto one or two blacklists that are based on such relay checks, even if you delete such email messages. To avoid this hassle, use this patch, written by Russel Nelson, to reject such relay attempts. This patch precludes using the percent hack, but you shouldn't be allowing that anyway so it's no big loss. (local copy) (qmail.org)
RFC 1870 (ESMTP SIZE)
One useful extension people have made to the SMTP protocol is defined in RFC 1870, which establishes the SIZE command. This has two primary consequences. First, it announces (as a response to the initial EHLO greeting) the maximum size of mail that the server is willing to accept. Qmail's default behavior without this patch is to accept any amount of data and if the amount exceeds the maximum allowed, to reject it after it has been received. This wastes bandwidth: smart ESMTP senders that also support the SIZE command could have avoided wasting that bandwidth. Second, as part of the MAIL FROM command, the sender can specify the size of the mail it is going to send, allowing the receiver (qmail) to reject it at that point for being too big. (local copy)(will.harris.ch)
SMTP AUTH
If all you're doing is sending email from your own machine, then you don't have to worry about who can relay email through you — only your machine can, of course. But, if you may want to send email from random locations (say, from your laptop as you travel around the country), you don't want to have to allow just any old computer to relay email through your server. You need some way to identify and authenticate yourself to the email server, so you can have it let you relay email through it without opening it up to every spammer in the world. To do this, you need support for SMTP AUTH (this, essentially, makes qmail compliant with the SUBMISSION protocol, from RFC 2476). This patch is written by Erwin Hoffmann, and requires a bit of extra configuration to set up. Full documentation is here. (local copy) (fehcom.de)
Caveat: This patch defaults to enabling CRAM-MD5 authentication. If you really want/need CRAM-MD5 authentication (that decision is up to you), you will need to use the cmd5checkpw implementation of checkpassword (fehcom.de). Otherwise, you'll want to disable CRAM-MD5 advertising, which has to be done at compile-time (unfortunately).
SSL (STARTTLS)
SMTP transmits email unencrypted. Other than privacy concerns, this is not typically considered a problem. However, if you use SMTP AUTH, you are sending your username and password across the network in plain text (which is easy for a hacker or spammer to extract if they wanted to). The solution is to use encryption in SMTP—in other words, make qmail support the STARTTLS ESMTP extension. Frederik Vermeulen wrote a patch to get it to work. It adds one minor step to the compilation of qmail: you must create a server certificate (run make cert before running make setup check). Also, you must create a cron job to rebuild the certs daily (because otherwise, over time, an attacker could figure out what they are). Commonly, when someone indicates that they want qmail to support SSL/STARTTLS they will be referred to a project like mailfront. While mailfront is a worthy project, it doesn't solve the entire problem. Specifically, it doesn't enable qmail to use SSL for sending mail to other servers that support STARTTLS (this is a problem of privacy; but keep in mind that if the email is being relayed, it may be transmitted via an unencrypted communication later—if you're really worried, use PGP). This patch, however, does enable qmail to do that. (local copy) (inoa.net)
Executable Banning
As anyone who has spent any time paying attention to computer news in the last decade or so knows, computer viruses are a problem. Email has become a common delivery mechanism for viruses, and even more specifically, most of them are Microsoft viruses (that is, most viruses are executable files that only run under some form of Microsoft Windows, and not under any other operating system). All Microsoft Windows programs begin with information that tells Windows how to load the program. This information is small and can easily be detected. This information is the same in all Windows programs and is required in order to run the file. It is not used in scripts (like Visual Basic scripts (.vbs)), but most email viruses these days aren't scripts anyway (nevertheless this patch is NOT a full anti-virus solution, just a quick and fast help). An email policy that may help prevent the spread of Windows viruses is to say that you may not send Windows executables (if you must send them, compress them first—that changes their header information). A way to enforce a policy like this is to look at all email attachments and see if they begin with Windows executable header information, and reject them if they do. Russell Nelson wrote a patch to do this. It is worth noting that this should NOT be considered sufficient virus protection for a large email server; you will want to use a real virus scanner. This patch is a fast way to enforce a policy against emailing plain executables. Because of the success of policies like this, such attachments have become less common, though they do still happen. (local copy) (qmail.org)
QMAILQUEUE
The complete anti-virus solution for your email server is to use something like qmail-scanner, simscan, or my current favorite, qmail-qfilter, in combination with a good antivirus program like clamav. These programs sit in the middle of your qmail installation, scan every email with whatever scanners you specify, and if the scanners approve, pass the messages along to the usual queue of email (think of them as filters). In order for these programs to sit in the middle of qmail like that, the easiest (and most common) method is to use the QMAILQUEUE patch, written by Bruce Guenter. The QMAILQUEUE patch has other uses, because it is used as a generic way to insert another program into the qmail queueing chain. This patch is part of netqmail. (local copy) (qmail.org)
Recipient Verification
One of the behaviors that many people dislike about qmail is that it accepts email based only on the domain of the recipient, rather than the full recipient address. The argument is that this behavior causes qmail to generate spurious and avoidable bounce messages (and I agree; some bounce messages are avoidable). There are several ways to check this, but perhaps the most flexible are the RCPTCHECK patch and the CHECKENV patch. Both are functionally equivalent. They run an admin-determined script to determine whether the recipient is valid. Full details on the RCPTCHECK patch are available here, but if Jay takes them down, they're also here. (local copy of the patch) (soffian.org) Kelly French's CHECKENV patch can be obtained here. I personally use the RCPTCHECK patch, because that's the first one I found, but they're both equally good (and CHECKENV is older).

More recently, Amitai Schlier pulled together a selection of useful tools for doing recipient validation using a variety of criteria. You can see his work here.

These patches are far more powerful and flexible than most people give them credit for. For example...

Using such a patch, it is trivial to implement things like three-tuple greylisting (based on RECIPIENT, SENDER, and TCPREMOTEIP). You can also, as Soffian suggests, use a script that queries another server to see if the recipient is valid. I like this script in part because I can use different verification techniques depending on the domain (for example, I can do one kind of check for lists.memoryhole.net, and another for memoryhole.net itself). I often hear questions on the qmail mailing list that could be solved simply and easily with this (relatively trivial) patch, and every time, I am re-impressed with the power and flexibility that this patch provides. It doesn't get enough credit.

Here's how it works: when the environment variable RCPTCHECK is set, qmail-smtpd will execute the program specified in that variable. Before the program is executed, the recipient in question is stored in the RECIPIENT environment variable, and the sender is stored in the SENDER environment variable. The exit code of the specified program determines whether qmail views the recipient as valid or not. Possible exit codes include:

100
Means that the recipient is invalid. The client receives a 553 error.
111
Means that there was a temporary problem verifying the recipient. The client recieves a 421 error code and the connection is closed.
120
Means that the recipient check program could not execute correctly. The client recieves a 421 error code and the connection is closed.
anything else
Means that the recipient is valid.

A trivial example script would be something like this:

#!/bin/sh
GoodRecipient=0
BadRecipient=100
Error=120
if grep "$RECIPIENT" /var/qmail/control/goodrcptto >/dev/null; then
        exit $GoodRecipient
else
        exit $BadRecipient
fi 

Here's a slightly more complex example:

#!/bin/sh
GoodRecipient=0
BadRecipient=100
Error=120
User=$( echo "$RECIPIENT" | cut -d@ -f1 )
Domain=$( echo "$RECIPIENT" | cut -d@ -f2- )
if ! type id >/dev/null 2>&1 ; then
        # id program not in PATH
        exit $Error
fi
if id "$User" >/dev/null 2>&1 ; then
        exit $GoodRecipient
else
        exit $BadRecipient
fi

A basic example of how to do greylisting with this patch is here (txt) (based on Kelly French's script). Note that you'd want to combine that script with some other kind of recipient validation as well, and needs a cron job to clean up after itself.

Hide Comment

Better Maildir
Some operating systems quickly recycle PIDs, which can lead to collisions between Maildir-style filenames, which must be unique and non-repeatable within one second. This patch is a means of updating qmail-local to use the format of the revised Maildir protocol, available here. This patch written by Toby Betts. (local copy) (vorlon.cwru.edu)
Better qmail-smtpd Logging
It is often convenient, for diagnosing problems and for monitoring what your server is up to, to have it log it's actions and decisions. The qmail-send-related programs (qmail-remote and qmail-local) do this well, but qmail-smtpd doesn't produce any logs at all. I wrote a patch to add some basic logging to it, so that it records its decisions in the log (i.e. prints them to stderr). There are several similar patches out there, which one you use depends largely on whose logging format you like best. My patch is available here. (Old versions are available here.)
DomainBindings
For servers that host multiple domains or have multiple IP addresses assigned to them, it is sometimes useful (or important) to have qmail use a specific IP address for its outgoing mail. By default, qmail uses whatever address the OS chooses for outbound connections by default. With this patch, you can specify which address to use. It uses a control file similar to smtproutes to specify the outbound IP address to use, based on the sender's domain (local copy) (pyropus.ca)
DKIM Wrapper
To sign all outbound messages with a DKIM and/or DomainKey, there are several alternatives. One is Russ Nelson's qmail-dk. While popular, it only handles DomainKeys (not DKIM, the more popular successor to DomainKeys), and doesn't sign all outbound messages, merely all inbound messages (that may then become outbound). Thus, things like bounce-messages cannot be signed, because they don't go through the usual qmail-smtpd/qmail-inject filters. An alternative that addresses this problem is to use a script wrapper around qmail-remote, like this (txt).

Integrating that script into your qmail setup is simple: rename the qmail-remote program to qmail-remote.orig and put that script in as qmail-remote (make sure it's readable and executable by everyone—note that that is different from the original qmail-remote permissions). The script can use two programs to do its job: the dktest program that comes with libdomainkeys (obsolete) and dkimsign.pl that comes with Perl's Mail::DKIM module. (That script expects that dkimsign.pl accepts the --key argument; if yours does not, you can use this simple patch (txt) to modify your dkimsign.pl so that it does accept the --key argument, or download the copy below.)

If you're interested in verifying both DKIM and DomainKey signatures, a similar script that can be used in much the same way as Russ Nelson's program is here (txt). This script relies on dktest as well, but also requires a script called dkimverify.pl. A similarly named script comes with Mail::DKIM, but is not particularly useful; I wrote one that generates some useful headers, which is available below.

Generic copies of those scripts can be had here: dkimsign.pl (txt) and dkimverify.pl (txt)

Patches for Big Sites

External Todo (qmail-todo)
The qmail mail queue has two sides: the ingestion side ("todo") and the delivery side. In vanilla qmail, the program that spawns email delivery agents to empty the delivery side of the queue is the same program that drains the ingestion side of the queue. It must recognize that new email has been added to the mail ingestion queue, parse it, generate the necessary metadata, and places the message into the mail delivery queue. When email comes into the server extremely quickly, qmail can sometimes spend so much time draining the ingestion queue that deliveries don't get scheduled and the email starts accumulating in the "ingest" side of the mail queue. If this is sustained, most of your email queue may be in an undeliverable state in the todo queue rather than the delivery queue (note that this is only a problem when you get massive amounts of email at the same time (many per second; the threshold is system-specific)). This behavior is often referred to as "silly qmail syndrome." André Opperman wrote a patch to solve the problem. It creates a separate qmail-todo program, whose only job is processing the ingestion/todo queue. This allows draining the todo queue to happen asynchronously and thus does not prevent deliveries from ocurring at the same time. The code of the solution is a bit complicated, and is not worth applying unless you are experiencing "silly qmail syndrome." This patch is referred to as the ext_todo patch. (local copy) (nrg4u.com)
Big Todo
Another issue that can come into play when email comes into your server extremely quickly is that if several thousand email messages accumulate in the todo queue, an inefficient filesystem that has trouble with large directories may restrict the speed of todo processing significantly. Such filesystems were standard back in 1998, and qmail had to work around this problem for the majority of the mail queue. It did so by creating many sub-directories (in essence, hash buckets), to limit the number of messages that would likely be in any one directory. For whatever reason, this workaround wasn't applied to the todo queue. The best way to address this is to use a filesystem capable of handling directories with large numbers of files. Ext3, for instance, uses a hash-based data structure to implement directories, which handles large numbers of files in a single directory with ease. But upgrading/modernizing your filesystem is not always possible, depending on your situation. Russ Nelson wrote a patch to make qmail use the same multi-directory hashing system qmail uses in its main queue in the todo portion of the queue as well. This patch is referred to as the "big-todo" patch. There's no reason to apply this patch unless you really really need it, because getting a better filesystem (or enabling the right features on the filesystem you're already using) is a more efficient option (though using the multi-directory scheme won't really cause any *trouble* on newer filesystems either); this patch can require that your queue be rebuilt, and so applying it to a running system can be a bit of a pain. (local copy) (qmail.org)