With qmail, it's trivial to pipe received emails into a different program.

This is not always a good solution though. You seem to recommend creating a perl process for every message that is received and piping the message to that process from qmail.This is vastly more resource-intensive than having a perl daemon that already runs with all required modules pre-loaded.

Of course, you could have a perl daemon running and have qmail send the message to that, but what's the point? You need to use a mail protocol for transfer between qmail and the daemon, so it's no less work to write and you might as well leave qmail out of the process entirely.

You also have the problem of what to do with messages your perl process can not or does not want to accept. If there's an intermediate MTA, rejecting the message will create a bounce, which may be superfluous. Running your own receiving daemon you can reject at SMTP time which means it's up to the sender to create a bounce or ignore.

Now if you're talking about a machine that receives mail for multiple destinations and has to deliver mail to mailboxes as well as to the perl process then I agree, use a real MTA (I personally would use Exim and not qmail but YMWV). But that doesn't seem to be what the OP wants to do.

Creating a new perl process for every message is more resource-intensive in some ways, not in others.

First off, all the SMTP-handling code is in C. That's got to be less resource-intensive than perl. Second, it's already written, most bugs worked out, and someone else will fix any new bugs found. That's less resource-intensive than writing it yourself (you are a resource, too). Third, spawning off a new perl process, compiling and then running a bunch of perl code (both a .pl and a bunch of modules) is really not that resource-intensive. The only really expensive part here is the DB connect, which you can mitigate with FastCGI or DBD::Proxy if it really becomes a problem (premature optimisation--). Oh, and by using qmail, you get the preforking done for you. No, your code isn't forked - but if multiple emails come in at the same time, qmail will kick you off multiple times, providing with better scaling without any thought or design on your part. That, too, makes it less resource-intensive (again, you're a resource). And spewing the message over a pipe (purely in RAM, remember) is trivial - we're talking about copying a couple of KB around (usually). That's not resource-intensive at all, especially considering perl already does that type of work when you pass around scalars instead of references to scalars.

Personally, I like the "use qmail or postfix" solution as a starting point, as it allows you to focus on the real work you're trying to accomplish (stuffing a database) without worrying about stuff you don't really care about (SMTP, preforking, dropping privileges, etc.). If it turns out that you need more, you can always go back and try cleaning up whatever bottleneck there really is, whether that's a db connection or it's the fork/exec overhead, or something else. But to solve performance issues you may never have is really wasting resources: you.

First off, all the SMTP-handling code is in C. That's got to be less resource-intensive than perl.

Not when I tested it. It's hard to accurately benchmark just the SMTP-talking part of an MTA, so I'll not claim those tests were perfect, but accepting multiple emails using Net::Server::Mail did not take significantly longer nor use more memory than doing so with a regular MTA.

Second, it's already written, most bugs worked out, and someone else will fix any new bugs found.

The same is true of Net::Server::Mail.

Regarding the comparative resource-intensiveness of multiple perl processes versus a daemon, I think you're forgetting copy-on-write. You're right in that the OS will almost certainly keep modules in the disk cache if they are called multiple times by quickly-spawning processes. However, the OS still has to create a new process and address space, compile the Perl code and fill the new process memory with it. Do that several thousand times within a second and you will run out of memory even on beefy machines. With a perl daemon that forks off instances of itself, the OS uses COW for the process address space, which means that real memory consumption is far lower.

Another point not mentioned yet is resource consumption limiting. If you have qmail spawning processes you have very little control over how many processes are created simultaneously. This can be especially fatal if your database connection is slow or some other resource is blocking, your OS will simply create processes too fast for the machine to handle and crash. This can be circumvented by using ulimit or PAM, but both of these have drawbacks, are not failsafe and far more complicated than simply setting max_servers in Net::Server::PreFork.


All dogma is stupid.

This is vastly more resource-intensive than having a perl daemon that already runs with all required modules pre-loaded.

If that were the case, all the setups that use spamc to talk to spamd (client and daemon of spamassassin) would come to a grinding halt. But that's not true. A light weight program talking to a daemon doesn't have to be resource intensive. If it's run very often, it'll be in memory anyway. ;-)

Of course, you could have a perl daemon running and have qmail send the message to that, but what's the point? You need to use a mail protocol for transfer between qmail and the daemon, so it's no less work to write and you might as well leave qmail out of the process entirely.

Why on earth would you need SMTP to talk to such a daemon? That what your MTA is for. Your MTA can also take care of queuing - you may get bursts of mail faster than your database can store them. Or your database may be down, and you still want to accept mail.

You also have the problem of what to do with messages your perl process can not or does not want to accept. If there's an intermediate MTA, rejecting the message will create a bounce, which may be superfluous. Running your own receiving daemon you can reject at SMTP time which means it's up to the sender to create a bounce or ignore.

Whether or not the MTA will create a bounce that depends on how you configure the MTA, or on the exit status of the delivery program. I can't answer whether there should be a bounce or not - that's up to the OP. I often set qmail systems up in such a way that they never bounce. They accept anything, and just deliver to /dev/null instead of bouncing. There's already enough spam. ;-)

Anyway, as I said, it's my personal preference. I like qmail, it does its job well and efficiently. It's really easy to configure, and I haven't encountered the resource problems you refer to in the beginning of your post.

If that were the case, all the setups that use spamc to talk to spamd (client and daemon of spamassassin) would come to a grinding halt.

Huh? It's precisely for this reason that any serious spamassassin setup will use spamd and not simply pipe messages through to spamc. The latter will come to a grinding halt under serious load. Thanks for making my point for me ;-)

Why on earth would you need SMTP to talk to such a daemon?

You wouldn't, you could also use LMTP. But your MTA has already accepted the message, which means you need a sensible way to communicate error states back to it. If (as you say) your database is down you want to pass back a temporary error to it. If you don't want to accept the message you need to pass a 5xx. Doing that in any other way than via a mail protocol is error-prone.

Your MTA can also take care of queuing - you may get bursts of mail faster than your database can store them.

That won't work well if you're piping messages to a new process, the process will either time out or die. In both cases your MTA will (normally) create a bounce, because this is a fatal error, the MTA was unable to deliver the message. You can configure the MTA to behave differently but again, that's hard to do correctly (if you care about RFCs) and much more complicated than writing the correct behaviour in Net::Server::Mail. An [SL]MTP-talking perl daemon can just give back a 4xx or 5xx error to the remote mail server, and that server can take care of the queueing for you. Much better.

I often set qmail systems up in such a way that they never bounce.They accept anything, and just deliver to /dev/null instead of bouncing.

Congratulations, you've just broken SMTP. SMTP guarantees that one of two things will happen to a message: 1.)The message will be delivered or 2.) you will get an error message back saying that it wasn't. If you set up a mail system to accept messages and then just blackhole them you have broken the 2nd part of that guarantee and are part of the problem email has these days. Superfluous bounces are another problem, but that can be solved by designing your system so that they do not need to bounce, not by breaking the protocol.


All dogma is stupid.

I used to be (administered a fairly active, but not huge, installation on a campus in the early to mid 90s) a big proponent of qmail for a general mail processing solution, even writing a delayed mail notifier for it (which I no longer recommend using in the general case due to the spamminess of the entire concept of DSN). Since that time, the network landscape has changed, and the default delivery method that qmail uses could be abusive.

For a specific end solution, where you don't use it as an outgoing delivery agent, I can still see it being a very good solution. However, tirwhan makes a very good point about resource usage. If you are aware of this, and deal with the local and remote resource piggyness that qmail can exhibit, it can be a good solution.

My personal preference is Postfix (for lesser patching ;-)), but choose the MTA you know/like best (or which you get best support in your environment) be it exim, postfix, qmail, or even sendmail (courier-mta i would not recommmend ...). I would prefer handing the Mail over to a full-blown MTA, if available (with filtering mechanisms like spamassassin attached etc.)