On my home network, I have a publically visible domain, complete with MX record sending email directly to me. I'm sure I'm not alone among monks that have the headache of 100s of spams hitting their mail server every minute of every hour, 24 hours a day, ... well, you get the idea. It doesn't stop.
Up until about 6 or 7 months ago, I had an old P2-233 running OS/2 which happily quashed most of the traffic with an OS/2 SMTP server that allowed filters written in any language to deal with inbound traffic while the server handled the protocol. Originally, my filter was in REXX, but eventually after I learned perl, I rewrote it. But that's not the point of this meditation, it's merely background. When the machine died (as in, I couldn't even turn the hardware on anymore), I moved the whole thing over to another box running Linux and Postfix. Unfortunately, I had no idea how to kill inbound email addressed to machines that didn't even exist.
So I didn't. I figured I'd get to it eventually. (Yes, I know, it was bad) That day was rushed when my ISP blocked outbound (but, thankfully, not inbound) anything on port 25, forcing all outbound email to go through their SMTP servers, which would then shut me down due to all the bounceback which they thought meant I had to be infected with a virus to be sending out that much email. So, I cleverly blocked my email server from outbound on 25 myself (at the router), and then wrote a cron-job that ran *every three minutes* to clear out the backlog of email that was no longer being able to be sent (reading the output from mailq, parsing it, and sending msg id's to "postsuper -d -" for deletion). It was a hack. And a bad one at that. But it worked.
Until a typo in my dns server worked around this. So I figured I really had to find a way to get postfix to stop even accepting bad emails. Unfortunately,I really have no idea how those postfix programs work - what is their API. AndI didn't notice anyone having solved this generically on CPAN. The next best thing appeared to be Sendmail::Milter as Postfix advertised Sendmail Milter compatibility. However, I read its ratings (http://cpanratings.perl.org++!!!) and decided that I didn't want to install sendmail to install this, so I checked out Sendmail::PMilter, a pure-perl implementation.
I quickly got to work:
#!/usr/bin/perl use strict; use warnings; use Sendmail::PMilter; my $conn = 'local:/var/run/mymilter.sock'; my $milter = Sendmail::PMilter->new(); $milter->setconn($conn); $milter->register('mymilter', {} Sendmail::PMilter::SMFI_CURR_ACTS ); $< = $> = getpwnam 'nobody'; $milter->main()
First thing - Postfix wasn't connecting. Without being able to waste a bunch of time here, I just changed $conn to 'inet:33333@127.0.0.1'. Problem solved - connections started happening.
So far, so good. So I went and added in my filter. In my case, I wanted toensure that the destination address for each email actually was a valid host name. So I had:
$milter->register('mymilter', { envrcpt => \&my_envrcpt_callback, }, Sendmail::PMilter::SMFI_CURR_ACTS ); openlog 'mymilter', 'pid', Unix::Syslog::LOG_MAIL();
and up top:
use Unix::Syslog qw(:macros :subs); sub is_valid_host { my $host = shift; return undef unless $host; my ($name,$aliases,$addrtype,$net) = gethostbyname($host); defined $name; } sub my_envrcpt_callback { my $ctx = shift; my $rcpt_addr = $ctx->getsymval('{rcpt_addr}'); my ($fqdn) = ($rcpt_addr =~ /\@(\S+)/); if (not is_valid_host($fqdn)) { # no such machine? syslog LOG_INFO, "$$: Rejected mail for $fqdn"; return Sendmail::PMilter::SMFIS_ACCEPT; } Sendmail::PMilter::SMFIS_ACCEPT; }
Seems pretty straight-forward, right? As you can see, I was accepting even the failures, but that was because I was testing still. Once I saw the syslog with correct entries, I changed that to SMFIS_REJECT, and suddenly email started dropping. Beautiful. However, there were still a few issues. First off, checking hosts is a network operation, and thus can be slow. Since these don't change often (it's a home network), I figured a bit of caching could help:
my %cache; sub is_valid_host { my $host = shift; return undef unless $host; $host = lc $host; unless (exists $cache{$host}) { my ($name,$aliases,$addrtype,$net) = gethostbyname($host); $cache{$host} = $name; } $cache{$host}; }
This sped things up noticeably. (Though if the dns server is on the same machine, this is probably negligible - I'm just not sure I'll always have the email and the dns on the same machine, and my testing of this code on a different machine showed this problem.) Normally, I don't bother with minor performance issues, but 100's of DNS queries per minute seemed like a candidate, even though the problem was larger in test than it would be in production.
The next issue was how the dozen or so child processes were being used. Sendmail::PMilter automatically spawned off subprocesses like any good daemon should, and I just wondered if I really needed them, and how they were being used. I had no idea. So I created a global $msg_count, set to 0, and in my_envrcpt_callback, simply did this:
++$msg_count; $0 = "[mymilter:$msg_count@" . scalar(localtime) . "] Checking + $fqdn ($rcpt_addr)";
Now I could see utilisation (or a reasonable facsimile thereof) via ps. I noticed that only about 3 or 4 processes were handling all the requests. This seemed odd to me. I tried one of Sendmail::PMilter's other dispatchers by setting $ENV{PMILTER_DISPATCHER} = 'prefork'; at the top of the script. Suddenly, the processes were being more evenly used. And fewer processes, mind you. It seemed also (from uptime) that I was using less CPU, so I stuck with it.
However, then I found that my milter stopped responding to Postfix after a while. I couldn't figure out why, so I added an "alarm 30" to the top of my_envrcpt_callback (if I stop hearing from postfix for 30 seconds, tell me, and reset each time I get an email to look at). And I added a global $ENV{ALRM} = sub { die "alarm\n" };.
And now this is what I have:
#!/usr/bin/perl use strict; use warnings; use Sendmail::PMilter; use Unix::Syslog qw(:macros :subs); my $conn = 'inet:33333@127.0.0.1'; $ENV{PMILTER_DISPATCHER} = 'prefork'; my %cache; sub is_valid_host { my $host = shift; return undef unless $host; $host = lc $host; unless (exists $cache{$host}) { my ($name,$aliases,$addrtype,$net) = gethostbyname($host); $cache{$host} = $name; } $cache{$host}; } $SIG{ALRM} = sub { die "alarm\n" }; { my $msg_count = 0; sub my_envrcpt_callback { my $ctx = shift; # if we're not used in 30 seconds, quit. alarm 30; my $rcpt_addr = $ctx->getsymval('{rcpt_addr}'); my ($fqdn) = ($rcpt_addr =~ /\@(\S+)/); ++$msg_count; $0 = "[mymilter:$msg_count@" . scalar(localtime) . "] Checking + $fqdn ($rcpt_addr)"; if (not is_valid_host($fqdn)) { # no such machine? syslog LOG_INFO, "$$: Rejected mail for $fqdn"; return Sendmail::PMilter::SMFIS_REJECT; #return Sendmail::PMilter::SMFIS_ACCEPT; } Sendmail::PMilter::SMFIS_ACCEPT; } } my $milter = Sendmail::PMilter->new(); $milter->setconn($conn); $milter->register('mymilter', { envrcpt => \&my_envrcpt_callback, }, Sendmail::PMilter::SMFI_CURR_ACTS ); openlog 'mymilter', 'pid', Unix::Syslog::LOG_MAIL(); $< = $> = getpwnam 'nobody'; syslog LOG_INFO, "Starting up: $$"; END { closelog } $milter->main(10,100);
And it seems to be working. My cron job is still running, but deals with a minor amount of email (addressed to a real host, but no user by that name there - gets rejected there, comes back to the mail server, but can't get back out dueto the block above) relatively speaking. I now have a bit of infrastructure set up such that I could find a way to validate the destination email address and reject immediately if I find the need/time. Not sure how, yet, but we'll see.
(PS - this was far too much for GrandFather's PMEdit to render and paste... *sigh* :-) )
|
---|
Replies are listed 'Best First'. | |
---|---|
Re: PMiltering fun
by rhesa (Vicar) on Jun 20, 2008 at 13:19 UTC | |
by Tanktalus (Canon) on Jun 20, 2008 at 14:30 UTC | |
by mr_mischief (Monsignor) on Jun 23, 2008 at 16:06 UTC | |
Re: PMiltering fun
by dwm042 (Priest) on Jun 23, 2008 at 19:40 UTC |