Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Direct SMTP send with full resolution.

by SilentMobius (Initiate)
on Nov 16, 2005 at 17:32 UTC ( [id://509123]=perlquestion: print w/replies, xml ) Need Help??

SilentMobius has asked for the wisdom of the Perl Monks concerning the following question:

I need to do a bulk mailing _again_ (don't worry this is to service subscribers about the actual service) and it’s occurred to me that I really need to check whether the remote end actually accepts the mail.

Now I could start parsing mailserver logs, but in this specific situation I can't help but wonder why I don't just resolve the MX records within the mailer script and send directly (assuming I identify myself correctly and don't have a blackholed IP) that way I could get a return status from Net::SMTP and properly record the transaction.

Now I assumed that someone would have already wrapped up Net::DNS and Net::SMTP in a direct sender module but my searches seem to only turn up perl SMTP receivers or senders that assume an upstream server. I'll write it myself if no one had done it, but I'm guessing there are a few gotya's in there somewhere and would prefer use known modules if available.

Has anyone come across such a thing? If not any guesses why it doesn't exist? I'm sure plenty of mail applications would like to know that the mail was received by the far destination successfully.


Thank for the replies so far. Just to clear a few things up. I'm already parsing bounce massages (the messages already contain unique ID's that can be read out of the bounce) however I'd like to recover the actual SMTP status as I'm finding more and more MTA replying with "temporary" failures when the problem will never be resolved. Additionally there are plenty of our customers whose MTA's don't generate NDR's .

Thanks for pointing out some of the gotyas but I guess people havn't written this yet as they don't perceive they need it or they parse their mailserver logs. Perhaps there is a niche here for me to contribute as I'm still convinced I need it.

Replies are listed 'Best First'.
Re: Direct SMTP send with full resolution.
by sgifford (Prior) on Nov 16, 2005 at 18:32 UTC
    The problem is that even if the mail server initially accepts the message, it may later reject it and send a bounce. qmail, for example, does this by default for all messages.

    One solution is to use VERPs, which let you track exactly what messages bounced. You can then write a handler for those bounced messages, which will do the right thing, and continue using your regular mailer to send. I think if you search for VERP you'll find some examples of how you might approach it in Perl.

Re: Direct SMTP send with full resolution.
by radiantmatrix (Parson) on Nov 16, 2005 at 19:17 UTC

    One of the issues with sending directly to an MX is that many, many MX's will refuse your mail because it doesn't come from a "real" SMTP server. Implementing enough of an SMTP server to play nice with that type of MX would be non-trivial.

    One thing I've had luck with to solve your bounce-detection problem is to add an X- header to the mailing list mails, and let bounces go to a real mailbox. I do this, today, all in Perl:

    1. Change .forward file for the account listed in the From header, so that mail there will be delivered (instead of silently discarded).
    2. Generate messages, tagging each with a common X-List-Message-ID: (in my case, a hash of the message body).
    3. Enable a cron job (which runs a Perl script) that checks the mailbox for bounce responses (which I do by looking for the X-List-Message-ID header) over the next 72 hours, logs the mail addresses which bounced, and clears the message. This runs hourly.
    4. The last run of the cron job from above resets the .forward file and shuts off the cron job until the next time I need them.

    This has worked very well for me, and catches the majority of bounced addresses. I even have a process that follows up on the bounces a week or two later with a "just checking" message before removing the address from my list.

    <-radiant.matrix->
    A collection of thoughts and links from the minds of geeks
    The Code that can be seen is not the true Code
    "In any sufficiently large group of people, most are idiots" - Kaa's Law
Re: Direct SMTP send with full resolution.
by tirwhan (Abbot) on Nov 16, 2005 at 23:14 UTC

    The main problem you'll run into when writing your own mail script will be how to handle all the different (possibly temporary) error conditions that exist. Examples of such errors are:

    • The remote mail server is heavily loaded and your transmissions time out. You need to recover gracefully and try sending the mail at a later time (preferably after some time has passed to avoid congesting the remote server even more)
    • The remote server uses greylisting and asks you to try again later
    • The remote server does spam filtering in an odd way and refuses some of your mails, but not others
    • Due to a temporary network problem almost none of your mail gets delivered on the first try. You need to write the mail out to a spool file, or else you memory fills up, the server crashes and all mails are lost.

    As I said, just a few examples. There's way to code around all of these of course, but thankfully some very clever and diligent people have already written programs that do this. These programs are called mail servers :-) (or Mail Transfer Agents if you want to be precise). I'd recommend letting these specialist programs help you by doing what they do best, a little time spent carefully RTFMing on the MTA of your choice will surely pay off more than writing another one by yourself. As for logging, I'm partial to exim, which is very flexible in what it will log, the logs are easy to parse and you should be able to get all the information you need out of them without much effort.


    Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. -- Brian W. Kernighan
Re: Direct SMTP send with full resolution.
by neilwatson (Priest) on Nov 16, 2005 at 19:26 UTC
    I wrote some applications that did some bulk mailing. First I validated the subscriber list as much as possible (node). The I prepared and sent the email (node).

    Bounces were collected in a separate mailbox. The list was pretty clean so manual checking was acceptable. With things collected in the mailbox it should be easy to script.

    Neil Watson
    watson-wilson.ca

Re: Direct SMTP send with full resolution.
by sunadmn (Curate) on Nov 16, 2005 at 18:37 UTC
    Why not take a look at Mail::Sendmail I think that may hadle everything you are wanting to do.
    SUNADMN
    USE PERL

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://509123]
Approved by herveus
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others admiring the Monastery: (3)
As of 2024-04-19 19:52 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found