Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much

Triggering Perl Scripts

by Perl Newby (Acolyte)
on Apr 23, 2001 at 19:52 UTC ( #74733=perlquestion: print w/replies, xml ) Need Help??

Perl Newby has asked for the wisdom of the Perl Monks concerning the following question:

I have a sports web site where I am going to post the box scores of pro sports games. I have already written script that will parse the data and put it into an HTML format. My problem is that I want to be able to set up some form of a trigger that will automatically fire when I receive a new file. This is the scenario I want to use: 1.) I receive the box score via FTP in a text file format, ex. mlb_boxscore.txt 2.) As soon as the file is FTP'd I would like a trigger to set off the code I have written that parses and formats the data 3) After the code runs, the web site is updated. Any suggestions on how I would write a trigger that could accomplish this?

Replies are listed 'Best First'.
Re: Triggering Perl Scripts
by stephen (Priest) on Apr 23, 2001 at 19:57 UTC
    Probably the best you can do is have a CRON job that runs every few minutes that checks for the existence of a new file. If you do this, I'd suggest that you have the job
    1. Spot the existence of the new file.
    2. Wait for a bit, then see if the file's size has changed.
    3. Only parse and format the data when the file size has stabilized.
    Otherwise, you're in danger of parsing the file before it's fully uploaded.


Re: Triggering Perl Scripts
by AgentM (Curate) on Apr 23, 2001 at 20:00 UTC
    A couple of options:
    • set up a cron job to scan the directory for files or new files and fire off the perl parsing script. This is the most wasteful option because it polls, but is most likely to be supported on your system.
    • Install tripwire. You may be subject to licensing fees, but this will most definitely get the job done.
    • Hack/modify/add a hook to the FTP server to fire off the perl script whenever a file is put into a specific directory. If the FTP server is used for nothing but this uploading thang (sic), then use some CPAN modules to throw together your own FTP server or modify a Perl FTP server that you find online. This is the least secure of your options- that is, if you're not pedantic about your UN*X programming.
    It looks like Option #2 is the winner!
    AgentM Systems nor Nasca Enterprises nor Bone::Easy nor Macperl is responsible for the comments made by AgentM. Remember, you can build any logical system with NOR.
Re: Triggering Perl Scripts
by arturo (Vicar) on Apr 23, 2001 at 20:07 UTC

    Here's another option, akin to AgentM's suggestion that you might write a Perl FTP server and handle the whole thing through that. Handle the uploads via CGI scripts, and have the results submitted via LWP (or whatever makes sense -- as long as you submit the files via HTTP and have a CGI handle them).

    Once the upload is complete, the CGI can check the file and update the site.


Re: Triggering Perl Scripts
by premchai21 (Curate) on Apr 23, 2001 at 20:12 UTC
    Have the file e-mailed to a special address that pipes the mail to your script.
Re: Triggering Perl Scripts
by arhuman (Vicar) on Apr 23, 2001 at 20:45 UTC
    To my mind, the CRON solution is the better one,
    But If you can't (or dont want to) use it
    (only root's allowed cron, Windoze boxes without cron, not used to CRON syntax...)
    You can simply write a perl script that'll check for the existence of a file, process it (and move/delete it)
    if it's present, and then sleep 5 mins (in an infinite loop).

    Quick and dirty but simple...<br

    "Only Bad Coders Badly Code In Perl" (OBC2IP)
Re: Triggering Perl Scripts
by suaveant (Parson) on Apr 23, 2001 at 20:06 UTC
    One way is to have the script on a cron, and every five minutes or whatever check the directory, if the file is there (and complete) then you can move it to a temporary spot and parse it. Of course, then you need a way to make sure the file is finished uploading.
                    - Ant
Re: Triggering Perl Scripts
by diarmuid (Beadle) on Apr 23, 2001 at 20:37 UTC
    Put your perl script in a crontab job to check the ftp site and run it every minute or whatever. Get it to check the modification date and only update the file if it has changed.

    I tried using a mkfifo but that was more messing than needed

Re: Triggering Perl Scripts
by asiufy (Monk) on Apr 24, 2001 at 19:52 UTC
    Instead of using the CRON, you can have your own parsing/formatting script poll for the new files in the specific directory. Something like this:
    while (1<2) { opendir (DIR, $dir) or die "cannot opendir $dir"; my @only_files = grep {-f "$dir/$_"} readdir(DIR); readdir DIR; closedir DIR; foreach $file (@only_files) { &process_file ($file); } sleep(10); }

    This will poll $dir every 10 seconds (set in the sleep line) for any new files.

    If your files are big, you might want to add some of the verification mentioned earlier, regarding the file size, etc.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://74733]
Approved by root
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (4)
As of 2023-05-29 17:48 GMT
Find Nodes?
    Voting Booth?

    No recent polls found