http://qs321.pair.com?node_id=283849

LupoX has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

what is one of the best ways in perl to monitor a directory for incoming files?

My first idea was:

while (true) { opendir (DIR, "/my/dir") or die "Cannot open /my/dir: $!\n"; my @Dircontent = readdir DIR; close DIR; my $items_in_dir = @Dircontent; if ($items_in_dir > 2) { # > 2 because of "." and ".." do_something_now(); # takes file(s) and moves them } else {sleep 100;} }

I am shure there must be a better idea somewhere out there...

Thank you in advance!!
Georg -> perlmonk.23142@viot.de

Replies are listed 'Best First'.
Re: how to permanently monitor a directory
by BrowserUk (Patriarch) on Aug 14, 2003 at 14:43 UTC

    If your running Win32 then Win32::ChangeNotify makes this very easy.

    #! perl -slw use strict; use Win32::ChangeNotify; my $path = 'p:\test'; my $notify = Win32::ChangeNotify->new( $path, 0, 'FILE_NAME' ); my %last; @last{ glob $path . '/*' } = (); while( 1 ) { print('Nothing changed'), next unless $notify->wait( 10_000 ); # Check every 10 seconds $notify->reset; print $/, 'Something changed'; my @files = glob $path . '/*'; if( @files < scalar keys %last ) { delete @last{ @files }; print 'These files where deleted: '; print for keys %last; print''; } elsif( @files > scalar keys %last ) { my %temp; @temp{ @files } = (); delete @temp{ keys %last }; print 'These files where created: '; print for keys %temp; print''; } else { print "A non-deletion or creation change occured"; } undef %last; @last{ @files } = (); } __END__ P:\test>changenotify Nothing changed Nothing changed Something changed These files where created: p:\test/fred Nothing changed Nothing changed Something changed These files where deleted: p:\test/fred Nothing changed Nothing changed Nothing changed Nothing changed

    You'll need control-break rather than ^C to interupt this script.

    To extending this to monitor the whole subtree, change the 0 in the new() call to 1, though you'll need some extra work to find out what changed where.

    I'm not aware of a similar mechanism under *nix.


    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
    If I understand your problem, I can solve it! Of course, the same can be said for you.

      The comment on the

      unless $notify->wait( 10_000 );
      line is misleading. The script doesn't check for changes every ten seconds. It waits for a change for 10 seconds, prints the "Nothing changed" message and goes back waiting. Any changes in the directories are noted immediately.

      Jenda
      Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.
         -- Rick Osborne

      Edit by castaway: Closed small tag in signature

        Good point. That was badly worded.


        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
        If I understand your problem, I can solve it! Of course, the same can be said for you.

Re: how to permanently monitor a directory
by monktim (Friar) on Aug 14, 2003 at 13:26 UTC
Re: how to permanently monitor a directory
by l2kashe (Deacon) on Aug 14, 2003 at 13:59 UTC
    #!/usr/bin/perl use strict; use DB_File; my $base = '/some/dir'; # dir to check my $data = '/some/file.db'; # file name cache my $sleep = 100; # how long to sleep for my $maxtime = 86400; # max time before key removal while (1) { # # we need to tie and untie each time just to be safe due # to file caching and flushing, which sometimes doesnt # happen untill the file has been untied. # If another monk can help here that would be nice. # tie(my %files, 'DB_File', $data, O_RDWR, 0644) or die "Cant tie $data: $!\n"; opendir(BASE, $base) or die "Cant open dir $base: $!\n"; for ( grep(!/^\./, readdir BASE) ) { # avoid . and .. :) if ( $files{$_} ) { $files{$_} = time(); } else { do_something(); $files{$_} = time(); } } closedir(BASE); for ( keys %files ) { # remove the file if it hasnt been around for a week delete($files{$_}) if ($files{$_} >= $maxtime); } untie(%files); sleep $sleep; }

    use perl;

Re: how to permanently monitor a directory
by Foggy Bottoms (Monk) on Aug 14, 2003 at 14:23 UTC
    You can use AdvNotify. It's a pretty powerful package that'll watch any folder and/or subfolder for changes. I've been using it lately and it's very convenient... Unfortunately, it doesn't run under Win98...
Re: how to permanently monitor a directory
by erasei (Pilgrim) on Aug 14, 2003 at 13:15 UTC
    Well, like all things Perl, TISMTOWTDI.

    If you know what the files will be (example, they are uploaded by a script so the filename will always be the same) you could do something like:

    while (1) { if (-e "/my/dir/filename.dat") { do_something_now(); } }
    If not, you could use your own code. I'm not sure I'd have it 'sleep' though. Run it from cron every so often would make more sense to me. Also, you can drop a couple of lines by changing
    my $items_in_dir = @Dircontent; if ($items_in_dir > 2) ..
    to
    if (scalar(@Dircontent) > 2) ..
      This:
      my $items_in_dir = @Dircontent; if ($items_in_dir > 2) ..
      won't work if some smart alec decides to create a subdir in your drop area. try using -f

      TISMTOWTDI:

      use strict; use warnings; sub scandir { my $dir = shift; my $fileProcessor = shift; opendir (DIR, $dir) or die "Cannot open $dir: $!\n"; for (readdir DIR) { if ( -f ) { &$fileProcessor($_); } } close DIR; } sub do_something { my $filename = shift; print "got file: $filename\n"; } sub do_something_else { my $filename = shift; print "got another file: $filename\n"; } scandir('.',\&do_something); scandir('.',\&do_something_else);
Re: how to permanently monitor a directory
by liz (Monsignor) on Aug 14, 2003 at 13:17 UTC
    Not Perl, but maybe the thing you need: Tripwire.

    Liz

Re: how to permanently monitor a directory
by Aristotle (Chancellor) on Aug 14, 2003 at 21:06 UTC
    Check out SGI::FAM (terribly named module in my opinion) for doing this on Unixoid systems.

    Makeshifts last the longest.

Re: how to permanently monitor a directory
by cfreak (Chaplain) on Aug 14, 2003 at 13:21 UTC
      Of course, if someone copies a file there and is still dropping it while your loop runs in the cycle you may see issues with truncating data. You should also build in tests to see if the file is done being written to. I assume you are moving the file to proccess it not just to move it.

      -Waswas