Watching Creation File and send it to Server

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I have a code running at my Client Machine, which use the Win32::ChangeNotify module ('FILE_NAME' flag) to watch the creation of file in the c:\windows\temp\ directory, by another application.

When that event is flagged, the code will send that brand new file, by ftp, to my Server Machine.

However, due the elapsed time during the writting process by application, the file is sending incomplete to Server.

My question is:

How can I to ensure that send process just begin after the writting to the file by another application had ended?

Thanks in Advance.

Comment on Watching Creation File and send it to Server

Replies are listed 'Best First'.
Re: Watching Creation File and send it to Server by clemburg (Curate) on Nov 06, 2001 at 19:56 UTC
Case 1: You can control the application that writes the file: Make the application output another "signal" file with size 0 (like "touch signal-file" under Unix) when processing of the file to be written has completed. Make the monitoring code react to the signal file instead of the original written file. Case 2: You can not control the application that writes the file: Have the monitoring code start another script that cyclically checks if the size of the file has changed (e.g., every 10 seconds). If the file size has not changed since the last check, send the file to the Server. Christian Lemburg Brainbench MVP for Perl http://www.brainbench.com	[reply]
Re: Watching Creation File and send it to Server by suaveant (Parson) on Nov 06, 2001 at 20:13 UTC
Aside from renaming or moving it when the upload is finished to specify such, your best option is to keep an eye on the file (programmatically). Either watch the modified time or the file size. When it hasn't changed in a minute or so, the file is probably done. Something like this... `my %files = ('foo.txt' => [0,time()], 'bar.csv' => [0,time()]); while(%files) { for(keys %files) { if((-s $_ == $files{$_}[0]) && (time()-$files{$_}[1]>60)) { &move_file($_); delete $files{$_}; } else { $files{$_} = [(-s $_),time()]; } sleep 1; } }` [download] this is not tested, and could be optimized, but what it does is have a hash of files (which you could add to as you go along). The values of the hash is an array holding file size and time it last changed. The code checks to see if the size has changed, and if the time it hasn't changed is over 60 seconds... You have to leave some amount of time since data tends to be written in chunks. If windows changes the modified time as the file gets written a simple `if(time() - (stat $file)[9] > 60)` [download] would be a sufficient test to see if $file has changed in the last 60 seconds... - Ant - Some of my best work - (1 2 3)	[reply] [d/l] [select]
Re: Watching Creation File and send it to Server by andye (Curate) on Nov 06, 2001 at 19:30 UTC
How about creating it with a different name, then renaming it when it's finished? andy.	[reply]
Re: Watching Creation File and send it to Server by jlongino (Parson) on Nov 06, 2001 at 20:29 UTC
Maybe I'm missing something here, but why can't you just try to periodically assert an exclusive lock on the file? If the application has finished writing the file you should be able to do this. If the application doesn't release the lock why would anyone guess that it is finished and try to copy or rename the file? This could screw up the file and possibly crash the application (unless you know the exact nature of reads/writes/locks the application uses). --Jim	[reply]
Re: Watching Creation File and send it to Server by hatter (Pilgrim) on Nov 06, 2001 at 22:21 UTC
I don't know what does and doesn't work under windows but your best bet would either be file locking (either using proper fcntl, or a separate lock files as other people have suggested) or there's probably some equivalent to the unix 'fuser' which tells you which other processes are using a file., and if no other processes are accessing it, then the write must have finished. the hatter	[reply]
(tye)Re: Watching Creation File and send it to Server by tye (Sage) on Nov 07, 2001 at 04:13 UTC
You can use Win32API::File's createFile() [or CreateFile() if you are already into Win32 APIs or are just a masochist] to try to open the file without allowing "write" sharing. Such an attempt will fail if any other process currently has the file open for writing. Keep a list of "new" files and periodically try to open each this way. When you succeed, remove the file from the list and send it. You may not want to keep the file open with no "write" sharing during the transfer. Although this sounds tempting as it ensures that you get a valid snap shot of the file contents and not half of the file contents from before some update and half from after that update, it also would cause other programs that attempt to write to the file to fail during that time (which will probably cause those programs to just give up). - tye (but my friends call me "Tye")	[reply]

Back to Seekers of Perl Wisdom