Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

Re: Problem using Net::FTP

by jarich (Curate)
on Mar 04, 2004 at 18:59 UTC ( [id://333972]=note: print w/replies, xml ) Need Help??


in reply to Problem using Net::FTP

Okay, the problem is that you're ftping as many files as the info file provides you with. If this usually contains 2 files then that's great, but you probably want to allow it to later contain 3 or 4 or more, for flexibility's sake.

If you're happy with this idea, then you'll probably realise that specifying the directory that you want the file to be ftped into inside the info file, with the renaming information is a good idea.

Before I get to that I just want to bring up one problem I immediately had with your script: 

open(FILE, "<info>");
while (<FILE> ) {

[download]

Firstly, I don't think you actually mean to have that > there. Secondly you're not checking that your open succeeded. What if you're not allowed to read the file? Permission denied? You should definately check this.

So anyway, we can fix your code like this: 

        # open the file safely or complain it wouldn't open
open(FILE, "<", "info") or die "Failed to open info file: $!";
while (<FILE> ) {
        s/\W*$//;            # remove trailing whitespace
        next if (!$_);       # skip empty lines
        
              # check that we get our match.  If not,
              # complain and move on
        unless(/^(.+?) \s+ (.+?) \s+ (.+?)$/x) {
                print STDERR "$_ is not a valid line";
                next;
        }
        my ($old, $new, $ftpdir) = ($1, $2, $3);
        rename $old, $new;
        
        # ftp transfer

        my $server = "X.X.X.X";
        my $ftp = Net::FTP->new ($server, Timeout => 9000, Debug => 3)
+;
        $ftp or die "$server: cannot connect: $@";
        # If you don't use ~/.netrc
        $ftp->login ('anonymous', 'mail@adress') 
            or die "$_: cannot logon: " . $ftp->message;

        # change remote directory for the first file
        $ftp->cwd($ftpdir);    # use directory from file

        # Send file to ftp server
        $ftp->put($new) or
        die "$server: cannot put $new: " . $ftp->message;

        # Sleep for 20 minutes before processing next file.
        sleep (20 * 60)
}

[download]

I really don't like your regular expression to pull the filenames out of the line. It's open to getting all sorts of things wrong. I'd suggest you instead change it to something like: 

#Expect 2 filenames and a directory path
/^([\w._-] \s+ ([\w._-]) \s+ ([\w._\/-]$)/;

[download]
This prevents people from entering the following lines and getting interesting results: 
/home/jarich/passwd /etc/passwd abc
a                   b                c

[download]
In the first case it's pretty easy to spot what $1, $2 and $3 are set to and I hope you understand why that might be a bad thing. In the second case we get this: 
$1 = "a                  ";
$2 = "b               ";
$3 = "c"

[download]
Why all those spaces when you have \s+? Because .* is greedy from the left, not from the right.

I'd suggest you work out what characters you want to allow in your filenames and only accept those. I've given you a good start. [\w._-] only accepts letters, numbers, dots, underscores and dashs. If you need anything else then add that in... with caution. Oh, and make sure you add them before the dash, not after. This will probably help you in the future too.

One final thing. Notice how using identation makes the script easier to read? You might want to make a special effort on that next time you post here.

all the best, and I hope this helps

jarich

Replies are listed 'Best First'.
Re: Re: Problem using Net::FTP
by cc (Beadle) on Mar 06, 2004 at 00:47 UTC
    thanks jarich
    for the great and very competent support,
    but I wrote my problems not so clearly.

    the info file looks :
    ----------------------
    AB00001 N120001
    BB00002 N200023
    ----------------------
    with info file I should get many times a day
    also 2 files: AB00001 and BB00002
    all files are TEXT files.
    left above is the name of the first file and at the bottom left hand corner the name of the second file.

    script should check if the files are complete and names on info file matches to the names of this 2 files.
    only if complete, rename first file to N120001 transfer,
    wait for 20 minutes, rename the second to N200023 and transfer to the different directory then the first one.
    the name of the info file never changed, but names of these 2 files and names on the info file change all the time.
    on info file I don't get any paths to change directory on the ftp server.
    these paths are fixed:
    already renamed first file N120001 should be transfered to:
    $ftp->cwd("FTP/A1");
    and second file N200023 to:
    $ftp->cwd("FTP/B2");
    this is, I think my biggest problem at the moment.

    when the transfer completed should create a new subdirectory with date and time, something like: 200402241200
    and put (backup) all files there, otherwise files will be overwritten each time.

    this script will be scheduled every 5 minutes from cron job and waits for the files.

    there are very important things:
    the second file cannot be transfered without the first one
    the second file must be transfered after 20 minutes after the first transfer.
    these files cannot be transfered twice and should be there.

    and I would like use your script, that you already posted.
    if you have time to help by this script, I 'll be very happy
    I know, it's a lot of changes, but I want to explain,
    what should do this script on the end.

    kind regards
    cc
[reply]
      the info file looks :
      ----------------------
      AB00001 N120001
      BB00002 N200023
      ----------------------
      However, because the code looks like: 
      while (<FILE> ) {

      [download]
      you will process as many files as the info file has lines. So, if the info file were to be replaced with something like this: 
      AB00001 N120001
BB00002 N200023
CB00003 N300011
DB00004 N400022

      [download]
      then your code will attempt to ftp and rename each of those 4 files.

      If you want to limit the number of files processed to always be two and only two then you don't want to use a while loop. Perhaps a for loop would be better: 

      # replace the while(<FILE>) { with the following two lines

for(my $i = 1; $i <= 2; $i++) {
        $_ = <FILE>;

      [download]

      script should check if the files are complete and names on info file matches to the names of this 2 files.
      I have no idea what this means.

      first file N120001 should be transfered to: $ftp->cwd("FTP/A1"); and second file N200023 to: $ftp->cwd("FTP/B2");
      If you can't change what gets put into the info file then my suggestion the other day won't work as well. If you can change it then that would probably make better sense. This way the directory names to ftp things to aren't hard coded in your script and therefore may be easier to change in future.

      Assuming you can't make the changes to the info script then you might want to change your script to look more like: 

              # ftp directories
my @ftp_locations = ("FTP/A1", "FTP/B2");

        # open the file safely or complain it wouldn't open
open(FILE, "<", "info") or die "Failed to open info file: $!";

while (<FILE> ) {
        s/\W*$//;            # remove trailing whitespace
        next if (!$_);       # skip empty lines
       
              # check that we get our match.  If not,
              # complain and move on.  Want to see two
              # filenames.
        unless(/^([\w.-]) \s+ ([\w.-])$/x) {
                print STDERR "$_ is not a valid line";
                next;
        }

        my ($old, $new) = ($1, $2);
        rename $old, $new;

              # Now that we have our filenames, grab an
              # ftp directory from the list
        unless(@ftp_locations) {
              die "Not enough specified ftp_locations for ".
                  "given number of files!";
        }
        my $destination = shift @ftp_locations;
      
        # ftp transfer

        my $server = "X.X.X.X";
        my $ftp = Net::FTP->new ($server, Timeout => 9000, Debug => 3)
+;
        $ftp or die "$server: cannot connect: $@";
        # If you don't use ~/.netrc
        $ftp->login ('anonymous', 'mail@address')
            or die "$_: cannot logon: " . $ftp->message;

        # change remote directory for the first file
        $ftp->cwd($destination);    

        # Send file to ftp server
        $ftp->put($new) or
        die "$server: cannot put $new: " . $ftp->message;

        # Sleep for 20 minutes before processing next file.
        sleep (20 * 60)
}

      [download]
      Note that this still allows you to deal with more than 2 files in the info file, but only if you add additional ftp locations into the array up the top.

      when the transfer completed should create a new subdirectory with date and time, something like: 200402241200 and put (backup) all files there, otherwise files will be overwritten each time.
      How about you give this bit a go and get back to us with how you do at it?

      and I would like use your script, that you already posted.
      Ah, but of course. Certainly, go ahead and use the code I've provided you with. Good luck with the rest.

      Hope this helps,

      jarich

[reply]
[d/l]
[select]
        hi

        thank you very much, it helps a lot.
        I'm also very happy that you explain so clearly.
        You are excellent teacher.
        I have only some little problems and don't know how to solve it.
        the first problem is this line:
        unless(/^([\w.-]) \s+ ([\w.-])$/x) {

        [download]
        and script doesn't want to process, although file are correct.
        I get this message:
        "AB0000011 LA120040212 is not a valid lineBB0000012 LB20040212 is not a valid lineErrors:"
        info file looks: 
        AB0000011   LA120040212
BB0000012   LB20040212

        [download]
        and I have these 2 files: AB0000011 and BB0000012
        I think, I did not explain the problem so well before.
        it should read the filenames in, compare them to what is in the info file and if they do not match don't process.
        for example: should NOT process with above info file and these files: AB0000015 / BB0000016
        the second problem is, that after transfer doesn't move these 2 files to the backup directory
        this line has an error:
        system("mv $new /var/save/$subfolder_name");
        my third problem is, it sends the mail or moved the files without ftp transfer

        #!/usr/bin/perl -w
use strict;
use warnings;
use File::Copy;
use Net::FTP;
use Net::Netrc;
my $linux = ".....";
my $recipient = ".....";
my $server = ".....";
my $user = ".....";
my $password = ".....";

# ftp directories
my $directory1 = "/ftp/B1";
my $directory2 = "/ftp/B2";

chdir "/var/files" or die "/var/files: $!\n";
-f "/var/files/info" or die "No info NO TRANSFER !\n";

my @ftp_locations = ($directory_1, $directory_2);
       
# open the file safely or complain it wouldn't open
open(FILE, "<", "info") or die "Failed to open info file: $!";

for(my $i = 1; $i <= 2; $i++) {
        $_ = <FILE>;

        s/\W*$//;            # remove trailing whitespace
        next if (!$_);       # skip empty lines
       
              # check that we get our match.  If not,
              # complain and move on.  Want to see two
              # filenames.
        unless(/^([\w.-]) \s+ ([\w.-])$/x) {
               print STDERR "$_ is not a valid line";
               next;
        }

        my ($old, $new) = ($1, $2);
        rename $old, $new;

              # Now that we have our filenames, grab an
              # ftp directory from the list
        unless(@ftp_locations) {
              die "Not enough specified ftp_locations for ".
                  "given number of files!";
        }

        my $destination = shift @ftp_locations;
        
# ftp transfer
        my $ftp = Net::FTP->new ($server, Timeout => 9000, Debug => 3)
+;
        $ftp or die "$server: cannot connect: $@";
        # If you don't use ~/.netrc
        $ftp->login ($user,$password)
            or die "$_: cannot logon: " . $ftp->message;

        # change remote directory for the first file
        $ftp->cwd($destination);

        # Send file to ftp server
        $ftp->put($new) or
        die "$server: cannot put $new: " . $ftp->message;
        #Quit FTP When finished
        $ftp->quit;

        # Sleep for 20 minutes before processing next file.
        sleep (20 * 60)
}

# send the mail, only when transfer completed
open(MAIL, "|/usr/sbin/sendmail -t") || die "Cant send mail. Reason: $
+!";
print MAIL "from:$linux\n";
print MAIL "to:$recipient\n";
print MAIL "subject: file transfer was successfully !\n";
print MAIL "file transfer was successfully ! \n\n";
print MAIL "Time: ", scalar localtime, "\n";
close(MAIL);

# when transfer completed, create backup subdirectory and move all fil
+e there
my @dt = localtime;
my $subfolder_name = ((((1900 + $dt[5]) * 100 + 1 + $dt[4]) * 100 + $d
+t[3]) * 100 + $dt[2]) * 100 + $dt[1];
mkdir "/var/save/$subfolder_name" or die "$subfolder_name: $!";

system("mv /var/files/info /var/save/$subfolder_name");
system("mv $new /var/save/$subfolder_name");
exit;

        [download]

        janitored by ybiC: Balanced <code> tags around codeblock

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?

www.com | www.net | www.org
Node Status?
node history
Node Type: note [id://333972]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? • Last hour • Other CB clients
Other Users?
Others browsing the Monastery: (5)
As of 2024-04-25 14:08 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found