http://qs321.pair.com?node_id=1161544

yulivee07 has asked for the wisdom of the Perl Monks concerning the following question:

Hi fellow Perlmonks! I am currently having fun with Net::SFTP::Foreign. I am modifying a deamon which monitors a directory and automatically uploads files via sftp to a server. If there are multiple files, the deamon forks a child process for every file and transfers them in parallel.
Now I want to do some better error handling. Wenn a transfer aborts, I want to print the number of bytes which were transferred. I can quite get this to STDOUT with the "more => -v" option.
my $sftp = Net::SFTP::Foreign->new(host =>$host, user => $user, key_pa
+th => [$privkey, $pubkey] , more => '-v');

[... debug output ...]
Transferred: sent 32700048, received 60544 bytes, in 1.6 seconds
Bytes per second: sent 20997978.1, received 38877.7
[... more debug output ..]

[download]
My first approach was, to grep that output for the "Transferred" line. While this does work, that seems a bit overkill.
I also tried utilizing the callback function of sftp->put like this: 
my $BYTES = 0;

$sftp->put(               
    "testfile",           
    "testfile",           
    cleanup  => 1,        
    callback => sub {        
                          
        my ( $sftp, $data, $offset, $size ) = @_; 
        print $offset ." of ". $size ." bytes copied\n";
last_bytes($offset);                                                  
+          
    } );     

print "\nTransferred $BYTES bytes\n";
                                                                      
+                
sub last_bytes {
    my ($bytes) = @_;
    $BYTES = $bytes;
}

[download]

However this is giving me a hard time, as I would have to track a variable per child I fork.

In contrast, WWW::Curl has a method for getting the number of transferred bytes:

$curl->getinfo( CURLINFO_SIZE_UPLOAD );

[download]
I hope I am not missing something from the documentation for getting the transferred bytes from Net::SFTP::Forerign.... Any Ideas? I get the feeling I am nearly there with the callback-solution...

Kind regards,
yulivee

Replies are listed 'Best First'.
Re: Net::SFTP::Foreign Transferred Bytes
by stevieb (Canon) on Apr 26, 2016 at 14:00 UTC

    If you fork() for every file, you can use a hash to store the information. This doesn't include all of the SFTP stuff, but hopefully its enough to give you an idea you can merge in with the callback sub feature of Net::SFTP::Foreign.

    use warnings;
use strict;

use Data::Dumper;
use Parallel::ForkManager;

my @files = qw(one.txt two.txt);
my %info;

my $pm = Parallel::ForkManager->new;

FILE:
for my $file (@files){
    $pm->start and next FILE;
    $info{$file}{size} = -s $file;
    $pm->finish;
}
$pm->wait_all_children;

print Dumper \%info;

__END__
$VAR1 = {
          'two.txt' => {
                         'size' => 21
                       },
          'one.txt' => {
                         'size' => 30
                       }
        };

    [download]

    Then you can process the collected data after all the workers have finished.

[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom