Net::SFTP::Foreign frequent connection server stall

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Net::SFTP::Foreign frequent connection server stall by syphilis (Archbishop) on Jun 19, 2012 at 23:57 UTC
Hi, I use Net::SFTP::Foreign::Backend::Net_SSH2 which (as you may have guessed) uses Net::SSH2 for the sftp transfers. I find it very good, though I don't think I've ever had to transfer anything bigger than about 10 megabytes. I used to transfer using scp via Net::SSH2, but sometimes experienced incomplete transfers. However, I think that problem (in Net::SSH2) has since been fixed. Cheers, Rob	[reply]
Re: Net::SFTP::Foreign frequent connection server stall by zentara (Archbishop) on Jun 19, 2012 at 17:54 UTC
You didn't mention trying Net::SSH2, it might be worth a try. `#!/usr/bin/perl use warnings; use strict; use Net::SSH2; my $ssh2 = Net::SSH2->new(); $ssh2->connect('localhost') or die "Unable to connect Host $@ \n"; $ssh2->auth_password('z','ztester') or die "Unable to login $@ \n"; #get a large file like a 100Meg wav file my $dir = '/home/whoever'; my $remote1 = $dir.'/1.wav'; use IO::File; my $local1 = IO::File->new("> 2.wav"); #it needs a blessed reference $ssh2->scp_get($remote1, $local1); __END__` [download] I'm not really a human, but I play one on earth. Old Perl Programmer Haiku ................... flash japh	[reply] [d/l]
Re^2: Net::SFTP::Foreign frequent connection server stall by Anonymous Monk on Jun 20, 2012 at 15:18 UTC
Hello Zentara, I have not tried using Net::SSH2, but interested in learning what considerations one should go through when deciding to use SSH2 vs. SFTP. (as I am still very new to using Perl and open source... I find very hard to navigate)	[reply]
Re^3: Net::SFTP::Foreign frequent connection server stall by zentara (Archbishop) on Jun 20, 2012 at 18:44 UTC
Hi, Net::SSH2 was designed to replace the old sftp and ssh module. You can ask the SSH experts on the maillist at ssh-sftp-perl maillist for all the details. The switch started about 10 years ago, and at the time the most important reason was to eliminate the numerous math library dependencies , like Math::Pari which was an installation hassle, and often resulted in sftp dropping back to very slow pure Perl math processing. It also brought all of the various ssh utilities, like sftp, scp, ssh under one module. I've found it easy to install and use, as it only requires libssh2 to be installed with it's development headers. I'm not really a human, but I play one on earth. Old Perl Programmer Haiku ................... flash japh	[reply]
Re^4: Net::SFTP::Foreign frequent connection server stall by salva (Canon) on Jun 21, 2012 at 07:55 UTC
Re^5: Net::SFTP::Foreign frequent connection server stall by zentara (Archbishop) on Jun 21, 2012 at 09:00 UTC
Some notes below your chosen depth have not been shown here
Re: Net::SFTP::Foreign frequent connection server stall by salva (Canon) on Jun 20, 2012 at 05:46 UTC
Hi, I am Net::SFTP::Foreign author. Set `$Net::SFTP::Foreign::debug = ~(8\|16\|1024\|2048);` and send me the output by email. Also, run your script with `truss` and send me the last hundreds of lines. BTW, which SSH client are you using, the one from Solaris or OpenSSH one and which version? I would try changing it. Net::SFTP::Foreign supports timeouts and resuming transfers. You can combine these features as a workaround until we find what is going wrong.	[reply] [d/l] [select]
Re^2: Net::SFTP::Foreign frequent connection server stall by Anonymous Monk on Jun 20, 2012 at 16:46 UTC
Hi Salva, Getting a dump from a failed run will be tricky. As it seems to only fail the first run of the day. I will try to get it. "ssh -V" shows: Sun_SSH_1.1.1, SSH protocols 1.5/2.0, OpenSSL I'm not sure what you mean by changing it or that I have any power to(?) PS. Just want to say kudos because I can see you actively answer user questions.	[reply]
Re^3: Net::SFTP::Foreign frequent connection server stall by salva (Canon) on Jun 20, 2012 at 18:29 UTC
As it seems to only fail the first run of the day Then, you must investigate what happens at that time of the day that is different than when it runs properly. It may be a problem related to the network, some firewall being rebooted every day at some time early on the morning or whatever. If you can run tcpdump on the machine, you may be able to discover what is happening with the connection at the TCP level. I'm not sure what you mean by changing it or that I have any power to(?) If you can, install OpenSSH on the machine and tell Net::SFTP::Foreign to use it instead of the SSH client from Solaris, just to see if it makes a difference.	[reply]
Re^2: Net::SFTP::Foreign frequent connection server stall by rider754 (Novice) on Jun 21, 2012 at 16:44 UTC
Hello, not sure how to email you (can you PM me your email address?). Thanks!	[reply]
Re^3: Net::SFTP::Foreign frequent connection server stall by salva (Canon) on Jun 21, 2012 at 19:22 UTC
CPAN authors can usually be reached via CPANID@cpan.org. In my case, salva@cpan.org.	[reply]
Re: Net::SFTP::Foreign frequent connection server stall by Illuminatus (Curate) on Jun 19, 2012 at 21:12 UTC
I would use scp over sftp as zentara suggests. If, however, 'future policies' means only sftp, then you'll have to figure out what the problem is: A code fragment showing what you're doing might be helpful You don't list the SSH command/version that you're using The latest version of Net::Sftp::Foreign is 1.74, and it looks like it should work with 5.8.8 Are both client and server Solaris 5.9? I'm not sure what you're referring to by 'SFTP v 1.35'. Your client is your perl program. Is this the server version? There are lots of different servers around. Most also create log files Can you use the sftp client on the command line to reliably transfer the file? tcpdump/wireshark of a hanging transfer can at least tell you what's crapping out at the tcp level fnord	[reply]
Re^2: Net::SFTP::Foreign frequent connection server stall by Anonymous Monk on Jun 20, 2012 at 15:45 UTC
fnord / Illuminatus, 2. "ssh -V" shows: Sun_SSH_1.1.1, SSH protocols 1.5/2.0, OpenSSL 3. Good to know!! 4. DataONTAP (NetApp OS) 5. I initially looked at the version of the SFTP.pm, so I just put it in here. Prolly extra info... 6. Yes, and one thing I forgot to mention is if I re-run the job, the sftp will go through. It also worked completely fine when ran daily on our test server. But also the same symptom on our dev box... (almost seems like it's only the first connection of the day, or maybe within 24hrs of last connection but it sounds ridiculous and I have no proof...) 7. I wouldn't know how to do that, or if I can. Since, I'm in a corporation and peons don't have access to anything, much less production. I'm just not sure. I mean, for us, in order to install a new Perl module it's 2 layers removed and we HOPE we get a knowledgeable SA... my $ssherr = File::Temp->new or die "File::Temp->new failed"; open my $stderr_save, '>&STDERR' or die "unable to dump STDERR"; open STDERR, '>&'.fileno($ssherr); my $sftp = Net::SFTP::Foreign->new( host => $properties{RemoteHostName}, user => $rmtUser, password => $rmtPass , timeout => 10, more=> '-v' , queue_size => 16); open STDERR, '>&'.fileno($stderr_save); if ($sftp->error) { print "sftp error: ".$sftp->error."\n"; seek($ssherr, 0, 0); while (<$ssherr>) { print "captured stderr: $_"; } $endprogram = 1; } close $ssherr; if ($endprogram) { exit($endprogram); } if (!$sftp->setcwd($properties{RemoteDirPath})) { print "File Transfer - unable to change directory: " . $sftp->error; exit(1); } if (!$sftp->get($properties{RemoteFileName}, "$fileTo")) { print "File Transfer - get failed: " . $sftp->error; exit(1); } [download]	[reply] [d/l]
Re: Net::SFTP::Foreign frequent connection server stall by Anonymous Monk on Jun 20, 2012 at 16:54 UTC
I just want to say thanks to all who've replied, for taking your time. Great sense of community!	[reply]
Re: Net::SFTP::Foreign frequent connection server stall by Anonymous Monk on Jun 20, 2012 at 18:05 UTC
One update: This morning's run (using FTP) also failed. Error Msg: Timeout at /opt/perl-5.8.8/lib/Net/FTP.pm line 491 which is... last unless $len = $data->read($buf, $blksize); Maybe it's a clue?	[reply]
Re^2: Net::SFTP::Foreign frequent connection server stall by salva (Canon) on Jun 21, 2012 at 09:12 UTC
Maybe it's a clue? Yes, investigate what happens in your network every day at that time!!!	[reply]
Re^3: Net::SFTP::Foreign frequent connection server stall by littlelion (Novice) on Oct 31, 2015 at 07:57 UTC
Good evening Salva, I am seeing the same problem. Every 50GB or so the SFTP connection will drop, retrys are not successful. scp an sftp work correctly, even in parallel to this activity so it looks like it is a Net::Foreign issue. Environment: `matt@aslan:185% cat /etc/redhat-release Fedora release 22 (Twenty Two) matt@aslan:186% perl -v This is perl 5, version 20, subversion 3 (v5.20.3) built for x86_64-li +nux-thread-multi (with 16 registered patches, see perl -V for more detail) ... matt@aslan:187% perl_module_version Net::SFTP::Foreign Net::SFTP::Foreign : 1.75 matt@aslan:188%` [download] I see no relevant detail in the ssh logs during the failure: ... debug3: Ignored env DISPLAY debug3: Ignored env SGI_ABI debug1: Sending subsystem: sftp debug2: channel 0: request subsystem confirm 1 debug2: callback done debug2: channel 0: open confirm rwindow 0 rmax 32768 debug2: channel 0: rcvd adjust 2097152 debug2: channel_input_status_confirm: type 99 id 0 debug2: subsystem request accepted on channel 0 # got it!, len:95, code:2, id:-, status: - Error: Connection lost (Connection to remote server is broke +n) Resuming transfer (retry 1) # queueing msg len: 71, code:17, id:1973 ... [1] # waiting for message... [1] # queueing msg len: 83, code:3, id:1974 ... [2] # waiting for message... [2] debug2: channel 0: read<=0 rfd 4 len 0 debug2: channel 0: read failed debug2: channel 0: close_read debug2: channel 0: input open -> drain ... [download] The error and resuming lines above was generated by this perl, the rest was from -vvv: `emit(" Error: " . $connect->status . ' (' . $connect->error . ')',` [download] emit() is just a wrapper around print STDERR; return undef; The code whole paragraph is: for (my $pass=0; $pass < $_max_retries; $pass++) { if ($connect->put($local, $remote, %$options)) { my $stat = $connect->stat($remote); $size = $stat->size; main::add2Log(" Finished $size of $lsize bytes", 'i +nfo') if ($main::debug >= 1 and $pass); $main::progressBar = 100; return 1; } main::append2Log("\b"x$_fll, ' 'x$_fll, "\b"x$_fll) unless (main::is_graphical()); emit(" Error: " . $connect->status . ' (' . $connect->error . ')', " Resuming transfer (retry " . ($pass+1) . ')') +; my $connect = sshConnect($url, $user, $password, 'sftp', undef +, 1, 1); return undef unless ($connect); } [download] The whole run sans debug looks like: Uploading marilyn/photos/1108BlueRidge/Original/CRW_2030.tif + Error: Connection lost (Connection to remote server is broke +n) Resuming transfer (retry 1) Error: Connection lost (Connection to remote server is broke +n) Resuming transfer (retry 2) Error: Connection lost (Connection to remote server is broke +n) Resuming transfer (retry 3) Error: Connection lost (Connection to remote server is broke +n) Resuming transfer (retry 4) Retries exceeded - failed to send '/Website/www/html/marilyn/p +hotos/1108BlueRidge/Original/CRW_2030.tif' Uploading marilyn/photos/1108BlueRidge/Original/CRW_2031.crw [download] What detail can I collect to help diagnose? Matt	[reply] [d/l] [select]


"be consistent"
	PerlMonks