Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Convert binary to ascii while sftp using Net::SSH2

by tsandras (Initiate)
on Mar 01, 2011 at 09:39 UTC ( [id://890708]=perlquestion: print w/replies, xml ) Need Help??

tsandras has asked for the wisdom of the Perl Monks concerning the following question:

Hi All,

Currently I'm utilising Net::SSH2 perl script to sftp file from Windows to Unix server. However, after transering the file, the file is having an extra character (^M) at the end of each line as the sample below.

0146669944 502126920956715 1^M 0146456870 502124950748150 1^M 0142811862 502122912649310 1^M

Is this due to the file format that's being transferred is in binary format and not in ASCII? If this is the case how do i incorporate the change of the file type from binary to ascii within the script below?

use warnings; use strict; use Net::SSH2; my $datafile = "D:/ETL/RPT/MD/IMSI_SICAP.lst"; my $ssh2 = Net::SSH2->new; $ssh2 -> connect("192.100.100.238") or die ("Error"); $ssh2 -> auth(username => 'user', password => 'aaa') or die ("Error"); $ssh2 -> blocking(1); $ssh2 -> scp_put("${datafile}IMSI_SICAP.lst","/timesten/IMSI_SICAP.lst +")or die ("Error"); $ssh2 -> disconnect();

Would really appreciate some guidance on this. Thanks.

Replies are listed 'Best First'.
Re: Convert binary to ascii while sftp using Net::SSH2
by roboticus (Chancellor) on Mar 01, 2011 at 10:56 UTC

    tsandras:

    The file is likely not being converted. On windows, the end-of-line convention is \r\n, while on unix, it's \n. So when you copy a windows text file to unix, you will normally have files with a carriage return (\r == ^M) at the end of your lines. That's the reason we have programs like dos2unix. If you're going to process the file in a perl program, though, I generally skip the dos2unix, and instead use something like:

    while (<$FH>) { # Remove *all* whitespace from the end of the line s/\s+$//; . . . the stuff you're gonna do anyway }

    ...roboticus

    When your only tool is a hammer, all problems look like your thumb.

Re: Convert binary to ascii while sftp using Net::SSH2
by salva (Canon) on Mar 01, 2011 at 09:59 UTC
    Probably, the easiest way is to just run dos2unix on the server once the file is transferred:
    ... $ssh2 -> scp_put("${datafile}IMSI_SICAP.lst","/timesten/IMSI_SICAP.lst +")or die ("Error"); $ssh2->channel->exec("dos2unix /timesten/IMSI_SICAP.lst");

    Another option is to use Net::SFTP::Foreign that supports the dos2unix operation natively:

    use Net::SFTP::Foreign; my $sftp = Net::SFTP::Foreign->new($host, backend => 'Net_SSH2', username => 'user', password => 'aaa'); $sftp->error and die "Unable to connect: " . $ssh->error; $sftp->put("${datafile}IMSI_SICAP.lst","/timesten/IMSI_SICAP.lst", con +version => 'dos2unix');
Re: Convert binary to ascii while sftp using Net::SSH2
by jethro (Monsignor) on Mar 01, 2011 at 09:54 UTC
    I checked the man page of sftp itself and not even that has options to automatically change ascii files, unlike the original ftp program. And Net::SSH2 seems unable to do that either. I would suggest that your script does the transformation before transfering the file.
Re: Convert binary to ascii while sftp using Net::SSH2
by 0xbeef (Hermit) on Mar 01, 2011 at 20:34 UTC
    The problem is that you are transferring a text file between systems that differ in how they represent an end of line character. This is precisely the reason why a program such as FTP implement the bin and ascii mode functions.

    When you transfer a file in binary mode, it remains unchanged (this mode is typically used to transfer compiled programs / BINaries).

    When dealing with end of line characters in text files that roboticus elaborated on, ascii mode would have to be used to do a transformation of the end of line character to make it appropriate to the target platform. In perl, you would need to use a s/\r\n/\n/; to do the equivalent transform when doing a Windows -> UNIX transfer.

    The reason for this madness lies in how early computer systems had to interact with teletype and typewriter devices that were unable to advance the print head to the next line in single character time (see Newline).

Re: Convert binary to ascii while sftp using Net::SSH2
by syphilis (Archbishop) on Mar 02, 2011 at 09:17 UTC
    Note that scp_put() is not really the same as sftp.

    I find that Net::SFTP::Foreign (as mentioned alteady in this thread by salva) with Net::SSH2 is really nice to use on Windows - and recommend it.

    It makes the coding a lot simpler, and the performance is excellent and trouble-free.

    Cheers,
    Rob
Re: Convert binary to ascii while sftp using Net::SSH2
by parkprimus (Sexton) on Mar 02, 2011 at 10:54 UTC
    The '^M' is a control character. It is your format type. Text should be transferred ascii, things like a .gz or .zip should be transferred binary. Good Luck

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://890708]
Approved by Corion
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others cooling their heels in the Monastery: (None)
    As of 2024-04-19 00:10 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      No recent polls found