Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re^4: Help Required - Problem Automating SFTP

by Kraythorne (Sexton)
on Jan 28, 2009 at 16:18 UTC ( [id://739624]=note: print w/replies, xml ) Need Help??


in reply to Re^3: Help Required - Problem Automating SFTP
in thread Help Required - Problem Automating SFTP

VERBOSE OUTPUT FROM SFTP

Connecting to *Host_IP*...
debug1: Reading configuration data /etc/ssh_config
debug2: ssh_connect: needpriv 0
debug1: Connecting to *Host_IP* *Host_IP* port *Port*.
debug1: Connection established.
debug3: Not a RSA1 key file C:\\id_cmf_mediatech.
debug2: key_type_from_name: unknown key type '-----BEGIN'
debug3: key_read: missing keytype
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug2: key_type_from_name: unknown key type '-----END'
debug3: key_read: missing keytype
debug1: identity file C:\\id_cmf_mediatech type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.1
debug1: match: OpenSSH_5.1 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_3.8.1p1
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr
debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib
debug2: kex_parse_kexinit: none,zlib
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour128,arcfour256,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr
debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour128,arcfour256,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: mac_init: found hmac-md5
debug1: kex: server->client aes128-cbc hmac-md5 none
debug2: mac_init: found hmac-md5
debug1: kex: client->server aes128-cbc hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug2: dh_gen_key: priv key bits set: 122/256
debug2: bits set: 517/1024
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug3: check_host_in_hostfile: filename /home/.ssh/known_hosts
debug3: check_host_in_hostfile: match line 1
debug1: Host '*Host_IP*' is known and matches the RSA host key.
debug1: Found key in /home/.ssh/known_hosts:1
debug2: bits set: 520/1024
debug1: ssh_rsa_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: C:\\id_cmf_mediatech (0x0)
debug1: Authentications that can continue: publickey,gssapi-with-mic,password
debug3: start over, passed a different list publickey,gssapi-with-mic,password
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: C:\\id_cmf_mediatech
debug1: read PEM private key done: type RSA
debug3: sign_and_send_pubkey
debug2: we sent a publickey packet, wait for reply
debug1: Authentication succeeded (publickey).
debug2: fd 4 setting O_NONBLOCK
debug2: fd 5 setting O_NONBLOCK
debug1: channel 0: new client-session
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug1: Entering interactive session.
debug2: callback start
debug2: ssh_session2_setup: id 0
debug1: Sending subsystem: sftp-server
debug2: channel 0: request subsystem
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
Request for subsystem 'sftp-server' failed on channel 0
Connection closed


If I don't put in the '-s sftp-server' switch the debug ends:


debug1: Entering interactive session.
debug2: callback start
debug2: ssh_session2_setup: id 0
debug1: Sending subsystem: sftp
debug2: channel 0: request subsystem
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
Request for subsystem 'sftp' failed on channel 0
Connection closed

Any Thoughts?

Replies are listed 'Best First'.
Re^5: Help Required - Problem Automating SFTP
by Kraythorne (Sexton) on Jan 28, 2009 at 16:59 UTC
    scp generates:

    exec: cp: Not Found

    Is that cos I'm using windows - searching the web makes it look like these are unix commands that I don't have installed?
      both scp and sftp should be part of the OpenSSH package. PuTTY has pscp and psftp.
Re^5: Help Required - Problem Automating SFTP
by Kraythorne (Sexton) on Jan 30, 2009 at 11:28 UTC
    Ok, the saga continues...

    This is the information I have back from the client:

    'We restrict external logins to sftp connections only and chroot the users into their home directories, i.e. you don’t see “/home/mediatech” when you log in, you just see “/”. Now the problem arises when you are chrooted, in this case the command line sftp won’t connect, when you are not chrooted command line sftp works fine. Unfortunately this server is shared with other external customers so it is considered good practice to have all accounts chrooted.

    I can’t find anything about why this is the case or how to fix it but I have found that psftp (command line sftp that comes with Putty) works. As far as I can see psftp is a generic command line sftp client that does everything Openssh sftp does. I assume that it would integrate into Perl scripts exactly the same as sftp would.'

    So my question is - Can I get Net::SFTP::Foreign to work with Putty psftp? If not is there another module I could use?

    Thanks
      I see, probably what is happening is that they are expecting you to use the old SSH version 1 approach for launching the SFTP server on the remote side.

      Just to test that this is the case, what happens if you run these commands...?

      plink *remoteHost* -p *remotePort* -v sftp-server plink *remoteHost* -p *remotePort* -v /usr/lib/sftp-server
      Does anyone of them hangs (that means that the sftp-server is listening) instead of exiting with an error?

      Once you find how to launch the remote sftp-server, tell Net::SFTP::Foreign to use the same command with the open2_cmd constructor option. For instance:

      my $sftp = Net::SFTP::Foreign->new(open2_cmd => "plink $host -p $port +/usr/lib/sftp-server"); $sftp->error and die "SFTP connection failed: " . $sftp->error;

      update: and BTW, I think the root of the problem is an incomplete chroot'ed configuration for sshd in the remote side. Probably, they have not configured sshd to use internal-sftp server for the SFTP subsystem. Ask them to check that the sshd_config file contains the following line:

      Subsystem sftp internal-sftp
      (a how-to: http://www.debian-administration.org/articles/590)
      I can get 'plink' to connect and hang up to this point:

      Offered public key
      Offer of public key accepted
      Authenticating with public key "rsa-key-20081029"
      Access granted
      Opened channel for session
      Allocated pty (ospeed 38400bps, ispeed 38400bps
      Started a shell/command

      ..and when I break out of it I get:

      Sent EOF message
      ^C

      So it looks like that works

      However, when I put the same syntax into Net::SFTP::Foreign, after the shell starts, I get:

      Server exited on signal "INT"
      Disconnected: All channels closed

      Are we getting closer?
        Post here the full plink command you are using.

        Also, set...

        $Net::SFTP::Foreign::debug = -1;
        in your script, before the constructor call, and post here the output you get.
      Command Line 'Plink':

      C:\Program Files\PuTTY>plink *mycompany@host* -P *port* -v -i c:/id_cmf_mediatech.ppk sftp-server
      Looking up host "*host_IP*"
      Connecting to *host_IP* port *port*
      Server version: SSH-2.0-OpenSSH_5.1
      We claim version: SSH-2.0-PuTTY_Release_0.60
      Using SSH protocol version 2
      Doing Diffie-Hellman group exchange
      Doing Diffie-Hellman key exchange with hash SHA-256
      Host key fingerprint is:
      ssh-rsa 2048 *FINGERPRINT*
      Initialised AES-256 SDCTR client->server encryption
      Initialised HMAC-SHA1 client->server MAC algorithm
      Initialised AES-256 SDCTR server->client encryption
      Initialised HMAC-SHA1 server->client MAC algorithm
      Reading private key file "c:/id_cmf_mediatech.ppk"
      Using username "mediatech".
      Offered public key
      Offer of public key accepted
      Authenticating with public key "rsa-key-20081029"
      Access granted
      Opened channel for session
      Started a shell/command

      *HANGS HERE UNTIL I BREAK*

      Sent EOF message
      ^C


      Net::SFTP::Foreign code:

      #228 ssh cmd: "c:/program files/putty/plink" *mycompany*@*host* -P *port* -v -i c:\id_cmf_mediatech.ppk sftp-server
      #228 queueing msg len: 5, code:1, id:3 ... 1 00 00 00 05 01 00 00 00 03 | .........
      #228 waiting for message... 1
      Looking up host "*host_IP*"
      Connecting to *host_IP* port *port*
      Server version: SSH-2.0-OpenSSH_5.1
      We claim version: SSH-2.0-PuTTY_Release_0.60
      Using SSH protocol version 2
      Doing Diffie-Hellman group exchange
      Doing Diffie-Hellman key exchange with hash SHA-256
      Host key fingerprint is:
      ssh-rsa 2048 *fingerprint*
      Initialised AES-256 SDCTR client->server encryption
      Initialised HMAC-SHA1 client->server MAC algorithm
      Initialised AES-256 SDCTR server->client encryption
      Initialised HMAC-SHA1 server->client MAC algorithm
      Reading private key file "c:\id_cmf_mediatech.ppk"
      Using username "mediatech".
      Offered public key
      Offer of public key accepted
      Authenticating with public key "rsa-key-20081029"
      Access granted
      Opened channel for session
      Started a shell/command
      #228 got it!, len:95, code:2, id:3, status: - 02 00 00 00 03 00 00 00 18 70 6f 73 69 78 2d 72 65 6e 61 6d 65 40 6f 70 65 6e 73 73 68 2e 63 6f | .........posix-rename@openssh.co 6d 00 00 00 01 31 00 00 00 13 73 74 61 74 76 66 73 40 6f 70 65 6e 73 73 68 2e 63 6f 6d 00 00 00 | m....1....statvfs@openssh.com... 01 32 00 00 00 14 66 73 74 61 74 76 66 73 40 6f 70 65 6e 73 73 68 2e 63 6f 6d 00 00 00 01 32 | .2....fstatvfs@openssh.com....2
      #228 Net::SFTP::Foreign=HASH(0x226e20)->DESTROY called (current pid: 228, discon nect_by_pid: )
      #228 Net::SFTP::Foreign=HASH(0x226e20)->disconnect called (ssh pid: 3028)
      #228 _conn_lost
      #228 _set_status code: 7, str: Connection lost
      #228 _set_err code: 37, str: Connection to remote server is broken
        so, it seems to be working!

        Are you doing anything else in your script, besides calling the constructor?

        Which version of Net::SFTP::Foreign are you using? I am missing some information in the debugging output...

      Version 1.46

      Yep, just calling the constructor at the mo, I need to get into how the module works a bit more now I know it's working!

      ...a nice bit of code. Thanks!!

      By the way, is there a 'move' method in Net::SFTP::Foreign module so that you can move files from one directory on the FTP site to another?
Re^5: Help Required - Problem Automating SFTP
by salva (Canon) on Jan 28, 2009 at 16:32 UTC
    That means that SFTP has been forbidden in the remote host configuration.

    But as you have said that FileZilla works, maybe they are allowing scp, so just try it!

Re^5: Help Required - Problem Automating SFTP
by Kraythorne (Sexton) on Jan 28, 2009 at 18:03 UTC
    Using Putty pscp:

    *Host_IP*:c:/test.txt
    Looking up host "*host_IP*"
    Connecting to *host_IP port *port*
    Server version: SSH-2.0-OpenSSH_5.1
    We claim version: SSH-2.0-PuTTY_Release_0.60
    Using SSH protocol version 2
    Doing Diffie-Hellman group exchange
    Doing Diffie-Hellman key exchange with hash SHA-256
    Host key fingerprint is:
    ssh-rsa 2048 *fingerprint*
    Initialised AES-256 SDCTR client->server encryption
    Initialised HMAC-SHA1 client->server MAC algorithm
    Initialised AES-256 SDCTR server->client encryption
    Initialised HMAC-SHA1 server->client MAC algorithm
    Reading private key file "c:/id_cmf_mediatech.ppk"
    Using username "*mycompany*".
    Offered public key
    Offer of public key accepted
    Authenticating with public key "rsa-key-20081029"
    Access granted
    Opened channel for session
    Primary command failed; attempting fallback
    Started a shell/command
    Using SCP1
    Connected to *host_IP*

    Ok, this looks like it can get me a connection - how can I use this with Net::SFTP::Foreign to automate file downloads and moving files on the clients FTP site?
      Net::SFTP::Foreign requires SFTP to be working (or allowed) on the remote server.

      If all you are allowed to do is to scp there, then you can try Net::SSH2, Net::SSH::Perl or Net::SCP, though, as you are using keys for authentication, probably just calling the scp (or pscp) program from Perl would be the simplest solution.

      Also, take into account that scp is very limiting and will allow you to transfer files from one side to the other only. For instance, you will not be able to move files around on the server (update: oh, well, unless they allow you to run any remote command and not just scp in client mode).

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://739624]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chilling in the Monastery: (6)
As of 2024-03-28 11:44 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found