http://qs321.pair.com?node_id=196996


in reply to scp cronjob

You are currently making a separate system call to run scp for each file that needs to be copied. This is inefficient because of the overhead of starting up a new process and the overhead of initiating a new connection to the remote machine. If you are running this every minute, and if the script takes more than a minute to run, you will also have multiple instances of the script running simultaneously.

Rsync, as mentioned above, would be a much better option. If you are primarily copying html files, you may want to also turn on compression (-z option).

/usr/bin/rsync -az -e ssh source_directory/ \ user@backupserver:destination_directory -e ssh tells rsync to use ssh for transport -a is archive mode, which gives recursion and preserves permissions an +d such. -ar would be harmlessly redundant. -v will show filenames during the transfer.
The trailing slash on the source directory is important. See man rsync for more info.