For files on the same server I would use File::DirCompare. Perhaps IPC::PerlSSH will do the trick across servers or you could use regular ol' SSH and gather listings and checksums. Beyond that I'm not sure, other than your approach: transferring the files to one server.

Update: Or mount a share (if feasible).