Copy the contents of the user's identity.pub on the local machine, and append it to that user's authorized_keys file on the remote machine.

Then, I run an instance of sshd, as that user, and define an obscure port to connect to. You will need to create an sshd_config_<user>, and tell sshd to use that file.