The only way I can see to make your solution work then is to have the clients connect to a proxy on the front side which then reconnects to the second server on the back side.

That's pretty close. In normal operation, the proxy sends the request to a "random" service in a pool (where "random" can just be round robin). Sometimes you have to take a service down. From here, there are differing approaches. One way is to have the service tell the proxy that it's no longer available to service requests. Another is to have the proxy tell the service to shutdown. Yet a third way would be to have the proxy wait for the service to respond to a request to handle a request...if it doesn't respont, the proxy forwards the request to a different service. In any case, the proxy needs to have a way to either know that a given service is down or to gracefully handle it when it happens. From there, you take down the service, update what you need to and bring it back into the fold.