The only siginficant difference was that once the client was assigned to a worker, the "server" got out of the way and let the two communicate directly.

I never considered doing that for my project, mainly because the jobs were usually (but definitely not always) finished within a fraction of a second - somewhere between 1 and 0.05 seconds is typical - and multiple clients can - and will, if the job queue is long enough - request the same job (which might be running or queued already).

Putting all the coordination in the server made the clients and the workers very simple. Having all the synchronization in a single-threaded single process made the server relatively simple too.