Thanks for sharing your advice. I have been talking to the lead developer. Though Rex doesn't out of the box run like a daemon, he says he has a method he uses that basically turns it into one by creating a task that waits for incoming requests. I'm going to investigate that route.
I also created my own little hack that does some symlink/bash trickery that loads a different, pared-down Rexfile when local commands are run so the load time is improved. 1/2 second response isn't perfect, but I can live with it. It's much better than 2.5 seconds.