1. Webservers should be implemented with load balancing or similar which allows for different versions to be driven simultaneously. Java world has Tomcat which supports sessions that are "linked" to different versions of the same software.

2. It makes the software more complicated, especially when running several threads. And therefore requires more developer discipline, and discipline must be maintained when developers and their management changes.

That sums it up nicely. There could be a niche where this kind of dynamic configuration would be useful and worth the extra price - but not for the everyday biz web application. :-)