It only fails because people have width. If we're willing to remove the concept of race from a relay race (and, in my mind, we shouldn't; if nobody's moving, that's not actually a race, IMHO), then we could also remove all non-essential physical distances -- such as human width and depth, and the length of the baton. With the baton being represented by a point and each runner represented similarly, the argument that enough people streamline the process becomes invalid.

(And I realize a relay race would be absurd to imagine in reality with a point-sized baton; however, I retort that a relay race where several thousand people pass a baton around a track could happen, but never would.)

While it might be faster, it's still not a very good argument against the metaphor. (Plus, any manager that says what you just said would immediately be thrown out of Manager Club*.)

* The first rule of Manager Club is you do not talk about Manager Club. The second rule of Manager Club is that you DO NOT TALK ABOUT MANAGER CLUB. The third rule is that the agenda for each meeting must be outlined by an Agenda Committee, consisting of at least three but no more than seven of the officers of the Manager Club, and provided to each included member at the outset of each meeting in hardcopy with a table of contents, a short summary, a summary of each item on the agenda, space for notes, and the bullet points each presenting member will be providing (for further rules on presenting members, please see Manager Club Rules 23, 24, 25, 26, 27, 28, 34, 35, 117, 118, 293, and appendices B, C, and F.)