UML and the like are for managers

I know where you're coming from - but UML is just a tool. It can be used well as well as badly. While I don't agree with everything Scott Ambler says the agile modelling approach is vaguely reasonable.

If any given piece is too big to hold in your hold, that means the piece is too big

Yup. And sometimes the best way to make it less big is to be shown a map. I know I end up scribbling stuff on whiteboards on occasion when presented with a big ball of mud. Some of them even (gasp) closely resemble UML.

Drawing and using UML diagrams doesn't mean you have to fall into BDUF RUP hell :-)