I'd suggest that you read up on
End-To-End Arguments in System Design and then a few essays by
Andrew Odlyzko. The point which nobody ever comes out and says, but is definitely true, is that the free-est tent is always the one that collects the most people. Once you get critical mass there, you definitely see a
Worse is Better phenomena, where the structure that is clearly "worse" does better in the long-run through positive network effects.
You might not draw the same conclusions that I do from those sources. I didn't until I spent a long time stewing with the ideas. And I don't have time and energy to really explain myself right now. However my conclusion after some thought is that the simplicity of CPAN that leads to your complaints also is what causes it to succeed. As much as we might like more form, imposing form imposes barriers to entry which would have kept CPAN from taking off.