Yes, these are good tests. I'm not sure if you're meaning that we should try to automate them or I should just spend time doing them manually. If it's the latter, while what you say is true, it misses my point which is that it would be a lot more efficient to just have a list of the "good" (sorry to be contentious again, let me call them "key") modules.
For example: I want to do object database work. Here is my vision:
I spend 10 mins looking through the list of "good" modules/libraries (whatever you want to call them) and see that either Rose or DBIx are the currently recommended starting places.
Here is the status quo:
I spend a couple of hours searching the Internet and perl monks and CPAN and find that there's a well-used module called Class::DBI. I spend half a day reading the documenation and tinkering with it and realise it's OK. I then post a couple of questions re implementation. Another half a day later, someone points out to me that Rose and DBIx would be a better bet. I spend another 2 or 3 hours trying to ascertain the truth of this, reading their documentation to see if it's good and nothing is deprecated and reading comparison reviews before finally realising that indeed I should switch to Rose or DBIx.
I prefer my 10 minute vision to the "days of wasted time" scenario.
The only part about this is that isn't true is the fact that the "days of wasted time" scenario isn't an example - it's the reality that happened to me last week. It's also about the third time that I have been through a similar frustrating process when after different modules. Probably some readers at this point are thinking "what a dork - he should have done this or realised that" Well, I realise that I am not an expert developer but I'm not especially stupid either. CPAN is all about not wasting people's time - resuable software is about not wasting time. So, my question is this - why does the CPAN index waste so much of my time?!
The less interesting is the one re finding someone to spend her time to save your time; the more interesting is "why would you expect one evaluator/indexer's estimate of 'the right size' to fit all?"
Or is it your vision that your needs will always be satisfied by someone else's evaluations?
So why does my remark bite chase its own tail? Perhaps because the pros and cons re the OP do so likewise?