In addition to the tests above mentioned by demerphq and adrianh, I also look at the following criteria :

1. Synopsis: Does the module have a good, concise synopsis? Does the code work/compile? Is the synopsis code too long?
2. Author: Does the author have a history of modules? Has the author a history of coming up with modules, releasing them (with much hype) and then abandoning them half-cooked? (I accuse some super-stars of this)
3. Prior art / existing wheels: Does the author reference and discuss similar modules? Are the differences mentioned? Are the reasons for reinventing the wheel mentioned?
4. Binaries / cross-platform: As I develop under three platforms (Win32, Linux, Solaris), I'm interested in whether there is a pure Perl version available, as this means a simple fallback solution in case compilation fails. Did the author cater for platform idiosyncrasies? Does the author care for other platforms?
5. Bug reporting: Does the author mention a preferred way of bug reporting? Does the author use/know RT?

perl -MHTTP::Daemon -MHTTP::Response -MLWP::Simple -e ' ;    # The  
$d = new HTTP::Daemon and fork and getprint $d->url and exit;#spider
($c = $d->accept())->get_request(); $c->send_response( new   #in the
HTTP::Response(200,$_,$_,qq(Just another Perl hacker\n))); ' #  web
[download]