Some of my red flags are:

1. Poor test suite. I have an immense distrust of any module without a good test suite. If all it does is check that the module loads okay then I'm going to be suspicious.
2. Poor documentation. If I can't understand what the module does from the synopsis and description I usually can't be bothered to read further. If something still has an unchanged POD template from h2xs then I'm very unlikely to look any further.
3. When there is a module that does something similar and there are no notes on why module X should be used instead of module Y (++ points for explaining when module Y should be used instead of module X).
4. Bad module names tend to put me off: Things that claim a new top-level name space; Lowercase modules that are not pragmas; Non-descriptive names; etc.

Update:: Additional red flags

• Having a long list of fails in CPAN testers