If you would only need to (poorly*) implement half of a module's functionality, then it means that by using it, your are also loading the other half of its functionality that you do not need. And that is deadweight.
Actually Moose and Class::MOP between them use every single function Text::Exception exports. Ack through the test suite for yourself and see how pervasive it is. In general I would agree with the sentiment you have there, but Test::Exception is (ironically) the Exception that disproves this rule.
As for 17 dependencies, I'm not sure where that number came from for Test::Exception, by my count it has 6 dependencies. One of the dependencies Sub::Uplevel is not a core Module, it in turn depends on Carp and Test::More. The rest of those dependencies are on dual life modules, sometimes that exist in a single dist (Test::Simple, Test::More, and Test::Builder are all explicitly named although last I checked they came in the same dist) and depend on versions that exist in 5.8.9's core but not in earlier core Modules. So if you are running perl 5.10.0 which has been out over a year, or 5.8.9 which has been out since December, you'll have those dependencies in core.