I very much doubt that you'll be able to remove Spiffy from the module. Spiffy tends to become very closely entwined with the code that uses it.
I suggest that you report the problem to the author. Hopefully he'll then pass it on to the author of Spiffy.
"The first rule of Perl club is you do not talk about
-- Chip Salzenberg