Personally, I think it's a good argument for making numbers true objects. It doesn't matter how they're represented internally, as long as operations (addition, subtraction, etc.) all work. You can specify the format of input and output, but that doesn't change how it operates.

Such a change wouldn't fit into Perl5, but would probably work in Perl6, and is probably already done in Python and Ruby.