in reply to Downvoting Aging
Dunno if it already works this way, but one option might be to have a downvote 'cost' something; either it uses up one of your votes but doesn't enter the XP lottery or perhaps the reverse; a -- puts you in the negative XP lottery (really bad odds?) so that it might take one of your own XP also. In the end, who cares, you'll be a perl monk regardless of your XP but on that other hand, there's the node now in code which isn't great but does it deserve a -5 just because its not wonderful?
Update: cowards ...