"I'd like a downvote to give the voter a chance of losing one XP."

I'd like to propose a somewhat less drastic change. I think it shouldn't be possible for the voter to gain XP by downvoting. As it is right now, I could spend all my votes downvoting and still climb in 'experience' as a result.