This is true, I was more thinking of the way people should vote- and the way I now do try and vote.
I think you get the personality cult problem on upvotes as well, and in an 'ideal world' it shouldn't be 'XP for ++s, no XP for --s', it should be 'XP for sensible voting, no XP for stupid voting'. By stupid voting I mean things like the personality cult (Both positive and negative), voting on the basis of personal opinion ('I disagree with you so I will downvote your opinion'), the spending of all votes in an effort to gain more personal XP, and so on.
I don't think the stopping of XP for --s will help here though. I'd hope it wouldn't make any difference, with those who'd -- sensibly not being overly dissuaded, but I can see it detracting those borderline people who still do have the XP lure but also the conscience.
Voting issues here, both good and bad, come ultimately down to the individual. I don't think trying to code round that will help any and it's going to end up being a case of taking the rough with the smooth.