When I vote (for or against) a political candidate, I don't feel compelled to comment on the way I have voted. The same goes here. I have commented a lot of nodes without voting for or against them and I've cast a lot of votes without comments.

The whole idea about voting being anonymous is that you cannot be held accountable for your vote. If that seems strange, go and live for a while in a dictatorship without secret voting and see if that makes you happy! Whatever the downside of not being accountable for your downvotes, a lot of worse issues rear their heads if you can be held accountable for your votes: "OK, let's all sue the voters who where in favour of the Government which made such a mess of the Hurricane Katrina relief effort."