From a purely UI point of view, it's a pain to have to scroll to the bottom of the page to click the 'Vote' button, especially if there are a lot of replies.

So... don't. The voting buttons are one large form on each page. In Firefox, at least, clicking a voting radio button and pressing Enter submits the form. Even without that, though, I've never found pressing Ctrl-End in order to get to the bottom of the current page to be "a pain".