I also had the same thought a while back, brother
Boogman, but have since changed my mind.
If a Monk Gone BadTM wanted to stalk someone, it could be done using "neutral" votes, such as you describe.
It's sad, but the Monastery has grown enough that such things do need to be considered. That, or I forgot to take my meds today and am having paranoid delusions. :^P (just joking about meds, but not about the chance that I'm way off base in my thinking on this.)
Update : a number of fine Monks have pointed out why this is highly unlikely, and I have no reason to doubt them. Not the first ybiC-brain-hiccup, certainly not the last.   ;^)
cheers,
ybiC