You seem to be confusing reputation and XP. Nodes gain/lose reputation, monks gain/lose XP.
Reputation is a measure of the degree to which other monks esteem a node and is an indication of its worth to the site. XP is an index of the degree of participation in PerlMonks which may be dominated by XP earned through node contributions, or may be dominated by XP earned through showing up every day and voting.
Votes cast for nodes influence XP to the extent that up votes may increase the author's XP and down votes may decrease the author's XP so to that extent a node's reputation adjusts an authors XP.
True laziness is hard work