$voteavg doesn't enter into the equation for XP earned for up-votes. But I think I'll change it to default to 1 as you suggest anyway (and then adjust existing $voteavg values).

Update: Default for new users is now 1. 1 added to the voteavg of the 20 users who had managed to get their voteavg below 0, all others had theirs set to 1 (like adding 1 to all but w/o overflowing above 1). Over 400 users had voted and raised their voteavg above 0 before I made this change.