Thank you for comprehensive comment, I'm jumping right in to try your suggestions!

First, I see no reason to avoid strict and warnings. The only overhead it adds is the clearing of the variables in each loop pass. If that's an issue, you can avoid the clearing by declaring the vars outside the loops.

Adding use strict; and declaring all vars right at the top with our increases my runtime by 5%.

I didn't know about strict clearing the variables, are you sure this is the reason?