First off, your code doesn't work.
$DIFF_OLD = <DIFF_OLD>; reads the
first line of the file, not the whole file. If you want to slurp it, set $/ to undef.
Second, you're on Unix (at least, I'm assuming that based on your directory separator). So, why aren't you using the some of the commandline utilities to help you out? For example, I'm assuming that the only thing that will change is the YYYY / NNNN, plus the _1_ / _2_. So, why not create a temporary file for each one, with the crud stripped out, then using sort and diff? You can using Unix's sort to sort based on various keys, split using your own delimiter. Even with the temp files, this shouldn't take more than a second per 20k rows, and probably much less than that.
Being right, does not endow the right to be rude; politeness costs nothing.
Being unknowing, is not the same as being stupid.
Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence.
Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.
