I had a data file that might as well have been generated this way:

for( my $i = 0; $i < 1_000_000; $i++ ) {
    print join ' ', map { int rand 256 } 0 .. 50;
    print "\n";
}
[download]

I read my data file this way:

my @d; $#d = 1_000_000; # presize the array!
my $i = 0;
while (<>) {
    chomp;
    $d[$i++] = [ split ];
}
[download]

After that, I do a lot of arithmetic with the resulting array. I expected this to take a lot of memory and a long time, but I did not expect my memory usage to increase once the array was populated. Much to my dismay, my program kept growing and growing until I killed it to save its slimmer siblings.

I spent a good hour commenting out parts and printing out pieces before I got around to blaming the arithmetic, and it finally came clear to me. My huge pile of strings (hot off the text file!) was being converted to numbers.

So now my read loop has this in it:

    $d[$i++] = [ map { int $_ } split ];
[download]

As a result, my program uses a lot less memory, and, more importantly, it does not grow.