http://qs321.pair.com?node_id=186838

TheFifthDeuce has asked for the wisdom of the Perl Monks concerning the following question:

Hello folks. I got a problem that I can't solve. I am working on a pretty cool encryption/decryption system. OK, I am reading a text file which consists of just 0's and 1's...no newlines or whitespace. The file can range in size and get to be VERY large. For this example, the file I am reading is 2,387,250 bytes in size. I need to get every byte of the file, so here are 3 different methods I tried using, and each one eats up ALOT of RAM:
sub test_loop_1{ ######## RAM used: 190 MB my(@all, $elements); @all = (); open(FILE, $file) or die; while(<FILE>){ push @all, /\d/og; } close(FILE); $elements = @all; print $elements; # Just for confirmation - prints 2387250 } sub test_loop_2{ ######## RAM used: 185 MB my(@all, @all2, $all, $elements); open(FILE, $file) or die; @all = <FILE>; close(FILE); $all = join('', @all); @all2 = split('', $all); $elements = 0; foreach(@all2){ $elements ++; } print $elements; # Just for confirmation - prints 2387250 } sub test_loop_3{ ######## RAM used: 120 MB my(@all, @all2, $all, $elements); open(FILE, $file) or die; @all = <FILE>; close(FILE); $all = join('', @all); @all2 = (); for($i = 1; $i <= length($all); $i ++){ push @all2, substr($all, $i, 1); } $elements = @all2; print $elements; # Just for confirmation - prints 2387250 }
Is there any way around this hogging of RAM, or being that the file is just so large in size, am I gonna have to deal with it?

Thanks for any advice,
David
http://www.trixmaster.com