Is there a good reason why you need to read the entire file into memory at once? If you're doing the removal process when you read the files, then you might want to do so while reading the file line-by-line. By the way, I'm not sure how your regexes line up with the data you showed, for example you say "foo ref n", but the regex seems to say there might be spaces before the "foo"? Please show an SSCCE that includes short but representative sample input data and the expected output for that input.

open my $fh, '<', $filename or die "$filename: $!";
while (<$fh>) {
    next if /^(?:foo )?ref \d+\b/;
    chomp;
    # process the line, for example:
    push @lines, $_;
}
close $fh;
[download]