comment on

Good observations and thanks! In fact I need it once to be created and then I will access it many times. so this one time processing for me takes lots of time however it's important to be accessed later easily and not very time consuming. I can have a big memory of 50 GB but still is not enough and it goes out of memory! I tried to create the hash and then tie it, it would not also possible for me or maybe I do some errors that it still goes out of memory!

my $t = tie(%hash, Tie::IxHash);
foreach my $line (@file){
    $line_count++;
    my @ngrams=produce_ngrams($line);
    foreach my $ngram (@ngrams) {
        #$t->Push(@{ $hash{$ngram} } => $line_count); 
        push(@{ $hash{$ngram} }, $line_count);
    }
}
[download]

I have also no idea if I tie the hash, later on how can I access it from my hard drive.

In reply to Re^2: a large text file into hash by perl_lover_always
in thread Reaped: a large text file into hash by NodeReaper

Are you posting in the right place? Check out Where do I post X? to know for sure.
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
Want more info? How to link or How to display code and escape characters are good places to start.


Think about Loose Coupling
	PerlMonks