http://qs321.pair.com?node_id=944816


in reply to Re: Displaying/buffering huge text files
in thread Displaying/buffering huge text files

Hey! I was looking for a way to handle big files and this looks awesome. I already tried and runs so cool. However, I am kind newbie on Perl. I was trying to modify in order to get secuencial access, not random. I haven't good luck on this. I just tried to remove the int rand( length( $index )/8) and let this with length of $index. Didn't work, the seek doesn't move the pointer in the file. Could someone can give me a hint? Thank you in advance!
  • Comment on Re^2: Displaying/buffering huge text files

Replies are listed 'Best First'.
Re^3: Displaying/buffering huge text files
by BrowserUk (Patriarch) on Dec 23, 2011 at 01:37 UTC
    I was trying to modify in order to get secuencial access, not random.

    Sorry, but that makes no sense at all. You have to read the entire file sequentially once in order to build an index.

    If you want to process the file sequentially, just read it sequentially. There is nothing to be gained by using an index unless you need random access.

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?
[reply]
Re^3: Displaying/buffering huge text files
by Anonymous Monk on Dec 23, 2011 at 01:03 UTC
    Us N instead of non-portable J. 
    my $index = pack 'N', 0;
$index .= pack 'N', tell FILE while <FILE>;

for( my $line_idx = $s; $line_idx < $fileLastLine; $line_idx++ ) {
    my $line = unpack( 'N', substr($index, $line_idx*4, 4));
    seek FILE, $line, 0;
    chomp( $_ = <FILE> );
    my $line_str = $_;

    [download]
[reply]
[d/l]
      Us N instead of non-portable J.

      'J' may be non-portable, but why oh why would you build an index on one machine and the export it to another? (Hint:You wouldn't unless you are naive!)

      The reason for using 'J' in preference to 'N', is that it is 'native'. That is, whatever architecture of machine you use it on it uses the format that is natural, and therefore most efficient, for that machine.

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

      The start of some sanity?
[reply]
      In case it isn't clear, $s is for start, the line to start on. For $fileLastLine I used: 
      my $fileLastLine = `wc -l ${filename}`;
chomp $fileLastLine;
$fileLastLine = int ($fileLastLine);

      [download]
[reply]
[d/l]

In Section Seekers of Perl Wisdom