chanakya has asked for the wisdom of the Perl Monks concerning the following question:
Monks,
I have a sample script to get the byte offset of a given string from an archive file.
The script reads a large archive file around 200GB and gets the byte offset at which the string was found.
Below is the sample script. This basically reads the archive file, which is a collection of email messages.
These email messages start with string "From " and end with a blank line. The marker here is "From" for every email
1) Should get the Subject and the byte offsets of every email from the archive(offset of "From " and offset of End of email)
2) The script should be very fast to read the archive files and print out the information
Please suggest the best possible ways to achieve the same. I want the script to complete the processing in the minimum time possible.
Thanks for your time.
I have a sample script to get the byte offset of a given string from an archive file.
The script reads a large archive file around 200GB and gets the byte offset at which the string was found.
Below is the sample script. This basically reads the archive file, which is a collection of email messages.
These email messages start with string "From " and end with a blank line. The marker here is "From" for every email
The syntax of the archive file is as below.my $infil = '20090101.arch'; my $begPat = '^From '; my $INFILE = new IO::File(); open($INFILE, "< $infil") or die("$infil: $!"); my $Num = 0; my ($line, $subject); while(1) { my $startOffset = tell($INFILE); $startOffset-=scalar(length($line)) if($line =~ /$begPat/); while($line = <$INFILE>) { $subject = $line if($line =~ /^Subject:/); last; } } close($INFILE);
Basically I want a script do the followingFrom 10.1.60.40 Accept: */* Transfer-Encoding: chunked Subject: Test mail 1 Mime-Version: 1.0 Content-Type: multipart/mixed; Content-Type: text/plain xxxxxxxxxxxxx xxxxxxxxxxxxxxxx xxxxxxxxxxxx Content-Type: application/x-gzip Content-Disposition: attachment; filename="Sampledata.gz" Content-Transfer-Encoding: base64 H4sIAAAAAAAAA+y9bXOkOLbv+74+BSfiztyeuG0Xz0nWnuu4rqfumq6nXa7qntlxIjIwiW +1O From 10.60.128.140 Accept: */* Transfer-Encoding: chunked Subject: Test mail 2 Mime-Version: 1.0 Content-Type: multipart/mixed; Content-Type: text/plain xxxxxx xxxxxx xxxxxx
1) Should get the Subject and the byte offsets of every email from the archive(offset of "From " and offset of End of email)
2) The script should be very fast to read the archive files and print out the information
Please suggest the best possible ways to achieve the same. I want the script to complete the processing in the minimum time possible.
Thanks for your time.
Back to
Seekers of Perl Wisdom