Your algorithm is fundamentally flawed. Assume for a moment that the input file consisted of the following:
1
S AGT
3
S AGT
5
R AGT
7
Your algorithm would proceed as follows:
- The outer loop scans until it reads record 2;
- The inner loop then scans on until it reads record 6;
- Then you reset the pointer to the start of record 3; and exit to the outer loop;
- The outer loop scans on to record 4;
- The inner loop scans on to record 6;
- Then you reset the pointer to the start of record 5; and exit to the outer loop;
- The outer loop scans on to EOF.
Note that the above has matched both S records with the same R record.
Now imagine that your 100MB file has (say) 1000 S-records near the top of the file, and a single R-record 10 million lines further on. Then each of those 100 S-records would get matched against that single R-record; but you would have to re-read the 10 million intervening records over and over to do so.
1e3 * 10e6 == a very long time. It might well look like it had hung.
If the S-record/R-record pairs should appear sequentially: ...
S AGT
...
R AGT
...
S AGT
...
R AGT
...
Then you should not be resetting the pointer after you've found the matching R-record.
If however, the S-record/R-record pairs can be interleaved: ...
S AGT #1
...
S AGT #2
...
R AGT #1
...
R AGT #2
...
Then you would have to be maintaining two pointers: one telling you where to start looking for the next S-record; and one telling you where to start looking for the next R-record. Whilst this could be made to work, there are other, simpler approaches to this latter problem.
For example: a simple, single loop down the file looking for both types of record. When an S-record is found, you push it onto an array; when an R-record is found, you shift off the least recently found S-record from the array and do your calculations.
my @Srecs;
while( <FILE> ) {
if( /^S AGT/ ) {
push @Srecs, $_;
}
elsif( /^R AGT/ ) {
die "R-record with no corresponding S-record" unless @Srec;
my @sRecBits = split ' ', shift @Srecs;
my @rRecBits = split ' ', $_;
## Calculate stuff
}
}
if( @Srecs) {
printf "There were %d unmatched S-records\n"; scalar @Srec;
}
## Other stuff
This single pass algorithm is far more efficient, less error prone and detects mismatches.
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
|