Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

IO::All speed issue

by glwtta (Hermit)
on May 11, 2004 at 15:21 UTC ( [id://352474]=perlquestion: print w/replies, xml ) Need Help??

glwtta has asked for the wisdom of the Perl Monks concerning the following question:

IO::All is one of the most convenient modules I've found in a long time. So I was a little dismayed to come across this little problem.

I have a file which is roughly half a GB (11 million lines), simply doing:

my $line; while ($line = <>){ print "$.\n" unless ($. % 10000); # to keep track of the progress }
takes about 7.5 seconds.
With
my $in = FileHandle->new($ARGV[0]); my $line; while ($line = $in->getline){ print "$.\n" unless ($. % 10000); }
takes about 23 seconds (same for IO::File); whereas this:
use IO::All; my $in = io($ARGV[0]); my $line; while ($line = $in->getline){ print "$.\n" unless ($. % 10000); }
takes over 10 minutes!

Are others seeing this sort of performance? Is this to be expected?

If it makes any difference, I am running perl 5.8.4 on RedHat Linux 7.3 (reiserfs)

Replies are listed 'Best First'.
Re: IO::All speed issue
by eserte (Deacon) on May 11, 2004 at 15:27 UTC
    You either get convenience, or speed, but never both (well, almost never...).
Re: IO::All speed issue
by duff (Parson) on May 11, 2004 at 15:34 UTC
    Magic and method dispatch conspire to make it slow. Use less magic and it will be fine. Good, fast, cheap--pick any two, but not that middle one if you're using IO::All :-)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://352474]
Approved by Limbic~Region
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others examining the Monastery: (4)
As of 2024-04-25 10:10 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found