Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

Re: Re: Efficient processing of large directory

by Elliott (Pilgrim)
on Oct 02, 2003 at 16:47 UTC ( [id://295970]=note: print w/replies, xml ) Need Help??


in reply to Re: Efficient processing of large directory
in thread Efficient processing of large directory

Thanks for the tip - but why? (Most of all I want it to work - but I also want to understand)

Replies are listed 'Best First'.
Re: Re: Re: Efficient processing of large directory
by dragonchild (Archbishop) on Oct 02, 2003 at 16:54 UTC
    It has to do with how foreach (and for, which is an exact synonym) works. foreach will construct the entire list, then iterate through it. This can be very memory-intensive, which will slow the processing speed (due to cache misses and virtual memory issues.) A nearly exact rewrite of foreach in terms of while would look something like:
    foreach my $n (<*.*>) { # do stuff } ---- my @list = <*.*>; my $i = 0; while ($i <= $#list) { my $n = $list[$i]; # do stuff } continue { $i++; }

    ------
    We are the carpenters and bricklayers of the Information Age.

    The idea is a little like C++ templates, except not quite so brain-meltingly complicated. -- TheDamian, Exegesis 6

    Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.

      Er, why on earth do you tell him to use while and then to use while to do the exact same thing the for loop had been doing previously? Your method is still going to need to build the 17,000 element list and iterate over it, it just uses a more explicit form. A rewrite which gets around this would be simply:
      while(my $x = <*.*>) { do_stuff($x); }
      This will only read a single file at a time and has no need to create huge lists.
        Because I'm not answering the original question. I'm answering the question "What's the difference between for and while?". In that context (which was obvious from the followup question), does my post make sense?

        ------
        We are the carpenters and bricklayers of the Information Age.

        The idea is a little like C++ templates, except not quite so brain-meltingly complicated. -- TheDamian, Exegesis 6

        Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://295970]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others admiring the Monastery: (6)
As of 2024-03-28 13:45 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found