Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Re: performance of huge array looping

by RMGir (Prior)
on Aug 14, 2002 at 11:58 UTC ( [id://190037]=note: print w/replies, xml ) Need Help??


in reply to performance of huge array looping

If you're dealing with that much data, do you REALLY need to load it all into memory in an array?

Given that you're only looking at each item once, it would make more sense to either process the data line-by-line from a text file (if it's that kind of data), or iterate over it record-by-record from a database using DBI.

You program will run MUCH faster, as a bonus, since you're not going to be swapping. This is definitely a case where loading things in memory won't be an optimization.
--
Mike

Replies are listed 'Best First'.
Re: Re: performance of huge array looping
by kommesel (Sexton) on Aug 15, 2002 at 08:26 UTC
    The source is an ASCII file. I've thought of it, and basically all I need is to load lines which begin with *| and work on them. and their size is much smaller than the file. So I'll read line by line, push into array only the lines that begin with *| (by the way, I need to use m/^\*\|/, don't I?) and loop over them, thus eliminating the need for swap.
[reply]
      Makes sense...

      Yes, you would need to escape those characters in your regex.
      --
      Mike

[reply]

In Section Seekers of Perl Wisdom

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?

www.com | www.net | www.org
Node Status?
node history
Node Type: note [id://190037]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? • Last hour • Other CB clients
Other Users?
Others surveying the Monastery: (5)
As of 2024-04-19 02:58 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found