Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Re^2: Performance Trap - Opening/Closing Files Inside a Loop

by iburrell (Chaplain)
on Dec 10, 2004 at 18:03 UTC ( [id://413925]=note: print w/replies, xml ) Need Help??


in reply to Re: Performance Trap - Opening/Closing Files Inside a Loop
in thread Performance Trap - Opening/Closing Files Inside a Loop

There is something to be said for letting the IO system and OS handle the buffering and writes. All filehandles have a write bufer which only written when it is full. One way to reduce seek times is increase the buffer size for the opened files. The advantage is that Perl decides when the write the buffers, the OS decides when the write it to disk, and both are pretty good at this.

The big advantage of caching filehandles is that the open files can hold output in the buffers until they are full. If they are continually being closed and reopened, then each line is being written individually.

What is need is some way to keep a limited number of filehandles opened to keep from hitting the limit. A LRU cache would be perfect. It see a couple of modules that implement this. Or reimplementing it would be pretty easy.

  • Comment on Re^2: Performance Trap - Opening/Closing Files Inside a Loop

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://413925]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others admiring the Monastery: (3)
As of 2024-04-25 17:15 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found