Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??

Creating and deleting all those threads will flay the heap. It's possible that you're experiencing heap fragmentation, in which the heap has lots of free blocks but none quite large enough to create a new thread. Moving to a boss--worker model, in which you create a pool of threads at startup and then distribute work to them in round-robin fashion, should help.

Whether threading helps performance will partly depend on your storage hardware. I happen to work for a maker of high-end file servers, handling hundreds of terabytes of storage each. If you send ten concurrent requests to one of those beaties, all those NFS or Cifs latencies will be handled in parallel, and the chances are that you'll be seeking on ten separate disks at once. That means you'll get nearly ten times the performance (and the client will be the limiting factor). OTOH, if the files are stored locally and there's only one direct-attached disk that's struggling to cope, multi-threading will help a little (especially if the OS is clever enough to do elevator seeking on the disk), but don't expect wonders.

Finally, consider moving to fork, rather than Perl threads. A while ago, I wrote a Linux-based Telnet proxy -- Telnet in, Telnet out. (It does logging, connection-sharing and a few other things, but proxying is the essence of it.) At startup, or when you add new ports at runtime, it forks two threads per server port; there are typically between fifteen and forty server ports per proxy. The threads communicate with each other using socket pairs. The proxy runs for months at a time without any apparent memory leaks or performance problems. I can recommend that approach, if it fits the problem you're trying to solve.


In reply to Re: Threads in Perl: Just leaky? by MarkusLaker
in thread Threads in Perl: Just leaky? by TheShrike

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others learning in the Monastery: (5)
As of 2024-04-26 07:57 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found