Re: Use more threads.

> editbin /stack:0x00100000 \yourperl\bin\perl.exe

Thank you so much for this!!! I just did it on Perl-5.12.3 and it allowed my script to use more of the available memory in the system. I couldn't pass a limit of around 80 threads, although on Perl-5.16.3 I could run over 120 threads with no problem. So I figured the versions must have been compiled with different limits (both ActivePerl), and so I found this post.

Unfortunately for now I must stick with 5.12.3, because 5.16.3 can't play with DBI connects in threads very well, producing tons of warnings about redefined subroutines. But with your "hack" I can still use 5.12.3 for a while.

Comment on Re: Use more threads.

Replies are listed 'Best First'.
Re^2: Use more threads. by BrowserUk (Patriarch) on Apr 28, 2013 at 06:53 UTC
Note also the `stack_size` use line option added to threads as a result of this thread. I quite routinely set this as low as 4096 without problems, and doing so I have had a proof of concept code with 3000 threads running in 4GB of memory. Not that there are many good uses for using more than 3 or 4 times as many threads as you have cores. So unless you are running your code on a system with 32 cores or more, seeking to run so many threads is usually a sign of naive coding. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice.	[reply] [d/l]
Re^3: Use more threads. by password (Beadle) on Apr 29, 2013 at 07:23 UTC
I agree completely with what you said about using many threads (or maybe using any threads at all) but what I've written is a proxy checker (with GUI), so most of the time my threads are waiting for a response or a timeout. My next idea is to see if I can create 100 threads any faster, so I'll tighten it up as much as it is possible, and try that stack_size option. Although my coding is a bit naive and far from professional, my only fear is that I'm inventing the wheel sometimes. he-he.	[reply]
Re^4: Use more threads. by BrowserUk (Patriarch) on Apr 29, 2013 at 08:36 UTC
t I've written is a proxy checker (with GUI), so most of the time my threads are waiting for a response or a timeout. Starting new threads for each connection is naive. And wasteful. A pool of threads and a queue of requests is more efficient (of time and resources), and actually easier to program and reason about. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice.	[reply]
Re^5: Use more threads. by password (Beadle) on Apr 30, 2013 at 00:26 UTC
Re^6: Use more threads. by BrowserUk (Patriarch) on Apr 30, 2013 at 00:43 UTC
Some notes below your chosen depth have not been shown here


Come for the quick hacks, stay for the epiphanies.
	PerlMonks