renodino has done some testing with home built versions of Perl for Linux...

Clarification: I didn't build a new Perl on Linux. Using the stock Perl 5.8.6 in FC4, I ran some tests. Linux died at 289 threads. I also ran tests on Solaris 10 (which dies at ~1900 threads, and starts thrashing the swapper around 1300 threads), and OS X 10.3.9, which dies around 450 threads.

Perhaps as importantly, I found a link that sheds a bit more light on the subject.

My current approach (which I hope to build/test today) is to add a couple new APIs to threads: set_stack_size() and get_stack_size(). The added code is pretty simple, though it may not be applicable to the root thread (the various editbin/setrlimit/ulimit solutions may address that issue).

Its important to point out that this issue isn't just about using more threads (tho thats my personal requirement); given the huge default stack size on Win32 and Linux, one of the biggest complaints of threaded Perl apps - its voracious memory appetite - may be addressed by just trimming the stack reserve down to a reasonable/minimal number.

Update:

After adding the set/get_stack_size() methods and applying the associated changes to the CreateThread()/pthread_attr_setstacksize() calls, and then calling set_stack_size(65536), I can crank out 1200 threads on Win32 (tho theres definitely some swapping kicking in at around 900 threads).

Likewise, on Linux FC4, I can get 1000 threads on a fairly small machine (an old 1GHz laptop w/512 meg), tho it starts thrashing at around 1000 threads. And the vsz report from ps shows a vast reduction in memory usage. (Since I can't get more than 120 threads using the original threads on Win32, I can't really make a useful memory usage comparison)

Note that in both cases, I'm using the stock perl 5.8.6 wo/ any ulimit'ing or editbin'ing.

I'm going to try it on OS X and Solaris and see what shakes out.

FWIW: my method for doing this was to copy the threads and threads::shared source directories into their own, and rename everything to "morethreads" package root. The module tests don't seem to pass w/ flying colors, but it may be related to using the unofficial threads::shared 0.95 against perl 5.8.6.

Update 2:

After testing on OS X 10.3.9 and Solaris 10, they both seem a bit less sensitive to the stack setting. Both reported ulimit -s == 8192 (ie, 8Meg).

When I ran a comparison test on OS X between stock threads, and my hacked morethreads, the overal performance was about the same, tho ps -o vsz reported about half as much memory being used when I set_stack_size(65536). So I'm assuming something in either the perl build, or the OS is throttling the per-thread stack size.

On Solaris, the test showed an even closer vsz between stock and hacked threads. Stock was always about 15-20 megs higher than hacked, so I'm assuming theres a build or OS limit there as well.

Following up on my Linux tests, ulimit -s reported 10240. The vsz differences were dramatic: at 200 threads, the stock version reported nearly 2Gig, while the hacked version reported around 125Meg.