How to pass sockets to different threads?

noslenj123 has asked for the wisdom of the Perl Monks concerning the following question:

I have been trying to work out a high connection perl server for a testing project at work. I have problems getting beyond 64 connections or about 110 (using threads).

I tried creating arrays of IO::Select objects which seemed to fall down just over 100 sockets. I tried using threads with a thread per socket which fell down around 110 sockets.

Recently I read something here that suggested that IO::Select can handle 64 sockets and that you can only have one IO::Select object per thread.

So I am trying to accept new sockets and push them onto a shared array or hash so that pre-started worker threads can lock that hash and suck the sockets out and handle them in it's own IO::Select. I have run into the dreaded "Invalid value for shared scalar" thingy. I also tried using Thread::Queue with no luck.

Does anyone have any wisdom on how to handle a large number of sockets (100-512) simultanieously? I am using ActivePerl 5.8.0 build 806 (I'll look at a newer version) but I can also run under Linux if that's preferred.

my $nuts = 'in pain';

Comment on How to pass sockets to different threads?

Replies are listed 'Best First'.
Re: How to pass sockets to different threads? by pg (Canon) on Dec 30, 2003 at 23:16 UTC
I noticed those limitations before. This thread maximum number of open ports per process on win98 has some details. To resolve your problem, just have multiple processes, as those limitations are per process. I will try 5.8.2 now, but don't think that's relevent. Any way I will try and come back with an update. Update: Same under 5.8.2.	[reply]
Re: How to pass sockets to different threads? by bl0rf (Pilgrim) on Dec 31, 2003 at 00:46 UTC
I remember reading in Apache's docs that it preforks several children, has them all try to lock onto a semaphore - the one that succeeds takes a connection and another one child takes its place as listener. If you really need Select functionality I also suggest forking and having a separate IO::Select object for each child ( as you can see I've gone fork hyper ). My site	[reply]
Re: How to pass sockets to different threads? by Mr_Person (Hermit) on Dec 31, 2003 at 15:45 UTC
In addition to the suggestions on how to fix threading problems, you might also take a look at POE. It has the advantage of running in one thread and being easier to program for than threads (you don't have to worry about multiple things happening at the exact same time). I haven't personally used it on Windows, but I know the core functionality is supposed to work without any issues. I've been using a program on Linux I wrote with POE that handles well over 100 simultaneous connections. It runs for months at a time (basically until we have to reboot the server for something else, like a kernel or hardware upgrade) and I haven't had a single problem with it. (The program is TN3270RG, if you're interested)	[reply]
Re: How to pass sockets to different threads? by LunaticLeo (Scribe) on Dec 31, 2003 at 17:19 UTC
My reading of your problem is that you need to use more than 64-110 sockets in a program. The best solution is to use an event driven framework for your program. As suggested previously, the POE (poe.perl.org) framwork is great. I use it extensively. I use it regularly to connect and interact with hundreds of sockets simultaneously. Actually, I've done upwards of two thousand concurrent sockets, but that was just to connect and disconnect quickly across several thousand hosts. My opinion on Perl Threads is that perl5-dev has not figured out how to do threads well yet and the code is shaky. Clearly, perl5-dev chose not to "share-all", and you can do "share-none" quite nicely with fork(). So they are playing games with "share-some" and "share-explicitly". Figuring out what memory to share seems to me very difficult in a generic case. You need to know that a perl file handler is a perl memory thing that contains a file descriptor, aka `$fh->fileno()`. Even if you are using perl threads, you need to pass the file descriptor then on the receiving end wrap the file descriptor in a perl file handle ala `IO::Handle->fdopen($fd)` . You might be able to pass the file descriptor from one thread to another with Thread::Queue cuz the file descriptor is just an integer and perl threads are really all in the same process. However, if you use separate processes via fork() you need to pass the file descriptor over a Unix Domain socket. See Advanced Programming in the Unix Environment by W. Richard Stevens. The chapter is "Advanced Interprocess Communication" section "Passing File Descriptors". However you are in luck. There is a CPAN module tested under Linux and Solaris for passing file descriptors called Socket::PassAccessRights. But like I said, use POE. There might be a learning curve, but it is totally worth it, you will thank me. Barring that, stop using perl thread and use processes, then do this file descriptor passing thing. Good Luck.	[reply] [d/l] [select]
Re: Re: How to pass sockets to different threads? by noslenj123 (Scribe) on Jan 03, 2004 at 05:57 UTC
Thanks for the tips gang. I have started taking a look at POE now and it looks promising. There definately is a learning curve there. I'll look around for a good tutorial if there is one. Any suggestions?	[reply]
Re: Re: Re: How to pass sockets to different threads? by rcaputo (Chaplain) on Jan 09, 2004 at 09:01 UTC
POE's web site has some tutorials. The main one shows how IO::Select relates to POE at various levels of abstraction. `-- Rocco Caputo - rcaputo@pobox.com - poe.perl.org`	[reply]


"be consistent"
	PerlMonks