Hi Jonny,
I have been working on exactly this problem lately and got a massive amount of help from BrowserUk++. Please see multithreaded tcp listener with IO::Socket for the discussion and some excellent code Re^7: multithreaded tcp listener with IO::Socket.
The main trick is in passing the file handles to a pool of threads that handle the connections and a little bit of trickery to stop them going out of scope too soon.
For my problem I have an architecture like this ...
1 main/listener Thread
listens for connections
passes socket via queue
cleans up old sockets
Pool of receiver threads
check data and send OK or ERROR to client,
instruction to close socket back to main thread
and puts ref to data on another queue for...
Router thread
Pulls refs off the queue and decides which of several
handler threads will get it (may be one of more).
spawns handler if it is not yet running then puts
ref on queue for that handler.
Handler threads
do thing like logging, hearbeat monitoring
forwarding exceptions to a console etc etc.
Self Heartbeat Thread
Send periodic heartbeats through its own
mechanism to another monitor console so
we know it is still working
Update
I am also using this for performance monitoring, stats gathering etc. Not sure how varied your data sources are, we have everything from heartbeats and disk space monitors to robots running synthetic transactions on our live web sites. One big step in making it all work was to define a common format for the event data. We have based ours on IBM's "Common Base Event". There is a paper out there called "Cannonical Situation Data Format: The common base event". It makes the rest of the code much simpler.
Cheers, R.
Pereant, qui ante nos nostra dixerunt!
-
Are you posting in the right place? Check out Where do I post X? to know for sure.
-
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big>
<blockquote> <br /> <dd>
<dl> <dt> <em> <font>
<h1> <h2> <h3> <h4>
<h5> <h6> <hr /> <i>
<li> <nbsp> <ol> <p>
<small> <strike> <strong>
<sub> <sup> <table>
<td> <th> <tr> <tt>
<u> <ul>
-
Snippets of code should be wrapped in
<code> tags not
<pre> tags. In fact, <pre>
tags should generally be avoided. If they must
be used, extreme care should be
taken to ensure that their contents do not
have long lines (<70 chars), in order to prevent
horizontal scrolling (and possible janitor
intervention).
-
Want more info? How to link
or How to display code and escape characters
are good places to start.
|