Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Re: Multithreading Parsers

by djantzen (Priest)
on Mar 22, 2005 at 01:06 UTC ( #441354=note: print w/replies, xml ) Need Help??


in reply to Multithreading Parsers

A streaming parser doesn't seem like a very threadable kind of thing insofar as you're talking about walking a document tree in parallel. I think you'd end up doing a great deal of work trying to rig something where, for example, you'd have dedicated threads for processing particular node types.

I think I'd be more inclined to try to break the raw input into separate chunks that could run in parallel processes, something your 28 processor machine could do very well. So, maybe rather than reengineering your existing code to use threads, you could write a frontend script to divide whole units of work and then fork off processes.

Hard to say much more without concrete examples.


"The dead do not recognize context" -- Kai, Lexx

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://441354]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (8)
As of 2020-05-26 18:27 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    If programming languages were movie genres, Perl would be:















    Results (150 votes). Check out past polls.

    Notices?