|
in reply to Re: Looking for alternative for IPC::Shareable (or increase size)
in thread Looking for alternative for IPC::Shareable (or increase size)
Dear jcb,
Storable looks pretty good, BUT: the locking mechanism is unsatisfactory. If more than one child is going to manipulate the array, it following could happen: child A lock_retrive array, change array (!), lock_store array. At position of ! another child B could do the same... This means, I'd have to build my own locking mechanism... (Yes, there can be more than one child at my application. ^^' )
The other way I see combining it with the hint of "Anonymous Monk": Generating separate name spaces for each child, and tell the parent only the name of the space when done... The actual problem here would be how many stores may I create until I reach any storage limits? (Multiple IPC::Shareables are limited, too...)
Anyways: It sounds faster than using database, so I surely will consider it. (Especially it seems to be implemented in Perl already.) |
Re^3: Looking for alternative for IPC::Shareable (or increase size)
by jcb (Parson) on Aug 07, 2020 at 02:01 UTC
|
The solution I use is for the child processes to feed array updates back to the master process, which applies them to the array. This has worked for me because the worker processes I have needed thus far have always had fully-defined tasks at the moment they are forked — the only communication needed is to report their results back to the master process.
I have used this technique to maintain Tk GUI responsiveness while issuing network queries in the background, but the data to scrape is known when the process forks and the child only needs to parse a reply and pass the "important bits" back up.
| [reply] |
|
