Re^2: Fork-safe DBI handles via ChildHandles and InactiveDestroy

Re: designing around it: true. Unfortunately, this code was originally for perl 5.5 (we only recently upgraded to 5.8), which didn't support weak referrences. Holding some sort of internal list of all the connected dbhs without weak referrences would have meant (indirectly) subverting the implicit disconnect when dbhs fall out of scope (which is generally a good thing). That would have been a bad thing.

As far as the fact that a fork can sort of corrupt various resources which aren't even used in the subprocess... well, that's unfortunate, but not isolated to DBI connections. A similar thing happens with IO buffers, necessitating that you do something like "flush all handles" before a fork. Unfortunately, this, too (at least used to be... someone may have fixed this since I last looked into it) has no good solution, as it requires the forking code to somehow reach out and grab a list of all of a process's resources of a type (in this case database handles, in that case buffered IO handles). Similarly, in apache, there's a necessary step of calling "cleanup_for_exec", so that child processes' sockets don't get selected for handling new requests. The list goes on, I'm sure... which is sad, because it means there's a whole laundry list of things you need to remember to do if you want to fork... forgetting to do any of them meaning a potential obscure bug.

------------
:Wq
Not an editor command: Wq
[download]

Comment on Re^2: Fork-safe DBI handles via ChildHandles and InactiveDestroy Download Code

Replies are listed 'Best First'.
Re^3: Fork-safe DBI handles via ChildHandles and InactiveDestroy by Zaxo (Archbishop) on Jun 07, 2007 at 04:41 UTC
As far as the fact that a fork can sort of corrupt various resources which aren't even used in the subprocess... well, that's unfortunate, but not isolated to DBI connections. Exactly. To be really careful, a program which forks should avoid letting the kids inherit such connections. After Compline, Zaxo	[reply]
Re^4: Fork-safe DBI handles via ChildHandles and InactiveDestroy by etcshadow (Priest) on Jun 07, 2007 at 05:08 UTC
To be really careful, a program which forks should avoid letting the kids inherit such connections. Ideally, that's exactly what I'd do. Unfortunately, I'm not aware that any such method really exists. If there is such a way, I'd love to learn about it. For anything that is some sort of file handle or socket (including all my examples above), the only way (that I know of) that you can "avoid letting the kids inherit" it is to close it before forking. Of course, that fails to meet the broader goal of isolating it from unintended side-effect. (Closing it is a pretty big side-effect.) By the way, if this sounds argumentative, it's not intended. I'm aiming for "inquisitive". (It can be very hard to accurately convey tone in a text-only medium.) I'd really like to learn more about what facilities there are for making this kind of mess easier (if any exist). I can't tell if what you're saying is "Too bad this sort of necessary when dealing with sockets and handles and forks" or if what you're saying is "Geez, I can't believe you don't know about MagicallyAwesomeFork that allows you to selectively not copy portions of your process in the fork." Although I'm confused as to how that could possibly be accomplished, I'd definitely like to learn. `------------ :Wq Not an editor command: Wq` [download]	[reply] [d/l]
Re^3: Fork-safe DBI handles via ChildHandles and InactiveDestroy by sgt (Deacon) on Jun 07, 2007 at 12:39 UTC
A few months ago, to make a point about some of the pitfalls of fork and IO, I wrote a few tests to show the potential problems (from a Unx perspective following the R. Stevens books for instance), and all my tests were negative in the sense that (at least on cygwin -- I think I tested HP-UX which has been my perl platform for years, but I cannot remember for sure) implicit flushing was going on ...It would be interesting to know how much perl departs from Unx on this (and since when). In general for this kind of problem, maybe the best way to go is a process manager maintaining a pool of processes each one with one DB connection, and some protocol to reach the process manager. Anyway I find the "forks" module interesting. cheers --stephan	[reply]