http://qs321.pair.com?node_id=892378


in reply to Re^4: Sharing XS object?
in thread Sharing XS object?

You are just guessing. I benchmarked.

I hardly call making a reasoned guess "just guessing". When a reasoned guess and experimental evidence conflict, one of three things can be happening: the reasoned guess was logically flawed, the reasoned guess failed to include key information and so was wrong no matter how perfect the logic, or the experimental design is wrong. Those are important questions and they can't just be swept under the rug.

No matter what the answer there is something to learn. True learning in my experience comes from reguarly checking experience with logic and logic with experience. Inductive and deductive reasoning need to work in tandem. I hardly need to tell you that.

Secondly, with all due respect, your "benchmark" is comparing apples and oranges. The OP said he wanted to share an object and not just any object but a pre-initialized tree. This is exactly the kind of object threads::shared says it can't handle well. According to the perl docs, sharing a pre-initialized object will wipe out all of the data (see 5.10 and 5.12 bugs on threads::shared). Additionally, share can only do shallow sharing. If you have a complex deeply nested data structure you have to share each component bottom up and of course you have to reinitialize them all as well because sharing will wipe out their data.

In short, given the current status of threads::shared a true benchmark is simply not possible. Either you level the playing field by applying a server/client solution to a situation where it is clearly overkill. Alternatively, you apply the client/server solution to an complex pre-initialized object, but then you wouldn't be able to use shared.

Where I will agree with you is that, threads::shared is clearly not nearly as inefficient as I first believed. If they ever do clear up the intialization wipe-out bug, it will be a very powerful thing indeed.

The very first line of my post (reply 1) made an assertion that is almost certainly false along with a disclaimer that I'm not a threads expert (implication I may be wrong). Given that you are an expert, why didn't you just come out and say:

threads::shared may not be applicable to the kind of object the OP wants to share, but it isn't nearly as inefficient as you think. In fact in some cases it even uses less memory than a single thread. Here's why ...

It would have been a lot more helpful to me and like to others as well. Further, your answer would have been right up there in reply 2 where it would be much more likely to be read and gotten exposure.

I was intrigued by your performance observations and I threw together a few additional benchmarks of my own comparing scripts using three different scenarios: single thread, multi-thread w/o share and shared.

What is really clear from the results below is that there is simply no way that thread::shared is keeping per-thread copies of data - i.e. unshared threads with a lot of behind-the-scenes automated copying. The unshared multi-threaded script used up so much memory it froze my whole system. Furthermore, the shared multi-threaded script used less memory than a single threaded script. Care to explain what is going on? It certainly isn't what I thought.

hash memory runtime keys sz vsz wall user sys ThreadShared 1K 22.6 90.1 0:02.6 0:01.6 0:00.1 ThreadCopy 1K 23.2 93.0 0:02:9 0:01.4 0:00.8 No Threads 1K 0.9 3.5 0:00.8 0:00.7 0:00.01 ThreadShared 1M 41.5 166.3 0:12.2 0:10.7 0:00.9 ThreadCopy 1M untestable - froze up my system No Threads 1M 45.0 180.0 0:06.9 0:04.8 0:02.1 *these numbers and their relationships are stable through multiple runs so they are unlikely to be artifacts of some sort of transient system state.