|
in reply to Standard way to convert timezone in multithreaded script
.... regardless of BrowserUk's belief that threads, especially Perl threads, will give you some sort of performance boost ..... try it yourself and see, .....huge overheads
....maybe they do in MSWindows....but not under linux...and frankly, i would not use an OS that let a process running on one core of a multi-core system, alter the goings on in a neighboring core..... that is what you seem to be suggesting...... i could start a perl script, start x number of threads, and be assured that each thread (code block) will be assigned to a separate core..... with cross-core variable sharing thru threads::shared? .....bad design...... on linux, each thread of a spawned thread set shares the same pid, and if you call exit from any thread, it takes down all threads under the master thread, including the master thread.... what more evidence do you need that they share
the same pid?... and probably have their nice level set as a whole based on the master thread's pid
...so even though in theory, BrowserUk's assertion that Perl threads will give you concurrent processing... in all likelihood, they won't..... i would bet for security, a pid gets spawned into 1 core, and since threads on linux share the same master pid, all threads probably will be assigned to the same core.... otherwise it would be chaos as programs try to spawn threads into other cores..... maybe it is done in sloppy OS's ?..... but not to abandon the intent of my reply..... threads are good if you want easy realtime sharing of data between threads..... but they are not for speed
...what you want to look at is forking and using shared memory segments for the best speed.... read "perldoc perlipc" and look for the section SysV IPC.... it is the fastest
.... and that is no baloney...i'm a vegetarian ;-)
|
Re^2: Standard way to convert timezone in multithreaded script
by tirwhan (Abbot) on Nov 24, 2009 at 11:37 UTC
|
i would bet for security, a pid gets spawned into 1 core, and since threads on linux share the same master pid, all threads probably will be assigned to the same core
Can I name an amount? Because I'll take that bet :-). Sorry, but this is completely wrong, and if you think about it for a minute you'll realize that it would be completely brainless and negate the usefulness of kernel threads entirely. Yes, threads are spawned under a single PID, but they can very well use more than a single core. This is simple to demonstrate (WARNING, following code will hog your CPUs, don't run it on a production system!):
perl -Mthreads -e'$n=3;for (1..$n){$t[$_]=threads->create(sub{my $r; while(1){$r++}});}; sleep 10;threads->exit()'
Set $n to 1 less than the number of cores your machine has (in this case 3 for a quad-core). Then run the command and observe its behaviour in the system monitor of your choice (top will do). You'll see that it runs on the number of CPU cores specified, despite retaining a single PID. So your assumption is wrong.
As for the usefulness/performance benefit of updating a shared variable from threads running on several cores, I'm staying firmly out of that debate. My own gut feeling would be that it is indeed not worth it on Linux, context switching Perl threads is more expensive than doing the same with processes (or at least it was the last time I measured), and a short test with the above command line using threads::shared seems to support the feeling. But the true answer will invariably depend (almost exclusively) on the concrete problem the OP is trying to solve, so he should benchmark competing solutions to that concrete problem and find out for himself.
| [reply]
[d/l] |
|
|
| [reply]
[d/l] |
|
|
However, n=1 (recommended setting) only pegs one core.
Yes, of course. A single thread will only execute on one core. Which core that is may change (it's up to the OS), but it won't run on more than one core simultaneously. Multiple threads will.
(I guess you already know this, just saying it explicitly in case someone reads all of this and is under the impression that simply saying "use threads;" will make a program run concurrently on several CPUs. That would be magic, not concurrency :-)
| [reply]
[d/l] |
|
|
... i don't have a multi-cpu system to test on, but i accept your explanation that a single process can be running things across core boundaries.... but that seems quite insecure and it would seem to be a good kernel option to have ( like in the NSA-security kernel )...why?.... because according to your explanation, any process can start injjecting code into all the other cores of a multicore machine, just by starting a few threads..... how does this affect the nice value of the parent thread... does it increase according to thread count?....... what does it mean then to give a nice value to a script?... if the script can start spawning an infinite number of threads..... or what is to stop core hijacking by me spawning a thread until it gets into the desired core.... then executing something that halts the core, or other malicious intent...oO( maybe promoting threading is good for national security ;-) )
... my original intent was to point out that threads are not the fastest IPC out there, and continuing in that vein, there is Why use threads over processes, or why use processes over threads? and Threads versus fork, which to use? and Using forks and threads ...then choose the IPC right for you
..... by the way.... you would have no such POSIX module thread-safety in a fork-and-exec
| [reply] |
|
|
The security concerns you name are completely irrelevant. Any user who can execute code can hog all cores of a system, any time they want to, by starting processes that do so (unless you restrict them with something like ulimit, which works just as well on threaded processes). The kernels job is to keep the system running responsive enough so that other user processes and the system in general run well. Try running the line I sent, it'll work on a single-core system just as well. If you're running a reasonably modern version of Linux you may experience some slowdown on certain other applications, but your system should not grind to a halt, nor show real signs of unresponsiveness. That's the kernel doing its job, no reason to panic, business as usual.
So all the talk about hijacking, injecting etc. completely misses the point of a multi-user system, yes you can do all these things with threads (if you're root), just as you could with processes. But no, it isn't a problem (if it were, by your reasoning, system security would be impossible on a single-core machine). The kernel only allows you as much access as the system policy dictates, and the system administrator sets system policy. You may have a point that multi-threaded apps can make things harder for the system administrator by obscuring what they're doing, because most Linux tools are built primarily with the multi-process model in mind. But, honestly, in over a decade of sysadmining multi-user systems I have never found this to be a problem.
BTW, the nodes you link to aren't really pertinent to a discussion about threads vs. processes today, they're from 2003 (only the last one, which contains very little helpful discussion on the topic, is from 2006), a time when most people were still running Perl 5.6 and (if on Linux) a 2.4 kernel. Lots of things have changed since then, and any serious discussion on the topic would have to take these changes into account (I have no desire to hold such a debate, I find it pretty pointless).
| [reply]
[d/l] |
|
|
|
|
|
|
| [reply] |
|
|
|
|
| |
|
Re^2: Standard way to convert timezone in multithreaded script
by whale2 (Novice) on Nov 23, 2009 at 21:39 UTC
|
I has already achieved required speed boost in my script using threads. Everything is clear and pretty straightforward here. The problem I ran into is converting timestamps using POSIX routines. Now I see that this will not work in multithreaded app. But I tried to do this directly from .xs and it works.
Thanks everyone for your attention :) | [reply] |
Re^2: Standard way to convert timezone in multithreaded script
by BrowserUk (Patriarch) on Nov 24, 2009 at 01:59 UTC
|
I'm afraid that this post just demonstrates how little you understand about how your favorite OS kernel works.
Don't feel too bad, you're far from alone in that.
| [reply] |
|
|
likewise, i know how my kernel assigns pids, and i thank God that my os dosn't confuse fork-and-exec with threading, as does your favorite ...... at least my destroyer won't be totally shut down because the gun turret is stuck
| [reply] |
|
|
| [reply] |
|
|
