Re: Queuing in multithread context

I try to implement the Thread::Queue module, but the provided example and the other resources I have been reading only consider a defined amount of queued tasks (when this may vary in my case).

What does that mean?

Its unclear what your diagram looks like, but I think Thread::Queue is workable like this


#!/usr/bin/perl --
use strict; use warnings;
use threads stack_size => 4096;
use Thread::Queue;

my %serverQ;
my $resultQ  = Thread::Queue->new;
my @workers  = map {
    my $workQ = Thread::Queue->new;
    threads->create( \&Worker, $workQ, $resultQ );
} 1..10;


...

while( @targets ){
    ...
    if( my $q = $serverQ{ $host } ){
        $q->enqueue( [
        ## 'Foo::bar::deployRoutine', $onearg, $twoarg, $threearg ,
            'deployRoutine' =>
                $targetServer, $deploySource, $deployDest,   "true",
        ] );
    
    ...
    FreeFreeWorkers( \%serverQ ); ## check empty Qs, unassociate from 
+a host
    

...

$_->join for threads->list;
exit( 0 );


sub Worker {
    my( $workQ, $resultQ ) = @_;
    while( defined( my $jobDesc = $workQ->dequeue ) ) {
        my( $package, $function, @args ) = @$jobDesc;
        require Module::Load;
        Module::Load::load( $package );
        $resultQ->enqueue( [ $function->( @args ) ] );
    }
    return;
}
[download]

Comment on Re: Queuing in multithread context Download Code

Replies are listed 'Best First'.
Re^2: Queuing in multithread context by Anonymous Monk on Jan 20, 2015 at 11:02 UTC
Bah, typo, idea is `### workQs , workerQs my @workers = map { my $workQ = Thread::Queue->new; threads->create( \&Worker, $workQ, $resultQ ); $workQ; } 1..10;` [download]	[reply] [d/l]
Re^2: Queuing in multithread context by Hardin (Novice) on Jan 20, 2015 at 11:28 UTC
Hi! Thanks for taking time to answer, sorry if I didn't make myself clear, my aim is to achieve queueing of my sub "deployRoutine" so that each copy task is executed after another and not sent alltogether. I have two main loops, the first loop through "targets" (ie: server where I need to copy files/directories) and the second embbeded is to loop through "deployments" which are several separated copy tasks. Lets say I have 3 servers : srv1, srv2 & srv3, I need to copy on each of those some remote content to local contents: copy1, copy2 & copy3 with copy1: source=\\some\remote\content1 dest=C:/some/local/content1 copy2: soure=\\some\different\remoteconte dest=C:/another/local/destination copy3: ... (you get the idea) I want the copy tasks (I said 3 but that could be 4, 5 or more, this can vary) sent to each servers one after another while currently they are sent at the same time since I create the thread its directly executed to the server. Right now I'm trying to understand your code as it's bit unclear to me. Cheers :)	[reply]
Re^3: Queuing in multithread context by BrowserUk (Patriarch) on Jan 20, 2015 at 11:33 UTC
my aim is to achieve queueing of my sub "deployRoutine" so that each copy task is executed after another and not sent alltogether. If you want to execute the subroutines sequentially; don't use threads! Just call the subroutine from within your loop; and they'll execute sequentially with no conflicts. Simples! With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked	[reply]
Re^2: Queuing in multithread context by BrowserUk (Patriarch) on Jan 20, 2015 at 11:11 UTC
Please ignore this. I had a brain fart! ~~Please don't advocate Thread::Queue. Its broken.~~ With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked	[reply]
Re^3: Queuing in multithread context by Anonymous Monk on Jan 20, 2015 at 11:23 UTC
Please don't advocate Thread::Queue. Its broken. Why? I learned to advocate it from your postings :)	[reply]
Re^4: Queuing in multithread context by BrowserUk (Patriarch) on Jan 20, 2015 at 11:30 UTC
Sorry. I brain-farted. I've scratched my post above. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked	[reply]


Do you know where your variables are?
	PerlMonks