Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation

Mojolicious / Hypnotoad / concurrency / data scope

by 1nickt (Abbot)
on Jan 06, 2018 at 15:10 UTC ( #1206819=perlquestion: print w/replies, xml ) Need Help??

1nickt has asked for the wisdom of the Perl Monks concerning the following question:

Hi brethren, I am newly developing with Mojolicious. I have a question about scope. I've read some of the Mojo doc but not found a clear answer to my concern.

If I understand correctly, by default running a Mojo app under Hypnotoad will allow each worker process to accept N concurrent client connections, where N defaults to 1000 (from max_connections in Mojo::IOLoop). One can override this with the clients option, all the way down to 1 so that the worker process exits and a new one is spawned after each request. This is recommended only for applications that are heavily blocking.

What I haven't been able top confirm is whether each client connection is completely separate from the others within a single worker process. Here's the scenario:

  • The application code instantiates an object for each HTTP request
  • The object consumes a Logger role, which gets a logger from Log::Any and puts it in the object ($obj->log)
  • The Log::Any logger object is global.
  • The parent object sets some value in the $obj->log->context hash (see Log::Any context data)
  • Another class that is not composed into the object gets the same logger and logs some message, which causes the same context data to be appended.
  • Goal achieved -- log the tracking ID of an object instance even when not using the object's Logger role.
Why bother composing the Logger role and not just get a logger in the main class? Because the existing code relies on the logger being provided in a role.

So what I want to make sure of is that the data stashed in the logger context hash is not shared, obviously. Do I need to set Hypnotoad so that only one client connection is handled per worker? Or is the scope limited to client connection already? Or, am I missing some fundamental Mojo concept?


The way forward always starts with a minimal test.

Replies are listed 'Best First'.
Re: Mojolicious / Hypnotoad / concurrency / data scope
by Arunbear (Prior) on Jan 06, 2018 at 18:30 UTC
    Looking at the code, it seems the context is stored in a singleton object and would also persist for the life time of the worker process.

    Why not rebuild/reset the context for each new request, or localize it as the example you linked to shows?

      That's what I expect to do, but I am unclear about the scope within the Hypnotoad server, since each process serves multiple requests.

      The way forward always starts with a minimal test.
Re: Mojolicious / Hypnotoad / concurrency / data scope (SOLVED)
by 1nickt (Abbot) on Jan 08, 2018 at 18:03 UTC

    Answering myself here. I couldn't get an answer from the doc, so I simply wrote a comprehensive test.

    With a Mojolicious app running under hypnotoad with 12 workers, set to accept up to 100 concurrent client connections per worker, with Log::Any code setting an ID value in the log context in the library used by the route controller, each request did indeed receive a unique ID. Tested with 10,000 requests from 50 parallel HTTP clients.


    Update: As suggested by Your_Mother, here is the code I used for testing. I wanted to see whether (a) I would get a unique ID per request when using the Log::Any context, and (b) this was true when using a second library, getting a logger there and logging messages.

    To run the test for yourself, save all these files into a single directory, change into it, and do:

    $ hypnotoad $ prove tracking.t

    # tracking.t use strict; use warnings; use Test::More; use Parallel::ForkManager; use HTTP::Tiny; use List::Util qw( all ); use Path::Tiny; my $log_file = '/tmp/test.log'; path( $log_file )->append( { truncate => 1 }, '' ); my $url = 'http://localhost:8080/hello'; my $pm = new Parallel::ForkManager(50); my $ua = HTTP::Tiny->new; for ( 1 .. 10_000 ) { $pm->start and next; my $res = $ua->get( $url ); $pm->finish; } $pm->wait_all_children; my %ids; my $count = 0; for ( path( $log_file )->lines({ chomp => 1 }) ) { $count++; (my $id) = /"([^"]*)/; $ids{ $id }++; } is( $count, 20_000, '20,000 lines in the log file' ) +; is( scalar keys %ids, 10_000, '10,000 IDs' ); ok( (all { $_ == 2 } values( %ids )), 'each key was seen twice' ); done_testing; __END__
    # use strict; use warnings; $|++; use Log::Any::Adapter 'File', '/tmp/test.log'; use Mojolicious::Commands; use MyApp; Mojolicious::Commands->start_app('MyApp'); __END__
    # package MyApp { use Mojo::Base 'Mojolicious'; sub startup { my $self = shift; $self->config( hypnotoad => { workers => 12, clients => 100, accepts => 100, }, ); $self->routes->get('/hello')->to('foo#hello'); } }; package MyApp::Controller::Foo { use Mojo::Base 'Mojolicious::Controller'; use Digest::MD5 qw( md5_hex ); use Time::HiRes qw( time ); use MyLib; sub hello { my $self = shift; my $obj = MyLib->new; $obj->log->context->{id} = md5_hex(time . $$); $obj->log->debug('Hello from the route'); my $txt = $obj->other_class_action(42); $self->render(text => $txt); } }; 1;
    # use strict; use warnings; package MyLib { use Log::Any; use OtherLib; use Moo; has log => ( is => 'ro', default => sub { Log::Any->get_logger }, ); sub other_class_action { my ( $self, $val ) = @_; OtherLib::do_action( $val ); } }; 1;
    # use strict; use warnings; package OtherLib { use Log::Any '$log'; sub do_action { my $val = shift; my $data = { value => $val }; $log->debugf( 'Acting with %s', $data ); return sprintf 'The answer is %s', $val; } }; 1;

    Hope this helps!

    The way forward always starts with a minimal test.

      That's great. You might post a gist of the code? Or even the code here, if you're happy with it, so others would have the skeleton for testing.

        I've updated my node above with the code I used for testing (stripped of in-house stuff).

        Hope this helps!

        The way forward always starts with a minimal test.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1206819]
Approved by johngg
Front-paged by Corion
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others having an uproarious good time at the Monastery: (4)
As of 2020-10-31 02:49 GMT
Find Nodes?
    Voting Booth?
    My favourite web site is:

    Results (286 votes). Check out past polls.