Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Implementing a forking SOAP::Lite server

by Tardis (Pilgrim)
on Nov 02, 2006 at 03:04 UTC ( [id://581816]=perlquestion: print w/replies, xml ) Need Help??

Tardis has asked for the wisdom of the Perl Monks concerning the following question:

Hi all,

I setup some time ago a SOAP service, using SOAP::Lite and in particular SOAP::Transport::HTTP::Daemon. It works fine.

However now I am finding that some queries are taking quite long (waiting on SQL) and the Daemon process thus created will only service one request at a time.

The official solution would seem to be to install Apache or possibly some other web server and run it via CGI/mod_perl/mod_soap.

I don't like the idea of installing a web server to solve a problem that might be solved with perl itself :-)

I've used Net::Server with a lot of success in the past, it's a very fast and efficient forking server framework.

However it's becoming clear to me that tying together SOAP::Lite and Net::Server might be a lot of work. Maybe so much that Laziness will dictate I install a web server after all. This would make me sad :-(

Has anyone either done this, or used some other technique to enable multiple simultaneous users of a SOAP::Lite service? In perl :-)

I noticed (while making my cpan:// links work :-) that there is a 'ForkOnAccept' module in the examples directory of the module, but I gave it a quick try and couldn't get it to work.

  • Comment on Implementing a forking SOAP::Lite server

Replies are listed 'Best First'.
Re: Implementing a forking SOAP::Lite server
by Tardis (Pilgrim) on Nov 02, 2006 at 03:11 UTC
    I noticed (while making my cpan:// links work :-) that there is a 'ForkOnAccept' module in the examples directory of the module, but I gave it a quick try and couldn't get it to work.
    OK I got it to work.

    It's obviously pretty raw, quite a few warnings while it is in use.

    I'd still prefer Net::Server or similar for more control (ie to make sure the number of children is kept under control, or even better, pre-forking.

    Any ideas very welcome :-)

      I have many systems with SOAP::Lite and prefer to use a standalone server (SOAP::Transport::(HTTP|TCP) because Fork on accept is a very bad option when you have many calls. Iīm prefer to use an single process with Listen is 10 or more, when one process is more quickly of ForkOnAccept!

      I think PreFork is a good solution, but only now (after some years) my database are upgrading to transaction mode!

      Pay attemption on timeouts on client side and a correct rollback actions (traped with SIGPIPE), or make an good "protocol" on client/server (Identify requests) if you need it.

      I donīt check Net::Server at now, but I think itīs a great solution, because Iīm implementing Config, Log Facilities and requestor control by hand, and itīs very hard!

        I don't like the idea of installing a web server to solve a problem that might be solved with perl itself :-)
      I have an Config_Server and a Dist_Server with some Targets_Agents. At Config_Server I think are good point to use Apache&mod_perl, but never simultaneous without transaction database, for others perl solve all on *nix servers and MS plataform ;)

      --
      Marco Antonio
      Rio-PM

Re: Implementing a forking SOAP::Lite server
by themage (Friar) on Nov 02, 2006 at 17:01 UTC
      I like POE a lot, and I had a quick go at the forking version, and tying it together with SOAP::Transport::HTTP. Unfortunately not knowing a lot about SOAP and the HTTP::* modules used, I'm not sure how to get them to cooperate. I don't really want to debug with tcpdump :-)

      It's probably the closest yet though...

        OK, scratch that, a bit more educated experimentation and I have it working!

        Initial performance results seem promising. It's driving the load up on my dev box, I consider that more useful than them waiting in line to be processed. Production box is much (much!) more grunty and I expect it to scream along.

Re: Implementing a forking SOAP::Lite server
by cLive ;-) (Prior) on Nov 02, 2006 at 16:28 UTC
    Oh, ours turned into a forking SOAP::Lite server without us having to write any code. Especially under mod_perl, trying to get authentication to work. In fact, I think f#@k was one of the milder terms used...
Re: Implementing a forking SOAP::Lite server
by Tardis (Pilgrim) on Nov 08, 2006 at 04:51 UTC
    What sort of a heel would I be if I didn't provide the sample code? :-)

    As mentioned, this is basically a quick hack of the POE Web Server With Forking example.

    This sample program sets up a (rather useless) SOAP service with a method called 'sleep' which .... well you can guess.

    Useless, but a good test of the success of the ability for this to service multiple requests at once, unlike SOAP::Transport::HTTP::Daemon.

    #!/usr/bin/perl use warnings; use strict; use SOAP::Transport::HTTP; use lib 'lib'; sub DEBUG () { 0 } # Enable a lot of runtime information +. sub MAX_PROCESSES () { 10 } # Total number of server processes. sub SERVER_PORT () { 8092 } # Server port to listen on. sub TESTING_CHURN () { 0 } # Randomly shutdown children to test +respawn. use POE; # Base features. use POE::Filter::HTTPD; # For serving HTTP content. use POE::Wheel::ReadWrite; # For socket I/O. use POE::Wheel::SocketFactory; # For serving socket connections. # These are HTTP::Request headers that have methods. my @method_headers = qw( authorization authorization_basic content content_encoding content_language content_length content_typ +e date expires from if_modified_since if_unmodified_since last_modifie +d method protocol proxy_authorization proxy_authorization_basic refere +r server title url user_agent www_authenticate ); # These are HTTP::Request headers that do not have methods. my @header_headers = qw( username opaque stale algorithm realm uri qop auth nonce cnonce nc response ); # Spawn up to MAX_PROCESSES server processes, and then run them. Exit # when they are done. server_spawn(MAX_PROCESSES); $poe_kernel->run(); exit 0; ### Spawn the main server. This will run as the parent process. sub server_spawn { my ($max_processes) = @_; POE::Session->create ( inline_states => { _start => \&server_start, _stop => \&server_stop, do_fork => \&server_do_fork, got_error => \&server_got_error, got_sig_int => \&server_got_sig_int, got_sig_chld => \&server_got_sig_chld, got_connection => \&server_got_connection, _child => sub { 0 }, }, heap => { max_processes => $max_processes, }, ); } ### The main server session has started. Set up the server socket and ### bookkeeping information, then fork the initial child processes. sub server_start { my ( $kernel, $heap ) = @_[ KERNEL, HEAP ]; $heap->{server} = POE::Wheel::SocketFactory->new ( BindPort => SERVER_PORT, SuccessEvent => "got_connection", FailureEvent => "got_error", Reuse => "yes", ); $kernel->sig( CHLD => "got_sig_chld" ); $kernel->sig( INT => "got_sig_int" ); $heap->{children} = {}; $heap->{is_a_child} = 0; warn "Server $$ has begun listening on port ", SERVER_PORT, "\n"; $kernel->yield("do_fork"); } ### The server session has shut down. If this process has any ### children, signal them to shutdown too. sub server_stop { my $heap = $_[HEAP]; DEBUG and warn "Server $$ stopped.\n"; if ( my @children = keys %{ $heap->{children} } ) { DEBUG and warn "Server $$ is signaling children to stop.\n"; kill INT => @children; } } ### The server session has encountered an error. Shut it down. sub server_got_error { my ( $heap, $syscall, $errno, $error ) = @_[ HEAP, ARG0 .. ARG2 ]; warn( "Server $$ got $syscall error $errno: $error\n", "Server $$ is shutting down.\n", ); delete $heap->{server}; } ### The server has a need to fork off more children. Only honor that ### request form the parent, otherwise we would surely "forkbomb". ### Fork off as many child processes as we need. sub server_do_fork { my ( $kernel, $heap ) = @_[ KERNEL, HEAP ]; return if $heap->{is_a_child}; my $current_children = keys %{ $heap->{children} }; for ( $current_children + 2 .. $heap->{max_processes} ) { DEBUG and warn "Server $$ is attempting to fork.\n"; my $pid = fork(); unless ( defined($pid) ) { DEBUG and warn( "Server $$ fork failed: $!\n", "Server $$ will retry fork shortly.\n", ); $kernel->delay( do_fork => 1 ); return; } # Parent. Add the child process to its list. if ($pid) { $heap->{children}->{$pid} = 1; next; } # Child. Clear the child process list. DEBUG and warn "Server $$ forked successfully.\n"; $heap->{is_a_child} = 1; $heap->{children} = {}; return; } } ### The server session received SIGINT. Don't handle the signal, ### which in turn will trigger the process to exit gracefully. sub server_got_sig_int { DEBUG and warn "Server $$ received SIGINT.\n"; return 0; } ### The server session received a SIGCHLD, indicating that some child ### server has gone away. Remove the child's process ID from our ### list, and trigger more fork() calls to spawn new children. sub server_got_sig_chld { my ( $kernel, $heap, $child_pid ) = @_[ KERNEL, HEAP, ARG1 ]; if ( delete $heap->{children}->{$child_pid} ) { DEBUG and warn "Server $$ received SIGCHLD.\n"; $kernel->yield("do_fork"); } return 0; } ### The server session received a connection request. Spawn off a ### client handler session to parse the request and respond to it. sub server_got_connection { my ( $heap, $socket, $peer_addr, $peer_port ) = @_[ HEAP, ARG0, AR +G1, ARG2 ]; DEBUG and warn "Server $$ received a connection.\n"; POE::Session->create ( inline_states => { _start => \&client_start, _stop => \&client_stop, got_request => \&client_got_request, got_flush => \&client_flushed_request, got_error => \&client_got_error, _parent => sub { 0 }, }, heap => { socket => $socket, peer_addr => $peer_addr, peer_port => $peer_port, }, ); delete $heap->{server} if TESTING_CHURN and $heap->{is_a_child} and ( rand() < 0.1 ); } ### The client handler has started. Wrap its socket in a ReadWrite ### wheel to begin interacting with it. sub client_start { my $heap = $_[HEAP]; $heap->{client} = POE::Wheel::ReadWrite->new ( Handle => $heap->{socket}, Filter => POE::Filter::HTTPD->new(), InputEvent => "got_request", ErrorEvent => "got_error", FlushedEvent => "got_flush", ); DEBUG and warn "Client handler $$/", $_[SESSION]->ID, " started.\n +"; } ### The client handler has stopped. Log that fact. sub client_stop { DEBUG and warn "Client handler $$/", $_[SESSION]->ID, " stopped.\n +"; } ### The client handler has received a request. If it's an ### HTTP::Response object, it means some error has occurred while ### parsing the request. Send that back and return immediately. ### Otherwise parse and process the request, generating and sending an ### HTTP::Response object in response. sub client_got_request { my ( $heap, $request ) = @_[ HEAP, ARG0 ]; my $soap = SOAP::Transport::HTTP::Server -> new -> dispatch_to('SOAPPackage') ; DEBUG and warn "Client handler $$/", $_[SESSION]->ID, " is handling a requ +est.\n"; if ( $request->isa("HTTP::Response") ) { $heap->{client}->put($request); return; } $soap->request($request); $soap->handle; my $response = $soap->response; $heap->{client}->put($response); } ### The client handler received an error. Stop the ReadWrite wheel, ### which also closes the socket. sub client_got_error { my ( $heap, $operation, $errnum, $errstr ) = @_[ HEAP, ARG0, ARG1, + ARG2 ]; DEBUG and warn( "Client handler $$/", $_[SESSION]->ID, " got $operation error $errnum: $errstr\n", "Client handler $$/", $_[SESSION]->ID, " is shutting down.\n" ); delete $heap->{client}; } ### The client handler has flushed its response to the socket. We're ### done with the client connection, so stop the ReadWrite wheel. sub client_flushed_request { my $heap = $_[HEAP]; DEBUG and warn( "Client handler $$/", $_[SESSION]->ID, " flushed its response.\n", "Client handler $$/", $_[SESSION]->ID, " is shutting down.\n" ); delete $heap->{client}; } ### We're done. package SOAPPackage; sub sleep { my $class = shift; my $sleep = shift; if (! defined $sleep) { die "no sleep time defined!"; } if ($sleep !~ /^\d+$/) { die "invalid sleep time '$sleep'"; } warn "$$ is sleeping for $sleep"; sleep $sleep; return "I slept for $sleep seconds"; }
    Here's a quick and dirty client program to connect to it:
    #!/usr/bin/perl use strict; use warnings; use SOAP::Lite; use Data::Dumper; my $object; my $method = shift; $object = SOAP::Lite -> uri('SOAPPackage') -> proxy("http://localhost:8092/") -> $method(@ARGV); if ($object->fault) { warn "FAULT " . $object->faultstring; exit 1; } else { print Dumper($object->result); exit 0; }
      Hey there. I realize this thread is quite old, however I was wondering how the posted code was working for you, or has anyone else done anything similar. I am looking to implement a SOAP::Lite service without mod_perl, and would like to know everyone's experiences.
        I just implemented this code with my own SOAP calls, and everything seems to work. However, it doesn't detach from terminal...the child processes spawn, but the parent process stays interactive. Any idea how to make this into a daemon?
Re: Implementing a forking SOAP::Lite server
by rahed (Scribe) on Nov 06, 2006 at 10:17 UTC
    I use HTTP::Daemon and ForkOnAccept module to accept soap requests. The traffic is low but was tested with 100 simultaneous connections without any noticeable service degradation. The processing of requests took more time but it doesn't matter in this situation.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://581816]
Approved by GrandFather
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others wandering the Monastery: (None)
    As of 2024-04-19 00:22 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      No recent polls found