http://qs321.pair.com?node_id=415524

We're doing some load testing on a Mason application that runs under SSL on Apache. We have some scripts that use WWW::Mechanize to interact with the application and complete a number of transactions. It all works, but the load on our web server was surprisingly high and the resulting transaction throughput not as high as we'd hoped.

Now I know that SSL is CPU intensive, but I wondered if there was any difference between the load imposed on the server by a request from a browser and a request from a Perl script. To test this, I set up two clients running on my Debian workstation:

  1. Mozilla 1.7.3 using some client-side JavaScript to cause a browser to repeatedly load a small image file (with a random query string added to defeat caching)
  2. A Perl script using WWW::Mechanize to request the same image file in a tight loop.

Now my workstation is not particularly fast and is geographically distant from the web server, so the absolute numbers are not significant, but the relative results are interesting:

Client Protocol Requests/s Server CPU Load
Mozilla HTTP 46.91 0.47%
Mozilla HTTPS 12.24 6.83%
Perl HTTP 28.11 0.65%
Perl HTTPS 5.97 20.50%

The fact that the Perl implementation of HTTP seems to give about half the throughput of Mozilla's highly optimised C implementation doesn't particularly surprise or concern me (and enabling keep_alive might narrow the gap). The fact that the web server has to work three times as hard to respond to half the volume of requests for Perl/HTTPS vs Mozilla/HTTPS is more worrying.

I was vaguely aware that the initial negotiation phase of an SSL interaction was relatively CPU intensive and that to alleviate this, the SSL protocol supports the concept of an SSL session identifier to allow a client and server to reuse the results of an earlier negotiation. In that vein, the POD for Net::SSLeay says:

"this module does not know to issue or serve multiple http requests per connection. This is a serious short coming, but using SSL session cache on your server helps to alleviate the CPU load somewhat"

So I added this to my Apache config:

SSLSessionCache dbm:/var/apache/ssl_session_cache

With this setting in place, the throughput for Mozilla+HTTPS climbed to about 15 requests/s but more importantly, the CPU load dropped to under 1%. Unfortunately, it made no difference at all to the Perl script since the low level Net::SSLeay doesn't do SSL session caching.

The README for IO::Socket::SSL refers to an unofficial later release of Net::SSLeay which does provide the appropriate hooks to allow IO::Socket::SSL to using SSL session caching.

We installed that unofficial release of Net::SSLeay and tested it by looping over this code:

my $client = new IO::Socket::SSL(PeerAddr => "servername", PeerPort +=> "https", SSL_session_cache_size => 100); if (defined $client) { print $client "GET /test/space.gif HTTP/1.0\r\n\r\n"; my @r = <$client>; close $client; } else { warn "SSL socket problem: ", IO::Socket::SSL::errstr(); }

Unfortunately, that still resulted in the same load on the server. Maybe extra parameters are required to enable session caching, I don't know. The next thing I tried was setting up a reusable IO::Socket::SSL::SSL_Context object like this:

my $context = new IO::Socket::SSL::SSL_Context( SSL_version => 'tlsv1', SSL_verify_mode => Net::SSLeay::VERIFY_NONE(), SSL_session_cache_size => 100 );

and reusing it on each request like this:

my $client = new IO::Socket::SSL(PeerAddr => "servername", PeerPort => + "https", SSL_reuse_ctx => $context);

Bingo! the server load dropped down to below 1% and the throughput rate for Perl climbed very slightly to 6.10.

The next question was how we could hack this low-level solution into the multi-layered script (WWW::Mechanise / LWP / IO::Socket::SSL / Net::SSLeay). It turned out to be remarkably easy. In our script, all we had to do was set a global SSL context before using the WWW::Mechanise object:

use IO::Socket::SSL; my $context = new IO::Socket::SSL::SSL_Context( SSL_version => 'tlsv1', SSL_verify_mode => Net::SSLeay::VERIFY_NONE(), SSL_session_cache_size => 100 ); IO::Socket::SSL::set_default_context($context);

Of course whereas the original test script put an unrealistically high load on the server by renegotiating too often; this modified version went too far the other way, by never renegotiating at all. The solution was to discard the default context object after a certain number of requests and create a new one.

Hopefully, these notes will be of use to someone else using LWP with SSL with high request rates.