Check if a scalar contains a complete HTTP request

FloydATC has asked for the wisdom of the Perl Monks concerning the following question:

Esteemed monks,

While reading from a non-blocking socket, I am trying to detect HTTP requests, complete with any POST data, within a scalar.

My first thought was to use HTTP::Request but it seems the ->parse() method will rather return an incomplete/invalid request than fail, and partial requests ruin everything.

As of now, I've been manually parsing content-length and chunked transfer encoding, but this is both slow and error-prone so I'd much rather lean on someone with broader experience than myself.

Any thoughts?

-- FloydATC

I got 99 problems, most of them have to do with printers.

Comment on Check if a scalar contains a complete HTTP request Download Code

Replies are listed 'Best First'.
Re: Check if a scalar contains a complete HTTP request by Corion (Patriarch) on May 20, 2017 at 09:30 UTC
Instead of doing this myself, I would look at one of the asynchronous frameworks, which all can handle that. See threads, AnyEvent, Mojolicious and IO::Async. There likely are more, but I boldly assume that all of them handle HTTP requests incoming and outgoing. If you have too much code written for your custom socket to rewrite for another framework, then running the HTTP part in a thread or separate process might be an approach that gives you only complete requests that can be read blocking.	[reply]
Re^2: Check if a scalar contains a complete HTTP request by FloydATC (Deacon) on May 20, 2017 at 11:02 UTC
Handling of the non-blocking sockets is not the issue, data from each client already flows beautifully into buffers thanks to IO::Select/sysread/syswrite and that bit works like a charm. It's just that I have about 60-70 lines of code just to grab HTTP requests from those buffers and besides being an eyesore I'm pretty sure it will fall over pretty soon if exposed to the real world. The main loop for each client/server socket pair kind of goes like this: Read from sockets that can be read from Fill inspection buffer with data from client If a buffer has been modified, see if it contains a HTTP request, complete with content (possibly chunked) If a HTTP request was found, remove it from the inspection buffer and allow/deny it Write allowed data to sockets that can be written to This is for SSL inspection if you didn't already guess, the prototype works but I'm trying to make it more robust. -- FloydATC I got 99 problems, most of them have to do with printers.	[reply]
Re^3: Check if a scalar contains a complete HTTP request by Marshall (Canon) on May 20, 2017 at 16:14 UTC
I guess I'm a bit lost here because this looks like the wrong application for a select server. Why aren't you using a normal TCP client/server model? What does this have to do with SSL inspection? Just curious..	[reply]
Re^4: Check if a scalar contains a complete HTTP request by FloydATC (Deacon) on May 22, 2017 at 07:57 UTC
Re^5: Check if a scalar contains a complete HTTP request by Marshall (Canon) on May 22, 2017 at 19:36 UTC
Some notes below your chosen depth have not been shown here
Re: Check if a scalar contains a complete HTTP request by Anonymous Monk on May 20, 2017 at 15:53 UTC
I doubt you'll find a module that does exactly what you want, but you might take a look at Net::Inspect::L7::HTTP.	[reply]
Re^2: Check if a scalar contains a complete HTTP request by FloydATC (Deacon) on May 22, 2017 at 15:07 UTC
Aha, this looks very promising indeed, thank you for the tip! -- FloydATC I got 99 problems, most of them have to do with printers.	[reply]


Perl: the Markov chain saw
	PerlMonks