Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot

Re: modern ways of doing web services

by Rhandom (Curate)
on Sep 26, 2013 at 13:47 UTC ( #1055833=note: print w/replies, xml ) Need Help??

in reply to modern ways of doing web services

If I had any recommendation to make for an API it would be: use JSON POSTing over HTTPS. Always.

Please do not use SOAP or any other XML based transport again. Ever.

Please do not use REST. REST ends up being wildly inconsistent and offers nearly no benefit. Do not use PUT or DELETE which are supposed to have obtuse HTTP statuses and truncated response data. And for consistency, don't use GET. Always use POST.

If you need to Create, Read, Update, Delete (CRUD), then simply consistently name your methods (server_add, server_info, server_update, server_delete, user_add, user_info, user_update, user_delete). Attach those on your endpoint (https://$host/$endpoint/server_info) and be done with it.

Consistency is one hallmark of a well designed API.

Update: The JSON bit isn't required - just trendy. Replace with sereal, amqp, storable and whatever the flavor of the decade is (as long as the flavor is not XML). But make sure you justify why you're not using JSON - which is widely more usable from other languages and platforms.

my @a=qw(random brilliant braindead); print $a[rand(@a)];

Replies are listed 'Best First'.
Re^2: modern ways of doing web services
by Your Mother (Bishop) on Sep 26, 2013 at 19:45 UTC

    I'm sorry, I agree with the XML/SOAP point but I think the rest is terrible advice. Doing webservices in a private idiom for each project is the opposite of consistency. REST only ends up inconsistent if it's abused/ignored. Using POST for every call is a kind of cognitive dissonance in design; why not just start over with POE or whatever and get away from HTTP completely. And Storable won't always even work between two different versions of Perl (even with nstore) let alone with anything else at all.

    I wrote a fairly half-baked REST service (some is not particularly RESTy and I regret and hope to phase out those misteps) for our main application at work about a year ago. Someone in another group picked it up and prototyped an integration with their application in a couple of weeks. Other teams had been working (well mostly arguing about how hard it would be and how to design it) to achieve this for two years. We were fairly astonished at how well it works. The dev said it was easy because the webservice was so simple/clear/atomic; "brilliant" was the word used. I brought nothing to that particular party except attempting to follow REST principles. It was the most gratifying thing that's happened to me in this particular job in a looooooooong time.

      I'm sorry in return for even hinting about using something other than JSON.

      REST is still wrong (even though it is better than SOAP). I actually feel strongly enough about it that I will explain in length why for posterity's sake.

      My post began with "use JSON POSTing over HTTPS. Always." Mistakenly I added an update to use "whatever" which you seem to have grasped onto. I stand by the first part: "use JSON POSTing over HTTPS. Always."

      Here are my reasons why to consistently use JSON POSTing over HTTPS always rather than use REST:

      You yourself mention that your "fairly half baked" REST services is "not particularly RESTy." This seems to happen often when people claim to follow REST. I have no doubt whatsoever that your solution you mentioned was absolutely brilliant (and I'm very glad you used REST instead of SOAP). But if it wasn't truly REST, or if even if it was, the client consumers likely had to write custom code.

      If you do have a REST client in any language on any platform - it will work with my "JSON POSTing over HTTPS" because JSON POSTing over HTTPS RESTish (1/4th of REST).

      If you don't have a REST client, it will take fewer lines of code and will be far more consistent to make your client speak to a consistent "JSON POSTing over HTTPS" than it will be to communicate with any REST based service.

      If you are writing a web based service, and are using *any* library to do your REST work (such as Mojo or the other frameworks, or some custom service), you can use that very same service to do consistent JSON POSTing over HTTPS for every method.

      If you intend to consume REST from the browser, you need a plugin to manage it. If you "always use JSON POSTing over HTTPS", it is easy to make beautiful unified API pages that make requests to the service (ajax consumption works just fine, but our consistent server layer also allows for normal x-www-urlencoded browser requests in addition to JSON POST for development and discovery purposes but we leave the normal client communication as JSON POSTing over HTTPS).

      I have consumed many REST services and each time I have these thoughts: PUT and DELETE are contrived; but at least this isn't SOAP.

      I don't like to use numbers to back up arguments, but we have at least 9 service APIs pulling over 12 million hits a day spiking to 20 million on some days (I know this is small compared to some shops). And they have consistently been doing this for over 7 years. Every single service is JSON POSTing over HTTPS. It is so consistent that our consuming code looks like this (seriously):
      my $client = API::Client->new({service => "dist"}); my $data = $client->server_info({hostname => ""}); # this maps out to POSTing +{"hostname":""}
      It is so consistent that our server code looks like this (our methods are self documenting and discoverable):
      package DistService; use API::Base; sub server_info__meta { return { desc => 'Returns information about a server', args => { server_id => {desc => 'The Id of this server', required => + 1}, }, resp => {success => 1}, }; } sub server_info { my ($self, $args) = @_; my %data; ... return \%data; }
      The API::Client code ends up being simple wrappers around configuration and existing perl network libraries. And we have consumers around the world running on many varied platforms. And they all had an easy time consuming because we "always use JSON POSTing over HTTPS." Just like REST (but moreso) it is even trivial to use curl or wget if you use consistent JSON POSTing over HTTPS.

      If you have commandline clients (every single one of our clients and servers has commandline interfaces) keeping the nouns and verbs in your consistent method names allows for calling the exact same method names on the commandline as you do in your API and even can keep them the same as listed in your code. You have to come up with new and creative ways of representing method names if you want to use REST since the noun and verb are separate. Here is a sample:

      [rhandom@somehost ~/%] ds server_info hostname Arguments: +----------+-------------+ | hostname | | +----------+-------------+ Data: +---------------+---------------------+ | access_tag | MDEV | | added_by_uid | 33 | | date_added | 2013-09-27 09:17:47 | | description | - | | hostname | | | ip | | | server_id | 26208 | +---------------+---------------------+
      Notice the client code, and the url, and the server code, and the commandline code all use the exact same method name - consistency will save time and frustration. You cannot do that with REST.

      Our implementation of JSON POSTing over HTTPS is consistent enough and simple enough and clean enough that we will likely eventually release our API based code to CPAN, despite the plethora of existing RPC tools on CPAN already.

      REST clients require you to determine which parts (if any) are the resource identifiers used in the URL, and which parameters are part of the JSON request, and then you have to keep your noun and verb separate. Consistent JSON POSTing over HTTPS allows for simple method/arg calls without having to do any parameter mapping.

      REST PUT and DELETE responses do not return data (or shouldn't if you are following real REST). Typically it is *very* useful to actually get back information from a delete (such as rows deleted, or if the record was already gone, or if it is pending, etc). Consistent JSON POSTing over HTTPS allows for returning data with any type of action.

      Lets look at the apache logs. Here is a tiny sampling from one of our lower hit services (with the ips munged to protect the innocent): - - [27/Sep/2013:08:12:50 -0600] "POST /dist/server_info HTT +P/1.0" 200 292 "-" "-" - - [27/Sep/2013:08:12:50 -0600] "POST /dist/unmanaged_add H +TTP/1.0" 200 21 "-" "-" - - [27/Sep/2013:08:12:50 -0600] "POST /dist/undone_list HTT +P/1.0" 200 218214 "-" "-" - - [27/Sep/2013:08:12:50 -0600] "POST /dist/unmanaged_add H +TTP/1.0" 200 21 "-" "-" - - [27/Sep/2013:08:12:50 -0600] "POST /dist/unmanaged_add H +TTP/1.0" 200 21 "-" "-" - - [27/Sep/2013:08:12:50 -0600] "POST /dist/undone_list HTT +P/1.0" 200 302337 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "POST /dist/unmanaged_add H +TTP/1.0" 200 21 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "POST /dist/server_info HTT +P/1.0" 200 290 "-" "-" - - [27/Sep/2013:08:12:50 -0600] "POST /dist/undone_list HTT +P/1.0" 200 146432 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "POST /dist/server_info HTT +P/1.0" 200 294 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "POST /dist/server_info HTT +P/1.0" 200 293 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "POST /dist/server_info_fil +es HTTP/1.0" 200 30321 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "POST /dist/unmanaged_add H +TTP/1.0" 200 21 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "POST /dist/unmanaged_add H +TTP/1.0" 200 21 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "POST /dist/unmanaged_add H +TTP/1.0" 200 21 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "POST /dist/server_info HTT +P/1.0" 200 273 "-" "-" - - [27/Sep/2013:08:12:52 -0600] "POST /dist/server_info HTT +P/1.0" 200 292 "-" "-" - - [27/Sep/2013:08:12:52 -0600] "POST /dist/server_info HTT +P/1.0" 200 294 "-" "-" - - [27/Sep/2013:08:12:52 -0600] "POST /dist/server_info HTT +P/1.0" 200 293 "-" "-" - - [27/Sep/2013:08:12:52 -0600] "POST /dist/server_info HTT +P/1.0" 200 290 "-" "-"
      Here is what the equivalent looks like if it was REST: - - [27/Sep/2013:08:12:50 -0600] "GET /dist/server/32 HTTP/1 +.0" 200 292 "-" "-" - - [27/Sep/2013:08:12:50 -0600] "POST /dist/unmanaged/33 HT +TP/1.0" 200 21 "-" "-" - - [27/Sep/2013:08:12:50 -0600] "GET /dist/undone HTTP/1.0" + 200 218214 "-" "-" - - [27/Sep/2013:08:12:50 -0600] "POST /dist/unmanaged/32 HT +TP/1.0" 200 21 "-" "-" - - [27/Sep/2013:08:12:50 -0600] "POST /dist/unmanaged/32 HT +TP/1.0" 200 21 "-" "-" - - [27/Sep/2013:08:12:50 -0600] "GET /dist/undone HTTP/1.0" + 200 302337 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "POST /dist/unmanaged/23 HT +TP/1.0" 200 21 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "GET /dist/server/36 HTTP/1 +.0" 200 290 "-" "-" - - [27/Sep/2013:08:12:50 -0600] "GET /dist/undone HTTP/1.0" + 200 146432 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "GET /dist/server/31 HTTP/1 +.0" 200 294 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "GET /dist/server/38 HTTP/1 +.0" 200 293 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "GET /dist/server/32_files +HTTP/1.0" 200 30321 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "POST /dist/unmanaged/23 HT +TP/1.0" 200 21 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "POST /dist/unmanaged/25 HT +TP/1.0" 200 21 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "POST /dist/unmanaged/26 HT +TP/1.0" 200 21 "-" "-" - - [27/Sep/2013:08:12:51 -0600] "GET /dist/server/38 HTTP/1 +.0" 200 273 "-" "-" - - [27/Sep/2013:08:12:52 -0600] "GET /dist/server/33 HTTP/1 +.0" 200 292 "-" "-" - - [27/Sep/2013:08:12:52 -0600] "GET /dist/server/34 HTTP/1 +.0" 200 294 "-" "-" - - [27/Sep/2013:08:12:52 -0600] "GET /dist/server/38 HTTP/1 +.0" 200 293 "-" "-" - - [27/Sep/2013:08:12:52 -0600] "GET /dist/server/33 HTTP/1 +.0" 200 290 "-" "-"
      REST *can* or *might* win here if you care about seeing the resource id (which I have greatly simplified in this REST example). However if I'm using most log analyzation tools or even live log viewing tools including toys like logstalgia, the consistent POST over HTTPS wins because the URL includes the noun AND the verb.

      REST cares too much about your abstractions and object representation. Consistent JSON POSTing over HTTPS doesn't care if you have a ${noun}_${verb} method or if you have a ${this_method_does_not_fit_anywhere} method. But it is always JSON POSTing over HTTPS. Should that random this_method_does_not_fit_anywhere method be a GET, a POST, a PUT, or a DELETE? - maybe it is just a method without object representation. Worse, what if my method updated *and* deleted a resource - which REST verb should I use? What a bout an import method that doesn't follow our normal and possibly even manages multiple resources at once? Consistent JSON POSTing over HTTPS does not care what your methods are. Conversly, you're not really REST unless you follow object abstractions to the letter.

      The only real benefits to REST are the ability to pull GET resources (assuming there is no authentication) directly from a browser (which you typically are not doing), and the ability to have a tiered system with caching based on URL alone (which I have never seen any company actually do in practice).

      I stand by the advice to always use JSON POSTing over HTTPS and to avoid REST. REST does not give you the benefits you think it does - but REST does put more work on the client consumers of your API.

      my @a=qw(random brilliant braindead); print $a[rand(@a)];

        I appreciate the simplification and see you're passionate about it.

        I only add: blaming crummy REST implementations on REST instead of the dev is a bit like blaming obesity on good food and exercise. REST does, like test driven development and other "cost up front" approaches, take discipline. I think it's worth it. I think the benefits cascade. I think working with HTTP instead of at right angles of convenience would have got web development as a whole further faster.

        the consistent POST over HTTPS wins because the URL includes the noun AND the verb. REST cares too much about your abstractions and object representation


Re^2: modern ways of doing web services
by Anonymous Monk on Sep 26, 2013 at 13:50 UTC

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1055833]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (7)
As of 2020-10-01 16:10 GMT
Find Nodes?
    Voting Booth?
    My favourite web site is:

    Results (16 votes). Check out past polls.