Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re: LWP::UserAgent Alternatives for API data

by johnfl68 (Scribe)
on Jul 06, 2020 at 17:04 UTC ( #11118977=note: print w/replies, xml ) Need Help??


in reply to LWP::UserAgent Alternatives for API data

Thanks for the input.

Sticking with LWP::UserAgent for the time being, in hope that NWS solves all the issues sometime soon.

I have been using DarkSky for years and there was seldom any issues like this, but that is going away in the future (thanks Apple).

I reported the issues to NWS, and got added to a large support ticket with many other people having similar issues, which confirms it's not me. Have added Retry coding for all the 500's and 502's, and trying to figure out the best way to check and retry when old data is returned, but I'm not sure an immediate retry will help any. Adding some code downstream to not update to new data if out of date and use slightly older (cached) data instead.

It's a nightmare. Getting data for 30 cities is taking almost 2 minutes now from Tier-1 Data Center (files are only about 13KB each). I can get the same from DarkSky in seconds with essentially the same code. OpenWeather's API works for some things, but does not provide some of the forecast wording that DarkSky and NWS provide.

Not many good weather API's left, they all keep going away. AccuWeather is about twice what I am paying for DarkSky, that's my next option but trying to avoid paying that much if I can.

Again thanks for the help and wisdom!



Update from NWS: "Since an upgrade API isn't behaving optimally, the 503 errors are a known issue protecting the application. A fix for the application's database is expected no later than late this summer. We apologize for the inconvenience. This ticket will remain open until the issue is resolved. If anyone would like to be removed from the distribution list, please let us know via reply email. Otherwise, you will remain on the distribution list and updates will be provided when available, perhaps in several weeks. (CJJ)"

So the issues are not going to get fixed anytime soon, so AccuWeather API it is.

  • Comment on Re: LWP::UserAgent Alternatives for API data

Replies are listed 'Best First'.
Re^2: LWP::UserAgent Alternatives for API data
by Aldebaran (Deacon) on Jul 07, 2020 at 05:38 UTC

    I think marto's reply to a similar question from me is relevant: Re: polishing up a json fetching script for weather data.

    # the API docs says you must identify yourself, please make this somet +hing legit my $name = '(example.com, contact@example.com)'; my $ua = Mojo::UserAgent->new; $ua->transactor->name( $name ); # get JSON response my $json = $ua->get( $url )->res->json->{properties};

    I wonder if we're looking at the same API...are you shy to post your source?

      I said it was the NWS API. Not hiding anything.

      https://api.weather.gov/

      I am working with 'Forecast' data. In the referenced Marto's reply post, they are trying to work with 'Latest Observations' which has even more issues than the Forecast data. I am sticking with the Latest Observations XML data from the NWS that comes from a different source than the NWS API. It has been much more reliable. Unfortunately they don't do the forecast data in XML like the latest observations, or I would use it instead right now.

      The NWS API is newer but it has been out for a few years now. It should be much more stable than it is at this point.

        He was probably referring to the fact that you haven't shown your source code with which you're having problems (which could reveal that there's something wrong / weird / off with what you're doing and that's where your problems are). Without your code (or an equivalent self-contained example snippet showing the same behavior) people are just guessing.

        The cake is a lie.
        The cake is a lie.
        The cake is a lie.

        Purely out of curiosity, can you be specific? Are you saying the API data differs from this XML source you have? Can you provide an example? If you believe there to be a discrepancy with the data have you reported this? It would likely be trivial to take the script provided and point it towards the XML source.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://11118977]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (3)
As of 2020-11-27 05:25 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?