Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

getting Sun info with Astro::Coord::ECI::Sun

by Aldebaran (Curate)
on May 23, 2020 at 07:51 UTC ( [id://11117145]=perlquestion: print w/replies, xml ) Need Help??

Aldebaran has asked for the wisdom of the Perl Monks concerning the following question:

I've been working up some scripts that query the US national weather service api for various and sundry data. In the last thread, I posted a clunky-looking thing with

use WWW::Mechanize; use HTML::TableExtract qw(tree);

, and marto simplified the whole business with Re: polishing up a json fetching script for weather data

where the above functionality was replaced with:

use Mojo::UserAgent;

I'd like to extend this by generalizing it to inputs of arbitrary lats and longs in the US as opposed to hard-coded to my location in Portland. Let's start by looking at a script, output first. It's longish, but important for people to see a sample of if they want to understand this. First, we find the stationID, gridX, gridY that correspond to a given ($lat, $long). Then we make a different query using these data, to find the forecast at this location.

Let's begin with the head and tail of the log:

2020/05/22 23:49:40 INFO ./3.6.pho.pl 2020/05/22 23:49:40 INFO 33.4 -112.1 112.1 2020/05/22 23:49:40 INFO PSR 2020/05/22 23:49:40 INFO 157 55 2020/05/22 23:49:40 INFO ============== 2020/05/22 23:49:40 INFO https://api.weather.gov/gridpoints/PSR/157,55 +/forecast/hourly 2020/05/22 23:49:40 INFO [ { "2020-05-22T23:00:00-07:00" => { "date-from-local" => "2020-05-22-23:00:00 PDT", "forecast-string" => "Clear", "forecast-temp-F" => 79, "human-readable" => "Fri 11 PM", "julian day" => "2458992.75", "number" => 1 } }, { "2020-05-23T00:00:00-07:00" => { "date-from-local" => "2020-05-23-00:00:00 PDT", "forecast-string" => "Clear", "forecast-temp-F" => 77, "human-readable" => "Sat 12 AM", "julian day" => "2458992.79166667", "number" => 2 } }, ... { "2020-05-29T09:00:00-07:00" => { "date-from-local" => "2020-05-29-09:00:00 PDT", "forecast-string" => "Sunny", "forecast-temp-F" => 94, "human-readable" => "Fri 9 AM", "julian day" => "2458999.16666667", "number" => 155 } }, { "2020-05-29T10:00:00-07:00" => { "date-from-local" => "2020-05-29-10:00:00 PDT", "forecast-string" => "Sunny", "forecast-temp-F" => 98, "human-readable" => "Fri 10 AM", "julian day" => "2458999.20833333", "number" => 156 } } ]

Source:

#!/usr/bin/env perl use strict; use warnings; use DateTime::Format::Strptime; use REST::Client; use Data::Roundtrip qw/:all/; use 5.016; use Log::Log4perl; use Data::Dump; # get rid of old log my $file = '/home/hogan/Documents/hogan/logs/4.log4perl.txt'; unlink $file or warn "Could not unlink $file: $!"; my $log_conf4 = "/home/hogan/Documents/hogan/logs/conf_files/4.conf"; Log::Log4perl::init($log_conf4); #info my $logger = Log::Log4perl->get_logger(); $logger->info("$0"); ## important...no trailing zeroes else 301 error my $lat = 33.4; my $long = -112.1; my $west_long = -$long; $logger->info("$lat $long $west_long"); # this is our fetcher, similar to LWP::UserAgent # but better suited for this kind of web service: REST my $rest = REST::Client->new() or die "failed to construct client"; # see examples in # https://www.weather.gov/documentation/services-web-api # set the host $rest->setHost('https://api.weather.gov'); # https://api.weather.gov/points/{latitude},{longitude} # https://api.weather.gov/points/45.4836356,-122.4170501 # and this is our query with lat,long specified above #my $query = "gridpoints/TOP/$lat,$long/forecast"; #https://api.weather.gov/stations/PQR/observations/latest KTTD my $query = "points/$lat,$long"; # make the request and check the response code, 200 is good my $response = $rest->GET($query) or die "failed to GET($query)"; if ( $rest->responseCode() != 200 ) { die "failed to GET(" . $rest->getHost() . "/$query) with " . $rest->responseCode(); } # we get back JSON my $jsonstr = $response->responseContent(); # convert JSON string to a perl variable my $pv = json2perl($jsonstr); if ( !defined $pv ) { warn "something wrong with this alleged json : '$jsonstr'"; } my $station = $pv->{'properties'}->{'cwa'}; my $gridX = $pv->{'properties'}->{'gridX'}; my $gridY = $pv->{'properties'}->{'gridY'}; $logger->info("$station"); $logger->info("$gridX $gridY"); # or print it all and examine it #$logger->info(perl2dump($pv)); ## use mojo now use Mojo::UserAgent; use open ':std', OUT => ':utf8'; use Mojo::Util qw(dumper); my $url = "https://api.weather.gov/gridpoints/$station/$gridX,$gridY/forecast/ +hourly"; $logger->info("=============="); $logger->info("$url"); # the API docs says you must identify yourself, please make this somet +hing legit my $name = '(example.com, contact@example.com)'; my $ua = Mojo::UserAgent->new; $ua->transactor->name($name); # get JSON response my $forecasts = $ua->get($url)->res->json->{properties}->{periods}; use DateTime::Format::Strptime; my $dateparser = DateTime::Format::Strptime->new( # parses 2020-04-26T06:00:00-05:00, # %F then literal T then %T then timezone offset pattern => '%FT%T%z' ) or die "failed to DateTime::Format::Strptime->new()"; my @ordered; my $refordered = \@ordered; my $index = 0; for my $aforecast (@$forecasts) { my $number = $aforecast->{number}; # start and end times of the prediction, convert them to objects my $start_time_str = $aforecast->{'startTime'}; say "start is $start_time_str"; my $start_time_dateobj = $dateparser->parse_datetime($start_time_str +) or die "parsing date '$start_time_str'."; my $jd = $start_time_dateobj->jd; $start_time_dateobj->set_time_zone('America/Los_Angeles'); my $st = $start_time_dateobj->strftime('%Y-%m-%d-%H:%M:%S %Z'); my $hour = $start_time_dateobj->hour_12(); my $which_m = $start_time_dateobj->am_or_pm(); my $abrv = $start_time_dateobj->day_abbr(); # sunny? my $forecast_str = $aforecast->{'shortForecast'}; # temperature as a number, see later for the units my $temp = $aforecast->{'temperature'}; # new scope for hash my %parsed; # store this record/prediction in our %parsed hash # keyed on this: my $key = $start_time_str; $parsed{$key} = { 'human-readable' => "$abrv $hour $which_m", 'date-from-local' => $st, 'number' => $number, 'forecast-string' => $forecast_str, 'julian day' => $jd, # we append temp unit to the key, e.g. 'F' 'forecast-temp-' . $aforecast->{'temperatureUnit'} => $temp, }; my $refparsed = \%parsed; $ordered[$index] = $refparsed; #$logger->info( dumper $refparsed); ++$index; } $logger->info( dumper $refordered); $logger->info("=============="); my $pturl1 = 'https://www.fourmilab.ch/yoursky/'; my $pturl2 = 'http://www.fourmilab.ch/cgi-bin/Yoursky?z=1&$lat&ns=North&lon=$west_l +ong&ew=West'; #my $tx = $ua->post( $pturl2 => form => { }); __END__

This script shows my learning curve with different modules. If one gives me malformed json, I can see what the other does. Where am I going with this?

I would like to add the altitude and azimuth of the sun to all of weather forecasts. I was going to do it with

my $pturl2 = 'http://www.fourmilab.ch/cgi-bin/Yoursky?z=1&$lat&ns=North&lon=$west_l +ong&ew=West'; #my $tx = $ua->post( $pturl2 => form => { utc => $dt->jd, date => '2' +});

, but I didn't want to make 156 different queries *AND* I reasoned that there had to be a reasonable way to calculate this value a priori. I didn't have long to search on cpan before I found Astro::Coord::ECI::Sun. This script delivers something near proof of concept with the example provided:

$ ./1.1.astro.pl ./1.1.astro.pl 33.4 -112.1 112.1 Sun rise is Sat May 23 09:49:05 2020 UT $ cat 1.1.astro.pl #!/usr/bin/env perl use strict; use warnings; use 5.016; use Log::Log4perl; # get rid of old log my $file = '/home/hogan/Documents/hogan/logs/4.log4perl.txt'; unlink $file or warn "Could not unlink $file: $!"; my $log_conf4 = "/home/hogan/Documents/hogan/logs/conf_files/4.conf"; Log::Log4perl::init($log_conf4); #info my $logger = Log::Log4perl->get_logger(); $logger->info("$0"); ## im my $deg_lat = 33.4; my $deg_long = -112.1; my $west_long = -$deg_long; $logger->info("$deg_lat $deg_long $west_long"); use Astro::Coord::ECI; use Astro::Coord::ECI::Sun; use Astro::Coord::ECI::Utils qw{deg2rad}; # 1600 Pennsylvania Ave, Washington DC USA # latitude 38.899 N, longitude 77.038 W, # altitude 16.68 meters above sea level my $lat = deg2rad (38.899); # Radians my $long = deg2rad (-77.038); # Radians my $alt = 16.68 / 1000; # Kilometers my $sun = Astro::Coord::ECI::Sun->new (); my $sta = Astro::Coord::ECI-> universal (time ())-> geodetic ($lat, $long, $alt); my ($time, $rise) = $sta->next_elevation ($sun); $logger->info("Sun @{[$rise ? 'rise' : 'set']} is ", scalar gmtime $time, " UT\n"); __END__ $

What am I trying to do here? I would like to gather whatever relevant information I can find about the sun with latitude, longitude, elevation, and time given as a DataTime object. I see tantalizing options like this:

($point, $intens, $central) = $sun->magnitude ($theta, $omega);

Where am I stuck? I have time like this

my $start_time_str = $aforecast->{'startTime'}; say "start is $start_time_str"; my $start_time_dateobj = $dateparser->parse_datetime($start_time_str +)

, and I need that to be simpatico with:

my $sun = Astro::Coord::ECI::Sun->universal ($time); # Figure out if the Sun is up at the observer's location. my ($azimuth, $elevation, $range) = $loc->azel ($sun); print "The Sun is ", rad2deg ($elevation), " degrees above the horizon.\n";

Thanks for your comment,

Replies are listed 'Best First'.
Re: getting Sun info with Astro::Coord::ECI::Sun
by bliako (Monsignor) on May 23, 2020 at 09:55 UTC

    Hello Aldebaran,

    Regarding the time, from what I read in the documentation of Universal-time of Astro::Coord::ECI, it should be seconds since the Linux (lololol) Unix Epoch in UTC (Coordinated Universal Time) time zone. Which is what you get from a DateTime's epoch() method. So, I think this is correct:

    my $sta = Astro::Coord::ECI-> universal ($start_time_dateobj->epoch())-> geodetic ($lat, $long, $alt);

    In general, if you ever need to convert between time zones, here is some extra information. The startTime you get from the api.weather.gov already contains a timezone. I remember output like 2020-04-26T06:00:00-05:00. Interesting question is what place's that time zone (-05:00) is: the enquiring coordinates or the enquirer's computer location (unlikely). You can deduce that by running a couple of tests for far away locations.

    A quick refresher on timezones from wikipedia tells us:

    Negative UTC offsets describe a time zone west of UTC±00:00, where the civil time is behind (or earlier) than UTC so the zone designator will look like "−03:00","−0300", or "−03".
    
    Positive UTC offsets describe a time zone east of UTC±00:00, where the civil time is ahead (or later) than UTC so the zone designator will look like "+02:00","+0200", or "+02".
    
    Examples 
    
        "−05:00" for New York on standard time (UTC-05:00)
        "−04:00" for New York on daylight saving time (UTC-04:00)
        "+00:00" (but not "-00:00") for London (UTC±00:00)
        "+02:00" for Cairo (UTC+02:00)
        "+05:30" for Mumbai (UTC+05:30)
        "+14:00" for Kiribati (UTC+14:00)
    

    If you must set a new timezone for a DateTime object: e.g. UTC (universal time; I have just learned that its name was born out of perfect delicate diplomatic balance) (equivalent to GMT for the layman). You can do it like this: my $UTCdateobj = $start_time_dateobj->set_time_zone('UTC');

    Given sun's relative position, the incoming solar energy per unit area can then be calculated, e.g. https://www.grc.nasa.gov/www/k-12/Numbers/Math/Mathematical_Thinking/sun12.htm

    btw, your link to Astro::Coord::ECI::Sun should be formatted as [mod://Astro::Coord::ECI::Sun] and not as [Astro::Coord::ECI::Sun].

    bw, bliako

      "btw, your link to Astro::Coord::ECI::Sun should be formatted as.."

      Your link calls a search on search.cpan, which redirects to metacpan. Cut out the middle man [metamod://Astro::Coord::ECI::Sun] -> Astro::Coord::ECI::Sun.

      >calculated

      not to mess up such a nice OP and reply, but since you used that word quoted above, I thought I'd mention the PDL for general purpose computations in the scientific realm.
      So, I think this is correct:   universal ($start_time_dateobj->epoch()

      Yes, I think so too. I may be re-ordering your response, because I don't have time to do a *good* write-up, just a sufficient one.

      your link to Astro::Coord::ECI::Sun should be formatted as Astro::Coord::ECI::Sun and not as Astro::Coord::ECI::Sun

      Thanks for the tip. How does a person strike out characters, like you did with 'Linux'?

      UTC (universal time; I have just learned that its name was born out of perfect delicate diplomatic balance) (equivalent to GMT for the layman)

      ...imagine that, we settled for Her Majesty at the center of the time world, of the empire on which the sun does not set....

      Negative UTC offsets describe a time zone west of UTC±00:00, where the civil time is behind (or earlier) than UTC so the zone designator will look like "−03:00","−0300", or "−03". "+14:00" for Kiribati (UTC+14:00)

      This was fascinating. These islands are below Hawaii. (You know, because north is up). We're at minus 7 for PDT (Pacific Daylight Time). What decides when someone's day starts? Is it 12 am local? (Is that a given?)

      The startTime you get from the api.weather.gov already contains a timezone. I remember output like 2020-04-26T06:00:00-05:00. Interesting question is what place's that time zone (-05:00) is: the enquiring coordinates or the enquirer's computer location (unlikely). You can deduce that by running a couple of tests for far away locations.

      I'm still testing and trying to make sense of what I have. I'm gonna put output and source between readmore tags:

      [Consider:] my $UTCdateobj = $start_time_dateobj->set_time_zone('UTC');

      I did not use this line, and it might be why I don't succeed.

      Anyways, thanks for your post, bliako, and your perspective from the world of positive time offsets. We'll be in the high desert this weekend, and you can be sure I'll have a weather report to take some of the guesswork out of it. Occasionally, the "data" can be quite beautiful: pictures of clouds (with data!)

        The thing that is not attaining is that the Altitude and Azimuth values are not changing as $start_time_dateobj does

        I am the last to dispense advise on celestial mechanics but I see all values changing.

        ...imagine that, we settled for Her Majesty at the center of the time world, of the empire on which the sun does not set....

        that's funny!

Re: getting Sun info with Astro::Coord::ECI::Sun
by etj (Deacon) on Apr 21, 2022 at 08:48 UTC
    This idea (of getting data over the web in a structured way for further processing) is vaguely similar to the thoughts in 11142744, which posits the idea of using a websocket to get streamed lightning strikes, and plotting them on a 3D model of the globe in real-time using PDL.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11117145]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others about the Monastery: (8)
As of 2024-04-24 10:49 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found