Come for the quick hacks, stay for the epiphanies. PerlMonks

### approximating geological problems with highway data

by Aldebaran (Curate)
 on Jan 25, 2023 at 07:19 UTC Need Help??

Aldebaran has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

I'm doing what people have to do in mormon country to save their sanity: study the rocks, which turns out to be, by far, the most interesting things in this region. So I came up with a script that was going to tell me something about the Bonneville flood, which I've been digging through the last 30 years of my life without having known it until last year. It is literally true that I brought the internet to Idaho in the form of walking in front of the caterpillar that ripped a channel for the fiberobtic cable from Preston to Spokane. (1989)

For the distance calc, I used geodatasource.com as a source. I supply a date and time that isn't relevant yet but stubbed in to be used eventually. Let me me just trot this thing out: 2.millcreek.pl. It's behaving pretty well, I think. I don't mean to thumb my nose at the metric system by using feet and miles, but it's definitely the parlance of the locals:

```fritz@laptop:~/Documents/gitlab1\$ ./2.millcreek.pl
Subroutine get_logger redefined at ./2.millcreek.pl line 68.
INFO: ./2.millcreek.pl
INFO: pi is 3.14159265358979
INFO: Bonneville max altitude in feet: 5200
INFO:  Boise 43.61  -116.2
INFO: return from the google is 821 meters
INFO: Altitude in feet is 2693.56964
INFO: Difference from max Bonneville elevation is 2506.43036 ft
INFO: ==============
INFO: distance is 240.161359349768 miles
INFO: ==============
INFO:  near sublett 42.3278114  -113.2104084
INFO: return from the google is 1455 meters
INFO: Altitude in feet is 4773.6222
INFO: Difference from max Bonneville elevation is 426.3778 ft
INFO: ==============
INFO: distance is 175.16979925839 miles
INFO: ==============
INFO:  snowville 41.9655701  -112.7105234
INFO: return from the google is 1384 meters
INFO: Altitude in feet is 4540.68256
INFO: Difference from max Bonneville elevation is 659.31744 ft
INFO: ==============
INFO: distance is 35.8058824322112 miles
INFO: ==============
INFO:  juniper 42.152429  -112.9842191
INFO: return from the google is 1577 meters
INFO: Altitude in feet is 5173.88468
INFO: Difference from max Bonneville elevation is 26.1153199999999 ft
INFO: ==============
INFO: distance is 19.0729839145012 miles
INFO: ==============
INFO:  on the outcrop 40.703684  -111.7849951
INFO: return from the google is 1752 meters
INFO: Altitude in feet is 5748.03168
INFO: Difference from max Bonneville elevation is -548.03168 ft
INFO: ==============
INFO: distance is 117.801976807568 miles
INFO: ==============
INFO: return from the google is 1524 meters
INFO: Altitude in feet is 5000.00016
INFO: Difference from max Bonneville elevation is 199.99984 ft
INFO: ==============
INFO: distance is 0.510598364633098 miles
INFO: ==============
INFO:  Mantua 41.5073303  -111.944728
INFO: return from the google is 1582 meters
INFO: Altitude in feet is 5190.28888
INFO: Difference from max Bonneville elevation is 9.71111999999994 ft
INFO: ==============
INFO: distance is 55.8975408837833 miles
INFO: ==============
INFO:  dry creek 41.5501001  -111.9537977
INFO: return from the google is 1810 meters
INFO: Altitude in feet is 5938.3204
INFO: Difference from max Bonneville elevation is -738.3204 ft
INFO: ==============
INFO: distance is 2.99196958759918 miles
INFO: ==============
INFO:  wellsville 41.6365147  -111.9288947
INFO: return from the google is 1376 meters
INFO: Altitude in feet is 4514.43584
INFO: Difference from max Bonneville elevation is 685.56416 ft
INFO: ==============
INFO: distance is 6.1074736112501 miles
INFO: ==============
fritz@laptop:~/Documents/gitlab1\$

So far, so good, and I welcome any style, usage, or numeric improvements. I might have 2 significant figures, but how much width can I fill up? Let's take a look at the dependencies:

```fritz@laptop:~/Documents/gitlab1\$ grep use 2.millcreek.pl
use v5.030;    # strictness implied
use warnings;
use feature qw[ signatures ];
use Data::Dumper;
use DateTime;
use DateTime::Format::ISO8601;
use DateTime::TimeZone;
use Log::Log4perl;
use Try::Tiny;
use LWP::UserAgent;
use HTTP::Request;
use Data::Roundtrip;

I have to use Log4perl to keep this type of data straight, and I use this hack of Log4perl so as to not have a .conf file dependency. Otherwise heavy with a lot of software from bliako. I wouldn't have been able to get these values right in geologic time without bliako's help at this level:

```
# this request was translated from Curl command-line
# by [Corion]'s https://corion.net/curl2lwp.psgi
my \$req = HTTP::Request->new(
'POST' =>
[
'Connection'      => 'keep-alive',
'Accept'          => '*/*',
'Accept-Encoding' => 'gzip, x-gzip, deflate, x-bzip2, bzip2',
'Accept-Language' => 'en-US,en;q=0.5',

#        'Host'            => 'www.mapcoordinates.net:443',
'Referer'    => 'https://www.mapcoordinates.net/en',
'User-Agent' =>
'Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firef
+ox/78.0',
'Content-Type'     => 'application/x-www-form-urlencoded; charse
+t=UTF-8',
'DNT'              => '1',
'Origin'           => 'https://www.mapcoordinates.net',
'X-Requested-With' => 'XMLHttpRequest'
],
);
die "call to HTTP::Request has failed" unless \$req;

What's the problem? 1. These values are ultimately dependent on Google, LLC. I'm happy that I can make use of them at this heartbeat, but I'm looking for other sources.

Q1) Does anyone know of another source for elevations on the interweb, given latitude and longitude? It seems like something more than one crappy company could pull off, in the age of satellites and lidar.

Q2) How do I make make such queries without associating it to my IP address?

## changing tack...

Rather than continuing in this line, let me try to get where I'm going by different methods. As I followed the path of the flood from Brigham City to Pocatello, I realized that I was on US highway 91 the entire time. If I instead had a dataset that represented that path, then I would also have a gradient for the flood. Let me try to represent what I have:

```https://www.google.com/maps/dir/Brigham+City,+UT/Logan,+UT/Preston,+ID
+/Pocatello,+ID/@42.1742739,-113.24861,8z/data=!3m1!4b1!4m26!4m25!1m5!
+1m1!1s0x87537b19bce31079:0xeeb9df2a3b92a006!2m2!1d-112.0155015!2d41.5
+102129!1m5!1m1!1s0x87547de05542a865:0xa9b33d0bcbbebcd5!2m2!1d-111.833
+8359!2d41.7369803!1m5!1m1!1s0x8754f78c7cda6c31:0xf1b3b4fc465a4a3f!2m2
+!1d-111.8766173!2d42.0963133!1m5!1m1!1s0x53554f20d5d09b0d:0x70c6a2484
+ce39b0!2m2!1d-112.4506191!2d42.8621042!3e0

I know that very few Germans will have been to Pocatello, but I find them to be some of the greatest enthusiasts for the rifted out sections of the American west. All that nothing, but then it's also so rich because it was a pluvial lake. The potatoes love it.

Q3) If not Highway 91, is there any other highway you can glean the points from?

Q4) Can we thus approximate the Mosel river from Trier to Koblenz? How many points do you need to drive a car next to this river (without falling in), as I remember doing many moons ago?

Ok, let me just throw this out there and see what I get.

Thanks for your comment and cheers,

Replies are listed 'Best First'.
Re: approximating geological problems with highway data
by bliako (Monsignor) on Jan 25, 2023 at 10:03 UTC

Welcome to the wonderful world of OpenStreetMap and, as marto said, Open Elevation. Here is a query to get you started with the latter: https://api.open-elevation.com/api/v1/lookup?locations=41.161758,-8.583933 (not sure lat/lon or lon/lat).

Try this at OpenStreetMap query at the sandbox at: https://turbo.overpass.kumi.systems/ :

```{{geocodeArea:Spokane}}->.searchArea;
way["highway"="path"][!foot][!bicycle][!segregated](area.searchArea);
(._;>;);
out;

make sure you click on the "Data" tab

bw, bliako

Welcome to the wonderful world of OpenStreetMap and, as marto said, Open Elevation.

Thx, man. You are honestly one of the people who makes the internet fun as well as manageable and informative.

Try this at OpenStreetMap query at the sandbox at: https://turbo.overpass.kumi.systems/ :

Just effin wow. What does this line in particular represent? [!foot], etc...

way["highway"="path"][!foot][!bicycle][!segregated](area.searchArea);

How might a person capture the data from the right?

Are node id's unique natural numbers that can be ordered into a graph?

Here is a query to get you started with the latter: https://api.open-elevation.com/api/v1/lookup?locations=41.161758,-8.583933 (not sure lat/lon or lon/lat).

I'm stretching my game but can't quite get there:

```\$ ./6.am.pl
Subroutine get_logger redefined at ./6.am.pl line 134.
INFO: ./6.am.pl
INFO: pi is 3.14159265358979
INFO: Bonneville max altitude in feet: 5200
Error fetching: 500 Server closed connection without sending any data
+back at ./6.am.pl line 298.
\$

The problem will be here:

```sub get_open_elevation {
my ( \$lat, \$lon, \$debug ) = @_;

use LWP::UserAgent;
use HTTP::Request;
use Data::Roundtrip;

\$debug //= 0;

#my \$ua = LWP::UserAgent->new( 'send_te' => '0' );
my \$ua = LWP::UserAgent->new( agent =>
'Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox
+/78.0', );
my \$r = HTTP::Request->new(
'GET' =>
'https://api.open-elevation.com/api/v1/lookup?locations=\$lat%2C\$lon',
[
'Accept'     => '*/*',
'User-Agent' => 'curl/7.55.1'
],

);
my \$response = \$ua->request( \$r, );

if ( \$debug > 0 ) {

\$logger->debug("\$0 : verbose");
}

die "Error fetching: " . \$response->status_line
unless \$response->is_success;

my \$content = \$response->decoded_content;
my \$data    = Data::Roundtrip::json2perl(\$content);
warn "failed to parse received data:\n\$content\n"
unless exists \$data->{'elevation'};

return 5000;

}

I may have worn out my welcome with the server because it didn't always not want to talk to me at all. Full script here for dedicated parties (80 lines less without dialing up google; I don't want to post it on gitlab until it looks and behaves better.):

Snagged on

`Σκύλλα`
,

I may have worn out my welcome with the server because it didn't always not want to talk to me at all

When I quote properly to allow interpolation, I get:

```\$ ./6.am.pl
...
Error fetching: 504 Gateway Time-out at ./6.am.pl line 298.
\$

, having changed the GET line to:

```    'GET' =>
"https://api.open-elevation.com/api/v1/lookup?locations=\$lat%2C\$lon",

Progress? Maybe.....

Re: approximating geological problems with highway data
by marto (Cardinal) on Jan 25, 2023 at 07:51 UTC

Q1. Open elevation was the first hit on a Web search for this. Q2 use answer for Q1 and host your own instance.

Q1. Open elevation was the first hit on a Web search for this.

Ok. Thx for your reply, marto, and several others who have pointed the way forward with better code. It will take me some time before I can incorporate the suggestions and have a different source for elevations than google, as well as avoiding having to going at it numerically by constraining hunting and pecking.

Q2 use answer for Q1 and

..., so get better data and...

Do you mean an instance of a HTTP::Request object as in:

``` my \$req = HTTP::Request->new(

I would still think that The Google would still know it's my IP address doing the asking. (?)

Cheers,

'Host your own', the second item on the linked page describes hosting your own instance of Open-Elevation and how to acquire the dataset you want. That way nobody can see what queries you're making.

Re: approximating geological problems with highway data
by Anonymous Monk on Jan 25, 2023 at 11:33 UTC
Re: approximating geological problems with highway data
by Anonymous Monk on Jan 25, 2023 at 11:47 UTC
GPS::Point
```use GPS::Point;
my \$gps  = GPS::Point->new(lat => \$lat1, lon => \$lon1);
my \$dist = \$gps->distance(\$lat2, \$lon2);
Re: approximating geological problems with highway data
by Anonymous Monk on Jan 25, 2023 at 18:51 UTC
If that's in reference to https://gitlab.com/perlmonks/betelgeuse/-/blob/main/2.millcreek.pl then don't worry about it. I have signed in to gitlab and that script is a 404 anyway. In fact it seems that even the perlmonks user is 404.

I thought I was riding off into the sunset as a successful gitlab user. Not so fast...

All of my visibility is private. I find myself unable to select anything other than private. I wondered whether I am the owner, and I am. I see this but do not understand it:

Change Project visibility to either Private, Internal, or Public. The visibility setting for a project must be at least as restrictive as the visibility of its parent group.

I also have to wonder out loud if this is when the hosting must not be gratis, but I don't find evidence for needing to find some bursar now, so I'm kind of just stuck. (?)

I can't see your code because Gitlab...

Nuts. I apologize for the dry link. I don't mean to be coy about posting source, but rather more mindful of the initial S in SSCCE. It's not punitively long:

Source:

Cheers,

In sub init_event { ... }
```{
"name"     => "snowville",
"date"     => \$date_str,
"time"     => \$time_str,
"location" => { lon => -112.7105234, lat => 41.9655701 },
42.152429, -112.9842191
},

Is the 42.152429, -112.9842191 key/value pair from the posted code perhaps a tyop? It's syntactically correct, but looks semantically fishy. Looks like it escaped from the following event record anonymous hash. (Update: OTOH, it looks quite harmless since you're unlikely to ever encounter a "42.152429" key in your code. :)

Also, if init_event() isn't just stub code for the SSCCE, a more succinct way to write it would be

```c:\@Work\Perl\monks>perl

use strict;
use warnings;

use Data::Dump qw(dd);

sub init_event {

# define unique event records as anonymous hash refs.
my \$ar_events = [
{ "name"     => "Boise",
"location" => { lon => -116.2, lat => 43.61 },
},
{ "name"     => "near sublett",
"location" => { lon => -113.2104084, lat => 42.3278114 },
},
{ "name"     => "snowville",
"location" => { lon => -112.7105234, lat => 41.9655701 },
},
{ "name"     => "juniper",
"location" => { lon => -112.9842191, lat => 42.152429 },
},
# and so on...
];

# add standard date/time to each event record.
my \$date_str = "2021-10-14";
my \$time_str = "03:22:31";

@{\$_}{ qw(date time) } = (\$date_str, \$time_str) for @\$ar_events;

return \$ar_events;

}

my \$ar_ev = init_event();
dd \$ar_ev;

^Z
[
{
date     => "2021-10-14",
location => { lat => "43.61", lon => "-116.2" },
name     => "Boise",
"time"   => "03:22:31",
},
{
date     => "2021-10-14",
location => { lat => "42.3278114", lon => "-113.2104084" },
name     => "near sublett",
"time"   => "03:22:31",
},
{
date     => "2021-10-14",
location => { lat => "41.9655701", lon => "-112.7105234" },
name     => "snowville",
"time"   => "03:22:31",
},
{
date     => "2021-10-14",
location => { lat => "42.152429", lon => "-112.9842191" },
name     => "juniper",
"time"   => "03:22:31",
},
]

Give a man a fish:  <%-{-{-{-<

```use Math::Trig qw[:pi deg2rad rad2deg];

Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11149859]
Approved by GrandFather
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others learning in the Monastery: (5)
As of 2024-04-13 00:38 GMT
Sections?
Information?
Find Nodes?
Leftovers?
Voting Booth?

No recent polls found