Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

approximating geological problems with highway data

by Aldebaran (Curate)
on Jan 25, 2023 at 07:19 UTC ( [id://11149859]=perlquestion: print w/replies, xml ) Need Help??

Aldebaran has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

I'm doing what people have to do in mormon country to save their sanity: study the rocks, which turns out to be, by far, the most interesting things in this region. So I came up with a script that was going to tell me something about the Bonneville flood, which I've been digging through the last 30 years of my life without having known it until last year. It is literally true that I brought the internet to Idaho in the form of walking in front of the caterpillar that ripped a channel for the fiberobtic cable from Preston to Spokane. (1989)

For the distance calc, I used geodatasource.com as a source. I supply a date and time that isn't relevant yet but stubbed in to be used eventually. Let me me just trot this thing out: 2.millcreek.pl. It's behaving pretty well, I think. I don't mean to thumb my nose at the metric system by using feet and miles, but it's definitely the parlance of the locals:

fritz@laptop:~/Documents/gitlab1$ ./2.millcreek.pl Subroutine get_logger redefined at ./2.millcreek.pl line 68. INFO: ./2.millcreek.pl INFO: pi is 3.14159265358979 INFO: Bonneville max altitude in feet: 5200 INFO: Boise 43.61 -116.2 INFO: return from the google is 821 meters INFO: Altitude in feet is 2693.56964 INFO: Difference from max Bonneville elevation is 2506.43036 ft INFO: ============== INFO: distance is 240.161359349768 miles INFO: ============== INFO: near sublett 42.3278114 -113.2104084 INFO: return from the google is 1455 meters INFO: Altitude in feet is 4773.6222 INFO: Difference from max Bonneville elevation is 426.3778 ft INFO: ============== INFO: distance is 175.16979925839 miles INFO: ============== INFO: snowville 41.9655701 -112.7105234 INFO: return from the google is 1384 meters INFO: Altitude in feet is 4540.68256 INFO: Difference from max Bonneville elevation is 659.31744 ft INFO: ============== INFO: distance is 35.8058824322112 miles INFO: ============== INFO: juniper 42.152429 -112.9842191 INFO: return from the google is 1577 meters INFO: Altitude in feet is 5173.88468 INFO: Difference from max Bonneville elevation is 26.1153199999999 ft INFO: ============== INFO: distance is 19.0729839145012 miles INFO: ============== INFO: on the outcrop 40.703684 -111.7849951 INFO: return from the google is 1752 meters INFO: Altitude in feet is 5748.03168 INFO: Difference from max Bonneville elevation is -548.03168 ft INFO: ============== INFO: distance is 117.801976807568 miles INFO: ============== INFO: Cascade way 40.7062734 -111.7941259 INFO: return from the google is 1524 meters INFO: Altitude in feet is 5000.00016 INFO: Difference from max Bonneville elevation is 199.99984 ft INFO: ============== INFO: distance is 0.510598364633098 miles INFO: ============== INFO: Mantua 41.5073303 -111.944728 INFO: return from the google is 1582 meters INFO: Altitude in feet is 5190.28888 INFO: Difference from max Bonneville elevation is 9.71111999999994 ft INFO: ============== INFO: distance is 55.8975408837833 miles INFO: ============== INFO: dry creek 41.5501001 -111.9537977 INFO: return from the google is 1810 meters INFO: Altitude in feet is 5938.3204 INFO: Difference from max Bonneville elevation is -738.3204 ft INFO: ============== INFO: distance is 2.99196958759918 miles INFO: ============== INFO: wellsville 41.6365147 -111.9288947 INFO: return from the google is 1376 meters INFO: Altitude in feet is 4514.43584 INFO: Difference from max Bonneville elevation is 685.56416 ft INFO: ============== INFO: distance is 6.1074736112501 miles INFO: ============== fritz@laptop:~/Documents/gitlab1$

So far, so good, and I welcome any style, usage, or numeric improvements. I might have 2 significant figures, but how much width can I fill up? Let's take a look at the dependencies:

fritz@laptop:~/Documents/gitlab1$ grep use 2.millcreek.pl use v5.030; # strictness implied use warnings; use feature qw[ signatures ]; use Data::Dumper; use DateTime; use DateTime::Format::ISO8601; use DateTime::TimeZone; use Log::Log4perl; use Try::Tiny; use LWP::UserAgent; use HTTP::Request; use Data::Roundtrip;

I have to use Log4perl to keep this type of data straight, and I use this hack of Log4perl so as to not have a .conf file dependency. Otherwise heavy with a lot of software from bliako. I wouldn't have been able to get these values right in geologic time without bliako's help at this level:

# this request was translated from Curl command-line # by [Corion]'s https://corion.net/curl2lwp.psgi my $req = HTTP::Request->new( 'POST' => 'https://www.mapcoordinates.net/admin/component/edit/Vpc_MapCoordinate +s_Advanced_GoogleMapCoords_Component/Component/json-get-elevation', [ 'Connection' => 'keep-alive', 'Accept' => '*/*', 'Accept-Encoding' => 'gzip, x-gzip, deflate, x-bzip2, bzip2', 'Accept-Language' => 'en-US,en;q=0.5', # 'Host' => 'www.mapcoordinates.net:443', 'Referer' => 'https://www.mapcoordinates.net/en', 'User-Agent' => 'Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firef +ox/78.0', 'Content-Length' => $payloadlen, 'Content-Type' => 'application/x-www-form-urlencoded; charse +t=UTF-8', 'DNT' => '1', 'Origin' => 'https://www.mapcoordinates.net', 'X-Requested-With' => 'XMLHttpRequest' ], $payload ); die "call to HTTP::Request has failed" unless $req;

What's the problem? 1. These values are ultimately dependent on Google, LLC. I'm happy that I can make use of them at this heartbeat, but I'm looking for other sources.

Q1) Does anyone know of another source for elevations on the interweb, given latitude and longitude? It seems like something more than one crappy company could pull off, in the age of satellites and lidar.

Q2) How do I make make such queries without associating it to my IP address?

changing tack...

Rather than continuing in this line, let me try to get where I'm going by different methods. As I followed the path of the flood from Brigham City to Pocatello, I realized that I was on US highway 91 the entire time. If I instead had a dataset that represented that path, then I would also have a gradient for the flood. Let me try to represent what I have:

https://www.google.com/maps/dir/Brigham+City,+UT/Logan,+UT/Preston,+ID +/Pocatello,+ID/@42.1742739,-113.24861,8z/data=!3m1!4b1!4m26!4m25!1m5! +1m1!1s0x87537b19bce31079:0xeeb9df2a3b92a006!2m2!1d-112.0155015!2d41.5 +102129!1m5!1m1!1s0x87547de05542a865:0xa9b33d0bcbbebcd5!2m2!1d-111.833 +8359!2d41.7369803!1m5!1m1!1s0x8754f78c7cda6c31:0xf1b3b4fc465a4a3f!2m2 +!1d-111.8766173!2d42.0963133!1m5!1m1!1s0x53554f20d5d09b0d:0x70c6a2484 +ce39b0!2m2!1d-112.4506191!2d42.8621042!3e0

I know that very few Germans will have been to Pocatello, but I find them to be some of the greatest enthusiasts for the rifted out sections of the American west. All that nothing, but then it's also so rich because it was a pluvial lake. The potatoes love it.

Q3) If not Highway 91, is there any other highway you can glean the points from?

Q4) Can we thus approximate the Mosel river from Trier to Koblenz? How many points do you need to drive a car next to this river (without falling in), as I remember doing many moons ago?

Ok, let me just throw this out there and see what I get.

Thanks for your comment and cheers,

Replies are listed 'Best First'.
Re: approximating geological problems with highway data
by bliako (Monsignor) on Jan 25, 2023 at 10:03 UTC

    Welcome to the wonderful world of OpenStreetMap and, as marto said, Open Elevation. Here is a query to get you started with the latter: https://api.open-elevation.com/api/v1/lookup?locations=41.161758,-8.583933 (not sure lat/lon or lon/lat).

    Try this at OpenStreetMap query at the sandbox at: https://turbo.overpass.kumi.systems/ :

    {{geocodeArea:Spokane}}->.searchArea; way["highway"="path"][!foot][!bicycle][!segregated](area.searchArea); (._;>;); out;

    make sure you click on the "Data" tab

    bw, bliako

      Welcome to the wonderful world of OpenStreetMap and, as marto said, Open Elevation.

      Thx, man. You are honestly one of the people who makes the internet fun as well as manageable and informative.

      Try this at OpenStreetMap query at the sandbox at: https://turbo.overpass.kumi.systems/ :

      Just effin wow. What does this line in particular represent? [!foot], etc...

      way["highway"="path"][!foot][!bicycle][!segregated](area.searchArea);

      How might a person capture the data from the right?

      Are node id's unique natural numbers that can be ordered into a graph?

      Here is a query to get you started with the latter: https://api.open-elevation.com/api/v1/lookup?locations=41.161758,-8.583933 (not sure lat/lon or lon/lat).

      I'm stretching my game but can't quite get there:

      $ ./6.am.pl Subroutine get_logger redefined at ./6.am.pl line 134. INFO: ./6.am.pl INFO: pi is 3.14159265358979 INFO: Bonneville max altitude in feet: 5200 Error fetching: 500 Server closed connection without sending any data +back at ./6.am.pl line 298. $

      The problem will be here:

      sub get_open_elevation { my ( $lat, $lon, $debug ) = @_; use LWP::UserAgent; use HTTP::Request; use Data::Roundtrip; $debug //= 0; #my $ua = LWP::UserAgent->new( 'send_te' => '0' ); my $ua = LWP::UserAgent->new( agent => 'Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox +/78.0', ); my $r = HTTP::Request->new( 'GET' => 'https://api.open-elevation.com/api/v1/lookup?locations=$lat%2C$lon', [ 'Accept' => '*/*', 'User-Agent' => 'curl/7.55.1' ], ); my $response = $ua->request( $r, ); if ( $debug > 0 ) { $logger->debug("$0 : verbose"); } die "Error fetching: " . $response->status_line unless $response->is_success; my $content = $response->decoded_content; my $data = Data::Roundtrip::json2perl($content); warn "failed to parse received data:\n$content\n" unless exists $data->{'elevation'}; return 5000; }

      I may have worn out my welcome with the server because it didn't always not want to talk to me at all. Full script here for dedicated parties (80 lines less without dialing up google; I don't want to post it on gitlab until it looks and behaves better.):

      Snagged on

      Σκύλλα
      ,

        I may have worn out my welcome with the server because it didn't always not want to talk to me at all

        When I quote properly to allow interpolation, I get:

        $ ./6.am.pl ... Error fetching: 504 Gateway Time-out at ./6.am.pl line 298. $

        , having changed the GET line to:

        'GET' => "https://api.open-elevation.com/api/v1/lookup?locations=$lat%2C$lon",

        Progress? Maybe.....

Re: approximating geological problems with highway data
by marto (Cardinal) on Jan 25, 2023 at 07:51 UTC

    Q1. Open elevation was the first hit on a Web search for this. Q2 use answer for Q1 and host your own instance.

      Q1. Open elevation was the first hit on a Web search for this.

      Ok. Thx for your reply, marto, and several others who have pointed the way forward with better code. It will take me some time before I can incorporate the suggestions and have a different source for elevations than google, as well as avoiding having to going at it numerically by constraining hunting and pecking.

      Q2 use answer for Q1 and

      ..., so get better data and...

      host your own instance

      Do you mean an instance of a HTTP::Request object as in:

      my $req = HTTP::Request->new(

      I would still think that The Google would still know it's my IP address doing the asking. (?)

      Cheers,

        'Host your own', the second item on the linked page describes hosting your own instance of Open-Elevation and how to acquire the dataset you want. That way nobody can see what queries you're making.

Re: approximating geological problems with highway data
by Anonymous Monk on Jan 25, 2023 at 11:33 UTC
Re: approximating geological problems with highway data
by Anonymous Monk on Jan 25, 2023 at 11:47 UTC
    GPS::Point
    use GPS::Point; my $gps = GPS::Point->new(lat => $lat1, lon => $lon1); my $dist = $gps->distance($lat2, $lon2);
Re: approximating geological problems with highway data
by Anonymous Monk on Jan 25, 2023 at 18:51 UTC
    I can't see your code because Gitlab says "You need to sign in or sign up before continuing." and "By signing in you accept the Terms of Use and acknowledge the Privacy Policy and Cookie Policy." Gitlab also rejects the use of what they consider outdated browsers, which is ridiculous, and not their decision to make.
        If that's in reference to https://gitlab.com/perlmonks/betelgeuse/-/blob/main/2.millcreek.pl then don't worry about it. I have signed in to gitlab and that script is a 404 anyway. In fact it seems that even the perlmonks user is 404.

        I thought I was riding off into the sunset as a successful gitlab user. Not so fast...

        All of my visibility is private. I find myself unable to select anything other than private. I wondered whether I am the owner, and I am. I see this but do not understand it:

        Change Project visibility to either Private, Internal, or Public. The visibility setting for a project must be at least as restrictive as the visibility of its parent group.

        I also have to wonder out loud if this is when the hosting must not be gratis, but I don't find evidence for needing to find some bursar now, so I'm kind of just stuck. (?)

      I can't see your code because Gitlab...

      Nuts. I apologize for the dry link. I don't mean to be coy about posting source, but rather more mindful of the initial S in SSCCE. It's not punitively long:

      Source:

      Cheers,

        In sub init_event { ... }
        { "name" => "snowville", "date" => $date_str, "time" => $time_str, "location" => { lon => -112.7105234, lat => 41.9655701 }, 42.152429, -112.9842191 },

        Is the 42.152429, -112.9842191 key/value pair from the posted code perhaps a tyop? It's syntactically correct, but looks semantically fishy. Looks like it escaped from the following event record anonymous hash. (Update: OTOH, it looks quite harmless since you're unlikely to ever encounter a "42.152429" key in your code. :)

        Also, if init_event() isn't just stub code for the SSCCE, a more succinct way to write it would be

        c:\@Work\Perl\monks>perl use strict; use warnings; use Data::Dump qw(dd); sub init_event { # define unique event records as anonymous hash refs. my $ar_events = [ { "name" => "Boise", "location" => { lon => -116.2, lat => 43.61 }, }, { "name" => "near sublett", "location" => { lon => -113.2104084, lat => 42.3278114 }, }, { "name" => "snowville", "location" => { lon => -112.7105234, lat => 41.9655701 }, }, { "name" => "juniper", "location" => { lon => -112.9842191, lat => 42.152429 }, }, # and so on... ]; # add standard date/time to each event record. my $date_str = "2021-10-14"; my $time_str = "03:22:31"; @{$_}{ qw(date time) } = ($date_str, $time_str) for @$ar_events; return $ar_events; } my $ar_ev = init_event(); dd $ar_ev; ^Z [ { date => "2021-10-14", location => { lat => "43.61", lon => "-116.2" }, name => "Boise", "time" => "03:22:31", }, { date => "2021-10-14", location => { lat => "42.3278114", lon => "-113.2104084" }, name => "near sublett", "time" => "03:22:31", }, { date => "2021-10-14", location => { lat => "41.9655701", lon => "-112.7105234" }, name => "snowville", "time" => "03:22:31", }, { date => "2021-10-14", location => { lat => "42.152429", lon => "-112.9842191" }, name => "juniper", "time" => "03:22:31", }, ]


        Give a man a fish:  <%-{-{-{-<

        use Math::Trig qw[:pi deg2rad rad2deg];

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11149859]
Approved by GrandFather
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others pondering the Monastery: (6)
As of 2024-03-28 14:05 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found