http://qs321.pair.com?node_id=1226131


in reply to Perl+PostgreSQL+GeoIP = Awesome!

NICE indeed. But TIMTOWTDI and here's one more (not better, just shorter and how I would probably have done it)

(bash + perl, Pg connects via PG* envvars)

wget --timestamping http://geolite.maxmind.com/download/geoip/database +/GeoIPCountryCSV.zip wget --timestamping http://geolite.maxmind.com/download/geoip/database +/GeoIPv6.csv.gz echo " create table geoip ( netblock cidr not null, country_code text not null, country_name text not null, constraint geoip_pk primary key (netblock) );"| psql \ && ( unzip -p GeoIPCountryCSV.zip ; gunzip -c GeoIPv6.csv.gz ) \ | perl -MNet::CIDR -ne ' chomp; my @arr = split( /\"\,\ ?\"/, $_ ); my @cidr = Net::CIDR::range2cidr(sanitize($arr[0]) . "-" . sani +tize($arr[1])); for my $subcidr (@cidr) { print $subcidr, "\t", sanitize($arr[4]), "\t", sanitize($arr +[5]), "\n"; } sub sanitize { my ($val) = @_; $val =~ s/^\"//; $val =~ s/\"$//; return $val; } ' | psql -c "copy geoip from stdin with (format csv, header false, del +imiter E'\t');"

(I get 326604 rows (in 12 sec))

Replies are listed 'Best First'.
Re^2: Perl+PostgreSQL+GeoIP = Awesome!
by cavac (Parson) on Nov 23, 2018 at 10:00 UTC

    As just noted in the update/edit in my thread starter, i just found out that we need to switch to the new GeoLite2 databases, because the ones i have used here are no longer supported. I'll post a new version in CUFP after i get it working and fully tested.

    Otherwise, nice use of the psql COPY command ;-)

    perl -e 'use MIME::Base64; print decode_base64("4pmsIE5ldmVyIGdvbm5hIGdpdmUgeW91IHVwCiAgTmV2ZXIgZ29ubmEgbGV0IHlvdSBkb3duLi4uIOKZqwo=");'