http://qs321.pair.com?node_id=11149191

Aldebaran has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

Trying to read the tea leaves with:

cpan[1]> install XML::DoubleEncodedEntities + Reading '/root/.cpan/Metadata' Database was generated on Thu, 29 Dec 2022 05:17:02 GMT Running install for module 'XML::DoubleEncodedEntities' Warning: checksum file '/root/.cpan/sources/authors/id/D/DC/DCANTRELL/ +CHECKSUMS' not conforming. The cksum does not contain the key 'cpan_path' for 'XML-DoubleEncodedE +ntities-1.1.tar.gz'. Proceed nonetheless? [no]

I'm trying to understand this, and I would say that this seems not to be the usual problem of not having installed the headers that this needs. I ran this command seemingly successfully:

sudo apt install libxml2-dev

, but ended up with the same warning before as after.

As I read it, I would think to not trust the source, but "CANT" seems to be the author, so that shoe doesn't fit either.

Q1) What does this message mean? It's a prereq for Astro::SIMBAD::Client.

The answer could also be that the the SIMBAD project has sunseted in significant ways, as indicated by the first paragraph at Astro::SIMBAD::Client:

As of release 0.027_01 the SOAP interface is deprecated. The Universit +y of Strasbourg has announced at https://cds.u-strasbg.fr/resources.g +ml?id=soap that this interface will not be maintained after April 1 2 +014, and that this interface will be stopped on December 31 2018.

Fishing for tips,

Replies are listed 'Best First'.
Re: cpan warning for XML::DoubleEncodedEntities
by marto (Cardinal) on Dec 29, 2022 at 22:22 UTC
      You didn't say which version of cpan you have, super search is your friend -> Re^4: pod2html: link (L<...>) formatting code (timeloop)

      Thx, marto. I know that's not gonna quote well, but your link led to correction: choroba's response in Re^5: pod2html: link (L<...>) formatting code (timeloop). Essentially, the fix is:

      rm /home/ken/.cpan/sources/authors/id/?/??/*/CHECKSUMS

      Regarding Super Search, I haven't been able to make useful queries with it lately. Is that just me being Super Rusty?

      Continuing:

      root@laptop:~/.cpan# pwd /root/.cpan root@laptop:~/.cpan# ls sources/authors/id/?/??/*/CHECKSUMS sources/authors/id/B/BL/BLIAKO/CHECKSUMS ... sources/authors/id/C/CO/CORION/CHECKSUMS ... sources/authors/id/D/DA/DAGOLDEN/CHECKSUMS ... sources/authors/id/D/DC/DCANTRELL/CHECKSUMS ... sources/authors/id/D/DR/DROLSKY/CHECKSUMS ... sources/authors/id/M/MI/MIYAGAWA/CHECKSUMS ... sources/authors/id/R/RU/RURBAN/CHECKSUMS sources/authors/id/S/SA/SALVA/CHECKSUMS

      Those are certainly the usual suspects, and when I have a problem implementing software with such names on it, I strongly suspect that the problem is local, either on my machine or between my ears. Proceeding:

      root@laptop:~/.cpan# rm sources/authors/id/?/??/*/CHECKSUMS root@laptop:~/.cpan# ls sources/authors/id/?/??/*/CHECKSUMS ls: cannot access 'sources/authors/id/?/??/*/CHECKSUMS': No such file +or directory root@laptop:~/.cpan# cpan

      Alles klar...und...

      cpan[1]> install Astro::SIMBAD::Client

      I went over this install very closely, reading many of the sources:

      The simbadc script is a simple wrapper for Astro::SIMBAD::Client, whic +h may be used interactively or as a Unix filter. >>>> NOTICE <<<< As of version 0.032_01 the simbadc script is installed by default. If you do not wish it installed. run this script with the -n option. The +-y option is still accepted for backward compatibility. ... Running Build test for WYANT/Astro-SIMBAD-Client-0.046.tar.gz t/basic.t .... 1/? # http://simbad.u-strasbg.fr/: SIMBAD4 1.8 - 2022-1 +2 t/basic.t .... ok #I verified this one in the debugger t/release.t .. ok t/simple.t ... ok t/vo.t ....... ok t/yaml.t ..... ok All tests successful. Files=5, Tests=13, 65 wallclock secs ( 0.07 usr 0.01 sys + 1.59 cusr + 0.23 csys = 1.90 CPU) Result: PASS ... Installing /usr/local/share/perl/5.30.0/Astro/SIMBAD/Client.pm Installing /usr/local/share/perl/5.30.0/Astro/SIMBAD/Client/WSQueryInt +erfaceService.pm Installing /usr/local/man/man3/Astro::SIMBAD::Client.3pm Installing /usr/local/bin/simbadc WYANT/Astro-SIMBAD-Client-0.046.tar.gz ./Build install -- OK cpan[2]> quit

      And then, and then, and gentleman and then:

      root@laptop:~/.cpan# perl -MAstro::SIMBAD::Client -le '$s=Astro::SIMBA +D::Client->new;@q=split /\n/, $s->url_query(id => Ident => "$ARGV[0]" +, NbIdent => 1);@r=grep/ICRS/,@q;print for @r' aldebaran Coordinates(ICRS,ep=J2000,eq=2000): 04 35 55.23907 +16 30 33.4885 (Op +t ) A [7.38 5.70 90] 2007A&A...474..653V root@laptop:~/.cpan#

      Q1) How can I repeat the tests in a comprehensive fashion?

      I stepped through one of the tests in the debugger:

      DB<1> n + main::(basic.t:8): require_ok 'Astro::SIMBAD::Client' main::(basic.t:9): or BAIL_OUT q{Can't do much testing if can't + load module}; DB<1> n + ok 1 - require Astro::SIMBAD::Client; main::(basic.t:12): my $smb = Astro::SIMBAD::Client->new (); DB<1> n + main::(basic.t:14): ok $smb, 'Instantiate Astro::SIMBAD::Client' main::(basic.t:15): or BAIL_OUT "Test aborted: $@"; DB<1> n + ok 2 - Instantiate Astro::SIMBAD::Client main::(basic.t:17): is $smb->get( 'debug' ), 0, 'Initial debug sett +ing is 0'; DB<1> n + ok 3 - Initial debug setting is 0 main::(basic.t:19): $smb->set( debug => 1 ); DB<1> p debug + debug DB<2> p $smb->get('debug') + 0 DB<3> n + main::(basic.t:21): is $smb->get( 'debug' ), 1, 'Able to set debug +to 1'; DB<3> n + ok 4 - Able to set debug to 1 main::(basic.t:23): $smb->set( debug => 0 ); DB<3> p $smb->get('debug') + 1 24 25 { 26: my $scheme = Astro::SIMBAD::Client::_is_scheme_valid( 27 $ENV{ASTRO_SIMBAD_CLIENT_SCHEME} ) ? 28 'http' : 29: lc $ENV{ASTRO_SIMBAD_CLIENT_SCHEME}; DB<6> n + main::(basic.t:26): my $scheme = Astro::SIMBAD::Client::_is_sch +eme_valid( main::(basic.t:27): $ENV{ASTRO_SIMBAD_CLIENT_SCHEME} ) ? main::(basic.t:28): 'http' : DB<6> c + ok 5 - Default scheme is 'http' # http://simbad.u-strasbg.fr/: SIMBAD4 1.8 - 2022-12 1..5 Debugged program terminated. Use q to quit or R to restart, use o inhibit_exit to avoid stopping after program termination, h q, h R or h o to get additional info. DB<6>

      I lengthened Tom's one-liner to a lexical script:

      #!/usr/bin/perl -w use v5.030; use Astro::SIMBAD::Client; use Data::Dumper; my $s = Astro::SIMBAD::Client->new; my @q = split /\n/, $s->url_query( id => Ident => "$ARGV[0]", NbIdent +=> 1 ); my @r = grep /ICRS/, @q; print Dumper(@r); #print Dumper(@q);

      Output with various choices of ARGV, and a few (edited) runs dumping @q:

      Finally, while I am writing up this simbad install project with what I'm calling success, the site and the client that gets it seem to be going just fine SOAP or no SOAP. (I still don't know what that is.) marto was completely correct that the interface had nothing to do with the problem reported. This is how you get this info with your browser:

      https://simbad.cds.unistra.fr/simbad/sim-id?Ident=betelgeuse&submit=su +bmit+id

      or betelgeuse.

      Thanks all for comments,

        [Minor attribution issue: s/link led to Corion's response/link led to choroba's response/ (2nd paragraph)]

        After successful "rm sources/authors/id/?/??/*/CHECKSUMS", I'd suggest the first thing to do would be to upgrade CPAN. As I indicated in "Re: cpan warning for XML::DoubleEncodedEntities", the vulnerability exists in v2.28; it's fixed in v2.29 (I left some notes about that in "Re^8: pod2html: link (L<...>) formatting code (timeloop)"); the current version is v2.34.

        "Q1) How can I repeat the tests in a comprehensive fashion?"

        For this, I'd generally use the prove utility.

        On the Astro::SIMBAD::Client page, there's a direct link to the tarball, there's also a link to the GitHub repository if you prefer that. Unpack the tarball and run the usual "perl Makefile.PL" followed by "make". At this point, the next step would normally be "make test"; instead of that, run prove here.

        There are a host of options. Some suggestions:

        # Run all tests with verbose output (possibly very lengthy) $ prove -vb t/*.t # Run individual test scripts, e.g. $ prove -vb t/basic.t

        You can run those tests as many times as you want.

        I can't see a "Q2) ...?". :-)

        — Ken

        > I lengthened Tom's one-liner to a lexical script:

        Tom didn't do that, it was just another perl hacker :)

        Also, your "Output with various choices" wrt "Mintaka" suggests it didn't work, but it does. I guess a few extra lines got pasted cause there's two "$VAR1".

Re: cpan warning for XML::DoubleEncodedEntities
by kcott (Archbishop) on Dec 30, 2022 at 02:42 UTC

    G'day Aldebaran,

    "Fishing for tips"

    Take a look at "Perl : Security Vulnerabilities". In particular, CVE-2020-16156 ("CPAN 2.28 allows Signature Verification Bypass."), which (at the time of writing) is at the top of the list.

    I seem to recall that, if you follow enough links from ++marto's "Super Search result", you'll get to that CVE information, eventually. I couldn't find it after a brief search, so I've posted it separately.

    That whole "Perl : Security Vulnerabilities" page may be of general interest to you; and, I imagine, quite a few other monks.

    — Ken

Re: cpan warning for XML::DoubleEncodedEntities
by Anonymous Monk on Dec 31, 2022 at 17:59 UTC

    It seems to me that I have gotten the checksum error from a local file corruption. My response was to go to CPAN's working directory (probably ~/.cpan) and delete file Metadata and directories build/ and sources/. After I did so the problem went away.

    As for Astro::SIMBAD::Client, I am not sure of the situation about SOAP, and I am the author of that module. I know they announced the deprecation of SOAP support, but the deadline passed and it was still there. I communicated with them and they confirmed it was going away, but to the best of my knowledge it still works. Trusting my memory (a foolish thing to do) I marked all the SOAP tests TODO, added that warning, and migrated all my code that uses that interface away from SOAP.

    As for XML::DoubleEncodedEntities, it may no longer be needed, though I am reluctant to drop the dependency completely for reasons that are not all technical.

    Dave Cantrell is a star CPAN tester (to me anyway; of course they all are but he's one of the brighter ones). At one point he noticed test failures in Astro::SIMBAD::Client due to the fact that U. Strasbourg was, in fact, double-encoding their XML entities. His response was to publish XML::DoubleEncodedEntities and write me about it.

    I can look into making XML::DoubleEncodedEntities optional, though author testing both with and without optional modules is a bit of a pain. Even if I do that it will remain strongly recommended.

      Replying to myself (since I'm anonymous and can't edit the entry): A development version of Astro::SIMBAD::Client went to PAUSE a couple days ago.

      It seems to me that I have gotten the checksum error from a local file corruption. My response was to go to CPAN's working directory (probably ~/.cpan) and delete file Metadata and directories build/ and sources/. After I did so the problem went away.

      The fix comes from choroba:

      rm /home/ken/.cpan/sources/authors/id/?/??/*/CHECKSUMS

      ...

      As for Astro::SIMBAD::Client, I am not sure of the situation about SOAP, and I am the author of that module. I know they announced the deprecation of SOAP support, but the deadline passed and it was still there. I communicated with them and they confirmed it was going away, but to the best of my knowledge it still works. Trusting my memory (a foolish thing to do) I marked all the SOAP tests TODO, added that warning, and migrated all my code that uses that interface away from SOAP.

      SOAP seems not to be an issue. Still don't know what it is. (No SOAP, radio?)

      As for XML::DoubleEncodedEntities, it may no longer be needed, though I am reluctant to drop the dependency completely for reasons that are not all technical.

      I don't think you should.

      Dave Cantrell is a star CPAN tester (to me anyway; of course they all are but he's one of the brighter ones). At one point he noticed test failures in Astro::SIMBAD::Client due to the fact that U. Strasbourg was, in fact, double-encoding their XML entities. His response was to publish XML::DoubleEncodedEntities and write me about it.

      Aha..yeah I know his name by reputation of the right variety. I'm very interested in XML::DoubleEncodedEntities, and I'd like to work up a handful of things that might pass or fail. I think a lot of what we do in encodings likens itself to cooking. You undercook the encoding by not decoding it correctly, with a pool of mojibake to show for the effort. OTOH, you might overcook it, which is what I think Dave was getting at. I think bliako has raised this issue in similar contexts.

      I can look into making XML::DoubleEncodedEntities optional, though author testing both with and without optional modules is a bit of a pain. Even if I do that it will remain strongly recommended.

      I don't see a reason for that, indeed I'd like to look under the hood here a bit. Does this occur in the context of XML::Parser doing its thing?

      Cheers,