Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re^2: CPAN modules for inspecting a Perl distribution?

by Tux (Canon)
on Oct 14, 2018 at 12:23 UTC ( [id://1224002]=note: print w/replies, xml ) Need Help??


in reply to Re: CPAN modules for inspecting a Perl distribution?
in thread CPAN modules for inspecting a Perl distribution?

I like your approach, but it gave me a lot of doubles and it took ages. Here's my stab at it

#!/usr/bin/env perl use 5.18.2; use warnings; use Digest::SHA qw(sha256_hex); use Devel::Examine::Subs; use SourceCode::LineCounter::Perl; use String::CommonPrefix qw(common_prefix); use Data::Peek; use File::Find; use Cwd; my @loc = @ARGV; @loc or @loc = (getcwd); my %file; my %seen; # Find all .pm and .pl files. (no .PL or .xs) find (sub { m/\.p[ml]$/ or return; m/^\./ || -l and return; # No symlinks or dot-files $File::Find::dir =~ m{\b(?:sandbox|tmp)(?:/|$)} and return; $seen{(lstat)[1]}++ and return; # dup on inode my $sha = sha256_hex ($_); $seen{$sha}++ and return; # dup on SHA $file{$File::Find::name} = $sha; }, @loc); my $pfx = common_prefix (keys %file); my %stats; for my $file (sort keys %file) { my $name = $file =~ s/^$pfx//r; my $flc = SourceCode::LineCounter::Perl->new; $flc->count ($file); $stats{$name} = { name => $name, file => $file, lines => $flc->total, blank => $flc->blank, doc => $flc->documentation, cmnt => $flc->comment, loc => 0, nobj => 0, }; # Now analyse subs my $des = Devel::Examine::Subs->new (file => $file); unless ($des) { say STDERR "Cannot analyse $name"; next; } my $objs = $des->objects; unless ($objs) { say STDERR "File $name has no objects"; next; } foreach my $obj (@{$des->objects}) { $stats{$name}{nobj}++; my $slc = SourceCode::LineCounter::Perl->new; $slc->count (\do { join "\n" => @{$obj->code} }); my $loc = $slc->code; $stats{$name}{loc} += $loc; push @{$stats{$name}{obj}}, { name => $obj->name, loc => $loc, lines => $slc->documentation, cmnt => $slc->comment, blnk => $slc->blank, doc => $slc->documentation, }; } } delete $SIG{__WARN__}; delete $SIG{__DIE__}; my @sum; say " subs lines LOC doc blank File"; say "----- ------- ------- ------ ------ ---------------------------- +----------"; for (sort { $b->{nobj} <=> $a->{nobj} || $b->{loc} <=> $a->{loc} } val +ues %stats) { printf "%5d %7d %7d %6s %6d %s\n", $_->{nobj}, $_->{lines}, $_->{loc}, $_->{doc} + $_->{cmnt}, $_ +->{blank}, $_->{name}; $sum[0] += $_->{nobj}; $sum[1] += $_->{lines}; $sum[2] += $_->{loc}; $sum[3] += $_->{doc} + $_->{cmnt}; $sum[4] += $_->{blank}; } say "----- ------- ------- ------ ------ ---------------------------- +----------"; printf "%5d %7d %7d %6s %6d\n", @sum;

Thanks for the motivation :)


Enjoy, Have FUN! H.Merijn

Replies are listed 'Best First'.
Re^3: CPAN modules for inspecting a Perl distribution?
by nysus (Parson) on Oct 14, 2018 at 14:49 UTC

    That's a sweet looking report. I'm going to steal that. Regarding slowness, I timed my script vs. yours:

    My script: real 0m12.515s user 0m11.919s sys 0m0.505s Your script: real 0m15.987s user 0m15.229s sys 0m0.660s

    This was done on the Rex v 1.6.0 module, which is pretty hefty. When you say "doubles," do you mean that some files were reported twice?

    $PM = "Perl Monk's";
    $MCF = "Most Clueless Friar Abbot Bishop Pontiff Deacon Curate Priest";
    $nysus = $PM . ' ' . $MCF;
    Click here if you love Perl Monks

      Doubles: it found files in blib as well as the original, of which some were symlinks to files in lib (or similar). My first run was faster than yours, but I noticed that when you add the numbers from the objects of the analysis, the __END__ docs and such are not taken into account for the whole file, which is why I analyze twice: once for the whole file (lines of docs and comments and total lines in the file) and once per object (lines of code). Another slowdown is because of the SHA256, as files might get copied (no link or symlink).

      It is an interesting project though. Have you tried it on your site_perl? For me it crashes halfway with some weird error (after running for 15 minutes), but then again, I do have 2118 modules installed, resulting in 12151 .pm files and 900 .pl files. That'll take a while to complete.


      Enjoy, Have FUN! H.Merijn

        Ah, it sounds like you are using it on all modules already installed on the machine. I'm only using it on a single distribution downloaded with cpanm --look which just downloads the raw module from cpan without installing it. So I have no blib directory. Yeah, I noticed the SHA256 in your code. I'm sure that's what makes it a bit slower. Not a big deal. I was just trying to figure out why mine was slower for you. Sounds like my code was probably slower for you because it was analyzing many files twice.

        $PM = "Perl Monk's";
        $MCF = "Most Clueless Friar Abbot Bishop Pontiff Deacon Curate Priest";
        $nysus = $PM . ' ' . $MCF;
        Click here if you love Perl Monks

        I was thinking of turning this into a simple CPAN distribution, maybe Distribution::Inspector or something like that. It would provide a command like cpan-inspect which would download the module and issue a report on the distribution for someone wishing to assess the module. It might provide useful information that could help a developer more quickly determine the health/quality of a distribution.

        $PM = "Perl Monk's";
        $MCF = "Most Clueless Friar Abbot Bishop Pontiff Deacon Curate Priest";
        $nysus = $PM . ' ' . $MCF;
        Click here if you love Perl Monks

Re^3: CPAN modules for inspecting a Perl distribution?
by nysus (Parson) on Oct 14, 2018 at 15:16 UTC

    Curious to know what these lines are for:

    delete $SIG{__WARN__}; delete $SIG{__DIE__};

    $PM = "Perl Monk's";
    $MCF = "Most Clueless Friar Abbot Bishop Pontiff Deacon Curate Priest";
    $nysus = $PM . ' ' . $MCF;
    Click here if you love Perl Monks

      Try a warn inside the main for loop or after it. Those handlers are "stolen" inside one of the analyzers and are still so after the loop.


      Enjoy, Have FUN! H.Merijn

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1224002]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others goofing around in the Monastery: (2)
As of 2024-04-26 04:55 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found