Beefy Boxes and Bandwidth Generously Provided by pair Networks
Come for the quick hacks, stay for the epiphanies.
 
PerlMonks  

Calculating the crc checksum of a file using perl?

by AGhoulDoingPerl (Sexton)
on Jul 09, 2011 at 02:54 UTC ( [id://913462]=perlquestion: print w/replies, xml ) Need Help??

AGhoulDoingPerl has asked for the wisdom of the Perl Monks concerning the following question:

I know that there is a module called Digest that can do some checksome stuffs including SHA-1 & MD5, but it seems that this module requires an extra installation of a CPAN module "Digest::CRC" for the crc part :(

But, since calculating crc-checksome is a so common task now-a-days, I wonder that there may be a more default way -- means no extra installation -- of doing it with perl.

Is there any easier way to get crc-checksome other than using the Digest module?

Replies are listed 'Best First'.
Re: Calculating the crc checksum of a file using perl?
by davido (Cardinal) on Jul 09, 2011 at 03:27 UTC
      I'll check that module out. Thanks:)
Re: Calculating the crc checksum of a file using perl?
by BrowserUk (Patriarch) on Jul 09, 2011 at 17:37 UTC

    If you are intending to use CRC32 to validate files against pre-existing checksums -- say those produced by a crc32 command for example -- then be sure to verify that whatever perl implementation you use matches. There are several (6) different polynomials used in various CRC32 standards.

    If you are going to be using it to detect differences in files -- and you are producing all the checksums -- then I strongly advise using Adler32 rather than CRC32. The former has a much lower collision rate than the latter, and is faster to calculate.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      Please keep in mind Adler is only good for LONG messages (large files). If you manipulate files of a few bytes it's a bad idea because most of the 32 bits will be unused.
      OK... Thank you for your tip
        Is there a more efficient way to get a checksum of all files in a directory?
        sub print_dir { my $dir_name = shift; print "DIR: $dir_name\n"; opendir ( my $dir_h , "$dir_name") or die "Unable to open dir :$dir +_name: $!\n"; while ( my $path = readdir($dir_h) ) { next if ($path eq "." or $path eq ".."); my $full_path = "$dir_name/$path"; if ( -l $full_path) { my $linkref = readlink($full_path); print " LINK: $full_path ==> $linkref\n" if $opt_debug; push(@bom_data, { Type => "symlink", Name => "$full_path", LinkR +ef => "$linkref" }); } elsif ( -d $full_path ) { push(@bom_data, { Type => "dir", Name => "$full_path" }); print_dir ( $full_path ); } else { my $ctx = Digest::Adler32::XS->new(); my $file_h = IO::File->new($full_path, "r"); die "? Error: Unable to open $full_path for read: $!" if (not de +fined($file_h)); binmode($file_h) || "? Error: Unable to set binmode on $file_h: +$!\n"; $ctx->addfile($file_h); $file_h->close; my $cksum = $ctx->hexdigest; print " FILE: $full_path ==> $cksum\n" if $opt_debug; push(@bom_data, { Type => "file", Name => "$full_path", Checksum + => "$cksum"}); } } close $dir_h; }
        Thanks for any suggestions.
Re: Calculating the crc checksum of a file using perl?
by kcott (Archbishop) on Jul 09, 2011 at 04:00 UTC

    In an emergency, a band-aid like the following may suffice but be aware this has many problems: non-portable, no error checking, assumptions about checksum program and target file, and so on. As already stated, the work's been done so the easiest solution is to use what's offered on CPAN.

    $ perl -E 'say +(split q{ } => qx{cksum compmesg.txt})[0];' 2782145713

    -- Ken

      So we got no built-in functions nor a core module for crc-checksome?
      That hurts....
      (But then, how come SHA-1 and MD5 do have a core module for them?,
      they are a lot less popular than crc's for sure:()

        So we got no built-in functions nor a core module for crc-checksome? That hurts.... (But then, how come SHA-1 and MD5 do have a core module for them?, they are a lot less popular than crc's for sure:()

        If you really wish to know, you should ask the perl-5-porters, its their decision :)

        Others who want more stuff with their perl than comes with CORE make their own perl distribution, like Perl::Dist::Strawberry

        I'm curious, why do you think crc is popular? Sure, your harddrive/os/filesystem/browser... use it, but they use md5 and sha1 too

Re: Calculating the crc checksum of a file using perl?
by Anonymous Monk on Jul 09, 2011 at 03:22 UTC

    But, since calculating crc-checksome is a so common task now-a-days, I wonder that there may be a more default way -- means no extra installation -- of doing it with perl.

    Yes, its called reinventing the wheel.

    Yes, even you can use CPAN

    Is there any easier way to get crc-checksome other than using the Digest module?

    No. Somebody else already did all the work, how can it get easier?

      Yes, its called reinventing the wheel.

      I think you misunderstood my question. I'm asking whether there is a crc checking functionality built-in into the perl core, or core modules.

      No. Somebody else already did all the work, how can it get easier?

      You call them "built-in funtions" and "core modules" .

      Come on, man. I thought there could be a built in fuction or a core module that can deal with crc-stuff since it is so common task now a day. You don't need to give me a lecture on how to use cpan nor on why reinventing will sucks:)

        Usually when there's a well-tested module on CPAN that I would like to use, and upon attempting to install it I find dependencies that I wasn't expecting (C headers, xml libraries, etc.), my next step is to dig into figuring out how to get those dependencies installed too. I figure that if a couple thousand other people (or more) find it useful, and have figured out how to get it installed and operating properly, it's probably not beyond me either, at least with a little help and advice from friends here from time to time. That's not to say there haven't been times where I just say, "This is not worth it because there are other solutions that are less demanding." But that is usually when another solution is evident.

        I've seen a great deal of patience here from other Monks when the question is, "I'm having difficulty installing this, here's what I've tried, and here's where I am." And I see a great deal of impatience around here with, "I can't install it because it requires xyz." That's like saying, "I can't install a faucet in my bathroom sink because I don't have a wrench. ...ok, so let's figure out how to get a wrench here. ;) Sometimes the response is, "Well, I don't want a wrench. I just want a faucet." Then the grumbling begins.


        Dave

        Come on, man. I thought there could be a built in fuction or a core module that can deal with crc-stuff since it is so common task now a day.

        There are many common tasks, not everything is in core....

        CRC site:perldoc.perl.orghttp://perldoc.perl.org/Compress/Zlib.html

        You don't need to give me a lecture on how to use cpan nor on why reinventing will sucks:)

        Well :D two sentences don't make a lecture, though it does appear you need it :)

Re: Calculating the crc checksum of a file using perl?
by Tux (Canon) on Jul 09, 2011 at 15:02 UTC

    I don't know if this suffices, but there is some less obvious extremely core way using unpack:

    my $sum = unpack "%64A*", $string;

    See $ perldoc packtut for more details. (under the head "Doing sums)


    Enjoy, Have FUN! H.Merijn
      As far as I know, what unpack does with "%" flag is just suming up the values and then modulate. CRC algorithm uses polynomial division and bitwise shifting instead of just summing things up, so I think they are quite different.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://913462]
Approved by ww
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others perusing the Monastery: (4)
As of 2024-03-29 07:57 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found