Exchanging md5s between Perl and C

damian45 has asked for the wisdom of the Perl Monks concerning the following question:

hi monks

I have a C app and a Perl app that I want to be able to exchange md5s for the same string data.

I've noticed that Digest::MD5's implementation of the standard rfc1321 version, produces different md5s that most other rfc1321 implementations (like those off http://userpages.umbc.edu/~mabzug1/cs/md5/md5.html) .

For example most C md5 implementations I've tried give 60b725f10c9c85c70d97880dfe8191b3

for 'a' (ex echo 'a' | md5sum)

Digest::MD5's implementation gives

0cc175b9c0f1b6a831c399e269772661

I've seen this behavoir on 64 and 32 bit machines (this is linux platform).
What's interesting is in t/md5-aaa.t there's a note that the test data was originally produced using the C system utility md5sum

# This data was generated with:
#
# perl -e 'for (1..256) { system("md5sum --string=" . ("a" x $_)); }'
#
__END__
0cc175b9c0f1b6a831c399e269772661  "a"
[download]

that utility does not give that md5 anymore however, it gives

60b725f10c9c85c70d97880dfe8191b3

It seems the perl implementation is the non standard one at this point. Trying to avoid getting heavily into the perl XS implementation to try and figure out how to make it produce the same md5s as all the other implementations.

Anyone know if there's anything simple to tweak in it, to get it to produce the more common md5s?

I've experimented a bit with linking against MD5.so and calling things direct but that was getting quite ugly. I don't really care which md5 I get, they just need to match between perl and my C app (without the user having to install a custom perl module or something).

cheers for any help

Comment on Exchanging md5s between Perl and C Download Code

Replies are listed 'Best First'.
Re: Exchanging md5s between Perl and C by jettero (Monsignor) on Oct 28, 2010 at 18:58 UTC
It seems to me it's your line endings... echo -n "a" \| md5sum gives me the same thing as md5_hex("a"). "a\n" and "a" should not hash to the same thing. "a" should be: 0cc175b9c0f1b6a831c399e269772661 "a\x0a" should be: 60b725f10c9c85c70d97880dfe8191b3 "a\x0d\x0a" should be: 0c629e1d3acda389c88a213374ec62fc It's also worth noting that Digest::MD5 really is C, as much as /usr/bin/md5sum is: MD5.xs.	[reply]
Re^2: Exchanging md5s between Perl and C by bart (Canon) on Oct 28, 2010 at 19:14 UTC
I'm pretty sure that is indeed the cause. Here's a test using md5sum from Perl: `open OUT, '>', 'tempfile'; print OUT 'a'; close OUT; system('md5sum', '-b', 'tempfile');` [download] Result: `0cc175b9c0f1b6a831c399e269772661 *tempfile` [download] So yeah, `md5sum` and Digest::MD5 agree. It's the OP's other tests that were flawed.	[reply] [d/l] [select]
Re^3: Exchanging md5s between Perl and C by damian45 (Novice) on Oct 28, 2010 at 19:54 UTC
ahhk! thank you very much, yes that was it. I previously had an issue with 32/64 bit and after going round and round until fixed, somewhere along the line I stared chasing my tail overlooking that minor but oh so important detail. cheers for you time! Glad it was as simple as that	[reply]


Perl: the Markov chain saw
	PerlMonks