http://qs321.pair.com?node_id=912014

adroc has asked for the wisdom of the Perl Monks concerning the following question:

Hello, I am attempting to do a search and replace of MD5 hashes contained in a file but am having issues with the special characters contained within the hash. For example: #Just generating random hashes for this example and storing them in variables.

$hash1= `makepasswd --clearfrom=$path/keys/temppw --crypt-md5 --verbos +e | grep Password=| cut -d"=" -f3`; chomp($hash1); $hash2=`makepasswd --clearfrom=$path/keys/temppw2 --crypt-md5 --verbos +e | grep Password=| cut -d"=" -f3`; chomp($hash2); #I need to update the hashes over ssh because I will be doing this on +remote machines, so I'm doing the following to test on the local mach +ine for now.... system("ssh -o StrictHostKeyChecking=no root\@localhost /usr/bin/perl +-p -i -e s/$hash2/$hash1/ee hashfile") == 0 or @errors=(@errors,"\n$server hash update error: $?\n");

Doing the above generates an error similar to this. Bareword found where operator expected at -e line 1, near "s/$/7OWwo4h/FWHShP3" syntax error at -e line 1, near "s/$/7OWwo4h/FWHShP3" Execution of -e aborted due to compilation errors.

Some example hashes..........

$1$UGXeHYG/$c8iQxpnMsSSCWlEPo0utZTxDjc.qCyTD59PjclG3iKW8p.ZRCySuul8mfl2xffClNH6m/0MqpbIaaHCsI/ziY0

$1$mOdDtc/f$iKpqV8NR7mfM9DrJ7ebS1/

What am I doing wrong here?

Thanks in advance...

Replies are listed 'Best First'.
Re: Find and replace MD5 hash from file
by philipbailey (Curate) on Jun 29, 2011 at 19:07 UTC

    I can see two problems with your system command: (1) parts of your hashed password strings contain $ signs which will be interpreted by the shell as environment variables. (2) If you then single quote your s/// expression to protect from shell interpolation, those same $ signs are interpreted as variables by Perl; the slashes are also interpreted by Perl. So replace this:

    system("ssh -o StrictHostKeyChecking=no root\@localhost /usr/bin/perl +-p -i -e s/$hash2/$hash1/ee ha +shfile")

    with this:

    system("ssh -o StrictHostKeyChecking=no root\@localhost /usr/bin/perl +-p -i -e 's/\Q$hash2\E/\Q$hash1\E/' ha +shfile")

      If you then single quote your s/// expression to protect from shell interpolation

      Adding single quotes does not protect from shell interpolation.

      That will fail if $hash1 or $hash2 can contain «'», I don't think that's possible, but I don't know for sure.

      That's why I use

      $s =~ s/'/'\\''/g; return "'$s'";

      instead of

      return "'$s'";

      Update: philipbailey has since pointed out that the hash cannot contains quotes. While you can't context text to a shell literal using the second snippet in general, it appears to be safe in this particular circumstance.

        We've had this conversation before, ikegami. In this case the OP's base64-encoded strings never contain single quotes.

        Update: this is clarified in later nodes in this thread. The OP's output largely consists of base-64 strings together with 4 other (non-quote) characters, explained below.

Re: Find and replace MD5 hash from file
by ikegami (Patriarch) on Jun 29, 2011 at 19:15 UTC

    First, you need to convert the text into a regex pattern. Then you need to convert the Perl program into a sh string literal.

    sub text_to_shell_lit(_) { return $_[0] if $_[0] =~ /^[a-zA-Z0-9_\-]+\z/; my $s = $_[0]; $s =~ s/'/'\\''/g; return "'$s'"; } my $pat = quotemeta($hash2); my $repl = quotemeta($hash1); my $perl_code = 's/^'.$pat.'$/'.$repl.'/'; my $remote_cmd = join ' ', map text_to_shell_lit, '/usr/bin/perl' => ( '-i', '-p', '-e' => $perl_code, 'hashfile', ); my $ssh_cmd = join ' ', map text_to_shell_lit, 'ssh' => ( '-o' => 'StrictHostKeyChecking=no', 'root@localhost', '--', $remote_cmd, ); system($ssh_cmd);

    That can be shortened to

    sub text_to_shell_lit(_) { return $_[0] if $_[0] =~ /^[a-zA-Z0-9_\-]+\z/; my $s = $_[0]; $s =~ s/'/'\\''/g; return "'$s'"; } system(q{ssh -o StrictHostKeyChecking=no root@localhost }.text_to_shel +l_lit(q{/usr/bin/perl -i -pe}.text_to_shell_lit("s/^\Q$pat\E\$/\Q$rep +l\E/").q{ hashfile}));

    But I wanted to present the safe but wordy version. If you do it in steps, you're more likely to get it right then if you start with this short but far more complex version.

    Alternatively, you could use the multiple argument form of system instead of forming a shell command since don't need the shell.

    my $pat = quotemeta($hash2); my $repl = quotemeta($hash1); my $perl_code = 's/^'.$pat.'$/'.$repl.'/'; my $remote_cmd = join ' ', map text_to_shell_lit, '/usr/bin/perl' => ( '-i', '-p', '-e' => $perl_code, 'hashfile', ); system( 'ssh' => ( '-o' => 'StrictHostKeyChecking=no', 'root@localhost', '--', $remote_cmd, ), );

    By the way, I suspect sudo is a more common way of escalating to root privileges.


    By the way,

    s/.../.../ee
    is the same as
    s/.../eval "..."/e

    which is very very very wrong. $hash1 does not contain Perl code to execute, so the /e is wrong, and if it was Perl code to execute, it doesn't return Perl code to execute, so the eval is wrong too.

      Thanks for the help, your solution works great but I forgot to mention that the file contains more then just the hashes.

      For example user:$1$7FWHShP3$QzHTcsaII5leWMSv6weND7vSS1KMvcsjfYpiDZxc.TCAVR6XUdLaLtUBVTTwPFXZBKs52DRzW1UKFZ5JB27yI0:15152:0:99999:7:::

      What needs to be modified in order for something like the above to work?

      Thanks again

        I switched from «s/.../.../» to «s/^...$/.../» to avoid possibly matching the wrong things. I incorrectly guessed your file format. You could revert to using «s/.../.../», or you could use the more restrictive «s/:...:/:...:/».