BuddhaNature has asked for the wisdom of the Perl Monks concerning the following question:
I am writing a script to run a series of perl-scripts/system commands, but I want to make sure that if something goes wrong nothing else happens. Is die the best way of going about this, or do you think there should be some form of error handling instead?
chdir("$staging_directory")
or die "Could not change directory to $staging_directory: $!\n";
system("svn", "update", "base")
or die "Could not svn update base: $!\n";
system("svn", "update", "hostgroup")
or die "Could not svn update hostgroup: $!\n";
system("svn", "update", "host")
or die "Could not svn update host: $!\n";
system("$copier_program_location", "$staging_directory", "$pre_launch_
+directory", "$copier_conf_file")
or die "Could not run copier $copier_program_location: $!\n";
system("$permissions_program_location", "-c", "$permissions_conf_file"
+)
or die "Could not run permissions program $permissions_program_loc
+ation: $!\n";
system("rsync", "-avz", "--exclude=.svn", "--delete-after", "$pre_laun
+ch_directory", "$live_directory")
or die "Could not perform rsync from $pre_launch_directory to $liv
+e_directory: $!\n";
On a side note, are system() calls the best way to call other perl scripts, or would some kind of use or sub better form? (The $blah_program_location above are perl scripts)
Thanks in advance.
-Shane
Re: Is "die" the best way to be atomic?
by dragonchild (Archbishop) on Apr 27, 2004 at 01:14 UTC
|
What you call atomic, most people call "error handling". die/eval is the standard Perl way of doing throwing and catching errors, similar to try/catch in Java. So, yes, die is what you want to do.
One more thing - you might want to look at named pipes instead of system. That way, you can parse the input of the command, if you want.
One more thing - you might want to take a look at the $? variable, which is set by the system() function. Read up on system.
One more thing - if the scripts are their own entities, it is a good thing. However, I would recommend looking at refactoring the functionality into various modules and packages, to benefit from code re-use.
One more thing - it looks like you're doing some sort of batch processing. There are very good applications that have already solved this problem, plus provide features you'll never think of. You might want to google for "batch process open source" and see what comes out. If your employer has the capital, I'd recommend either Oracle Scheduler or Control-M, for proprietary products. Stay away from Tivoli and SQL Scheduler.
------
We are the carpenters and bricklayers of the Information Age.
Then there are Damian modules.... *sigh* ... that's not about being less-lazy -- that's about being on some really good drugs -- you know, there is no spoon. - flyingmoose
| [reply] |
Re: Is "die" the best way to be atomic?
by saintmike (Vicar) on Apr 26, 2004 at 23:19 UTC
|
To save you some typing, you could possibly wrap your calls into
sub system_and_die {
system(@_) and die "system @_ failed";
}
eval {
system_and_die("svn", "update", "base");
system_and_die("svn", "update", "hostgroup");
system_and_die("svn", "update", "host");
};
if($@) {
print "Ouch! ($@)\n";
}
Updated: system and, not system or of course :). Thanks, bart. | [reply] [d/l] [select] |
Re: Is "die" the best way to be atomic?
by davido (Cardinal) on Apr 26, 2004 at 23:23 UTC
|
die is more of a friend than its name suggests. In fact, if your code needs to do some mandatory cleanup before its death, you can even trap the dies using eval or Carp and have a special death handler deal with the cleanup for you, after which time it's your choice whether you let your script return to duty or terminate execution.
If you have something going on that you want your script to complain about but not necessarily die, you can also use warn. It just spits a message to STDERR and continues on.
| [reply] |
Re: Is "die" the best way to be atomic?
by bart (Canon) on Apr 27, 2004 at 02:18 UTC
|
A bug on your behalf: system behaves opposite to most other built-in functions, in that it returns false on success. So system($foo) or die is a mistake. You can use
system($foo) and die "system() failed, error code " . ($? >> 8);
But actually the return value of system is more complex than this. You might want to check perlfunc -f system. The return value -1 in particular is quite revealing. | [reply] [d/l] [select] |
Re: Is "die" the best way to be atomic?
by castaway (Parson) on Apr 27, 2004 at 14:53 UTC
|
If by 'atomic' you mean 'do things all in one go, or not at all', then no, it isnt. To me it sounds like you want some sort of all-or-nothing transaction, in which either all the commands are complete, or none at all.
I'm not sure quite what those commands are doing, but what happens if you update the hostgroup, but fail to update the host, do you have an inconsistent system then? In which case you need to code some sort of 'undo' sub, which you can pass a value which indicates how far the script got before it encountered an error, and proceeds to undo the previous steps to return to the state the system was in before you started. And *then* it can die, or exit, or whatever..
Eg:
chdir("$staging_directory")
or undo(1);
system("svn", "update", "base")
or undo(2);
...
sub undo
{
my $state = shift;
if($state >= .. ) # highest one first
..
if($state >= 2)
{
system("svn" .. ) # undo svn/update/base command
}
if($state >= 1)
{
chdir("-"); # undo chdir command
}
die "Got to level X: $!\n";
}
.. or something.. but maybe I misunderstood what you wanted.
C. | [reply] [d/l] |
|
Firstly, I am glad someone saw threough my poorly worded question what I meant - I _am_ looking for your definition of atomic - "do it, or don't do it - don't go halfway..." Well the two main places of concern are the permissions script and the rsync - if something goes wrong with either it could be a BadThing(tm).
I like your idea of the undo() function, but am not sure it would be feasible in either case, meaning I am not sure if there would be a way to actually undo what had occurred. One way I might be able to cover myself on the rsync is to first run the rsync with the --dry-run option, and as long as all goes well run it without. Is there any kind of "dry run" in perl? Does eval or try work in that way - in that it tries to do it, but doesn't actually do it unless it is sucessful?
-Shane
| [reply] [d/l] [select] |
|
Not really, no. At least, there appear to be a few modules for doing transactional perl code, eg WhatIf and Transaction, but these are not likely to be able to rollback system calls. So you'll need to know how to do that yourself.
Seeing as the permissions call appears to be to a script you wrote yourself, (if I read the post correctly), then surely you know what that does internally, and how to roll it back? You might even make it take some sort of parameter, which makes it rollback itself.
As to the rsync, there are also some Rsync modules, but whether any support rollback/transactions I couldnt see. (I would have thought any decent backup mechanism would be atomic somehow)
C.
| [reply] |
Re: Is "die" the best way to be atomic?
by mce (Curate) on Apr 27, 2004 at 10:10 UTC
|
$rc=system(<<EOF);
set -e
ls
this_command_does_not_exist
EOF
die if $rc;
But this solution fits more in the ShellMonks :-)
PS: I don't know why, but $? is not set!!
---------------------------
Dr. Mark Ceulemans
Senior Consultant
BMC, Belgium
| [reply] [d/l] |
Re: Is "die" the best way to be atomic?
by revdiablo (Prior) on Apr 27, 2004 at 16:58 UTC
|
Looking at your program, it seems die is probably a nice way to make this whole thing atomic. I'm assuming it doesn't matter if the staging directory is in a state of disarray, as long as the rsync command is not run. Other than the issue with die's return value not being what you seem to expect, your code looks ok.
That said, there are some minor stylistic problems I see with your code. Here's how I would have written the lines you pasted, if I were using Perl at all (this thing reeks of a shell script, though if it's a learning experience, then hopefully I'm helping you learn a bit):
chdir $staging_dir or die "chdir: $!";
system qw(svn update base) and die "up base: $!";
system qw(svn update hostgroup) and die "up hostgroup: $!";
system qw(svn update host) and die "up host: $!";
system $copier_prog, $staging_dir,
$pre_launch_dir, $copier_conf and die "copy: $!";
system $perm_prog, -c => $perm_conf and die "permissions: $!";
system qw(rsync -avz --exclude=.svn --delete-after),
$pre_launch_dir, $live_dir and die "rsync: $!";
There are a few things I've done here. First, I removed superfluous quotes. In Perl, unlike most shells, you don't need quotes around your variables to be safe. In fact, doing string interpolation when you don't really want to can have negative consequences. I also made use of the qw and => operators to remove a few more quotes, and I removed some unneeded parentheses. I find these things just clutter up the program. I also shortened the error messages and variables, though that was mainly to make the code fit better in a browser.
Update: fixed silly bug caused by last-minute variable renaming. | [reply] [d/l] [select] |
|
| [reply] [d/l] |
|
|