Backup script

I keep forgetting making backups of my dotfiles (and other files and directories too), so I wrote the following script that somewhat automates the process. It can be run by cron.

#!/usr/bin/perl
#
# Script to backup dot- and other files
#
# Copyright (C) 2009, Sven-Thorsten Fahrbach
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses>.

use strict;
use warnings;
use File::Path;
use File::Basename;
use File::Spec;
use File::Copy;
use Cwd;
use Getopt::Long;
use Fcntl qw/ :DEFAULT :flock /;

my $VERSION = "0.9";

##############################################################
# VARIABLE DECLARATIONS AND THE LIKE
##############################################################

# get a timestamp in the format YYYYMMDDHMS
my ($year, $month, $dayofmonth, $hour, $minute, $second)
        = (localtime())[5, 4, 3, 2, 1, 0];
$year += 1900;
$month++;
my $timestamp = "$year$month$dayofmonth-$hour$minute$second";

# standard values for config file/command line options
my $dotbackup_home      = get_dotbackup_home();
my $working_dir         = $ENV{HOME};
my $filename            = "dotbackup$timestamp";
my $destination_dir     = getcwd;
my $path_to_conf_file   = get_path_to_conf_file();
my $path_to_log_file    = File::Spec->catfile($dotbackup_home, "log");
my $path_to_file_list   = File::Spec->catfile($dotbackup_home, "files"
+);
my $compression_level   = 1;
my $gzip_options        = '';
my $tar_options         = '';
my $show_help           = '';
my $show_version        = '';

# this is the array with the files that are to be backed up
my @backup_files        = get_files();

# $log_level can be:
#   0 - quiet mode, no logging
#   1 - errors
#   2 - warnings
#   3 - info
#   4 - debug
my $log_level           = 4;

# Get values from the configuration file. Default values will be overw
+ritten
# with values from the config.
parse_config($path_to_conf_file) if $path_to_conf_file;

# Get command line options. These will overwrite default options and v
+alues
# we got from the configuration file.
my $result = GetOptions("working-dir=s"         => \$working_dir,
                        "filename=s"            => \$filename,
                        "destination-dir=s"     => \$destination_dir,
                        "logfile=s"             => \$path_to_log_file,
                        "file-list=s"           => \$path_to_file_list
+,
                        "log-level=i"           => \$log_level,
                        "compression-level=i"   => \$compression_level
+,
                        "gzip-options=s"        => \$gzip_options,
                        "tar-options=s"         => \$tar_options,
                        "help"                  => \$show_help,
                        "version"               => \$show_version);

help_and_exit()    if $show_help;
version_and_exit() if $show_version;

$compression_level = 1 if $compression_level < 1;
$compression_level = 9 if $compression_level > 9;

# commands that have to be invoked by us
my $tar                 = "tar -cf ";
my $gzip                = "gzip -$compression_level ";

####################################################################
# MAIN
####################################################################

# declare LOG file handle
my $LOG;
sysopen ($LOG, $path_to_log_file, O_WRONLY | O_APPEND | O_CREAT)
    or die "Can't open $path_to_log_file: $!";
# We'll try and keep a lock on the log file for the duration of our sc
+ript
flock($LOG, LOCK_EX)        or die "Can't get a lock on $path_to_log_f
+ile";

logger("invoked script", 3);
logger("compression level is $compression_level", 3);
# try to append leftover command line arguments to our @backup_files
foreach (@ARGV) {
    my $cur_file;
    unless (File::Spec->file_name_is_absolute($_)) {
        $cur_file = File::Spec->catfile($working_dir, $_);
    }
    next unless -e $cur_file;
    push @backup_files, File::Spec->canonpath($cur_file);
}
if (@backup_files == 0) {
    print "Nothing to do\n";
    logger("file stack empty - nothing to do", 3);
    exit(0);
}

# build tar-string
$tar .= "$tar_options " if $tar_options;
my $absolute_filename = File::Spec->catfile($destination_dir, $filenam
+e . ".tar");
$tar .= "$absolute_filename ";
foreach (@backup_files) {
    # Remove trailing slash or else tar will archive to contents of th
+e
    # directory instead of the directory itself which is probably not 
+what
    # we want.
    s/\/$//;
    $tar .= "$_ ";
}
logger("Archiving " . scalar(@backup_files) . " entries", 3);
logger("tar string: $tar", 4);

# invoke tar
my $tar_result;
$tar_result = `$tar`;
if ($!) {
    logger("tar error: $!", 1);
    logger("exited abnormally", 1);
    die "tar exited abnormally: $!";
}
logger("tar exited successfully", 4);
logger("tar output: $tar_result", 4);

# invoke gzip
$gzip .= "$gzip_options " if $gzip_options;
$gzip .= $absolute_filename;
my $gzip_result;
$gzip_result = `$gzip`;
if ($!) {
    logger("gzip error: $!", 1);
    logger("gzip exited abnormally", 1);
    die "gzip exited abnormally: $!";
}
logger("gzip exited successfully", 4);
logger("gzip output: $gzip_result", 4);

logger("script exiting successfully", 3);

# release lock on log file, close FH and exit successfully
flock($LOG, LOCK_UN);
close($LOG);
exit(0);

###############################################################
# SUBROUTINES
###############################################################

sub parse_config {
    my $conf_filename = shift;
    unless (open(CONF, $conf_filename)) {
        warn "Could not open config $conf_filename";
        return;
    }
    
    while (<CONF>) {
        chomp;
        next if /^\s*$/;        # ignore blank lines
        next if /^\s*#/;        # ignore lines consisting of comments 
+only
        
        if (/^working-dir\w*=\w*(.+)$/) {
            unless (-d $1) {
                warn "in $conf_filename line $.: Directory does not ex
+ist";
                next;
            }
            $working_dir = $1;
            next;
        }
        if (/^destination-dir\s*=\s*(.+)$/) {
            unless (-d $1) {
                warn "in $conf_filename line $.: Directory does not ex
+ist";
                next;
            }
            $destination_dir = $1;
            next;
        }
        if (/^log-file\s*=\s*(.+)$/) {
            if (-d $1) {
                warn "in $conf_filename line $.: $1 is a directory";
                next;
            }
            unless (-e $1) {
                unless (open(LOG, ">", $1)) {
                    warn "in $conf_filename line $.: Cannot open $1 fo
+r writing: $!";
                    next;
                }
                print LOG "[" . scalar(localtime()) . "] Created logfi
+le\n";
            }
            close(LOG);
            $path_to_log_file = $1;
            next;
        }
        if (/^file-list\s*=\s*(.+)$/) {
            if (-d $1) {
                warn "in $conf_filename line $.: $1 is a directory";
                next;
            }
            unless (-e $1) {
                warn "in $conf_filename line $.: $1 does not exist";
                next;
            }
            $path_to_file_list = $1;
            next;
        }
        if (/^tar-options\s*=\s*(.*)$/) {
            $tar_options = $1;
        }
        if (/^gzip-options\s*=\s*(.*)$/) {
            $gzip_options = $1;
        }

        if (/^compression-level\s*=\s*(\d)$/) {
            $compression_level = $1;
        }
        if (/^filename\s*=\s*(.+)$/) {
            $filename = "$1$timestamp";
            next;
        }
    }
}

sub get_files {
    unless (open(FILES, $path_to_file_list)) {
        warn "Cannot open $path_to_file_list: $!";
        return
    }
    my @filelist;
    while (<FILES>) {
        chomp;
        next if /^\s*$/;         # ignore blank lines
        next if /^\s*#/;         # ignore lines containing only commen
+ts

        my $cur_file;        
        unless (File::Spec->file_name_is_absolute($_)) {
            $cur_file = File::Spec->catfile($working_dir, $_);
        }
        next unless -e $cur_file;
        push @filelist, File::Spec->canonpath($cur_file);
    }
    close(FILES);
    return @filelist;
}

sub logger {
    my ($message, $level) = @_;
    return if $level > $log_level;
    
    # name log levels
    my @level_names = qw( NONE ERROR WARNING INFO DEBUG );
    
    # autoflush output for current scope only
    local $| = 1;
    
    # DEBUG
    #print "\@level_names:\n";
    #print "\t$_\n" foreach @level_names;
    #print "\$level: $level\n";
    #print "\$message: $message\n";
    
    print $LOG "[" . scalar(localtime()) . "] " . $level_names[$level]
+ . ": $message\n";
}

sub get_dotbackup_home {
    my $dotbackup_home_path = File::Spec->catdir($ENV{HOME}, ".dotback
+up");
    unless (-d $dotbackup_home_path) {
        mkpath($dotbackup_home_path) or die "Could not create dir $dot
+backup_home_path: $!";
    }
    
    return $dotbackup_home_path;
}

sub get_path_to_conf_file {
    my $conf_file_path = File::Spec->catfile($dotbackup_home, "conf");
    #GetOptions("config=s"   => \$path);
    #return if $path == undef;
 
    unless (-e $conf_file_path) {
        # warn "File $conf_file_path does not exist.\n";
        return;
    }
    if (-d $conf_file_path) {
        warn "$conf_file_path is a directory.\n";
        return;
    }
    return File::Spec->canonpath($conf_file_path);
}

sub help_and_exit {
    print <<EOHELP;
Usage: $0 [OPTION]... [FILE]...
Make a backup from a list of files. Save backup tarball to a specified
+ path
(the current directory by default).
  -c, --compression-level   1-9, 1 is fastest, 9 is best
  -d, --destination-dir     path to the output directory, default is \
+$HOME
      --filename            filename of the generated tarball (without
+ the
                            suffix)
      --file-list           path to the list of files to be backed up
  -g, --gzip-options        additionals options to pass to gzip
  -h, --help                give this help
      --log-level           0-4, 0 is quiet, 4 is the most verbose
      --logfile             path to the log file
  -t, --tar-options         additional options to pass to tar
  -v, --version             display version and exit
  -w, --working-dir         path to the default directory where the fi
+le to be
                            backed up are located, default is \$HOME
                            
Report bugs to <joran\@alice-dsl.de>.
EOHELP
    
    exit(0);
}

sub version_and_exit {
    print <<EOVERSION;
$0 $VERSION
Copyright (C) 2009 Sven-Thorsten Fahrbach
This is free software. You may redistribute copies of it under the ter
+ms of
the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
There is NO WARRANTY, to the extent permitted by law.

Written by Sven-Thorsten Fahrbach.
EOVERSION
    
    exit(0);
}
[download]

The corresponding config file looks like this:

# config file for dotbackup
# Entries here override default options.
# Command line arguments override config-file options

# working-directory: default is ${HOME}
# working-directory = /home/sven

# destination-directory: default is pwd
destination-dir = /home/sven/backups

# log-file: default is ~/.dotbackup/log
# log-file = /home/sven/.dotbackup/log

# file-list: list of files to be backed up, default is ~/.dotbackup/fi
+les
# file-list = /home/sven/.dotbackup/files

# tar-options: additional options to be passed to tar, default is unde
+f
# Use the following argument to exclude .tmp files from the archive
# tar-options = --exclude=*.tmp

# gzip-options: dito gzip
# use the following option to suppress output from gzip:
# gzip-options = --quiet

# compression-level: compression level for gzip to be used, default is
+ 1
# use the following option for best compression (might be slow)
# compression-level = 9
compression-level = 3

# filename: filename to be used for the tarball, default is dotbackup 
++
# timestamp, do not include file suffixes, these are added automatical
+ly
# filename = mybackup
[download]

Here's an example for a file list, each file has to be on its own line.

.bashrc
bin
.fetchmailrc
.forward
.irssi
.msmtprc
.mutt
.muttrc
.procmail
.procmailrc
.purple
.urlview
.vim
.vimrc
[download]

Comment on Backup script Select or Download Code

Replies are listed 'Best First'.
Re: Backup script by jwkrahn (Abbot) on Jun 14, 2009 at 21:01 UTC
`36 # get a timestamp in the format YYYYMMDDHMS 37 my ($year, $month, $dayofmonth, $hour, $minute, $second) 38 = (localtime())[5, 4, 3, 2, 1, 0];` [download] Why not just assign the variables in the order that localtime returns them: `# get a timestamp in the format YYYYMMDDHMS my ( $second, $minute, $hour, $dayofmonth, $month, $year ) = localtime;` [download] 70 parse_config($path_to_conf_file) if $path_to_conf_file; 100 # declare LOG file handle 101 my $LOG; 102 sysopen ($LOG, $path_to_log_file, O_WRONLY \| O_APPEND \| O_CREA +T) 103 or die "Can't open $path_to_log_file: $!"; 104 # We'll try and keep a lock on the log file for the duration o +f our script 105 flock($LOG, LOCK_EX) or die "Can't get a lock on $path_ +to_log_file"; 201 if (/^log-file\s=\s(.+)$/) { 202 if (-d $1) { 203 warn "in $conf_filename line $.: $1 is a direc +tory"; 204 next; 205 } 206 unless (-e $1) { 207 unless (open(LOG, ">", $1)) { 208 warn "in $conf_filename line $.: Cannot op +en $1 for writing: $!"; 209 next; 210 } 211 print LOG "[" . scalar(localtime()) . "] Creat +ed logfile\n"; 212 } 213 close(LOG); 214 $path_to_log_file = $1; 215 next; 216 } [download] You call `parse_config()` on line 70 which opens and prints to and closes the log file and then sets the variable `$path_to_log_file` which is subsequently used to open the log file again. Do you really need to open the log file inside the `parse_config()` function? `268 sub logger { 269 my ($message, $level) = @_; 270 return if $level > $log_level; 271 272 # name log levels 273 my @level_names = qw( NONE ERROR WARNING INFO DEBUG ); 274 275 # autoflush output for current scope only 276 local $\| = 1; 277 278 # DEBUG 279 #print "\@level_names:\n"; 280 #print "\t$_\n" foreach @level_names; 281 #print "\$level: $level\n"; 282 #print "\$message: $message\n"; 283 284 print $LOG "[" . scalar(localtime()) . "] " . $level_names +[$level] . ": $message\n"; 285 }` [download] You are changing the value of the `$\|` variable which turns on autoflush for the currently selected file handle which by default is `STDOUT` so it will have no effect on the following print statement. Since you are opening the log file with sysopen perhaps you should write the log message using syswrite which bypasses buffered IO. `syswrite $LOG "[" . scalar(localtime()) . "] " . $level_names[$lev +el] . ": $message\n";` [download] `202 if (-d $1) { 203 warn "in $conf_filename line $.: $1 is a direc +tory"; 204 next; 205 } 206 unless (-e $1) { 218 if (-d $1) { 219 warn "in $conf_filename line $.: $1 is a direc +tory"; 220 next; 221 } 222 unless (-e $1) {` [download] To avoid stating the same file twice you can use the special `_` filehandle: `202 if (-d $1) { 203 warn "in $conf_filename line $.: $1 is a direc +tory"; 204 next; 205 } 206 unless (-e _) { 218 if (-d $1) { 219 warn "in $conf_filename line $.: $1 is a direc +tory"; 220 next; 221 } 222 unless (-e _) {` [download]	[reply] [d/l] [select]
Re^2: Backup script by svetho (Beadle) on Jun 15, 2009 at 19:32 UTC
Thanks for your comment. Your suggestions all make sense and I wasn't even aware of the _ file handle - thanks! I'll implement your improvements and post the updated script here (which will probably take until next weekend).	[reply]
Re: Backup script by heptadecagram (Novice) on Jun 18, 2009 at 19:50 UTC
I would like to point out that there is great utility in putting your home directory's dot files into version control, especially a distributed VCS like hg, bzr, or git. That way, not only do you have a backup, but you have a versioned backup, for when you want to find that shell function you deleted a few months back. Also, it's trivially easy to get set up on a new machine (just check out the repository).	[reply]
Re^2: Backup script by mpeg4codec (Pilgrim) on Jun 27, 2009 at 17:42 UTC
++, I keep many of my dotfiles in a git repo and can't recommend it enough. Being able to sit down on a virgin box, `git clone`, and have all my stuff completely customized (including/especially vim) is the bee's knees. Pushing local customizations out to all the boxes I use without having to think about it is even better.	[reply]


Perl Monk, Perl Meditation
	PerlMonks