Good to hear that you are finding Perl to be useful in solving problems such as this. In the most part your code looks in good shape too.

The one obvious thing I notice when reading through this script is that none of your subroutines take any arguments and none of them return anything (not explicitly at least and they are being called in void context anyway). It isn't critical in a script of this size but since you have it working it might be opportune to try to pass some arguments here and there and see how it goes.

For example you have these:

build_exclude_list() if $exclude_list_file;
build_exclude_dir() if $exclude_dir_file;
[download]

And each of those subs works on the filename declared outside their scope. If I were writing this, I would pass the filename as an argument and have the subs return immediately if the arg is undef. eg:

sub build_exclude_list {
    my $file = shift;
    return unless defined $file;
    open my $exclude_files, '<', $file or die "Can't open $file: $!";
    # ...
}
# ...

build_exclude_list ($exclude_list_file);
[download]

Then think about returning the list which it builds rather than assigning to a global hash. Here is how you might return a ref to the hash.

sub build_exclude_list {
    # ...
    return \%exclude_list;
}

my $exclude_list = build_exclude_list ($exclude_list_file);
[download]

This makes your subroutine independent from variables declared outside it which in general terms is A Good Thing. It allows for code re-use: you could make the sub perfectly general, put it in a module and use it from multiple scripts without duplicating the code.

Anyway, just something to consider.