For efficiency you want something like this:

my $dir = '/Path/to/a/data/directory';

opendir my $DH, $dir or die "Cannot open '$dir' $!";

my %hash;
while ( my $file = readdir $DH ) {
    next if $file =~ /~$/;
    open my $FH, '<', "$dir/$file" or die "Cannot open '$dir/$file' $!
+";
    while ( my $line = <$FH> ) {
        next if /^#/ || !/\S/;
        my ( $key, @values ) = split /\t/;
        $hash{ $file }{ $key } = \@values;
    }
}
[download]

Update: I realised that the while loop is incorrect, it should be:

    while ( my $line = <$FH> ) {
        next if $line =~ /^#/ || $line !~ /\S/;
        my ( $key, @values ) = split /\t/, $line;
        $hash{ $file }{ $key } = \@values;
    }
[download]