Thanks, I got your version to work with very few changes.

    my $dir = '/path/to/data/directory';  
    my %hash;
    opendir my $DH, $dir or die "cannot open '$dir' $!";
    while (my $file = readdir $DH ) {
        next if $file =~ /~$/;
        next if -d $file;
        open my $FH, "<", "$dir/$file" or die "Cannot open '$dir/$file
+' $!";
        while ( my $line = <$FH> ) {
            next if /^#/ || !length($line);
            my ($key, @values ) = split(/\t/, $line);
            $hash{ $file }{ $key } = \@values;
        }
    }
[download]

It even seems to work quite a bit faster than the ls/cat combo.