Your script also look for directories. Maybe you want to add a grep.

opendir my $d, $dir;

# read dir, keep only files sort the files by modification time
my @f = sort { -M $b <=> -M $a } grep { -f } readdir $d; 

# if we have more as 20 files remove the oldes two
unlink @f[0,1] if @f > 20;
[download]