I do not think that the name is the reason. Perhaps you have not the right permissions, or you ftpserver does not allow rmdir at all.
this should help to find the reason.
#!/usr/bin/perl
use Net::FTP;
my $host = 'my.host.com';
my $login = 'login';
my $passwd = 'mypasswd';
my $ftp = Net::FTP->new( $host, Debug => 1)
or die "Cannot connect $@";
$ftp->login($login, $passwd)
or die "Cannot login ", $ftp->message;
$ftp->cwd('your_dir')
or die "Cannot change working directory ", $ftp->message;
$ftp->rmdir('your_dir', 1) or die $@;
| 