It is not clear for me what you want to accomplish, but generally use HTML::Entities to process html entities:

use strict;
use warnings;
use HTML::Entities;

my $line="” 3”";
decode_entities ($line);
[download]

If this module is of no help, please, try to give a step backwards and explain the kind of conversions you want to be done

Update: A closer look at your code reveals that you are not using alternation correctly: [] in a regexp is a character class, but probably you are wanting () instead (see perlretut and perlre):

$line="” 3 ”";
$line =~ s/(?:&|&amp\;)rdquo;/'/g;
print "$line\n";
[download]

That prints:

' 3 '
[download]