http://qs321.pair.com?node_id=1096815

Jim has asked for the wisdom of the Perl Monks concerning the following question:

The function Encode::Repair::repair_double fixes the character U+201C LEFT DOUBLE QUOTATION MARK when double-encoded but not its companion character U+201D RIGHT DOUBLE QUOTATION MARK when double-encoded. Is there a bug in the module or a defect in my expectations? Or is something else wrong?

Here's a script that demonstrates the problem:

use v5.14; use strict; use warnings; use charnames qw( :full ); use Encode qw( encode decode ); use Encode::Repair qw( repair_double ); binmode STDOUT, ':encoding(UTF-8)'; my $ldqm = "\N{LEFT DOUBLE QUOTATION MARK}"; my $rdqm = "\N{RIGHT DOUBLE QUOTATION MARK}"; $ldqm = encode('UTF-8', decode('Windows-1252', encode('UTF-8', $ldqm)) +); $rdqm = encode('UTF-8', decode('Windows-1252', encode('UTF-8', $rdqm)) +); say repair_double($ldqm, { via => 'Windows-1252' }); say repair_double($rdqm, { via => 'Windows-1252' }); __END__
“
��?

Here's the output of the script piped through od:

C:\>perl demo.pl | od -h 0000000000 E2 80 9C 0D 0A EF BF BD EF BF BD 3F 0D 0A 0000000016 C:\>

E2 80 9C is the correct UTF-8 encoding of the Unicode character U+201C LEFT DOUBLE QUOTATION MARK.

EF BF BD is U+FFFD REPLACEMENT CHARACTER and 3F is U+003F QUESTION MARK. I expect the output to be the single Unicode character U+201D RIGHT DOUBLE QUOTATION MARK instead.