By explicitly invoking encode ("UTF-8", ..., the mixed string contains bytes with the high bit set, but not UTF-specific characters. Outputting a byte string as binary is natural, but outputting a Perl string that contains wide characters does not map without specifying an encoding.

Does this clarify? If you describe the task you are trying to accomplish, we can probably help with the appropriate set of I/O specifications.

in your first case you have a properly formatted Perl string that contains UTF characters

Yes

but in the second you have a UTF-8 byte string, not a character string.

No. Second case does look like UTF-8 character string, because it prints "UTF IS ON" and "LENGTH DIFFERS"

Note that if you modify line 8 to

my $ascii_but_utf = '123';
[download]

the output changes to

SV = PV(0x22ae1d0) at 0x2300b20
  REFCNT = 1
  FLAGS = (PADMY,POK,pPOK)
  PV = 0x22d21f0 "123\321\204\321\204\321\204\321\204"\0
  CUR = 11
  LEN = 16
ALL OK
[download]

This is because that UTF is on is just a historical artifact of your initialization.

If we take a look at the two output files generated by these two cases, you'll note that both contain 11 bytes, despite the fact that the byte dump of the UTF-upgraded case should have output 19 bytes. This is because the internal representation of high-bit, 1-byte characters under Perl's implementation of UTF is multi-byte even though they cleanly map to 1-byte characters on output. You wouldn't expect these 1-byte characters to output a wide-character warning any more that you'd expect an ASCII character to.

Second case does look like UTF-8 character string,

You're thinking of that wrong; it could be a UTF character string, or a UTF-8 byte string. When dealing with non-ASCII characters in Perl, rare is the case when you should actually be thinking about Perl's internal representation.

---

#11929 First ask yourself `How would I do this without a computer?' Then have the computer do it the same way.

#!/usr/bin/perl

use strict;
use warnings;
use Devel::Peek;
use Encode;
#use utf8;

binmode STDOUT, ":utf8";

my $string1 = "123\x{444}\x{444}\x{444}\x{444}";
_display ($string1, 'STRING1' );

my $utfstring = "123 \x{439}\x{439}\x{439}\x{439}";
_display ($utfstring, 'UTF_STRING' );

my ($ascii_but_utf, undef) = split ' ', $utfstring;
_display ($ascii_but_utf, 'ASCII_BUT_UTF' );

#my $bytestring = encode ("UTF-8", "\x{444}\x{444}\x{444}\x{444}");
my $bytestring = "\x{444}\x{444}\x{444}\x{444}";
_display ($bytestring, 'BYTESTRING' );

my $mixedstring = "$ascii_but_utf$bytestring"; # simulate The Unicode 
+Bug here
_display ($mixedstring, 'MIXEDSTRING' );

print "MIXEDSTRING and STRING1 are supposed to be identical...\n";

exit;

###############

sub _display {
   my ($string, $name) = @_;

   print "$name:\n";
   Dump $string;

   my $l1 = length($string);
   my $l2 = bytes::length($string);
   if ($l1 != $l2) {
       print "LENGTHs DIFFERS: length: $l1, bytes: $l2\n" 
   }

   print "UTF IS ON\n" if utf8::is_utf8($string);
   print "\n";
}
[download]

Yes, I understand that result of encode("utf8", ... ) is a byte string with UTF-8 flag off. But that does not answer the question in my post. In my example, both poc1.pl and poc2.pl print strings with UTF-8 on, with length <> bytes::length, but those strings behave differently. Why?