Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Problem with escape sequence \x for hex with spaces "\x{ 263A }"

by LanX (Saint)
on Jun 10, 2022 at 20:00 UTC ( [id://11144667]=perlquestion: print w/replies, xml ) Need Help??

LanX has asked for the wisdom of the Perl Monks concerning the following question:

Dear brethren,

please help me understand either

  • what is meant °
  • what I'm missing
  • if this a bug

from perlop#Quote-and-Quote-like-Operators

\x{263A} [1,8] hex char (example shown: SMILEY) \x{ 263A } Same, but shows optional blanks inside and adjoining the braces

DB<55> p join ",",map {ord} split //, "\x{ 41 }" + 0 + DB<56> p join ",",map {ord} split //, "\x{41}" + 65 + DB<57> p join ",",map {ord} split //, "\x{263A}" + 9786 + DB<58> p join ",",map {ord} split //, "\x{ 263A }" + 0 + DB<59> p "\x{41}" + A + DB<60> p "\x{ 41 }" + ^@

tested with strawberry perl 5.032001 on win

Question: In which way is this

"Same, but shows optional blanks inside and adjoining the braces" ?

Cheers Rolf
(addicted to the Perl Programming Language :)
Wikisyntax for the Monastery

update

°) OK - ironically - reading the footnotes help

https://perldoc.perl.org/perlop#%5B1%5D

    Note that any escape sequence using braces inside interpolated constructs may have optional blanks (tab or space characters) adjoining with and inside of the braces, as illustrated above by the second \x{ } example.

    [1] The result is the character specified by the hexadecimal number between the braces. See "8" below for details on which character.

    Blanks (tab or space characters) may separate the number from either or both of the braces.

    Otherwise, only hexadecimal digits are valid between the braces. If an invalid character is encountered, a warning will be issued and the invalid character and all subsequent characters (valid or invalid) within the braces will be discarded.

    If there are no valid digits between the braces, the generated character is the NULL character (\x{00}). However, an explicit empty brace (\x{}) will not cause a warning (currently).

update

and activating warnings shows the bug, somehow the blank is not recognized.

DB<65> use warnings; "\x{ 263A }" + Non-hex character ' ' terminates \x early. Resolved as "\x{00}" at (e +val

update

different warnings by version ... this is getting weirder and weirder

  • ActiveState 5.24

    C:\tmp>%PERL% -w print $]; print "\x{ 41 }"; Illegal hexadecimal digit ' ' ignored at - line 2. __END__ 5.024001
  • Strawberry 5.32

    C:\tmp>%PERL_532% -w print $]; print "\x{ 41 }"; Non-hex character ' ' terminates \x early. Resolved as "\x{00}" at - +line 2. __END__ 5.032001
could someone please test this on linux?

Replies are listed 'Best First'.
Re: Problem with escape sequence \x for hex with spaces "\x{ 263A }"
by kcott (Archbishop) on Jun 10, 2022 at 20:41 UTC

    G'day Rolf,

    That feature was introduced in v5.34: "perl5340delta: Blanks freely allowed within but adjacent to curly braces".

    My tests:

    $ alias perle alias perle='perl -Mstrict -Mwarnings -Mautodie=:all -MCarp::Always -E +' $ perlbrew switch perl-5.32.0 $ perle 'say $]; say join ",",map {ord} split //, $_ for "\x{ 41 }", " +\x{41}", "\x{263A}", "\x{ 263A }"; say for "\x{41}", "\x{ 41 }"' Non-hex character ' ' terminates \x early. Resolved as "\x{00}" at -e + line 1. Non-hex character ' ' terminates \x early. Resolved as "\x{00}" at -e + line 1. Non-hex character ' ' terminates \x early. Resolved as "\x{00}" at -e + line 1. 5.032000 0 65 9786 0 A $ perlbrew switch perl-5.34.0 $ perle 'say $]; say join ",",map {ord} split //, $_ for "\x{ 41 }", " +\x{41}", "\x{263A}", "\x{ 263A }"; say for "\x{41}", "\x{ 41 }"' 5.034000 65 65 9786 9786 A A

    It's not mentioned in perlop, so a doco omission but not a bug in Perl itself. Perhaps raise a doco bug report.

    — Ken

      Hi Ken,

      > hat feature was introduced in v5.34

      Thanks, I'm feeling stupid now.

      I'm having the newest available Strawberry Perl 5.32 installed and didn't think about the perldoc being even newer.

      > It's not mentioned in perlop, so a doco omission

      I searched for "version" in perlop and there are many mentions of differences to older Perl.

      Not sure if this is always done, but I certainly would have preferred seeing a "(since v5.34)" somewhere.

      And I found the description rather cryptic too.

      • but shows optional blanks inside and adjoining the braces
      shouldn't it rather be "ignores" or "tolerates blanks"?

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      Wikisyntax for the Monastery

        shouldn't it rather be "ignores" or "tolerates" whitespace?

        My suggested phrasing would be on "but allows° optional blanks between the braces and the hex value"

        (And whether I said "blanks" or "spaces" or "horizontal whitespace" or "whitespace" would probably depend on whether tests show that it allows only space, or any horizontal space, or any horizontal or vertical space. Like you, I only have Strawberry so limited to 5.32 and cannot test the feature.)

        update: °: Rereading the original, I think they intended that sentence to mean the equivalent of "This second example of the syntax means the same as the example above, but shows the optional blanks that are allowed..."

        'I searched for "version" ...'

        On the basis that version notes often look like "introduced in 5.nn", I searched for "34": 8 instances found; none relate to this issue.

        '... but shows ... shouldn't it rather be "ignores" or "tolerates" whitespace?'

        I believe that was intended to contrast with the previous line (\x{263A} vs. \x{ 263A }). Depending on device, browser and font, the difference between \x{263A} and \x{ 263A } may not be all that obvious.

        Regardless, a ninth note (e.g. [9] Introduced in v5.34.0.) would add clarity.

        — Ken

Re: Problem with escape sequence \x for hex with spaces "\x{ 263A }"
by Aldebaran (Curate) on Jun 11, 2022 at 03:18 UTC

    Na Rolf, I had this same failure but didn't post it, because I had *a lot* of failures and first assume that I'm doing something backwards. I can get there partially now, but I can't clear the warning:

    $ perl -de1 ... DB<1> $str = "\x{ 41 }" + DB<2> p $str + DB<3> $str1 = "\x{41}" + DB<5> p $str1 + A DB<6> $str2 = "\x{263A}" + DB<7> p $str2 + Wide character in print at (eval 30)[/usr/share/perl/5.28/perl5db.pl:7 +38] line 2. ... &#9786; DB<8> binmode STDOUT, ':utf8'; + DB<9> p $str2 Wide character in print at (eval 30)[/usr/share/perl/5.28/perl5db.pl:7 +38] line 2. + ... &#9786;

    I tried everything I could throw at it. Am I on STDOUT whilst invoking perl -de1 ? What am I missing?

      > What am I missing?

      the interactive debugger is redirecting STDOUT with every eval. You need to do it inline

      DB<3> binmode STDOUT, ':utf8'; print "\x{263A}" \u263A

      of course your terminal° must also be capable to display smileys :)

      I think if you want to change STDOUT globally for the debugger, some of the unicode flags in perlrun should do. (untested)

      Cheers Rolf
      (addicted to the Perl Programming Language :)
      Wikisyntax for the Monastery

      °) in my case emacs inside the console

        "I think if you want to change STDOUT globally for the debugger, some of the unicode flags in perlrun should do. (untested)"

        I tried all of these:

        $ perl -C -de1 ... $ perl -de1 ... DB<1> use open OUT => qw{:encoding(UTF-8) :std} ... $ perl -Mopen=OUT,:encoding\(UTF-8\),:std -de1 ...

        All gave the "Wide character ..." warning. All still printed "". There's probably other avenues of investigation for a global change: I didn't try anything further.

        — Ken

        the interactive debugger is redirecting STDOUT with every eval. You need to do it inline

        Aha...

          DB<1> binmode STDOUT, ':utf8'; print "\x{263A}"                               
        ☺
          DB<2>   
        

        Endlich erlungen. Vielen Dank....

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11144667]
Approved by kcott
Front-paged by kcott
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others perusing the Monastery: (7)
As of 2024-04-23 11:47 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found