What happens when you run the script with wide-characters explicitly turned off, i.e. in an xterm launched with xterm -u8 +wc?
No difference whatsoever.
seems to think certain byte sequences are escape sequences meant to control the terminal.
That may be.
The things is, those normally start with ESCape (^[). UTF-8 doesn't produce anything that contains ESC except for ESC itself. Other control character respected by terminals are also found in the ASCII range and thus not produced by UTF-8.
I don't know much about terminals, and less about xterm. I didn't even have xterm installed until this came up.
You mentioned something about an "Xemacs shell". Is that a variable that can be eliminated?
| [reply] |
You mentioned something about an "Xemacs shell". Is that a variable that can be eliminated?
In this case the xemacs shell was being used as a control rather than something to be eliminated. One way of assuring myself that this was xterm specific behavior was to run the script in an alternative shell and see what happened. As it turned out, there was no disappearing output in the xemacs shell (which is really just a file buffer pretending to be a shell). I also dumped the output to a file instead of the terminal (as suggested above by BrowserUK and not surprisingly it was all there - no disappearing output. This really does seem to be an xterm problem.
those normally start with ESCape (^[).
Like you, that was my first assumption too, but googling around I see that there does appear to be some overlap between UTF-8 and xterm escape sequences. For example,
Under normal mouse mode, positions outside (160,94) result in byte pairs which can be interpreted as a single UTF-8 character; applications which do treat their input as UTF-8 will almost certainly be confused unless extended mouse mode is active. Source: http://invisible-island.net/xterm/ctlseqs/ctlseqs.html#Mouse%20Tracking
I'm not sure how that explains what I'm seeing, but that may not be the only case of overlap.
I don't know much about terminals, and less about xterm. I didn't even have xterm installed until this came up.
Wow. Many thanks for the effort you have put into this!
| [reply] |
Under normal mouse mode, positions outside (160,94) result in byte pairs which can be interpreted as a single UTF-8 character;
For there to be an issue, a sequence of UTF-8 characters has be interpreted as an escape sequence, not the other way around.
From higher up in that linked document comes this:
The xterm program recognizes both 8-bit and 7-bit control characters. It generates 7-bit controls (by default) or 8-bit if S8C1T is enabled.
It proceeds to say 0x9B and ESC [ are equivalent, for example.
More relevant, it says 0x90 and ESC P are equivalent. U+05D0 is 0xD7 0x90 in UTF-8.
Are these equivalent for you?
perl -e'print "\x1B[31m", "foo", "\x1B[0m", "bar", "\n";'
perl -e'print "\x9B31m", "foo", "\x9B0m", "bar", "\n";'
Perhaps you can tell xterm to stop recognising the "8-bit" codes.
| [reply]
[d/l] |
