3

System: Debian 8 with XFCE, Cli: Konsole Version 2.14.2 (The terminal is called "Konsole" -> https://konsole.kde.org/)

I friend asked me to help him with a strange problem.

Some keys are having a weird behavior.

For example Shift + § makes the current line to show only (arg: 0) (We have a swiss keyboard https://en.wikipedia.org/wiki/QWERTZ#/media/File:KB_Swiss.svg, you can find this key on the upper left)

Pressing ö gives (arg: 6). And so on...

I can get out of this "mode" by the following commands:

set -o vi
set +o vi

But I am not in vi mode before. As far as I can tell. But something seems to be "reset". The thing is that I also loose some settings which I want if I use the two commands above.

EDIT: What I forgot to mention; On another Terminal emulator - UXTerm - the behaviour is "normal". So the way I want. But in XTerm I have the same behaviour like in Konsole.

EDIT2: As asked for:

user@myMachine:~$ bind -p | grep arg
"\M--": digit-argument
"\M-0": digit-argument
"\M-1": digit-argument
"\M-2": digit-argument
"\M-3": digit-argument
"\M-4": digit-argument
"\M-5": digit-argument
"\M-6": digit-argument
"\M-7": digit-argument
"\M-8": digit-argument
"\M-9": digit-argument
"\M-.": insert-last-argument
"\M-_": insert-last-argument
# universal-argument (not bound)
# vi-arg-digit (not bound)
# vi-yank-arg (not bound)
"\M-.": yank-last-arg
"\M-_": yank-last-arg
"\M-\C-y": yank-nth-arg
bioinf@ags-wav-debian-MandS:~$ set +o emacs
bioinf@ags-wav-debian-MandS:~$ bind -p | grep ö
bash: bind: warning: line editing not enabled

Additional info:

user@machine:~$ locale charmap
ANSI_X3.4-1968

user@machine:~$ bind -v | grep meta
set convert-meta on
set enable-meta-key on
set input-meta on
set meta-flag on
set output-meta on
jlliagre
  • 61,204
Michael
  • 193
  • If you move to the virtual terminal Ctrl + Alt + F1, all works as expected? Or the problem occurs there too? – MiniMax Jun 02 '17 at 14:17
  • 2
    Try to run subshell by this command bash --noediting (if problem only in Konsole, then do it there). This option disable readline library. And check which control sequencies send to the terminal in the raw form. Also, the same behavior will be if you do set +o emacs - disable emacs mode. – MiniMax Jun 02 '17 at 14:29
  • 1
    OK, running bash --noediting this behaviour is gone. But the arrows are now showing ^[[A, etc. for example. – Michael Jun 02 '17 at 14:34
  • It is normal behavior in this mode :). But which symbols are showing in this mode, when you are pressing problem characters like ö and Shift + §? – MiniMax Jun 02 '17 at 14:37
  • The correct symbols are shown. Meaning ö and ° – Michael Jun 02 '17 at 14:38
  • 1
    Well... Then try this: bind -p | grep arg and this: set +o emacs then bind -p | grep ö. Add their output to your question. My thoughts - may be you have keys like ö binded for another action. In the normal state, the (arg: 6) and (arg: 0) behavior is achived by pressing Alt + 6 and Alt + 0. In your case this behavior is changed. – MiniMax Jun 02 '17 at 15:13
  • See my second edit above – Michael Jun 02 '17 at 15:26
  • Do you have file .inputrc in your home directory? If you have, rename its like .inputrc.rename and test again. – MiniMax Jun 02 '17 at 15:57
  • Similar to your question: https://unix.stackexchange.com/questions/200255/bash-displays-international-characters-as-escape-sequences – MiniMax Jun 02 '17 at 15:59
  • 1
    You have set enable-meta-key on because your charset is ASCII (ANSI_X3.4-1968), you shouldn't be entering those characters if your charset is ASCII, ASCII has no such character. There's something wrong in your locale settings. – Stéphane Chazelas Jun 02 '17 at 16:06
  • 1
    The reason it works with UXTerm is that that's a script that sets LC_CTYPE to something.UTF-8 if the locale was not initially using the UTF-8 charmap. – Stéphane Chazelas Jun 02 '17 at 16:13
  • @StéphaneChazelas then it also shouldn't work in the terminal emulator (tty), right? – MiniMax Jun 02 '17 at 16:20
  • 2
    Depends where the locale is being specified. Possibly your login manager has an option to specify the locale. You may also have it configured in your X session manager / desktop configuration, and that would override the system's locale. dpkg-reconfigure locales to set the system's locale on Debian (you may need to reboot after that). – Stéphane Chazelas Jun 02 '17 at 16:24
  • @StéphaneChazelas but how is it possible? - (copied from our previous comments) "OK, running bash --noediting this behaviour is gone.", "The correct symbols are shown. Meaning ö and °". If readline was disabled, symbols are shown. And you wrote, that "you shouldn't be entering those characters if your charset is ASCII, ASCII has no such character." Where did ö and ° characters come from? – MiniMax Jun 02 '17 at 17:21
  • I have some ideas, how it is possible, have looked at outputs in the question - user@myMachine, bioinf@ags-wav-debian-MandS, user@machine - check was made on different machines? :) – MiniMax Jun 02 '17 at 17:30
  • Nope, sorry same machine. Forgot to edit at the bottom. – Michael Jun 02 '17 at 18:03

1 Answers1

6

It seems to all boil down to the fact that the charset in your locale is ASCII. ASCII is an American charset from the 60s and that is the common denominator for most modern character sets.

That typically happens when locales are not configured (LANG, LC_* environment variables all unset) or when they are set to C or POSIX.

That charset defines 128 characters with byte values 0 to 127. All the characters that make up the sh language and found in most command names are in ASCII. But § and ° are not.

Now those symbols are on your keyboard, what should happen when you type it in a terminal emulator?

If you start your terminal emulator in a locale where the charset is ASCII, you're telling your terminal emulator that when you press A, (XK_a X11 keypress event) it needs to send the encoding of the a character in the ASCII encoding, that is 0x61. But when you press Shift+§ (XK_degree X11 keypress event), it should send the encoding for the ° character, but there's no such character in ASCII, so what should it do?

xterm and rxvt choose to send the encoding of ° in the iso-8859-1 character set. That's the 8 bit charset of most of the western world (extends ASCII to cover most of the characters used in languages like German, French, British English, Spanish...). Before UTF-8, that was the most commonly used charset.

My konsole, Eterm and xfce4-terminal send ? for every non-ASCII character.

My gnome-terminal and terminator send the UTF-8 encoding of °.

When bash receives the iso8859-1 encoding of ° (0xb0) like in xterm, what shall it do?

We've told it the charset was ASCII, so that 0xb0 corresponds to no known character.

In the old days in the US, Ctrl+X was used to enter characters from 0 to 31 (the control characters). In ASCII, that would send the lower 5 bits of the corresponding character. For instance, A being 0x41, a being 0x61, Ctrl+A would send 0x61 & 0x1f, so 0x1 (the ^A, aka C-A character). While Meta+X would send the encoding of x with the the 8th bit set. a being 0x61, Meta+A would send 0x61 | 0x80, that is 0xe1. Pressing Meta+0 would send 0x30 | 0x80, that is 0xb0 also known as the M-0 (non-)character.

In ASCII, 0x0 -> 0x1f are used for Control characters for the teletypewriter days, most of which are no longer used, and 0x80 to 0xff is not used, so that was a way to input codes that could be used to do other things than enter text. emacs for instance used those for the editing keys, the C-B character moves the cursor one character to the left, the M-B (non-)character moves the cursor one word to the left.

Nowadays, since most people use charsets that extend ASCII by making use of byte values 0x80 to 0xff, those are no longer commonly understood as those meta characters. Meta+X now generally sends two characters instead: the ESC and X characters.

Still, when in a locale where byte values 0x80 to 0xff can't possibly be characters, when bash (actually readline) reads a byte value from the tty device like 0xb0, it understands it as being M-0 (which by default is bound to digit-argument, which explains your arg: 0).

That's the convert-meta setting in the readline configuration. You'll find in the readline documentation (man 3 readline) that when readline detects that the charset is 7 bit, it sets that to on to convert those 0xb0 bytes to ESC + 0.

If you turn that off with:

bind 'set convert-meta = off'

(and assuming input-meta and output-meta are also on). Then you'll find that pressing ° gets you a ° displayed. But that would be a 0xb0 iso8859-1 encoding of ° which applications won't know what to do about.

What you need to do is fix the locale to one that has those ° characters. Nowadays, you should only be considering UTF-8 as that covers all characters and is widely supported.

So, check your desktop configuration for your internationalisation setting and choose something like de_CH.UTF-8/fr_CH.UTF-8/it_CH.UTF-8 (German/French/Italian speaking Swiss with UTF-8 as the charset) that best match your environment.

You may need to logout and login again for that to be fully taken into account.

Some login managers sometimes allow you to select the locale at login time via some drop-down menu as well.