Mapping otherwise conflicting or unmappable keys in terminal Vim

Question

Resources in the network (1, 2, 3) claim that some key combinations, among which Ctrl-Shift-letter, Ctrl-number, Ctrl-i/Tab, Ctrl-m/Enter, Esc/Ctrl-[, cannot be mapped reliably in Vim because the terminal does not distinguish them from their unmodified counterparts (more background in this Gilles' answer and this ASCII table article). As a concrete example, the maps

nnoremap <Tab>   :!echo A<CR>
noremap  <C-S-X> :!echo B<CR>
noremap  <C-1>   :!echo C<CR>
noremap  <C-F1>  :!echo D<CR>

cause both Tab and Ctrl-i to print A (also in Gvim) and both Ctrl-x and Ctrl-Shift-x to print B. Ctrl-{1,2,...} and Ctrl-{F1,F2,...} cannot be mapped, the former not even in Gvim.

This answer to "How to map Ctrl-a and Ctrl-Shift-a differently?" shortly describes a solution for Xterm, but it lacks several tricky details that may escape some. This is an attempt to provide a more complete answer.

Although this is a Vim centered question, other terminal applications with customizable mappings can also benefit from it. Vifm is an obvious case.

Answer 1

I'll start with Xterm because it is the most complicated one. Kitty and Urxvt are tackled at the end.

X-resources

The ~/.Xresources file configures Xterm (and some other Xlib applications). Whenever you are done editing it, issue xrdb ~/.Xresources and open a new Xterm to test the changes. Start with

XTerm*metaSendsEscape: true
XTerm*eightBitInput:   false

^{See man xterm or configuring Xterm for further options, such as colors and fonts — the defaults are admittedly ugly.}

Sending keycodes

We will follow Leonerd's article proposal, and have the terminal send CSI codepoint;modifier u, where

• CSI stands for an Esc character followed by [.

• codepoint is the decimal Unicode value of the character to be mapped. ASCII characters have the same decimal representation in Unicode.

• modifier is chosen from the table below:

  None	Shift	Alt	Alt+Shift	Ctrl	Ctrl+Shift	Ctrl+Alt	Ctrl+Alt+Shift
  1	2	3	4	5	6	7	8

Ctrl-Shift-x

Look up X in an ASCII table and find that its decimal value is 88. Therefore CSI 88;5 u should be sent to Vim. This goes in .Xresources:

XTerm*Translations: #override\
    Ctrl ~Meta  Shift <Key>x   :string("\033[88;5u")

Ctrl-Shift-x now produces the sequence in quotes. A tilde negates the modifier, i.e., ~Meta means that Alt is not pressed (Meta means Alt). 033 is Esc in octals.

There should be no spaces after the backslashes (Xrdb would warn you of the mistake) and, as will be seen ahead, multiple bindings should be separated from each other with \n\.

Tab and Ctrl-i

i is decimal 105 and Tab is decimal 9, but both Ctrl-i and Tab send 9, as the ASCII table article explains. So Ctrl-i must send a different sequence to disambiguate them, and by the same reasoning as before, we conclude it is CSI 105;5 u.

XTerm*Translations: #override\
    Ctrl ~Meta ~Shift <Key>i   :string("\033[105;5u")

One could also add ~Ctrl ~Meta ~Shift <Key>Tab :string("\011") but that would be redundant since Tab already sends decimal 9 (octal 11).

Modified Tab is special as listed under "Modified C0 controls".

XTerm*Translations: #override\
   ~Ctrl ~Meta  Shift <Key>Tab :string("\033[Z")    \n\
    Ctrl ~Meta ~Shift <Key>Tab :string("\033[9;5u") \n\
    Ctrl ~Meta  Shift <Key>Tab :string("\033[1;5Z")

More special keys

Here belong F1-F12, Home and others. In Xterm they already have unambiguous codes, which can be straightforwardly determined by pressing Ctrl-vKey in Vim's insert mode. For example, Ctrl-vCtrl-F1 produces <ESC>[1;5P.

Vimrc

Now we just have to add the bindings to .vimrc:

" Disambiguate Tab and Ctrl-i
nnoremap <ESC>[105;5u <C-I>
nnoremap <Tab>        :!echo A<CR>
noremap  <ESC>[88;5u  :!echo B<CR>
noremap  <ESC>[49;5u  :!echo C<CR>
noremap  <ESC>[1;5P   :!echo D<CR>

The 5th line is very important: Ctrl-i, used to move back in the jump list, is not Tab anymore in Xterm; instead it sends a different sequence to Vim, thus that sequence should be mapped to what Vim has under Ctrl-i.

Extra notes

• If you notice a delay when Esc is pressed in insert mode, adjust Vim's timeout settings, e.g. set timeoutlen=1000 ttimeoutlen=20 (see timeoutlen vs ttimeoutlen).

• If in doubt what keysym should go in .Xresources, use xev to find it out. For example, pressing the left Windows/Super key outputs Super_L.

• In Xterm, Ctrl-Q and Ctrl-S are reserved for flow control, a legacy feature. To map them, first deactivate flow control by adding

  " Disable XOFF/XON
  silent !stty -ixon
  " Redraw screen
  silent !resize>/dev/null
  

  to .vimrc.

• To make the maps invisible to other TUI programs, you can keep Vim under a different Xterm classname, such as

  xterm -name vimterm -e vim file
  

  and use vimterm instead of XTerm in the .Xresources file. How to open new files in a same Vim instance may prove helpful.

Other terminal emulators: Urxvt and Kitty

Urxvt uses a different syntax in .Xresources. A interfering binding caused by ISO 14755 also has to be disabled.

URxvt*iso14755:    false
URxvt*keysym.C-i:  \033[105;5u
URxvt*keysym.C-X:  \033[88;5u
URxvt*keysym.C-1:  \033[49;5u
URxvt*keysym.C-F1: \033[1;5P

Kitty does not use .Xresources, the bindings go in ~/.config/kitty/kitty.conf:

map ctrl+shift+x send_text application \033[88;5u
map ctrl+i       send_text application \033[105;5u
map ctrl+1       send_text application \033[49;5u
map ctrl+F1      send_text application \033[1;5P