Relationship of keyboard layout and xmodmap

Question

I'm using Xubuntu. Before login I can choose a keyboard layout. I'm using xmodmap for remapping some keys.

I'm interested in two things:

1. How the state of keyboard mapping changes (a) when I turn the laptop on, (b) during the boot process and (c) login to the system (in these three phases) and when working with the system (logged in).
2. What causes which symbols will be displayed on the screen (and control keys sent) during the individual phases. When I press some key, it sends some signal to the keyboard driver(?) and then there must be some decision process (applications and configuration files) determining what symbols will be displayed. Answer to this question should be the list of applications and paths to those configuration files (I'm especially interested in Ubuntu (Debian based system), but you can describe some other system, but Ubuntu is preferred).

Answer 1

There are two layers here, KEYCODE to KEYSYM mapping and KEYSYM to text mapping. There are more layers if you count the kernel, which has to map AT Keyboard scancodes to a XT-style KEYCODE or a USB Keyboard HID code to a KEYCODE. A KEYCODE is simply an 8-bit unsigned integer which the kernel of an Operating System passes to the X11 server. It can vary between Operating Systems such as Linux and Solaris. On Linux, these KEYCODEs are typically the same number used on old XT PC Keyboards. Newer computers with AT, PS/2, or USB keyboards typically just map those keyboards to the old XT code for the key to keep life simple.

Raw keyboard codes, whether they be XT, AT, PS/2, or USB represent a physical location on a keyboard. The XT keyboard only sends a single 8-bit number on press or release of a key. The q key on an US/British XT keyboard sends the number 16. On a French keyboard that same physical key is labeled a, but it still sends 16. It's the higher layers in the operating system that assign it a real meaning. When a key is released on an XT keyboard, the same keycode is sent plus 128. For this example, when q is pressed, a 16 is sent, but on release, the number 142 (16+128) is sent. AT Keyboards use scancodes which are a series of numbers and can get quite long. Key releases add additional codes. For example, the scancode for Pause is E1, 1D, 45, E1, 9D, C5. Most Operating Systems including DOS, Windows, Linux, FreeBSD, and the BIOS all map scancodes into much simpler XT-style scancodes. It also keeps it easier to support newer keyboards which use different codes such as USB Keyboards which send HID codes. All codes get mapped to the same consistent set of codes by the Operating System before X11 or the application sees them.

X11 is ignorant of this part of the process, it just gets the KEYCODE from the kernel and applies its own mapping to convert that KEYCODE to a KEYSYM. Xmodmap is the standard tool for controlling that mapping. Much of the behavior of the keyboard mapping is configurable, but there are several special cases such as Num Lock, Mode Switch, and Caps Lock/Shift Lock that are hard coded into X11. Other aspects like Shift are actually configurable. Any key can be mapped to act as shift, unlike Mode Switch or Num Lock.

KEYCODEs represent physical keys sent by the Operating System's kernel. Every KEYCODE can map to 8 possible KEYSYMs. Only 4 are used and are sometimes called levels 1-4. Level 1 specifies the KEYSYM that gets printed when no modifiers are active. These are often lowercase letters and digits. Modifiers are KEYCODEs that modify the KEYSYM generated by other KEYCODEs when the modifier is active (pressed or toggled on.) Modifier keycodes are also controlled through Xmodmap. Level 2 specifies a KEYSYM to be sent when the shift modifier is pressed. Level 3 is activated whenever the Mode Switch KEYSYM has been pressed. Level 4 is activated when both a shift key and Mode Switch are active.

Once a KEYSYM has been generated, this may be interpreted directly, but most often will be converted to text. Not all KEYSYMs turn into text or may only affect a future KEYSYM. One example is Shift_L, of course, which has no textual representation, but there are also a number of KEYSYMs that are used to Compose another character. A list of them on my system is under /usr/share/X11/locale/en_US.UTF-8/Compose. One such example is the dead_acute KEYSYM which, when pressed, will attempt to convert the next KEYSYM into an acute accented letter. There is a standard mapping for turning KEYSYMs into Unicode.

Now that all this has been said, note that Xmodmap is obsolete and replaced by XKB which is much more sophisticated. This affects how KEYCODEs are mapped to KEYSYMs, but not how the kernel generates KEYCODEs nor how KEYSYMs are converted into text or composed which is still the same. XKB can be disabled restoring Xmodmap behavior. It also has a compatibility layer to support Xmodmap, but it can have issues as it's not completely compatible. XKB rules are under /usr/share/X11/xkb/ and are much more sophisticated. There is some good documentation elsewhere on how it generates keyboard layouts for mapping KEYCODEs to KEYSYMs.

As for the Linux console, it has its own keyboard layouts which are stored in /usr/share/keymaps and loaded with the loadkeys command. When in the BIOS and earlier boot loader stages, including GRUB2, the keyboard mapping is whatever the number the BIOS decides to map the key to.