nhexl-mode shows bytes 80-ff with 6-digit hex 3fff80–3fffff

Question

When I open a file and activate nhexl-mode, non-ASCII characters are shown as 3fffxx in the hex column instead of xx. This disrupts the alignment.

I observed this on Emacs 23.2 and 24.4 started with emacs -Q. On Emacs 22.2, it works fine.

What's wrong here, and how do I fix it?

Answer 1

I think the problem is that unibyte characters in an unknown encoding — which is what non-ASCII characters are in a binary file — are represented internally with character values #x3fff80–#x3fffff.

Arguably, not taking this into account is a bug in nhexl-mode, or at least an incompatibility with MULE and Emacs ≥23.

A possible fix is to switch the buffer to unibyte mode. The code below does this automatically on entry to nhexl-mode. Note that it doesn't restore multibyte mode if you switch nhexl-mode off.

(defun my-nhexl-mode-hook ()
  (when nhexl-mode
    ;; Turn off multibyte, otherwise nhexl 0.1 displays non-ASCII characters
    ;; in hexa as values in the range 3fff80-3fffff instead of 80-ff.
    (if (and (fboundp 'toggle-enable-multibyte-characters)
             enable-multibyte-characters)
        (toggle-enable-multibyte-characters))))
(add-hook 'nhexl-mode-hook 'my-nhexl-mode-hook)

This feels more like a partial workaround though. It would be nice to be able to display multibyte characters in the text display and individual bytes in the hexadecimal display.