Debian thinks my file system is encoded as ISO-8859-1

Question

This may not be the appropriate site for this question, because it's more a complaint than a question, but here goes.

Today I updated my computer from Mint 15 to Debian 7.6. And after I'd done that, my computer thought that all the existing filenames were encoded as ISO-8859-1 rather than UTF-8. So, for instance, filenames containing "à" now got displayed as "Ã " and so on.
Turns out I needed to tell my system explicitly that the file system uses UTF-8, by putting export LANG=en_US.UTF-8 in my .bashrc file.

And the question is, why? Why should something like that be necessary in this day and age, 2014? The default was UTF-8 for Mint, obviously; why should Debian be any different? What would cause Debian to use such an obscure and outdated default, even if I told it at install time what I wanted my locale to be, and /etc/locale.gen only contains en_US.UTF8 UTF8 as the only (uncommented) line. And do I need to remember to edit the .bashrc file for every new user account that I create from now on to edit the export line in?

Answer 1

The default locale on Debian is in a UTF-8 encoding, at least since wheezy (I don't remember when it switched). So you must have chosen some non-default setting during the installation.

putting export LANG=en_US.UTF-8 in my .bashrc file.

That's not going to work, because .bashrc is the wrong file. An environment variable defined there only applies to programs launched from a terminal.

The right place to define this environment variable is /etc/environment to make it a system setting or ~/.pam_environment to make it a per-user setting. Both files are read at login time (by all login methods).

The specific locale variable for the character encoding is LC_CTYPE. Setting LANG sets all settings, including LC_COLLATE which can have surprising consequences. You should set only LC_CTYPE.

So remove the LANG setting from your .bashrc and add LC_CTYPE=en_US.utf8 where it belongs, i.e. in /etc/environment or ~/.pam_environment.

Answer 2

do I need to remember to edit the .bashrc file for every new user account that I create from now on to edit the export line in?

The best idea in this case is to put it in a system wide file; first check to see if it's already being set somewhere:

grep -R LANG /etc

You may or may not want to change anything there.

Gilles' answer probably has the most effective place to set this, but I'm leaving what I have here partly to explain what he means by "read at login time (by all login methods)".

You can create a ZZ-local.sh file and put it in /etc/profile.d. The files there get sourced in lexicographical order when a user logs in, so ZZ should override anything previous. The file does not need to be executable or contain a shebang, so the one line export LANG=... should be fine.

Some GUI login display managers don't read .profile -- e.g., lightdm (you may have noticed one of the grep hits to do with this; lightdm exports $LANG in an init script). If that's the case, you'll want to do the same thing as just described WRT /etc/profile.d, but for /etc/X11/Xsession.d. In fact, there's no harm in:

ln -s /etc/profile.d/ZZ-local.sh /etc/X11/Xsession.d/ZZ-local

Symlinking the two files. Just keep in mind that this may mean it will get sourced twice in succession, although if all you are doing is setting $LANG, that's fine.

Answer 3

First, I am surprised that you are able to update Mint 15 to Debian 7.6. They after all are different distributions.

I assume Debian made the decision to use ISO 8859.1 because you are located in Europe and answered some question to that effect regarding the locale.