22

I have several utility programs that do not have their own directory and are just a single executable. Typically I put the binary in /usr/local/bin. A problem I have is how to manage preference settings.

One idea is to use environment variables and require the user to define such variables, for example, in their bash.rc. I am a little reluctant, however, to clutter up the bash.rc with miscellaneous preference settings for a minor program.

Is there a Standard (or standard recommendation), that defines some place or method that is appropriate for storing preferences for small utility programs that do not have their own directory?

Chris Davies
  • 116,213
  • 16
  • 160
  • 287
Tyler Durden
  • 5,631
  • 14
    When you add user preferences, this makes this tool pretty much unusable for scripting, because transferring a script from one system to another can then change its behaviour. – Simon Richter Nov 24 '20 at 13:42
  • @SimonRichter It's not a script, its a C binary. – Tyler Durden Nov 24 '20 at 14:19
  • 6
    The question is whether a script may want to call your binary and expect that it always behaves the exact same way when called with the same parameters. There is absolutely room for local configuration here, e.g. gpg has a configuration file with the user's identity, so scripts don't have to list it, but you wouldn't want a config file to switch the output format, because a script using gpg to verify a signature would not expect that. – Simon Richter Nov 24 '20 at 15:21
  • 3
    @SimonRichter There can be a flag to select a config file, including no config file at all. For example vim can be run with vim -u NONE. This way it behaves the same on all systems. Or the tool can have a special scripting interface because e.g. parsing its normal output is not recommended. Alternatively the tool can check if it's being run interactively and omit config when it's not, but this might be confusing in some cases. – user31389 Nov 24 '20 at 16:46
  • 1
    @SimonRichter An example of a CLI tool (often used for scripting) with a config file is wget: https://www.gnu.org/software/wget/manual/html_node/Startup-File.html#Startup-File – Sam Estep Nov 26 '20 at 05:07
  • @SamEstep, most of the options typically configured there are things needed for the command to work in the local environment, not changes in behaviour -- but I would absolutely expect scripts to break when my wgetrc specified to send an Accept-Type header, for example. – Simon Richter Nov 26 '20 at 10:13
  • @SimonRichter "git" is one of those tools where the configuration can change behavior. Still very useful though. – Thorbjørn Ravn Andersen Nov 26 '20 at 10:20

4 Answers4

43

Small utilities for interactive desktop use would be expected to follow the XDG Base Directory Specification and keep their config files under

$XDG_CONFIG_HOME

or (if that is empty or unset) default to

$HOME/.config

The picture is a little less clear for non-GUI tools, since they might run on systems which are headless or which don't otherwise adhere to XDG/freedesktop standards.

However, there's no obvious drawback to using $XDG_CONFIG_HOME if set or $HOME/.config if not, and it should be relatively unsurprising everywhere.

Useless
  • 4,800
  • confirmed here as well https://golang.org/pkg/os#UserConfigDir – Zombo Nov 25 '20 at 04:03
  • 4
    The key phrase is for interactive desktop use. Command line executables should probably not have definable preferences at all and should prefer command line arguments for customization of behaviour. – J... Nov 25 '20 at 20:33
  • 2
    I'm not sure I agree with that. Lots of command-line executables have persistent per-user config - for example git has user-level config, and I wouldn't really object if my ~/.gitrc lived in ~/.config/git instead. – Useless Nov 26 '20 at 12:35
  • I'd still say that git falls under "interactive desktop use" though and you can ask it to not use any config file if you want to use it within a script. But tools like cp, ls, dirname etc. shouldn't have a config file, which was what @J... meant. – ljrk Nov 27 '20 at 08:35
  • Sure, but then htop does have a config file. Perhaps the correct distinction is mostly used interactively vs. mostly used in scripts or something, but "CLI programs" is definitely too broad. – Useless Nov 27 '20 at 10:53
  • Yes, I think more precisely the idea being that CLI executables should always produce deterministic results. If the config options don't change that, then OK. – J... Nov 27 '20 at 16:44
  • 1
    No, we're back to the fact that git is an interactive CLI executable, and I absolutely want it to produce commits that depend on the per-user config. You mean "non-interactive" CLI executables, or CLI executables when used non-interactively. It's a fuzzy distinction. – Useless Nov 27 '20 at 16:49
19

Although historically, a lot of programs used $HOME/.$PROGNAME the result is that home directories are becoming a bit messy. The preferred convention seems to be $HOME/.config/$PROGNAME nowadays (I currently have 173 dotfiles in ~, most of which are historical artefacts, and 80 in ~/.config).

Note that there is an important difference between config and preferences. The former is the system wide policy and should only be editable by root, while the latter should controlled by users. Preeferences should NOT be stored anywhere outside of $HOME.

symcbean
  • 5,540
  • Yes, I notice that trend, but the problem is that they are creating a bunch of directories to store just one file. For example, I have ~/.config/sway/config where config is just a single file and it is the only thing in that directory. – Tyler Durden Nov 23 '20 at 22:07
  • 7
    There's nothing to stop you putting a single file in .config rather than a directory - but a single inode is not a huge cost. – symcbean Nov 23 '20 at 22:08
  • 2
    They don't need to do that (but perhaps they want to allow multiple config files in future). Just ~/.config/swaycfg or whatever should be fine too. – Useless Nov 23 '20 at 22:10
  • I see, so I would have something like ~/.config/my-util.config for example? – Tyler Durden Nov 23 '20 at 22:18
  • Yep, that should be fine. – Useless Nov 23 '20 at 22:19
  • 9
    Two small, related advantages to creating a subdirectory instead of just a file: 1) if you ever in the future need another file, you've already got a directory to keep them together, and won't need to handle migrating from single file -> subdir; 2) similarly, if a user wants to keep two versions of the config file for some reason, both can be kept together in the subdir (also pairs nicely with allowing them to pass the config file to use on the command line) – solarshado Nov 24 '20 at 05:34
  • 7
    Please note that ~/.config is the default path, but the actual user-set path is stored in $XDG_CONFIG_HOME. Make sure to use the user's preferred path when it's set and not empty. – user31389 Nov 24 '20 at 16:26
  • @symcbean while "huge cost" is of course subjective and you are correct, a directory is more than an inode, and more importantly, filesystems often try to keep files in a directory together, so creating a subdirectory will likely destroy locality. If all single-file configs are together in one directory, they are likely near to each other on disk, which can have a human-noticable difference in speed. – Remember Monica Nov 26 '20 at 11:53
9

Opinion

You put the utilities into /usr/local/bin so you could reasonably use /usr/local/etc/{program_name} for system-wide settings, or $HOME/.{program_name} for local per-user configuration

In a shell script, since $0 contains the program name, these pathnames could be determined as "/usr/local/etc/${0##*/}" and "$HOME/.${0##*/}"

Chris Davies
  • 116,213
  • 16
  • 160
  • 287
2

Where should small utility programs store their preferences?

This is a matter of opinion !

Is there a Standard (or standard recommendation),

You might read the GNU coding standards then hier(7)

And you should study for inspiration the source code and the documentation of several open source software. BTW the conventions for zsh are different of those for GNU bash and your login shell (often described in /etc/passwd, see passwd(5)) could be changed with chsh(1); I prefer to use zsh


A source of inspiration might be existing GNU programs, including GNU emacs or GCC

When you compile one of them from its source code, you run a ./configure script (generated by GNU autoconf) and that script usually accepts some --help command.

As an example, for GNU emacs, ./configure --help gives

 `configure' configures GNU Emacs 28.0.50 to adapt to many kinds of systems.

Usage: ./configure [OPTION]... [VAR=VALUE]...

To assign environment variables (e.g., CC, CFLAGS...), specify them as VAR=VALUE. See below for descriptions of some of the useful variables.

Defaults for the options are specified in brackets.

Configuration: -h, --help display this help and exit --help=short display options specific to this package --help=recursive display the short help of all the included packages -V, --version display version information and exit -q, --quiet, --silent do not print checking ...' messages --cache-file=FILE cache test results in FILE [disabled] -C, --config-cache alias for--cache-file=config.cache' -n, --no-create do not create output files --srcdir=DIR find the sources in DIR [configure dir or `..']

Installation directories: --prefix=PREFIX install architecture-independent files in PREFIX [/usr/local] --exec-prefix=EPREFIX install architecture-dependent files in EPREFIX [PREFIX]

By default, make install' will install all the files in/usr/local/bin', /usr/local/lib' etc. You can specify an installation prefix other than/usr/local' using --prefix', for instance--prefix=$HOME'.

For better control, use the options below.

Fine tuning of the installation directories: --bindir=DIR user executables [EPREFIX/bin] --sbindir=DIR system admin executables [EPREFIX/sbin] --libexecdir=DIR program executables [EPREFIX/libexec] --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] --datarootdir=DIR read-only arch.-independent data root [PREFIX/share] --datadir=DIR read-only architecture-independent data [DATAROOTDIR] --infodir=DIR info documentation [DATAROOTDIR/info] --localedir=DIR locale-dependent data [DATAROOTDIR/locale] --mandir=DIR man documentation [DATAROOTDIR/man] --docdir=DIR documentation root [DATAROOTDIR/doc/emacs] --htmldir=DIR html documentation [DOCDIR] --dvidir=DIR dvi documentation [DOCDIR] --pdfdir=DIR pdf documentation [DOCDIR] --psdir=DIR ps documentation [DOCDIR]

Program names: --program-prefix=PREFIX prepend PREFIX to installed program names --program-suffix=SUFFIX append SUFFIX to installed program names --program-transform-name=PROGRAM run sed PROGRAM on installed program names

X features: --x-includes=DIR X include files are in DIR --x-libraries=DIR X library files are in DIR

System types: --build=BUILD configure for building on BUILD [guessed] --host=HOST cross-compile to build programs to run on HOST [BUILD]

Optional Features: --disable-option-checking ignore unrecognized --enable/--with options --disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no) --enable-FEATURE[=ARG] include FEATURE [ARG=yes] --disable-ns-self-contained disable self contained build under NeXTstep --enable-locallisppath=PATH directories Emacs should search for lisp files specific to this site --enable-checking[=LIST] enable expensive checks. With LIST, enable only specific categories of checks. Categories are: all,yes,no. Flags are: stringbytes, stringoverrun, stringfreelist, structs, glyphs --enable-profiling build emacs with low-level, gprof profiling support. Mainly useful for debugging Emacs itself. May not work on all platforms. Stops profiler.el working. --enable-autodepend automatically generate dependencies to .h-files. Requires gcc, enabled if found. --enable-gtk-deprecation-warnings Show Gtk+/Gdk deprecation warnings for Gtk+ >= 3.0 --disable-build-details Make the build more deterministic by omitting host names, time stamps, etc. from the output. --disable-largefile omit support for large files --enable-gcc-warnings[=TYPE] control generation of GCC warnings. The TYPE 'yes' means to fail if any warnings are issued; 'warn-only' means issue warnings without failing (default for developer builds); 'no' means disable warnings (default for non-developer builds). --enable-check-lisp-object-type Enable compile time checks for the Lisp_Object data type, which can catch some bugs during development. --enable-link-time-optimization build with link-time optimization (experimental; see INSTALL) --disable-silent-rules verbose build output (undo: "make V=0") --enable-cross-guesses={conservative|risky} specify policy for cross-compilation guesses --disable-acl do not support ACLs

Optional Packages: --with-PACKAGE[=ARG] use PACKAGE [ARG=yes] --without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no) --without-all omit almost all features and build small executable with minimal dependencies --with-mailutils rely on GNU Mailutils, so that the --without-pop through --with-mailhost options are irrelevant; this is the default if GNU Mailutils is installed --with-pop Support POP mail retrieval if Emacs movemail is used (not recommended, as Emacs movemail POP is insecure). This is the default only on native MS-Windows. --with-kerberos support Kerberos-authenticated POP --with-kerberos5 support Kerberos version 5 authenticated POP --with-hesiod support Hesiod to get the POP server host --with-mail-unlink unlink, rather than empty, mail spool after reading --with-mailhost=HOSTNAME string giving default POP mail host --with-sound=VALUE compile with sound support (VALUE one of: yes, alsa, oss, bsd-ossaudio, no; default yes). Only for GNU/Linux, FreeBSD, NetBSD, MinGW, Cygwin. --with-pdumper=VALUE enable pdumper support unconditionally ('yes', 'no', or 'auto': default 'auto') --with-unexec=VALUE enable unexec support unconditionally ('yes', 'no', or 'auto': default 'auto') --with-dumping=VALUE kind of dumping to use for initial Emacs build (VALUE one of: pdumper, unexec, none; default pdumper) --with-x-toolkit=KIT use an X toolkit (KIT one of: yes or gtk, gtk2, gtk3, lucid or athena, motif, no) --with-wide-int prefer wide Emacs integers (typically 62-bit); on 32-bit hosts, this allows buffer and string size up to 2GB, at the cost of 10% to 30% slowdown of Lisp interpreter and larger memory footprint --without-xpm don't compile with XPM image support --without-jpeg don't compile with JPEG image support --without-tiff don't compile with TIFF image support --without-gif don't compile with GIF image support --without-png don't compile with PNG image support --without-rsvg don't compile with SVG image support --without-lcms2 don't compile with Little CMS support --without-libsystemd don't compile with libsystemd support --without-cairo don't compile with Cairo drawing --without-xml2 don't compile with XML parsing support --with-imagemagick compile with ImageMagick image support --without-native-image-api don't use native image APIs (GDI+ on Windows) --with-json compile with native JSON support --without-xft don't use XFT for anti aliased fonts --without-harfbuzz don't use HarfBuzz for text shaping --without-libotf don't use libotf for OpenType font support --without-m17n-flt don't use m17n-flt for text shaping --without-toolkit-scroll-bars don't use Motif/Xaw3d/GTK toolkit scroll bars --without-xaw3d don't use Xaw3d --without-xim at runtime, default X11 XIM to off --without-xdbe don't use X11 double buffering support --with-ns use Nextstep (macOS Cocoa or GNUstep) windowing system. On by default on macOS. --with-w32 use native MS Windows GUI in a Cygwin build --without-gpm don't use -lgpm for mouse support on a GNU/Linux console --without-dbus don't compile with D-Bus support --with-gconf compile with Gconf support (Gsettings replaces this) --without-gsettings don't compile with GSettings support --without-selinux don't compile with SELinux support --without-gnutls don't use -lgnutls for SSL/TLS support --without-zlib don't compile with zlib decompression support --without-modules don't compile with dynamic modules support --without-threads don't compile with elisp threading support --with-file-notification=LIB use a file notification library (LIB one of: yes, inotify, kqueue, gfile, w32, no) --with-xwidgets enable use of xwidgets in Emacs buffers (requires gtk3 or macOS Cocoa) --without-makeinfo don't require makeinfo for building manuals --without-compress-install don't compress some files (.el, .info, etc.) when installing. Equivalent to: make GZIP_PROG= install --with-gameuser=USER_OR_GROUP user for shared game score files. An argument prefixed by ':' specifies a group instead. --with-gnustep-conf=FILENAME name of GNUstep configuration file to use on systems where the command 'gnustep-config' does not work; default $GNUSTEP_CONFIG_FILE, or /etc/GNUstep/GNUstep.conf --with-x use the X Window System --without-libgmp do not use the GNU Multiple Precision (GMP) library; this is the default on systems lacking libgmp. --without-included-regex don't compile regex; this is the default on systems with recent-enough versions of the GNU C Library (use with caution on other systems).

Some influential environment variables: CC C compiler command CFLAGS C compiler flags LDFLAGS linker flags, e.g. -L<lib dir> if you have libraries in a nonstandard directory <lib dir> LIBS libraries to pass to the linker, e.g. -l<library> CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if you have headers in a nonstandard directory <include dir> CPP C preprocessor PKG_CONFIG path to pkg-config utility PKG_CONFIG_PATH directories to add to pkg-config's search path PKG_CONFIG_LIBDIR path overriding pkg-config's built-in search path ALSA_CFLAGS C compiler flags for ALSA, overriding pkg-config ALSA_LIBS linker flags for ALSA, overriding pkg-config XMKMF Path to xmkmf, Makefile generator for X Window System OBJC Objective C compiler command OBJCFLAGS Objective C compiler flags RSVG_CFLAGS C compiler flags for RSVG, overriding pkg-config RSVG_LIBS linker flags for RSVG, overriding pkg-config IMAGEMAGICK_CFLAGS C compiler flags for IMAGEMAGICK, overriding pkg-config IMAGEMAGICK_LIBS linker flags for IMAGEMAGICK, overriding pkg-config GTK_CFLAGS C compiler flags for GTK, overriding pkg-config GTK_LIBS linker flags for GTK, overriding pkg-config WEBKIT_CFLAGS C compiler flags for WEBKIT, overriding pkg-config WEBKIT_LIBS linker flags for WEBKIT, overriding pkg-config DBUS_CFLAGS C compiler flags for DBUS, overriding pkg-config DBUS_LIBS linker flags for DBUS, overriding pkg-config GSETTINGS_CFLAGS C compiler flags for GSETTINGS, overriding pkg-config GSETTINGS_LIBS linker flags for GSETTINGS, overriding pkg-config GCONF_CFLAGS C compiler flags for GCONF, overriding pkg-config GCONF_LIBS linker flags for GCONF, overriding pkg-config GOBJECT_CFLAGS C compiler flags for GOBJECT, overriding pkg-config GOBJECT_LIBS linker flags for GOBJECT, overriding pkg-config LIBGNUTLS_CFLAGS C compiler flags for LIBGNUTLS, overriding pkg-config LIBGNUTLS_LIBS linker flags for LIBGNUTLS, overriding pkg-config LIBSYSTEMD_CFLAGS C compiler flags for LIBSYSTEMD, overriding pkg-config LIBSYSTEMD_LIBS linker flags for LIBSYSTEMD, overriding pkg-config JSON_CFLAGS C compiler flags for JSON, overriding pkg-config JSON_LIBS linker flags for JSON, overriding pkg-config KQUEUE_CFLAGS C compiler flags for KQUEUE, overriding pkg-config KQUEUE_LIBS linker flags for KQUEUE, overriding pkg-config GFILENOTIFY_CFLAGS C compiler flags for GFILENOTIFY, overriding pkg-config GFILENOTIFY_LIBS linker flags for GFILENOTIFY, overriding pkg-config CAIRO_CFLAGS C compiler flags for CAIRO, overriding pkg-config CAIRO_LIBS linker flags for CAIRO, overriding pkg-config FREETYPE_CFLAGS C compiler flags for FREETYPE, overriding pkg-config FREETYPE_LIBS linker flags for FREETYPE, overriding pkg-config FONTCONFIG_CFLAGS C compiler flags for FONTCONFIG, overriding pkg-config FONTCONFIG_LIBS linker flags for FONTCONFIG, overriding pkg-config XFT_CFLAGS C compiler flags for XFT, overriding pkg-config XFT_LIBS linker flags for XFT, overriding pkg-config LIBOTF_CFLAGS C compiler flags for LIBOTF, overriding pkg-config LIBOTF_LIBS linker flags for LIBOTF, overriding pkg-config M17N_FLT_CFLAGS C compiler flags for M17N_FLT, overriding pkg-config M17N_FLT_LIBS linker flags for M17N_FLT, overriding pkg-config HARFBUZZ_CFLAGS C compiler flags for HARFBUZZ, overriding pkg-config HARFBUZZ_LIBS linker flags for HARFBUZZ, overriding pkg-config LCMS2_CFLAGS C compiler flags for LCMS2, overriding pkg-config LCMS2_LIBS linker flags for LCMS2, overriding pkg-config PNG_CFLAGS C compiler flags for PNG, overriding pkg-config PNG_LIBS linker flags for PNG, overriding pkg-config XRANDR_CFLAGS C compiler flags for XRANDR, overriding pkg-config XRANDR_LIBS linker flags for XRANDR, overriding pkg-config XINERAMA_CFLAGS C compiler flags for XINERAMA, overriding pkg-config XINERAMA_LIBS linker flags for XINERAMA, overriding pkg-config XFIXES_CFLAGS C compiler flags for XFIXES, overriding pkg-config XFIXES_LIBS linker flags for XFIXES, overriding pkg-config LIBXML2_CFLAGS C compiler flags for LIBXML2, overriding pkg-config LIBXML2_LIBS linker flags for LIBXML2, overriding pkg-config

Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations.

Report bugs to <bug-gnu-emacs@gnu.org>. GNU Emacs home page: <https://www.gnu.org/software/emacs/>. General help using GNU software: <http://www.gnu.org/gethelp/>.

I often compile the latest GNU emacs snapshot. And I did:

'./configure'  'CC=gcc-10' 'CFLAGS=-O2 -g' '--program-suffix=-trunk' \
               '--prefix=/usr/local' '--sysconfdir=/etc/local'

which is kept in the config.status generated file.

BTW, to get the configuration options of your GCC just run gcc -v

Consider also studying the package management system of your Linux

E.g. Debian (or your favorite GNU/Linux distribution), and reading Linux From Scratch.

Read also Linux Filesystem Hierarchy Standard.

On other POSIX or Unix-like systems (e.g. Solaris or MacOSX) things are slightly different.

You could decide to pass options thru program options. Then see also this answer.

In several open source programs I am coding, I also document conventions about environment variables (see environ(7) and query them using getenv(3) ...) and I often decide to have a common prefix for them. So in the RefPerSys project (an open source symbolic artificial intelligence project) we are using of course REFPERSYS_, for example coding getenv("REFPERSYS_HOME") ... And in Bismon (a static analysis / software engineering tool funded by CHARIOT and DECODER European projects, see this draft report) the prefix is BISMON_

For graphical user interface applications, your favorite GUI framework (such as Qt or GTK) defines its own preferences.

Consider also embedding some interpreter inside your programs

You could easily embed Lua or GNU Guile (or Python) inside your programs coded in C or C++. These interpreters have each their own convention and syntax. The advantage of extending some existing interpreter is potential familiarity for advanced users of these scripting languages.

Consider also extending your programs with plugins.

On Linux, a plugin is loaded by dlopen(3) and used thru dlsym(3) in programs coded in C or C++. If you accept them, please document the public API of your program.

Many major Linux programs (in particular the Firefox browser and the GCC compiler, but also GNU make or zsh) accept plugins. The advantage is that extensibility becomes easier, and the advanced user don't have to recompile some huge open source program to customize it. Another advantage is speed. However, learning a plugin API is a significant effort.