how to use custom-set-variables safely?

Question

The help for custom-set-variables says:

The arguments should each be a list of the form:

  (SYMBOL EXP [NOW [REQUEST [COMMENT]]])

This stores EXP (without evaluating it) as the saved value for SYMBOL.
If NOW is present and non-nil, then also evaluate EXP and set
the default value for the SYMBOL to the value of EXP.

REQUEST is a list of features we must require in order to
handle SYMBOL properly.

So if NOW is nil then EXP won't be evaluated immediately, which suggests to me that it should be possible for EXP to refer to variables or functions provided by features in the REQUEST list which are not yet loaded.

Even if my interpretation is wrong, it should be possible to safely use custom-set-variables to customize variables in features which are not yet loaded. For example, if I have something like:

(custom-set-variables
 '(yas-snippet-dirs
   ('("~/.config/yasnippets/rails")) nil (yasnippet)))

Then I would expect this variable only to be set once yasnippet is loaded.

It turns out that custom-set-variables tries to load yasnippet immediately. This means that the custom-file containing this directive has to be loaded after all other user init code which takes care of ensuring that yasnippet is available (e.g. loading it from MELPA or el-get or via req-package). That's all well and good if yasnippet is available, but what if it isn't available, or at least not yet available, due to some deferred or lazy-loading?

Currently this will result in a backtrace ending in something like this:

Debugger entered--Lisp error: (file-error "Cannot open load file" "no such file or directory" "yasnippet")
  require(yasnippet)
  mapc(require (yasnippet))
  custom-theme-set-variables(user (yas-snippet-dirs (quote ("~/.config/yasnippets/rails")) nil (yasnippet)))

which will then break the rest of the user's initialisation, which is really bad. OTOH, if I just remove the REQUEST argument in order to allow deferred loading, then presumably the value of specifying that argument is lost.

So what's the correct way to safely customize variables for packages which are lazy-loaded or loaded on demand?

(BTW in case it's not obvious, I'm looking for a solution which doesn't bypass custom.el, because then I lose the ability to customize via custom.el's nice UI. Otherwise I would simply do setq inside the :config section of my use-package declaration ... Doing custom-set-variables in the same place doesn't work either, because updating the value via the custom.el UI would still update the file pointed to by custom-file rather than this correct location.)

Answer 1

When NOW is non-nil, what happens is that EXP is evaluated (and used to set the corresponding variable) only when the variable's defcustom is evaluated.

The REQUEST part will always be used right away.

If the var's defcustom has already been evaluated (or is evaluated while handling the REQUEST), then the value of NOW will have no effect.

Why not just do

(custom-set-variables
 '(yas-snippet-dirs '("~/.config/yasnippets/rails")))

which will set yas-snippet-dirs when/if yasnipet is loaded?

The REQUEST argument is very rarely needed. The typical case where it was needed was for minor modes, where enabling foo-mode should load the corresponding file (since nothing else would load it otherwise), and this is now handled differently: those minor mode defcustom are expected to be marked as ;;;###autoload, so Emacs knows to autoload the file without needing a REQUEST. The advantage is also that the actul name of the file that defines this variable is not hard-coded in the user's ~/.emacs, so it won't fail if the function is moved or its file is renamed.