AUCTeX isn't loading local per-package style files, is ignoring `TeX-auto-local`

Question

My AUCTeX setup isn't loading any local per-package auto-generated style files. Suppose I have a simple package doc-preamble.sty

% doc-preamble.sty
\usepackage{cleveref}
\newcommand*{\mycmd}[1]{Hello, #1!}

and I use that in doc-main.tex (in the same directory):

% doc-main.tex
\documentclass{article}

\usepackage{doc-preamble}

\usepackage{url}
\newcommand*{\myothercmd}[1]{Goodbye, #1!}

\begin{document}

This is a test.

\end{document}

Then, when working in doc-main.tex, the configuration for doc-preamble.sty isn't loaded. In this case, that means that \cref (from cleveref) and \mycmd don't show up as autocomplete results for TeX-insert-macro (C-c C-m), the \cref reference style isn't available to RefTeX, etc.

The configuration for doc-main.tex, on the other hand, is available, so configuration in general works: commands like \url, as well as \myothercmd, are both available from TeX-insert-macro (C-c C-m), and \url is correctly formatted as a verbatim command.

The value of TeX-active-styles in doc-main.tex is, strangely,

("url" "doc-preamble" "art10" "article" "latex2e" "doc-main" "LATEX")

So AUCTeX knows that doc-preamble is being used, but not what to do when using it.

Relatedly, the AUCTeX .el files are being generated, so that's not the issue. They even look right:

;; doc-preamble.el
(TeX-add-style-hook
 "doc-preamble"
 (lambda ()
   (TeX-run-style-hooks
    "cleveref")
   (TeX-add-symbols
    '("mycmd" 1)))
 :latex)

;; doc-main.el
(TeX-add-style-hook
 "doc-main"
 (lambda ()
   (add-to-list 'LaTeX-verbatim-macros-with-braces-local "url")
   (add-to-list 'LaTeX-verbatim-macros-with-braces-local "path")
   (add-to-list 'LaTeX-verbatim-macros-with-delims-local "url")
   (add-to-list 'LaTeX-verbatim-macros-with-delims-local "path")
   (TeX-run-style-hooks
    "latex2e"
    "article"
    "art10"
    "doc-preamble"
    "url")
   (TeX-add-symbols
    '("myothercmd" 1)))
 :latex)

The doc-preamble.el file simply isn't being loaded as far as I can tell.

---

Thanks to giordano, I was able to find a possible reason for this behavior. (This reason is still a symptom of whatever the underlying problem is.)

My AUCTeX configuration includes the line (setq TeX-auto-local ".auctex-auto"). And while this variable retains its value, when I query TeX-style-path with C-h C-v, I get (reformatted)

TeX-style-path is a variable defined in `tex.el'.
Its value is
("/usr/local/var/auctex" "/Users/antal/.emacs.d/elpa/auctex-11.89.4/style"
 "/Users/antal/.emacs.d/auctex/auto" "/Users/antal/.emacs.d/auctex/style"
 "auto" "style")

Original value was 
("/usr/local/var/auctex" "/Users/antal/.emacs.d/elpa/auctex-11.89.4/style"
 "/Users/antal/.emacs.d/auctex/auto" "/Users/antal/.emacs.d/auctex/style"
 ".auctex-auto" "style")

Note that, somehow, my setting for TeX-auto-local (which correctly has the value ".auctex-auto") is being overwritten in TeX-style-path: the new value uses "auto" where the original used ".auctex-auto"!

Thus, TeX-auto-local is being overridden, but not changed! This pushes the question back to "why is this happening"?

---

I'm using AUCTeX 11.89.4. I'm not sure if I'm doing something wrong, if I've got a bug in my configuration, or what, so I'm not sure which parts of my configuration are important, but here are some potentially relevant pieces of my configuration:

(require 'tex)
(require 'latex)
(require 'reftex)

(setq TeX-auto-save  t
      TeX-parse-self t)

(add-hook 'LaTeX-mode-hook #'reftex-mode)
(setq reftex-plug-into-AUCTeX '(nil t t t t))

(setq TeX-auto-local ".auctex-auto")

(TeX-global-PDF-mode t)

(add-hook 'LaTeX-mode-hook #'auto-fill-mode)
(add-hook 'LaTeX-mode-hook #'flyspell-mode)

There's also some custom advice that I don't think is relevant, but I've included the advice-add calls below in case one of those functions is dangerous to advise (the definitions of the advice functions are available if necessary):

(advice-add #'LaTeX-indent-calculate-last
            :around #'LaTeX-indent-calculate-last--outdent-label)
(advice-add #'TeX-command-sequence-sentinel
            :after #'TeX-sentinel-manage-error-buffer)
(advice-add #'reftex-reference
            :around #'reftex-reference-negatable-prompt
            '((depth . -50)))
(advice-add #'reftex-format-special
            :filter-return #'reftex-format-special-handle-cleveref)
(advice-add #'reftex-ref-style-list
            :filter-return #'reftex-ref-style-list-maybe-preferentially)
(advice-add #'reftex-reference
            :around #'reftex-reference-no-promt-preferentially)

Answer 1

The problem was that I did

(require 'tex)

before

(setq TeX-auto-local ".auctex-auto")

In AUCTeX, the definition of TeX-style-path is via a defcustom form, which sets TeX-style-path to the result of evaluating an expression that builds up a list by referring to the values of TeX-auto-local and similar variables. (require 'tex) before setting TeX-auto-local, the value of TeX-style-path got set to a value that referenced the old value of TeX-auto-local, and was never reinitialized.

If I remove the (require 'tex), or simply move (setq TeX-auto-local ".auctex-auto") before it (there are later parts of my config that require it to be loaded), then everything works fine.

Another option would be to re-initialize TeX-style-path, about which see the next section.

---

One piece of the mysteriousness, though, is that describe-variable (C-h v) for TeX-style-path claimed (reformatted):

TeX-style-path is a variable defined in `tex.el'.
Its value is
("/usr/local/var/auctex" "/Users/antal/.emacs.d/elpa/auctex-11.89.4/style"
 "/Users/antal/.emacs.d/auctex/auto" "/Users/antal/.emacs.d/auctex/style"
 "auto" "style")

Original value was 
("/usr/local/var/auctex" "/Users/antal/.emacs.d/elpa/auctex-11.89.4/style"
 "/Users/antal/.emacs.d/auctex/auto" "/Users/antal/.emacs.d/auctex/style"
 ".auctex-auto" "style")

Why was the "original value" different if nothing had ever changed TeX-style-path? The answer is that defcustom stores, not the initial value of a variable, but the initial expression that created it, and re-evals it to see if the "original value" was different. Since one of the variables that TeX-style-path's definition referenced was the now-changed TeX-auto-local, the "original value" was inaccurate upon recomputation.

This is also why it's possible to reset the value of TeX-style-path to its "default" value, as mentioned above.

---

For reference, the defcustom form that sets TeX-style-path to its default value in AUCTeX version 11.89.4:

(defcustom TeX-style-path
  (let ((path))
    ;; Put directories in an order where the more local files can
    ;; override the more global ones.
    (mapc (lambda (file)
            (when (and file (not (member file path)))
              (setq path (cons file path))))
          (append (list TeX-auto-global TeX-style-global)
                  TeX-auto-private TeX-style-private
                  (list TeX-auto-local TeX-style-local)))
    (nreverse path))
  "List of directories to search for AUCTeX style files.
Per default the list is built from the values of the variables
`TeX-auto-global', `TeX-style-global', `TeX-auto-private',
`TeX-style-private', `TeX-auto-local', and `TeX-style-local'."
  :group 'TeX-file
  :type '(repeat (file :format "%v")))

As we can see, it references TeX-auto-local, and this is the only place that TeX-style-path is set.

Answer 2

Instead of bending TeX-auto-local and TeX-style-path to your will, you might want give TeX-auto-generate a try instead? Also see here: https://tex.stackexchange.com/questions/410393/how-to-properly-set-up-auctex-to-parse-macros-from-my-own-sty-files