Unexpected indentation in python chunks

Question

Already one year ago, I described a problem on github with polymode and unexpected indents. Since there was no answer there so far and the problem is really annoying, I ask here for ideas for a workaround of the problem:

I am using polymode like this for pycode environments in latex documents:

(defcustom pm-inner/python   (pm-inner-chunkmode "python"
         :head-matcher "\\\\begin{\\(pycode\\|pythontexcustomcode\\|sagecode\\)}"
         :tail-matcher "\\\\end{\\(pycode\\|pythontexcustomcode\\|sagecode\\)}"
         :mode 'python-mode)   "python typical chunk."   :group 'innermodes   :type 'object)

(defcustom pm-poly/latex-python   (pm-polymode "latex-python"
         :hostmode 'pm-host/latex
         :innermodes '(pm-inner/python))   "latex-python typical polymode."   :group 'polymodes   :type 'object)

(define-polymode poly-latex+python-mode pm-poly/latex-python)
(add-to-list 'auto-mode-alist '("\\.tex$" . poly-latex+python-mode))

Now the pycode environment may have an optional argument for the python session name, for example

\begin{pycode}[SessionName] a = 2 \end{pycode}

If I hit Enter at the end of a line, in the next line it automatically adds some unwanted tabs like in the following screen cast:

In a pycode environment without optional argument however it works as expected.

My polymode version is 20190624.1927 in emacs 26.1

Answer 1

When you enter a newline in Python mode, it calls the hook electric-indent-post-self-insert-function. This function calls (eventually) pm-indent-line-dispatcher, provided by polymode to fill the role of indent-according-to-mode.

In order for polymode to know what mode you're in, and what the appropriate indentation for that mode is, it needs to know the mode of the current span, and, most crucially here, what the extent of that span is. Knowing where the span starts depends on the value of :head-matcher. That is your problem!

You defined your head matcher as:

:head-matcher "\\\\begin{\\(pycode\\|pythontexcustomcode\\|sagecode\\)}"

This matches the first part of your pycode block:

\begin{pycode}[SessionName]

This means that polymode considers your python span to start with [SessionName], and it correctly tries to match the indentation when you enter a new line. That's why you get tabs inserted. They are unwanted by you, but necessary to bring the indentation out to match the beginning of [SessionName].

Knowing that, we can fix your problem by correcting the head-matcher, such that it matches all possible span heads:

(defcustom pm-inner/python   (pm-inner-chunkmode "python"
         :head-matcher "\\\\begin{\\(pycode\\|pythontexcustomcode\\|sagecode\\)}\\(\\[[^]\\]+]\\)?"
         :tail-matcher "\\\\end{\\(pycode\\|pythontexcustomcode\\|sagecode\\)}"
         :mode 'python-mode)   "python typical chunk."   :group 'innermodes   :type 'object)

This works for my tests, but may require additional tweaking if there are edge cases I'm not aware of. Here, I'm assuming you might have a single option surrounded by [], with no following text. It works ok if there is whitespace after the option, but additional non-whitespace will confuse it.

If you want to match everything to the end of the line, you could use this:

:head-matcher "\\\\begin{\\(pycode\\|pythontexcustomcode\\|sagecode\\)}\\(\\[[^]\\]+]\\)?.*$"