Highlight a comma-separated list by using rx macro

Question

I want to create font locking for string like:

interface Foo extends B, C, D, E {};

And I can't understand why my regexp rule doesn't work for font locking:

(defvar my-font-lock-keywords
  `(
    ;; Some code is omitted for brevity
    ;; ...

    ;; Just for example
    (,(rx "interface Foo extends B"
          (1+ ?,
              (syntax whitespace)
              (group (or "A" "B" "C" "D"))))
     1 font-lock-type-face)

    ;; Some code is omitted for brevity
    ;; ...
))

This test fails:

(ert-deftest my-mode-syntax-table/fontify-extends/3 ()
  :tags '(fontification syntax-table)
  (test-with-temp-buffer
   "interface Foo extends B, C, D, E {}; "
   (should (eq (test-face-at 23) 'font-lock-type-face))))

As I can see there is no problem with regexp:

ELISP> (require 'rx)
rx
ELISP> (string-match-p
        (rx (1+ ?, (syntax whitespace) (group (or "A" "B" "C" "D"))))
        "B, C, D, E")
1 (#o1, #x1, ?\C-a)

Of course I'm managed to succeed for other cases e.g.:

interface Foo {};

My real code is more complicated. I tried to give a simple example to show my problem.

How to correctly make font-lock regexp for the comma-separated lists?

Update: I'm managed to debug regexp (thanks to @phils) via M-x re-builder so as I said before there is no problem with regexp

Answer 1

"interface Foo extends B "

You have a space after B and therefore cannot match B,

Handy hint: Use M-x re-builder to test your regexps. You can switch it to rx syntax via C-c TAB (although it actually uses rx-to-string with the consequence that the implied sequence in your form is lost, so you would need to make that explicit with '(sequence "interface Foo extends B " ....) or equivalent.

Answer 2

Your code works straight out of the box, in a way. When you apply it, the letter "D" is highlighted.

The problem with the code, as it is written, is that only one match is highlighted. In your case, it matches a number of ,<LETTTER>, but when the face is applied, the match data points to the last in the list. (The reason E isn't highlighted is that your regexp only match A, B, C, and D.

A better approach is to use anchored keywords. Think of them as a search within a search. For example:

(defvar my-font-lock-keywords
  '(("\\_<interface .* extends\\_>"
     ("[, \t]*\\(\\<[a-zA-Z_]+\\_>\\)"
      nil   ;; Pre-match form
      nil   ;; Post-match form
      (1 font-lock-type-face)))))

First, font-lock search for interface NAME extends. (If you like, you can extend the rule to highlight the name and the keywords.) Once this is found, it repeatedly search for identifiers (skipping commas and whitespace) and highlight each one using font-lock-type-face.

Normally, the sub-search is performed on the current line, but this can be changed by making the pre-match form return an integer representing the end.

Update: You can use this when defining a new major mode, for example:

(define-derived-mode my-mode fundamental-mode "MY"
  "My mode."
  (setq font-lock-defaults '(my-font-lock-keywords nil)))

Or, if you want to add the rules to an existing mode (in this case the fictitious whatever-mode), you can do something like:

(defun my-whatever-mode-hook ()
  (font-lock-add-keywords
   nil
   '(("\\_<interface .* extends\\_>"
      ("[, \t]*\\(\\<[a-zA-Z_]+\\_>\\)"
       nil
       nil
       (1 font-lock-type-face))))))
(add-hook 'whatever-mode-hook #'my-whatever-mode-hook)

Answer 3

I'm managed to succeed with fontification multiple inheritance list by using following approach:

(defun syntax-propertize-inheritance (start end)
  "Fontify all items in a inheritance list between START and END."
  (save-match-data
    (save-excursion
      (goto-char start)
      (while (re-search-forward rx-to-match-each-inherit-item end t)
        (put-text-property (match-beginning 1) (match-end 1)
                           'face 'font-lock-type-face)
      ))))

Where rx-to-match-each-inherit-item is my regexp to match item in multiple inheritance list.

(defun custom-syntax-propertize-function (start end)
  "Apply propertize rules from START to END."
  (let ((case-fold-search))
    (goto-char start)
    (funcall
     (syntax-propertize-rules
      ;; Propertize multiple inheritance causes.
      (rx-to-match-full-inherit-list
       (0 (ignore (syntax-propertize-inheritance
                   (match-beginning 0)
                   (match-end 0))))))
     start end)))

Where rx-to-match-full-inherit-list is my regexp to match full multiple inheritance list.

;; Used in mode definition
(setq-local syntax-propertize-function #'custom-syntax-propertize-function)