SMIE Basic Identation and Block Example

Question

I've tried for a while to create a simple major mode that uses SMIE to perform navigation and indentation, but it seems like even the simple examples that I've found seems to break quite easily.

The mode I'm trying to create consist of semi-colon separated statements and blocks of the construct "AttributeBegin" and "AttributeEnd". E.g.:

test;

AttributeBegin
    test;

    AttributeBegin
        test;
    AttributeEnd
AttributeEnd

AttributeBegin
    test;
AttributeEnd

test;

# ...

(This also shows how I want it to be indented.)

To this end, I created the following SMIE grammar and SMIE rules:

(defvar smie-sample-grammar
  (smie-prec2->grammar
   (smie-bnf->prec2
    `((insts (inst) (insts ";" insts))
      (inst ("AttributeBegin" inst "AttributeEnd")))
    '((assoc ";"))))
  "Sample BNF grammar for `smie'.")

(defun smie-sample-rules (kind token)
  "Perform indentation of KIND on TOKEN using the `smie' engine."
  (pcase (list kind token)
    (`(:elem basic) smie-sample-indent-offset)
    (`(:elem arg) 0)))

This mostly works, but I'm getting some strange behaviors in several cases:

test;
AttributeBegin
    test;
AttributeEnd

    # (1)

    AttributeBegin # <--- Why is this intended?
        test;
    AttributeEnd

    AttributeBegin # (2)
        test;
    AttributeEnd

    test;

For some reason, the block pointed to above is erroneously indented, but for some reason, (2) is not. Additionally, if I add a statement at (1), the following block is correctly indented.

I believe that I've written an incorrect grammar/rule pair, but I don't see what I need to change in order to fix it. Most examples I've seen so far doesn't seem to require special rules for blocks like this, so is there something I'm missing?

For completeness, the entire code I used for testing these things are here.

score 3 · Accepted Answer · answered Jul 26 '19 at 18:44

Your problem lies within the indentation rules.

You must align the text after "AttributeEnd" to the parent token of "AttributeEnd", i.e., its opener "AttributeBegin".

You can do that with case

('(:after "AttributeEnd") (smie-rule-parent))

in function smie-sample-rules.

There follows a complete minimal working example.

First the Elisp code:

(require 'smie)

;; We are using SMIE's default lexer.
;; A token is:
;; 1. any sequence of characters that have word or symbol syntax
;; 2. any sequence of characters that have punctuation syntax

(defvar smie-sample-grammar nil
  "Sample BNF grammar for `smie'.")

(setq smie-sample-grammar
  (smie-prec2->grammar
   (smie-bnf->prec2
    `((insts (insts ";" insts) (inst))
      (inst ("AttributeBegin" insts "AttributeEnd")))
    '((assoc ";")))))

(defun smie-sample-rules (kind token)
  "Perform indentation of KIND on TOKEN using the `smie' engine."
  (pcase (list kind token)
    ('(:after "AttributeEnd")
     (smie-rule-parent))
    ('(:elem arg) 1)))

(define-derived-mode smie-sample-mode prog-mode "ExSMIE"
  "Usage example for the Simple Minded Indentation Engine."
  :syntax-table nil
  (modify-syntax-entry ?\# "<")
  (modify-syntax-entry ?\n ">")
  (setq comment-start "#"
    comment-end "")
  (smie-setup smie-sample-grammar #'smie-sample-rules)
  (font-lock-add-keywords nil '(("AttributeBegin" . font-lock-keyword-face)
                ("AttributeEnd" . font-lock-keyword-face)))
  (font-lock-mode)
  (font-lock-ensure (point-min) (point-max)))

Second an extented example text after identation by smie:

test;
AttributeBegin
    test;
AttributeEnd

# (1)

AttributeBegin # <--- Why is this indented?
    test;
    AttributeBegin
        test2;
    AttributeEnd;
    test3;
    AttributeBegin
        test2;
    AttributeEnd
AttributeEnd

test4;

AttributeBegin # (2)
    test;
AttributeEnd

test;

Tested with:
emacs-version: GNU Emacs 26.2 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.22.30) of 2019-04-12

score 3 · Answer 2 · answered Jul 26 '19 at 22:18

If you replace AttributeBegin and AttributeEnd with open and close parens, you code is basically of the form:

test;
(...)
    (...)
    (...)

The reason for this indentation is that there is no ; between the three parenthesized elements. Basically, SMIE parses the above as a ; operator with test on the left and (...) (...) (...) on the right. And SMIE doesn't know how those 3 thingys relate and its default is to consider that these are part of a kind of "function call", which it intends like:

funct
    arg1
    arg2

If instead (like here) all 3 should be aligned, you can tell that to SMIE via the :list-intro where you'll need to tell SMIE that "arguments to the ; operator are not like function calls but more like lists":

(pcase (list kind token)
  (`(:list-intro . ";") t)
  (`(:elem basic) smie-sample-indent-offset)
  (`(:elem arg) 0)))

You may need to do that for more than just ; depending on your language. If you M-x smie-edebug RET you should be able to see the calls to your smie-rules-function where kind is equal to :list-intro.

SMIE Basic Identation and Block Example

2 Answers2

Linked