How can I search between the lines with multiline search?

Question

I use C-u 4 M-x helm-swoop for interactive search also show several surrounding lines (this is analogous to grep -C 4).

When I find the word I narrow my search interactively, for example, C-u 4 M-x helm-swoop foo bar And this is analogous to grep -C 4 foo files | grep bar.

I want to narrow my search more and search only code between the lines #+BEGIN_SRC python and #+END_SRC.

#+BEGIN_SRC python
    search this code with "grep -c"
#+END_SRC

How can I do it with helm-swoop or helm-occur? Is there another solution?

Drew · Answer 1 · 2019-07-14T02:16:06.913

You apparently want to search only within certain contexts, defined by those two Org labels, #+BEGIN_SRC python and #+END_SRC.

You can use Icicles search to do that, defining the search contexts as text that matches a regexp, which you can input at the prompt:

#[+]BEGIN_SRC python\(.\|[\n]\)+#[+]END_SRC

But as a shortcut (and easier on the eyes) you can use the toggle key C-M-. to make typing just . have the effect of typing \(.\|[\n]\) (to match any char, including a newline), so just:

#[+]BEGIN_SRC python.+#[+]END_SRC

Then type 0 (accept the default), to match the whole regexp, (subgroup 0).

That searches all text from your start to your finish text. If you don't want to include the delimiters then you can use this instead, and enter 1 for the subgroup:

#[+]BEGIN_SRC python\(.+\)#[+]END_SRC

Then, to search within the contexts, just type whatever pattern (e.g. a regexp) you want to match into the minibuffer and hit S-TAB to see the matches as completions in buffer *Completions*. The matches update as you change the pattern in the minibuffer. You can navigate among contexts by cycling (e.g. C-<down>). You can narrow the candidates further with S-SPC and another search pattern. Etc.

But that context-defining regexp is far too simple. As you noted in a comment, it matches the text between the first #+BEGIN_SRC and the last #+END_SRC, producing only one context, which is useless.

Defining the contexts you really want is trickier. For that, I looked in the Org code for a suitable regexp and found org-babel-src-block-regexp, which matches the text of any source block (not just python), including matching the delimiters #+begin_src and #+end_src.

Here's the value of org-babel-src-block-regexp:

"^\\([  ]*\\)#\\+begin_src[     ]+\\([^ \f  \n
]+\\)[
]*\\([^\":\n]*\"[^\"\n*]*\"[^\":\n]*\\|[^\":\n]*\\)\\([^\n]*\\)\n\\([^\0]*?\n\\)??[
]*#\\+end_src"

That's the string form. To use icicle-search interactively you would need to enter this at the prompt to define the regexp, and then enter 3 as the subgroup of the regexp to use as the context:

^\([    ]*\)#[+]BEGIN_SRC[  ]+\([^  

]+\)[
]*\(\([^\":
]*\"[^\"
*]*\"[^\":
]*\|[^\":
]*\)\([^
]*\)
\([^\0]*?
\)??\)[
]*#[+]END_SRC

That's org-babel-src-block-regexp, but:

With an extra \(...\) added around the part between the two delimiters.
As a value that you would input in the minibuffer as a regexp. E.g., for \n you would use C-q C-j (which inserts a newline char).

Clearly you wouldn't want to be entering a regexp like that interactively! Fortunately, you can also use function icicle-search to define your own search command.

Here's an Icicles-search command that incorporates org-babel-src-block-regexp, but with an extra \\(...\\) added around the part between the two delimiters:

(defvar org-python-section-regexp "^\\([    ]*\\)#\\+BEGIN_SRC[     ]+python[
]*\\(\\([^\":\n]*\"[^\"\n*]*\"[^\":\n]*\\|[^\":\n]*\\)\\([^\n]*\\)\n\\([^\0]*?\n\\)??\\)[
]*#\\+END_SRC"
  "Regexp to match the content of Org-bable Python source sections.")

(defun icy-search-org-python-sections (beg end)
  "`icicle-search' with Org-babel python sections as contexts.
Type a regexp to match within each context."
  (interactive `(,@(icicle-region-or-buffer-limits)))
  (let ((icicle-search-context-level  2)
        (icicle-transform-function    (and (not (interactive-p))
                                           icicle-transform-function)))
    (unwind-protect
        (progn (face-remap-set-base 'icicle-search-main-regexp-others nil)
               (icicle-search beg end org-python-section-regexp t))
      (when icicle-search-cleanup-flag (icicle-search-highlight-cleanup))
      (face-remap-reset-base 'icicle-search-main-regexp-others))))

This is the part of that regexp that matches the text between the delimiters:

\\(\\([^\":\n]*\"[^\"\n*]*\"[^\":\n]*\\|[^\":\n]*\\)\\([^\n]*\\)\n\\([^\0]*?\n\\)??\\)

That's thanks to the Org code -- glad I didn't have to come up with it!

All I did was take the value of org-babel-src-block-regexp and substitute python for the more general pattern \\([^ \f \n ]+\\), and then wrap the between-delimiters part in \\( ... \\) so I could tell Icicles search to pick up all of that stuff.

Then I bound icicle-search-context-level to 2, which is the regexp subgroup number to use for the context.

(See also the Isearch answer below. There are advantages to each: Icicles search and Isearch.)

With ```M-x icicles-search``` highlighted all text beetween first ```'#[+]BEGIN_SRC'``` and last ```'#[+]END_SRC'```, then ```Moving to sole context``` and ```Removing search highlighting...done```. That's all. — Pfedj, Jul 10 '19 at 16:46
Right; thx. Updated to take care of that. Hope the explanation is clear enough. — Drew, Jul 11 '19 at 15:33

Drew · Answer 2 · 2019-07-14T14:54:44.147

This answer uses Isearch, not Icicles search. To make use of it you need library isearch-prop.el: Search zones of text that have certain text properties (or overlays that have certain overlay properties).

First, here's a regexp that identifies the content of the Org-babel sections you cite. (This is the same defvar as in the Icicles answer here - see that answer for more about the regexp.)

(defvar org-python-section-regexp "^\\([    ]*\\)#\\+BEGIN_SRC[     ]+python[
]*\\(\\([^\":\n]*\"[^\"\n*]*\"[^\":\n]*\\|[^\":\n]*\\)\\([^\n]*\\)\n\\([^\0]*?\n\\)??\\)[
]*#\\+END_SRC"
  "Regexp to match the content of Org-bable Python source sections.")

This command isearches forward in the contexts you want. If you want regexp isearch forward then substitute function isearchp-regexp-context-regexp-search for isearchp-regexp-context-search in this code.

(defun isearch-org-python-sections (&optional beg end)
  "Isearch within org-babel python sections, as contexts.
If ‘isearchp-complement-domain-p’ is non-nil then search *outside* the
contexts defined by the regexp.  (Use ‘M-= ~’ during Isearch to toggle
this variable.  Search is limited to the region if it is active."
  (interactive
   (let ((beg  (if (and transient-mark-mode  mark-active) (region-beginning) (point-min)))
         (end  (if (and transient-mark-mode  mark-active) (region-end) (point-max))))
     (list beg end)))
  (let ((isearchp-context-level  2))
    (isearchp-regexp-context-search nil beg end nil org-python-section-regexp)))

While searching, the area outside the zones is dimmed slightly, by default. This is from option isearchp-dim-outside-search-area-flag. (You can toggle this dimming during Isearch using C-M-D (aka C-M-S-d.)

Tobias · Answer 3 · 2019-07-15T08:47:38.200

The newest master of Elgrep provides searching of context.

It is available on Melpa. You can install it by M-x package-install RET elgrep RET if melpa is in your package-archives.

The command elgrep-menu, bound to the new menu item Tools -> Search Files (Elgrep)..., provides you with the Elgrep menu where you can specify your search.

The items "Context Lines Before The Match" and "Context Lines After The Match" are bold. If you click on them you can choose between "Number of Lines" and "Regexp" for the context.

Input the regexp

^[[:space:]]*#\+BEGIN_SRC python

for the context lines before the match and the regexp

^[[:space:]]*#\+END_SRC

for the context lines after the match.

Clearly, you should also specify the regexp you want to search for within the specified context as "Regular Expression" in the Elgrep menu.

How can I search between the lines with multiline search?

3 Answers3