8

I am reading detailed line oriented log file (100MB).

I want to skip some parts by using regex.

Usually I M-x flush-lines but this is destructive operation.

I want to hide many lines by regex and expect it would be efficient when I navigate across file. Extra plus if interactive search skips hidden parts.

From the comments on yue's answer:

  • I want incremental exclusion (first regex, then next regex).

  • I search for anomalies in log and know what to exclude but I don't know what I will have found.

Drew
  • 75,699
  • 9
  • 109
  • 225
gavenkoa
  • 3,352
  • 19
  • 36
  • 2
    Why don't you like `flush-lines`? With a [tiny trick](https://emacs.stackexchange.com/questions/47357/why-is-cloning-of-file-buffers-prohibited) you can clone the file buffer into one that does not visit any file and flush lines there. (1) Do you want a list of flushes you have performed to roll back? (2) Do you want to edit the log file (maybe inserting some kind of markers) and keep these edits in hidden regions? Please, expand your question in that regard. – Tobias Apr 30 '19 at 11:44
  • @Tobias It is exploration of log for some issues and it can be possible that I applied `flush-lines` too aggressively and losing problem. Probably should work on file copy... – gavenkoa Apr 30 '19 at 11:50
  • I.e. you want to be able to roll-back `flush-lines` commands in an [`undo-tree`](https://github.com/apchamberlain/undo-tree.el) fashion maybe including the `undo-tree-visualizer-toggle-diff` feature. It would be even nicer to see the flushed regexps aside the undo tree nodes. Something like that? You can use all that on a buffer clone as I stated [in my first comment](https://emacs.stackexchange.com/questions/50233/hide-lines-by-regex#comment77352_50233). – Tobias Apr 30 '19 at 11:59
  • The [mentioned strategy](https://emacs.stackexchange.com/questions/50233/hide-lines-by-regex?noredirect=1#comment77354_50233) is even reasonably fast with large text files (just tried it with 40MB). – Tobias Apr 30 '19 at 12:06
  • 1
    I know the file is quite large, duplicating the file might not be optimal, but I ended up doing just that. It's just a few steps. `C-x C-w`, `foo.log`, `M-x flush-lines`. After that, you can do whatever you want with the duplicated file. – frogcoder Nov 18 '19 at 12:18

4 Answers4

5

I agree with comments that suggest using flush-lines. I would likely use that in this case, especially since your use case is line-oriented.

But if you want to make text that matches a regexp invisible, so that search, query-replace, etc. ignore it but it is still present, then you can do that with libraries Zones and Isearch+.

For example, you can use C-x n R (command zz-set-zones-matching-regexp) to set the value of the current izones variable to the list of zones that match a given regexp. Then use command isearchp-make-zones-invisible to make those zones invisible.

If you want to make other lines, which match other regexps, then use C-x n r, not C-x n R. The former adds zones to the zone set; the latter sets the zone set (removing any existing zones in it).

There are also these related commands: isearchp-make-zones-visible, isearchp-toggle-zones-invisible, and the same for anti-zones (complement of zones).

C-x n R and C-x n r match a regexp against your text, like re-search-forward does - the search hits define the zones. They are not line-oriented like flush-lines, occur, and grep. So if you want an entire line to become invisible then you need to use a regexp that matches the entire line (containing whatever text you're interested in).

Drew
  • 75,699
  • 9
  • 109
  • 225
3

You want to use occur. This feature is built-in and it does exactly what you want.

There is also a package called loccur, which does the same, but does not create a new window (it just hides all non matching lines).

jue
  • 4,476
  • 8
  • 20
  • I want incremental exclusion (first regex, then next regex). `occur` has positive logic instead of negative and I can't apply exclusion sequentially... – gavenkoa Apr 30 '19 at 11:05
  • I search for anomalies in log and know *what to exclude*. – gavenkoa Apr 30 '19 at 11:07
  • @gavenkoa not sure if I understand correctly, but you can run `occur` in an occur-buffer, then it filters from the already filtered results. – jue Apr 30 '19 at 12:30
  • @gavenkoa you could also run `occur` again in your logfile-buffer, recall the old regex (by using cursor key ) and edit the old regex – jue Apr 30 '19 at 12:39
  • `occur` in `occur` adds level of line numbers as prefix + it is additive logic instead of negative. – gavenkoa Apr 30 '19 at 13:35
0

Try this:

M-x query-replace-regexp RET .*FOO.* C-qC-jRET \,(propertize \& 'invisible t) RET

See also C-hig (elisp)Invisible Text

phils
  • 48,657
  • 3
  • 76
  • 115
0

I found this most easy solution. https://github.com/vapniks/hide-lines

Some functions are here which can be useful for your scenario

hide-lines : Hide lines matching the specified regexp.
Keybinding: M-x hide-lines
hide-lines-not-matching : Hide lines that don’t match the specified regexp.
Keybinding: M-x hide-lines-not-matching
hide-lines-matching : Hide lines matching the specified regexp.
Keybinding: M-x hide-lines-matching
hide-lines-show-all : Show all areas hidden by the filter-buffer command.
Keybinding: M-x hide-lines-show-all
hide-blocks-not-matching Hide text that is not between lines matching START-TEXT and END-TEXT. Keybinding: M-x hide-blocks-not-matching
hide-blocks-matching Hide text that is between lines matching START-TEXT and END-TEXT. Keybinding: M-x hide-blocks-matching
hide-lines-kill-hidden Kill or delete all hidden areas. Keybinding M-x hide-lines-kill-hidden
itirazimvar
  • 568
  • 4
  • 15
  • I need to evaluate this solution because `flush-lines` is quite performant on jumble files (like 200MB of text logs). – gavenkoa Jan 20 '21 at 12:21