Emacs: Open a buffer with all lines between lines X to Y from a huge file

Question

In the same spirit as this other question: cat line X to line Y on a huge file:

Is there a way to open from within Emacs (and show on a buffer) a given set of lines (e.g. all lines between line X and Y) from a huge text file?

E.g. Open and show in a buffer all lines between lines 57890000 and 57890010 from file huge.txt

Update: I am interested in a solution that at least can open the lines in read-only (just for display purposes), although it would be great if I can also edit the lines (and save to the original file).

Answer 1

If you want to open the whole file (which requires ), but show only part of it in the editor window, use narrowing. Select the part of the buffer you want to work on and press C-x n n (narrow-to-region). Say “yes” if you get a prompt about a disabled command. Press C-x n w (widen) to see the whole buffer again. If you save the buffer, the complete file is selected: all the data is still there, narrowing only restricts what you see.

If you want to view a part of a file, you can insert it into the current buffer with shell-command with a prefix argument (M-1 M-!); run the appropriate command to extract the desired lines, e.g. <huge.txt tail -n +57890001 | head -n 11.

There is also a Lisp function insert-file-contents which can take a byte range. You can invoke it with M-: (eval-expression):

(insert-file-contents "huge.txt" nil 456789000 456791000)

Note that you may run into the integer size limit (version- and platform-dependent, check the value of most-positive-fixnum).

In theory it would be possible to write an Emacs mode that loads and saves parts of files transparently as needed (though the limit on integer sizes would make using actual file offsets impossible on 32-bit machines). The only effort in that direction that I know of is VLF (GitHub link here).

Answer 2

You can do this with View Large Files, an emacs minor mode designed for exactly this case.

Answer 3

You may find this perl and elisp combination useful. It allows you to pipe data to a buffer. Subsequent invocations using the same buffer-name will append the new lines to the same buffer.

You can "edit" the buffer, but the edit in no way reflects back to the source (which is a pipe) ... It doesn't show any line numbers, though you can tweak the input to include a numbered prefix for each line.

from=50000000
  to=50000010
<file_50 head -n "$to" | tail -n +"$from" | e-sink.pl

In the buffer:

<<<<< start: 2012-09-09T01:39:49
1000000
VSjU K97X5Z dFcc ZZd2OqQ PzbnphT
yQBTt LOic Ks sPXrq tty oy
dA8 SD BvO daZ KFPr44X
X0m3BI eR4go YjFp7e vbJr3oe Y0OGgH3 uPfz yfq59
we rm L9iD ugcJBND daS

7pO lwUFzNE HPlPW fmPZ vpRs Rx EFeHaFM
b0 1B ncr Db324 vwO Un34R
HDZS wq9zg W013 5JGly
kAfP QPpjjyh pXMAw I1 CGKDc23 qCBnP
<<<<<   end: 0.630s

Or, with line numbers added:

from=50000000
  to=50000010
<file_50 head -n "$to" | tail -n +"$from" | nl -v$from -ba -w${#to} | e-sink.pl

In the buffer:

<<<<< start: 2012-09-09T01:53:44
50000000    1000000
50000001    VSjU K97X5Z dFcc ZZd2OqQ PzbnphT
50000002    yQBTt LOic Ks sPXrq tty oy
50000003    dA8 SD BvO daZ KFPr44X
50000004    X0m3BI eR4go YjFp7e vbJr3oe Y0OGgH3 uPfz yfq59
50000005    we rm L9iD ugcJBND daS
50000006    
50000007    7pO lwUFzNE HPlPW fmPZ vpRs Rx EFeHaFM
50000008    b0 1B ncr Db324 vwO Un34R
50000009    HDZS wq9zg W013 5JGly
50000010    kAfP QPpjjyh pXMAw I1 CGKDc23 qCBnP
<<<<<   end: 0.768s

I found this on a StackOverflow Q/A