0

Is there a utility that dumps a file line by line to stdout and deletes those lines?

I have a number of large files that I want to merge via cat. Of course I could do

cat *.txt > merged.txt 

But this will take double the space originally occupied by the txt files. Deleting them after cat like so

cat *.txt > merged.txt && rm -rf *.txt

reduces the storage requirements but still needs 2x the space to be present in the meantime

I'm looking for a command that would stream lines to stdout while at the same time deleting them in the original files:

<some command> *.txt > merged.txt

such that the total space used by *.txt + merged.txt and is never more that what *.txt needed originally

luksen
  • 107

1 Answers1

0

Line by line sounds a bit tricky, but if you accept to do it file by file (so that storage never exceeds all *.txt files + one of them:

find -name "*.txt" -exec cat {} >> result.txt \; -exec \rm -f {} \;

BE VERY CAREFUL! Always check the result of your find command before running it, or you'll be rm'ing stuff that should not be deleted at lightspeed.