162

I've got 14 files all being parts of one text. I'd like to merge them into one. How to do that?

Ivan
  • 17,708

6 Answers6

276

This is technically what cat ("concatenate") is supposed to do, even though most people just use it for outputting files to stdout. If you give it multiple filenames it will output them all sequentially, and then you can redirect that into a new file; in the case of all files just use ./* (or /path/to/directory/* if you're not in the directory already) and your shell will expand it to all the filenames (excluding hidden ones by default).

$ cat ./* > merged-file

Make sure you don't use the csh or tcsh shells for that which expand the glob after opening the merged-file for output, and that merged-file doesn't exist before hand, or you'll likely end up with an infinite loop that fills up the filesystem.

The list of files is sorted lexically. If using zsh, you can change the order (to numeric, or by age, size...) with glob qualifiers.

To include files in sub-directories, use:

find . ! -path ./merged-file -type f -exec cat {} + > merged-file

Though beware the list of files is not sorted and hidden files are included. -type f here restricts to regular files only as it's unlikely you'll want to include other types of files. With GNU find, you can change it to -xtype f to also include symlinks to regular files.

With the zsh shell,

cat ./**/*(-.) > merged-file

Would do the same ((-.) achieving the equivalent of -xtype f) but give you a sorted list and exclude hidden files (add the D qualifier to bring them back). zargs can be used there to work around argument list too long errors.

Stéphane Chazelas
  • 544,893
Michael Mrozek
  • 93,103
  • 40
  • 240
  • 233
  • 26
    Beware that your quoted command will probably only do what the poster wants if they're numbered in such a way that the shell expands * in "natural" order. If you have "file1.txt...file9.txt...file14.txt" it won't work because file1?.txt will sort between file1.txt and file2.txt. You'd have to rename them to "file01.txt...file09.txt...file14.txt". Say echo * if you're not sure. – Warren Young Nov 04 '10 at 21:43
  • 3
    @Warren: good point (or you can use zsh and set its numeric_glob_sort option). – Gilles 'SO- stop being evil' Nov 04 '10 at 23:04
  • 2
    @warren-young a correct, useful warning comment. But in my actual case the order makes no difference (because files contain just simple SQL statements inserting data records which have no dependencies). – Ivan Nov 04 '10 at 23:16
  • 2
    Beware, if the count of files exceeds a certain limit, you can run in errors like - /bin/cat: Argument list too long – Nupur Aug 05 '15 at 13:46
  • Beware that command can go wild with recursion, infinitely adding merged-file to itself. Always specify merged-file path outside of the current dir. – ARA1307 Sep 20 '18 at 02:08
  • 1
    @ARA1307 Only if the file already exists; otherwise the glob will be expanded before the shell opens the file to write to it. Good point in that situation though – Michael Mrozek Sep 20 '18 at 03:45
  • @MichaelMrozek, this is not true for csh\tcsh. In these shells recursion happens even if resulting file doesn't exist. – ARA1307 Sep 20 '18 at 04:27
  • I think different about -ing vs -ed. If you have a lot of files. unfortunately your system down on merging. you don't know merging or merged difference. so I recommend cat * > merging; mv merging merged – seunggabi Jan 09 '21 at 15:17
33

If your files aren't in the same directory, you can use the find command before the concatenation:

find /path/to/directory/ -name *.csv -print0 | xargs -0 -I file cat file > merged.file

Very useful when your files are already ordered and you want to merge them to analyze them.

More portably:

find /path/to/directory/ -name *.csv -exec cat {} + > merged.file

This may or may not preserve file order.

Wildcard
  • 36,499
3nrique0
  • 461
15

The command

$ cat * > merged-file

actually has the undesired side-effect of including 'merged-file' in the concatenation, creating a run-away file. To get round this, either write the merged file to a different directory;

$ cat * > ../merged-file

or use a pattern match that will ignore the merged file;

$ cat *.txt > merged-file
Christopher Jones
  • 175
  • 15
    cat * > merged-file works fine. Globs are processed before the file is created. If merged-file already exists, cat (mine at least) will detect that it's the output file and refuse to read it. IF the file already exists AND you have the redirect later in the pipeline, then it obviously can't do that, so then and only then do you get the runaway file. – Kevin Feb 21 '12 at 22:48
  • cat has no way to detect if the file is the output one. The redirection happens in the shell; cat only prints on stdout. – bfontaine Sep 11 '17 at 18:48
11

Like the other ones from here say... You can use cat

Lets say you have:

~/file01
~/file02
~/file03
~/file04
~/fileA
~/fileB
~/fileC
~/fileD

And you want only file01 to file03 and fileA to fileC:

cat ~/file01 ~/file02 ~/file03 ~/fileA ~/fileB ~/fileC > merged-file

Or, using brace expansion:

cat ~/file0{1..3} ~/file{A..C} > merged-file

Or, using fancier brace expansion:

cat ~/file{0{1..3},{A..C}} > merged-file

Or you can use for loop:

for i in file0{1..3} file{A..C}; do cat ~/"$i"; done > merged-file
Florin Idita
  • 131
2

You can specify the pattern of a file then merge all of them as follows:

cat *pattern* >> mergedfile
Kevdog777
  • 3,224
user182845
  • 39
0

Another option is sed:

sed r 1.txt 2.txt 3.txt > merge.txt

Or...

sed h 1.txt 2.txt 3.txt > merge.txt

Or...

sed -n p 1.txt 2.txt 3.txt > merge.txt # -n is mandatory here

Or without redirection ...

 sed wmerge.txt 1.txt 2.txt 3.txt

Note that last line write also merge.txt (not wmerge.txt!). You can use w"merge.txt" to avoid confusion with the file name, and -n for silent output.

Of course, you can also shorten the file list with wildcards. For instance, in case of numbered files as in the above examples, you can specify the range with braces in this way:

sed -n w"merge.txt" {1..3}.txt
Harini
  • 101