Cannot merge files using cat

Question

I tried to open a lot of files (10,000+) with cat and them merging them into one like this:

cat * > ~/Desktop/lol.xml

But it returned this error:

-bash: /bin/cat: Argument list too long

This means that its too long/large, what other way could I do this?

You could check the answer here. – Ramesh Nov 10 '14 at 16:25 — Ramesh, Nov 10 '14 at 16:25
Also, some more useful read from here as well. – Ramesh Nov 10 '14 at 16:30 — Ramesh, Nov 10 '14 at 16:30

score 4 · Answer 1 · answered Nov 10 '14 at 16:24

4

find . -maxdepth 1 -type f --exec cat {} + > ~/Desktop/lol.xml

This calls cat with the maximum possible number of arguments. For the remaining arguments new instances of cat are started.

answered Nov 10 '14 at 16:24

Hauke Laging

Huh, that was quick, could you provide an explanation for this? It seems like it launches another cat process for every file. – DisplayName Nov 10 '14 at 16:25
2

-maxdepth is not POSIX, you can use find ! -name . -prune -type f instead. – cuonglm Nov 10 '14 at 16:28
Is this compatible with FreeBSD (OS X) tools? Just checking before. – DisplayName Nov 10 '14 at 16:33
@DisplayName I don't know. -exec + is part of the standard but has not always been. If that doesn't work then you can get a similar effect with find ... -print | xargs -d '\n' cat >... assuming there are no newlines in the file names. – Hauke Laging Nov 10 '14 at 17:21

score 3 · Accepted Answer · answered Nov 10 '14 at 16:25

3

There's a limit for the number of arguments a command can take. A workaround is to use a for loop :

for file in *; do cat "$file"; done

The maximum can be displayed with :

$ getconf ARG_MAX

answered Nov 10 '14 at 16:25

Gilles Quénot

1

One cat instance per file, really? – Hauke Laging Nov 10 '14 at 16:26
For a one-shot interactive command, this is sufficient I think. – Gilles Quénot Nov 10 '14 at 16:45

2 Answers2