The question may be trivial but with this simple for loop
for i in {1..10000000}; do echo "$i" line >> file
done
my shell crashed after some elaboration time (the tab in the terminal emulator was automatically closed) and file was not created.
It is GNU bash, version 4.1.11(2)-release.
Is this due to a memory limit or what else? And if yes which should be this limit?
With {1..1000000} it worked correctly.
The reason this happens, is because the brackets are expanded before the command is invoked. Thus you actually end up with a command like:
for i in 1 2 3 ... 10000000 ; do ...
.. and thus it uses up a lot of memory or it crashes.
The solution when working with long loops is to use the bash's c-style syntax:
for ((i = 1; i <= 10000000; i++))
do
...
done
Or if you want to remain compatible with other shells you can pipe to a while loop:
seq 1 10000000 | while IFS= read -r line
do
...
done
My bash process performing this command takes a lot of memory, yes. 1.9GB in my case (x86_64). You might have run into your limit and the kernel out-of-memory killer has taken it out. For me, it works though. It takes about 3 minutes
$ wc -l file
10000000 file
$ tail -n1 file
10000000 line
$ ls -hl file
-rw-r--r-- 1 seb users 123M Sep 5 13:19 file
To confirm the hypothesis of RAM limitation: If you have swap, you can try to watch it being filled up. Use e.g. htop for this.
If I increase further to looping over 100.000.000 elements, the process eats also ten times more RAM: 18GB.
@user000001 gives an explanation and an alternative. I tested with the C style syntax (which he provided):
for ((i = 1; i <= 10000000; i++))
do
...
done
This process then doesn't take more RAM than a regular bash process.
