So, first of all, as is discussed briefly here the common form...
for arg in {brace..expanded..set}; do ...
... is, in my opinion, exemplary of a shell programming mistake. It generates a formulaic set in a loop, stores every result in a delimited array, then hands another loop said array as a list over which to iterate.
As a general rule, brace expansion is not particularly useful in a programming context because all shell programming is in one way or another macro-like and the brace-expansion already is a macro. While brace-expansion is (also in my opinion) a very useful interactive shell feature precisely because it is a macro, it is probably wasteful otherwise.
Brace-expansion is also not portable (read: POSIX-portable). You can usually rely on it to work in current versions of zsh
, bash
, yash
, and ksh93
but not elsewhere, as far as I know. And even in those shells it can be disabled with shell options and so cannot be definitely relied upon in a shell-function context, though its availability can usually be tested for in $-
or in the output of set +o
. Between them, each of the aforementioned shells will require different types of tests so if you plan to use it in a function and want to test for it then check your targeted shell's manual for specifics.
In other shells you can get practically the same behavior - except that an outside executable must be called to generate the set - by splitting out a command-substitution's output on $IFS
. Where the seq
program is available this is easily demonstrated like...
unset IFS
### ^ensure $IFS default behavior
for arg in $(seq 7000)
do printf %d\\n "$arg"
done
...or...
unset IFS
printf %d\\n $(seq 7000)
...but, as written, neither offers anything but disadvantage over...
seq 7000
Still - for the first example - the method is very nearly the same as the brace-expanded version. In both cases, some other function or process generates an entire list of arguments which the for
control construct then iterates over. In the latter case, though, (for shells other than ksh93
) there is also the added overhead of a pipe open/teardown, a forked shell process and its child the seq
process, and last the work of splitting its output on $IFS
.
$IFS
splitting, by the way, happens at very different speeds depending on the shell used. bash
, for example, is typically atrocious at simple $IFS
splitting performance-wise, whereas dash
- depending on the line-length for split strings - can often do it as quickly as some other shells do brace-expansion.
Whether you use brace-expansion or $IFS
splitting, though, the end result is that you will have run some loop to generate a formulaic list and then another to iterate that list. What you might have - and probably should have - done instead, is run a single loop over the application of the formula that generates the list, disposed of used iterators as soon as can be, and tested the result of each of each iteration for a satisfactory outcome. It is this kind of thing which you would do in C like...
for ( x = 0; x < 10; x++ ){ ...
That is also practically the same syntax which you can use in bc
. And it is very like convenience-syntax forms which might be used in ksh93
, zsh
or bash
like:
for (( x=0; x<10; x++ )); do ...
That is, however, also not portable (again, read: POSIX). Instead, portably, you can do:
iterator=$(( start_value - interval ))
while [ "$(( iterator += interval ))" -le "$end_value" ]; do ...
...or...
iterator=$(( start_value - interval ))
while [ "$(( ( iterator += interval ) <= end_value ))" -ne 0 ]; do ...
POSIX specifies something close to a feature parity for shell arithmetic expansions and C language arithmetic operations - where integers are concerned, that is. And so you can often conveniently and portably iterate over even complicated formulae with only a single expansion - and can even iterate several variables at once in synchronicity in same.
Such a construct, on the other hand, will likely bring you hard against another C language vs. shell language inequity. Shell reads and writes are almost always literally read()
s and write()
s - each a syscall unto itself - and there is no portable analog to fread()
, fwrite()
and/or setbuf()
in the shell syntax. And so any loop which performs any of those kinds of operations per iteration - even when it does so only with builtins - is bound to result in a performance dog.
You can sometimes reach a happy medium between generating and working an entire set in series or in paralell in a shell script, though.
set 0 1 2 3 4 5 6 7 8 9
for n do printf "$n%d\n" "$@"; done
That does ten write()
s in ten iterations and prints a new-line divided sequence from 00 - 99. It's a tiny example - and still wasteful as a 30-byte write()
is hardly a significant improvement over a 3-byte write()
but it is demonstrative of the kind of method you might use. As I mentioned before, the shell programming language is very macro-like in that everything is a string. And so if you can break your goal set out into smaller, more manageable sets - and then call upon them in a loop context repeatedly to eventually generate your goal set you can usually arrive at a more performant result.
Here's a more complicated, macro-like example:
sh -c '
i=0 _i=-25
set "$1" "$1" "$1" "$1" "$1"
for n do eval "
printf %b%d $@ $@ $@ $@ $@"
done' -- \
'"\01$((1+(i<(i+=!(_i=(_i+=25)%275)))))" "$i$_i"'
Which iterates 5 times for a single write()
of - on average - about 100 bytes per and prints:
10 125 150 175 1100 1125 1150 1175 1200 1225 1250
20 225 250 275 2100 2125 2150 2175 2200 2225 2250
30 325 350 375 3100 3125 3150 3175 3200 3225 3250
40 425 450 475 4100 4125 4150 4175 4200 4225 4250
50 525 550 575 5100 5125 5150 5175 5200 5225 5250
60 625 650 675 6100 6125 6150 6175 6200 6225 6250
70 725 750 775 7100 7125 7150 7175 7200 7225 7250
80 825 850 875 8100 8125 8150 8175 8200 8225 8250
90 925 950 975 9100 9125 9150 9175 9200 9225 9250
100 1025 1050 1075 10100 10125 10150 10175 10200 10225 10250
110 1125 1150 1175 11100 11125 11150 11175 11200 11225 11250
120 1225 1250 1275
until
form given in the linked answer; it works onmksh
too. – Stephen Kitt Apr 27 '15 at 21:39for
iterates a list of arguments. But if you're going to be generating the set for sake of thefor
you're doing twice the work. Just don't. And your wholefor
loop example can be done as easily asseq 7000
. – mikeserv Apr 27 '15 at 21:42until . . do . .
. But if one of you guys wants to post a solution, I'll upvote – Sergiy Kolodyazhnyy Apr 27 '15 at 21:50for
w/ the cfor
- andzsh
,ksh93
, andbash
actually allow for it withfor ((num;num++;...__
and so on out of convenience. That syntax isn't portable though, and is a very different thing thanfor arg in {set...set}
in any case. The answer here, by the way, is practically the same as thefor arg in {set...set}
form actually, except it doesn't mention that it relies on default values for$IFS
to work. – mikeserv Apr 27 '15 at 21:53COUNTER=20; while [ $COUNTER -gt 1 ]; do printf "COUTERN IS $COUNTER\n"; let COUNTER=COUNTER-1;sleep 1; done
? – Sergiy Kolodyazhnyy Apr 27 '15 at 23:07let
is not portable. You can portably use just about any c math function you're used to, though, with a few exceptions. This is portablec=100; while [ "$((c-=1))" -gt 0 ]; do ...
but if you're going to be printing per iteration you're going to come up against another c v shell inequity - buffering. The shell wont buffer output - every echo or printf is a literalwrite()
syscall. You can compromise between the extremes likeset '"$((i-=1))"'; set "$1" "$1"; set "$@" "$@" "$@" "$@" "$@"; i=100; while [ "$i" -gt 0 ]; do eval "printf 'i = %d\n' $@"; done
, if you take my meaning. – mikeserv Apr 27 '15 at 23:23write()
syscalls, I am assuming that will be loading the cpu and memory, isn't that right ? – Sergiy Kolodyazhnyy Apr 28 '15 at 01:14mksh
is:x=0; while (( ++x < 7000 )); do …; done
– For anything else, mikeserv’s answer is worth reading, but it must be noted thatseq
is not portable either, it’s GNU-specific, the BSDs havejot
instead, but iterating that way is even worse than brace expansion, and thewhile
loop and arithmetics are the best way to do it in mksh. – mirabilos Apr 02 '16 at 21:29