1

So I've tried to do this

for x in {1..7000}
do
   echo $x
done

The output is {1..7000}

I've also learned that the C-style for loop doesn't work in mksh

I've done a bit of googling but mostly there is information about ksh, not mksh

So what is the correct way to iterate through a range in mksh ?

NOTE: I am more interested in proper way to create iterations and c-like behavior rather than printing a string.

  • You really should not do that. It's not the correct way to iterate a range in any shell. – mikeserv Apr 27 '15 at 21:19
  • So if I gathered correctly from that answer, for loop isn't suitable for iterations , is that right ? – Sergiy Kolodyazhnyy Apr 27 '15 at 21:37
  • Yes, you should use the until form given in the linked answer; it works on mksh too. – Stephen Kitt Apr 27 '15 at 21:39
  • @Serg - it is suitable for a set you already have - a shell for iterates a list of arguments. But if you're going to be generating the set for sake of the for you're doing twice the work. Just don't. And your whole for loop example can be done as easily as seq 7000. – mikeserv Apr 27 '15 at 21:42
  • Ok, so my question isn't exactly duplicate ( I am more interested in iterations rather than printing something N times, because I might need to iterate some other action 10 times, and I had in mind c-like behavior of for loop ), but I suppose it could be closed, since the basic answer is to use until . . do . . . But if one of you guys wants to post a solution, I'll upvote – Sergiy Kolodyazhnyy Apr 27 '15 at 21:50
  • @Serg - I agree that it's not a duplicate - and a lot of people do equate the shell for w/ the c for - and zsh, ksh93, and bash actually allow for it with for ((num;num++;...__ and so on out of convenience. That syntax isn't portable though, and is a very different thing than for arg in {set...set} in any case. The answer here, by the way, is practically the same as the for arg in {set...set} form actually, except it doesn't mention that it relies on default values for $IFS to work. – mikeserv Apr 27 '15 at 21:53
  • @mikeserv small question about portability: would something like this work accross different shells ? COUNTER=20; while [ $COUNTER -gt 1 ]; do printf "COUTERN IS $COUNTER\n"; let COUNTER=COUNTER-1;sleep 1; done ? – Sergiy Kolodyazhnyy Apr 27 '15 at 23:07
  • No. let is not portable. You can portably use just about any c math function you're used to, though, with a few exceptions. This is portable c=100; while [ "$((c-=1))" -gt 0 ]; do ... but if you're going to be printing per iteration you're going to come up against another c v shell inequity - buffering. The shell wont buffer output - every echo or printf is a literal write() syscall. You can compromise between the extremes like set '"$((i-=1))"'; set "$1" "$1"; set "$@" "$@" "$@" "$@" "$@"; i=100; while [ "$i" -gt 0 ]; do eval "printf 'i = %d\n' $@"; done, if you take my meaning. – mikeserv Apr 27 '15 at 23:23
  • Make that c operator not function. And it does include ternary and subexpressions so stuff like `echo "$((i=${#var}?true_count:(${#other}?false_count:0)))" is totally portable. – mikeserv Apr 27 '15 at 23:35
  • I didn't understand what does the code in the last comment do, can you clarify that a little ? I admit, I am still a noob when it comes to shell scripting. Also, what effect does the buffering have ? Since it's a bunch of write() syscalls, I am assuming that will be loading the cpu and memory, isn't that right ? – Sergiy Kolodyazhnyy Apr 28 '15 at 01:14
  • The native way to do this in mksh is: x=0; while (( ++x < 7000 )); do …; done – For anything else, mikeserv’s answer is worth reading, but it must be noted that seq is not portable either, it’s GNU-specific, the BSDs have jot instead, but iterating that way is even worse than brace expansion, and the while loop and arithmetics are the best way to do it in mksh. – mirabilos Apr 02 '16 at 21:29

1 Answers1

3

So, first of all, as is discussed briefly here the common form...

for arg in {brace..expanded..set}; do ...

... is, in my opinion, exemplary of a shell programming mistake. It generates a formulaic set in a loop, stores every result in a delimited array, then hands another loop said array as a list over which to iterate.

As a general rule, brace expansion is not particularly useful in a programming context because all shell programming is in one way or another macro-like and the brace-expansion already is a macro. While brace-expansion is (also in my opinion) a very useful interactive shell feature precisely because it is a macro, it is probably wasteful otherwise.

Brace-expansion is also not portable (read: POSIX-portable). You can usually rely on it to work in current versions of zsh, bash, yash, and ksh93 but not elsewhere, as far as I know. And even in those shells it can be disabled with shell options and so cannot be definitely relied upon in a shell-function context, though its availability can usually be tested for in $- or in the output of set +o. Between them, each of the aforementioned shells will require different types of tests so if you plan to use it in a function and want to test for it then check your targeted shell's manual for specifics.

In other shells you can get practically the same behavior - except that an outside executable must be called to generate the set - by splitting out a command-substitution's output on $IFS. Where the seq program is available this is easily demonstrated like...

unset IFS       
### ^ensure $IFS default behavior
for arg in $(seq 7000)
do  printf %d\\n "$arg"
done

...or...

unset IFS
printf %d\\n $(seq 7000)

...but, as written, neither offers anything but disadvantage over...

seq 7000

Still - for the first example - the method is very nearly the same as the brace-expanded version. In both cases, some other function or process generates an entire list of arguments which the for control construct then iterates over. In the latter case, though, (for shells other than ksh93) there is also the added overhead of a pipe open/teardown, a forked shell process and its child the seq process, and last the work of splitting its output on $IFS.

$IFS splitting, by the way, happens at very different speeds depending on the shell used. bash, for example, is typically atrocious at simple $IFS splitting performance-wise, whereas dash - depending on the line-length for split strings - can often do it as quickly as some other shells do brace-expansion.

Whether you use brace-expansion or $IFS splitting, though, the end result is that you will have run some loop to generate a formulaic list and then another to iterate that list. What you might have - and probably should have - done instead, is run a single loop over the application of the formula that generates the list, disposed of used iterators as soon as can be, and tested the result of each of each iteration for a satisfactory outcome. It is this kind of thing which you would do in C like...

for ( x = 0; x < 10; x++ ){ ...

That is also practically the same syntax which you can use in bc. And it is very like convenience-syntax forms which might be used in ksh93, zsh or bash like:

for (( x=0; x<10; x++ )); do ...

That is, however, also not portable (again, read: POSIX). Instead, portably, you can do:

iterator=$(( start_value - interval ))
while [ "$(( iterator += interval ))" -le "$end_value" ]; do ...

...or...

iterator=$(( start_value - interval ))
while [ "$(( ( iterator += interval ) <= end_value ))" -ne 0 ]; do ...

POSIX specifies something close to a feature parity for shell arithmetic expansions and C language arithmetic operations - where integers are concerned, that is. And so you can often conveniently and portably iterate over even complicated formulae with only a single expansion - and can even iterate several variables at once in synchronicity in same.

Such a construct, on the other hand, will likely bring you hard against another C language vs. shell language inequity. Shell reads and writes are almost always literally read()s and write()s - each a syscall unto itself - and there is no portable analog to fread(), fwrite() and/or setbuf() in the shell syntax. And so any loop which performs any of those kinds of operations per iteration - even when it does so only with builtins - is bound to result in a performance dog.

You can sometimes reach a happy medium between generating and working an entire set in series or in paralell in a shell script, though.

set 0 1 2 3 4 5 6 7 8 9
for n do printf "$n%d\n" "$@"; done

That does ten write()s in ten iterations and prints a new-line divided sequence from 00 - 99. It's a tiny example - and still wasteful as a 30-byte write() is hardly a significant improvement over a 3-byte write() but it is demonstrative of the kind of method you might use. As I mentioned before, the shell programming language is very macro-like in that everything is a string. And so if you can break your goal set out into smaller, more manageable sets - and then call upon them in a loop context repeatedly to eventually generate your goal set you can usually arrive at a more performant result.

Here's a more complicated, macro-like example:

sh -c '
    i=0 _i=-25
    set "$1" "$1" "$1" "$1" "$1"
    for n do eval "
        printf %b%d $@ $@ $@ $@ $@"
done' -- \
    '"\01$((1+(i<(i+=!(_i=(_i+=25)%275)))))" "$i$_i"'

Which iterates 5 times for a single write() of - on average - about 100 bytes per and prints:

10      125     150     175     1100    1125    1150    1175    1200    1225    1250
20      225     250     275     2100    2125    2150    2175    2200    2225    2250
30      325     350     375     3100    3125    3150    3175    3200    3225    3250
40      425     450     475     4100    4125    4150    4175    4200    4225    4250
50      525     550     575     5100    5125    5150    5175    5200    5225    5250
60      625     650     675     6100    6125    6150    6175    6200    6225    6250
70      725     750     775     7100    7125    7150    7175    7200    7225    7250
80      825     850     875     8100    8125    8150    8175    8200    8225    8250
90      925     950     975     9100    9125    9150    9175    9200    9225    9250
100     1025    1050    1075    10100   10125   10150   10175   10200   10225   10250
110     1125    1150    1175    11100   11125   11150   11175   11200   11225   11250
120     1225    1250    1275
mikeserv
  • 58,310