0

context: https://stackoverflow.com/questions/52299524/duplicate-output-looping-through-multiple-values-in-while-loop-bash/52301168#52301168

Nota: this will work simply while each file do contain 1 word by line.

And if you plan to use space in your source files, you could use another delimiter:

while IFS=@ read -r aws_user_name aws_key aws_account_num ;do
    printf "User: %-16s Key: %-22s AccNum: %s\n" \
        "$aws_user_name" "$aws_key" "$aws_account_num"
    # ....
done < <(
    paste -d@ "$aws_users_all" "$aws_env_list" "$aws_account_numbers"
)

I am not sure IFS=@ paste -d@ mean? I understand paste -d "|,", I can aslo figure it out IFS=";" mean. I also following https://stackoverflow.com/questions/7427262/how-to-read-a-file-into-a-variable-in-shell https://man7.org/linux/man-pages/man1/bash.1.html

jian
  • 567

2 Answers2

4

Since you understand paste -d "|," then you can also infer the meaning of paste -d@ - it's the same process - instead of separating fields with |, they're separated with @. (In common with many UNIX/Linux tools a flag and its argument can be joined together or separated with a space; standard parsing libraries don't care - see POSIX Utility Argument Syntax, point 2a.)

In this situation the IFS variable determines the way that the read verb parses its input. The variables aws_user_name, aws_key, and aws_account_num are read from the output of paste -d@ and split at the @ symbol that paste has just added.

You can partially disect the command to see what's going on; running the paste command (with appropriate filenames for the three variables) will show you what the while read … loop is going to process

paste -d@ "$aws_users_all" "$aws_env_list" "$aws_account_numbers"

Be aware that this kind of processing will break if any of the source values contains the delimiter (@).

Chris Davies
  • 116,213
  • 16
  • 160
  • 287
  • Something maybe worth mentioning is that it's important that the delimiter used here be a single-byte and non-whitespace one. Single-byte because several paste implementations including GNU paste don't support multibyte ones (and some shells, not bash, don't support them in IFS either), and non-white space ones because the whitespace ones receive a special treatment when it comes to $IFS splitting. So @ is a good choice because it's single byte and non whitespace (and presumably cannot occur in the data) but paste's default delimiter (TAB) would be a poor choice. – Stéphane Chazelas Dec 24 '21 at 11:35
  • Ah, yes. I do appreciate your knowledge here; I'd not seen (i.e. read and understood the detail of), "If IFS has a value other than the default, then sequences of the whitespace characters space, tab, and newline are ignored at the beginning and end of the word, as long as the whitespace character is in the value of IFS" until now. – Chris Davies Dec 24 '21 at 14:07
  • Since 5.0, it's all single byte characters classified as whitespace that receive the special treatment like in ksh93. POSIX requires it to be all whitespace characters, but AFAIK, yash is the only shell that honours that requirement. Traditionally (in ksh which introduced that behaviour and most other shells), that was limited to SPC, TAB and NL. ksh93 and zsh allow the special treatment to be removed by doubling the character in $IFS (with IFS=$'\t\t', you would be able to use the default paste delimiter there). – Stéphane Chazelas Dec 24 '21 at 14:20
4

FWIW, more as a comment than an answer (which has already been given by @roaima), it would have made more sense to write it:

while
  IFS= read -r aws_user_name <&3 &&
    IFS= read -r aws_key <&4 &&
    IFS= read -r aws_account_num <&5
do
  printf "User: %-16s Key: %-22s AccNum: %s\n" \
    "$aws_user_name" "$aws_key" "$aws_account_num"
    # ....
done 3< "$aws_users_all" 4< "$aws_env_list" 5< "$aws_account_numbers"

Instead of joining with @ to later split again on @.

That way, having @ in values doesn't cause problems, and missing elements in the 3 lists are correctly detected. It also avoids the <(...) kshism, making it standard sh code.

If all the loop does is print stuff, it would even have made more sense to use awk / perl or some proper text processing utility than running several invocations of some clunky tools in a loop like that. More on that at Why is using a shell loop to process text considered bad practice?

Stéphane Chazelas
  • 544,893