1

I have a file, which contains ids:

123abc
456dbc

I want to pass them in to a command while I iterate over the the file.

Currently, my command works like this: part_of_command '{"id":{"S":"123abc"}}'. However, now with the file, I want to replace 123abc with whatever is in each line of the file.

I am trying to accomplish something like:

    while read line; do
   command '{"id":{"S":"$line"}}'
    done <output.txt

where line is 123abc or 456dbc or any other id. But the $line is giving errors with the single and double quotes. Any idea on how to fix this?

random
  • 11
  • 3

3 Answers3

2

It seems you want to pass a nested JSON object as an argument to some command. The shell won't expand a variable that is inside single quotes.

The solution here is not to switch to using double quotes, though, as the data you are injecting into the JSON document might need encoding for it to be valid JSON (if it contains tabs, quotes, or other characters that would break the format).

There are two good tools for creating the JSON for you. The simplest to use is jo:

while IFS= read -r line; do
    utility "$( jo id="$( jo S="$line" )" )"
done <input.txt

The other is jq:

while IFS= read -r line; do
    utility "$( jq -c --arg data "$line" -n '{ id: { S: $data } }' )"
done <input.txt

Both of these loops would correctly create a nested JSON object with your read data, possibly JSON-encoded, and call the utility utility with that as an argument.

With jq, you could even turn the loop inside-out, as it were, to avoid having to call jq in every iteration. Doing this relies on using -c, which makes jq output each element of the resulting set is a single line ("compact output").

jq -c -R '{ id: { S: . } }' input.txt |
while IFS= read -r json; do
    utility "$json"
done

... or get it to produce your commands and eval them:

eval "$(
   jq -r -R '[ "utility", ({ id: { S: . } } | @json) ] | @sh' input.txt
)"

Since you're calling the file that you read from output.txt in the question, there are presumably some steps in some workflow that generates it. It may be possible to integrate the calling of utility much earlier in your pipeline or bypass it altogether, depending on what you are doing. Especially if the IDs are coming from some JSON document.

Kusalananda
  • 333,661
1

Just put your string in double quotes and not in simple quotes. Don't forget to backslashes internal double quotes.

while read line; do
    command "{'\"id\"':{\"S\":\"$line\"}}"
done <output.txt
0

You can mix quotes in the same shell string, so if one part of the string is easier to write with single quotes and another part requires a substitution, you can use single quotes for the literal part and double quotes around the substitution.

However, note that the resulting JSON will not be well-formed if $line contains a double quote, a backslash or control characters.

Note also that a plain read line does some backslash interpretation and strips whitespace. See Understanding "IFS= read -r line"

while IFS= read -r line; do
  case $line in
    *[\\\"]*) echo >&2 "I'm not smart enough to handle special characters. Giving up."; exit 1;;
  esac
  command '{"id":{"S":"'"$line"'"}}'
done