2

If I run in the CLI:

curl time.com | sed -n 's/.*href="\([^"]*\).*/\1/p' | tr " " "\n"

then, as expected, I get a list of sanitized links from the page to STDOUT, each on new line.

However, when I save that to a variable and try to echo that from a script.sh:

PAGE_LINKS=$(curl time.com | sed -n 's/.*href="\([^"]*\).*/\1/p' | tr " " "\n")
echo $PAGE_LINKS

I get all links on one line, space separated. So as if the tr was ignored.

I tried multiple things including something like

HREFS=$(tr " " "\n" < "{PAGE_LINKS}")
echo $HREFS

But then I get file too long error. Any suggestions?

Andrejs
  • 315
  • 2
  • 4
  • 10

2 Answers2

5

According to the bash man page for the $(command) construct:

Bash performs the expansion by executing command and replacing the command substitution with the standard output of the command, with any trailing newlines deleted. Embedded newlines are not deleted, but they may be removed during word splitting.

So tr is not the problem, but rather bash either deletes the newlines if they are trailing, and removes any other newlines during word splitting. This is behavior as documented.

I believe you want this behavior in most places. If you have a file with a list of file names in it, then:

for FILENAME in $(cat somefile)
do
     ...
done

Iterates over the list of file names. You wouldn't want the newlines in somefile to mess up your list of words to use as filenames, and maybe even mess up your for-do-done loop.

4

The problem is not tr, the problem resides in how you are output-ing the variable expansion:

echo $PAGE_LINKS

Quote the variable expansion:

echo "$PAGE_LINKS"

otherwise the expansion will go through word splitting according to the value of IFS (space, tab, newline by default), and pathname expansion (*, ?, []).

In your case, the word splitting is happening, and each newline separated element is being taken individually, and being shown as space separated entities finally. Using quotes will prevent word splitting (and pathname expansion), so the whole expansion will be taken as a single entity.

heemayl
  • 56,300