3

When I want to replace, say Dot with

Dot
##empty line##

I do this:

sed 's/Dot/Dot\
/g'

Yes, a literal new line, this is the way it works on BSD sed. But, I can't do the same when the sed is inside parentheses. Example command:

lol=$(echo $varcontaining something | sed 's/Dot/Dot\
/g')

Because it will only replace is with . (whitespace)

Braiam
  • 35,991
DisplayName
  • 11,688
  • Because trailing newlines are removed in command substitution, even if you quote or not. See: http://unix.stackexchange.com/questions/17747/why-does-shell-command-substitution-gobble-up-a-trailing-newline-char – cuonglm Nov 28 '14 at 16:13

4 Answers4

2

What about these methods mentioned in this Stackoverflow Q&A titled: How do I use a new-line replacement in a BSD sed?:

sed -e 's/ /\'$'\n/g'

Or another method mentioned where you put the newline in a variable like so:

cr="
"

Then use the variable:

sed "s/ /\\${cr}/g"
Community
  • 1
slm
  • 369,824
  • i usually use the $var and i often include the backslash in its value as well - backslashes suck in doublequotes. – mikeserv Nov 30 '14 at 12:11
1

Shell removes the trailing newlines. The common workaround is to add a character at the very end in the command substitution, and then remove it.

x=$( echo a; echo ; echo :)
echo "${x%:}"
choroba
  • 47,233
  • This will produce two newlines, not one. – jimmij Nov 28 '14 at 16:47
  • How would i do if I were to echo the variable again and do some other text processing stuff? – DisplayName Nov 28 '14 at 16:50
  • @jimmij: This will keep all the trailing newlines. If you don't want two of them, don't output them. Or, remove the last one, too: "${x%$'\n':}" (bash syntax). – choroba Nov 28 '14 at 16:52
  • @DisplayName: What do you mean? You can assign the substitution to a variable: y=${x%:} – choroba Nov 28 '14 at 16:53
  • I mean, say echo "${x%: | sed blablabla | awk afdiusf | grep | iiadfhidsuhfisdu | It doesn't work. – DisplayName Nov 28 '14 at 16:53
  • @DisplayName: Missing right }. – choroba Nov 28 '14 at 16:54
  • Thats not what i meant, but whatever i fixed it now. – DisplayName Nov 28 '14 at 16:57
  • This gives me a bad substitution error ?sqlinsertconvert8="${echo sqlinsert8%: | sed la | sed '/kal/d'}" – DisplayName Nov 28 '14 at 17:12
  • @DisplayName: Misplaced }: sqlinsertconvert8=$(echo "${sqlinsert8%:}" | sed la | sed '/kal/d') – choroba Nov 28 '14 at 17:14
  • @DisplayName: But it's a command substitution again, so it removes the trailing newlines... – choroba Nov 28 '14 at 17:15
  • Yeah... Its probably a bad solution to loop over and over again. – DisplayName Nov 28 '14 at 18:40
  • @Displayname - well, looping over and over again is exactly what sed would do - at least per line. Still, the subshell is the biggest problem here - there's a clone() for every one invoked. And for each |pipe within it there are two more - and an execve() for sed. It is better to stage the processing - do as much with what youve got in the current process as you possibly can do in a sane way - then pass the whole of it on to another process at one outpoint to do its part in the same way. Very often people store data which the shell shouldnt store anyway in shell variables. – mikeserv Nov 30 '14 at 13:37
1

Probably the most effective way to do this - as I consider it - is with eval. If you want literal values out of a shell expansion then the most straightforward way to get them is out of literal input - and the only way to get that out of a shell's output is to feed it right back in.

It is usually easiest to work with 'hard-quotes. The simple quote rule for a 'hard-quoted' string is that it cannot contain a hard-quote - the only way to get one interpreted in such a context is to concatenate 2 or more quoted strings together like...

VAR='string'\''more string'

In that way the first quoted string ends at the second quote, the second quoted string is only a single backslash-escaped hard-quote, and third is the fourth hard-quote through the end.

And so what I might do if were you is...

eval "lol='$(nl='\
';    printf %s\\n "$lol" |
      sed "s/Dot/&$nl/g
           s/'"'/&\\&&/g
          $s/$/'"'/
")"

In that way the same trailing newline which printf adds to the var's value is the one that the command substitution strips - which is also the last character in output because the second-to last character is always a hard-quote - and that safely delimits the eval statement because sed escapes any hard-quotes which may appear in its input and there is one at the head of the string.

In truth, though, using command substitutions like that is often not a good practice anyway - and perhaps that is why it doesn't fit quite right all of the time. It is generally more effective to gather all output from a process for each of its process loops and pass it off at once on a single stream to another filter process in a pipeline than it is to nickle-and-dime the filters in that way.

And unless you have huge values in your shell variables - which is also usually a bad idea - something like this is probably a better way to go:

set -- "$lol" 
while case "$1" in (*Dot*)
set -- "${1%Dot*}" "${1##*Dot}Dot
$2";;(*) ! lol=$1$2;; esac
do :; done
mikeserv
  • 58,310
0

You can use newline \n together with carriage return \r. I don't have BSD sed at hand to test, but it works with GNU sed.

$ var=Dot
$ lol=$(echo $var | sed 's/Dot/Dot\n\r/g')
$ echo "$lol"
Dot

$

If BSD sed doesn't support \r then another (rather ugly) solution is to print \n\r with echo -e or printf:

lol=$(echo $var | sed "s/Dot/Dot\\$(printf '\n\r')/g")
jimmij
  • 47,140