2

How can I make sed not append a newline character to an input stream that was missing one? If I can't, what's the quickest way to remove a newline character from a variable?

A code sample was requested in the comments, so here you go. Imagine a .csv file that has no newline at the end. Pipe it into a script.

Then inside that script:

function foobarize() {
    sed \
    -e "s|foo|bar|g" \
    $1
}

INPUT=`tee`
printf "$INPUT" | foobarize

If the input has no newline at the end, foobarize will add a newline to it 100% of the time.

slm
  • 369,824
CommaToast
  • 375
  • 2
    Can you please include the example you're dealing w/ along with any code you've been trying to get to work? – slm Sep 09 '14 at 05:30
  • I tried it and got no added newlines. – John1024 Sep 09 '14 at 05:46
  • When you say you tried it, how did you try it? – CommaToast Sep 09 '14 at 06:02
  • echo -n abc | foobarize on a linux box. – John1024 Sep 09 '14 at 06:07
  • but sed adds a newline. It says so in its documentation. Maybe it's just the version of sed that's on BSD – CommaToast Sep 09 '14 at 06:11
  • @CommaToast - a command substitution will strip that newline. So printf %s "$(printf %s "$INPUT" | foobarize)" < that works. – mikeserv Sep 09 '14 at 06:12
  • See: Why does sed add a new line in OSX? – John1024 Sep 09 '14 at 06:19
  • @mikeserv a command substitution will strip the newline, but if the newline was already in the file that would ruin it. Anyway your other answer on my other question has answered this one, in a way. When git streams its input to my filter, now I add a character to the end of the stream before feeding it to sed. Then I feed it back through command substitution and then strip the added character at the end before sending it along. I hate unix. It's as if it thinks its a typewriter. They really need to invent a version of unix that has no reserved characters, and uses an extended space for such. – CommaToast Sep 09 '14 at 06:38
  • @CommaToast - that's all shell stuff. I don't fully understand why you have a need to push it through a variable at all. In fact, as I suspect, your real problem is that you're saving information you don't have to. Anyway, I'm glad this much has worked out for you. – mikeserv Sep 09 '14 at 07:29
  • @mikeserv - well actually, my real problem was being accustomed to modern programming languages, I never anticipated that trailing newlines would be a "gotcha" in shell scripting. That being said, ultimately it is much cleaner to have used a variable assignment based on the nature of the code I was writing. Take a look, maybe you'll think there was a better way to do it but I felt it made the code nice and readable: https://github.gistya.com/expandr (see the expandr.sh file). That being said there are always several ways to skin a cat. – CommaToast Sep 10 '14 at 14:41
  • i did look - you need to change the bangline for that - its not an sh script - its a bash script maybe. it would be better if it were modular - you can do a sed bangline, you know. i prefer to do everything possible at the start and then to pass off all of the args i need onto the next process - in a row. the shell is glue you know. its great at mangling arguments any which way y0u like. it sucks at storing/understanding them. – mikeserv Sep 10 '14 at 14:43
  • Why is it not an sh script? It works on my system. Just curious, I'm new to this. – CommaToast Sep 10 '14 at 14:45
  • @CommaToast - well there are many reasons, but the most standout one is your use of shell array type variables. Portably the shell is not specified to handle arrays like that - rather each function/script/shell-process is assigned its very own argument array that manipulated with commands like getopts, set, and shift while each parameter is stored in $[num], the total index count in $# and the entire array in special parameters $* and $@. What you're using are bash arrays - they don't work in sh. Likely readlink /bin/sh will point to bash on your machine. – mikeserv Sep 10 '14 at 16:11
  • I realize we're off-topic here now, but thanks for that information. Indeed on Darwin 12.5, sh --version reports GNU bash, version 3.2.48(1). For now I'll change the bangline to bash. Regarding modularity, even in bash you can't export arrays which is why I was forced to make it all one script. However one script may be more CPU-efficient in my use case because git is multithreaded and will be running n number of instances of this script simultaneously. If it's spawning multiple shells due to each script calling multiple sub-scripts, I would expect that to slow things down. – CommaToast Sep 10 '14 at 17:46

0 Answers0