33

I have a text file with following data and each row ends with |END|. 

T|somthing|something|END|T|something2|something2|END|

I am tryig to replace |END| with \n new line with sed. 

 sed 's/\|END\|/\n/g' test.txt

But it's producing wrong output like below:

 T
 |
 s
 o
 m
 e
 ...

But what I want is this:

T|somthing|something
T|something2|something2

I also tried with tr. It didn't work either.

Gilles 'SO- stop being evil'
  • 829,060
Sas
  • 1,083

7 Answers7

41

Use this:

sed 's/|END|/\n/g' test.txt

What you attempted doesn't work because sed uses basic regular expressions, and your sed implementation has a \| operator meaning “or” (a common extension to BRE), so what you wrote replaces (empty string or END or empty string) by a newline.

Gilles 'SO- stop being evil'
  • 829,060
A.B.
  • 3,442
  • Need to comment out the \ in \n: sed 's/|END|/\n/g – Baazigar Apr 02 '15 at 18:42
  • @Baazigar No, what A.B. wrote is correct (for Linux at least, some sed implementations would emit \n). The question asks how to replace |END| by a newline, not by \n. – Gilles 'SO- stop being evil' Apr 03 '15 at 12:08
  • The characters for newline are '\n'. The \n is needed because \ also is an escape character, so if you do only \n, you are saying 'escape this n character'. When you do \n you are saying 'don't treat this next \ as an escape.'. – Baazigar Apr 17 '15 at 13:28
8

The following worked fine for me:

$ sed 's/|END|/\
/g' foobar
T|somthing|something
T|something2|something2

Notice that I just put a backslash followed by the enter key.

unxnut
  • 6,008
5

You can use awk:

$ awk -F'\\|END\\|' '{$1=$1}1' OFS='\n' file
T|somthing|something
T|something2|something2
  • -F'\\|END\\|' set field separator to |END|
  • OFS='\n' set ouput field separator to newline
  • $1=$1 cause awk reconstruct $0 with OFS as field separator
  • 1 is a true value, causeawk print the whole input line
cuonglm
  • 153,898
3

Another possibly command and using its RS option would be:

awk '$1=$1' RS="\|END\|" file

Will print those records (based on awk's Record Separator) which are not empty( has at least one field) to prevent printing empty lines.

Tested on this input:

T|somthing|something|END|T|something2|something2|END|
Test|END|
|END|

Gives this output:

T|somthing|something
T|something2|something2
Test

That cleared all empty lines :) If you want to have newlines too, replace $1=$1 with $0 in command:

awk '$0' RS="\|END\|" file
αғsнιη
  • 41,407
  • $1=$1 condenses sequences of blanks into one space character and returns false if the first field is 0. Doesn't make sense. You probably want awk 1 RS='\\|END\\|' or awk NF RS='\\|END\\|' or awk length RS='\\|END\\|' here. Note that a regexp RS requires gawk or mawk – Stéphane Chazelas Apr 02 '15 at 21:49
3

Another way with sed that doesn't print empty lines:

sed 's/|END|/\
/g;/^$/!P;D' infile

e.g. input:

T|one|two|END|T|three|four|END|
T|five|six|END|T|seven|eight|END|
T|nine|ten|END|T|eleven|twelve|END|

output:

T|one|two
T|three|four
T|five|six
T|seven|eight
T|nine|ten
T|eleven|twelve

same thing with ed:

ed -s infile <<'IN'
1,$j
s/|END|/\
/g
,p
q
IN
don_crissti
  • 82,805
2

As mentioned here by Walter Mundt, we can achieve this using an ANSI C quoted string

sed $'s/|END|/\\\n/g'

~ $ echo 'T|somthing|something|END|T|something2|something2|END|' | sed 
$'s/|END|/\\\n/g'
T|somthing|something
T|something2|something2

~ $

Check out the above here link for other alternatives.

You can use the following syntax as well, I'm not sure if it works on all flavours of Unix/Linux

sed 's/|END|/\'$'\n''/g'

~ $ echo 'T|somthing|something|END|T|something2|something2|END|' | sed 
's/|END|/\'$'\n''/g'
T|somthing|something
T|something2|something2

~ $
k_vishwanath
  • 198
  • 1
  • 7
1

I had a same issue in strict posix shell I did it in two pass with a unused char

cat data.json|tr '§' '?'|sed -e 's/"[^"]":/§&/g'|tr '§' '\n'
Benamar
  • 111