Replace new lines at end of lines starting with pattern

Question

I have a file on my Ubuntu machine where I've marked the start of some lines using '@':

@A
bbb
@B
bbb
@D
ccc

I want to remove the new lines at the end of lines starting '@' so the above file becomes

@Abbb
@Bbbb
@Dccc

I can match the lines starting @ using sed:

sed /^@/ ...

but I'm lost at trying to find the way to remove the new line character at the end of the string. There must be a way of doing this easily using sed without having to manually open the file in a text editor and remove them myself.

What operating system are you using? We need to know this so we can know what tools, and what implementations of tools, you have available. — terdon, Oct 04 '21 at 14:11

score 5 · Accepted Answer · answered Oct 04 '21 at 14:16

$ printf '%s\n' 'g/^@/j' ',p' 'Q' | ed -s file
@Abbb
@Bbbb
@Dccc

The above uses a short ed script to join each line in the file that matches the regular expression ^@ with the line after. It then prints all the lines in the editing buffer and quits without saving.

$ sed '/^@/ { N; s/\n//; }' file
@Abbb
@Bbbb
@Dccc

This sed command appends the next line to the input buffer if the current line matches ^@. The inserted newline character is removed and the line printed.

Both of these variations would miss a line starting with @ if it was immediately preceded by such a line. This means, they would turn

@1
@2
X
@3
Y

into

@1@2
X
@3Y

rather than

@1@2X
@3Y

It is unclear whether this is ok or not.

The lines are broken due to faulty copy-paste from a MacOS terminal window to a (remote) Ubuntu 18.04 window hence why I added the "@" so I know where the lines are supposed to start — user36196, Oct 04 '21 at 14:22

score 4 · Answer 2 · answered Oct 04 '21 at 14:17

4

Perl to the rescue!

perl -pe 'chomp if /^@/' -- file

-p reads the input line by line and prints each line after processing
/^@/ matches the @ at the beginning of a line
chomp removes the trailing newline

answered Oct 04 '21 at 14:17

choroba

47,233

schrodingerscatcuriosity · Answer 3 · 2021-10-04T14:28:03.033

3

Here's an option with GNU sed:

$ sed -E 'N;s/\n([^@])/\1/' foo.txt 
@Abbb
@Bbbb
@Dccc

Using the N flag:

N

appends line from the input file to the pattern space.

([^@]) matches the firts character after the newline that is not a @, capturing it and then adding it to the previous line.

If you need to modify the file you have to use the option -i: sed -i..., but make a backup first.

edited Oct 04 '21 at 14:28

answered Oct 04 '21 at 14:17

schrodingerscatcuriosity

12,396

score 3 · Answer 4 · answered Oct 04 '21 at 14:19

3

Non-sed

if you know that all the lines alternate @ and non-@, then
```
paste -d '' - - < file
```

awk

awk '{printf "%s%s", $0, (/^@/ ? "" : ORS)}' file

answered Oct 04 '21 at 14:19

glenn jackman

85,964

score 2 · Answer 5 · answered Oct 04 '21 at 14:19

2

With perl:

$ perl -pe 's/^(@.*)\n/\1/' file
@Abbb
@Bbbb
@Dccc

or

$ perl -pe 's/\n// if /^@/' file
@Abbb
@Bbbb
@Dccc

answered Oct 04 '21 at 14:19

terdon

242,166

αғsнιη · Answer 6 · 2021-10-04T15:21:40.377

1

awk '{ printf "%s", (/^@/?(NR>1?ORS:""):"")$0 } END{ print "" }' infile

Sample input:

iiii
@A
bbb
xyz
@B
bbb
@D
ccc

Output:

iiii
@Abbbxyz
@Bbbb
@Dccc

edited Oct 04 '21 at 15:21

answered Oct 04 '21 at 14:21

αғsнιη

41,407

Replace new lines at end of lines starting with pattern

6 Answers6