9

I'm trying to take the contents of a file and insert it after a matching pattern in another file using sed. My question is very similar to this question, but I wish to insert the contents of a file inline rather than on a new line. How can I do this?

Using the example question I referenced, the first answer does exactly what I want; however, I want the insertion to happen inline:

sed '/First/r file1.txt' infile.txt 

The actual data I want to insert is a JSON file:

[
    {
        "foo": "bar", 
        "baz": "biff",
        "data": [
            {
                "a": 1945619, 
                "b": [
                    {
                        "c": 512665, 
                        "d": "futz"
                    }
                ]
            }
        ]
    }
]
jimmij
  • 47,140
turtle
  • 2,697
  • 5
  • 20
  • 17
  • I think this answers your question. Would you look at it and let me know? – mikeserv Dec 09 '14 at 21:43
  • @mikeserv Thanks for the help. I'd happy to try your solution, but after reading your solution, I'm not sure how to implement what you suggest. Can you post an example? – turtle Dec 09 '14 at 22:04
  • I'm definitely considering it - it looks simple enough - but I dunno where to start... OOps. the pattern is First - duh. sorry. – mikeserv Dec 09 '14 at 22:05
  • Wait - you want to insert the JSON newline free - like remove the newlines from the inserted file as it is inserted? I can do that with GNU sed but I just wanna make sure... – mikeserv Dec 09 '14 at 22:07
  • The actual JSON does not need to be altered, I want to insert the JSON after the pattern so that there is not a newline between the pattern and the start of the JSON. – turtle Dec 09 '14 at 22:08

3 Answers3

7

In your linked question there is already good awk answer, just modify it a little bit by using printf instead of print to insert the content without newline:

awk '/First/ { printf $0; getline < "File1.txt" }1' infile.txt

Result:

Some Text here
FirstThis is text to be inserted into the File.
Second
Some Text here

You may want to add space or other delimeter after "First" with printf $0 " "; ...


If inserted file has many lines then:

awk '/First/{printf $0; while(getline line<"File1.txt"){print line};next}1' infile.txt

Result:

Some Text here
First[
    {
        "foo": "bar", 
        "baz": "biff",
        "data": [
            {
                "a": 1945619, 
                "b": [
                    {
                        "c": 512665, 
                        "d": "futz"
                    }
                ]
            }
        ]
    }
]
Second
Some Text here
jimmij
  • 47,140
  • In my actual use case, I'm inserting a large JSON file and this awk example doesn't work. I just get a single left brace, [ inserted after the pattern match. Normal text works fine, but not JSON? – turtle Dec 09 '14 at 21:53
  • Added an example JSON data file that does not insert correctly for me. – turtle Dec 09 '14 at 22:01
  • @turtle that maybe because getline reads in one line. Use a loop around getline: http://unix.stackexchange.com/a/32911/70524 – muru Dec 09 '14 at 22:04
  • Ok, that worked, but the JSON is getting inserted right BEFORE the pattern now: awk '/First/ { while(getline line<"File1.txt"){print line} }1' infile.txt – turtle Dec 09 '14 at 22:11
  • @turtle try awk '/First/ { print; while(getline line<"File1.txt"){print line} }' infile.txt instead. – muru Dec 09 '14 at 22:15
  • @turtle see the update. – jimmij Dec 09 '14 at 22:25
  • Ah, didn't notice it. +1. – muru Dec 09 '14 at 22:28
  • This doesn't seem to append after PATTERN if PATTERN is in the middle of a string, it appends at the end of line, irrespective of PATTERN position. – don_crissti Dec 10 '14 at 09:45
7

You could use perl (get the file content and substitute pattern with pattern+file content):

perl -pe '$text=`cat insert.txt`; chomp($text); s/PAT/$&$text/' file.txt

add -i to edit in place; g to append after each PAT (pattern) occurrence, e.g.:

perl -i -pe '$text=`cat insert.txt`; chomp($text); s/PAT/$&$text/g' file.txt

Another way, using ed:

printf '%s\n' /PAT/s/PAT/\&\\ \/ - kb ". r insert.txt" j \'b j ,p q | ed -s file.txt

to edit in-place, replace ,p with w:

 printf '%s\n' /PAT/s/PAT/\&\\ \/ - kb ". r insert.txt" j \'b j w q | ed -s file.txt

Probably no one is interested how this works but anyway, printf passes a list of commands to ed:

/PAT/s/PAT/&\             #   set address to first line matching PAT and
/                         #   split the line right after PAT
-                         #   set address one line before (return to the line matching PAT)
kb                        #   mark the current line
. r insert.txt            #   insert content of insert.txt after this line         
j                         #   join current line and the next
'b                        #   set  address to marked line (return to the line matching PAT)
j                         #   join current line and the next one
,p                        #   print file content
q                         #   quit editor

Or, without using printf and |:

ed -s file.txt <<< $'/PAT/s/PAT/&\\\n/\n-\nkb\n. r insert.txt\nj\n\'b\nj\nw\nq\n'
don_crissti
  • 82,805
  • I once spent a solid 8 hours shoulders deep in ex docs - I really like the vi console window thing you can do. A single line or an adjustable size or...well, lots of stuff probably, that I just never got the hang of. Anyway, I really do wanna figure it out though - I was just poring over the POSIX docs. Do you have any recommendations? – mikeserv Dec 10 '14 at 18:54
  • Not many resources (that I know of, alas) covering ed... the POSIX docs, there's also this page... I recommend using the tool for simple tasks/experimenting (at least, that's how I've learned, reading only the docs, Kernighan's short intro and Krummins' cheat sheet). With your knowledge of sed, you should have no problems. Unless your question is about ex (I don't use ex much, if at all)... – don_crissti Dec 10 '14 at 20:18
  • it is about ex - which, as I understand it, is just an ed extension? one thing that threw me when I played with it was the tmp storage - I tried loading up a big file and had to wait for it. on the other hand, what I find attractive about that is the innumerable buffers - I just wasn't good enough to split them out efficiently. thanks for the links. – mikeserv Dec 10 '14 at 21:29
6

So it would be a little tricky to make this work portably in sed - you should be looking to cut and/or paste with some regex precursor generating their script in that context - and this is because sed will always insert a \newline before the output of a read. Still, w/ GNU sed:

sed '/First/{x;s/.*/cat file/e;H;x;s/\n//}' <<\IN 
First
Second
Third
IN

That works by executing cat every time it encounters your /First/ address. It does this in the hold space (kind of - an alternate buffer anyway - because I exchange them it actually happens in pattern space which used to be hold space) so as to preserve the contents of the line matching First and then appends cat's output to your line and removes the intervening \newline.

OUTPUT:

First[
    {
        "foo": "bar", 
        "baz": "biff",
        "data": [
            {
                "a": 1945619, 
                "b": [
                    {
                        "c": 512665, 
                        "d": "futz"
                    }
                ]
            }
        ]
    }
]
Second
Third

Now if you want the entire contents of the file to fit between two portions of a line that has to work a little differently because with the above command I just remove the trailing newline between the end of the matching line and the beginning of the file. Still, you can do this, too:

sed '/First/{s//&\n/;h
         s/.*/{ cat file; echo .; }/e;G
         s/\(.*\).\n\(.*\)\n/\2\1/
}' <<\IN
Third
Second
First Second Third
Third
Second
First Second Third
IN

That splits the line at the match with a \newline character, saves it in hold space, executes cat - which replaces pattern space with its output - Gets the contents of the hold space appended to our new pattern space following another \newline character, and then rearranges on \newline delimiters.

I do echo . to preserve any trailing \newline characters in file - but if that is not your wish (and isn't very relevant to your example anyway) you can do without it and remove the first . before .\n in the following s///ubstitution.

Just before the rearrange pattern space looks like this:

^cat's output - any number of newlines.*.\nmatch on First\nrest of match$

OUTPUT:

Third
Second
First[
    {
        "foo": "bar", 
        "baz": "biff",
        "data": [
            {
                "a": 1945619, 
                "b": [
                    {
                        "c": 512665, 
                        "d": "futz"
                    }
                ]
            }
        ]
    }
] Second Third 
Third
Second
First[
    {
        "foo": "bar", 
        "baz": "biff",
        "data": [
            {
                "a": 1945619, 
                "b": [
                    {
                        "c": 512665, 
                        "d": "futz"
                    }
                ]
            }
        ]
    }
] Second Third
mikeserv
  • 58,310
  • 1
    That's a great answer Mike. – don_crissti Dec 10 '14 at 09:47
  • @don_crissti - thanks very much. The s///e is syntax is pretty cool - you can do anything you with it that you might do at a prompt. In fact, it makes for a pretty handy prompt processor as well. I've experimented with it and some more basic shells which don't provide much in the way of line editing and come away impressed. If you use it though, remember to take care that you remove all of a pattern space when you do - replacing only a portion of arbitrary data and then executing the results can be disastrous. I know - I've been there. – mikeserv Dec 10 '14 at 11:23