How to do a text replacement in a big folder hierarchy?

Question

I want to search and replace some text in a large set of files excluding some instances. For each line, I want a prompt asking me if I need to replace that line or not. Something similar to vim's :%s/from/to/gc (with the c to prompt for confirmation), but across a set of folders. Is there some good command line tool or script that can be used?

Answer 1

Why not use vim?

Open all files in vim

vim $(find . -type f)

Or open only relevant files (as suggested by Caleb)

vim $(grep 'from' . -Rl)

And do then run the replace in all buffers

:bufdo %s/from/to/gc | update

You can also do it with sed, but my sed knowledge is limited.

Answer 2

You can do something crude with a small Perl script which is instructed to perform replacements line by line (-l -pe) in place on the files passed as arguments (-i):

perl -i -l -pe '
    if (/from/) {                            # is the source text present on this line?
        printf STDERR ("%s: %s [y/N]? ", $ARGV, $_);  # display a prompt
        $r=<STDIN>;                                   # read user response
        if ($r =~ /^[Yy]/) {                          # if user entered Y:
            s/from/to/g;                              # replace all occurences on this line
    }' /path/to/files

Possible improvements would be to color parts of the prompt and support things like “replace all occurences in the current file”. Separately prompting for each occurrence on a line would be harder.

Second part, matching the files. if there aren't too many files involved and you're running zsh, you can match all the files in the current directory and its subdirectories recursively:

perl -i -l -pe '…' **/*(.)

If your shell is bash ≥4, you can run perl … **/*, but that will produce spurious error messages because sed will try (and fail) to run on directories. If you only want to perform the replacement in a set of files such as C files, you can restrict the matches (that works in either bash ≥4 or zsh):

perl -i -l -pe '…' **/*.[hc]

If you need finer control over which files you're replacing, or your shell doesn't have the recursive directory matching construct **, or if you have too many files and get a “command line too long” error, use find. For example, to perform a replacement in all files named *.h or *.c in the current directory and its subdirectories (on older systems, you may need to use \; instead of + at the end of the line (the + form is faster but not available everywhere).

find . -type f -name '*.[hc]' -exec perl -i -l -pe '…' {} +

---

That being said, I'd stick to an interactive editor if you need interaction. Gert has shown a way to to this in Vim, though it requires opening all the files that you want to search through, which may be a problem if there are a lot.

In Emacs, here's how you can do this:

1. Gather the file names with M-x find-name-dired (specify a toplevel directory) or M-x find-dired (specify an arbitrary find command line).
2. In the resulting dired buffer, press t to mark all files, then Q (dired-do-query-replace-regexp) to perform a replacement with prompting on the marked files.

Answer 3

sdiff (see http://www.gnu.org/software/diffutils/manual/diffutils.html#Invoking-sdiff) might come in handy in here. With it you can do interactive patching. So doing it with a temporary file you created by doing replacement operations using sed might be a possible solution:

# use file descriptor 3 to still allow use of stdin
while IFS= read -r -d '' file <&3; do

  # write the result of the replacement into a temporary file
  sed -r 's/something/something_else/g' -- "$file" > replacer_tmp

  if cmp -s -- "$file" replacer_tmp; then
    continue; # nothing was replaced
  fi

  echo "There is something to replace in '$file'! Starting interactive diff."
  echo

  sdiff -o "$file" -s -d -- "$file" replacer_tmp

  echo

done 3< <(find . -type f -print0)

(File loop using non-POSIX process substitution and read -d as supported e.g. by bash.)