10

Is it possible to use gawk's -i inplace option and also print things to stdout?

For example, if I wanted to update a file, and if there are any changes print the name of the file and the changed lines to stderr I could do something like

find -type f -name 'myfiles' -exec gawk -i inplace '{if(gsub(/pat/, "repl")) { print FILENAME > "/proc/self/fd/2" ; print > "/proc/self/fd/2"; } print;}' {} +

but is there a way to use stdout instead, or a cleaner way to print that block to the alternate stream?

Eric Renouf
  • 18,431
  • 2
    In this particular case you might want to run something like find -type f -name 'myfiles' -exec grep -q 'pattern' {} \; -print -exec gawk -i inplace '{do_your_sub_and_print_to_dev/stderr_too}' {} \; that way you will only use awk to edit the files that actually contain lines matching that pattern. – don_crissti Nov 08 '16 at 01:14
  • @don_crissti I often forget how much quicker grep can be. My first instinct is to avoid having to "process" the file twice, but I wouldn't be at all surprised if it is indeed faster nonetheless. It just goes to show why I can trust my gut on profiling tasks – Eric Renouf Nov 08 '16 at 01:19
  • 1
    Yes, it's quite fast and keep in mind that: 1) if there's no match, you only process the file once (the part with -print -exec gawk is not executed anymore) and 2) it will stop at 1st match so unless the 1st match is on the last line you're still not processing the whole file twice (it's more like 1.X times) . Also, if gawk -i inplace works like sed -i it will edit the file in-place anyway - i.e. updating timestamps, inode etc even if there was nothing to edit... – don_crissti Nov 08 '16 at 01:27
  • 1
    Beware using gawk -i inplace like that introduces a security vulnerability unless you modify $AWKPATH not to include . or make sure you run that command from within a working directory where nobody could create a file called inplace or inplace.awk. – Stéphane Chazelas Jun 24 '23 at 21:14

2 Answers2

13

You should use /dev/stderr or /dev/fd/2 instead of /proc/self/fd/2. gawk handles /dev/fd/x and /dev/stderr by itself (regardless of whether the system has those files or not).

When you do a:

print "x" > "/dev/fd/2"

gawk does a write(2, "x\n"), while when you do:

print "x" > "/proc/self/fd/2"

since it doesn't treat /proc/self/fd/x specially, it does a:

fd = open("/proc/self/fd/2", O_WRONLY|O_CREAT|O_TRUNC);
write(fd, "x\n");

First /proc/self/fd is Linux specific and on Linux they are problematic. The two versions above are not equivalent when stderr is to a regular or other seekable file or to a socket (for which the latter would fail) (not to mention that it wastes a file descriptor).

That being said, if you need to write to the original stdout, you need to save it away in another fd like:

gawk -i /usr/share/awk/inplace.awk '{
   print "goes into the-file"
   print "to stdout" > "/dev/fd/3"}' the-file 3>&1

gawk does redirect stdout with in-place to the file. It's needed because for instance, you'd want:

awk -i /usr/share/awk/inplace.awk '{system("uname")}' file

to store the uname output into the file.

In any case, do not use -i inplace as gawk tries to load the inplace extension (as inplace or inplace.awk) from the current working directory first, where someone could have planted malware. The path of the inplace extension supplied with gawk may vary with the system, see the output of gawk 'BEGIN{print ENVIRON["AWKPATH"]}'

Stéphane Chazelas
  • 544,893
2

Just for fun, an alternate approach using only POSIX features of find, sh, grep and (most notably) ex:

find . -type f -name 'myfiles' -exec sh -c '
  for f do grep -q "pat" "$f" &&
    { printf "%s\n" "$f";
      printf "%s\n" "%s/pat/repl/g" x | ex "$f";
  }; done' find-sh {} +

(All line breaks in the above command are optional; it can be condensed to a one-liner.)

Or, arguably more legibly:

find . -type f -name 'myfiles' -exec sh -c '
  for f
  do
    if grep -q "pat" "$f"; then
      printf "%s\n" "$f"
      printf "%s\n" "%s/pat/repl/g" x | ex "$f"
    fi
  done' find-sh {} +

:)

(This command is untested; edits welcome if I made any goofs.)

Wildcard
  • 36,499