How to replace all matched strings in the files recursively?

Question

There are about 50 HTML/js files in the folder name site,

some of the files contain (below lines are combined from files)

    {"rendered":"http:\/\/localhost:4542\/?page_id=854"}
                 http:\/\/localhost:4542\/wp-content\/uploads\/2022\/09\/
           src=\"http:\/\/localhost:4542\/wp-content\/uploads\/2022\/09\/B
                 http:\/\/localhost:4542\/wp-content\/uploads\/2022\/09\/A
replies":[{"embeddable":true,"href":"http:\/\/localhost:4542\/en\/wp-json

Any tool/ commands to replace http:\/\/localhost:4542 to https:example.com recursively in all files of a folder?

Working on a macOS now.

Macos has no some gui editor with find/replace in files? – gapsf Sep 10 '22 at 17:41 — gapsf, Sep 10 '22 at 17:41

Stéphane Chazelas · Answer 1 · 2022-09-10T17:46:33.073

2

Portably:

LC_ALL=C find . '(' -name '*.[Jj][Ss]' -o \
                    -name '*.[Hh][Tt][Mm][Ll]' -o \
                    -name '*.[Hh][Tt][Mm]' \
                ')' -exec perl -pi -e '
  s{\Qhttp:\/\/localhost:4542\E}{https:example.com}g' {} +

Note that it ends up rewriting even the files that have nothing to replace.

With GNU grep/xargs or compatible (and here a shell with brace expansion support to avoid too much typing), that can be avoided with:

LC_ALL=C grep -r --include='*.'{'[jJ][sS]','[Hh][Tt][Mm]'{,'[Ll]'}} \
  -Fl --null 'http:\/\/localhost:4542' . |
  xargs -r0 perl -pi -e '
    s{\Qhttp:\/\/localhost:4542\E}{https:example.com}g'

Except maybe for the -r option to xargs which you can remove without too much harm (just a perl warning if grep finds not file), it should also work on macos whose grep has copied most of the GNU API.

edited Sep 10 '22 at 17:46

answered Sep 10 '22 at 13:58

Stéphane Chazelas

544,893

I really liked this answer. I had a doubt about LC_ALL and just found your answer in another question. – Edgar Magallon Sep 10 '22 at 17:41
I spent 30+ years working on a system (which still exists, I just don't work there any longer) that only has the mandatory POSIX tools and we weren't allowed to install any others. We also occasionally see question on SO (I don't recall seeing any here) where people also say that's all they can have so a solution involving perl isn't entirely portable across Unix systems (which I assume is what you meant by "Portably"). – Ed Morton Sep 11 '22 at 13:18

score 0 · Answer 2 · answered Sep 10 '22 at 01:56

0

You can use find to locate all files of certain type (e.g. *.txt) in the given folder (e.g. .), and then feed them into a sed command using xargs:

find .  -type f -name '*.txt' -print0 | xargs -0 sed -i 's/http:\\\/\\\/localhost:4542/https:example.com/g'

Note the escaping of the slashes in above.

answered Sep 10 '22 at 01:56

tinlyx

678

if our files are not type specified, we can just use -name '*' ? – Maxfield Sep 10 '22 at 02:13
@user95432 yes, you can use -name '*' if type is not specified. – tinlyx Sep 10 '22 at 11:56
Rather than -name '*', just don't use -name at all in that case – Ed Morton Sep 11 '22 at 13:21
Rather than having to escape all your /s, just don't use / as the sed delimiter, e.g. use # as in s#http://local#foo# instead of s/http:\/\/local/foo/ – Ed Morton Sep 11 '22 at 13:23

Edgar Magallon · Accepted Answer · 2022-09-10T03:22:15.400

0

You can try using the command find with -exec argument:

find /path/to/folder -type f -regex '.*\.\(js\|html\)' -exec sed -i 's#http:\\/\\/localhost:4542#https:example.com#g' {} +

On Mac Os you can use a similar syntax:

find /path/to/folder -E -type f -regex '.*\.(js|html)' -exec sed -i '.bak' 's#http:\\/\\/localhost:4542#https:example.com#g' {} +

I really hope that works, I don't have a Mac to test it, but it should work. On Mac Os when you specify the -i option the backup file name extension is required. You can use any value instead of '.bak'

Credits:

How to use find command to search for multiple extensions,

sed command with -i option failing on Mac

Note: I'm not sure if this code will work in Mac Os, but you can try it too:

find /path/to/folder -iname "*.html" -o -iname "*.js" -exec sed -i 's#http:\\/\\/localhost:4542#https:example.com#g' {} +

edited Sep 10 '22 at 03:22

answered Sep 10 '22 at 02:13

Edgar Magallon

4,820

find: illegal option -- t usage: find [-H | -L | -P] [-EXdsx] [-f path] path ... [expression] find [-H | -L | -P] [-EXdsx] -f path [path ...] [expression] – Maxfield Sep 10 '22 at 02:34
@user95432 Ok! I just saw you are on Mac OS. I will edit the post to use find in this os. – Edgar Magallon Sep 10 '22 at 02:37
I don't find it works since files are unchanged. – Maxfield Sep 10 '22 at 03:07
@user95432 I searched and I saw that in Mac Os the sed command works differently in some ways. – Edgar Magallon Sep 10 '22 at 03:17
@user95432 I edited again, try to test it. The problem was with sed -i – Edgar Magallon Sep 10 '22 at 03:32
1

In place editing without backup with the sed of FreeBSD/macos is with sed -i '' 'sed-code' file – Stéphane Chazelas Sep 10 '22 at 14:00
@StéphaneChazelas thanks for the tip! I guess these basic commands follow the same syntax for both operating systems. I also guess these problems are part of the POSIX compliant. – Edgar Magallon Sep 10 '22 at 17:28
1

@EdgarMagallon, POSIX sed doesn't support -i. Both GNU and FreeBSD added a -i to their respective sed inspired by perl's independendly in the early 2000s, unfortunately with incompatible interface, so there's little hope it will be standardised by POSIX soon. The GNU API (which most other implementations have copied including some other BSDs) is also incompatible with the POSIX option parsing which doesn't support options with optional arguments like GNU's -i / -i.back does. – Stéphane Chazelas Sep 10 '22 at 17:35
@StéphaneChazelas thanks again for the answer. All this information is useful, I don't know much about POSIX but I really have to learn more about it. – Edgar Magallon Sep 10 '22 at 17:50

How to replace all matched strings in the files recursively?

3 Answers3