sed to redirect data to a file

Question

I am trying to replace >>> with "" in file

cat table_files.md | egrep ">>>$"                               
  \>>>
  \>>>
\>>>
\>>>
\>>>
\>>>
\>>>
\>>>
\>>>

replace it with sed

cat table_files.md | sed "s\>>>$\\g" > table_files.md

Unfortunately, all my data gone, it feedback to me a blank file. I check this should be done using -i option in place.

How could that happen, is it possible to recover the data?

Related: redirect to the same file as the source file processed by the command — steeldriver, Nov 03 '18 at 02:05
First, you don't need cat. You can use sed on the file without -i which will send the changes to standard output without changing the file itself and then add -i once it does what you want. Instead, you redirected the output of nothing back into the file itself so, no You can't recover it unless you have a backup. — Nasir Riley, Nov 03 '18 at 02:22
Sorry, but no, it is not possible to recover the data. Next time, be safe: work on a copy. — , Nov 03 '18 at 03:27

score 2 · Accepted Answer · answered Nov 03 '18 at 02:40

THE REASON the file was overwritten/truncated was that it's your shell that interprets redirects, and it does this interpretation before any commands get run. A big part of the shell's job is to handle the plumbing to connect stdin and stdout between various things. You issued a command line like:

$ command1 < somefile | command2 > somefile

Even before command2 was run, the shell had interpreted the redirect that would direct the output from that command to a file, truncating the file. And with the file truncated, there was no input for command1 to read, and so it quit, closing the pipe to command2, which also quit, without having issued any data to be redirected to the file.

It seems like you have two questions here...

1. Can I recover the file I overwrote?

Probably not. If you can, it'll be done by restoring a backup. Overwriting a file with other data generally eliminates the old version of the file.

2. What's the best way to replace data in a file?

As you surmised, the best way to do this with sed would be to use the -i (inplace) option. Bear in mind that this isn't really an in-place edit. The -i option causes sed to read all its input, modify it and stow it in a temporary storage, and upon completion, move everything from the temporary storage back to the original file.

In BSD sed (including macOS):

sed -i '' 's:>>>$::' $filename

But in GNU sed (most Linux distros), an unspecified optarg to -i is considered to be null:

sed -i -e 's:>>>$::' $filename

This removes the >>> at the ends of lines. If you want to remove the string from within lines, modify the regex so that it isn't anchored to the end of the line ($), and add the g flag to have it apply to multiple strings per line:

sed -i '' 's:>>>::g' $filename

Note that different operating systems run sed in different ways. When you're writing tools that you may want to share, it's important to remember that not everyone runs the same operating system as you. Portability is important. To make things portable, you may have to dumb-down your scripts so that they use the "lowest common denominator" of functionality available across all the platforms you want to support. Thus:

tmpfile=$(mktemp /tmp/$$XXXX)
sed -e 's:>>>$::' $filename > $tmpfile &&
mv $tmpfile "$filename"

sed to redirect data to a file

1 Answers1