2

I have many txt files in a folder. Each file contains one line in this format;

8852 0.53451 0.55959 0.65494 0.36047

I would like to add, in the next line, a fixed number and the rest 4 columns copying from the original line, like so

8852 0.53451 0.55959 0.65494 0.36047

9997 0.53451 0.55959 0.65494 0.36047

So far I managed to add 9997 by the below command inside the folder

find *.txt -exec sh -c 'echo "9997" >> $0' {} \;

and managed to copy the rest 4 columns by

for files in .; do awk '{$1=""; print substr($0,2)}' *.txt;done

but it prints out in the output and not in the next line in each file.

I haven't managed to put them together yet and felt like I come so close but maybe in a wrong direction. I tried awk '{print "\n"$2,$3,$4,$5}'but it also didn't work.

Any help is appreciated! Thank you.

Pimpwhippa
  • 23
  • 4
  • Are you trying to update the files in one directory or recursively update files in this directory and all directories under it? – Ed Morton Jul 03 '19 at 22:27
  • @Ed in this directory and other directories next to it. sorry for late reply. thank you for your suggestions. – Pimpwhippa Jul 04 '19 at 11:01

3 Answers3

3

I'd suggest awk '{print; $1 = 9997; print}'

$ echo '8852 0.53451 0.55959 0.65494 0.36047' | awk '{print; $1 = 9997; print}'
8852 0.53451 0.55959 0.65494 0.36047
9997 0.53451 0.55959 0.65494 0.36047

You mention applying the transformation to multiple files: you can do so either using a loop

for f in *.txt; do  awk '{print; $1 = 9997; print}' "$f"; done

or (recursively) using find

find . -name '*.txt' -exec awk '{print; $1 = 9997; print}' {} \;

however unless you have a recent version of GNU Awk with the -i inplace extension, the results will be concatenated and written to standard output.

steeldriver
  • 81,074
  • yes I tried the one above, and the results are written to standard output. thank you for your proper answer. – Pimpwhippa Jul 04 '19 at 11:10
0

Given your comment that you want to update files in this directory and other directories next to it., lets assume you have a directory under which there are other directories under which there are the files you want to modify. This is how you'd do that after cd-ing to that top level directory:

find . -maxdepth 2 -mindepth 2 -exec awk -i inplace '{print} $1==8852{$1=9997; print}' {} +

The above uses GNU awk for "inplace" editing and GNU find for +. If you don't have GNU tools then, assuming your file names don't contain newlines:

tmp=$(mktemp)
while IFS= read -r file; do
    awk '{print} $1==8852{$1=9997; print}' "$file" > "$tmp" &&
    mv "$tmp" "$file"
done < <(find . -maxdepth 2 -mindepth 2) )
rm -f "$tmp"

If you don't want to change the timestamp on files that don't containing 8852 then:

tmp=$(mktemp)
while IFS= read -r file; do
    awk '$1==8852{f=1; exit} END{exit !f}' "$file" &&
    awk '{print} $1==8852{$1=9997; print}' "$file" > "$tmp" &&
    mv "$tmp" "$file"
done < <(find . -maxdepth 2 -mindepth 2) )
rm -f "$tmp"
Ed Morton
  • 31,617
  • 1
    thanks for the proper way to make it works. not a one liner finally! anyway, to update all other directories from the top level directory, I have to modify it further, as not all the directories has 8852 to start with. I'll try to figure that out. Thank you – Pimpwhippa Jul 05 '19 at 12:52
  • I meant other directories have other numbers that's not 8852, but I also want to change them :) I think I really have to write a script for that e.g. directories with files that contain 26053 0.xxxxx 0.xxxxx 0.xxxxx 0.xxxxx also needs to add in the next line 9997 0.xxxxx 0.xxxxx 0.xxxxx 0.xxxxx – Pimpwhippa Jul 06 '19 at 10:59
  • OK, feel free to ask a new question with more accurate/representative sample input/output if you have a question about how to do that. – Ed Morton Jul 06 '19 at 11:02
  • 1
    yes, will figure out the overview of what I want and will see if I can do it myself first. thanks again. – Pimpwhippa Jul 06 '19 at 11:06
-1

So you have many files and they have one line in the below format.

8852 0.53451 0.55959 0.65494 0.36047

Now you want to open the file and replace the first column with a value of your choice and leave the rest as it is, then append it to the next line.

8852 0.53451 0.55959 0.65494 0.36047

9997 0.53451 0.55959 0.65494 0.36047

If I get this question right, the below should work.

for file in ``find . -name "*.txt"``; do awk '{print "\n9997 "$2,$3,$4,$5}' $file >> $file; done

Get all the files in a folder, with awk read the file and take 2345 columns and for the first column add your value and append it to the same file.

Apologies if I got your question wrong.

Vignesh SP
  • 307
  • 2
    Not only is this a bad idea due to parsing the output of ls but it's also going to give a bunch of errors iif there are any directories inside of the folder and if there are any files (or directories) with spaces in them. It would also perform the operateion on every file in that directory whether it's needed or not. You also don't need the double quotes in between the column parameters for awk as a space as a field separator can be denoted with a comma. – Nasir Riley Jul 03 '19 at 21:08
  • I tried. it didn't work. maybe something to do with ls -1 as @Nasir mentioned? but yes you understand what I want correctly – Pimpwhippa Jul 03 '19 at 21:10
  • @NasirRiley, thanks for the comment. I'm a newb myself. I will edit the answer. – Vignesh SP Jul 03 '19 at 21:13
  • @Pimpwhippa, what is the output when you execute this? – Vignesh SP Jul 03 '19 at 21:14
  • It just returns, gives no output. then I go nano into a few random files. nothing happens there either. I just copied pasted what you typed. – Pimpwhippa Jul 03 '19 at 21:17
  • Take out the extra apostrophes before and after ls -1, like this - for file in ls -1 *txt*; do awk '{print "\n9997 "$2,$3,$4,$5}' $file >> $file; done – Vignesh SP Jul 03 '19 at 21:18
  • @Vignesh SP yes, now it works. Thank you! nonetheless, if it is not recommended to parse the output of ls, what else could be done instead? just would like to know for educational purpose.. – Pimpwhippa Jul 03 '19 at 21:26
  • Well, Riley was mentioning that ls will provide everything from the directory as output which includes directory and all. We can fixe this by what exactly we want. Now all you want is files with .txt extension. So we can ask ls to provide files only with .txt extension by adding ls -1 *txt*. Hopefull this solves the downside of using ls. Other than this, I do not see any downsides in using ls. – Vignesh SP Jul 03 '19 at 21:29
  • 1
    @VigneshSP The downsides are that your command would break if any of the files contain spaces and/or if there are any files that contain the string txt that you don't want to operate on. A Google search will give plenty of examples of why parsing ls isn't a good idea starting with this one: https://unix.stackexchange.com/questions/128985/why-not-parse-ls-and-what-do-to-instead – Nasir Riley Jul 03 '19 at 21:31
  • @NasirRiley I guess piping ls to sed may solve the space issue? like this? ls -1 | sed 's/\ /\\\ /g' – Vignesh SP Jul 03 '19 at 21:36
  • @VigneshSP No, because that's still parsing ls which is what you want to avoid. – Nasir Riley Jul 03 '19 at 21:37
  • @VigneshSP combining the awk print to the find command? – Pimpwhippa Jul 03 '19 at 21:41
  • @NasirRiley How about find . -name "*.txt" than ls? Would that work? – Vignesh SP Jul 03 '19 at 21:43
  • @Pimpwhippa, didn't understand what you meant. – Vignesh SP Jul 03 '19 at 21:52
  • @VigneshSP something like find *.txt -exec sh -c 'echo "\n"9997 $(awk '{print $2,$3,$4,$5}')' >> $0' {} ; – Pimpwhippa Jul 03 '19 at 21:56
  • @Pimpwhippa for file infind . -name "*.txt"; do awk '{print "\n9997 "$2,$3,$4,$5}' $file >> $file;done this works too. We can find so many other ways to do this if we sit and do a research on this :p – Vignesh SP Jul 03 '19 at 22:00
  • @VigneshSP hahaha thanks for researching this with me! now we can move on to the next task! ;D – Pimpwhippa Jul 03 '19 at 22:06
  • 1
    Do not execute this code. So many bugs/anti-patterns in so little code - see https://mywiki.wooledge.org/Quotes, http://mywiki.wooledge.org/BashFAQ/001, http://pubs.opengroup.org/onlinepubs/9699919799, /xrat/V4_xcu_chap02.html#tag_23_02_06_03, https://unix.stackexchange.com/q/10241/133219, and I'm sure there's more... – Ed Morton Jul 03 '19 at 22:26
  • 1
    oh thank you @Ed Morton – Pimpwhippa Jul 03 '19 at 22:33