2

I have a text file that looks like this -

1 29 12
1 22 17
1 20 43
1 48 30
1 3 0
1 21 25
1 18 9
1 14 1
1 44 45
1 52 26
1 NaN 27
1 13 46
1 5 37
1 34 23
1 39 40

whose columns I want to store into different array variables. I have used the following command -

#!/bin/csh
set Logfile=$proj_path/data/individual/AAR"$subj"/activation/trialWise/"AAR"$subj"_PseudoOrder.txt"

set acc_full= (awk -F' ' '{print $1}' "$Logfile")


set stim1= (awk -F' ' '{print $2}' "$Logfile")


set stim2= (awk -F' ' '{print $3}' "$Logfile")

The problem is that the 3rd awk command only stores the last number in the 3rd column.

αғsнιη
  • 41,407
Sanjana Hari
  • 23

2 Answers2

2

Your file has windows line endings. You must have edited on a Windows machine at some point. This means there are trailing \r characters which are part of your csh variables, so that when you print the variable's value, the \r cause the previous values to be overwritten:

$ printf 'a\rb\rc\re\r\n'
e
$ printf 'a\rb\rc\re\r\n' | od -c
0000000   a  \r   b  \r   c  \r   e  \r  \n
0000011

You can use one of these commands to fix your file and after that your script will work (although you should really think about whether you want to use csh as a scripting language: it is a very, very bad choice for scripting):

dos2unix foo_PseudoOrder.txt

or (assuming GNU sed or compatible)

sed -i 's/\r$//' foo_PseudoOrder.txt

or

tr -d '\r' < foo_PseudoOrder.txt > tmpFile && mv tmpFile foo_PseudoOrder.txt

Note that the tr solution above will delete all \r from the file, even if for some reason you have one that isn't at the end of the line and want to keep it.

terdon
  • 242,166
  • Hello @terdon, Your third solution did the trick nicely.

    About changing from csh to another language, I am working on a large set of Pre written codes, already in csh, so changing it now may need some rewriting on my Part, which I can't tackle at the moment.

     – Sanjana Hari Feb 02 '23 at 10:42
  • @Stéphane, what GNUism did I use there, is it the \r that isn't understood by all seds? – terdon Feb 02 '23 at 11:02
  • 1
    @SanjanaHari yeah, that makes sense. My condolences then :) – terdon Feb 02 '23 at 11:02
  • 1
    @terdon, both -i and \r. On FreeBSD and derivatives, it's -i '' for in-place editing without backup. Might as well use perl: perl -pi -e 's/\r$//' file – Stéphane Chazelas Feb 02 '23 at 11:25
2

You have DOS line endings, see why-does-my-tool-output-overwrite-itself-and-how-do-i-fix-it.

The -F ' 's in your script are setting FS to the value it already has by default so don't do that as it's doing nothing useful.

If you have GNU awk (for multi-char RS) change this:

awk '{print $3}' "$Logfile"

to this:

awk -v RS='\r\n' '{print $3}' "$Logfile"

or with any awk:

awk '{sub(/\r$/,""); print $3}' "$Logfile"
Ed Morton
  • 31,617