How to cut date and time in Unix Scripting

Question

I'd like to ask how to use cut or other similar command to get date and time... So basically, $line was retrieved from a for loop:

for line in $(cat $file);
do
   getdatetime=$(echo $line | cut -f4 -d,)
done

where the sample value for $line is:

883427446627317909,1114259,1573178423,2019-11-08 02:00:23,RD,4.7,0,351442429

The file $file contain multiple lines similar to the sample $line.

The expected value for getdatetime is:

2019-11-08 02:00:23

But i'm getting the date only:

2019-11-08

Is there any one-liner cut command to extract the date and time? I need to preserve the structure of the for loop. Thank you.

Answer 1

When you iterate over the unquoted command substitution $(cat $file), you iterate over all words that cat $file results in. A word will be anything delimited by a whitespace (space, tab or newline by default). This means that for a line in $file that is

883427446627317909,1114259,1573178423,2019-11-08 02:00:23,RD,4.7,0,351442429

you will have the two words 883427446627317909,1114259,1573178423,2019-11-08 and 02:00:23,RD,4.7,0,351442429 (i.e. the loop would iterate twice for this single line). This means that you'd get 2019-11-08 in the first iteration and 0 in the second from this line.

The solution is not to quote the command substitution as doing so would cause the loop to iterate once over the full contents of the file which is read into $line. The solution is not to set IFS to a newline, as that is inelegant (it would necessitate one call to cut in each iteration).

Instead, parse out the data that you want in a single call to cut and read that:

while IFS= read -r datetime; do
    # use "$datetime" here
done < <( cut -d, -f4 "$file" )

This uses a process substitution to create an input stream for the while loop to read from. The data in this stream will consist of the 4th comma-delimited field in the file named by $file.

Alternatively, with the while loop in a subshell:

cut -d, -f4 "$file" |
while IFS= read -r datetime; do
    # use "$datetime" here
done

With awk, the processing would be cleaner (unless you need to use the date/time value as a shell variable for whatever reason):

awk -F , '{ datetime = $4; ... more code here using the datetime variable }' "$file"

Related reading:

• Why is using a shell loop to process text considered bad practice?
• Understanding "IFS= read -r line"

Answer 2

We can actually do without cut if we want to:

while IFS=, read _ _ _ stamp _ ; do
  echo "do something with $stamp here"
done < "$file"

Breaking it down a bit:

IFS=, temporarily set the record separator to be ,

read _ _ _ stamp _ store fields 1-3 and 5 onwards in a throwaway variable, while capturing field 4 (your date/time) as stamp

< "$file" read in the source file (this is caught by our read command).

Another way to skin this cat, by populating an array with each line (this works in bash for sure, other shells may not support this or implement it in a different manner):

while IFS=, read -a line ; do echo "${line[3]}" ; done < "$file"