what is "read" in this context?

Question

Considering:

cat filename | while read i
do
    echo $i
done

all other answers failed, this one works! this one will cut text file to line by line.
I want to know more about the key part read. But I cannot found the source code. which read no output. what is "read" in this context?

See info bash read and Understanding "IFS= read -r line" and Why is using a shell loop to process text considered bad practice? and Why is printf better than echo? — Stéphane Chazelas, Aug 13 '23 at 07:06
@StéphaneChazelas thx. my use case is cut text line by line then copy to database. since sql have delimiter and quote other messy case. while read i successfully copied to database. will try printf. — jian, Aug 13 '23 at 07:23
You'd be better of generating the whole SQL code from the input using things like awk bearing in mind that it's very hard to avoid SQL injection vulnerabilities that way (same with your read+echo approach), or use proper programming languages like perl that have modules to talk to databases. — Stéphane Chazelas, Aug 13 '23 at 07:30
Most database systems have a configurable fast-loader tool (which will for example fix up indexes all at once at the end, instead of updating after every commit). First time I used Oracle's SQL*Loader, I ran it about five times because I simply did not believe it had done anything useful that fast. — Paul_Pedant, Aug 13 '23 at 08:12

Chris Davies · Answer 1 · 2023-08-13T09:54:02.577

this one works

Actually it doesn't. The read will parse the line and rebuild it according to shell rules. For example,

echo '  hello  world  ' |
    ( read x; echo ">$x<" )

The captured $x has neither leading nor trailing space:

>hello  world<

(Change read x to IFS= read -r x and compare the result.) Further, by not double-quoting the variable $x when you use it, it is subject to further processing. For example,

touch a b c    # Create three empty files
x='here is my star: *'
echo "$x"      # Outputs: here is my star: *
echo $x        # Outputs: here is my star: a b c

In your case the files a, b, c will also be supplemented by the set of files you have in your current directory. (I should point out that touch only creates empty files as a side effect; if the files already exist it will simply update the last-modified time to now and will most definitely not truncate or recreate them.)

After all that, you won't be able to use the value of $x outside the loop as you've got the loop in a subshell. (Set x='demo' before the loop starts and then echo "$x" after the loop ends. It will have been unaffected by the loop.) You don't need the cat in this case as there's only one file:

while IFS= read -r x
do
    printf '%s\n' "$x"
done <filename

But even after all that, typically there is little need to loop over input like this as many tools process files line by line anyway. c.f.

cat filename

Back in SunOS 4 days, cat was in /usr/bin and so not available at boot time. So in /etc/rc.boot we could find a function definition for shcat which was "good enough". Basically shcat() { while test $# -ge 1 do while read i do echo "$i" done < $1 shift done }. Was sufficient to allow boot processes to read config files (eg hostname.??0) :-) — Stephen Harris, Aug 13 '23 at 12:28
@StephenHarris while I liked SunOS 4 at the time (and SVR3, just to play fair) I don't think I'd choose to go back to it now :-) — Chris Davies, Aug 13 '23 at 12:38

score 2 · Accepted Answer · edited Aug 13 '23 at 07:50

2

This is the read shell builtin command, which is specified by POSIX. The Bash implementation is described in the Bash manual, and its source code is part of the Bash repository.

edited Aug 13 '23 at 07:50

Stéphane Chazelas

544,893

answered Aug 13 '23 at 07:39

Stephen Kitt

434,908

what is "read" in this context?

2 Answers2