11

Bash lets you specify a redirected input before a command:

$ <lines sed 's/^/line: /g'
line: foo
line: bar

Bash also lets you redirect input to a compound command like a while loop:

$ while read line; do echo "line: $line"; done <lines
line: foo
line: bar

However, when I try to specify a redirected input before a while loop, I get a syntax error:

$ <lines while read line; do echo "line: $line"; done
bash: syntax error near unexpected token `do'

What's wrong with this? Is it not possible to specify a redirected input before a compound command in Bash? If so, why not?

  • (I specifically want to know because I'm directing a command's output to a while loop in the current shell context via process substitution, and I'd prefer it if the first part of the code didn't have to appear after the second part.) – Stuart P. Bentley Sep 17 '14 at 14:04

4 Answers4

12

man bash says:

... redirection operators may precede or appear anywhere within a simple command or may follow a command.

while is not a simple command.

choroba
  • 47,233
7

You can in zsh, not in bash and choroba has already pointed you to the documentation, but if you want to have the redirection before, you can do things like:

< file eval '
  while IFS= read -r line; do
    ...
  done'

Or (on systems with support for /dev/fd/n):

< file 3<< 'EOF' . /dev/fd/3
while IFS= read -r line; do
  ...
done
EOF

(not that's you'd want to do it).

You can also do:

exec 3< file
while IFS= read -r line <&3; do
  ...
done
exec 3<&-

(note that exec will exit the script if file can't be open).

Or use a function:

process()
  while IFS= read -r line; do
    ...
  done

< file process
  • Wonder if eval and /dev/fd/ routes are equally fast. – Deer Hunter Sep 17 '14 at 19:31
  • 1
    @DeerHunter, << EOF involves creating a temp file, filing it up and then open it, read it and interpret as you read it. It's bound to be less fast, though probably not noticeably as it's all going to happen in memory anyway. – Stéphane Chazelas Sep 17 '14 at 20:09
  • "Use a function" doesn't really help - I still have to write the function before the code that gets directed into it (although I suppose I can write a function for that, with the two functions in either order). – Stuart P. Bentley Sep 17 '14 at 22:21
  • command exec can handle the exit issue as w/ other builtins. but it gets in way too late too handle the reserved word bits. – mikeserv Sep 18 '14 at 05:38
  • @StéphaneChazelas What's the reason for using a temporary file rather than a pipe? – kasperd Sep 18 '14 at 10:09
  • @kasperd, that should be a separate question, but a pipe involves forking an additional process to feed the other end so is generally less efficient. Having said that, rc/es and ash use a pipe there, though they are in minority (ash doesn't fork an additional process unless the content doesn't fit in the pipe buffer though) – Stéphane Chazelas Sep 18 '14 at 11:04
  • @StéphaneChazelas In typical use cases there would be no need to fork. Instead of the shell calling wait to wait for the child process to terminate, it could write all the data to the pipe, and then call wait. Using fork would only be needed, if the shell need to keep running with the child continuing in the background. Detecting when fork is needed is slightly tricky. It would be simpler to always fork, that is also simpler to implement than a temporary file, and less prone to security bugs. And with CoW fork shouldn't be a major performance problem. – kasperd Sep 18 '14 at 11:35
  • @kasperd, yes, that's how ash works. It does nor fork except when necessary, but forking has side effects (co-holds file descriptors for instance) as well and possible security implications. While the temp file is something that is proven as it has been used for over 35 years (note that the temp file is deleted before being filled up). It also has the benefit of being seekable. There's good and less good in both approaches. – Stéphane Chazelas Sep 18 '14 at 12:19
3

You can use that substitution, if you want to precede the input:

cat lines | while read line; do echo "line: $line"; done
chaos
  • 48,171
  • 1
    I need this to be executed within the current shell context - no pipelines. (The code in the question was just a simplified example.) – Stuart P. Bentley Sep 17 '14 at 14:35
2

You can use exec to redirect the stdin.

In a script:

exec < <(cat lines)
while read line ; do echo "line: $line"; done

You can't use in a login shell though (it will dump the file on the stdout and exit). In that case you can open a different file descriptor:

exec 3< <(cat lines)
while read -u 3 line ; do echo "line: $line"; done

For reference: Using exec

spider
  • 206
  • And if I'm using stdin for something else I ihave to surround the code in exec 3<&0 and exec <&3, right? (that or use &3 for the redirect ie. your second example) – Stuart P. Bentley Sep 17 '14 at 18:07
  • Yes, you "save" the current stdin to another file descriptor and when you finished you can restore it. (it's well explained in the first example in the linked doc) – spider Sep 18 '14 at 08:55