How to check if a pipe is empty and run a command on the data if it isn't?

Question

I have piped a line in bash script and want to check if the pipe has data, before feeding it to a program.

Searching I found about test -t 0 but it doesn't work here. Always returns false. So how to be sure that the pipe has data?

Example:

echo "string" | [ -t 0 ] && echo "empty" || echo "fill"

Output: fill

echo "string" | tail -n+2 | [ -t 0 ] && echo "empty" || echo "fill"

Output: fill

Unlike Standard/canonical way to test whether foregoing pipeline produced output? the input needs to be preserved to pass it to the program. This generalizes How to pipe output from one process to another but only execute if the first has output? which focuses on sending email.

Answer 1

There's no way to peek at the content of a pipe using commonly available shell utilities, nor is there a portable way to read a character from the pipe then put it back. The only way to know that a pipe has data is to read a byte, and then you have to get that byte to its destination.

So do just that: read one byte; if you detect an end of file, then do what you want to do when the input is empty; if you do read a byte then fork what you want to do when the input is not empty, pipe that byte into it, and pipe the rest of the data.

first_byte=$(dd bs=1 count=1 2>/dev/null | od -t o1 -A n | tr -dc 0-7)
if [ -z "$first_byte" ]; then
  # stuff to do if the input is empty
else
  {
    printf "\\$first_byte"
    cat
  } | {
    # stuff to do if the input is not empty
  }      
fi

The ifne utility from Joey Hess's moreutils runs a command if its input is not empty. It usually isn't installed by default, but it should be available or easy to build on most unix variants. If the input is empty, ifne does nothing and returns the status 0, which cannot be distinguished from the command running successfully. If you want to do something if the input is empty, you need to arrange for the command not to return 0, which can be done by having the success case return a distinguishable error status:

ifne sh -c 'do_stuff_with_input && exit 254'
case $? in
  0) echo empty;;
  254) echo success;;
  *) echo failure;;
esac

test -t 0 has nothing to do with this; it tests whether standard input is a terminal. It doesn't say anything one way or the other as to whether any input is available.

Here with 254 chosen arbitrarily, avoiding 255 as commands doing some exit(-1) (which usually result in $? being 255) are relatively common (-exec(-2) less so).

Answer 2

A simple solution is to use ifne command (if input not empty). In some distributions, it is not installed by default. It is a part of the package moreutils in most distros.

ifne runs a given command if and only if the standard input is not empty

Note that if the standard input is not empty, it is passed through ifne to the given command

Answer 3

check if file descriptor of stdin (0) is open or closed:

[ ! -t 0 ] && echo "stdin has data" || echo "stdin is empty"

Answer 4

If you like short and cryptic one-liners:

$ echo "string" | grep . && echo "fill" || echo "empty"
string
fill
$ echo "string" | tail -n+2 | grep . && echo "fill" || echo "empty"
empty

I used the examples from the original question. If you don't want the piped data use -q option with grep

Some commenters rightfully noted that this version will ignore new lines. My answer is focused on the original question. From this point of view a newline symbol is an empty string. Using grep ^, as suggested by @EvgEnZh solves this corner case, but I am not sure it is what the requestor wanted. Please, add this variant in comments, it is a useful correction.

Answer 5

Old question, but in case someone comes across it as I did: My solution is to read with a timeout.

while read -t 5 line; do
    echo "$line"
done

If stdin is empty, this will return after 5 seconds. Otherwise it will read all the input and you can process it as needed.

In these situations where -t is supported, you can test for input before reading any data with -t0. (also specifying -u0 for STDIN because awslinux needs it, whereas ubuntu assumes it)

if [ $(read -u0 -t0) ]; then ....

And you can start reading as normal afterward

from help read:

If TIMEOUT is 0, read returns immediately, without trying to read any data, returning success only if input is available on the specified file descriptor.

Answer 6

One easy way to check if there's data available for reading in Unix is with the FIONREAD ioctl.

I cannot think of any standard utility doing just that, so here is a trivial program doing it (better than the ifne from moreutils IMHO ;-)).

fionread [ prog args ... ]

If there's no data available on stdin, it will exit with status 1. If there's data, it will run prog. If no prog is given, it will exit with status 0.

This should work with most kinds of file descriptors, not just pipes.

fionread.c

#include <unistd.h>
#include <poll.h>
#include <sys/ioctl.h>
#ifdef __sun
#include <sys/filio.h>
#endif
#include <err.h>
int main(int ac, char *av){
        int r; struct pollfd pd = { 0, POLLIN };
        if(poll(&pd, 1, -1) < 0) err(1, "poll");
        if(ioctl(0, FIONREAD, &r)) err(1, "ioctl(FIONREAD)");
        if(r < 1) return 1;
        if(++av, --ac < 1) return 0;
        execvp(av, av);
        err(1, "execvp %s", *av);
}

Answer 7

In bash:

read -t 0 

Detects if an input has data (without reading anything). Then you can read the input (if the input is available at the time the read is executed):

if     read -t 0
then   read -r input
       echo "got input: $input"
else   echo "No data to read"
fi

---

Note: Understand that this depends on timing. This detects if input already has data only at the time read -t runs.

For example, with

{ sleep 0.1; echo "abc"; } | read -t 0; echo "$?"

the output is 1 (read failure, i.e.: empty input). The echo writes some data but is not very fast to start and write its first byte, thus, read -t 0 will report that its input is empty, since program has not written anything yet.

Answer 8

You may use test -s /dev/stdin (in an explicit subshell) as well.

# test if a pipe is empty or not
echo "string" | 
    (test -s /dev/stdin && echo 'pipe has data' && cat || echo 'pipe is empty')

echo "string" | tail -n+2 | 
    (test -s /dev/stdin && echo 'pipe has data' && cat || echo 'pipe is empty')

: | (test -s /dev/stdin && echo 'pipe has data' && cat || echo 'pipe is empty')

Answer 9

If you need this functionality for use in xargs and you have GNU xargs, you can use xargs -r. Ex. curl won't run here:

echo -n '' | xargs -r curl 

Answer 10

The following approach worked like a charm for me:

function _read_from_stdin() {
  less <&0 2>/dev/null
}

It reads from STDIN and returns its contents (if any). All you need to do afterwards is to check if contents is empty:

stdin_contents="$(_read_from_stdin)"
if [[ ! "${stdin_contents}" ]] ; then
  echo "STDIN is empty!"
else
  echo "Have something in STDIN: ${stdin_contents}"
fi

Answer 11

Full POSIX solution

Compile the following file containing a call to poll() into a binary called has_data:

#include <poll.h>
int main() {
    struct pollfd fds = { 0, POLLIN, 0};
    poll(&fds, 1, -1);
    return fds.revents & POLLIN ? 0 : 1;
}

gcc -o has_data has_data.c

Usage

with a pipeline

cat only runs if has_data succeeds

echo foobar | ( has_data && cat ) | ...

with a file descriptor

N is the number of the file descriptor opened in the current Shell Execution Environment

if has_data <&N; then
    echo "Has data"
fi

with a filename

if has_data <filename; then
    echo "Has data"
fi

There is a problem with this approach when filename is a FIFO, though. According to the documentation of close:

When all file descriptors associated with a pipe or FIFO special file are closed, any data remaining in the pipe or FIFO shall be discarded.

This means that when has_data closes the FIFO file and the file is not opened by another process, the data yet to be read will be lost.

Answer 12

Another hack that you can try is using timeout to read from /dev/stdin like this:

timeout 1s cat /dev/stdin > /tmp/input
if [ $? -eq 124 ] && ! [ -s /tmp/input ]
then
  echo "No input provided."
else
  echo "Input provided"
  # Optionally, you can move or read from /tmp/input...whatever your program needs to do.
fi

Answer 13

Use grep

chris@SR-ENG-P18 /cygdrive/c/Projects
$ ( ! echo -n "" | grep -q '.' && echo "empty" || echo "fill" )
empty
chris@SR-ENG-P18 /cygdrive/c/Projects
$ ( ! echo -n "string" | grep -q '.' && echo "empty" || echo "fill" )
fill

grep will return 1 (i.e. error) when nothing is matched. You want the opposite thus the not operator !.

Be sure to use echo -n "" in your testing because echo "" will output a newline.

Answer 14

I find the following is the most elegant, I use it in scripts:

function isempty {
    ifne false
}
function isnotempty {
    ifne -n false
}

It uses ifne from moreutils, as others already suggested.

Answer 15

This seems to be a reasonable ifne implementation in bash if you're ok with reading the whole first line

ifne () {
        read line || return 1
        (echo "$line"; cat) | eval "$@"
}


echo hi | ifne xargs echo hi =
cat /dev/null | ifne xargs echo should not echo

Answer 16

This works for me using read -rt 0

example from original question, with no data:

echo "string" | tail -n+2 | if read -rt 0 ; then echo has data ; else echo no data ; fi

Answer 17

Another approach is reformulating the problem as a pure problem of running a command with the input from a stream and massaging that stream to fit your use case. Ie, you want a command to operate on a stream of data and this data may come from stdin and/or another stream(s).

AND

If its stdin and another stream it should be pretty straightforward using cat

echo bar |
  cat - <(echo biz) |
  cat -A
# output: bar$biz$

This will pipe the contents of stdin and the output of the process substitution echo bar to the stdin of cat -A.

OR

Or on the other hand is a bit trickier. Let's say you want to run a command with input from stdin but fallback to something else if nothing was passed to stdin.

In this case an approach is reformulate the problem as combining streams (including a delimiter sequence say two null-bytes) and sinking the stream IFF stdin is non-empty by using sed and matching the delimiter in the combined stream.

echo bar |
  cat - <(printf '\x0\x0\n') <(echo biz) |
  sed \
    -e '1{/\x0\x0/s@@@;N;s@\n@@}' \
    -e '1!{/\x0\x0/s@@@;q}' | # stdin was non-empty, sink the rest
    cat -A
# output: bar$
< /dev/null |
  cat - <(printf '\x0\x0\n') <(echo biz) |
  sed 

    -e '1{/\x0\x0/s@@@;N;s@\n@@}' 

    -e '1!{/\x0\x0/s@@@;q}' | # stdin was non-empty, sink the rest
    cat -A
output: biz$

We can combine this approach with xargs -I% for an additional fallback. It uses process substitution but other than that it should be pretty POSIX compatible.

#!/bin/bash
This script takes a list of paths from stdin
IFF it's empty the list of files is the existing file 'foo'
Each file and/or directory in this list is passed to stat
IFF this list is empty we run stat on the current directory
combined_stream() {
  if [ -t 0 ]; then
    cat <(printf '\x0\x0\n') <(find foo -type f 2> /dev/null)
  else
    cat - <(printf '\x0\x0\n') <(find foo -type f 2> /dev/null)
  fi
}
combined_stream |
  sed 

    -e '1{/\x0\x0/s@@@;N;s@\n@@}' 

    -e '1!{/\x0\x0/s@@@;q}' | # stdin was non-empty, sink the rest
  xargs -I% stat % |
  grep . || stat .

Answer 18

Is more difficult to be achieved on C Shell, but I made it.

#!/bin/tcsh -f
while 1
  head -c1 | sed p | 

  ( exit ( ! head -c1 | wc -c ) && 

  tail -c1 ) || exit
end

Also achievable on Bourne Shells, though I think may not be the best solution.

#!/bin/sh
while :
do
  head -c1 | sed p | 

  ( ! ( exit head -c1 | wc -c ) && 

  tail -c1 ) || exit
done

Answer 19

Simplest ternary

doSomethingAndCheckTruth && echo 'yes' || echo 'no'

The right part (after the &&) will not run unless the left part is complete, so you only check truth upon successfully running something else, saving computation.

Example

using homebrew and grep we check if the bat program is installed

• If true you will see "yes"
• If false you will see "no"

I added the -q to suppress the grepped string output here, so you only see "yes" or "no"

brew list | grep -q bat && echo 'yes' || echo 'no'

Tested with bash and zsh