10

The special variable $RANDOM has a new value every time it's accessed. In this respect, it is reminiscent of the "generator" objects found in some languages.

Is there a way to implement something like this in zsh?

I tried to do this with named pipes, but I did not find a way to extract items from the fifo in a controlled manner without killing the "generator" process. For example:

% mkfifo /tmp/ints
% (index=0
   while ( true )
   do
       echo $index
       index=$(( index + 1 ))
   done) > /tmp/ints &
[1] 16309
% head -1 /tmp/ints
0
[1]  + broken pipe  ( index=0 ; while ( true; ); do; echo $index; index=$(( ...

Is there some other way to implement such a generator-type object in zsh?


EDIT: This does not work:

#!/usr/bin/env zsh

FIFO=/tmp/fifo-$$
mkfifo $FIFO
INDEX=0
while true; do echo $(( ++INDEX )) > $FIFO; done &
cat $FIFO

If I put the above in a script, and run it, the output rarely the expected single line

1

rather, it usually consists of several integers; e.g.

1
2
3
4
5

The number of lines produced varies from one run to the next.

EDIT2: As jimmij pointed out, changing echo to /bin/echo takes care of the problem.

kjo
  • 15,339
  • 25
  • 73
  • 114

3 Answers3

11

ksh93 has disciplines which are typically used for this kind of thing. With zsh, you could hijack the dynamic named directory feature:

Define for instance:

zsh_directory_name() {
  case $1 in
    (n)
      case $2 in
        (incr) reply=($((++incr)))
      esac
  esac
}

And then you can use ~[incr] to get an incremented $incr each time:

$ echo ~[incr]
1
$ echo ~[incr] ~[incr]
2 3

Your approach fails because in head -1 /tmp/ints, head opens the fifo, reads a full buffer, prints one line, and then closes it. Once closed, the writing end sees a broken pipe.

Instead, you could either do:

$ fifo=~/.generators/incr
$ (umask  077 && mkdir -p $fifo:h && rm -f $fifo && mkfifo $fifo)
$ seq infinity > $fifo &
$ exec 3< $fifo
$ IFS= read -rneu3
1
$ IFS= read -rneu3
2

There, we leave the reading end open on fd 3, and read reads one byte at a time, not a full buffer to be sure to read exactly one line (up to the newline character).

Or you could do:

$ fifo=~/.generators/incr
$ (umask  077 && mkdir -p $fifo:h && rm -f $fifo && mkfifo $fifo)
$ while true; do echo $((++incr)) > $fifo; done &
$ cat $fifo
1
$ cat $fifo
2

That time, we instantiate a pipe for every value. That allows returning data containing any arbitrary number of lines.

However, in that case, as soon as cat opens the fifo, the echo and the loop is unblocked, so more echo could be run, by the time cat reads the content and closes the pipe (causing the next echo to instantiate a new pipe).

A work around could be to add some delay, like for instance by running an external echo as suggested by @jimmij or add some sleep, but that would still not be very robust, or you could recreate the named pipe after each echo:

while 
  mkfifo $fifo &&
  echo $((++incr)) > $fifo &&
  rm -f $fifo
do : nothing
done &

That still leaves short windows where the pipe doesn't exist (between the unlink() done by rm and the mknod() done by mkfifo) causing cat to fail, and very short windows where the pipe has been instantiated but no process will ever write again to it (between the write() and the close() done by echo) causing cat to return nothing, and short windows where the named pipe still exists but nothing will ever open it for writing (between the close() done by echo and the unlink() done by rm) where cat will hang.

You could remove some of those windows by doing it like:

fifo=~/.generators/incr
(
  umask  077
  mkdir -p $fifo:h && rm -f $fifo && mkfifo $fifo &&
  while
    mkfifo $fifo.new &&
    {
      mv $fifo.new $fifo &&
      echo $((++incr))
    } > $fifo
  do : nothing
  done
) &

That way, the only problem is if you run several cat at the same time (they all open the fifo before our writing loop is ready to open it for writing) in which case they will share the echo output.

I would also advise against creating fixed name, world readable fifos (or any file for that matters) in world writable directories like /tmp unless it's a service to be exposed to all users on the system.

4

If you want to execute code whenever the value of a variable is read, you can't do that inside zsh itself. The RANDOM variable (like other similar special variables) is hard-coded in the zsh source code. You can however define similar special variables by writing a module in C. Many of the standard modules define special variables.

You can use a coprocess to make a generator.

coproc { i=0; while echo $i; do ((++i)); done }
for ((x=1; x<=3; x++)) { read -p n; echo $n; }

However this is pretty limited because you can only have one coprocess. Another way to progressively get output from a process is to redirect from a process substitution.

exec 3< <(i=0; while echo $i; do ((++i)); done)
for ((x=1; x<=3; x++)) { read n <&3; echo $n; }

Note that head -1 does not work here, because it reads a whole buffer, prints out what it likes, and exits. The data that's been read from the pipe remains read; this is an intrinsic property of pipes (you can't stuff data back in). The read builtin avoids this issue by reading one byte at a time, which allows it to stop as soon as it finds the first newline but is very slow (of course that doesn't matter if you're just reading a few hundred bytes).

1

I think I would do it with a signal of some kind.

(   trap   "read zero </tmp/ints" PIPE
    while  kill -s PIPE -0
    do     i=$zero
           while echo $((i++))
           do :; done 2>/dev/null >/tmp/ints
    done
)&

It works for me, anyway.


$ echo  15 >/tmp/ints; head -n 5 </tmp/ints
15
16
17
18
19
$ echo  75 >/tmp/ints; head -n 5 </tmp/ints
75
76
77
78
79

On an only slightly related note, here's something weird I discovered the other day:

mkdir nums; cd nums
for n in 0 1 2 3 4 5 6 7
do  ln -s ./ "$n"; done
echo [0-3]/*/*

0/0/0 0/0/1 0/0/2 0/0/3 0/0/4 0/0/5 0/0/6 0/0/7 0/1/0 0/1/1 0/1/2 0/1/3 0/1/4 0/1/5 0/1/6 0/1/7 0/2/0 0/2/1 0/2/2 0/2/3 0/2/4 0/2/5 0/2/6 0/2/7 0/3/0 0/3/1 0/3/2 0/3/3 0/3/4 0/3/5 0/3/6 0/3/7 0/4/0 0/4/1 0/4/2 0/4/3 0/4/4 0/4/5 0/4/6 0/4/7 0/5/0 0/5/1 0/5/2 0/5/3 0/5/4 0/5/5 0/5/6 0/5/7 0/6/0 0/6/1 0/6/2 0/6/3 0/6/4 0/6/5 0/6/6 0/6/7 0/7/0 0/7/1 0/7/2 0/7/3 0/7/4 0/7/5 0/7/6 0/7/7 1/0/0 1/0/1 1/0/2 1/0/3 1/0/4 1/0/5 1/0/6 1/0/7 1/1/0 1/1/1 1/1/2 1/1/3 1/1/4 1/1/5 1/1/6 1/1/7 1/2/0 1/2/1 1/2/2 1/2/3 1/2/4 1/2/5 1/2/6 1/2/7 1/3/0 1/3/1 1/3/2 1/3/3 1/3/4 1/3/5 1/3/6 1/3/7 1/4/0 1/4/1 1/4/2 1/4/3 1/4/4 1/4/5 1/4/6 1/4/7 1/5/0 1/5/1 1/5/2 1/5/3 1/5/4 1/5/5 1/5/6 1/5/7 1/6/0 1/6/1 1/6/2 1/6/3 1/6/4 1/6/5 1/6/6 1/6/7 1/7/0 1/7/1 1/7/2 1/7/3 1/7/4 1/7/5 1/7/6 1/7/7 2/0/0 2/0/1 2/0/2 2/0/3 2/0/4 2/0/5 2/0/6 2/0/7 2/1/0 2/1/1 2/1/2 2/1/3 2/1/4 2/1/5 2/1/6 2/1/7 2/2/0 2/2/1 2/2/2 2/2/3 2/2/4 2/2/5 2/2/6 2/2/7 2/3/0 2/3/1 2/3/2 2/3/3 2/3/4 2/3/5 2/3/6 2/3/7 2/4/0 2/4/1 2/4/2 2/4/3 2/4/4 2/4/5 2/4/6 2/4/7 2/5/0 2/5/1 2/5/2 2/5/3 2/5/4 2/5/5 2/5/6 2/5/7 2/6/0 2/6/1 2/6/2 2/6/3 2/6/4 2/6/5 2/6/6 2/6/7 2/7/0 2/7/1 2/7/2 2/7/3 2/7/4 2/7/5 2/7/6 2/7/7 3/0/0 3/0/1 3/0/2 3/0/3 3/0/4 3/0/5 3/0/6 3/0/7 3/1/0 3/1/1 3/1/2 3/1/3 3/1/4 3/1/5 3/1/6 3/1/7 3/2/0 3/2/1 3/2/2 3/2/3 3/2/4 3/2/5 3/2/6 3/2/7 3/3/0 3/3/1 3/3/2 3/3/3 3/3/4 3/3/5 3/3/6 3/3/7 3/4/0 3/4/1 3/4/2 3/4/3 3/4/4 3/4/5 3/4/6 3/4/7 3/5/0 3/5/1 3/5/2 3/5/3 3/5/4 3/5/5 3/5/6 3/5/7 3/6/0 3/6/1 3/6/2 3/6/3 3/6/4 3/6/5 3/6/6 3/6/7 3/7/0 3/7/1 3/7/2 3/7/3 3/7/4 3/7/5 3/7/6 3/7/7

It gets weirder, too:

rm *
for a in  a b c d e f g h \
          i j k l m n o p \
          q r s t u v x y z
do 
    ln -s ./ "$a"
done
for a in *
do  echo "$a"/["$a"-z]
done

a/a a/b a/c a/d a/e a/f a/g a/h a/i a/j a/k a/l a/m a/n a/o a/p a/q a/r a/s a/t a/u a/v a/x a/y a/z
b/b b/c b/d b/e b/f b/g b/h b/i b/j b/k b/l b/m b/n b/o b/p b/q b/r b/s b/t b/u b/v b/x b/y b/z
c/c c/d c/e c/f c/g c/h c/i c/j c/k c/l c/m c/n c/o c/p c/q c/r c/s c/t c/u c/v c/x c/y c/z
d/d d/e d/f d/g d/h d/i d/j d/k d/l d/m d/n d/o d/p d/q d/r d/s d/t d/u d/v d/x d/y d/z
e/e e/f e/g e/h e/i e/j e/k e/l e/m e/n e/o e/p e/q e/r e/s e/t e/u e/v e/x e/y e/z
f/f f/g f/h f/i f/j f/k f/l f/m f/n f/o f/p f/q f/r f/s f/t f/u f/v f/x f/y f/z
g/g g/h g/i g/j g/k g/l g/m g/n g/o g/p g/q g/r g/s g/t g/u g/v g/x g/y g/z
h/h h/i h/j h/k h/l h/m h/n h/o h/p h/q h/r h/s h/t h/u h/v h/x h/y h/z
i/i i/j i/k i/l i/m i/n i/o i/p i/q i/r i/s i/t i/u i/v i/x i/y i/z
j/j j/k j/l j/m j/n j/o j/p j/q j/r j/s j/t j/u j/v j/x j/y j/z
k/k k/l k/m k/n k/o k/p k/q k/r k/s k/t k/u k/v k/x k/y k/z
l/l l/m l/n l/o l/p l/q l/r l/s l/t l/u l/v l/x l/y l/z
m/m m/n m/o m/p m/q m/r m/s m/t m/u m/v m/x m/y m/z
n/n n/o n/p n/q n/r n/s n/t n/u n/v n/x n/y n/z
o/o o/p o/q o/r o/s o/t o/u o/v o/x o/y o/z
p/p p/q p/r p/s p/t p/u p/v p/x p/y p/z
q/q q/r q/s q/t q/u q/v q/x q/y q/z
r/r r/s r/t r/u r/v r/x r/y r/z
s/s s/t s/u s/v s/x s/y s/z
t/t t/u t/v t/x t/y t/z
u/u u/v u/x u/y u/z
v/v v/x v/y v/z
x/x x/y x/z
y/y y/z
z/z
mikeserv
  • 58,310
  • What's weird in it? – Stéphane Chazelas Jan 13 '16 at 12:44
  • @StéphaneChazelas - it just seemed weird that the links would self-recurse. And so easily. I thought it was weird. And cool. I also thought there was supposed to be some kind of depth recursion limit - seems the shell would trigger that - or does it actually need to do 40 links in a single path? – mikeserv Jan 13 '16 at 12:47
  • @StéphaneChazelas - That's good. But maybe bash's behavior has changed? I think the statement about pwd not checking and referring only to $PWD is incorrect. mkdir /tmp/dir; cd $_; PS4='$OLDPWD, $PWD + '; set -x; OLDPWD=$OLDPWD PWD=$PWD command eval ' cd ..; cd ..; cd ~; pwd'; pwd; cd .; pwd might show you what I mean. It's a problem that bugged me w/ this ns() thing. – mikeserv Jan 13 '16 at 13:07