Expose awk over tcp (inetd, socat, etc.)

Question

I was playing around with awk, and I wanted to expose it over a tcp socket.

At first I tried, but failed with micro-inetd. Thinking that maybe the issue could be in the way that carriage returns/newlines characters are handled, I decided to try to switch to socat.

Using a remote command with socat is easy enough:

socat TCP4-LISTEN:9000 EXEC:/bin/cat
socat TCP4-LISTEN:9000 SYSTEM:"/bin/python -i"

both of these work

But when trying the same with a simple awk command (for example `awk '{print NR}' to progressively count the number of lines received) I don't get anything back when connecting with socat from another shell on the same machine.

I tried plenty of things (some of these fail with a syntax error that is a bit obscure to me. I cannot even try to strace socat to see what exactly is being executed, since the actual command exec'ing is delegated to the shell):

socat TCP4-LISTEN:9000 SYSTEM:"/usr/bin/awk '{print NR}'"
socat TCP4-LISTEN:9000 SYSTEM:'/usr/bin/awk "{print NR}"'
socat TCP4-LISTEN:9000 SYSTEM:'/usr/bin/awk "{print\ NR}"'

I also created a small script /tmp/testawk

#! /bin/sh
/usr/bin/awk '{print NR}'

(also with an exec variant)

#! /bin/sh
exec /usr/bin/awk '{print NR}'

Both of these work locally, but when invoking

socat TCP4-LISTEN:9000 EXEC:/tmp/testawk

It doesn't send me back the output

Thinking that the issue might be inheritance of the stdin/stdout handles and/or again handling of crlf... I tried also with:

socat TCP4-LISTEN:9000 EXEC:/tmp/testawk,nofork
socat TCP4-LISTEN:9000,crnl EXEC:/tmp/testawk
socat TCP4-LISTEN:9000 EXEC:/tmp/testawk,crnl
socat TCP4-LISTEN:9000,crnl EXEC:/tmp/testawk,crnl

I've found someone else who complained about a similar issue, but apparently it failed to garner any attention

Any idea on what is going wrong?

score 2 · Answer 1 · edited Apr 13 '17 at 12:36

2

Awk is printing to standard output, which is line buffered when it points to a terminal, and fully buffered if not; this means you should get output after it fills the buffer, ie it has printed (eg) 4KB.

You can test this by sending a lot of output (eg using the yes program); to fix you should either: - unbuffer awk's stdout (see eg https://unix.stackexchange.com/a/25375/63847) - use the shutdown(2) system call from the sender (ugly)

Any better solution is welcome

edited Apr 13 '17 at 12:36

Community

1

answered Feb 20 '15 at 17:34

loreb

121

Thanks, I thought about it, but overlooked it when playing around with different options. Anyhow I tried with stdbuf -oL -eL /usr/bin/awk '{print NRq}' but I still cannot see anything from the other side – berdario Feb 20 '15 at 18:13
The pty option apparently was what I was looking for: http://stackoverflow.com/questions/6344501/make-socat-write-to-tcp-or-unix-socket-line-by-line-for-log-gathering – berdario Feb 20 '15 at 18:24

score 1 · Accepted Answer · answered Feb 20 '15 at 22:47

You can tell socat to associate a pty with the awk program; this forces awk to run in line buffered mode and you get the immediate responses you're looking for.

Here's the command line to listen on port 9000:

socat TCP4-LISTEN:9000 SYSTEM:'/tmp/awk.sh',pty,echo=0

And here's the contents of the script /tmp/awk.sh:

#!/usr/bin/awk -f
#
BEGIN   { print "This is the awk socket" }
        { print NR, $0 }
END     { print "All done" }

Sample run:

$ nc -vvv otherhost 9000
otherhost [192.168.1.15] 9000 (?) open
This is the awk socket
hello world
1       hello world
how are you
2       how are you
boo
3       boo
^C sent 28, rcvd 61

Expose awk over tcp (inetd, socat, etc.)

2 Answers2