Grep for a string on remote server returns no results

Question

#!/bin/bash
params=$(cat /var/tmp/patterns.txt)
remoteLog="/home/remotefile.txt"
nodes=("192.168.1.2" "192.168.1.3")

for node in "${nodes[@]}";do
    ssh  ${node} << 'EOF'
    grep  -i "${params}" "$remotelog" >/var/tmp/output.txt
EOF
done 

cat /var/tmp/patterns.txt 
abc 
efg 
ijk

I am trying to login to a remote server and grep for strings that are specified in the patterns.txt file which resides on my local machine. When I run the script, it logins to the remote machine but doesn't grep on the patterns specified in my patterns.txt file which resides on my local machine.

The script works fine if I create the patterns.txt file on the remote server but it doesn't when that file resides on my local machine from where I am running the script.

Note that using files with fixed guessable file names in world writable directories such as /var/tmp is bad practice from a security standpoint as attackers can booby-trap them by creating symlinks by the same name to some files you have write access to which will end up overwritten if write something to them. — Stéphane Chazelas, Jan 30 '23 at 19:38

Kusalananda · Answer 1 · 2023-01-30T19:36:21.083

1

Your here-document is introduced with <<'EOF'. Since you quote the initial delimiter, the whole here-document will be quoted. This in turn means that the shell will not perform expansions in it, which means that both variables will be untouched by the local shell and instead expanded by the remote shell, which will most likely expand them to empty strings.

Another issue is the way you use $params with grep. It will be used as a single regular expression, not a separate regular expression per line (which is what I assume is what you intend).

Instead:

for node in "${nodes[@]}"; do
    ssh "$node" "cat -- '$remotelog'" |
    grep -f "$params" |
    ssh "$node" 'cat >/var/tmp/output.txt'
done

This performs the grep operation locally on data fetched from each host. The grep operation uses the regular expressions from the $params file through the -f option and then outputs the result back to the file on the remote host.

The alternative is to transfer the patterns over to each host and perform the operation remotely:

for node in "${nodes[@]}"; do
    scp "$params" "$node:/var/tmp/patterns.txt" &&
    ssh "$node" <<EOF
    grep -f /var/tmp/patterns.txt '$remotelog' >/var/tmp/output.txt
    rm -f /var/tmp/patterns.txt
EOF
done

edited Jan 30 '23 at 19:36

answered Jan 30 '23 at 19:30

Kusalananda

333,661

Beware that ssh "$node" cat "$remotelog" is the same as ssh "$node" "cat $remotelog". The concatenation of cat, a space and the contents of $remotelog will be evaluated as shell code by the login shell of the remote user. So it only works if $remotelog doesn't contain any character special in the syntax of the remote shell (and doesn't start with - as the -- is missing). See also How to execute an arbitrary simple command over ssh without knowing the login shell of the remote user? – Stéphane Chazelas Jan 30 '23 at 19:34
@StéphaneChazelas Thanks! Hopefully a bit better now. Would the alternative loop be a safer way of doing it? – Kusalananda Jan 30 '23 at 19:36
How to execute an arbitrary simple command over ssh without knowing the login shell of the remote user? gives approaches for doing it safely. Here you could construct proper sh code dynamically and pipe to ssh host sh. – Stéphane Chazelas Jan 30 '23 at 19:41
If you know the client shell is the same as the login shell on the target and is Korn-like, you can also do things like ssh host "$(typeset -p var1 var2);"' code that uses "$var1" "$var2"' to transmit variables from local shell to remote shell (though that's only totally safe if both OSes are the same and the locales end up being the same. See Escape a variable for use as content of another script) – Stéphane Chazelas Jan 30 '23 at 19:43

Grep for a string on remote server returns no results

1 Answers1