2

this is super trivial and i did find a lot of answers close, but still can't work this out. i made a script with an argument, basically something like:

var1=$1
echo "outside the pipe, my variable is set: $var1"
ls *.* | xargs sh -c 'echo "$0" "$@"; echo "inside the pipe, my variable is empty: $var1"'

my problem is, that i need to pipe a list of file names and also my one variable. how would i do this? i tried to write my file list into an array, then append/prepend my argument and pass this into the pipe but this does create problems later since i process this in chunks, my original script is something like (imagemagick):

ls *.png | xargs -n 100 sh -c 'convert "$0" "$@" -evaluate-sequence $var1 ../out2/"$0"'

any help is appreciated a lot!

user3647558
  • 77
  • 1
    wow! i'm stunned by all this help, thank all of you so much!!! for the time being, i first tried the export method which does get the job done (actually i even remember stumbling over this but just did not get it at the time). it will take me some time to understand the other solutions but i will look into this - and eventually maybe get to a point, where i can avoid asking such dumb questions... again, thanks to all of you! – user3647558 Jul 14 '18 at 21:04

3 Answers3

1

No, you do not have to pipe a list of filenames.

var1=$1

find . -maxdepth 1 -type f -name '*.png' -exec sh -c '
    var1=$1; first=$2; shift 2
    convert "$first" "$@" -evaluate-sequence "$var1" ../out2/"$first"' sh "$var1" {} +

That is, give the internal child shell $var1 as the first argument (the zeroth argument should be the name of the shell).

The internal shell takes $var1 off of the parameter list, and also does the same for the first pathname that find finds (I don't know why, but this at least makes it equivalent to what I believe your code does), and then shifts off these from $@ using shift 2.

If you need to do this in batches of exactly 100:

var1=$1

find . -maxdepth 1 -type f -name '*.png' -print0 |
xargs -0 -n 100 sh -c '
    var1=$1; first=$2; shift 2
    convert "$first" "$@" -evaluate-sequence "$var1" ../out2/"$first"' sh "$var1"

The reason this is safer than piping the output of ls is because the pathnames are passed as a nul-terminated list rather than a newline-terminated list. The sh script is the same in both variations and takes $var1 as a command line argument.

The reason your code doesn't work is because the sh -c shell doesn't know $var1 from its parent shell.

Related:

Kusalananda
  • 333,661
0

The sh -c is a new shell so you need to do an export of $var1 so that children procs have it or pass it into sh -c as an argument to it.

The other issue with your example is that you have the variable inside of single ticks.

slm
  • 369,824
0

You don't really have to run a another shell to do this. To handle all the files matching the glob expression, just use the glob on the command line of the program you want to run in the end:

convert *.png -whatever /some/output/path

If you need to access just part of the files, store them in an array and slice or index it:

files=(*.png)
convert "${files[0]}" "${files[@]:1}" -evaluate-sequence "$var1" ../out2/"${files[0]}"

("${files[0]}" is the first filename, "${files[@]:1}" expands to all the others)

From there, we can process the files one hundred at a time:

for (( i=0; i < "${#files[@]}" ; i+=100 )); do 
    convert "${files[@]:i:100}" -evaluate-sequence "$var1" ../out2/"${files[i]}"
done

That is, if I interpreted your xargs command correctly. It seems to pass the first file name as the zeroth argument to sh -c (usually as the name of the shell).

But to answer the question as you asked it, you can export the variable to make it visible to the inner shell, or put it in double quotes to have it expanded by the outer shell. E.g. 

var=foo
export var
echo bar | xargs sh -c 'echo "$var" "$1"' sh

or 

var=foo
echo bar | xargs sh -c "echo '$var' "'"$1"' sh

Both output foo bar. Passing through the environment is better in that the syntax is cleaner (note the different types of quotes around $var and $1 in the latter), and any quotes in the variable value will not cause issues, as they would in the second case.

ilkkachu
  • 138,973