bash array acting weird

Question

I have two folders under /tmp.

From terminal:

ls -d /tmp/firefox-*
/tmp/firefox-sy2vakcj.default-esr-charlie-cache
/tmp/firefox-sy2vakcj.default-esr-charlie-profile

or

compgen -G /tmp/firefox-*
/tmp/firefox-sy2vakcj.default-esr-charlie-cache
/tmp/firefox-sy2vakcj.default-esr-charlie-profile

And i can store the output in an array as well:

arr=( $(ls -d /tmp/firefox-*) )
echo $arr
tmp/firefox-sy2vakcj.default-esr-charlie-cache  /tmp/firefox-sy2vakcj.default-esr-charlie-profile
echo $arr[1] 
tmp/firefox-sy2vakcj.default-esr-charlie-cache
echo $arr[2]
/tmp/firefox-sy2vakcj.default-esr-charlie-profile

so far so good.

But if i attempt the same thing from a script:

    ...
    ... 
    arr=( "$(ls -d /tmp/firefox-*)" ) ||( echo "directory doesn't exist" && exit 1)
    #arr=( "$(compgen -G /tmp/firefox-*)" ) ||( echo "directory doesn't exist" && exit 1)
    echo "this is a test for arr[1]: $arr[1]"
    echo "this is a test for arr[2]: $arr[2]"
    ...

I get the output:

From script:

for ls -d this is the output:

+ arr=("$(ls -d /tmp/firefox-*)")
++ ls -d '/tmp/firefox-*'
ls: cannot access '/tmp/firefox-*': No such file or directory
+ echo 'directory doesn'\''t exist'
directory doesn't exist

and for compgen -G , this is the output:

this is a test for arr[1]: /tmp/firefox-sy2vakcj.default-esr-charlie-cache
/tmp/firefox-sy2vakcj.default-esr-charlie-profile[1]
this is a test for arr[2]: /tmp/firefox-sy2vakcj.default-esr-charlie-cache
/tmp/firefox-sy2vakcj.default-esr-charlie-profile[2]

My questions:

1. Why is the glob not expanding in the subshell for the command ls -d ?

2. With compgen -G, how are the values being stored in the array? the output seems like each entry in the array is storing both the directory entry with the second one with its own index array?

3. Is the output from terminal for both commands different from the script, or am i missing something?

Why did you enclose the command substitution in double-quotes in your script? It would lead to the multi-word output being treated as one token (and hence only one array element). Also, array elements are referenced as ${arr[1]} - $arr[1] (as you can see) is the same as $arr followed by the literal string [1], and $arr when used on an array is equal to the first array element. I would recommend shellcheck to check for such errors. — AdminBee, Oct 19 '21 at 09:30
Does this answer your question? Why does my shell script choke on whitespace or other special characters? — Wieland, Oct 19 '21 at 09:34
that part with echo $arr and echo $arr[1] looks like zsh. Except for the part where the first array element is missing the leading slash. Check to make sure what shell you're running, and copypaste the complete commands and their complete output, and if you're asking about scripts, include the complete script, and the complete command line you use to run the script. — ilkkachu, Oct 19 '21 at 09:41
@Wieland why would that answer the question? Is this related to quoting and whitespace? How? — terdon, Oct 19 '21 at 09:46
Are you maybe using macOS? That has a default shell of zsh which would explain part of what you're showing here as ilkkackhu pointed out. — terdon, Oct 19 '21 at 09:46
Are you sure that is the script you are running? Did you maybe have single and not double quotes here: arr=('$(ls -d /tmp/firefox-*)')? The output of set -x suggests you did. — terdon, Oct 19 '21 at 09:56
Okay, after reading @AdminBee 's comment, removed the "". i am still reading into the details and update when i get the whole picture. FIrst thing first, yes my interactive shell is zsh and i was testing on it while my script is in bash and that's why the simple $arr references. I changed to bash for testing. And the command substitution was the main problem (as pointed out by @AdminBee). Second issue was escaping the * glob. so compgen -G /tmp/firefox-\* is the correct. Third, Bash index starts at 0 while for zsh it starts at 1 so that was another source of confusion. — Just Khaithang, Oct 19 '21 at 11:03
another point: while in zsh, ls -d works both from terminal as well as from script (#!/bin/zsh). While from bash shell, it works in the terminal but not in script. in bash (terminal), i have extglob and gobalasciiranges shell options turned on while the others globstar,dotglob,failglob,nocaseglob,nullglob are turned off. I suspect it has to do with the subhsell not getting the required environment to expand the * glob — Just Khaithang, Oct 19 '21 at 11:10
Please [edit] your question and show us the entire script you are using (make it minimal, just enough to reproduce the issue). The way that you set shell options is also very important. They don't work the same in scripts and in interactive use. So, please [edit] and add all the information you've given in comments and make it clear what commands/scripts are being run by what shell and also in what way (bash script.sh or /path/script.sh). — terdon, Oct 19 '21 at 11:39

score 1 · Accepted Answer · answered Oct 19 '21 at 14:41

Why is the glob not expanding in the subshell for the command ls -d ?

You've probably turned off globbing by using set -f. To demonstrate:

$ touch firefox-1 firefox-2
$ arr=( firefox-* ); declare -p arr
declare -a arr=([0]="firefox-1" [1]="firefox-2")
$ set -f
$ arr=( firefox-* ); declare -p arr
declare -a arr=([0]="firefox-*")

With compgen -G, how are the values being stored in the array? the output seems like each entry in the array is storing both the directory entry with the second one with its own index array?

When you do arr=( "$(compgen -G /tmp/firefox-*)" ), the double quotes force the compgen output to be stored as a single element in the array. In this case, to read the lines of output into an array, use mapfile with a process substitution:

$ mapfile -t arr < <(compgen -G ./firefox-*)
$ declare -p arr
declare -a arr=([0]="./firefox-1" [1]="./firefox-2")

Is the output from terminal for both commands different from the script, or am i missing something?

Looks like your interactive shell is zsh. Aside from that, you're missing the parameter expansion syntax for array elements requiring the braces (3.5.3 Shell Parameter Expansion), and that bash arrays are indexed starting from zero:

echo "${arr[0]}"

bash array acting weird

1 Answers1