What is the setting in bash for globbing, to control whether * matches dot files

Question

I was surprised recently when I did something like mv ./* ../somedirectory and found that files like .gitignore were not moved.

I do most of my work in zsh on OS X, and this surprise bit me in bash on CentOS. I tried bash on OS X and found the same behavior: * does not match dot files. This seems very undesirable to me, but apparently it's the bash default. (It may be the zsh default too for all I remember, but I may have changed it years ago in my .zshrc and forgotten it ever worked differently.)

How can I configure bash to behave as I expected: for * to match all files, and not ignore dot files.

In case this is at all unclear, here's how to reproduce it

cd /tmp
mkdir {t,d}est
touch test/{.,}{1,2,3,4,5,6,7}
ls -hal test
mv test/* dest
ls -hal test     # notice dot files are still there
ls -hal dest     # notice only some files were mv'ed

possible duplicate of How do you move all files (including hidden) in a directory to another? — Gilles 'SO- stop being evil', Jun 14 '12 at 00:56
Yes, they're related. That didn't show up when I searched (probably because the questioner of that didn't mention globbing or dot files), but it's really a different question. He was asking how to move the files, and I was asking how to change the shell behavior, and specifically for bash. I might not have asked if that question had shown up for me (since your extremely complete and thorough answer there includes the bash setting) but it's still a different question, and someone looking for an answer to my question isn't necessarily going to find his question. — iconoclast, Jun 18 '12 at 14:41
That whole “someone looking for an answer to my question isn't necessarily going to find his question” business is exactly why we have this way of closing questions as duplicates. — Gilles 'SO- stop being evil', Jun 18 '12 at 22:05
yes, but you seem to be missing the main point, which is that it's a different question. — iconoclast, Oct 25 '16 at 20:14

Ulrich Dangel · Accepted Answer · 2012-06-14T00:38:37.723

19

Bash

As you already noticed bash won't match a . at the start of the name or a slash. To change the matching regarding the dot you have to set the dotglob option - man bash:

dotglob If set, bash includes filenames beginning with a `.'  in
    the results of pathname expansion.

To enable/set it with bash use shopt, e.g:

shopt -s dotglob

For zsh you can also use the dotglob option but but you will have to use setopt to enable it, e.g:

setopt dotglob

edited Jun 14 '12 at 00:38

answered Jun 13 '12 at 03:15

Ulrich Dangel

25,369

2

With zsh, the better option is to enable dotglob on a per-glob basis: mv test/*(D) /dest – Stéphane Chazelas Apr 27 '16 at 11:28
1

Just because I stepped here looking to turn it back off, to disable it the command is shopt -u dotglob. – mazunki Aug 26 '20 at 22:20

Stéphane Chazelas · Answer 2 · 2016-09-29T06:48:19.573

* is a glob that is expanded by the shell. By default shells don't include files whose name starts with a . (called hidden files or dotfiles) unless the leading . is entered literally.

* or [.]* or ?* or *.* or dir/* will not include dotfiles.

.* or dir/.* will.

So you could do:

mv -- * .* /dest/

however some shells including bash (but not zsh, mksh nor fish) have that misfeature that the expansion of .* include the . and .. special directory entries, which you don't want here (and generally never want a glob to include which is why I call it a misfeature).

For that reason, you'll find that sometimes people use (in Bourne-like shells):

mv -- * .[!.]* ..?* /dest/

That's three globs, the first one matching non-hidden files, the second one filenames starting with . followed by a character other than . and the 3rd one filenames starting with .. followed by at least one character.

However some modern shells have better ways around that

zsh

With zsh, you can use the (D) glob qualifier to specify that the glob should include dotfiles:

mv -- *(D) /dest/

zsh also fixed that other misfeature of the Bourne shell in that if the pattern doesn't match, the mv command is not run.

As said above it also will never include . nor .. in its globs, so

mv -- * .* /dest/

will be safe. However if there's no file matching * or no file matching .* the command will be aborted, so it would be better to use:

mv -- (*|.*) /dest/

Like in some other shells, you can also force all globs to include dotfiles (for instance if you find yourself wanting dotfiles included more often than not) with:

setopt dotglob

or:

set -o dotglob

After that, if you want a particular glob not to include dotfiles, you can write it:

echo *(^D)

Or:

echo [^.]*

Bash

Unfortunately bash doesn't have glob qualifiers. So you're left with enabling dotfile inclusion globally. In bash, the syntax is:

shopt -s dotglob

(and use [^.]* for globs without hidden files).

With dotglob, bash doesn't include . nor .. in globs like *, but still does for globs like .*.

If you set the GLOBIGNORE variable to something non-empty, it then automatically enables the dotglob option and excludes . and .. from .* globs but not from dir/.* or .*/file ones (!) so that safeguard is pretty useless. You could do GLOBIGNORE='*/.:*/..:./*:../*:*/./*:*/../*' but then it would break globs like */. or ./* or ../*.

A better work around is to use [.]* or dir/[.]* or [.]*/file (with dotglob enabled) to expand dotfiles except . and ...

fish

fish globs don't include . nor ... When there's no match, depending on the version, it will work like either zsh (or bash -o failglob) or bash -o nullglob.

mv -- * .* /dest/

Would work if there are both hidden and non-hidden files. Otherwise, YMMV and with some versions, it may call mv -- /dest if there's no file at all.

ksh93

No glob qualifier in ksh93 either. You can include dotfiles in globs with:

FIGNORE='@(.|..)'

Contrary to bash's GLOBIGNORE, that's done properly and also fixes the problem of .* including . and ...

yash

yash has a dot-glob option (set -o dot-glob), but contrary to bash, the glob expansions (even of *) include . and .. so it's pretty useless.

tcsh

set globdot

Works like in bash, that is * include dot files except . and .. but .* still includes . and .. (and you can use [.]* to expand hidden files except . and ..).

score 7 · Answer 3 · answered Jun 13 '12 at 03:15

7

I tested this and it solves the issue:

shopt -s dotglob

Output:

~/stackexchangeanswers/40662$ ls -hal dest
total 8.0K
drwxr-xr-x 2 jodiec jodiec 4.0K 2012-06-12 22:15 .
drwxr-xr-x 4 jodiec jodiec 4.0K 2012-06-12 22:15 ..
-rw-r--r-- 1 jodiec jodiec    0 2012-06-12 22:15 1
-rw-r--r-- 1 jodiec jodiec    0 2012-06-12 22:15 .1
-rw-r--r-- 1 jodiec jodiec    0 2012-06-12 22:15 2
-rw-r--r-- 1 jodiec jodiec    0 2012-06-12 22:15 .2
-rw-r--r-- 1 jodiec jodiec    0 2012-06-12 22:15 3
-rw-r--r-- 1 jodiec jodiec    0 2012-06-12 22:15 .3
...snipped....

answered Jun 13 '12 at 03:15

Jodie C

1,879

Sorry but Ulrich answered first. – iconoclast Jun 13 '12 at 03:44
5

We're all on the same team. It's ok. – Jodie C Jun 13 '12 at 04:46

score 4 · Answer 4 · 2016-05-01T06:25:38.907

The easiest way in bash to print matching dot files, ignoring . and .. is:

$ GLOBIGNORE=/+/
$ printf '%s\n' *

To test:

$ cd tmp; mkdir empty; cd empty; touch {,.}{a..h}
$ GLOBIGNORE=/+/
$  ls *
a  .a  b  .b  c  .c  d  .d  e  .e  f  .f  g  .g  h  .h

It printed all files with dots or not, with the notable exception of . and ...

Your example will also work correctly:

$ cd /tmp; mkdir {t,d}est; touch test/{,.}{a..h}
$ mv test/* dest/
$ ls -a test
.  ..
$ ls -a test/*
ls: cannot access test/*: No such file or directory

Empty of files that matter.
The system files . .. still exist but a shell expansion (using *) will not include them.

And the destination directory contains all of the files:

$ cd dest
$ ls -a *
a  .a  b  .b  c  .c  d  .d  e  .e  f  .f  g  .g  h  .h

Why?

This works because setting GLOBIGNORE has this side effects:

Set dotglob (shopt -s dotglob) to math dot files on expansions.
File names . and .. are ignored when GLOBIGNORE is set and not null.

Details in the manual: LESS=+'/The GLOBIGNORE' man bash.

Also, we need to set the variable GLOBIGNORE to contain a name that no filename could match:
an slash (and something else just as a double precaution).

$ GLOBIGNORE=/+/

It may be useful to also set nullglob if it is needed to avoid getting an * when there is no file to match the *.

score 0 · Answer 5 · answered Oct 25 '16 at 17:21

Very interested this setting of GLOBIGNORE=/+/ @user79743

From my experience unsetting dotglob is bad business to almost all your commands (cp, mv etc.) BUT the same is true setting the nullglob (especially with regular expressions that then need escaping!).

Nevertheless, if you do need to set them in the beginning of your program but interfere in specific parts where you call commands like cp, mv etc. or use regular expressions just do this:

  # set dotglob, unset nullglob AFTER this
  is_nullglob=$( shopt -s | egrep -i '.*nullglob' )
  is_dotglob=$( shopt -s | egrep -i '.*dotglob' )
  [[ $is_nullglob ]] && shopt -u nullglob
  [[ ! $is_dotglob ]] && shopt -s dotglob
  # ... call commands ...
  # .....................
  # reset dotglob, nullglob to their previous values
  [[ $is_nullglob ]] && shopt -s nullglob
  [[ ! $is_dotglob ]] && shopt -u dotglob

To check whether a shopt option is set, you'd rather do if shopt -q nullglob; then... — Stéphane Chazelas, Jan 19 '21 at 13:21

What is the setting in bash for globbing, to control whether * matches dot files

5 Answers5

Bash

zsh

Bash

fish

ksh93

yash

tcsh

Why?

Linked

Related