62

I put together a script to do some file operations for me. I am using the wild card operator * to apply functions to all files of a type, but there is one thing I don't get. I can unzip all files in a folder like this

unzip "*".zip

However, to remove all zip files afterward, I need to do 

rm *.zip

That is, it does not want the quotation marks. The unzip, on the other hand, does not work if I just give it the * (gives me a warning that "files were not matched").

Why is this different? To me, this seems like the exact same operation. Or am I using the wild card incorrectly?

Introductions to the wild card in Unix do not really go into this, and I could not locate anything in the rm or zip docs.

I am using the terminal on a Mac (Yosemite).

Stephen Kitt
  • 434,908
patrick
  • 1,022
  • 5
    I had no idea unzip could do this without the normal for f in *.zip;do...done shell loop. Such a weird non-unix-like command line UI. – Peter Cordes May 22 '16 at 18:49
  • @Peter I think you misunderstand the situation. unzip applies the glob to the contents of an archive; you cannot get them from bash with a wildcard. (You'd need ```for f in unzip -l archive.zip; do ... done`) – alexis May 23 '16 at 10:50
  • 1
    @alexis: I knew about unzip accepting globs to match inside a single zip file. But this is different; I actually tried unzip '*.zip' in a directory with multiple zip files, and it extracts all files from all zips. Like I said, super-weird. tar doesn't have any mode of operation like that. – Peter Cordes May 23 '16 at 11:12
  • 2
    @Peter I see... yeah that's weird, especially since unzip will not accept multiple commandline arguments! Clearly a Windows-only implementation. I misinterpreted the OP's description of the task. – alexis May 23 '16 at 11:15
  • 2
    @alexis: PKZip pre-dates Windows. It's a DOS command-line program, first released in 1989. The Unix port uses basically the same cmdline-parsing code, AFAIK. – Peter Cordes May 23 '16 at 11:21

5 Answers5

75

You've explained the situation very well. The final piece to the puzzle is that unzip can handle wildcards itself:

http://www.info-zip.org/mans/unzip.html

ARGUMENTS

file[.zip]

...

Wildcard expressions are similar to those supported in commonly used Unix shells (sh, ksh, csh) and may contain:

* matches a sequence of 0 or more characters

By quoting the * wildcard, you prevented your shell from expanding it, so that unzip sees the wildcard and deals with expanding it according to its own logic.

rm, by contrast, does not support wildcards on its own, so attempting to quote a wildcard will instruct rm to look for a literal asterisk in the filename instead.

The reason that unzip *.zip does not work is that unzip's syntax simply does not allow for multiple zip files; if there are multiple parameters, it expects the 2nd and subsequent ones to be files in the archive:

unzip [-Z] [-cflptTuvz[abjnoqsCDKLMUVWX$/:^]] file[.zip] [file(s) ...] [-x xfile(s) ...] [-d exdir]

Community
  • 1
Jeff Schaller
  • 67,283
  • 35
  • 116
  • 255
  • 6
    thanks, that makes sense! if i understand correctly, in one case I am speaking unzip's own language, in the other case the general unix lingo? – patrick May 21 '16 at 15:17
  • 7
    Correct. It's important to keep in mind what your shell does vs what a program does. – Jeff Schaller May 21 '16 at 15:21
  • 8
    pkzip originated on DOS which did not expand wildcards passed to programs. – Thorbjørn Ravn Andersen May 21 '16 at 20:11
  • 13
    @patrick the unix way of processing multiple files with a program that can only work with one file at a time is to use a loop. e.g. for f in *.zip ; do unzip -v "$f" ; done. and a big part of the reason WHY the shell does the filename expansion etc itself is sot that each individual program doesn't have to (which would result in having lots of independently-written implementations of wildcard expansion that differed in small but annoying ways). – cas May 22 '16 at 00:14
25

The difference between those two commands is the quoted * character. If you call a command in a shell and use the * character for an argument, the shell itself will evaluate the argument. See this example:

$ ls
file1.zip  file2.zip  file3.zip  file4.txt

Now with a *:

$ ls *.zip
file1.zip  file2.zip  file3.zip

The shell evaluates the wildcard and builds a command as follows:

$ ls file1.zip  file2.zip  file3.zip

With a quoted wildcard, it is interpreted as a file named (literally) *.zip:

$ ls "*".zip
ls: cannot access *.zip: No such file or directory

The unzip utility cannot be called with multiple zipped files as arguments. But, the developer chose another way for this. From the manpage:

file[.zip]

[...] Wildcard expressions are similar to those supported in commonly used Unix shells (sh, ksh, csh) [...] (Be sure to quote any character that might otherwise be interpreted or modified by the operating system, particularly under Unix and VMS.)

The Nate
  • 103
  • 3
chaos
  • 48,171
  • Do you know why the authors of unzip chose to go that route, rather than allow for multiple zipped files as arguments? – David Etler May 22 '16 at 21:25
  • @DavidEtler I don't know too. – chaos May 22 '16 at 22:42
  • 1
    I can't say why either, @DavidEtler, but as-built, unzip's syntax accepts filenames after the zipfile that are assumed to be contents of that zipfile. It'd be ambiguous whether you meant for a second zip file to be an "unzip me" parameter or a "unpack this internal zip file from the preceding archive" one. – Jeff Schaller May 23 '16 at 00:30
  • @DavidEtler don't know what the developers were thinking, but everything was much slower and smaller back then. You usually weren't dealing with more than one zip file at a time. You had floppies which held 90 or 250kB and you were really happy to have a 10MB disk drive. Things were compressed because they had to be, not just for intersystem transport. – Joe May 28 '16 at 01:22
  • To really highlight this shell expansion example replace ls with echo – silicontrip Feb 13 '21 at 04:35
8

The difference is in the first case the shell itself expands the glob:

% cd /                                                       
% echo *
Applications Library Network System Users Volumes bin cores ...
%

while in the second case the application itself Does Something™ with that literal character:

% cd /
% perl -E 'chdir "/tmp" or die; say for glob($ARGV[0])' "*"
com.apple.launchd.aj4FEhYqm5
...

If unquoted, the shell first expands out the glob, and the command will be run with whatever that shell glob expanded out to.

thrig
  • 34,938
2

A command will receive the arguments after they have been processed by the shell.

On first processing, an unquoted * will be expanded by the shell (to the list of files in the present directory (pwd) that match the pattern):

echo *.zip

Will list all .zip files. But echo "*".zip" will not.

On first processing, a quoted "*" will not be expanded, it will be given to the command unzip as a parameter (after quoting has been removed). The command unzip will receive a parameter of *.zip:

$ echo unzip "*".zip
unzip *.zip

It is the command unzip which expand the * to the list of files.

It is also interesting that this two commands will not perform the exact same final action, and who expands the * changes:

unzip "*".zip                ### the command unzip expands `*.zip`.
unzip *.zip                  ### the shell expands `*.zip`.

The first command receives a *.zip which it expands to process all files. The second command unzip will receive a list of all .zip files in the pwd, which it will not process, as the unzip developer has chosen to reject the expansion of more than one zip file.

0

The quotes are needed because of the way zip handles multiple arguments:

rm: remove all files in the argument list

zip: unzip the file in the first argument. only extract the files in the remaining arguments.

$ ls *.zip
file1.zip  file2.zip  file3.zip
$ unzip *.zip
Archive:  file1.zip
caution: filename not matched:  file2.zip
caution: filename not matched:  file3.zip

as you can see, it tries to find file2.zip and file3.zip inside file1.zip

in order to allow you to extract multiple zip files at once, zip supports interpreting the glob by itself, with a different result.

GAD3R
  • 66,769
eMBee
  • 532
  • 4
  • 7