How to properly use tail to concatenate all hidden files

Question

Issue

I want to be able to :

concatenate all files in a directory (regular and hidden),
but I would also like to display the title of each file at the beginning of each concatenation.

I found some solutions on the web, where I can do #2 with tail -n +1 * 2>/dev/null super neat trick, but it doesn't include hidden files like if I were to do: cat * .* 2>/dev/null or even head * .* 2>/dev/null

The cat command will do the trick but doesn't include the filename, and the head command will not print/concatenate the whole contents of each file.

Question

Is there a way to do what I need to do with tail, if not what is a good substitute to achieve the same result/output.

Update with an example

The tail command when attempting to concatenate all files (regular and hidden)


[kevin@PC-Fedora tmp]$ ls -la
total 8
drwx------   2 user user 4096 Jun 23 09:24 .
drwxr-xr-x. 54 user user 4096 Jun 23 08:21 ..
-rw-rw-r--   1 user user    0 Jun 23 09:24 .f1
-rw-rw-r--   1 user user    0 Jun 23 09:24 f1
-rw-rw-r--   1 user user    0 Jun 23 09:24 .f2
-rw-rw-r--   1 user user    0 Jun 23 09:24 f2
-rw-rw-r--   1 user user    0 Jun 23 09:24 .f3
-rw-rw-r--   1 user user    0 Jun 23 09:24 f3
-rw-rw-r--   1 user user    0 Jun 23 09:24 .f4
-rw-rw-r--   1 user user    0 Jun 23 09:24 f4
-rw-rw-r--   1 user user    0 Jun 23 09:24 f5
[user@PC-Fedora tmp]$ tail -n +1 *
==> f1 <==
==> f2 <==
==> f3 <==
==> f4 <==
==> f5 <==
[user@PC-Fedora tmp]$ tail -n +1 * .*
==> f1 <==
==> f2 <==
==> f3 <==
==> f4 <==
==> f5 <==
==> . <==
tail: error reading '.': Is a directory
[user@PC-Fedora tmp]$

Is there a problem with tail -n +1 * .*? You've tried that with both head and cat, why not tail? Is it that you sometimes only have one file so you don't get the file name? — terdon, Jun 23 '21 at 16:23
Yes, good find @StephenKitt. Rennitbaby, you can just do tail -n +1 * .[^.]*, which is basically explained in the suggested duplicate. If that works for you, please accept the duplicate suggestion. — terdon, Jun 23 '21 at 16:32
That solved the issue, beautiful find @StephenKitt. And thank you @terdon for the help! I think from the look of it head and tail process arguments is different, hence .[^.]* regex is needed to match hidden files. — 0x5929, Jun 23 '21 at 16:48

Stéphane Chazelas · Accepted Answer · 2021-06-24T05:51:59.363

With zsh and GNU tail (not all tail implementations can take more than one filename arguments, and not all that do will display the file names):

() { (($# == 0)) || tail -vn +1 -- "$@" < /dev/null; } *(ND)

-v is to still print the filenames even if there's only one file, D for dotglob, N for nullglob, using an anonymous function which is passed the expansion of that glob and checks for the special case of the current directory being empty.

</dev/null is to partly mitigate the fact that GNU tail, when passed a filename called -, will read stdin instead. Here, we just prevent it from reading stdin, but it still won't read the file called -. Another approach would be to use "${@/#%-/./-}" in place of "$@" to replace - with ./-, but that would also mean that you'd see ==> ./- <== instead of ==> - <== for the - file (still probably better than ==> standard input <==).

You may also want to add the . (or -.) glob qualifier to restrict to regular files only to avoid the errors or worse if there are directories or other types of non-regular files in the current directory (*(ND-.)).

Same with ksh93 and GNU tail:

(
  FIGNORE='@(.|..)'
  set -- ~(N)*
  (($# == 0)) || tail -vn +1 -- "$@" < /dev/null
)

Same with bash and GNU tail:

(
  setopt -s nullglob dotglob
  set -- *
  (($# == 0)) || tail -vn +1 -- "$@" < /dev/null
)

Or with GNU tail and any POSIX sh (including zsh, but only in sh emulation), also restricting to regular or symlink to regular files and replacing - with ./-, but potentially processing the files in a different order as we're doing .* ones before the other ones:

(
  set --
  for file in .* *
    [ -f "$file" ] || continue
    [ "$file" = - ] && file=./-
    set -- "$@" "$file"
  done
  [ "$#" -eq 0 ] || exec tail -n +1 -- "$@"
)

Or you could use GNU awk (here with zsh):

() {
  (( $# == 0 )) ||
    gawk 'BEGINFILE{
            print sep "==> "substr(FILENAME, 3)" <=="
            sep = "\n"
          }
          {print}' "$@"
} *(ND-.)

awk has the same problem as tail with - and also with filenames that contain =. We work around both by adding that ./ prefix which we strip upon display. Note that awk adds a newline character if missing at the end of the files.

Or use a loop:

sep=
for f (*(ND-.)) {
  print -r "$sep==> $f <=="
  sep=$'\n'
  cat < $f
}

(cat has the same problem with -, which we work around by passing the file on stdin instead of arguments)

How to properly use tail to concatenate all hidden files

1 Answers1