hardlink/softlink multiple file to one file

Question

I have many files in a folder. I want to concatenate all these files to a single file. For example cat * > final_file; But this will increase disk space and also will consume time. Is there is a way where I can hardlink/softlink all the files to final_file? For example ln * final_file.

Related: Virtual file made out of smaller ones (for mac-like sparse bundle solution) and How to split a ddrescue disk image and how to use it again? — Stéphane Chazelas, Jul 31 '13 at 20:33
You could use FUSE for this task. I've created a simple example on how to accomplish this: cat-fuse. — Jakub Nowak, Aug 19 '18 at 19:35

Marco · Accepted Answer · 2013-07-31T20:21:32.823

5

With links, I'm afraid, this will not be possible. However, you could use a named pipe. Example:

# create some dummy files
echo alpha >a
echo beta  >b
echo gamma >c

# create named pipe
mkfifo allfiles

# concatenate files into pipe
cat a b c >allfiles

The last call will block until some process reads from the pipe and then exit. For a continuous operation one can use a loop, which waits for a process to read and starts over again.

while true; do
  cat a b c >allfiles
done

edited Jul 31 '13 at 20:21

answered Jul 31 '13 at 20:09

Marco

33,548

Wouldn't cat block until a process starts reading from the named pipe? – Joseph R. Jul 31 '13 at 20:10
@JosephR. Yes, it would. Append & if that's not desired. – Marco Jul 31 '13 at 20:14
1

Great. But wouldn't the OP need to repeat these steps every time he/she wants to access the full content via allfiles? – Joseph R. Jul 31 '13 at 20:15
Indeed. Use while true; do cat a b c >allfiles; done if that's not desired. – Marco Jul 31 '13 at 20:18
Thanks for bearing with me (I'm still grappling with the fifo concept). I think your notes should be added to the answer as the OP might not be aware of that. – Joseph R. Jul 31 '13 at 20:19

score 2 · Answer 2 · answered Jul 31 '13 at 20:05

2

This is not possible.

N files mean N inodes. Hard links, by definition, are simply different names for the same inode. Symlinks are files that point to a certain inode (their target). Either way, soft or hard, the link can refer to a single inode.

answered Jul 31 '13 at 20:05

Joseph R.

39,549

score 2 · Answer 3 · edited Jul 31 '13 at 20:42

2

In a straight way, no ... You cannot hard/soft link to a single file. links are nothing more and nothing less than pointer from one file to another.

Now if you are worried about space and want to release the space you can do the following:

for i in *
do
    cat < "$i" >> destination_file &&
      rm -f -- "$i"
done

Basically, it will append the output to destination_file and remove the file afterwards. Also I'm assuming you don't need the original files.

edited Jul 31 '13 at 20:42

Stéphane Chazelas

544,893

answered Jul 31 '13 at 20:07

BitsOfNix

5,117

1

Why do you parse ls? Just use for i in *. And why the loop in the first place? Just do cat * >> destination. – Marco Jul 31 '13 at 20:10
Why not quote the variable ("$i") to allow for spaces in the file name? – Joseph R. Jul 31 '13 at 20:13
@JosephR. That doesn't help. If you have special characters it'll break. – Marco Jul 31 '13 at 20:16
@Marco But it would at least help with filenames with spaces in them. – Joseph R. Jul 31 '13 at 20:19
@Marco I'm not talking about the ls part. I agree with you that it's unnecessary (and incorrect). I meant to use the double quotes as well as for i in *. – Joseph R. Jul 31 '13 at 20:28
@JosephR. No, it would not. The quoting is applied too late. The string is already split at the spaces and the individual parts assigned to i. – Marco Jul 31 '13 at 20:29
@JosephR. Sorry, I misunderstood you. Totally! cat $i would not work at all, regardless of * or ls. – Marco Jul 31 '13 at 20:31
1

@Marco in the OP he's worried about space. otherwise cat * >> destination was more than enough ... hence the loop to cat and remove the file. – BitsOfNix Jul 31 '13 at 20:36
1

@AlexandreAlves My point is: i) This loop is terrible. ii) It it totally unnecessary. Just use cat * >dest followed by rm !(dest) (bash) or rm ^dest (zsh). – Marco Jul 31 '13 at 20:41
@Marco 1st why is terrible? 2nd, from my point of view you are assuming that he OP has small files. if you have 10 files of 1G each and want to cat in a single one you need extra 10G if you only have 7G available cat * > dest_file will not work due to lack of space. – BitsOfNix Jul 31 '13 at 20:46
@AlexandreAlves For starters, you were trying to parse the output of ls, which you shouldn't. – Joseph R. Jul 31 '13 at 20:49

hardlink/softlink multiple file to one file

3 Answers3