-1

I don't know anything about Unix/Linux. I am primarily a web programmer.

I am running the following command in my code:

exec("tar -caf '/backups/_structure.tar.gz' --ignore-failed-read -C /source .");

This code makes a file backup of all the contents of my /source directory and stores it in the /backup directory. This code works fine, except for the fact that sometimes my files are HUGE.

I would like to split the tar file by 100MB segments. In this single line, what would I add to do that? I've read some things about | split -b 100MB, but I don't know where in the line to put it and I'm very confused.

Jason Axelrod
  • 99
  • 1
    https://unix.stackexchange.com/questions/61774/create-a-tar-archive-split-into-blocks-of-a-maximum-size – marc Jul 23 '20 at 15:45

2 Answers2

2
tar -cf - --ignore-failed-read -C /source . | gzip |
  split -b 100M - /backups/_structure.tar.gz.

The whole point of tar is that it sends a stream, that we compress on the fly, split on the fly, send over the network, etc with pipelines without ever having to store temporary files.

-f - tells tar to send the archive on stdout, which here is connected via a pipe to gzip, which compresses it on the fly and sends it to split.

- tells split to take its input from stdin. The second argument is the prefix for the chunks. See the man page of split for how to change the naming scheme for those chunks if you don't like the default.

Same thing, to extract that archive, no need to store the full tar.gz file, just use:

cat _structure.tar.gz.* | gunzip | (cd /dest && tar xf -)
Stéphane Chazelas
  • 544,893
1

First, create the tar file:

tar -caf '/backups/_structure.tar.gz' --ignore-failed-read -C /source . Then, split the tar file in 100Mbs

split -b 100M /backups/_structure.tar.gz

This will create a file with weird names, like xaa, xab...

To have the original file again, just cat all those files:

cat xaa xab xac... > _structure.tar.gz