We shouldn't forget that the essence of the task is indeed quite simple; as put in a tutorial on Haskell (which is written around the working through of the solution for this task, incrementally refined)
Now let's think for a moment about how
our program will operate and express
it in pseudocode:
main = Read list of directories and their sizes.
Decide how to fit them on CD-Rs.
Print solution.
Sounds reasonable? I thought so.
Let's simplify our life a little and
assume for now that we will compute
directory sizes somewhere outside our
program (for example, with "du -sb *")
and read this information from stdin.
(from Hitchhikers guide to Haskell, Chapter 1)
(Additionaly, in your question, you'd like to be able to tweak (edit) the resulting disk layouts, and then use a tool to burn them.)
You could re-use (adapt and re-use) a simple variant of the program from that Haskell tutorial for splitting your file collection.
Unfortunately, in the distribute tool that I've mentioned here in another answer, the simplicity of the essential splitting task is not matched by the complexity and bloatedness of the user interface of distribute (because it was written to combine several tasks; although performed in stages, but still combined not in the cleanest way I could think of now).
To help you make some use of its code, here's an excerpt from the bash-code of distribute (at line 380) that serves to do this "essential" task of splitting a collection of files:
# Splitting:
function splitMirrorDir() {
if [[ ! -d "$THIS_BASES_DIR/$BASE/$type" ]]; then
echo $"No base fixed for $type" >&2
exit 1
fi
# Getting the list of all suitable files:
local -a allFiles
let 'no = 0' ||:
allFiles=()
# no points to the next free position in allFiles
# allFiles contains the constructed list
for p in "$THIS_BASES_DIR/$BASE/$type"/*.rpm; do
if [[ ! -e "$p" ]]; then
# fail on non-existent files
echo $"Package file doesn't exist: " "$p" >&2
return 1
fi
if [[ "$ONLY_REAL_FILES" == "yes" && ! -f "$p" ]]; then
continue
fi
if [[ "$DIFF_TO_BASE" ]]; then
older_copy="$DIFF_TO_BASE/$type/${p##*/}" # using shell param expansion instead of `basename' to speed up
if [[ -h "$older_copy" || -a "$older_copy" ]]; then
continue
fi
fi
allFiles[$(( no++ ))]="$p"
done
readonly -a allFiles
# Splitting the list of all files into future disks:
#
local -a filesToEat allSizes
let 'no = 0' ||:
filesToEat=()
allSizes=($(getSize "${allFiles[@]}"))
readonly -a allSizes
# allSizes contains the sizes corrsponding to allFiles
# filesToEat hold the constructed list of files to put on the current disk
# no points to the next free position in filesToEat
# totalSize should hold the sum of the sizes
# of the files already put into filesToEat;
# it is set and reset externally.
for p in "${allFiles[@]}"; do
if (( totalsize + ${allSizes[$(( no ))]} > CDVOLUME )); then
eatFiles "${filesToEat[@]}"
filesToEat=()
finishCD
startTypedCD
fi
let "totalsize += ${allSizes[$(( no ))]}" ||:
filesToEat[$(( no++ ))]="$p"
done
eatFiles "${filesToEat[@]}"
}
function eatFiles() {
#{ oldIFS="$IFS"; IFS=$'\n'; echo "$FUNCNAME: args: " "$*" | head >&2; IFS="$oldIFS"; }
zeroDelimited "$@" | xargs -0 --no-run-if-empty \
cp -s \
--target-dir="$THIS_LAYOUTS_DIR/cd$(( cdN ))/$PREFIX/$type$DOT_SUFFIX"/ \
--
}
function startTypedCD() {
# set -x
mkdir -p "$THIS_LAYOUTS_DIR/cd$(( cdN ))/$PREFIX/$type$DOT_SUFFIX"
start_action $" %s with %s" "$(( cdN ))" "$type"
# set +x
}
function finishCD() {
(read more after line 454)
Note that the eatFiles function prepares the layouts of the future disks as trees where the leaves are symlinks to the real files. So, it is meeting your requirement that you should be able to edit the layouts before burning. The mkisofs utility has an option to follow symlinks, which is indeed employed in the code of my mkiso function.
The presented script (which you can take and rewrite to your needs, of course!) follows the simplest idea: to sum the sizes of files (or, more precisely, packages in the case of distribute) just in the order they were listed, don't do any rearrangements.
The "Hitchhikers guide to Haskell" takes the optimization problem more seriously and suggests program variants that would try to re-arrange the files smartly, in order for them to fit better on disks (and require less disks):
Enough preliminaries already. let's go
pack some CDs.
As you might already have recognized,
our problem is a classical one. It is
called a "knapsack problem"
(google it up, if you don't know
already what it is. There are more
than 100000 links).
let's start from the greedy solution...
(read more in Chapter 3 and further.)
Other smart tools
I've been told also that Debian uses a tool to make its distro CDs that is smarter than my distribute w.r.t. collections of packages: its results are nicer because it cares about inter-package dependencies and would try to make the collection of packages that gets on the first disk closed under dependencies, i.e., no package from the 1st disk should require a package from another disk (or at least, I'd say, the number of such dependencies should be minimized).
par2protection. – Aaron D. Marasco Feb 20 '12 at 16:22