I have a list of filenames that I want to create soft links to (call it filenames.txt). When using
cat filenames.txt | awk 'system("ln -s "$0" ~/directory\ with\ spaces/subdirectory/")`
it seems that it chokes up when encountering anything (either a filename or the destination directory) with a space.
How do I create these soft links without having to first rename everything to ensure there are no spaces?
NOTE: All examples presented here are to be run from the directory where files and the list of filenames are located. For instance, if they are in /mnt/Hard\ Drive/some\ files/ folder, please ensure the filenames.txt is stored there as well. When executing the commands from shell, please cd /mnt/Hard\ Drive/some\ files/ and then run the examples.
AWK's system call can be used with items that contains spaces, but it's slightly complicated. Your original code had a few syntax errors and useless cat there. Regardless of that fact, proper way would be for command to be build with sprintf() into variable first, and pass that variable to system(). Like so:
$ ls dir\ with\ spaces/
$ awk '{cmd=sprintf("fpath=$(realpath -e \"%s\" );%s %s \"$fpath\" \"%s\"",$0,"ln","-s","dir with spaces/");system(cmd) }' filenames.txt
$ ls dir\ with\ spaces/
ascii.txt@ disksinfo.txt@ input.txt@ process info.txt@
The path of lesser resistance, would be via small shell script given below:
#!/bin/bash
# uncomment set -x for debugging
#set -x
# Preferably, the directory should be full path
dir="dir with spaces/"
while IFS= read -r line
do
if [ "x$line" != "x" ] && [ -e "$line" ];
then
fpath=$( realpath -e "$line" )
ln -s "$fpath" "$dir"
fi
done < filenames.txt
Script in action:
$ ls -l dir\ with\ spaces/
total 0
$ cat filenames.txt
input.txt
ascii.txt
disksinfo.txt
process info.txt
$ ./makelinks.sh
$ ls -l dir\ with\ spaces/
total 0
lrwxrwxrwx 1 xieerqi xieerqi 9 1月 15 00:47 ascii.txt -> ascii.txt
lrwxrwxrwx 1 xieerqi xieerqi 13 1月 15 00:47 disksinfo.txt -> disksinfo.txt
lrwxrwxrwx 1 xieerqi xieerqi 9 1月 15 00:47 input.txt -> input.txt
lrwxrwxrwx 1 xieerqi xieerqi 16 1月 15 00:47 process info.txt -> process info.txt
The way it works is simple: while IFS= read -r line; do . . . done < filenames.txt reads filenames.txt line by line, each time line going into line variable. We check if that line isn't empty and if the file exists ( this script is to be executed from the same directory where original files and the file-list are located ). If both conditions are true, we make links.
Note that you should ensure the file ends with a newline - if there's a last line that doesn't end with newline character ( which does happen ) , that final line will be skipped, thus there won't be any link made for that file.
Although not ideal and not without quirks, this is a working solution for cases where awk's system() is not available ( which is probably rare nowadays ).
A slightly simpler shell approach would be via xargs and bash -c '' sh , where we read filenames.txt , again, line by line with -L1 flag, and use each line as arguments to bash. Using $@ we take whole array of command-line arguments ( $1,$2,$3 . . . etc.) and turn in into string variable, which is then passed on to ln. The sh at the end is to set $0 variable. It's arguable whether it's necessary to do, but that's not the topic of this question, hence let's skip it:)
$ xargs -I {} -L1 bash -c 'file="$@";fpath=$(realpath -e "$file"); ln -s "$fpath" ./dir\ with\ spaces/"$file" ' sh < filenames.txt
$ ls dir\ with\ spaces/
ascii.txt@ disksinfo.txt@ input.txt@ process info.txt@
$ ls dir\ with\ spaces/
$ python -c "import os,sys;fl=[f.strip() for f in sys.stdin];map(lambda x: os.symlink(os.path.realpath(x),'./dir with spaces/' + x),fl)" < filenames.txt
$ ls dir\ with\ spaces/
ascii.txt@ disksinfo.txt@ input.txt@ process info.txt@
The way this works is simple - we redirect filenames.txt into python's stdin, read in each line into list fl, and then run os.symlink() for each item in list. Lengthy one-liner, but works.
A shorter version is achieved via Perl:
$ perl -lane 'use Cwd "realpath";symlink(realpath($_),"./dir with spaces/" . $_)' < filenames.txt
$ ls dir\ with\ spaces/
ascii.txt@ disksinfo.txt@ input.txt@ process info.txt@
$sourcedir variable like "$sourcedir" ? I'll look into other solutions, will see if I can fix those up. I'm using Ubuntu Linux, so I'm on different system, but in theory that should still work
– Sergiy Kolodyazhnyy
Jan 16 '17 at 04:59
ln needs to know full path of the original file. I've added os.path.realpath() function call to python, tested it out - it resolves properly. Without it, it makes symlink to itself. I'll see what I can do about perl one
– Sergiy Kolodyazhnyy
Jan 16 '17 at 05:03
For this kind of task, don't use a scripting language which calls an external command with system() or a similar function. It presents the same dangers and pitfalls as Bourne Shell's eval.
xargs allows you to do it safely and efficiently:
xargs -d '\n' ln -s -t ~/directory\ with\ spaces/subdirectory/ < filenames.txt
What it does is build one command by adding the content of filenames.txt at the end of the command you have given as parameter. Then it executes it. With the example above, the command it would build would look like:
ln -s -t directory file1 file2 ... fileN
What's interesting is that you don't have to quote anything in filenames.txt, and you don't have to use nested quoting for the directory.
xargs isn't that "safe": only use plain xargs if you your file names don't contain whitespace or \'" otherwise see How to make xargs handle spaces and special chars from cat?
– don_crissti
Jan 15 '17 at 20:52
In the string literal, use a double backslash to get a single backslash in the string that is passed to the shell, which causes the shell to interpret the next character literally:
awk 'system("… ~/directory\\ with\\ spaces/subdirectory/")'
To quote the input for use in a shell snippet, one method is to add a backslash before every character. Portable awk doesn't make this easy (GNU awk has extensions that do). Alternatively, put a single quote around each element, and replace all single quotes in the element by '\''. Awk makes this solution easy: just use gsub. If your awk snippet is in a shell script, you can use a backslash-octal escape to put a single quote in an awk string literal.
<filenames.txt awk '{
gsub(/\047/, /\047\\\047\047/, $0);
system("ln -s \047" $0 "\047 ~/directory\\ with\\ spaces/subdirectory/");
}'
filenames.txtonly filenames or full paths ? – Sergiy Kolodyazhnyy Jan 15 '17 at 07:30awkfor you. There was some trickery that's needed to be done viasprintf()– Sergiy Kolodyazhnyy Jan 15 '17 at 10:18