15

I'm doing my first ever moving of user files from an old system to a new system. My goal is to use SCP for it's simple syntax of scp -r source destination. I tried the following command to copy the files first:

scp -r root@10.0.0.11:/home/someuser/* .

In retrospect, and from past experience, this copied all files without a leading .. In my attempt to fix this, I did this:

scp -r root@10.0.0.11:/home/someuser/.* .

meaning wildcard for anything starting with a .. Obviously (why I'm asking the question) it didn't do what I wanted. The observed result, was that it interpreted the . as moving up a level in the path, and it started copying /home/* instead, also (I think) placing the files one level up from my working directory, rather than the working directory itself.

Is my interpretation of the execution of the second command correct? I think it was easy to fix since I was in ~/backup, so one level up was ~. I just rm -rf ~/someuser on each username that had copied before interrupted the command. Those someuser directories were supposed to be in ~/backup

I have since learned how to copy the files I wanted by specifying the directory only, not the files contained in the directory.

muru
  • 72,889
Brian
  • 292
  • Just use scp -r root@10.0.0.11:/home/someuser/. . or ssh root@10.0.0.11 'cd /home/someuser && tar cf - . | gzip -1' | tar zxpf - – Stéphane Chazelas Sep 25 '22 at 16:14
  • 2
    @StéphaneChazelas my main concerning is understanding the command I did first, so I feel confident that I reversed the effects. Can you explain your first syntax? It's similar to what I did, so I want to understand the differences. Not interested in tar and gzip as I'm not worried about speed or archiving at this time. – Brian Sep 25 '22 at 16:33
  • 1
    I understand that you want to use scp but you could also just use rsync -avhH where the h will account for files beginning with . – Nasir Riley Sep 25 '22 at 17:00
  • @NasirRiley yes, I've recently learned that literal scp is deprecated now (and the algorithm by default is sftp or something like that, unless you specify -o for original algorithm or something. I haven't used rsync, so while it doesn't directly answer my question, it is a useful comment. Thanks! – Brian Sep 25 '22 at 18:10

1 Answers1

32

It's not the . that goes up a level, but the fact that in the shells' pattern match syntax, * means any number of any characters, so it matches the .. entry that exists in every directory and points to the next upper directory. You can get the same with something like ls .* in Bash and many other shells.

Having .* match the special entries . and .. is a pain that's never useful. You might see people use ugly workarounds like .??* .[!.] to explicitly navigate around the .. entry. (*)

Luckily some smarter shells remove those two from glob results, zsh probably being the most prominent, but not the only one. (At least the development versions of Bash also have that as the globskipdots option.)

In this particular case, there's the added complication that it's the SSH server that expands the glob but I expect the root cause is the same.

(* .??* matches a dot and at least two other characters, .[!.] matches a dot and one character that's not a dot. In regular expressions, * and ? have slightly different meanings.)

ilkkachu
  • 138,973
  • 2
    Dr. Nick: you mean .* includes .. ? What a country! That is a very clear and concise answer. Maybe I'm also remembering my working directory incorrectly so that might be confusing me to. Either way, I need to understand globbing more. I use it with rdiff-backup, but didn't explore these obvious intricacies. – Brian Sep 25 '22 at 18:09
  • 2
    Among shells, there's also the Forsyth shell and derivatives including pdksh and derivatives and fish. Note that we know the OP doesn't use zsh on the local host as that unquoted * would have resulted in an error unless there was a root@10.0.0.11:/home/someuser directory and files in it. – Stéphane Chazelas Sep 25 '22 at 18:15
  • @StéphaneChazelas very good point! I'm still new to details Unix command crafting and syntax. I was using bash, and I'll be sure to point to my shell in the future when dealing with command interpretation! – Brian Sep 25 '22 at 18:35
  • 2
    @Brian, in bash as well, you'd want to make sure that * is quoted as you do not want that * to be interpreted as a glob operator by the local shell, but be passed asis to scp so that the remove scp/sftp end interprets that wildcard – Stéphane Chazelas Sep 25 '22 at 18:39
  • 1
    @Brian for future reference, .* in glob (not regex) context, means "anything starting with a . and 0 or more other characters, so it matches both . and .., yes. – terdon Sep 26 '22 at 08:28
  • 1
    It may be helpful to note that, in the vast majority of cases, .??* will glob all dot-files and folders without getting . or .. - it'll get any dot-files/folders with 2 or more characters in their names (eg., it'll miss .a). That said, it's almost always better with stuff like this to pull the folder itself rather than try to glob the contents of the folder. – minnmass Sep 26 '22 at 13:31
  • @minnmass you're right about folder vs folder/*, and that's been my process since discovering this. I feel so noob, but everyone has to start somewhere. Thanks! – Brian Sep 28 '22 at 12:14