Fundamental metaphor: Directories are not Folders
The fundamental concept to understand is the concept of a directory. This is important: Unix does not have folders, it has directories.
This distinction is important: a folder contains things. A directory lists things. If you think of folders, you have the wrong metaphor stuck in your head. Think of the company directory you find in the lobby of an office building: the directory does not contain the offices, it lists them. Likewise, your phonebook does not contain your friends, it lists them. On the other hand, a (physical) folder actually does contain the (physical) files it holds.
File Serial Numbers
In Unix, files do not have names. Files only have a File Serial Number (FSN). FSNs are unique within a single filesystem: no two files on the same filesystem have the same FSN and single file only has a single FSN over its lifetime. Note, however, that files on different filesystems can have the same FSN.
inodes
Note: you may encounter the term inode. It essentially means the same thing. File Serial Number is a name that was invented for the standardization of Unix, because inodes are actually an implementation detail of some filesystems and many modern filesystems don't even have inodes anymore … but they still need to have FSNs. Also, many modern Unices allow mounting foreign filesystems, which obviously do not have inodes either since that is purely a Unix thing (e.g. mounting a FAT, exFAT, or NTFS filesystem in Linux). The Unix drivers for those foreign filesystems are still required to synthesize FSNs, though, since much of the filesystem APIs are based on FSNs.
Directories
Directories are special files that associate names to FSNs. In very early, very simple Unix filesystems, a directory was quite literally just a file (similar to a text file) that looked something like
12345 foo
54321 hello.txt
34343 My 2021 taxes.xlsx
Modern filesystems are much more sophisticated, of course, but conceptually, that is still how it works today in even the most sophisticated filesystems.
The three fundamental concepts
If you understand these three concepts, you understand everything you need to know about Unix filesystems and Unix permissions:
- Files don't have names, they only have File Serial Numbers.
- Directories are files.
- Directories associate names with FSNs.
Think about how our telephone system works: we only have numbers, but we have directories (phonebooks, Yellow Pages, or in more modern times, a Contact app on our phone) which associate names with those numbers.
(Hard)links
The association between a name and an FSN is called a link or sometimes a hardlink (in contexts where it could be confused with a symlink / softlink).
Consequences of the three fundamental concepts
Once you internalize those three concepts, everything else should fall into place.
It should now be immediately obvious that there is nothing stopping you from creating multiple name→FSN links for the same FSN in different directories, or even in the same directory. In other words, the same file can have multiple names and exist at multiple paths.
It should also be immediately obvious that this trivially leads to hierarchical directory structures: since directories are just files like any other file, they have FSNs just like any other file, and thus, I can list a directory inside a directory.
It is less obvious but important to understand that the rm utility does not actually remove a file! It removes a link, in other words, it removes an entry in a directory. The library function and the kernel system call used by rm is actually called unlink. In fact, you cannot actually remove files in Unix. Files will be automatically removed if there is no link pointing to them (in other words if they have 0 names) and there is no open file handle for the file.
This also explains something that many Windows users get confused about: how is it possible that I can delete a file that is still open in some application? And how does that application not crash? Well, with what we learned in the last paragraph, the answer is simple: they didn't delete the file, they only deleted the name. The file still exists. It could still have other names, but even if this was the last name, the file will not be deleted until the application has closed it. In fact, creating a temporary file, opening it and then immediately "deleting" it again is a common design pattern for Unix applications that need to temporarily store data but don't want to clutter the filesystem with temporary files.
Applying the concepts to permissions
If you understand that directories are files, and that directories associate names with numbers, then the permissions become clear:
- If you want to rename, you need to write the new name to the directory and delete (which is also a form of writing) the old one, ergo you need
write permission to the directory, but no access whatsoever to the file. In fact, renaming doesn't actually have anything to do with the file itself at all, it only concerns the name which is not a property of the file itself.
- If you want to list a directory, you need to read its contents and therefore need
rread permissions on the directory.
- Creating a hardlink is just writing to the directory, ergo you need
write permissions to the directory. And so on.
Concrete examples
With our three core concepts in mind, we should be able to answer all of your questions:
copy
I can copy the /etc/passwd file which is owned by root to my home directory with the following permissions:
-rw-r--r--. 1 root root 2751 Dec 24 21:26 /etc/passwd
I can't do the same with:
-rw-------. 1 root root 43591 Dec 27 18:32 /var/log/messages
So I am guessing that the read permission for others is making the copying possible?
That is correct. Just think about what does "copy a file" actually mean: you are creating a new file (which automatically also means you need to give it a name, i.e. create a link entry in the target directory) which has the same content as the original file.
In order to do that, you need two permissions:
- You need
write permission on the target directory in order to create the hardlink to the newly created file.
- You need
read permission on the original file in order to read its content because if you can't read its content then how can you recreate it?
link
For making a hardlink, I created file1 as root user in my home directory and tried creating a hardlink as a regular user. I wasn't able to.
-rw-r--r--. 1 root root 0 Dec 27 18:39 file1
There is nothing in the standard Unix permission model preventing you from creating the hardlink. All you need is write permission to the directory. The permissions for the file are irrelevant since you don't need to touch it.
I just tested it on my OS (macOS 12.1 "Monterey") on an APFS filesystem, and it works just fine.
fs.protected_hardlinks
What is preventing the creation of hardlink?
Nothing in the Unix permission system does. However, since you are using Linux, there is a Linux-specific configuration option, which may prevent you from doing so: the fs.protected_hardlinks sysctl.
Quoting from the official sysctl documentation [bold emphasis mine]:
protected_hardlinks
[…]
When set to "0", hardlink creation behavior is unrestricted.
When set to "1" hardlinks cannot be created by users if they do not already own the source file, or do not have read/write access to it.
It looks like you have configured it to 1 (restricted). Check /proc/sys/fs/protected_hardlinks to be sure.
rename
The directory permissions where file1 resides are:
drwx------. 18 student student 4096 Dec 27 18:42 .
I am able to rename the file1 to file2 so I am guessing, same as copy, the read permission on others is making that happen?
For a rename, you only need write permission on the directory, which you have. You never touch the content of the file, therefore, its permissions are irrelevant.
move
Lastly, I am not able to move file2 to a different location. Why?
That depends on what, exactly, you mean by "move", and what, exactly, you mean by "different location", and what, exactly you mean by "not able to".
There are two ways how you could interpret "move":
- Delete the original and recreate the same content under a different name.
- Delete the link in the source directory and create a link in the target directory.
If you use the mv utility, it actually could do either of those, so let's look at both of them.
#2 is just a rename: you need write permission to the source directory so that you can delete the link and you need write permission to the target directory so that you can create the link. At no point do you need to touch the file, or access its contents, so the permissions are irrelevant. In fact, mv will use the rename syscall for this.
mv will try to do #2 whenever possible.
So, when is #2 not possible? Well, remember that FSNs are only unique within one filesystem. Also, files only exist within a filesystem.
So, it should be clear that #2 is only possible if the source directory and the target directory are on the same filesystem. If they are on different filesystems, mv will instead copy the file to the target and then unlink the original. Which means, as we discussed above, in addition to write access to the source directory and write access to the target directory, it also needs read access to the original file.
Execute permissions
Note: so far, we have only looked at the read and write permissions, but we have completely ignored the execute permission.
… for regular files
For regular files, the execute permission is fairly obvious: it gives permission to execute the file. (No, really?) However, note that "execute" has a specific meaning here: it means "passing it to the kernel to execute".
In particular, you do not need execute permission to pass a file to a script interpreter. So, if you have a script file called myscript in a fictional language called MyLanguage that looks something like this:
#!/usr/bin/env mylanguage
print "Hello, World!"
Then you do need execute permission if you execute it like this:
/path/to/myscript
but you do not need execute permission if you execute it like this:
mylanguage /path/to/myscript
The language interpreter only needs rread permission so that it can read the content of the script file.
Or, to be more precise, the system does not require you to have execute permission, but the mylanguage interpreter, could, if it wanted to, check if you have execute permission. And in fact, some interpreters do that, and some don't.
… for directories
What does execute permission mean for directories? What is being executed here?
For directories, execute permission is interpreted as "enter" permission or "lookup" permission. In other words, the execute permission allows you to ask the directory for an FSN, given a name.
You can think of the directory as a very simple program which takes a name as a parameter and returns an FSN, then the idea of "execute permission" kind of makes sense. But it is a really strenuous idea. Just ignore the fact that the x stands for execute and think of x meaning "lookup" or "enter" for directories.
Other kinds of special files
Directories are not the only kind of special files there are. There are five others:
- symbolic links,
- pipes,
- sockets,
- character special files, and
- block special files.
Symlinks
Of these five, the ones you encounter most often are symbolic links or symlinks.
A symlink is essentially a file that contains a path. In very early, very simple Unix filesystems, symlinks where quite literally just text files that contained a path. And that's more or less still how you can think of them today.
Once you understand this, it should be clear that in order to create a symlink to somewhere, all you need to be able to do is create the file. You do not need access to the path that you put into the file. In fact, that path does not even have to exist!
For example, assuming you are allowed to create files at all, you can easily do
ln -s https://www.google.com/ google
This is a perfectly valid symlink: google is a perfectly valid name and https://www.google.com/ is a perfectly valid path (it means "the directory named www.google.com inside the directory named https: inside the current directory), but of course it is not pointing to an actually existing file, although you could do
mkdir -p https:/www.google.com
to make it point to somewhere if you wanted to:
cd google
# no error
