Why is filesystem time always some msecs behind system time in Linux?

Question

In Linux, it seems that filesystem time is always some milliseconds behind system time, leading to inconsistencies if you want to check if a file has been modified before or after a given time in very narrow time ranges (milliseconds).

In any Linux system with a filesystem that supports nanosecond resolution (I tried with ext4 with 256-byte inodes and ZFS), if you try to do something like:

date +%H:%M:%S.%N; echo "hello" > test1; stat -c %y test1 | cut -d" " -f 2

the second output value (file modification time) is always some milliseconds behind the first one (system time), e.g.:

17:26:42.400823099
17:26:42.395348462

while it should be the other way around, since the file test1 is modified after calling the date command.

You can get the same result in python:

import os, time
def test():
    print(time.time())
    with open("test1", "w") as f:
        f.write("hello")
        print(os.stat("test1").st_mtime)
test()

1698255477.3125281
1698255477.3070245

Why is it so, and is there a way to avoid it, so that system time is consistent with filesystem time? The only workaround I found so far is to get filesystem "time" (whatever that means in practice) by creating a dummy temporary file and getting its modification time, like this:

def get_filesystem_time():
    """
    get the current filesystem time by creating a temporary file and getting
    its modification time.
    """
    with tempfile.NamedTemporaryFile() as f:
        return os.stat(f.name).st_mtime

but I wonder if there is a cleaner solution.

Answer 1

The time used for file timestamps is the time at the last timer tick, which is always slightly in the past. The current_time function in inode.c calls ktime_get_coarse_real_ts64:

/**
 * current_time - Return FS time
 * @inode: inode.
 *
 * Return the current time truncated to the time granularity supported by
 * the fs.
 *
 * Note that inode and inode->sb cannot be NULL.
 * Otherwise, the function warns and returns time without truncation.
 */
struct timespec64 current_time(struct inode *inode)
{
    struct timespec64 now;
ktime_get_coarse_real_ts64(&now);

if (unlikely(!inode->i_sb)) {
    WARN(1, "current_time() called with uninitialized super_block in the inode");
    return now;
}

return timestamp_truncate(now, inode);

}

and the latter is part of a family of functions documented as follows:

These are quicker than the non-coarse versions, but less accurate, corresponding to CLOCK_MONOTONIC_COARSE and CLOCK_REALTIME_COARSE in user space, along with the equivalent boottime/tai/raw timebase not available in user space.

The time returned here corresponds to the last timer tick, which may be as much as 10ms in the past (for CONFIG_HZ=100), same as reading the 'jiffies' variable. These [functions] are only useful when called in a fast path and one still expects better than second accuracy, but can't easily use 'jiffies', e.g. for inode timestamps. Skipping the hardware clock access saves around 100 CPU cycles on most modern machines with a reliable cycle counter, but up to several microseconds on older hardware with an external clocksource.

Note the specific mention of inode timestamps.

I’m not aware of any way of avoiding this entirely, short of modifying the kernel. You can reduce the impact by increasing CONFIG_HZ. There has been a recent proposal to improve this, which is still being worked on.

Answer 2

Stephen Kitt's answer seems to be spot-on.

We can reproduce this very nicely by actually getting the same "coarse" clock that the filesystem uses, at least on my kernel configuration; a C program that always gets the coarse realtime clock before accessing the file takes either exactly the timestamp of the file, or (rarely) a timestamp one system tick earlier:

// excerpt from the program linked above, not a relicensing
// …
    clock_gettime(CLOCK_REALTIME_COARSE, &now_coarse);
    clock_gettime(CLOCK_REALTIME, &now);
    int fd = open("temp", O_WRONLY | O_CREAT);
    write(fd, data, length);
    close(fd);
    clock_gettime(CLOCK_REALTIME, &now_after);
stat("temp", &props);

printf("Differences relative to coarse clock before:\n"
       "Fine Realtime before:   %+8jd ns\n"
       "File Modification Time: %+8jd ns\n"
       "Realtime clock after:   %+8jd ns\n",
       ns_difference(&now_coarse, &now),
       ns_difference(&now_coarse, &props.st_mtim),
       ns_difference(&now_coarse, &now_after));

yields something like

Differences relative to coarse clock before:
Fine Realtime before:   +1551810 ns
File Modification Time:       +0 ns
Realtime clock after:   +1626199 ns

with the aforementioned rare occurrence of a circa tick-duration delta, which happens when the system tick falls just between the getting of the now_coarse and the modification of the file:

Differences relative to coarse clock before:
Fine Realtime before:   +1497562 ns
File Modification Time:  +999992 ns
Realtime clock after:   +1609943 ns

By the way, statistics show that if we do the above until we collected 10,000 occurrences where this "tick jump" happened, that the range of possible delays is quite small:

Total processed:     777618
Observed tick progressions:      10000
Percentage:                 0.013
Minimum tick delta:     999992
Maximum tick delta:     999993
Average tick delta: 999992.905900

In other words, we have extremely close timing when it comes to the deltas between ticks.