1

I have a Xen PV node running RHEL6 on the 3.10 kernel. This node has mdadm RAID1 and the following partitions:

[root@node ~]# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/md1         15G  4.1G  9.9G  30% /
tmpfs           966M     0  966M   0% /dev/shm
/dev/md0        477M   84M  365M  19% /boot
xenstore        966M  184K  966M   1% /var/lib/xenstored

The node also has an LVM that occupies the remaining portion of the SSDs less 20GB that I've left unallocated for SSD over-provisioning.

I've noticed some interesting behavior recently when I execute fstrim from the dom0. fstrim will continue to trim bytes even though there's been no significant amount of disk activity occurring in the dom0 (with the exception of nightly backups). There's tons of disk activity occuring in the domUs, but almost none in dom0.

Each of these fstrim commands below were separated by a ~2 minute delay:

[root@node ~]# fstrim -v /
/: 10452664320 bytes were trimmed
[root@node ~]# fstrim -v /
/boot: 0 bytes were trimmed
[root@node ~]# fstrim -v /
/: 181784576 bytes were trimmed
[root@node ~]# fstrim -v /
/boot: 0 bytes were trimmed
[root@node ~]# fstrim -v /
/: 283222016 bytes were trimmed
[root@node ~]# fstrim -v /
/boot: 0 bytes were trimmed
[root@node ~]# fstrim -v /
/: 334065664 bytes were trimmed

The first fstrim was the first executed for the day and, as expected, it found about 10GB worth of pages to trim -- correlates to nightly backups that were taken. If I run the command a couple minutes later, there's nothing to trim. But if I wait another couple minutes and trim again, then it will always find something to trim -- not an insignificant amount of data either (e.g., ~300MB).

As I mentioned before, the dom0 has very little disk activity outside of the nightly backup run. The domUs have tons of disk activity, but I wouldn't have thought the dom0 could trim pages associated with a domU filesystem.

In either case, when I execute fstrim on dom0, the performance on the domUs will noticeably improve -- iowait falls.

Does anyone know how it is possible that dom0 could continually trim hundreds of MB when there's very little disk activity on dom0?

Elliot B.
  • 203
  • 1
    Don't put too much weight on those sizes reported by fstrim. It's down to filesystem implementation. See also https://unix.stackexchange.com/a/371665/30851 – frostschutz Sep 19 '17 at 16:03
  • @frostschutz Thanks for the tip. I just read over your answer in the other post -- you describe that ext4 tries to avoid trimming the same things over and over, so you will see 0 bytes trimmed. Since my filesystem is ext4, shouldn't it report 0 bytes trimmed over and over? Seems odd that the number bytes trimmed has a direct correlation with how long I want before re-running the fstrim command if it indeed is not trimming any additional bytes. – Elliot B. Sep 19 '17 at 16:11
  • That's if there are no writes. Once you make a small change, it will trim a large area (but still smaller than free space). Also it might not remember across reboots. I haven't looked at the implementation in detail, I guess it's just not that fine grained. – frostschutz Sep 19 '17 at 16:22
  • @frostschutz I'd be happy to accept your answer to this question if you wrote one up -- even if it's just a pointer to your previous answer. – Elliot B. Sep 20 '17 at 19:38

0 Answers0