I'm working on an embedded application using Raspberry Pi CM4 running DietPi OS. I want to minimize MMC flash writes as much as possible for maximum lifespan. In my installation, there are a few applications frequently writing data throughout the /var directory (not just to the usual offenders of /var/log and /var/tmp). This new data in /var is not mission critical so I want to mount all of /var as an overlayfs and then have a shutdown script to synchronize changes to the disk one time. I have the overlay part working fine. Here is my startup script:
#!/bin/bash
#Startup script to mount /var as an overlay to minimize eMMC writes
if mount | grep "overlay on /var type overlay"; then
echo Mount Point at /var already exists, exiting.
exit 1
else
if [ ! -d "/var.tmpfs" ]
then
mkdir -p /var.tmpfs
fi
mount -t tmpfs -o rw,nosuid,nodev,noexec,relatime,size=256m,mode=0775 tmpfs /var.tmpfs
mkdir -p /var.tmpfs/upper
mkdir -p /var.tmpfs/work
chown -R root:root /var.tmpfs
chmod -R u=rwX,g=rwX,o=rX /var.tmpfs
if [ ! -d "/tmp/realvar" ]
then
mkdir -p /tmp/realvar
fi
mount --bind /var /tmp/realvar
sudo mount -t overlay -o rw,lowerdir=/tmp/realvar,upperdir=/var.tmpfs/upper,workdir=/var.tmpfs/work overlay /var
fi
exit 0
This results in zero MMC writes to anything written to /var while still having use of the data being written there using the overlayfs. The hard part is working well. Now I want to synchronize updates in the upperdir to disk to survive a planned reboot/shutdown event. I realize a power loss event will cost me all the data in upperdir and that is fine.
Synchronization script:
#!/bin/bash
#Shutdown script to sync the contents of the temporary overlay to actual /var on disk
if [ -d "/tmp/realvar" ]
then
rsync -vaz --delete /var/ /tmp/realvar
fi
exit 0
Credit for most of the above goes to LeonidMew and Tobby Fox from this very relevant but older thread: Mount /var/logs as tmpfs, with help of overlayfs to save changes sometimes
Here's the issue: the rsync doesn't work and I cannot figure out why. To test, I create a file in /var/lib after the overlay is established. I then execute my rsync command string which appears to execute as expected. I reboot and my test file is gone. In other words, the contents of the upperdir are not being written to disk the way I want them to be.
An interesting observation: After creating my test file in /var/lib, it exists instantly in /var.tmpfs/upper/lib and in /tmp/realvar/lib also. I do not expect it in /tmp/realvar/lib as this is the "lower" directory of the overlay. Not sure if this is related but it is an odd thing I noticed.
I use the command below to confirm my overlay is actually working by observing all my apps reading/writing to the /var overlay and confirming the result of this command does not change:
cat /sys/fs/ext4/mmcblk0p2/lifetime_write_kbytes
All changes in /var are lost after reboot since my rsync function is not working. With the overlay on /var and the minimalist install I have with DietPi, there are zero MMC writes after boot yet if I change a setting in /etc or change my embedded application in /usr then it gets written instantly and survives a reboot. If I can just get this rsync of the /var overlay to work then I will have a perfect (for me) embedded filesystem configuration.
Any thoughts would be appreciated!!!
