4

I have a home server, which - between other things - stores my documents, mail archives etc.. It uses a ZFS RAIDZ, based on spinning hdds. The server also runs an off-line IMAP service, synchronizing my mail in 5 minutes intervals.

The problem is, that frequently I do not need to access mail (e.g. during the night, or when I am away on weekends). Still the disks keep spinning, making noise and wasting electric power, and the only process that wakes them up is this mail checking. I can disable the sync process for the night, but I am looking for a more universal solution.

I thought about putting the mail folders on an SSD, which holds the system anyway.

However, I do want my mail to be safe. Storing it purely on an SSD while a safe RAIDZ storage is at hand is something I would not call wise. So, my idea is to synchronize the SSD mail storage with the RAIDZ array once or twice per day - I think I need something like a very delayed write cache.

I thought about calling rsync between SSD and RAIDZ, but this has a drawback: if an old mail gets corrupted on the SSD, the error will propagate to the array. Perhaps rsync --ignore-existing will do the job, but maybe there is something that could work better?

Braiam
  • 35,991
Paweł Rumian
  • 1,656
  • "if an old mail gets corrupted on the SSD, the error will propagate to the array" Are you saying you aren't running ZFS on the SSD? – user Mar 31 '14 at 11:54
  • @MichaelKjörling - yes, indeed :) So, using ZFS on SSD guarantees that a rotten bit won't pass unseen? – Paweł Rumian Mar 31 '14 at 15:09
  • ZFS doesn't care about the type of storage media, so you get the same sort of guarantees that you'd get using a traditional HDD in the same manner. – user Mar 31 '14 at 18:35

1 Answers1

2

I thought about calling rsync between SSD and RAIDZ, but this has a drawback: if an old mail gets corrupted on the SSD, the error will propagate to the array. Perhaps rsync --ignore-existing will do the job, but maybe there is something that could work better?

There should be no need to go through such hoops.

Just use ZFS to make a second pool backed by SSD storage.

  • You will get the same data integrity guarantees as on your RAIDZ "archival" array
  • You use the same management commands (zfs, zpool) to manage the storage
  • You can add redundancy to your SSD storage, either through the copies attribute on the file system or through redundant hardware (mirroring or RAIDZ), to allow ZFS to repair (rather than just detect) any possible degredation, within the limitations of your chosen level of redundancy

Using ZFS on both sides also has the advantage that you can zfs send | zfs receive to copy datasets between the two locations. If you want to, you can even keep "active" email in a file system separate from "archival" email, which would allow you fast and low-power access to active data while keeping archival data on slower but much cheaper per GB storage. I do something very similar myself and it's working like a charm.

Separating classes of storage (SSDs and HDDs, and possibly with different levels of redundancy) is a good use for multiple pools on a single system, as it allows you to in each specific instance pick storage characteristics appropriate for the type of data you are working with and its access patterns. Since any ZFS dataset can be made accessible at any mountpoint, switching back and forth between various storage characteristics available through different pools can be made almost seamless.

ZFS itself doesn't really care much what sort of storage media you are using to hold the data, so with a similar setup you get similar capabilities (but very different performance characteristics) with SSDs as you do with HDDs.

user
  • 28,901