Kopia backups of array that is not always mounted

Support
1

Hi,
I’m testing Kopia and other backup solutions before committing to a particular software (or actually, probably multiple solutions in parallel with different repository backends).

So far, I’ve got Kopia working successfully for manual snapshots to a repository hosted in a GCE nearline bucket. Manual use seems straightforward and the documentation is clear (good work by the devs). However, I now need to automate backups, and here I’m starting to struggle a bit.

Use case:
We are an small knowledge based business with less than 5 people. We need to backup:
(1) a ZFS pool that is mounted on our server, and,
(2) an encrypted physical RAID array which is manually mounted when that data is needed (i.e. mounted/unmounted on demand).

I think use (1) should be is easy enough, there is good documentation on creating a point in time snapshot of a ZFS filesystem in the advanced topics ‘actions’ documentation. I think I will be able to create the relevant shell scripts and a systemd unit file to get this all working automatically after boot (well, after the local-fs.target).

The data on (2) is sensitive and therefore only mounted when needed, and this requires a passphrase (which is not stored on the server anywhere) to be manually entered. However, this data is also important to our business and needs to be backed up.

From what I understand, if Kopia tries to make a backup of (2) when the array is not mounted, it will create an empty backup (presumably then deduplicated with all other empty backups in the repository). This then leads to issues with empty backups being kept in preference to older backups made when the array was actually mounted. I wonder if enabling --ignore-identical-snapshots would prevent this?

If not, is there a way to configure Kopia to skip empty backups or detect when the array is not mounted? Could I achieve the desired behaviour using –enable-actions on the repository and then write a –before-folder-action script that checks if (2) is mounted and fails if it is not, which I understand would lead to the backup being aborted if action-command-mode=essential is set? This seems like it should prevent empty backups being created, but result in normal behaviour if the check script exit returns 0.

Alternatively, I could set up a cron job script that checks if the array is mounted, and makes a manual snapshot if it is. In this case, would I have to manage flags/maintenance of the repository manually?

Any thoughts, tips or alternatives to this approach would be appreciated before I head down this particular rabbit hole?

2

Yeap - this is the most straightforward way I use myself.

3

Excellent, thanks for confirming. The before/after actions make Kopia a pretty powerful tool.

I’m mildly surprised that systemd unit files are not distributed with Kopia, as I imagine many people would want to run it as a service started at boot time. However, as I say, writing unit files is not that challenging and there are some examples on the web. I might turn my hand to some documentation of this if I’m successful though.

4

I think most people turn into kopia because of GUI and do not even think about CLI.

Sure some systemd examples would be useful for Linux users. But to make it complete it would be nice to include launchd (macOS) and whatever Windows is using:)

PS. I use actions for both managing local file system snapshots and various conditions I need to make sure are met to run backup.

5

Agreed, running Kopia in a server environment is an area that is less well documented across any of the major OS.
We are also testing restic, which also runs as a normal process rather than daemon (service) by default. It will be interesting to see which is easier to set up to fully meet our needs, though I think we will end up using both (with different cloud storage providers) so that we are covered if one project or the other dies (or more likely, if I manage to accidentally configure one service to wipe out its repo in a fit of incompetence).
Anyway, thanks for confirming how you have implemented a similar use case, it is reassuring to know it can be done and I’m not heading down a blind alley.

6

hahahaha. funnily I do the same (for critical data). After long time playing with both I know that none is perfect. They both have their shortcomings.

1 Like