I’d like to schedule a regular job to verify that my backups are actually happening, and notify me if they aren’t. So I need some way to query a kopia repository to find out when the latest snapshot happened, for each hostname/path.
Right now, AFAICT, the only way to do this is to run kopia snapshot list and then parse the textual output. This doesn’t seem to be something the output is designed to support, e.g. there’s significant whitespace, and I have to either be prepared to parse localtimes or set TZ=UTC.
Is there any easier way I’m missing? I see --json options for a number of commands, but nothing for listing snapshots. If not, could one be added?
Has anyone written anything like this already?
And for that matter, it would be nice if there was some kind of built-in kopia command I could run from cron that would print a nice message and give a non-zero exit status if any of my backups were more than X days out-of-date.
Not Kopia developer, but try to help. If you make policy to schedule backup in interval, and keep the kopia server running, why do expect the backup could fail constantly? Do your backup source or remote have unstable nature (e.g. on a network that intermittently disconnects)? If for example, your source is a local disk and remote is Google Drive, you set it up, tested it’s working, and schedule a daily backup. The only reasons the backup would stop happening for a week continuously are 1) the kopia server process is down; 2) your disk is dead; 3) your network is shut down. 1) can be detected via OS API. The other two should be easily noticeable.
I mean, adding JSON support to snapshot list should be easy. But what is your fundamental concern? Maybe there is a better solution for your problem.
I guess there are two main reasons why I want this.
One is that I have a laptop to back up, so it’s expected that some backups will randomly fail because I happened to close the lid and suspend in the middle of the backup. If that happens once or twice it’s not a big deal; if I get unlucky and it happens a bunch of times in a row, that’s a problem. So I want some mechanism that doesn’t live on the laptop to keep an eye on things.
The other is, well… these are backups we’re talking about. The whole point is that you need them to work when everything is breaking :-). And it’s easy to find many, many, many stories about times when people thought their backup system was working fine, and then only after a disaster discover that something obscure broke months ago and they never noticed, and now they have no backups. I’m just not comfortable with a backup system unless I can have some end-to-end monitoring to double-check it’s actually running properly. Belt and suspenders.
Makes sense. If the data source is unstable, the backup may not be reliable, depending on how fast each snapshot can be made.
Suppose each incremental snapshot takes 10 minutes to finish, I think the easiest solution would be finding a time of day that you will sure to keep the laptop up for more than 10 minutes (e.g. 14:00 when you are working). Then create a daily scheduled task / cron job that runs kopia snapshot create --all. There is no guarantee each backup would complete successfully, but it’s very unlikely it fails several times in a row (unless you travel to the other side of the earth and forgot to sync the clock ).
Also, if what you described happened, shouldn’t those snapshot ended up as “incomplete” state? snapshot list --incomplete should list those versions, and all you need is to grep on those and notify if any exists, assuming you have set up scheduled backup and keep the server running.
The last possibility is that say you never power on the laptop for several days, thus causing the scheduled backups to skip. But since you never used the laptop, the data should remain the same, so the last backup should still be fine.
Or schedule something for 14:00 that doesn’t involve the laptop, etc. Basically my philosophy on backups is that since I’m a human, I should assume that if there’s any conceivable way I could screw things up, then I will screw things up, and it’s the computer’s job to protect me against my own stupidity.
But it doesn’t matter if one backup is incomplete, as long as another one completes soon enough afterwards. It’s expected and normal that sometimes I’ll suspend my laptop in the middle of a backup. So I don’t want to get annoying alerts about every incomplete backup, because I’ll just tune them out. I only want to get alerted when there’s an actual danger of data loss.
You’d have to write your own little script for that. After having played with Kopia for some months, I can say, that Kopia has incorporated some very nice new features which will ensure that snapshots will finally succeed. Kopia will always try to get it’s schedule done, no matter what.
If you are afraid that Kopia will fail you, you can write a little snippet, which connects to the repo and queries the snapshots and alerts you, if the timestamp of the latest snapshot is x days off.
All great ideas totally aligned with Kopia goals. I’ll be happy to review and merge pull requests that add end to end monitoring, alerting, etc.
My bandwidth is limited so to make those things happen faster I really would like to encourage folks to contribute to Kopia and make it better for everyone.
).
I would really like to deploy kopia for real, but not sure how to do that without some end-to-end monitoring…