Hi Jarek, Julio, and all devs, thanks for Kopia, super nice product, many features and coherent design. I did an extensive review of backup solutions, and selected and use Kopia for user data, and Borg for system. I especially like the builtin scheduling and web UI, plus robust deduplicated data model, and many backends.
A couple bugs and feature reqs:
(all these notes refer to a local storage repository)
BUG: MIGRATE dropped GLOBAL policy
Regarding the v1 to v2 format change, I was recently migrating a repo. I made a new repo, then migrated all my snapshots from the old. I did this via
$ kopia snapshot migrate --source-config âŚPathOld.config --all --policies
Everything seems to have migrated correctly except the GLOBAL policy did not get migrated. Afterwards I tried again with --policies --overwrite-policies --latest-only --sources=
but it still did not bring GLOBAL.
This is a serious bug, because if I had started the server on my new repo, it would have been deleting old snapshots based on the default retention.
BUG: Then I tried to fixup the policy via
kopia policy edit global
This doesnât work, despite the help says it should. All the fields are empty and it shows "Editing policy for â%vâ "
It works to give
kopia policy edit --global
so I was able to edit the retention policy back.
FEATURE: I couldnât find a way to SET a full dump of the policy settings via cmdline. You can GET via policy show --json, but how to set them in bulk from that json?
FEATURE: The gap between Month to Annual is long. What about a Quarterly period, possibly instead of Annual (see next)
BUG/FEATURE: Retention periods are counter-intuitive, lossy.
Retention period seems to be based on absolute dates? ie. week is cycling on Sunday? annual at end of year? And itâs preserving the latest in the time period, instead of the earliest. This results in being too cavalier about throwing away old backups.
Maybe would rather have earliest, or relative logic, such that the earliest/first snapshot defines the start of epoch?
There is general counter-intuitiveness in the retention policy timeframe as currently designed, which is most noticeable on the longer intervals. Most egregiously with Annual, which currently refers to the latest backup of a calendar year.
This seems completely wrong. If I specify Annual = 1 year, I want that to be protecting up to 1 year backwards. The latest ones are already protected by the smaller intervals daily/weekly/monthly.
To my mind, the goal is recovery and rollback coverage, not some âyear-end accountingâ notion. So I want the longest spans possible.
The periods should mean that I can rollback that far.
Example: Setting Annual = 2, I expect would cover 2 years span, say if today is 2022jan1 then back to 2020jan1 or the oldest one within this time span.
NOT AS with the current design, that Annual=2 setting gives you only 2 days coverage! (2022jan1 and 2021dec31)
In general, to protect the oldest backups with the current design, the Annual is useless, and I have to dial the weekly/monthly way up.
Found some other discussion on this:
A little baffled by retention policy
Trying to understand retention policies - #4 by jkowalski
FEATURE: Empty password.
I would like to have no password on some repo, but it currently requires a non-empty string.
Itâs ok that the files/info would still actually be stored encrypted, per the whole data content is sliced and diced into blobs anyway, not actually files.
For local backups with physical control of the media, the password/encryption is less necessary, and there is risk of losing it, and it puts another step/requirement in the recovery process.
FEATURE: How about support for usually OFFLINE media? I would like to have some backups offline, then attach the media and a Kopia backup runs automatically (or by demand), then detach when itâs done. And have this automated to a high degree, so Kopia can sense when the repo is attached, without failing or too much system polling during the 99% time itâs offline. Also, a distinction between how often it checks for the media, say every 5 minutes, versus the actual backup period, say once per day, or once per week.
The idea being, I attach the media, Kopia notices it within some settable short period, but only does a new backup if itâs time for a new one, within a longer period. (i.e. not backuping every 5 minutes, just because the polling time was 5 min)
What would be the best way to accomplish this currently?
FEATURE: Scheduler to be more powerful, set actual times to run, not just elapsed seconds from prior/first. (either explicit times, or windows/intervals; also times to not run)
DOC The way CONFIG files work is not adequately documented. It took some research/experiments to figure them out. Buried under Reference.CommandLine, itâs listed as just another option, I initially thought was just a naming thing, to put the config-file somewhere else. But itâs really an overall operational attribute, the key to having multiple simultaneous connections and repos running.
Itâs a good design, that you can have specific .configs, and/or the single system default config â just needs more intro/overview paragraphs, and documented at a higher level of the overall system operation.
FEATURE: Help should list single line commands first, otherwise they are buried in the commands which take many sub-commands. Or, you might consider grouping them by the most important commands first
Also what about -h or -? for help.
FEATURE It would be nice if cmds could accept a unique prefix or shorter form, for example ârepoâ for repository, âsnapâ for snapshot, etc.
Itâs laborious to type the long cmds and options at cmdline.
This would be a more generally useful efficiency, and could serve in lieu of those default subcommands like server [start], and snapshot [create] which are somewhat desultory thus error prone.
OLD VERSION POSSIBLE BUGS
Here are a few more bug notes from my earlier tests around July 2021.
These are from old version, and/or I did not well understand the application yet, so these might not be actual bugs. I have not reexamined them lately.
BUG?: Restore via web UI, is making ALL SHALLOW, all .kopie-entry restores, doesnt actually restore whole tree. (v0.8.4, 2021jul16)
BUG?: Restore via cmdline, doesnt preserve dirtimes. Seems to be setting dirtime to time of newest contained file or dir, and this ripples upwards.
BUG? Scheduling per Path â when a Repo has multiple Paths in it, want to schedule backup them at different intervals. Itâs doing both Paths, despite that the other Path has no Schedule set.
Thanks again for all your great work on Kopia.
â JBThiel
