Recompressing objects in repository

Hello,

I have been using Kopia for about a year now with compression left disabled. My disk space has since started to run out, so I would like to reduce the amount of storage the Kopia repository needs.

In other threads (e.g. Kopia server taking too much ram->swapping) I learned that Kopia will not automatically recompress already uploaded files. Is there a way to do such recompression manually, or is that something that is simply not supported (yet)?

I understand that it would be possible to scratch the whole repository and create new initial snapshots with compression already enabled. That would, however, mean losing all the snapshots that were already created. Is there any way to reduce the size of the repository (possibly by enabling compression) while retaining old snapshots?

All I can think of would be to tranfer all snapshots to a new repo, which has the appropriate configuration…

That would definitely work for me. However, I am not so sure how to achieve this.

One way that comes to mind is using kopia snapshot migrate. Is it the case that this command does not just copy the repository data as-it-is, but actually compresses the snapshot according to policies in the target repository?

Yeah… that’s the question. One can only hope that, other then the --sync-to command, the migration command honors the target repo’s settings. However, the documentation doesn’t make this clear.

Okay, thanks a lot for the advice. I will try to look at the source code, understand it a little and eventually experiment. Should I come across anything, I will try to keep this thread updated – but right now I am not certain when/if that happens.

Of course if anyone has more insight into how the snapshot migrate command actually works, I will be thankful for any information.

I got around to trying kopia snapshot migrate and I can confirm it can be used for this purpose – the migrated snapshots indeed honor policy settings of the target repository, including compression settings.

To summarize, a repository can be recompressed including existing snapshots in the following way:

  1. Create a repository config file for the existing repository, e.g. by making a copy of <kopia-config-dir>/repository.config
  2. Create a new repository using kopia repository create
  3. Set the compression method for the new repository using kopia policy --global --compression <compression-method> and optionally set other compression settings
  4. Migrate snapshots from the original repository using kopia snapshot migrate --all --source-config <path-to-orig-repo-config>
  5. (Optional) Delete the original repository and move the newly created repository where the original repository was
3 Likes