Retention policies documentation

Hi!

I was trying to set up my backups with kopia. Surprisingly, the biggest problem was understanding how retention policies work. Lack of documentation on this topic doesn’t really help. So I searched the forums and Reddit and all that. I think I read every post regarding that, and I think I kinda understand how it all works, but I’m not 100% sure. Either way, I wrote a “documentation” (more like ELI5) in hope someone will correct me if I’m wrong, or help someone else set the policies up.

I still don’t know when the weekly ones come up, Monday or Sunday :stuck_out_tongue:

Kopia rentention policies

Kopia adds a label to a snapshot (day, week, month…) every time it creates one.
Kopia deletes snapshots that do not have a label.

Assume snapshot every 30 mins.

Annual

Let’s say, you start creating snapshots on the 1st of June. Kopia adds Annual-1 label to the first snapshot. When another snapshot is created in 30 mins, Kopia moves the label to the newest snapshot. This is done every 30 mins, until the year finishes — the label is moved to the newest snapshot every time one is created. When a new snapshot is created in the following year, the first snapshot gets the label Annual-1, and the old Annual-1 is changed to Annual-2. This is what happens every year, until x in Annaul snapshots: x is met. In other words, snapshot with Annual-x+1 has the label removed, and it gets deleted.

Monthly

Again, 1st of June. Monthly-1 is added to the 00:01 snapshot and then moved to the newest snapshot when it’s created, until the end of the month when it’s changed to Monthly-2 and the new snapshot in July has Monthly-1. At the end of July, Monthly-2 moves to Monthly-3, Monthly-1 moves to Monthly-2 and the new snapshot gets Monthly-1. All that until the Monthly snapshots: x is met. Monthly-x+1 has the label removed, but not necessarily deleted, as it may still have other labels (like Annual-y if it’s December).

Weekly

Works the same as above, but everything happens at the end of the week (Monday or Sunday).

Daily

Same thing but at the end of the day.

Hourly

Same thing but at the end of every hour.

Latest

Let’s say Hourly snapshots: 1, Daily snapshots: 1 and Latest snapshots: 3.

1st snapshot

1st June - 10:01 - snapshot - (Hourly-1, Daily-1, Latest-1).

2nd snapshot

1st June - 10:01 - snapshot - (Latest-2)
1st June - 10:31 - snapshot - (Hourly-1, Daily-1, Latest-1)

3rd snapshot

1st June - 10:01 - snapshot - (Latest-3)
1st June - 10:31 - snapshot - (Latest-2)
1st June - 11:01 - snapshot - (Hourly-1, Daily-1, Latest-1)

4th snapshot

1st June - 10:31 - snapshot - (Latest-3)
1st June - 11:01 - snapshot - (Latest-2)
1st June - 11:31 - snapshot - (Hourly-1, Daily-1, Latest-1)

As you can see, even though Hourly snapshots: 1, the latest tag keeps them “alive”.

When you combine all of the above, you see why the most recent snapshot always has (Latest-1, Hourly-1, Daily-1, Weekly-1, Monthly-1, Annual-1)

Summary

Let’s assume 30-min intervals, 8th of June 2023, 12:00PM, and these policies:

  Annual snapshots:                     3
  Monthly snapshots:                   12
  Weekly snapshots:                     4
  Daily snapshots:                      7
  Hourly snapshots:                    24
  Latest snapshots:                    24

This is how the snapshots would look like if we didn’t do any manual snapshots.

31st Dec   2021 - 23:31 - snapshot - (Annual-3)
31st July  2022 - 23:31 - snapshot - (Monthly-12)
...
30th Nov   2022 - 23:31 - snapshot - (Monthly-8)
31st Dec   2022 - 23:31 - snapshot - (Monthly-7 ,Annual-2)
...
31st March 2023 - 23:31 - snapshot - (Monthly-4)
30th April 2023 - 23:31 - snapshot - (Monthly-3)
21st May   2023 - 23:31 - snapshot - (Weekly-4)
28th May   2023 - 23:31 - snapshot - (Weekly-3)
31st May   2023 - 23:31 - snapshot - (Monthly-2)
 2nd June  2023 - 23:31 - snapshot - (Daily-7)
 3rd June  2023 - 23:31 - snapshot - (Daily-6)
 4th June  2023 - 23:31 - snapshot - (Daily-5, Weekly-2)
 5th June  2023 - 23:31 - snapshot - (Daily-4)
 6th June  2023 - 23:31 - snapshot - (Daily-3)
 7th June  2023 - 12:31 - snapshot - (Hourly-24)
...
 7th June  2023 - 22:31 - snapshot - (Hourly-15)
 7th June  2023 - 22:31 - snapshot - (Hourly-14)
 7th June  2023 - 23:31 - snapshot - (Hourly-13, Daily-2)
 8th June  2023 - 00:01 - snapshot - (Latest-24)
 8th June  2023 - 00:31 - snapshot - (Latest-23, Hourly-12)
 8th June  2023 - 01:01 - snapshot - (Latest-22)
 8th June  2023 - 01:31 - snapshot - (Latest-23, Hourly-11)
...
 8th June  2023 - 10:01 - snapshot - (Latest-4)
 8th June  2023 - 10:31 - snapshot - (Latest-3, Hourly-2)
 8th June  2023 - 11:01 - snapshot - (Latest-2)
 8th June  2023 - 11:31 - snapshot - (Latest-1, Hourly-1, Daily-1, Weekly-1, Monthly-1, Annual-1)

There might be mistakes in dates or times, because my brain was frying typing this table haha

4 Likes

It is actually very handy - thank you!