Retention:
Annual snapshots: 2 (defined for this target)
Monthly snapshots: 6 (defined for this target)
Weekly snapshots: 16 (defined for this target)
Daily snapshots: 60 (defined for this target)
Hourly snapshots: 96 (defined for this target)
Latest snapshots: 10 (defined for this target)
âlatestâ is a quite literal prefix - it simply refers to the latest snapshot of each kind. I guess is is meant to be more approaching for the average user.
After each snapshot is taken, kopia will assign retention tag to snapshots and delete snapshots that donât have any assigned tag.
Tags are assigned as follows:
latest-N will be assigned to all snapshots going backwards where the most recent snapshot gets latest-1, previous is latest-2 and so on.
hourly-N will be assigned to the latest snapshot within each respective hour, going back no more than M hours where M is defined by --keep-hourly parameter.
For example if you have done snapshots at:
2001-01-01 14:01 - not hourly because 14:30 was the last snapshot of this hour
2001-01-01 14:30 - hourly-2
2001-01-01 15:01 - not hourly because 15:55 was the last snapshot of this hour
2001-01-01 15:02 - not hourly because 15:55 was the last snapshot of this hour
2001-01-01 15:40 - not hourly because 15:55 was the last snapshot of this hour
2001-01-01 15:50 - not hourly because 15:55 was the last snapshot of this hour
2001-01-01 15:55 - hourly-1
Similarly daily-N is assigned to the latest snapshot within each day:
2001-01-01 14:01 - not daily because 15:55 was done the same day
2001-01-01 15:55 - daily-5
2001-01-02 14:01 - daily-4
2001-01-03 15:55 - daily -3
2001-01-04 14:01 - not daily because 15:01 was done the same day
2001-01-04 15:01 - daily-2
2001-01-05 15:55 - daily-1
Same rule applies to weekly, monthly, quarterly and annual retention rules - each of them retains latest snapshot within a given time period up to a defined limit.
Note the limit applies to both count and time, so if you have keep-hourly 48, it will keep no more than 48 hourly snapshots going back no more than 48 hours since the last snapshot.
Letâs consider only those hourly snapshots: if, for instance, youâre out of office with your laptop for three days, when you come back and connect to the server again, all the 48 hourly snapshots would get deleted (since they all are older than 48 hours), except for those also bearing the latest tag.
So, to respectfully correct @budy here, latest is not to make it approachable but is more of a safety net.
Personally, after it was explained I saw it as a totally different workflow.
i.e. some people may just want to keep the latest X regardless of when it was backed up.
It also is way to retain additional hourly backups. I have one scenario where I have this
Need minimal loss of data during business hours. Currently have ZFS snapshots every 5 minutes during business hours
After 2 days just hourlies is fine
If I was to move that to Kopia it would be
12 * 8 * 2 latest
12 backups per hour⌠every 5 minutes
8 hours
2 days
I would set latest to at least 192 to be able to keep those 5 minutes periods for 2 days in addition to hourly 48 or higher (to keep the hourlies after the latest expires)
That said, does kopia treat (from the retention standpoint) identical snapshots differently from others?
Thereâs not much value in retaining lots of identical snapshots. But, as it doesnât cost space anyway, it doesnât hurt. As long as it doesnât trigger deletion of more useful (since they are not identical) snapshots simply because they are older.
My understanding is that those two identical snapshots will be maintained.
My understanding is retention is as follows for each snapshot
Is this within the last retention number? If retention is 10 then at least 10 snapshots will be retained regardless of other policies.
Is this within the last number of hours for the hours policy. If hour policy is 48 one backup per hour will get flagged for retention for at least two days.
Same for daily, weekly, monthly, yearly
Then as maintenance runs it looks for any snapshots that donât have a flag. Imagine you have latest to 4 and you do 2 backups per hour for 8 hours a day; hourly set to 48, dailies to 7 and weekly to 4
The last four backups will be retained because of the last copy policy, Then after 2 hours only one hourly is kept for up to 48 hours.
After 48 hours 1 daily is kept
After 7 days 4 weeklies are kept
âIdenticalâ backups are simply a CLI/UI feature to collapse them. They are not really any different than the regular ones and have the same retention tags applied to them, so they can be retained by virtue of being hourly, daily, weekly or whatever.
You can see that using kopia snapshot ls -l - for example this is my backup of my /opt - notice how last 38 snapshots are identical but they carry different tags:
2023-07-28 06:46:16 +07 pins:initial-backup
2023-07-30 04:34:14 +07 (daily-7) <-- This is not the latest time in the day
2023-07-30 09:53:45 +07 (latest-10)
2023-07-30 23:53:15 +07 (latest-9,daily-6,weekly-2) <-- Second snapshot of the day
2023-08-01 01:35:07 +07 (latest-8,daily-5,monthly-2) <-- Not the latest time again
2023-08-01 15:48:48 +07 (latest-7)
2023-08-02 06:11:23 +07 (latest-6,daily-4) <-- Not the latest time
2023-08-02 15:35:20 +07 (latest-5)
2023-08-02 21:52:25 +07 (latest-4,daily-3) <-- Two daily snapshots again
2023-08-03 15:15:50 +07 (latest-3,hourly-3)
2023-08-04 05:12:52 +07 (latest-2,hourly-2,daily-2)
2023-08-05 01:00:57 +07 (latest-1,hourly-1,daily-1,weekly-1,monthly-1,annual-1)
Days seem to be based on the oldest snapshot, that rather than checking from 0:00, theyâre broken into 24-hour bins based from the oldest snapshot, then the latest are checked from those bins. In your example, that would be:
2023-07-28 06:46:16 to 2023-07-29 06:46:15 (none)
2023-07-29 06:46:16 to 2023-07-30 06:46:15 (2023-07-30 04:34:14 is the only one in this duration, so daily)
2023-07-30 06:46:16 to 2023-07-31 06:46:15 (2023-07-30 23:53:15 is the latest in this duration, so another daily)
2023-07-31 06:46:16 to 2023-08-01 06:46:15 (2023-08-01 01:35:07 is the only one in this duration, so another daily)
2023-08-01 06:46:16 to 2023-08-02 06:46:15 (2023-08-02 06:11:23 is the latest in this duration, so another daily)
2023-08-02 06:46:16 to 2023-08-03 06:46:15 (2023-08-02 21:52:25 is the latest in this duration, so another daily)
2023-08-03 06:46:16 to 2023-08-04 06:46:15 (2023-08-04 05:12:52 is the latest in this duration, so another daily)
2023-08-04 06:46:16 to 2023-08-05 06:46:15 (2023-08-05 01:00:57 is the only one in this duration, so another daily)
Then assign numbers to them. This⌠is not intuitive at all, so I think this is a bug that should be rectified, @jkowalski.