Trying to "reset" a repo

Support
1

I created a snapshot of a home directory (roughly 9GB) then I deleted all snapshots with

kopia snapshot delete --all-snapshots-for-source /home/user --delete
kopia maintenance --full --safety=none

After that the size of the repo folder was still 128MB. Then I ran

kopia snapshot gc --delete --safety=none
kopia logs cleanup --max-age=1s
kopia index optimize --advanced-commands=enabled
kopia blob gc --advanced-commands=enabled

But after that the size of the repo folder is still 28MB

I am wondering what is in those 28MB and how to get rid of them? There shouldn’t be more than a few KB, right?

It is not critical issue, of course. Just trying to understand Kopia.

PS

repo is of type “filesystem”

2

What’s your kopia blob ls output?

3

The output is (slightly edited to fit better)

kopia.maintenance                                                  2938
kopia.repository                                                   1118
xw1637174032                                                         18
xw1637174093                                                         18
xw1637174126                                                         18
xw1637174159                                                         18
xw1637174193                                                         18
xw1637263936                                                         18
xw1637263938                                                         18
xw1637266344                                                         18
xn0_ff75950d8c891ce410c1c9c77d038952-s37c8532c1d18e17a10a-c1      12508
xn0_f9cdb9b50cab3809eebc0f1de09d6a9c-s39903bdefc6d74bb10a-c1    3417639
xn0_f7b4d505cbfd405f86f7365849c1e9d1-s9ac05be50eeed44610a-c1    2919944
xn0_f641c5e1c53c8b0651b7a74d70abea68-sf5222e46b9a9e8a110a-c1      31403
xn0_f581284057025ca4172c9f4d5841141b-sc181150b15edc11710a-c1        143
xn0_eac7521d3335bca0369ee0b3960009bc-s9398dccf981ad9dd10a-c1       2011
xn0_d42ff23aca4fe2b0e327a4049a7673c0-sda451cbe88c037e210a-c1       1909
xn0_cd32a1cc268aa8bc48b20b1a48af9385-s818722e50995175310a-c1        143
xn0_b73a6a8b863634b2ee2dda41280619f5-s4ec57f60009f158e10a-c1        143
xn0_9c4f7d8d62073cd2f954906648459fb8-se139f753012a6fea10a-c1        143
xn0_9137f1dde00fc833d6a0aa5c98ed7940-s5bfed5028e0f6f0b10a-c1       5683
xn0_89e00ba6e9401da7578269a102c251f8-s6192f5f0a34dc2c510a-c1       1615
xn0_79be6b62b2869f6d652b76f7401fe892-sc5f201c8b9ae301610a-c1    6186121
xn0_6894dc52130dbe9bfb16b72503d39a3c-se51d3f6dc6f5261910a-c1        143
xn0_53dc0325c9103a18a9b0f7aa58ae4711-s1a88f4c1be1058ec10a-c1        143
xn0_51aba139090aeeeb86b6aaf9a1982267-s0f1797e7cb95a4d410a-c1      34701
xn0_33b7973d180994ab555f4f9e177600a1-sdbad3483149c36d010a-c1       2555
xn0_310938bd8ea346883c34365e2f77cba7-sef339a324462646510a-c1        143
xn0_294d15660a91997fdef83ef6fa343480-se4c9f09148d8562010a-c1        143
xn0_26fa2c3670b2531ee46b4de43750de42-s4896e08db3c7db8610a-c1       3439
xn0_1f21a77b2c4a6bffa4b37a147563cd0d-s547e0222352b1ad510a-c1      16078
xn0_12aba2d2eb4add1eaf3ad8f9e3af8067-se292920ffb16685610a-c1        143
xn0_002dc4d3338c3e9d10cdb1c8fa91d316-s5ae39222bf78e24610a-c1      60447
q18808366a76854e43307ecafa6c1185-s0f1797e7cb95a4d410a          11303407
4

It looks like those are indices and metadata?

From the documentation

* `q` represents packs containing metadata (e.g. `q7a9939814e8aba1fdda2d87965f324d3` )
* `x` represents indices (e.g. `xn0_20db7984bd71c4042cea471a61fbcea1` )

BTW I also did a similar test on Linux . Snappshotted 100GB, did the same steps above, and the size of the repo was just a few KB.

5

Do you need more information? (Or are you out of ideas here? :innocent:)

6

You can try to see see what’s in the live blobs by:

kopia index inspect --active

7

I did that. And the output is 370,000 lines of the type below.

How can I clean this repo, there is no content in the repo, so that should be all “garbage”?

2021-11-18 01:33:51.026 +07 xn0_f7b4d505cbf405f826f7365849c1e9d1-s9ac05be50eeed44610a-c1 bb185f6a07c8b8fe02db105189b7dfdf deleted 2021-11-18 01:33:47.000 +07 pa06bf389b7d1ab04cc9a5caf4d0d9f2c-s5ae39222bf78e24610a 4252700 6159
2021-11-18 01:33:47.539 +07 xn0_f9cdb9b50ca3809e0ebc0f1de09d6a9c-s39903bdefc6d74bb10a-c1 0073c6fff83d7dc1b4ddc589166b2c16 deleted 2021-11-18 01:33:43.000 +07 p410847760915fa4fab39d9167f6155de-sc5f201c8b9ae301610a 90040 355
2021-11-18 00:38:00.212 +07 xn0_f641c5e1c538b06591b7a74d70abea68-sf5222e46b9a9e8a110a-c1 468586ea380ad39761b5fbf3f89bb9c6 created 2021-11-18 00:37:46.000 +07 pc00c87936f3d831846223097f3041f95-sf5222e46b9a9e8a110a 13902549 17193

369,997 lines follow :-)

(obviously I did not read all lines in detail, but it seems like that)

8

Any idea how to proceed now?

9

If the repo is completely empty, you can simply delete all files except kopia.repository and it should be back to clean slate.

10

That’s good to know!

“In the name of bug hunting”, Is this still of interest though?

11

delete all files except kopia.repository

Two questions:

  1. What about policies? They are not in the .repository.f file. Deleting all except kopia.repository.f would lose those right?
  2. Why do these x blobs not reclaimed? Are they reference counted or is there special rule for “meta” blobs (which prevent them from being deleted)? Can we make Kopia smarter to know when to delete them (safely of course)?