Another invalid checksum

Support
1

I have a kopia repository on pCloud and when doing kopia maintenance un --full I get

2022-07-05T12:27:41.045817+02:00  Rewriting contents from short packs...
2022-07-05T12:28:20.497298+02:00  unable to rewrite content "b4b977f5adeb7096b499b09fcf991c5b": unable to get content data and info: invalid checksum at pef1fadaf23dee5217e577debf3a5cc81-sbbb06912f257eb53111 offset 7797637 length 3152/3152: decrypt: unable to decrypt content: cipher: message authentication failed
2022-07-05T12:28:20.499827+02:00  unable to rewrite content "f3c5e6f6ebbcabeafca6278560b6f400": unable to get content data and info: invalid checksum at pef1fadaf23dee5217e577debf3a5cc81-sbbb06912f257eb53111 offset 7800789 length 40/40: decrypt: unable to decrypt content: cipher: message authentication failed
2022-07-05T12:28:31.237874+02:00  Finished full maintenance.
2022-07-05T12:28:31.548228+02:00 ERROR error rewriting contents in short packs: failed to rewrite 2 contents

kopia content verify runs ok

Running, for example, sha256sum daf23dee5217e577debf3a5cc81-sbbb06912f257eb53111.f works.

What can I do?

2

You need to run kopia snapshot verify --verify-files-percent=100 if you want to test for corrupt blobs and then kopia snapshot fix invalid-files --verify-files-percent=100 to try to fix the blobs. kopia content verify will not identify corrupt blobs. See Best method to ensure valid snapshots: snapshot verify vs snapshot fix invalid-files - #2 by jkowalski.

1 Like
3

Thanks a lot @basldfalksjdf . After the verify you gave, I started kopia snapshot fix invalid-files --verify-files-percent=100.

Now running for 11.5 hours. I think my organization with a large repository in pCloud, it has around 2TB, is wrong. Have to split into multiple repos I guess.

4

I don’t know the specifics about pCloud, and it is possible that pCloud cannot support such large repositories. However, Kopia should have no problems with a 2TB repo. Maybe try a different cloud storage?

5

Kopia is fine I know.

My point is that in case I need to run above command it takes awfully long with such a large repository.

This is the same with other cloud providers and might be dependent upon the bandwidth one has. In my case 50Mit/s.

6

It is downloading the whole repository to check. If you have a provider where you have to pay for that it will also cost you some money.

7

@gogolathome I am aware of that and of course I have an internet flat. :slight_smile:

8

I guess that depends on what you mean by “slow”.

Using --verify-files-percent is going to be slower than not using --verify-files-percent because with the command you are downloading, decrypting, and decompressing the files. Essentially, pretty close to doing an actual restore. In your case, you are doing that for a 2TB repo, and I suspect pCloud is not exactly fully saturating your pipe either.

Did you make sure to enable --file-parallelism and --parallel? Increasing them may help speed up the process (depends on your hardware, as --parallel is already set to 8 by default and increasing it further may bottleneck you).

For reference, it takes about 1.5 hours for me to do --verify-files-percent=100 for a 200GB repo from Amazon, whose servers are really close to me and fully saturates my pipes as needed.

9

I just used the command as you gave me. Not sure if I could cancel this now and try again with parallel options?!

It is running now for 21h and it may take still 8x as much to get thru all of the 2TB. Which is too long for me.

10

I have to admit when running the command I forgot to do the math in advance. Then I would have seen that with the bandwidth I have it has to take a long time to get all read. :innocent: