Verification failures

Support
1

Following the discussion in this thread I realized I had to start running verification on my repository. I did that and can’t make sense of the results I’m getting:
snapshot verify succeeds without errors and reported processing 312256 objects:

Processed 239883 objects.
Processed 262007 objects.
Processed 284964 objects.
Finished processing 312567 objects.

snapshot verify --verify-files-percent=100 finished running and reported a single error and 287744 objects:

ERROR error processing eran@homepc:C:\Users\eran@2025-02-01 01:59:58 IST/AppData/Local/Google/DriveFS/Logs/drive_fs_232.txt: error reading object 5f0e8088163c1b71075bfa540ddacb95: unable to open object 5f0e8088163c1b71075bfa540ddacb95: unexpected content error: invalid checksum at p8dec91ff45a45091b370164a6a27dc89-s6ab50597524a9eca12b offset 8534285 length 36352/36352: decrypt: Error computing ECC: no shard data
Finished processing 287744 objects.
ERROR error reading object 5f0e8088163c1b71075bfa540ddacb95: unable to open object 5f0e8088163c1b71075bfa540ddacb95: unexpected content error: invalid checksum at p8dec91ff45a45091b370164a6a27dc89-s6ab50597524a9eca12b offset 8534285 length 36352/36352: decrypt: Error computing ECC: no shard data

content verify --download-percent=100 reported 17 errors after going through just 6.5% of the “contents”, whatever that means, and then I stopped it because I didn’t see a point in letting it run further.
All the errors looked like this one but with different IDs:

ERROR error content 00f19d8445444031e9732b2a2e7d861f is invalid: invalid checksum at p0359106897c73385184511aab5239328-s98d57780b3300555124 offset 440 length 46080/46080: decrypt: Error computing ECC: no shard data

Can anyone please explain what is going on? Why are there more content verification errors than snapshot verification errors if the latter is supposed to be a superset of the former? And why are they all reporting different object counts?

Finally, any suggestion how to deal with the reported errors? I don’t really care about the file it’s complaining about, assuming it is the only damaged file and in fact I excluded the whole folder from recent backups.

2

I’m going to take a shot in the dark & say Google Drive is touching small files like text files. Have you confirmed the checksum for %APPDATA%/Local/Google/DriveFS/Logs/drive_fs_232.txt is the same on both ends (eg: when decrypted fr the download to a tmp dir)?

I don’t use Windows or Google Drive but if I were, in this case, I’d drop MobaXTerm (x64 version) on the machine. That’ll given you a basic environment to get some proper *Nix tools like md5sum, sha256sum, sha512sum, etc. It’s available on Chocolatey or direct download.

(Unsolicited opinion: …but if you’re doing Python dev I’d simply run Linux in a VM if not going as far as dumping MSFT altogether.)

3

Trying to bump this thread for hopefully a more relevant response.

I’m now running daily verification with partial download of the repo and occasionally getting errors such as this:
error reading object 9ba4bf3a958a8f7189fe0685a6b691be: unable to open object 9ba4bf3a958a8f7189fe0685a6b691be: unexpected content error: invalid checksum at p0359106897c73385184511aab5239328-s98d57780b3300555124 offset 5814792 length 2072/2072: decrypt: Error computing ECC: no shard data

I found three similar such errors over the last month, which were not reported when I ran the full verification, so I assume they are new problems in the repo. Is that normal? Is there a way to fix them?

4

I have no idea if it’s normal. I’m still trying to get this abandonware-adjacent software into a sliver’s stance of being semi-usable. I’m sure you’ve seen all the threads I’ve been spewing out on this forum. It’s going to be a while before I start burn in tests.

But goddamn if I ain’t gonna be running kopia snapshot verify --verify-files-percent=100 --file-parallelism=10 --parallel=10 on a weekly basis for the foreseeable future.

If you value your sanity don’t start crawling through their GitHub issues.

5

I don’t think it’s an abandonware, there is new work being done on this project consistently. It does however seem to have too few developers involved for a complex product with a lot of advanced users who push it to its limits. That said, it’s still better than other alternatives I’ve tried so I’m very thankful to the people who spend their time working on it.

1 Like
6

Respectfully, I can’t agree. Encrypted snapshots are nothing new, nor using AES-256 crypto. Marcium Reflect (Windows, paid) has supported it for years. It also has a form of checksumming, IIRC.

Kopia’s first commit was nine years ago. The blake3 checksumming algo is supported by Kopia. It was only released in 2020. RFC 7616 is ten years old this year. That’s not even the tip of the proverbial iceberg I’ve seen since digging deep into their GitHub, forum, commits.

It’s not hard to see where are their priorities… & I don’t mean getting it to a production/stable release.

7

It’s a community driven open source project and no one owes anyone anything. The developers are volunteers and “their priorities” are on whatever they like to work on. Users are free to make the choice if they want to use it or not. They are also free to contribute themselves by making PRs to improve things they care about. That’s how open source works and it may not be the right fit for everyone.

8

Uh huh.

Would you mind telling me which supposed ‘members’ of this software ‘community’ (as if that’s the sort of litmus test of what constitutes a community instead of vapid, manipulating marketing-speak so easily slurped up so often, so readily) registered the domain, designed their slick web site, set up this forum, published the docs, setup a security disclosure contact, publishes must reads but doesn’t update the docs?

Can you point out the member of their ‘community’ who can’t even validate a basic desktop shortcut before unleashing it on an unsuspecting userbase?

Is Kasten, owned by Veeam Software Group a member by your metrics?

You are no more apart of any ‘community’ than you are based on your choice of file system or the fact you have notepad.exe present on your PC.

Again: their repo is nine years old. This isn’t someone’s COVID lockdown make work/keep busy project to stave off cabin fever.

Furthermore I’m not going to be lectured to on the philosophical nature of F/OSS from someone who’s idea of computing is dictated to him by his corporate overlords at Redmond. I’m old enough to remember when programming was a profession. Do you even predate Windows, Kunter?

I’m going to let you in on a little secret you clearly have yet to learn: just because it’s F/OSS doesn’t make it good.

See: the first post ITT.

9

You are literally complaining that someone spent their time a money setting up a domain and building a website. Did you pay them anything? What exactly gives you the right to have this shitty attitude. And why exactly are you still here? If you don’t like it don’t use it. Nobody is forcing you to use this software and no one would care if you stopped.

10

Huh; so that’s what you chose to first zero in on, to retort with. Fascinating. I’ve got a bridge in Brooklyn for you. Wanna buy it, boy-o?

That’s the wrong question. Here’s the more relevant one: am I expected to pay with my data or yours? Given your deficiencies at comprehension, attention I refer you back to the first post ITT.

Oh, I’m sorry; should I have limp-wristed, mealy mouthed responses like others on here who 1.) can’t see through the copy or grasp the faults because they’ve no technical acumen & are drawn in by the slick presentation or 2.) don’t know what it is they don’t know so they end up placing themselves in a false state of security only to have it all burn down (again: first post, this very thread)?

Your kind will do anything but rise to discussing the merits of the subject, won’t’ch’a?

Tell me: what’s your post secondary education level, Kutner? You’re very sophomoric.

Yeah, it would be easier if I just F’d off, wouldn’t it? Meanwhile some poor SOB like this comes along… risking more than just his job:

Now let me flip that question back: why are you here when you don’t even have the sense to seek primary sources before unleashing half-baked code on unsuspecting users, openly encourging its use, leading 'em down a garden path?:

And you would know that how? My git repos include four different methods to nullify an undocumented but openly acknowledged vector for a failure cascade reported years ago. Here’s a taste of the end result testing one of 'em, MSFT serf:

dav_20250427_182756
dav_20250427_182756400×166 16.5 KB

You, on the other hand, are obviously, painfully oblivious to said source of repo corruption.

Another brilliant insight. Thank you for your non-input.

pip install pathetic

11

Just do everyone a favor and go troll somewhere else

12

You’re obviously too vacant to understand of what it is to troll, wannabe tech bro.