I’m wanting to run maintenance on my Kopia repo. I thought this was happening automatically.
As it turns out, this is not the case.
kopia maintenance run --full
Running full maintenance...
Looking for active contents...
Processed 278 contents, discovered 1419...
Processed 290 contents, discovered 1446...
Finished full maintenance.
ERROR snapshot GC failure: error running snapshot gc: unable to find in-use content ID: error walking snapshot tree: error verifying b8a82c683b31d5561b07a3cf0fa3286c: error getting content info for b8a82c683b31d5561b07a3cf0fa3286c: content not found
According to kopia maintenance info my last successful maintenance run was back in early March. Yikes!
Notably, performance is still quite good! It’s just that the repo is probably much larger than necessary.
I’d like to clear the error so that I can get back to running maintenance like normal. I likely corrupted something when doing a Google Drive Trash Restore.
I decided to try kopia index optimize --drop-contents=b8a82c683b31d5561b07a3cf0fa3286c but received a harsh warning about doing this.
This command could be dangerous or lead to repository corruption when used improperly.
Running this command is not needed for using Kopia. Instead, most users should rely on periodic repository maintenance. See Maintenance | Kopia for more information. To run this command despite the warning, set --advanced-commands=enabled
Now I’m hesitant to proceed, and I’m just looking for confirmation that I’m about to do the right thing. I would prefer not to destroy my backup repo.
My proposed next step is to run kopia index optimize --drop-contents=b8a82c683b31d5561b07a3cf0fa3286c --advanced-commands=enabled
followed by kopia maintenance run --full
I’m assuming I wouldn’t need to run a kopia blob gc because this is taken care of by the full maintenance run.
I’ve decided to take a poll. I appreciate your willingness to participate in this unusual survey.
Doing this will ruin your backed-up digital life. I will recommend an alternative solution that is far superior.
Doing this is an excellent idea. You’re a full-on genius.
This repo is on a google drive.
I just ran rclone dedupe kopia: --dedupe-mode newest against the entire repo.
Now I’m trying the dangerous commands.
(I used rclone to make an entire replica of the repo, so I won’t actually lose anything at any point by experimenting)
I used --advanced-commands=enabled and it went through. But now running kopia maintenance just tells me it had another error verifying some other string. Running multiple times, I get the same error with a different string.
Boring.
Time to try SAFETY=NONE or whatever the dev warns us to never do.
At this point, I don’t know what else to do…
UPDATE: Now trying kopia blob gc --delete=true --safety=none --advanced-commands=enabled
UPDATE2: The above command took a very long time, and I didn’t want to leave it running too long and risk having a client operate on the repository while I was using the unsafe flag. So I canceled that.
I have no idea how I can run maintenance on my repo at this point. I’m willing to delete snapshots, contents, indexes, whatever it takes to get this to work again.
How long did it take to solve itself? And did the other repository recover?
I’m not sure what to do at this point.
I’m considering mounting snapshots and snapping them to a new repo. Not sure about that yet.
EDIT:
Next I’m trying:
count=25; while [ $count -ge 0 ]; do kopia maintenance --full | grep "error processing" >> kopia.errors 2>&1; let "count=count-1"; done
Then I’ll take a look at the list and reduce it to the content ids, then for i in $(cat kopia.errors); do kopia index optimize --drop-contents=$i --advanced-commands=enabled; done
I don’t remember but I guess it shouldn’t take more than 2 days. It wasn’t a big deal for me as I had other backups (differnet software). Besides the maintenance error Kopia was working as usual.
I lost patience and deleted the repository after ~2 weeks as nobody seemed to have a clue what was going. You can read the thread I linked to if you want to know the whole story.
Maybe open an issue at Github and hope for a developer to help you solve this puzzle instead of running random commands out of desperation?
I updated 5 days ago, and I haven’t had it self-correct.
I read the whole story in the other thread a few days ago, maybe even before I posted this.
It’s an annoying thread. Nobody seems to understand what you’re really asking.
Side note, when a command finishes with no output, that’s typically a good sign. No output means it succeeded.
I may try deleting these blobs if I can.
I didn’t want to necessarily open a github issue, because I think the problem is my own fault. I’m doing this in a Google Drive account, and I ran a restoration of deleted data on the account, which places everything back into the folders that it came from. Anything Kopia deleted, returned. So my repo is loaded with junk, hence why I want to run maintenance.
I ran kopia snapshot fix invalid-files --commit and it gave me a ton of output.
Now maintenance is running and I’m not getting those errors. Some interesting output it has now given:
GC found 1654036 unused contents that are too recent to delete (427.2 GiB) GC found 284811 in-use contents (34.7 GiB) GC found 1640 in-use system-contents (4.4 MiB)
I expect that it will delete this in the not too distant future. It is currently rewriting contents from short packs.
