# Snapshot GC failure: Could not undelete referenced content (deleted session blob, BLOB not found) - Kopia 0.22.3 / S3/MinIO **URL:** \ **Category:** Support\ **Created:** [April 2, 2026, 2:29pm UTC](https://kopia.discourse.group/t/snapshot-gc-failure-could-not-undelete-referenced-content-deleted-session-blob-blob-not-found-kopia-0-22-3-s3-minio/8845 "2026-04-02T14:29:16Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![spetrov](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/spetrov/32/1985_2.png) [@spetrov](https://kopia.discourse.group/u/spetrov)\ **Post date:** [April 2, 2026, 2:29pm UTC](https://kopia.discourse.group/t/snapshot-gc-failure-could-not-undelete-referenced-content-deleted-session-blob-blob-not-found-kopia-0-22-3-s3-minio/8845/1 "2026-04-02T14:29:16Z")
# Support **Environment:** - Kopia Version: 0.22.3 - Storage: S3 (MinIO, endpoint: s3 own storage) - Deployment: Docker container (kopia/kopia:latest) via Podman Quadlet on Rocky Linux 9 - Repository format: v3, Epoch Manager enabled, Current Epoch: 27 **Problem:** Since March 14, 2026 every `snapshot-gc` run fails with: ```auto snapshot GC failure: error running snapshot gc: error iterating contents: Could not undelete referenced content: {pe15bce50646ecb824423a908daf46592-sb5dddc2d36f62fe913e 1774695556 198443 198471 32 0 f2a5ef14efd7a9d31b8be03cd79d0212 true 3 0}: unable to get content data and info: error getting cached content from blob "pe15bce50646ecb824423a908daf46592-sb5dddc2d36f62fe913e": failed to get blob with ID pe15bce50646ecb824423a908daf46592-sb5dddc2d36f62fe913e: BLOB not found ``` The content entry has `deleted: true` – it belongs to an interrupted backup session from March 14. The physical blob no longer exists in S3 (confirmed via `blob list | grep`). However, Kopia’s GC tries to `undelete` it and fails because the blob is gone. **What we confirmed:** - `kopia snapshot verify` → 356,508 objects, zero errors – all snapshots intact - The blob `pe15bce50646ecb824423a908daf46592-sb5dddc2d36f62fe913e` does not exist in S3 - `full-rewrite-contents` has been failing daily since March 5 (growing from 996 → 21,108 failed contents) - Index blob count has grown to ~15,000+ due to failed maintenance **What we tried:** - `kopia blob delete` on the blob (already gone, exit 0) - `kopia blob delete` on xn/xs/q-epoch blobs referencing the session ID - `kopia index recover --dangerous-commands=enabled --commit` (twice, with server stopped) - `kopia cache clear` (caused SIGSEGV – separate bug?) - Clearing cache directory manually + service restart - `kopia maintenance run --full --safety=none` None of these resolved the issue. After each `index recover`, the content entry reappears in the next maintenance run. **Question:** Is there a way to permanently remove a `deleted: true` content entry from the Epoch index when its backing blob no longer exists? The entry seems to survive `index recover` because it gets re-written from the xn-epoch blobs which still contain it. ---
**Author:** ![Statick](https://avatars.discourse-cdn.com/v4/letter/s/7feea3/32.png) [@Statick](https://kopia.discourse.group/u/Statick)\ **Post date:** [April 10, 2026, 9:28pm UTC](https://kopia.discourse.group/t/snapshot-gc-failure-could-not-undelete-referenced-content-deleted-session-blob-blob-not-found-kopia-0-22-3-s3-minio/8845/2 "2026-04-10T21:28:08Z")
also having this problem and would appreciate some guidance ---
**Author:** ![spetrov](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/spetrov/32/1985_2.png) [@spetrov](https://kopia.discourse.group/u/spetrov)\ **Post date:** [April 13, 2026, 12:45pm UTC](https://kopia.discourse.group/t/snapshot-gc-failure-could-not-undelete-referenced-content-deleted-session-blob-blob-not-found-kopia-0-22-3-s3-minio/8845/3 "2026-04-13T12:45:32Z")
I have since disconnected, deleted, and rebuilt the S3 storage. This resolved the issue. ---
**Author:** ![Statick](https://avatars.discourse-cdn.com/v4/letter/s/7feea3/32.png) [@Statick](https://kopia.discourse.group/u/Statick)\ **Post date:** [April 16, 2026, 8:20am UTC](https://kopia.discourse.group/t/snapshot-gc-failure-could-not-undelete-referenced-content-deleted-session-blob-blob-not-found-kopia-0-22-3-s3-minio/8845/4 "2026-04-16T08:20:38Z")
I’m not on S3 but having this same error and have tried the same things what exactly do you mean by disconnect, delete and rebuild? ---
**Author:** ![spetrov](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/spetrov/32/1985_2.png) [@spetrov](https://kopia.discourse.group/u/spetrov)\ **Post date:** [April 16, 2026, 8:54am UTC](https://kopia.discourse.group/t/snapshot-gc-failure-could-not-undelete-referenced-content-deleted-session-blob-blob-not-found-kopia-0-22-3-s3-minio/8845/5 "2026-04-16T08:54:19Z")
I disconnected the S3 repository from the Kopia server, then deleted everything related to Kopia from the S3 storage, reconnected the repository, and had all clients create new snapshots or waited until they did so on their own. ---
**Author:** ![budy](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/budy/32/38_2.png) [@budy](https://kopia.discourse.group/u/budy)\ **Post date:** [April 17, 2026, 2:31pm UTC](https://kopia.discourse.group/t/snapshot-gc-failure-could-not-undelete-referenced-content-deleted-session-blob-blob-not-found-kopia-0-22-3-s3-minio/8845/6 "2026-04-17T14:31:55Z")
Well… by “reconnected” you mean recreate, right? If you deleted everything from the S3 bucket you effectively removed the repo… ---
**Author:** ![spetrov](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/spetrov/32/1985_2.png) [@spetrov](https://kopia.discourse.group/u/spetrov)\ **Post date:** [April 17, 2026, 2:43pm UTC](https://kopia.discourse.group/t/snapshot-gc-failure-could-not-undelete-referenced-content-deleted-session-blob-blob-not-found-kopia-0-22-3-s3-minio/8845/7 "2026-04-17T14:43:47Z")
That’s right. I haven’t found a better solution yet. ---
**Author:** ![Statick](https://avatars.discourse-cdn.com/v4/letter/s/7feea3/32.png) [@Statick](https://kopia.discourse.group/u/Statick)\ **Post date:** [April 17, 2026, 11:35pm UTC](https://kopia.discourse.group/t/snapshot-gc-failure-could-not-undelete-referenced-content-deleted-session-blob-blob-not-found-kopia-0-22-3-s3-minio/8845/8 "2026-04-17T23:35:49Z")
okay yeah I’m ideally looking for the kind of fix that doesn’t involve re-uploading 5TB of data over a slow-ish connection ---
**Author:** ![spetrov](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/spetrov/32/1985_2.png) [@spetrov](https://kopia.discourse.group/u/spetrov)\ **Post date:** [April 27, 2026, 9:58am UTC](https://kopia.discourse.group/t/snapshot-gc-failure-could-not-undelete-referenced-content-deleted-session-blob-blob-not-found-kopia-0-22-3-s3-minio/8845/9 "2026-04-27T09:58:22Z")
The problem was that the S3 storage was behind a load balancer that wasn’t configured properly. I configured the S3 storage directly as a repository on the Kopia server, and everything has been working fine ever since. Maintenance runs without errors. ---
**Author:** ![Statick](https://avatars.discourse-cdn.com/v4/letter/s/7feea3/32.png) [@Statick](https://kopia.discourse.group/u/Statick)\ **Post date:** [April 30, 2026, 5:01pm UTC](https://kopia.discourse.group/t/snapshot-gc-failure-could-not-undelete-referenced-content-deleted-session-blob-blob-not-found-kopia-0-22-3-s3-minio/8845/10 "2026-04-30T17:01:17Z")
I fixed this by creating new snapshots, each one being a sub directory of the main snapshot content (there were about 8 sub directories so I created that many new snapshots) because the data for these snapshots already existed in the repository, they did not require a full upload and creating them was quite quick once they were all created, I was able to delete the original snapshot entirely. maintenance has been working ever since so not the simplest or most practical solution in all cases but it worked for me, and maybe posting it here will help someone else as well ---
**Author:** ![jd\_power\_associate](https://avatars.discourse-cdn.com/v4/letter/j/cc9497/32.png) [@jd\_power\_associate](https://kopia.discourse.group/u/jd_power_associate)\ **Post date:** [July 13, 2026, 4:22pm UTC](https://kopia.discourse.group/t/snapshot-gc-failure-could-not-undelete-referenced-content-deleted-session-blob-blob-not-found-kopia-0-22-3-s3-minio/8845/11 "2026-07-13T16:22:11Z")
I’m in a similar situation, full maintenance is failing (maybe it’s been failing since 2022? I just noticed this 😆) and I’d prefer not to start a fresh S3 repo because my Canadian internet connection would take over a week to re-upload. In my case, I have all the data locally, so I would be comfortable running a dangerous command like “nuke the problematic bits and rebuild from my local files” if such a thing exists. ---
**Author:** ![jd\_power\_associate](https://avatars.discourse-cdn.com/v4/letter/j/cc9497/32.png) [@jd\_power\_associate](https://kopia.discourse.group/u/jd_power_associate)\ **Post date:** [August 26, 2026, 7:01am UTC](https://kopia.discourse.group/t/snapshot-gc-failure-could-not-undelete-referenced-content-deleted-session-blob-blob-not-found-kopia-0-22-3-s3-minio/8845/12 "2026-08-26T07:01:23Z")
> [@Statick](#): > > I fixed this by creating new snapshots, each one being a sub directory of the main snapshot content (there were about 8 sub directories so I created that many new snapshots) > > because the data for these snapshots already existed in the repository, they did not require a full upload and creating them was quite quick That’s a good idea. I did something similar and created a new snapshots of the subdirectories that contained the bulk of the size. If you know where the problematic blob is, maybe you could just create a snapshot of that directory? Anyway, hashing those terabytes took some time but it only uploaded a few KB. Then i was able to run full maintenance, and delete the new snapshots I made. Now the old snapshots are working again. Leaves me with a question: what is _undeleting_ in the context of Kopia? Why does Kopia need to undelete anything?