Any tips for faster snapshots of large (+50 TB) backups?

I’m backing up a large (FUSE-mounted) drive to GCS. The first snapshot took over a day which is okay, but I’m really trying to speed up subsequent snapshots. Before I start tweaking anything, there’s a couple things I’d like to understand:

  • How does Kopia decide if a file should be hashed? Is it based on size, modification time, or something else? I hope its not based on inodes which might change due to FUSE
  • What do the values in the UI mean? The ones that aren’t completely self explanatory to me are “Cached Bytes”, “Cached Files”, “Processed Bytes”, and “Processed Files”.
  • How do Cached Bytes and Cached Files relate to the Kopia cache: Caching | Kopia ? (I’d like to estimate how large to make my local cache drive)
  • A snapshot necessitates walking the whole directory tree, correct? This is a little slow over FUSE but possibly nothing I can do about it.

Thanks for the help - I apologise if some of this is covered in docs and posts, I really did look!

1 Like