I used kopia snapshot verify --verify-files-percent=10 --file-parallelism=10 --parallel=10 on a 17GB backup to Storj through rclone and next thing I know, I’m over my egress limit, having gone from basically nothing to 29GB. Any ideas as to what went wrong here?
I now have access to a test account with a 300 MB backup. I started a 10% verification again and interrupted it partway through after it had reached 3GB egress. Now that I can reproduce this, what logs should I look through to troubleshoot this?
I use kopia with Storj too and observe similar behavior. Usually after 4-7 days the egress used will drop way down on the dashboard as it takes the Storj network a bit of time to settle the egress bandwidth used. Once this happens the actual egress used is shown on the dashboard. This happens because Storj will “allocate” egress bandwidth based on the maximum you could have possibly used if you downloaded the complete files from every node that stores them. Then as the individual nodes submit there actual usage the egress will “settle” on your account and the usage shown will drop back down.
I heard similar when researching this myself so I guess this makes sense. Still a pain that it completely locks you out in the meantime while it considers your egress used up. I switched to the self-hosted S3 gateway (running on my machine, so Kopia points to 127.0.0.1) and have experienced no such issue since. I also tested simply downloading a few things using rclone from the command line and didn’t experience the high egress, so it’s still a mystery to me why the specific kopia+rclone combination causes such problems.
I’ve seen it happen with the S3 gateway too. I’ve got kopia setup to use the S3 self-hosted gateway with caching enabled. If you do this make sure you exclude the kopia.maintenance file from the cache as this can cause problems. See Here for more information.