Similar question to How to speed up repository sync-to, but in my case I am not using rclone.
I have created a local repository (using the kopia docker image), which is about 1 TiB, and want to sync this offsite. I have tried both webdav and sftp remotes, and experience long waiting times “Looking for BLOBs to synchronize”.
I have about 40k blobs and it takes about 1.5 hours to read the blobs in the destination repository. Setting the flag --parallel=8 does not seem to make a difference. Interestingly, the log file only shows 31 minutes even though the timestamps are 85 minutes apart. Copying the missing ~180 blobs takes about 90 seconds:
2024-04-07T10:26:57.075704Z INFO kopia/cli Looking for BLOBs to synchronize...
2024-04-07T11:51:26.596629Z DEBUG kopia/repo [STORAGE] ListBlobs {"prefix":"","resultCount":44523,"error":null,"duration":"31m42.091803961s"}
2024-04-07T11:51:26.596738Z INFO kopia/cli Found 0 BLOBs to delete (0 B), 44344 in sync (1 TB)
...
2024-04-07T11:53:02.986635Z DEBUG kopia/repo [STORAGE] Close {"error":null,"duration":"1.147µs"}
My command line is: docker-compose exec kopia kopia repository sync-to from-config --file=/app/config/hetzner_sftp.config --parallel=8 --delete
Is there any way to speed up this initial scan of the destination repository? I have tried both webdav and sftp with similar results.