Good news is, my Kopia setup is mostly running well. I’m backing up 3 Macs to a Synology NAS; and then use
kopia repository sync-to s3 \
--endpoint=s3.... \
--bucket=... \
--access-key=... \
--secret-access-key=... \
--delete
To sync the Synology repository over to B2. The sync-to is working fine, but …
I had started to do the sync between Synology and B2 using Synology Cloud Sync, but based on comments here, I decided to use sync-to instead.
My problem is, I reused the same bucket that I had used with Cloud Sync, and though I had intended to clear the data that Cloud Sync had already synced over to that bucket, I screwed up and didn’t.
Because the data Cloud Sync had pushed was actually the identical repository that sync-to was syncing, I was thinking that sync-to would clear any unnecessary data on the destination bucket. But it appears not to. I tried adding the --delete flag but that still doesn’t seem to clear out data in the destination bucket.
The source repository shared folder on Synology is 1.1TB; the B2 bucket is 1.7TB.
I can always create a new bucket and do a clean re-sync; but is there a way to tell sync-to to clear the extra data from the current bucket?
Here’s log output:
Synchronizing repositories:
Source: Filesystem: /volume1/kopia-backups
Destination: S3: s3.us-west-001.backblazeb2.com …
Looking for BLOBs to synchronize…
Found 1 BLOBs in the destination repository (4.3 MB)
Found 1001 BLOBs in the destination repository (21.3 GB)
Found 2001 BLOBs in the destination repository (44.2 GB)
Found 3001 BLOBs in the destination repository (67 GB)
Found 4001 BLOBs in the destination repository (89.8 GB)
Found 5001 BLOBs in the destination repository (112.8 GB)
Found 7001 BLOBs in the destination repository (158.6 GB)
Found 8214 BLOBs in the destination repository (186.4 GB)
Found 10001 BLOBs in the destination repository (227.4 GB)
Found 11001 BLOBs in the destination repository (250.2 GB)
Found 12561 BLOBs in the destination repository (286.1 GB)
Found 13001 BLOBs in the destination repository (296.1 GB)
Found 14001 BLOBs in the destination repository (319 GB)
Found 15001 BLOBs in the destination repository (341.8 GB)
Found 16001 BLOBs in the destination repository (364.7 GB)
Found 17001 BLOBs in the destination repository (387.6 GB)
Found 18001 BLOBs in the destination repository (410.5 GB)
Found 19001 BLOBs in the destination repository (433.4 GB)
Found 20001 BLOBs in the destination repository (456.2 GB)
Found 22001 BLOBs in the destination repository (501.7 GB)
Found 24001 BLOBs in the destination repository (547.5 GB)
Found 26001 BLOBs in the destination repository (593.3 GB)
Found 27001 BLOBs in the destination repository (616.2 GB)
Found 29001 BLOBs in the destination repository (661.8 GB)
Found 31001 BLOBs in the destination repository (707.6 GB)
Found 33001 BLOBs in the destination repository (753.1 GB)
Found 35001 BLOBs in the destination repository (798.9 GB)
Found 37001 BLOBs in the destination repository (844.5 GB)
Found 39001 BLOBs in the destination repository (890.2 GB)
Found 41001 BLOBs in the destination repository (0.9 TB)
Found 42001 BLOBs in the destination repository (1 TB)
Found 44001 BLOBs in the destination repository (1 TB)
Found 46001 BLOBs in the destination repository (1 TB)
Found 48001 BLOBs in the destination repository (1.1 TB)
Found 49107 BLOBs in the destination repository (1.1 TB)
Found 1 BLOBs (30 B) in the source repository, 0 (0 B) to copy
Found 270 BLOBs (3.9 GB) in the source repository, 27 (407.4 MB) to copy
Found 762 BLOBs (15.2 GB) in the source repository, 31 (493.6 MB) to copy
Found 1313 BLOBs (27.8 GB) in the source repository, 37 (624.7 MB) to copy
Found 2040 BLOBs (44.6 GB) in the source repository, 45 (788.1 MB) to copy
Found 2674 BLOBs (59.1 GB) in the source repository, 49 (873.8 MB) to copy
Found 3081 BLOBs (68.5 GB) in the source repository, 51 (0.9 GB) to copy
Found 3420 BLOBs (76.3 GB) in the source repository, 54 (1 GB) to copy
Found 3953 BLOBs (88.4 GB) in the source repository, 57 (1 GB) to copy
Found 4793 BLOBs (107.7 GB) in the source repository, 69 (1.3 GB) to copy
Found 5914 BLOBs (133.4 GB) in the source repository, 79 (1.5 GB) to copy
Found 7081 BLOBs (160.2 GB) in the source repository, 95 (1.9 GB) to copy
Found 8717 BLOBs (197.7 GB) in the source repository, 109 (2.2 GB) to copy
Found 10432 BLOBs (237.1 GB) in the source repository, 121 (2.4 GB) to copy
Found 12763 BLOBs (290.5 GB) in the source repository, 145 (2.9 GB) to copy
Found 15399 BLOBs (351.1 GB) in the source repository, 166 (3.4 GB) to copy
Found 17842 BLOBs (407 GB) in the source repository, 181 (3.7 GB) to copy
Found 20756 BLOBs (474 GB) in the source repository, 205 (4.2 GB) to copy
Found 23899 BLOBs (546.1 GB) in the source repository, 228 (4.7 GB) to copy
Found 27078 BLOBs (619.3 GB) in the source repository, 249 (5.2 GB) to copy
Found 30210 BLOBs (691 GB) in the source repository, 268 (5.6 GB) to copy
Found 33399 BLOBs (764.4 GB) in the source repository, 288 (6 GB) to copy
Found 36638 BLOBs (838.8 GB) in the source repository, 322 (6.7 GB) to copy
Found 39867 BLOBs (0.9 TB) in the source repository, 350 (7.4 GB) to copy
Found 43061 BLOBs (1 TB) in the source repository, 368 (7.7 GB) to copy
Found 46345 BLOBs (1.1 TB) in the source repository, 389 (8.2 GB) to copy
Found 48983 BLOBs (1.1 TB) in the source repository, 410 (8.6 GB) to copy
Found 533 BLOBs to delete (12 B), 48573 in sync (1.1 TB)
Copying…
Also, I am able to connect successfully to the B2 repo and spot checks of files look right. Though the raw data structure of the B 2 repo doesn’t match what’s on the NAS …
Current size of B2 bucket is 1.6 TB so there appears ~500GB of pre-existing data still in the bucket …