I’ve unsuccessfully tried to migrate the repo of ~ 420GB mostly static source files to V2.
The total size of the repo came to over 600GB.
What I’ve done: Migrated, made a new snapshot, deleted all previous snapshots, initiated full maintenance which deleted just about 25GB. The second full maintenance performed back-to-back surprisingly freed another 11GB. The third one finished freeing 0. However, the total size of the repo is still over 570GB.
Totally new repo with the same settings (except 1M splitter instead of 4M) containing a single initial snapshot is 288GB.
New repo (DYNAMIC-1M-BUZHASH)
Count: 316606 Total Bytes: 359.1 GB Total Packed: 308.8 GB (compression 14.0%) By Method: (uncompressed) count: 106239 size: 104.5 GB zstd-best-compression count: 210367 size: 254.6 GB packed: 204.3 GB compression: 19.8% Average: 1.1 MB Histogram: 0 between 0 B and 10 B (total 0 B) 373 between 10 B and 100 B (total 24.5 KB) 14978 between 100 B and 1 KB (total 6.1 MB) 8865 between 1 KB and 10 KB (total 30.3 MB) 13927 between 10 KB and 100 KB (total 642.9 MB) 140216 between 100 KB and 1 MB (total 89.4 GB) 138247 between 1 MB and 10 MB (total 218.7 GB)
Old repo (DYNAMIC-4M-BUZHASH):
Count: 213680 Total Bytes: 662.8 GB Total Packed: 608.3 GB (compression 8.2%) By Method: (uncompressed) count: 150304 size: 412.7 GB zstd-best-compression count: 63376 size: 250 GB packed: 195.5 GB compression: 21.8% Average: 3.1 MB Histogram: 0 between 0 B and 10 B (total 0 B) 435 between 10 B and 100 B (total 28.2 KB) 27724 between 100 B and 1 KB (total 10.2 MB) 6939 between 1 KB and 10 KB (total 22.5 MB) 10155 between 10 KB and 100 KB (total 416.4 MB) 22319 between 100 KB and 1 MB (total 10.6 GB) 146108 between 1 MB and 10 MB (total 597.2 GB) 0 between 10 MB and 100 MB (total 0 B)
(A high percentage of uncompressed files is expected due to extensive no-compress policy.)
Old repo had some snapshots made with different compression settings (pgzip-best), which, as I now understand, leads to no deduplication on V1. However, after removing all pre-migration snapshots and performing forced full maintenance I expect the size to be at least in the ballpark of a new repo.
Didn’t save the log, but at least 40% of the total repo size was “uploaded” to the old repo during snapshotting post-migration so even if the rest was totally uncompressed the results are still totally off: the repo size is double the size of the new one.
snapshot list --all of the old repo:
Zoom@hostA:d:\dir1 Zoom@hostA:d:\dir2 Zoom@hostA:e:\dir2 # different mount point of an external drive Zoom@hostB:h:\dir2 2021-11-23 k22e78a355a6a2fa3183793ef00cc7f42 353.1 GB drwxrwxrwx files:21604 dirs:1392 (latest-1,annual-1,monthly-1,weekly-1,daily-1,hourly-1) Zoom@hostB:h:\dir1 2021-11-23 kc136bb3d1847faa65f3a1685367fb3d3 70.7 GB drwxrwxrwx files:2434 dirs:334 (latest-1,annual-1,monthly-1,weekly-1,daily-1,hourly-1)
Shouldn’t the user@host of the deleted snapshots disappear? Is it an indication of some orphaned blobs remaining?