Does anyone have experience comparing/contrasting zstd and S2 for compression of backups? My shop has been using zstd routinely for a long time, but are not married to it if there’s a better candidate, and S2 might be promising for this case. We’ve been really impressed with Kopia overall, and are planning to use KopiaUI for this situation.
Background: I’m working on a system backup design to have 5 Wi-Fi-connected Windows machines backup important data (not OS/applications binaries) to each other to minimize the chance of data loss in an environment where computers sometimes just die.
Because of that uncertainty, I’ve settled on a round-robin scheme where each machine’s important data is backed up to two other machines on the network, with backup operations staggered over time across the day. Each repo will be pointed to a shared (SMB) folder on a “remote” machine.
The equipment: They’re so-so in the horsepower department, with latest-gen i7s, 32GB RAM, and fast NVME storage. Backups will be across Wi-Fi (6), not Ethernet.
My take is that zstd yields better compression, while S2 uses less CPU and RAM, and is less likely to cause noticeable slowdowns during backup operations.
The ask: If you’ve been through this kind of trade study – or even just have an informed opinion – and have any useful information to share, I’d love to hear it.
According to this doc page: Compression | Kopia
the S2 compression creates files which are ca. two times bigger than with zstd. Your repo will be twice the size with S2. If you are paying your storage provider by the Byte it will be more expensive.
On the other hand S2 is much faster with less memory consumption. But that does not make a difference in real live. The zstd compression speed is fast enough to saturate the backup media if it is not nvme drives.
So my conclusion is: With S2 your backup repo will increase in size while you will not notice a speed difference.
Thanks for the response, yes, that is consistent with my understanding.
We’ll be doing some testing soon on this, a head-to-head comparison for our use case to try to determine exactly how much CPU and RAM are consumed by each during the backup (using Windows perfmon), and also the difference in the resulting repository size for each type.
For our use case the hardware is identical for all machines, and they are loaded from the same OS/software image, so I expect we can leverage that uniformity to predict overall system performance impact.
I’ll post the results, or at least our high-level conclusions, when that is complete.
Candidly, I was hoping I’d hear from someone who had already done that kind of test/trade study and had their own experience or results to share. Not that I’m lazy about doing it myself. Well, okay, maybe I’m a little bit lazy.
