I have and old server with 6Gb of ram running a kopia server (and other services…).
Every now and then kopia starts taking 20% of ram and the server starts swapping. I tried to stop other services to allow more ram, but it is useless, the only solution is restarting kopia service.
Is there any CLI command to see what is kopia doing? I tried accessing web interface, but server is barely responding to a SSH session.
Can you try with the latest v0.11.0 - there have been a bunch of improvements to reduce the memory usage in that version.
Ok, I just upgraded my version using apt.
Maybe there is a parameter I could add to my kopia.service (systemd) to make it more verbose?
Checking the changelog I see zstd-best-compression is not recommended, and it is the algorithm I was using… which one is recommended and how should I proceed to change it? I have:
- multiple clients (family PCs) connecting to my server
- server is low on CPU, but I prefer “best compression possible”, I thing for long retention it pays back.
- the storage is a S3 (wasabi, no egress fee, only storage billed), the bucket is ~1Tb
Just changing the policy in the server will affect new upload and repacking? will deduplication still work?
You can see some benchmarking statistics for the different compression algorithm at Compression | Kopia. Since you are low on RAM, I would pick one that uses less memory.
ztd is the algorithm that provides one of the best compression ratios with very good speed, but RAM usage is on the higher side. Looking at Compression | Kopia,
gzip uses very little RAM but it is very slow.
pgzip is the middle ground – less RAM than
ztd but more than
gzip with decent compression ratios and faster speed than both. May I suggest you try
pgzip to see if your server can handle that? If not, change it
Changing the policy now would change compression on all future uploads. Kopia will not go back and recompress already uploaded files. Deduplication should still work.
My server is a dual core machine, so I don’t think there should be much of a difference between gzip and pgzip.
I will play with gzip, and change it whenever I upgrade the server.
If it might help, - I having one server with 4Gb of RAM and just
zstd works fine ( I stay away from “zstd-best-compression”) even so there working MySQL, Web server + email server (postfix+dovecot+spamassasin+antivirus+DKIM+DCC…)
No, it don’t work fine, it worked a few months Ok, but today’s latest maintenance got failed due to out of memory because of
zstd. I rerun
kopia benchmark compression and it looks like
s2 is the fastest and much less hungry.
It looks like
pgzip usage is more than
zstd in my tests (320kb data file)
Compression Compressed Throughput Allocs Usage
0. s2-parallel-8 104.5 KiB 606.2 MiB/s 2159 2.2 MiB, --compression=s2-parallel-8
1. s2-parallel-4 104.5 KiB 605.8 MiB/s 2187 2.2 MiB, --compression=s2-parallel-4
2. s2-default 104.5 KiB 603.1 MiB/s 2195 2.2 MiB, --compression=s2-default
3. s2-better 99.6 KiB 435.2 MiB/s 2263 2.2 MiB, --compression=s2-better
4. zstd-fastest 78.8 KiB 298.9 MiB/s 7368 9.7 MiB, --compression=zstd-fastest
5. zstd 75.8 KiB 206.3 MiB/s 4879 19.3 MiB, --compression=zstd
6. deflate-best-speed 98.2 KiB 205.2 MiB/s 29 1.1 MiB, --compression=deflate-best-speed
7. zstd-better-compression 71 KiB 147.9 MiB/s 4872 37.8 MiB, --compression=zstd-better-compression
8. gzip-best-speed 111.5 KiB 144.3 MiB/s 35 1.5 MiB, --compression=gzip-best-speed
9. deflate-default 84.7 KiB 132.8 MiB/s 29 1.3 MiB, --compression=deflate-default
10. lz4 109.2 KiB 122.4 MiB/s 2135 3.4 GiB, --compression=lz4
11. pgzip-best-speed 98.2 KiB 109.2 MiB/s 11456 1.1 GiB, --compression=pgzip-best-speed
12. pgzip 84.7 KiB 77.7 MiB/s 11315 1.1 GiB, --compression=pgzip
13. gzip 80.6 KiB 63.7 MiB/s 33 1.1 MiB, --compression=gzip
14. gzip-best-compression 80.4 KiB 56.1 MiB/s 33 1.1 MiB, --compression=gzip-best-compression
15. deflate-best-compression 80.2 KiB 46 MiB/s 30 1.4 MiB, --compression=deflate-best-compression
16. pgzip-best-compression 80.2 KiB 38.2 MiB/s 11807 1.2 GiB, --compression=pgzip-best-compression
17. zstd-best-compression 68.3 KiB 22.1 MiB/s 4819 100 MiB, --compression=zstd-best-compression
Yes, you are correct. I was focusing on large files, but RAM usage on small files for
pgzip is quite high.
It looks like it isn’t an issue with
zstd. Replaced 0.11 back to
kopia 0.10.7 and it finished maintenance successfully, but I will keep s2 compression from now on anyway.
I’m coming back to try kopia again after almost a year and I’m currently creating a new snapshot of a folder that has 675,077 files and as far as I’ve seen kopia uses up to 20GB of RAM (I’ve had to update that number 3 times in the making of this comment because it first was 16 and then 19), sometimes I have seen it go down to 8 or 9 GB but I think that is due to memory compression because that process’ RAM usage goes up too. Is this a currently known problem?
I’m using the default parallel file reads which should be 16 and using zstd-better-compression, CPU usage is pretty low, only thing is the RAM usage, it’s too much. restic uses a lot too but it goes to 7GB or less.