We create a snapshot for a big directory(about 28.9T)
the full copy take more than one day, after the full copy is done, we try to test the incremental copy speed after 2 days, we use the default parameter: kopia snapshot create xxx
but it seems the incremental copy take much long times, it running more than 16 hours:
0 hashing, 128578 hashed (203.5 GB), 48060696 cached (28.7 TB), uploaded 44.1 GB (192 errors ignored), estimating…
we plan to backup the directory every 6 hours, but it seems the incremental copy takes too long time, is there any way to improve it so we can backup the directory in time?
1 Like
Hi @ray,
Can you share some more details on the setup you have?
- basic config of the device running kopia
- Operating systems used
- Anti virus if any
- HDD or SSD
- Backend storage: krs/Samba/nfs/SFTP/ftp/rclone
- internet/network speed if you use remote backups
- defined compression policy if any
This would help to answer your questions better.
Sd
Depending on how your data is organized, you will probably need to parallelize your snapshots. E.g., we are snapshotting a 77TB Samba server, which holds approx 400 shares in shy of an hour, by parallelizing and running a separate job for each share.
Running on a dual 10 core CPU, we spin up up to 15 seperate snapshot jobs at once.