Stranges behaviors using kopia snapshot create

Hello, I am new to Kopia. I have two differents servers using Ubuntu server 24.04, and I want to use Kopia to backup the data from one server (data-server) to store it in my backup-server (quite the classic use-case !)
On my data server, I have an old xeon 1225 v3 and 32GB of DDR3 which has 4 threads.

On the backup server, I created a Kopia repository and created an Ubuntu service which automatically launches kopia server.
On the data server, I installed Kopia and I am trying different scenarios to create a snapshot that is uploaded to the repository of the backup server using kopia snapshot create, for a set of 28.9GB of data (mainly pictures and videos). I first had an extremely slow process (2h for 10GB) by using the following command :
sudo kopia snapshot create /data

I stopped the snapshot and cleaned the data with listing the incomplete snapshot and running
kopia snapshot delete
And then on the backup-server I run
kopia maintenance run --full --safety=none

Which from my understanding delete everything and makes my repository clean.I run theses command after every test.
I then started to make some tests playing with the parameter --parallel.

Here are the result I had
–parallel 5000

  • 0 hashing, 2222 hashed (28.9 GB), 0 cached (0 B), uploaded 25.3 GB, estimated 28.9 GB (100.0%) 0s left
    Created snapshot with root k823d94524065b5293f43db74c4724d6b and ID 550380de7d2c76cd1ad0aaa595df794e in 32m49s

With --parallel = 4

  • 0 hashing, 2222 hashed (28.9 GB), 0 cached (0 B), uploaded 14 GB, estimated 28.9 GB (100.0%) 0s left
    Created snapshot with root kb9847720104737d0b9de255e9376e066 and ID fd7355d07be45b9f4186d03ee6073ddc in 57m35s

With --parallel 100000

  • 0 hashing, 2222 hashed (28.9 GB), 0 cached (0 B), uploaded 28.8 GB, estimated 28.9 GB (100.0%) 0s left
    Created snapshot with root k4fbfc9a54143eb752218ad5657402607 and ID 79ff4c010272da9f18df256f0b6ae9bd in 20m31s

When I look at the different data I have while the process in ongoing, with a higer parameter for --parallel, the hashing is going much faster and it starts to upload at full speed directly. When it’s low (4) or when the parameter is by default (0), the hashing seems to takes forever. The upload is really low (probably because the data is not ready to be uploaded ? ). Shouldn’t the default parameter be using more of the CPU capabilities ? Is it because the processor is quite old ?

Another interogation I have is why the value of “uploaded” is different in each test I am doing ? I checked on the backup-server how much data was added using the command “df -h” and it matchs with the value of "uploaded ". How is it possible that with the same command with only the parameter --parallel, and without changing anything compression related, it uploads differents size ?

I also have this warning in the end of the snapshot :

WARN unable to send notification        {"err":"unhandled session request","errVerbose":"unhandled session request\ngithub.com/kopia/kopia/repo.errorFromSessionResponse\n\t/home/runner/work/kopia/kopia/repo/grpc_repository_client.go:660\ngithub.com/kopia/kopia/repo.unhandledSessionResponse\n\t/home/runner/work/kopia/kopia/repo/grpc_repository_client.go:666\ngithub.com/kopia/kopia/repo.(*grpcInnerSession).SendNotification\n\t/home/runner/work/kopia/kopia/repo/grpc_repository_client.go:496\ngithub.com/kopia/kopia/repo.(*grpcRepositoryClient).SendNotification.func1\n\t/home/runner/work/kopia/kopia/repo/grpc_repository_client.go:473\ngithub.com/kopia/kopia/repo.inSessionWithoutRetry[...]\n\t/home/runner/work/kopia/kopia/repo/grpc_repository_client.go:606\ngithub.com/kopia/kopia/repo.maybeRetry[...]\n\t/home/runner/work/kopia/kopia/repo/grpc_repository_client.go:572\ngithub.com/kopia/kopia/repo.(*grpcRepositoryClient).SendNotification\n\t/home/runner/work/kopia/kopia/repo/grpc_repository_client.go:472\ngithub.com/kopia/kopia/notification.Send\n\t/home/runner/work/kopia/kopia/notification/notification_send.go:118\ngithub.com/kopia/kopia/cli.(*commandSnapshotCreate).run\n\t/home/runner/work/kopia/kopia/cli/command_snapshot_create.go:153\ngithub.com/kopia/kopia/cli.(*App).repositoryWriterAction.func1.1\n\t/home/runner/work/kopia/kopia/cli/app.go:496\ngithub.com/kopia/kopia/repo.WriteSession\n\t/home/runner/work/kopia/kopia/repo/repository.go:413\ngithub.com/kopia/kopia/cli.(*App).repositoryWriterAction.func1\n\t/home/runner/work/kopia/kopia/cli/app.go:492\ngithub.com/kopia/kopia/cli.(*App).repositoryWriterAction.(*App).maybeRepositoryAction.func2\n\t/home/runner/work/kopia/kopia/cli/app.go:579\ngithub.com/kopia/kopia/cli.(*App).repositoryWriterAction.(*App).maybeRepositoryAction.(*App).baseActionWithContext.func3.1.1\n\t/home/runner/work/kopia/kopia/cli/app.go:564\ngithub.com/kopia/kopia/cli.(*profileFlags).withProfiling\n\t/home/runner/work/kopia/kopia/cli/profile.go:45\ngithub.com/kopia/kopia/cli.(*App).repositoryWriterAction.(*App).maybeRepositoryAction.(*App).baseActionWithContext.func3.1\n\t/home/runner/work/kopia/kopia/cli/app.go:559\ngithub.com/kopia/kopia/cli.(*App).runAppWithContext.func1\n\t/home/runner/work/kopia/kopia/cli/app.go:530\ngithub.com/kopia/kopia/cli.(*App).runAppWithContext\n\t/home/runner/work/kopia/kopia/cli/app.go:531\ngithub.com/kopia/kopia/cli.(*App).repositoryWriterAction.(*App).maybeRepositoryAction.(*App).baseActionWithContext.func3\n\t/home/runner/work/kopia/kopia/cli/app.go:558\ngithub.com/alecthomas/kingpin/v2.(*actionMixin).applyActions\n\t/home/runner/go/pkg/mod/github.com/alecthomas/kingpin/v2@v2.4.0/actions.go:28\ngithub.com/alecthomas/kingpin/v2.(*Application).applyActions\n\t/home/runner/go/pkg/mod/github.com/alecthomas/kingpin/v2@v2.4.0/app.go:568\ngithub.com/alecthomas/kingpin/v2.(*Application).execute\n\t/home/runner/go/pkg/mod/github.com/alecthomas/kingpin/v2@v2.4.0/app.go:398\ngithub.com/alecthomas/kingpin/v2.(*Application).Parse\n\t/home/runner/go/pkg/mod/github.com/alecthomas/kingpin/v2@v2.4.0/app.go:230\nmain.main\n\t/home/runner/work/kopia/kopia/main.go:77\nruntime.main\n\t/opt/hostedtoolcache/go/1.22.7/x64/src/runtime/proc.go:271\nruntime.goexit\n\t/opt/hostedtoolcache/go/1.22.7/x64/src/runtime/asm_amd64.s:1695"}

1 Like

Your test results indicate that Kopia is using data that is still available in the repository or in the cache. You should create a new repository for each test run to make sure Kopia isn’t using any existing data.

The default is probably tuned for conventional HDDs which can perform worse with too much IO.

To achieve optimal performance you can tune some parameters upon creation of the repository. Run these commands on both servers:

  • kopia benchmark crypto
  • kopia benchmark compression --data-file=DATA-FILE
  • kopia benchmark splitter

Kopia v0.18.0 introduced a new notification feature that tries to send out notifications if an error is encountered. That warning means you didn’t create any notification profiles.

Yes I had the same thought, I finally achieved consistents results after rebooting both servers after each tests. Likely a cache issue.

Weirdly, it now works fine without fine-tuning the --parallel parameter. - maybe everything was related to this cache issue.

Thanks, I will try that.

Thanks, I created a profile but I encoutered another error. I will create another post as I think it’s another problem.