# My benchmark of kopia and bupstash

**URL:** <https://kopia.discourse.group/t/my-benchmark-of-kopia-and-bupstash/502>\
**Category:** General\
**Created:** [July 1, 2021, 1:02am UTC](https://kopia.discourse.group/t/my-benchmark-of-kopia-and-bupstash/502 "2021-07-01T01:02:25Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![nh2](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/nh2/32/171_2.png) [@nh2](https://kopia.discourse.group/u/nh2)\
**Post date:** [July 1, 2021, 1:02am UTC](https://kopia.discourse.group/t/my-benchmark-of-kopia-and-bupstash/502/1 "2021-07-01T01:02:25Z")

</div>

I recently benchmarked current versions of `kopia` and [`bupstash`](https://bupstash.io/) for various scenarios to evaluate these tools.

Report: [GitHub - nh2/bupstash-kopia-comparison: Evaluation benchmarks of backup tools bupstash and kopiia](https://github.com/nh2/bupstash-kopia-comparison)

I will likely update the report a couple times as the tools improve over time.

* * *

For `bupstash`, I’ve implemented (but not benchmarked yet) a WIP for per-directory parallel `stat()`ing, which is its current bottleneck for my use case.

For `kopia`, my main request is to allow to runtime-configure the number of threads it uses (instead of having it hardcoded to 16), as my networked file system would benefit a lot from that. There are also a couple issues I found (and linked).

I would also appreciate that if you find some answers to the open questions in there (e.g. why my `kopia` run didn’t deduplcate the data within the first run on the _“4 GB, small files”_ dataset), please answer them here or file an issue in my report’s repo.

Thanks, and happy backuping!

---

<div class="post-metadata">

**Author:** ![nh2](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/nh2/32/171_2.png) [@nh2](https://kopia.discourse.group/u/nh2)\
**Post date:** [July 1, 2021, 1:06am UTC](https://kopia.discourse.group/t/my-benchmark-of-kopia-and-bupstash/502/2 "2021-07-01T01:06:45Z")

</div>

Btw, this discourse is configured such that

> Sorry, new users can only put 2 links in a post.

which is somewhat annoying for technical work (e.g. linking to issues).

---

<div class="post-metadata">

**Author:** ![jkowalski](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/jkowalski/32/3_2.png) [@jkowalski](https://kopia.discourse.group/u/jkowalski)\
**Post date:** [July 2, 2021, 2:16pm UTC](https://kopia.discourse.group/t/my-benchmark-of-kopia-and-bupstash/502/3 "2021-07-02T14:16:34Z")

</div>

Interesting report. Kopia will generally deduplicate across “contents” which are sections of large files between 1MB and 8MB - this is to keep the number of entries in the index low. This also makes deduplication across small files not effective, but those are typically compressible (esp. log/source files).

If you can, try with:

```auto
$ kopia policy set --global --compression=zstd-fastest

```

or even better run:

```auto
$ kopia benchmark compression --data-file=sample-file.log

```

and pick compression method that is fastest on your machine.

You have some excellent points about parallelism, memory consumption, etc. All things we should improve over time - I’d be happy to review PRs for those.

Definitely please file individual GH issues for proposed improvements. That would be highly appreciated.