# Does kopia use content-defined-chunking (CDC)? **URL:** \ **Category:** General\ **Created:** [September 9, 2022, 3:35pm UTC](https://kopia.discourse.group/t/does-kopia-use-content-defined-chunking-cdc/1417 "2022-09-09T15:35:11Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![bbigras](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/bbigras/32/297_2.png) [@bbigras](https://kopia.discourse.group/u/bbigras)\ **Post date:** [September 9, 2022, 3:35pm UTC](https://kopia.discourse.group/t/does-kopia-use-content-defined-chunking-cdc/1417/1 "2022-09-09T15:35:11Z")
I saw [a post on lobster](https://lobste.rs/s/h38buk/restic_0_14_0_deduplicating_encrypted#c_fztmd3) saying that restic used content defined chunking. Does kopia use something similar? > Not only rolling hashes, but also content defined chunking [GitHub - restic/chunker: Implementation of Content Defined Chunking (CDC) in Go](https://github.com/restic/chunker) which is just magic really. Deduplicating segments not at block boundaries. ---
**Author:** ![stpr](https://avatars.discourse-cdn.com/v4/letter/s/b3f665/32.png) [@stpr](https://kopia.discourse.group/u/stpr)\ **Post date:** [September 10, 2022, 3:12am UTC](https://kopia.discourse.group/t/does-kopia-use-content-defined-chunking-cdc/1417/2 "2022-09-10T03:12:40Z")
Yes, Kopia does indeed use something similar. Restic calls them chunking, Kopia calls them ‘Splitters’. Both tools use so called [rolling hash](https://en.wikipedia.org/wiki/Rolling_hash), I am not exactly sure what Restic uses, but I think it is Rabin-Karp algorithm. Kopia also provides you with a variety of splitters, though this is not well documented, if you run `kopia benchmark splitter`, you might see something like this: ```nohighlight splitting 16 blocks of 32MiB each, parallelism 1 DYNAMIC 307.8 MB/s count:107 min:9467 10th:2277562 25th:2971794 50th:4747177 75th:7603998 90th:8388608 max:8388608 DYNAMIC-1M-BUZHASH 319 MB/s count:428 min:9467 10th:612999 25th:766808 50th:1158068 75th:1744194 90th:2097152 max:2097152 DYNAMIC-1M-RABINKARP 96.2 MB/s count:391 min:213638 10th:623062 25th:813953 50th:1328673 75th:2097152 90th:2097152 max:2097152 DYNAMIC-2M-BUZHASH 301.5 MB/s count:204 min:64697 10th:1210184 25th:1638276 50th:2585985 75th:3944217 90th:4194304 max:4194304 DYNAMIC-2M-RABINKARP 96.7 MB/s count:204 min:535925 10th:1235674 25th:1675441 50th:2525341 75th:3658905 90th:4194304 max:4194304 DYNAMIC-4M-BUZHASH 296.4 MB/s count:107 min:9467 10th:2277562 25th:2971794 50th:4747177 75th:7603998 90th:8388608 max:8388608 DYNAMIC-4M-RABINKARP 98 MB/s count:110 min:535925 10th:2242307 25th:2767610 50th:4400962 75th:6813401 90th:8388608 max:8388608 DYNAMIC-8M-BUZHASH 294.2 MB/s count:49 min:677680 10th:5260579 25th:6528562 50th:11102775 75th:16777216 90th:16777216 max:16777216 DYNAMIC-8M-RABINKARP 100.2 MB/s count:60 min:1446246 10th:4337385 25th:5293196 50th:8419217 75th:12334953 90th:16777216 max:16777216 FIXED 94 TB/s count:128 min:4194304 10th:4194304 25th:4194304 50th:4194304 75th:4194304 90th:4194304 max:4194304 FIXED-1M 56.9 TB/s count:512 min:1048576 10th:1048576 25th:1048576 50th:1048576 75th:1048576 90th:1048576 max:1048576 FIXED-2M 112.8 TB/s count:256 min:2097152 10th:2097152 25th:2097152 50th:2097152 75th:2097152 90th:2097152 max:2097152 FIXED-4M 159 TB/s count:128 min:4194304 10th:4194304 25th:4194304 50th:4194304 75th:4194304 90th:4194304 max:4194304 FIXED-8M 320.9 TB/s count:64 min:8388608 10th:8388608 25th:8388608 50th:8388608 75th:8388608 90th:8388608 max:8388608 ``` Notice the names on the splitters: BUZHASH and RABINKARP. Both of these are rolling hashes (see the linked Wiki article above). Kopia allows you to select how long each split is: from 1M to 8M; and also how to compute them: fixed, BUZHASH, RABINKARP. Based on your benchmarks that you use on your data, you can select the one that suits you the best.