Caching suddenly gets slow in ~15TB repo

When creating a snapshot of ~15TB repo (also differential one), first ~7-8TB are very fast to cache (it takes minutes).
Afterwards, caching becomes extremely slow, processing 10k small files (less than 100GB) takes more than e.g. ramping up from 0TB to 6TB.
Do you have any ideas where this sudden change comes from and if there is a way to tweak it?

Thank you in advance!

[EDIT] Forgot to mention: the source of data is NFS and destination is S3.