ERROR failed to open repository: [...] too many open files

Support
1

I am recovering from a failed root SSD on my Raspberry Pi. When reattaching my Kopia repository, I’m hitting this “too many open files” error, which, when I google it, hasn’t previously appeared in connection with Kopia.

Kopia-UI on Windows is stuck on “Initializing repository,” with an endless string of messages like

12:25:00.757 finished sweeping {"cache":"metadata","duration":"66.8833ms","totalRetainedSize":466424725,"tooRecentBytes":0,"tooRecentCount":0,"maxSizeBytes":5242880000,"inUsePercent":8}

Connecting to the repository Kopia-UI on the Pi (arm64 Ubuntu 22.04 LTS) is simply failing with request failed with status code 400.

Kopia on the command line of the Pi yields the following:

$ sudo kopia repository connect filesystem --path -- "/tank/--- Backups/Kopia"
Enter password to open repository:

[a few minutes wait]

ERROR failed to open repository: unable to create shared content manager: error loading indexes: unable to open pack index "ncb2f39362e1c07cd4dee689595ae6de6-s139d3956a2c98620111": unable to open file despite retries: open /root/.cache/kopia/186ab428dc760d97/indexes/ncb2f39362e1c07cd4dee689595ae6de6-s139d3956a2c98620111.sndx: too many open files
ERROR error connecting to repository: unable to create shared content manager: error loading indexes: unable to open pack index "ncb2f39362e1c07cd4dee689595ae6de6-s139d3956a2c98620111": unable to open file despite retries: open /root/.cache/kopia/186ab428dc760d97/indexes/ncb2f39362e1c07cd4dee689595ae6de6-s139d3956a2c98620111.sndx: too many open files


$ sudo kopia repository repair filesystem --path -- "/tank/--- Backups/Kopia"
looking for format blob...
format blob already exists, not recovering, pass --recover-format=yes

$ kopia repository repair --recover-format=yes filesystem --path -- "/tank/--- Backups/Kopia"
looking for replica of format blob in pffff383cc649811fef14f946e104b210-sa61f037b9abfd9b8107...
recovered replica block from pffff383cc649811fef14f946e104b210-sa61f037b9abfd9b8107

$ kopia --version
0.11.3 build: 317cc36892707ab9bdc5f6e4dea567d1e638a070 from: kopia/kopia
(The kopia version of the previous pre-crash machine may or may not have been the same, but it was no earlier than 0.10.)
2

Are you sure this is being caused by Kopia? The error “too many files open” is typically an issue with the Linux distro. I suspect the same for you – you will need to tweak your distro settings. See https://www.howtogeek.com/805629/too-many-open-files-linux/

3

Indeed, raising the limit from 1024 to 2096 did the trick.

What determines the number of files Kopia will open at once? Wondering how high to set the limit. For now I’ve set it ridiculously high (524288, or 512x the default).

(Note to anyone arriving from Google: follow the instructions at the linked article starting in the “Making Permanent Changes” section; and don’t use “sudo kopia”, because the root user has its own files-open limit which won’t be changed by those steps.)

4

I have a similar fault MacOS with my first attempt to run 0.11.3.

Error:

ERROR open repository: unable to open repository: unable to create shared content manager: error loading indexes: unable to open pack index "n1151b05bfbe19e6f5bf587a32110f67d-s4070d5803ea83b3910f": unable to open file despite retries: open /Users/rjkroege/Library/Caches/kopia/e83b2b11f50b8ea2/indexes/n1151b05bfbe19e6f5bf587a32110f67d-s4070d5803ea83b3910f.sndx: too many open files

This does not happen with versions before v0.11.0 with the same repository.

I have raised the limit to the max:

;    ulimit -a
ulimit -c 0
ulimit -d unlimited
ulimit -f unlimited
ulimit -l unlimited
ulimit -m unlimited
ulimit -n 5000
ulimit -s 8372224
ulimit -t unlimited
ulimit -u 10666
ulimit -v unlimited

to no avail.

Any suggestions?

5

This also started happening for me with v0.11.0 on MacOS.

My workaround is to run the following:

sudo launchctl limit maxfiles unlimited unlimited
ulimit -n 10240

However, I’d also love to know if there’s a way to limit the number of open files via a kopia flag or config option.