Kopia unable to read FS?

Support
1

I’ve recently discovered Kopia while trying to decide between restic and duplicacy, and Kopia looks to better meet all my needs! Unfortunately, it appears that Kopia is unable to read my main personal data drive. I’m using Windows 10 (a fresh installation), and all the native Windows utilities can read my data drive, and I’ve tested restic successfully on it as well.

Error reproduced below, along with a successful ‘dir’ listing. Note that I set the policy to ignore errors.

PS F:\> kopia snapshot estimate "F:\"
Analyzing ....
Ignored error in .: unable to stat directory entry "System Volume Information": CreateFile F:\/System Volume Information: Access is denied.
Snapshot includes 0 files, total size 0 B

Snapshots excludes no files.
Snapshots excludes no directories.

Estimated upload time: 0s at 10 Mbit/s
PS F:\> dir

    Directory: F:\

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
da---          2021-05-26    12:54                !Not Installed
da---          2021-05-24    13:33                BOINC
da---          2021-05-26    12:13                Epic Games
da---          2021-05-26    12:22                GOG Galaxy
da---          2021-05-23    15:28                IRC
da---          2021-05-25    08:18                KSP
d-r--          2021-05-27    14:41                My Documents
da---          2021-05-26    12:09                Origin
da---          2021-05-31    06:55                Steam

I suspect the reason for this is that I’m using btrfs on my F drive (via WinBtrfs). However, I don’t see why all normal FS interactions are fine, but Kopia cannot see it.

Is there any hope of being able to work with Kopia?

Thanks for any support!

- nickersonm

2

The folder you’re trying is not something you want to backup:

You can try excluding this system folder:

kopia policy set --global --add-ignore "System Volume Information"

Also, the issue you’re seeing is with estimation, not with actual snapshot, which I’m guessing should be able to proceed fine. Can you try on a small directory first then on the root of the drive?

3

Thanks for the quick response!

I am already ignoring “System Volume Information”; here’s my policy:

{
    "retention": {
        "keepLatest": 20,
        "keepHourly": 48,
        "keepDaily": 7,
        "keepWeekly": 4,
        "keepMonthly": 24,
        "keepAnnual": 3
    },
    "files": {
        "ignore": [
            "System Volume Information/",
            "\\$RECYCLE.BIN/",
            "Config.msi/"
        ],
        "ignoreDotFiles": [
            ".kopiaignore"
        ]
    },
    "errorHandling": {
        "ignoreDirectoryErrors": true
    },
    "scheduling": {},
    "compression": {
        "compressorName": "zstd-better-compression"
    },
    "actions": {}
}

If I remove the first entry, then it gets the same error but with the second entry in the unable to stat line.

Unfortunately snapshotting doesn’t work any better:

PS F:\> kopia snapshot "F:\"
Snapshotting nickersonm@bigwhite:F:\ ...
ERROR: upload error: unable to stat directory entry "System Volume Information": CreateFile F:\/System Volume Information: Access is denied.

Snapshotting individual files directories does work:

PS F:\> kopia snapshot "F:\BOINC"
Snapshotting nickersonm@bigwhite:F:\BOINC ...
 * 0 hashing, 117 hashed (21.6 MB), 0 cached (0 B), uploaded 8.9 MB, estimating...
Created snapshot with root k8e223948dcb668bf945e617a67819436 and ID c724abf386b3829697782cb42babcdd6 in 1s

so I could technically back up the entire drive, I’d just have to list the root and feed all subdirectories into kopia (now I’ll have to see what the command is to feed a subdirectory into the F:\ snapshot root).

I was hoping there might be an easy solution to allowing Kopia to either ignore this error and continue, or to resolve it.

Edit: no changes by running in an elevated prompt.

- nickersonm

1 Like
4

Surprisingly, kopia ends up working if I grant Users read access to the various system directories on my F drive:

PS F:\> kopia snapshot estimate "F:\" --quiet
Snapshot includes 680350 files, total size 587 GB
   10 GB ...100 GB:       2 files, total size 25.3 GB
   1 GB  ... 10 GB:      25 files, total size 58.4 GB
   100 MB...  1 GB:     666 files, total size 155.2 GB
   10 MB ...100 MB:    6382 files, total size 169 GB
   1 MB  ... 10 MB:   45748 files, total size 138.7 GB
   100 KB...  1 MB:   99215 files, total size 33.8 GB
   10 KB ...100 KB:  153770 files, total size 5.7 GB
   1 KB  ... 10 KB:  229032 files, total size 870.5 MB
   0 B   ...  1 KB:  145510 files, total size 55.6 MB

Snapshots excludes no files.
Snapshots excludes 2 directories. Examples:
 - ./$RECYCLE.BIN
 - ./System Volume Information

Estimated upload time: 130h26m50s at 10 Mbit/s

This is not required for kopia-inaccessible system folders on NTFS or ReFS formatted drives, so I assume there’s a difference between the stat errors returned between NTFS or ReFS and WinBtrfs when there are no +x permissions. I don’t know why kopia can’t get past this error even with ‘ignore errors’ active, either.

For now, this solutions works for me - but please let me know if I can be of any assistance in debugging kopia’s behavior for this (failing even with ignore errors active), or if it would be helpful for me to open a github issue about this.

- nickersonm

5

I think the trailing slash in ignore rule may be a problem. Try ignoring without trailing slash.

6

I tried with and without the trailing slash with no changes, but regardless the issue seems to be that kopia hard failed in this case even when ignoreDirectoryErrors is true.

7

Remove the two leading backslashes from the $RECYLE.BIN and try it again…

8

The backslashes are required to interpret the $ literally; you can see from the estimate when I granted read access to the system directories that it was successfully ignored.

The read access permissions work for me now, and I’m happily using Kopia; the underlying issue seems to be the same as this thread: for some filesystems Kopia hard fails on stat even when ignoreDirectoryErrors is true, and will stat directories before checking if it should ignore them.

- nickersonm