Does anyone know how to find out why a specific file may fail to restore when restoring a whole snapshot via snapshot restore --write-files-atomically?
I doublechecked and the file is correctly backed up in the repo: (1) I have run snapshot verify --verify-files-percent=100 and it returned zero errors and (2) I tried restoring just that specific file from the snapshot rather than restore the whole snapshot and it downloads just fine. However, when doing a full restore of the snapshot, Kopia throws the error ERROR error restoring: restore error: copy file: error creating file: cannot create temp file: open \\?\\\nas\path\to\file\file_name.pdf2619871429: The filename, directory name, or volume label syntax is incorrect.
I think it may have to do with the length of the file name. The file name is 145 characters but the whole file path is 266 characters. But, anyway to confirm this and is there anything that could be changed in Kopia to deal with this scenario? Interestingly enough, if I manually save the file to the target restoration location, it saves just fine.
I did two test restores so far, first with KopiaUI and then with Kopia CLI. 99.99% of the files restored just fine the first round. This was one of the files that did not, but I did that restore via KopiaUI so I couldn’t read what the errors where. The second time around I tested the restore via CLI and Kopia threw the above error. That restore is still going, so I am not yet sure if the other files will be successfully restored, but I expect the same as the earlier restore with KopiaUI.
If this is an issue with the atomic library, then presumably if I did not restore atomically, then the error would not happen, correct? On the flipside, not restoring atomically risks partial restore of files but no errors would be thrown in that situation?
In windows, \\?\ prefix disables following string parsing as well it allows to bypass MAX_PATH limit set by WinAPI and called sometimes as “raw path”(or “extended” path), so it shouldn’t be a case.
Are you sure you have 3 backslashes after ? (question mark character) ?
It should be actually \\?\nas\path\to\file\file_name.pdf2619871429 unless \\nas is a network path.
Also, does your NAS supports long file namespaces ?
Since you on Windows, what is registry value of this key:
I copy and pasted exactly what the error said. The 3 backslashes should be correct, since it is a network share.
As far as I know, this shouldn’t be a NAS issue because I am able to save the file just fine in the exact same location Kopia was supposed to restore it.
The full restore completed, and it threw the following 10 errors:
2022-06-23T07:07:36.873739Z ERROR restore ignored error copy file: error creating file: cannot create temp file: open \\?\\\nas\path\to\file\file_name.pdf4100174292: The filename, directory name, or volume label syntax is incorrect. on path/to/file/file_name.pdf.
2022-06-23T07:19:46.272056Z ERROR restore ignored error copy file: CreateFile \\?\\\nas\path\to\file\file_name2.pdf.kopia-entry: The filename, directory name, or volume label syntax is incorrect. on path/to/file/file_name2.pdf
2022-06-23T07:19:54.294754Z ERROR restore ignored error copy file: CreateFile \\?\\\nas\path\to\file\file_name3.txt.kopia-entry: The filename, directory name, or volume label syntax is incorrect. on path/to/file/file_name3.txt
2022-06-23T07:19:55.663458Z ERROR restore ignored error copy file: CreateFile \\?\\\nas\path\to\file\file_name4.txt.kopia-entry: The filename, directory name, or volume label syntax is incorrect. on path/to/file/file_name4.txt
2022-06-23T08:00:07.704560Z ERROR restore ignored error copy file: CreateFile \\?\\\nas\path\to\file\file_name5.docx.kopia-entry: The filename, directory name, or volume label syntax is incorrect. on path/to/file/file_name5.docx
2022-06-23T08:03:16.293650Z ERROR restore ignored error copy file: error creating file: cannot create temp file: open \\?\\\nas\path\to\file\file_name6.exe.deploy542837994: The filename, directory name, or volume label syntax is incorrect. on path/to/file/file_name6.exe.deploy
2022-06-23T08:03:16.628422Z ERROR restore ignored error copy file: error creating file: cannot create temp file: open \\?\\\nas\path\to\file\file_name7.exe.manifest2367062697: The filename, directory name, or volume label syntax is incorrect. on path/to/file/file_name7.exe.manifest
2022-06-23T08:30:01.070672Z ERROR restore ignored error copy file: error creating file: cannot create temp file: open \\?\\\nas\path\to\file\file_name8.exe.deploy1528924582: The filename, directory name, or volume label syntax is incorrect. on path/to/file/file_name8.exe.deploy
2022-06-23T08:30:01.073881Z ERROR restore ignored error copy file: error creating file: cannot create temp file: open \\?\\\nas\path\to\file\file_name9.exe.manifest119476697: The filename, directory name, or volume label syntax is incorrect. on path/to/file/file_name9.exe.manifest
2022-06-23T09:29:56.845058Z ERROR restore ignored error copy file: CreateFile \\?\\\nas\path\to\file\file_name10.pdf.kopia-entry: The filename, directory name, or volume label syntax is incorrect. on path/to/file/file_name10.pdf
I am not sure if the issue is file path length, as some of the file paths have a total length of around 240 characters (less than 255) while others have a total length of around 260 characters (above 255).
Also, to be clear, the blobs for these files are not corrupt in the backup. The issue seems to be with the restore.
You can lower path length to verify if it long path issue or using of “raw path” isn’t handled correctly, if you would map first NAS drive to some drive letter, a N: for example and direct kopia to restore to N: drive instead of network path \\server
A few other tests I did to try to pin down what is happening:
(1) I tried doing the restore but without writing atomically. I stopped the restore when it threw the same error for the same file in my OP. I did not complete the restore (because it takes about 12 hours), so I am not sure if the errors for the other files would have persisted, but it looks like the issue may not be with the atomic library.
(2) Just for testing purposes, I create a new policy of just the one folder that contains the file that keeps erroring out. The idea here is to allow quick restores to try to pin down the issue, rather than having to wait hours. To my surprise, the problematic file restored just fine when restoring the test snapshot, without any errors. I tried this both atomically and not atomically. Same result.
I am not quite sure what is up, because I know the files that threw errors are not corrupt in my original snapshot. I’ve run snapshot verify --verify-files-percent=100 and I am able to successfully download the individual files from the snapshot. But the fact that the file restores fine after I created a new policy is weird. If it helps, the original policy backs up about 1TB of files. The new test policy only backed up about 100 MB of data.
(3) I deleted all my snapshots for my original policy but did not delete the underlying blobs (so Kopia will not reupload the data). I then rerun the snapshot and did the restore again. It started throwing the same errors for the same files as before, so I stopped the restore.
So, I didn’t do exactly what you suggest. Rather, I copied file_name.pdf and pasted it in the same source location, but made the name of the new file shorter by about 100 characters. Then I let the snapshot run and then restored the snapshot. file_name.pdf still throws an error but the exact same file but renamed to a shorter name does not throw any errors.
(1) I tried restoring from a different repo. Both the original repo and this new repo store identical snapshots. I restored to the same NAS. Same errors.
(2) I tried restoring from the original repo and snapshot to a local USB drive rather than NAS. To my surprise, the files that were throwing errors before restored just fine. So…???
Windows can do relatively good network handling only when it talk to “native” windows shares, but NAS(es) used mostly SAMBA that not always completely the same as “windows’es” SMB. My guess is that the case. The same kinda issues I saw when used cloud based mappings and not only with kopia but with other programs that actively using mapped cloud based storage.
You might want to try (if you have such option) to disable on NAS supports for CIFS completely and allow only SMB3
It is a Synology NAS and AFAIK it is already using SMB.
Further, I am not sure if that is exactly the issue. It seems to be part of the issue, but the issue is a bit more complicated because of point (2) I noted at Why is Kopia failing to restore file? - #8 by basldfalksjdf. In other words, restoring the problematic files to the NAS works just fine if I create a new policy. Perhaps the error is triggered when restoring to the NAS in combination with something else.
There’re 3 versions of SMB and since begging of this year Windows actively disabled on update CIFS and older SMB versions enforcing push to SMB3, so if NAS isn’t running latest fixes it might be a case.
Do you have any spare windows computer where you can share some folder and try to restore to that share instead of NAS, just to spot who is in charge of this issue