Backup Incomplete

Support
1

Hello,

First of all, thank you for Kopia project and for the work already done on this project! It is promising :slight_smile:

I am evaluating this solution for backup (compared to restic). I have done a few successful test, so I tested with a larger repository of about 90GB with 55’000 files and 11’000 folders.

I used the KopiaUI-0.7.3-win, with the following policy (for *):

Compression: zstd
Never compress extensions:

.zip
.ZIP
.7z
.7Z
.rar
.RAR
.pdf
.PDF
.png
.PNG
.jpg
.JPG
.jpeg
.JPEG
.webp
.WEBP
.avi
.AVI
.mp3
.MP3
.mp4
.MP4
.mkv
.MKV
.ogg
.OGG
.opus
.OPUS
.gif
.GIF
.dropbox
.dropbox.attr
.dropbox.cache

Files - Ignore rules:

Thumbs.db
Thumbs.db:encryptable
ehthumbs.db
ehthumbs_vista.db
.stackdump
[Dd]esktop.ini
$RECYCLE.BIN/
.lnk
~.
.~lock*
.tmp
~$.doc*
Backup of .doc
~$.xls
.xlk
~$.ppt*
.~vsd
sync.ffs_db
.dropbox
.dropbox.attr
.dropbox.cache
.fuse_hidden*
.directory
.Trash-*
.nfs*
.DS_Store
.AppleDouble
.LSOverride
.DocumentRevisions-V100
.fseventsd
.Spotlight-V100
.TemporaryItems
.Trashes
.VolumeIcon.icns
.com.apple.timemachine.donotpresent
.AppleDB
.AppleDesktop
Network Trash Folder
Temporary Items
.apdisk
__FREEFILESYNC/**
__FREEFILESYNC/

The result is an “Incomplete” backup.

On the root of the source directory, I have 6 folders. Only 1 is backup and not completely (for about 57.5GB, 34591 files and 6270 folders, instead of 68GB, 39877 files and 6701 folder).

I tried to “Snapshot now” a second time without better success.

Can I do something to try to help debugging it?

Thanks and best regards,

Fabien

2

Cn you post a screenshot or even better results of

$ kopia snap ls -a -l -i
3

Here it is. Thanks.

FB@lp59:C:\Users\FB\Desktop\My Stuff
2020-12-07 10:38:45 CET k9affdf7a1e832d54055960615de9c82b 5.6 GB drwxrwxrwx files:4276 dirs:812 (latest-1,annual-1,monthly-1,weekly-1,daily-1,hourly-1)

FB@lp59:T:
2020-12-07 12:49:38 CET k2f29502bbce84c03a31f3543bb906eb4 incomplete:checkpoint 57.5 GB drwxrwxrwx files:34591 dirs:6270 (incomplete)
Running quick maintenance…
Rewriting contents from short packs…
Looking for unreferenced blobs…
Deleted total 4 unreferenced blobs (7.3 MB)
Compacting indexes…
Finished quick maintenance.

4

So if you snapshot again, what happens (Kopia snapshot —all)

5

Here is the result, using the command line, this time.

Note that I have added a “.kopiaignore” file on “T:” (the 2nd backup source) now, with the same content of the ignore rules mentioned on my first post.

kopia.exe snapshot --all

Snapshotting FB@lp59:C:\Users\FB\Desktop\My Stuff …

  • 0 hashing, 0 hashed (0 B), 4276 cached (5.6 GB), 3 uploaded (4.5 KB), 0 errors, estimated 5.6 GB (100.0%) 0s left
    Created snapshot with root k9affdf7a1e832d54055960615de9c82b and ID c4b33c71ee8a83e57aa674a4ca398a1f in 2s
    Snapshotting FB@lp59:T:\ …
    | 0 hashing, 23 hashed (13.8 KB), 34564 cached (52.4 GB), 0 uploaded (0 B), 0 errors, estimated 89.2 GB (58.7%) 8s left
    / 4 hashing, 23 hashed (57.3 MB), 34590 cached (55 GB), 0 uploaded (0 B), 0 errors, estimated 89.2 GB (61.7%) 8s left
    / 2 hashing, 179 hashed (10.1 GB), 34590 cached (55 GB), 0 uploaded (0 B), 0 errors, estimated 89.2 GB (73.0%) 24s left
    (…)
    | 4 hashing, 18798 hashed (34.1 GB), 34590 cached (55 GB), 0 uploaded (0 B), 0 errors, estimated 89.2 GB (99.9%) 0s left
    / 1 hashing, 18816 hashed (34.1 GB), 34590 cached (55 GB), 0 uploaded (0 B), 0 errors, estimated 89.2 GB (99.9%) 0s left
    / 4 hashing, 18854 hashed (34.1 GB), 34590 cached (55 GB), 0 uploaded (0 B), 0 errors, estimated 89.2 GB (99.9%) 0s left
    \ 4 hashing, 18923 hashed (34.1 GB), 34590 cached (55 GB), 0 uploaded (0 B), 0 errors, estimated 89.2 GB (100.0%) 0s left
    | 4 hashing, 18941 hashed (34.2 GB), 34590 cached (55 GB), 0 uploaded (0 B), 0 errors, estimated 89.2 GB (100.0%) 0s left
    / 0 hashing, 18954 hashed (34.2 GB), 34590 cached (55 GB), 1 uploaded (438.4 KB), 0 errors, estimated 89.2 GB (100.0%) 0s left
    kopia.exe: error: encountered 1 errors:
    unable to process directory “System Volume Information”: open T:\System Volume Information: Access is denied., try --help

I think that the cause is this “System Volume Information” folder in the root of a Windows mount point, which is not accessible.

I’ll try again with this folder added to my ignore list. Could it be “hard coded” ignored?

Thanks and regards :slight_smile:

6

You can also ignore read errors. That may help get the first snapshot going.

7

Thank you very much.

Could I have identified the reason for this incomplete backup and which file or folder caused this problem, from a log file, knowing that I started initially the snapshot from the user interface (KopiaUI)?