I’m using Kopia in a docker container on my home server to back up various devices around the house to one repository.
I’d like to make a copy of this central repository at another location, and I’m currently testing that with my linux desktop. Obviously, I would prefer to pull from the server than push from it. Using rsync does not work because my server uses ZFS and the destination file system is EXT4.
I’ve tried connecting from the CLI. I’m able to connect as
user@desktop, having created that user on the server already. However, trying to use
sync-to results in the
ERROR sync only supports directly-connected repositories. I’m guessing that’s because I’m not authenticated as the root server user (which was automatically assigned as
abc@kopia. Any thoughts on how I can back up my server’s kopia repository?
To sync the repository from your home server to the linux desktop on the linux desktop by
sync-to, a directly connection to both source and target repository of the home server is needed (that is, not by connecting via
kopia server). Use other ways like sftp and webdav.
If you don’t want to expose the repository directly via sftp or webdav and want to protect the repository behind kopia server, you can use
snapshot migrate command to copy snapshots from server to desktop.
sync-to need directly connection, while
snapshot migrate can copy snapshots from
kopia server connection.
sync-to can only mirror the whole state of whole repository, including creation and deletion of every snapshots (by every one), policy changes, snapshot changes, maintenance status, etc. Whether
--delete is used, deletion of snapshots is synchronized too.
snapshot migrate can copy any selection of snapshots or policies, and never delete snapshots that only exists on the target.
sync-to copy blobs and indexes from the filesystem directly (that is why it can only mirror repository), while
snapshot migrate read snapshots and policies from source repository and create same things in the target repository.
rsync does not care really about filesystem used. There is nothing stopping you from using it between ZFS and EXT4 filesystems.
I would only be not sure if using exFAT (as it does not support spare files). Still
rsync would work but could not probably use delta updates.
Rsync doesn’t care, but I think Kopia does? When I copied the repository folder over directly and tried to open it, I got
ERROR error connecting to repository: repository not initialized in the provided storage. I assumed this had to do with Jarek’s note on sharding.
No if you
rsync local to local. You said ZFS to EXT4…
Sharding can only play part when you copy local ↔ some cloud (like S3). But then you can not use
rsync for simple reason that it does not support cloud providers:)
Got it. Do you know what caused that error then?
This is weird… do you have
kopia.blobcfg.f files there? at destination.
I copied repos many times between different locations and never seen such error.
Are you trying to connect to it on the destination machine? or maybe on the source one? In that case maybe copia is confused that two repos have the same ID - the error might be side effect only.
Thanks for all this info! I want to mirror the repository, so it sounds like I want to use
sync-to rather than
Just to confirm. You’re saying that a “direct connection” is NOT between kopia servers but rather from one kopia server to a target filesystem. In order to mirror my repository, I would need to initiate the sync from the repository source (in this case, my server). This would require access to the target filesystem via sftp, webdav etc. Is that all correct?
Yes. Need sftp or webdav. Rclone is OK too.
I tried again from scratch and did not get the error.
In this case, I accessed my server’s filesystem via SMB and copied over the repository folder using my file explorer application on my desktop. Then using the Kopia CLI on my desktop, I ran ``kopia repository connect filesystem`. This connected successfully, after which I was able to view and mount the snapshots normally.
Thanks for the tip! Very strange to get that error though. I wonder in which cases it’s supposed to show up.
I really think that this error in this case is just side affect of some other problem - hard to guess without exact details what/how you rsynced it
If rsync have mirror the repository correctly, kopia should be able to open it like the original repository. Could you check if here is any wrong with the commandline (like a wrong path, same path on different machine, rsync ignored some files, etc)
Closing the loop for anyone reading this in the future.
I believe that the
ERROR sync only supports directly-connected repositories message was caused by an incomplete
rsync operation that left the repository in a partially updated state. In my case, rsync failed because it did not have read permissions for every folder and file in the repository.
The official docker image doesn’t currently support
UMASK. When taking a snapshot, it’ll create files and folders that can’t be read by any user other than
root. To ensure a successful transfer, you’ll either need to change the file permissions or give the remote access user sufficient privileges to copy the file.