I have a local repository with a lot of users and I want to make a continuously updated remote copy of it.
To protect from ransomware, I want to do it in a pull fashion, without storing credentials for remote server with r/w access on local machine.
I know about sync-to command, but, unfortunately, I can’t run it on remote host because it works only with local repos. I’m also aware of object locks for S3 storage, but using it doesn’t quite fit my case.
I’d like to ask: is it possible to continuously replicate contents of repository directory to another machine (with a tool like Syncthing or something like that)?
Wouldn’t there be any problem if there is a snapshot or a maintenance task running on repository during replication?
I’m OK with losing the most recent snapshots which weren’t fully replicated yet before something bad happened to the local repo, but won’t there be any problems with the older ones?
Also, is Kopia repository crash-consistent? Would Kopia read repo’s content without problems if there was a power outage during backup/maintenance going on, or if storage was restored from a filesystem snapshot?
First of all - this is not how kopia works. There is no thing as continous replication with kopia. 2nd, kopia repo sync-to, of course, works also with remote repos, I am doing it on a daily basis, where I sync my local repo to a S3 bucket.
@budy Maybe I expressed my intention not clear enough.
What I want is to basically initiate replication from external server.
I know that I can run sync-to from my local repository machine, buy this means I have to store credentials to external storage there, so if this machine is compromised, I can lose external backup too.
I tried to run sync-to on remote server, while connected to repository on local machine, but I get this error: ERROR sync only supports directly-connected repositories.
Your main issue is, that the repos for local storage and remote (S3) storage have different layouts, so you can’t simply copy those over. However, what about a write-only user on your S3 repo and then having your local repo be synched over? Then you’ll have another user on the S3 repo to perform the maintenance - on a different host, probably. That should keep any ransome malware from messing with your good data in the S3 repo.
As for now, I am not using S3 anywhere.
Both local and remote storage I use are just Linux boxes, which can communicate over SSH/SFTP/another protocol if needed.
I guess, I could install something like Minio on remote host, but this would definitely take some time to set up properly.
Well, regardless of the protocol used… the solution with a add-only user should resolve your concerns about someone malicious destroying or encrypting all your old snapshots - which I guess would probably no one bother to do anyway, since kopia is not wide-spread and thus not a juicy target.
the solution with a add-only user should resolve your concerns about someone malicious destroying or encrypting all your old snapshots
Yes, that’s what Syncthing can do - it allows to set retention rules for files on remote host.
I just don’t know if continuous synchronization of Kopia filesystem repository to another host would produce consistent result. I mean, if my local storage does down in the middle of most recent snapshot getting replicated to remote server, would Kopia be able to restore a snapshot prior to the most recent from such not fully synchronized repo. That’s the main question.
No, syncthing will probably damage your kopia repo, by deleting blobs, which fall out of it’s retension, when the repo might still depend on it. You need to have kopia perform the maintenance on its repo objects, that’s paramount.
By “retention rules” of Syncthing I meant versioning. It can store different versions of files for the last N days, but it won’t delete it by itself - only if they were removed from the source and all the previous versions are expired.
So, if ransomware encrypts the blobs, it’s possible to restore the original files.
Well, this is inherent also true for a remote kopia repo, since the malware simply can’t get to it via the os of the infected victim - it would have to use kopia’s api to do so - veery unlikely, actually on the verge of impossible, I’d say. The malware wouldn’t even know, that the repo exists. And even if it did - as long as you local kopia client has write-only access, nothing could be encrypted in the repo.