Centralized backup of multiple servers and ransomware protection

General
1

Centralized backup of multiple servers and ransomware protection

Hi everyone,

I’d like you to review the following solution I came to, revealing weak points and/or suggesting better ways to achieve the same. Thanks.

I know the canonical way of backing up using Kopia is to let source servers run their own Kopia instance, and send backups to a single repository, or to a Kopia server. But such approach makes it difficult for me to meet the following requirements, I’ll highlight the problematic ones:

  1. I need to backup several servers, residing in DMZ and LAN.
  2. Just one backup a day, during the night.
  3. The main repository should be local (a secondary cloud repository is optional).
  4. DMZ to LAN connections are denied, LAN to DMZ are allowed.
  5. The main repository is a set of USB hard disks, rotated daily: one of them stays connected to the backup server, the others stay in a safe.
  6. I need to send a single daily backup report: positive in case all of the backups went fine, negative in case at least one backup failed (read error on any source file, no free space on the repository, etc.) or the repository verification failed.
  7. To protect the backups from ransomwares trying to delete previous snapshots, backup clients shouldn’t be able to delete or modify in any way the old snapshots.

I know, #7 could be achieved by using “append only” ACL on a Kopia server.

#6 could be achieved by parsing the repository snapshots list and reporting if all the required snapshots are present or not.

But I can’t find a solution to #4… unless I completely change the backup approach.

Here is my idea…

I setup a Linux “backup server” like this:

  • Residing in LAN.
  • Exclusively dedicated to backing up all of the LAN and DMZ servers.
  • No TCP listening services running, except for SSH (for administration purposes), accepting public key authentication only.
  • It has the USB hard disk repository attached, daily rotated.
  • It has SSH access credentials to all of the Linux servers it needs to backup, so that it can mount them via sshfs. SSH authentication is done via public key.
  • It has Windows user credentials for all of the Windows shared folders it has to backup, so that it can mount them via cifs.
  • It runs Kopia, of course! :slight_smile:

And here is a “pseudo code” script doing the job:

$error = false

mount USB_hard_disk /mnt/repository
if mount_exit_status != 0
    $error = true
    skip to to the end of the script, just before notification

kopia repository connect filesystem --path=/mnt/repository
if kopia_exit_status != 0
    $error = true
    skip to to the end of the script, just before notification

for $source in $backup_sources_list:
    case $source
        is Linux:
            mount -t sshfs $source /mnt/$source
        is Windows:
            mount -t cifs $source /mnt/$source
    if mount_exit_status != 0
        $error = true
    else
        kopia snapshot create /mnt/$source
        if kopia_exit_status != 0
            $error = true

kopia snapshot verify
if kopia_exit_status != 0
    $error = true

umount USB_hard_disk_filesystem
if umount_exit_status != 0
    $error = true

if $error == false
    send notification "relax, backup is ok!"
else
    send notification "there were errors, please review the logs"

So, do you think it will work? Any weak spots? Any improvement ideas? Any completely different way to achieve the same? Thanks.

2

Question: Kopia needs to access the most recent version of the repository in oder to efficiently create a new snapshot, that is, in order to deduplicate contents against the prior snapshot. Does rotating the drives have an effect on the performance or correctness of the backups? Are contents across drives synchronized in any form?

3

That’s a very good question! In fact, to be on the safe side, I was thinking of creating a different repository on each disk: this way there should be no risk of messing things up.

4

You could use sshfs ineed. You would connect from your LAN-based backup server to your DMZ-machine and mount the paths you wish backed up (e.g. /etc) locally on your backup server. Snapshot. Umount. The Kopia feature of ‘actions’ allows pre- and post-snapshot scripts to be ran which could accomplish this. From the 0.8.0 release notes " Added support for Actions which allow running custom scripts before and after a directory is snapshotted with lots of useful applications."

Another approach could be to set a firewall rule from DMZ → LAN only allowing port 51515/tcp (TLS-enabled) from the DMZ IP address, possibly scheduled and ACL-limited. This is what also works well for remote VPSes. Once scheduling at a specific point in time is possible in Kopia (now it’s only ‘every x seconds’ I believe) you could also limit the firewall rule to the timeframe needed to perform the backup. An attacker would have a very very small attack surface but defies your core question of not allowing any traffic.