(newbie questions) Backing up KVM VMs

General Topics
1

Starting over with kopia from scratch, having my first small setup with an sftp repo on one server and kopia writing there from 3 other servers. Basics work fine, I still learn about the concepts etc

On one server I run ~5 VMs with QEMU/KVM. The VMs are mostly LVM-based, their virtual disks are raw Logical Volumes in terms of LVM. I have an old and reliable script that pauses the VMs, does LVM-snapshots, unpauses the VMs and archives the snapshots to a separate directory, then syncs it over to the backup server.

Now I think of optimizing this with kopia.

So far I would keep the old script in place and let it create compressed tarballs in a local directory /mnt/virt-backup. Each VM gets its own subdir there.

From there I would create kopia snapshots to the remote sftp-repo.

I wonder if it makes sense to create one big snapshot of /mnt/virt-backup? Or is it better to do separate snapshots of

  1. /mnt/virt-backup/VM1
  2. /mnt/virt-backup/VM2
  3. /mnt/virt-backup/VM3

?

I think of writing some wrapper (bash-)script and call it via systemd-timer, btw. As far as I understand it’s good (or necessary?) to add a connect-command at first, right? Or does kopia connect by itself, just by reading the config-file?

Should I maybe skip compression in my initial backup script and better let kopia compress things?

As soon as I have this in place, I will also add another stage and sync the sftp-repository to a GCS-repository. Only mentioning, not in place yet (I wait for an admin to assign me a bucket, different story).

So far I haven’t fully understood all the features of kopia’s policies. I assume they will help me to maybe only sync selected snapshots to GCS etc … looking forward to all these possibilities.

Help and hints welcome, thanks all for this great software.

2

Yes - 100%. so you will benefit from deduplication. E.g. if you have the same big file in more than one VM kopia will deduplicate it nicely saving you remote storage requirement.

3

Unrelated to Kopia, but (somewhat) on-topic… do you use any manager for our KVM VMs? If not, I’d suggest to take a look at Promox VE. It is free and its a wonderful, complete solution for managing KVM VMs.

Plus… and that’s the “on-topic” part… they also offer the PBS Proxmox Backup Server (also free) which also uses the content-addressable-storage setup, which Kopia also uses. I am running it at work for three clusters and at home for my personal needs

4

@budy thanks for your suggestions. The servers are rather basic. Currently moving from older gentoo-based servers to debian, just using qemu, libvirtd etc. I use virt-manager as a GUI on my desktop/laptop to manage the VMs. The VMs are rather static, no big management to do so I don’t feel like adding anything if not really necessary.

Currently my “draft” is to use Kopia plus Wasabi S3 storage, maybe with sftp-storage as a local stage.
Setting up tests right now and learning all the concepts etc …

Changing a backup solution at a customer is scary: you have to be able to fully trust that shiny new thing, but as long as your are still learning to use that tool, it’s a bit thrilling :wink:

5

Sure, just wanted to make sure, you know about PVE and just since both apps deploy the same kind of backup storage.

6

I am starting over now with a fresh sftp-repo.

What I don’t fully understand yet:

How to set up things if I want a second gcs-repo to sync to? How do I create that second repository? Is creating that repo simply adding it as another repository along the first sftp-repo?

I want to avoid overwriting something of the config.

While I type this it seems to get clearer anyway, but asking still helps :smirk:

7

The most efficient way of cloning your repo is to use Kopia’s repo sync command. It will take care of different repo typs, e.g. cloud-based (flat) repos vs. repos based on regular file systems.

8

sure, I understand that.

But I don’t yet understand the handling of multiple repositories, as I asked in Multiple repositories - newbie doesn't understand yet

9

Okay, so… you have setup a sftp repo. You can use primarily that one with your clients and then run a repo sync-to on your kopia server, which hosts the sftp repo. Or, if you should be running Kopia entirely on your VM host, have it perform the repo sync.

You will have to create a repo on GCS of course, but you can do this using KopiaUI and then use the credentials/repo password which you used to set the offsite repo up.

I myself, like to use the cli client better, but that’s just a personal preference.

10

please share or point to an example of actual commands …

How to switch between the repos, how to sync from a to b etc.

edit: cli preferred here, too.