I’m new to Kopia CLI and Linux (usually a Windows sysadmin). I have data on a Debian LXC server that I want to back up to both a Windows SMB share and Backblaze B2.
I can manually snapshot to the SMB share. I need the best approach to also back up to B2 and automate backups to both destinations daily.
Proposed Automation Plan (Bash Script):
Connect to Windows SMB repository, run kopia snapshot create /path/to/data.
Disconnect from SMB repository.
Connect to B2 repository, run kopia snapshot create /path/to/data.
Disconnect from B2 repository.
My Questions:
Is this sequential connect/snapshot/disconnect approach the standard or recommended way to back up the same source to multiple distinct Kopia repositories?
What’s the best practice for automating this script on Debian (cron, systemd-timer)? Are example scripts for repository connect/snapshot/disconnect available?
I’m confused about policies (kopia policy set --global ...). My understanding is that scheduled snapshots only work if the Kopia UI is running (and I’m using Kopa CLI). Does that mean retention policies wont apply either and I need to manually recreate Kopia’s default retention policies in the bash script?
First things first: I’m still working out, around the numerous rough edges of this software before I begin full shake down & burn in tests but these questions are pretty basic though quite important when trying to get oriented. I’ve already set similar confs up so:
SOP would be to keep all repos connected unless there’s a specific need to disconnect (eg: failed drive on target). You can specify which repo to run a snapshot against via kopia snapshot --path="$path" --config-file="$HOME/.config/kopia/repository-N.config"
Use systemd-timers for server/desktop. Cron is going to eventually be phased out as SystemD is more mainstream than ever on mainline distros. It looks to instead be regulated more & more to embedded Linux. For SystemD you might want to give timertab a punt.
There’s a Kopia policy you can set to regulate snapshot frequency. Eg: --snapshot-interval="10m"
--ignore-identical-snapshots="true" (should) ensure you’re not making needless writes of metadata for identical snapshots.
You’ll have to write your own pre & post actions shell scripts.
Kopia Actions support bash arguments so you can save considerable effort when having to use the same script for multiple repos. I won’t write an action-related sh without them.
Keep it all in git before publishing for deployment. You’ll thank yourself later.
Highly recommended: set up powerline-go; it really helps navigating git repos & makes statuses/dirs much more readable
Set up zoxide; cd $path is a waste of time, keystrokes.
Kopia UI is merely Kopia CLI with a Electron wrapper. Note it does not seem to refresh when making changes via the CLI unless you relaunch Kopia UI.
Once global policies are applied they are to apply to any child repos (via 'inherit`). That can be toggled per-repo if needed (supposedly).
Be sure to give a quick read of my thread of a related issue.
You may want to check what sort of files you are going to back up. There’s no sense compressing what’s already compressed. See below.
If you want full access to Kopia’s advertised capabilities you’ll need the CLI. The GUI does not have feature parity.
If you have the option for a ZFS file system do it. Otherwise trial BTRFS starting with blake2b until you find an acceptable perf v hash rate.
Never update Kopia in situ, in production. I’ve read there’s been at least one repo breaking change (IIRC) but cannot recall the source ATM. Regardless be sure every endpoint has its ‘auto-update’ file removed (rm $HOME/.config/kopia/*.update-info.json). I block it from communicating to any Google (which hosts most golang packages which Kopia itself is programmed in) & GitHub ASN in the firewall to help as extra insurance.
Keep a separate browser up for the docs & keep a log while confirming your changes while testing, bench marking, confirming, reproducing, deploying. Be sure to document/cite your sources.