Policy Schedule

Support
1

Is there any further documentation detail on what the policy scheduling commands are actually doing? Ideally I would like to set a schedule timer using systemd to run a script that runs various tasks such as kopia snapshsot create, kopia snapshot verify. That way I can wake computers at the appropriate time also. I guess I could do that now independently of the two schedule flags discussed below.

When I set the policy

kopia policy set \
	  --global \
	  --snapshot-time 06:00

what is this actually doing in the background?

If I have --snapshot-time 06:00 set, does that negate the need to use --snapshot-interval 24h0m0s, or should they be used in combination?

Also there are no examples of time format to use with --snapshot-interval but I did stumble on a suggestion that e.g. 24h0m0s is okay but 7d is not when reading the output errors.

2

Schedule parameters only take effect when you use kopia server or KopiaUI and is mostly helpful on Windows, where neither cron nor systemd are available. If you want to use systemd timers, that’s totally ok. I personally use crontab and run kopia snaphot create --all twice/hour.

3

Hey,

Seeing you write " Schedule parameters only take effect when you use kopia server or KopiaUI" is assuring, some text in KopiaUI gave me a different impression, which led me to search online.

This is the text under Snapshot Frequency in KopiaUI:
“How frequently to create snapshots in KopiaUI or Kopia server (has no effect outside of the server mode)”

The part in parentheses made me think creating snapshots automatically might only work in server mode, instead of both server mode and the UI.

4

KopiaUi is a Kopia Server in disguise though…

5

This is news to me. I’m installing Kopia CLI this weekend after using Kopia UI for years and I always thought Kopia CLI had a scheduler service built in. Does it still use policies in pure CLI mode? In other words if I had backups scheduled for weird times like 9:03am, 7:47pm, etc. would I need to run a cron task every minute to make that happen on time?

Do I need to run cron tasks for maintenance as well? Every 30 minutes, and leave it up to the policy whether or not it actually does anything?

6

The policy schedule will apply and be used by any running Kopia server. However, since Kopia server does not pull data, this doesn’t work with remote Kopia clients. Those have to be managed by something like cron.

Remeber that KopiaUI is also “just” a Kopia server in disguise.

7

Now that I’ve tested Kopia CLI, I realized a lot of my assumptions from reading the docs are incorrect. :melting_face:

Can anyone please confirm these points and let me know if I’m thinking about it correctly now?

  • Kopia CLI does not run as a service; it just does one-shot commands that you manually invoke
  • Kopia CLI is not aware of schedule policies, but it is aware of every other policy? (Ignored files, retention, etc.)
  • Maintenance runs automatically after a snapshot so you don’t need a cron for maintenance
  • Kopia CLI snapshots don’t respect your schedule policy, but maintenance vs. full maintenance does follow an internal schedule policy
  • The web UI is only available for Kopia server and not Kopia CLI
  • Kopia server does run as a service and has a scheduler that respects schedule policies
  • Kopia server is intended for multi-user setups because you can’t access snapshots from other clients?
  • The documentation doesn’t mention that you need to set up a cronjob or a systemd timer task or snapshots and maintenance will not run

I’m backing up my desktop (Kopia UI), my laptop (Kopia UI), and my home server (Kopia CLI). I want to be able to access the snapshots from all 3 machines on any of the machines. Is that not possible if I use Kopia in server mode?

The main thing that I think would help immensely here is if Kopia CLI had a schedule-aware snapshot command like kopia snapshot create-overdue or something, which would run the snapshots that are due to run according to the policy I defined. Is there a feature request somewhere I can add a “+1” to? :smile:

Or should I just be running Kopia in server mode? There’s a disconnect in features between CLI, UI, and Server that seems needlessly confusing at the moment.

8

Well…you’re right with all of your assumptions. I think things start to clear up, once you realize, that Kopia CLI is just a CLI client for Kopia, nothing more. It will carry out any action you want it to against a “connected” Kopia repo, given that you don’t have an error in your invocation.

Kopia Server is, as you noticed, a service, which is also at the heart of Kopia UI. So Kopia UI is a GUI to a running Kopia server, which usually is bundled and started with Kopia UI. Its the service that handles all the scheduling, while Kopia CLI always respects and adheres to the policies stored in the repo.

Since you can connect multiple servers to the same repo, just as you can run different Kopia UIs against the same repo, you can always run the Kopia service on, say your Linux host, to benefit from automatic scheduling - I tend not to do that and have cron handle that for me using Kopia CLI.

Regarding access to different user backups: you can grant access via Kopia Server’s ACL instructions to achieve that and there’s the all-mighty repo UI user, which can access all snapshots - that’s what I am using, if I want to browser all my snapshots. I wouldn’t grant access to my different “agents” though, since I’d like to keep that clean.

As for your FR… that doesn’t work with Kopia CLI. Again, you could do that using Kopia Server. What would the use case for this be anyway? Kopia Server would run snapshots as soon, as they’re due - thats what KopiaUI does as well…

9

I still don’t see why it can’t be a feature in Kopia CLI; If we wanted to DIY, couldn’t we use eg:

policy show --global --json | jq -r '.scheduling'

To get our scheduling policies:

{
  "timeOfDay": [
    {
      "hour": 6,
      "min": 0
    }
  ],
  "runMissed": true
}

Parse that JSON to know when the snapshot should run, then check recent snapshots to see if it’s overdue? If overdue: run it.

Edit: to be clear, I meant we’d still invoke it manually, perhaps on a cron every few minutes, but it wouldn’t do anything unless there are snapshots due to run.

I used KopiaUI before I ever tried CLI, so I had a lot of incorrect assumptions that are now clearing up, but I still think the disconnect between some things available only for Server/UI and not in the CLI doesn’t make sense. And the docs could dispel a lot of confusion if they had some common usage examples for single/multi user use cases.

For my “local only / single user” use case, I like not having to worry about ACLs, but I wish there was a schedule policy-aware snapshot command so I could use my schedules I set up in Kopia UI.

10

I’d say, you’re overthinking this issue. As soon as cron is involved, you will have to know, when your snapshot should be taken, no? So if you have to use cron anyway… there’s no reason for Kopia to check the schedule policy, because you want it to take a snapshot - otherwise, you wouldn’t have invoked it.

The only possible use case would be, if Kopia would always run a snapshot when being invoked and you’d use the repo’s schedule policy to keep Kopia from doing so. The only benefit would be, that you wouldn’t have to concern yourself with changing your cron config, if you change the schedule policy, but come on…

Kopia is actually all about Kopia server. Just install Kopia server on that client and connect it to the repo, that would do what you want - regard Kopia CLI as a tool, that also happens to be able to run snapshots, but its basically more for maintenance stuff…

11

The workaround I was describing would have a cron running every 10 minutes or so, and calling this imaginary “snapshot run-overdue” CLI command that would only run if it’s due to run according to the policy. Effectively just polling for any overdue snapshots in the repo all day long.

I bet you’re right: I probably am overthinking it, and if server mode is the intended use, even for single-user setups, I’ll look into doing that this weekend. I was just avoiding it because I’m already having enough trouble understanding the fundamentals from the current documentation.

Is it the kind of thing I’d configure and then set up with systemd to auto-run when the machine boots? And then it takes care of itself after that?

If my other machines already connect and run just fine with KopiaUI, do I need to change them to connect to the server instead? (if not required, is there a benefit to doing so?)

I suppose if it doesn’t work, I can always shut down the server and go back to cron + CLI until I make sense of it…

12

Having queried this topic when starting out with Kopia, I have since implemented what @jkowalski suggested purely using CLI and systemd scheduling.

I am able to wake sleeping computers before the backup schedule, and backup to a central local network storage.

For a novice like my self just focusing on the CLI commands made sense to me, as does systemd scheduling, which of course if very flexible and replaced anything cron.

All the CLI commands are implemented using bash scripts, which are called by systemd.
Never tried using the server feature as I didn’t think it was needed or relevant, but I might read up on it again to see if it has any benefit, but I doubt I would need it over and above what I’ve achieved without it.

13

Yeah, I agree!! Running the server on a Kopia client is just over-engineering. I was just trying to layout exactly that. Just because KopiaUI has a Kopia Server running underneath doesn’t mean, that you should do that on a Linux box, or a headless setup.

14

I’ve cleared up more of my own confusion through trial and error :sweat_smile:
I mentioned the documentation has some real gaps, and for what it’s worth, ChatGPT and Claude couldn’t make sense of it either! I think I was even more confused by so many folks on the forums talking about Kopia Server, while the author of the software itself said he’s just using the CLI command with a cron.

  • Kopia server is pretty much a headless version of KopiaUI, as @budy said :+1:
  • You can use it as a Kopia client to connect to a repo, the same as Kopia UI
  • You can also connect other clients to the Kopia server and use ACLs to proxy their access to the repo, but:
  • The ACL/proxy stuff is optional

The key was to think about Kopia server as a stand-in for Kopia UI, and ignore the features related to the repo proxy. I finished with the setup I was hoping for, but I’d appreciate a sanity check from the experts:

I created htpasswd for the basic auth protecting the web UI, and control.env for the password protecting the “server control” API (though I doubt I’ll use the server control commands when the web UI is available.)

Registered a systemd service for each repo I want to connect to:

[Unit]
Description=Kopia server (Backblaze B2 repo)
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=me
EnvironmentFile=/home/me/.config/kopia/control.env
ExecStart=/usr/bin/kopia --config-file=/home/me/.config/kopia/backblaze-b2.config server start --address=http://0.0.0.0:51517 --insecure --htpasswd-file=/home/me/.config/kopia/htpasswd
Restart=on-failure
RestartSec=10

[Install]
WantedBy=multi-user.target

Now, I can access the web UI from any browser on my network at http://myserver.local:51517 with username kopia and the password I set in my htpasswd file.
I can run server control commands with the control password:

source ~/.config/kopia/control.env
kopia server status --address=http://127.0.0.1:51517 --server-control-password="$KOPIA_SERVER_CONTROL_PASSWORD"

And I can shut down the server and run normal CLI commands too, if I ever need to.

(Repeat with different kopia config files and different ports for my other repos.)

Now my snapshots run according to the schedule policies I set up, and I can access the snapshots from my other devices, just like the setup I had with Kopia UI :+1: