I just published this early-stage design doc on support for notifications. Comments and suggestions are welcome.
opened 01:47AM - 20 May 22 UTC
design
app
htmlui
ux
## Background
There are situations where Kopia needs to communicate with the … user to let them know something important has happened.
Examples:
- data corruption detected
- snapshot failed
- maintenance failed
- an important upgrade is available
Today there's no good way to communicate such things to the user in the UI. Things are even worse in the CLI, since we don't know whether the user is looking at the output at all.
## Proposal
1. We introduce the notion of notifications, which will be stored in repository as `manifest` entries.
2. Notifications will persist in the repository until the user sees them, acknowledges them or until they expire.
3. Notifications can be addressed to: all users or a particular users on the machine.
4. Notifications will be associated with severity: normal, warning, error.
5. We will implement UX to show notifications to users in both the CLI and UI.
6. We will add flags and policy options with useful defaults to control when the notifications are emitted.
7. We should have a goal of not showing notifications to most users unless something really important or catastrophic has happened or if the user explicitly opts in to notifications.
8. In the future, we may also add ability to forward notifications via email, slack, pushover, pagerduty, etc. and via other integrations. This could be specified via user-level configuration or via policies.
## CLI behavior changes
At the end of each command execution in the CLI, if there are any pending notifications for the current user, Kopia will display (to stderr) a one-line summary similar to:
```
$ kopia snapshot create ...
<...>
Created snapshot ...
NOTICE: There are N pending notifications. To read them use kopia notifications.
```
Depending on severity, this will say something like.
* There are X pending notifications. To read them use `kopia notification list`
* There are X pending notifications including Y warnings and Z errors. To read them use `kopia notification list`
This behavior will be controlled by a flag:
--notifications=hidden | summary | details
* In `hidden` mode, nothing is displayed
* in `summary` mode the one-line notice is displayed if there are any notification (red if there are any errors). Also, any warnings or errors will cause exit code 1 to be returned. This will hopefully raise alerts in systems such as crontab and will alert appropriate system administrator who may otherwise be unaware of issues.
* In `details` mode, all notifications are directly displayed following regular output of a command
This behavior can also be configured using `KOPIA_NOTIFICATIONS` environment variable.
## Listing and acknowledging notifications
```
$ kopia notification list
2022-05-19 ERROR Maintenance failed on SOME_MACHINE
To see the details `kopia notification show aaaaaaaaa`
Go to https://kopia.io/docs/<some-page> for more information on resolving this issue.
2022-05-12 WARNING Snapshot failed on SOME_MACHINE
To see the details `kopia notification show bbbbbbbbb`
$ kopia notification ack --all | --item= aaaaaaaaa
All pending notifications have been acknowledged.
$ kopia notification list
There are no notifications.
```
Additionally we may support auto-acking:
```
$ kopia notification list --auto-ack
```
## UI changes
In the web UI, kopia will have a new tab showing notifications including some alert indicators on the main page. Details TBD.
In the KopiaUI kopia will additionally be showing incoming notifications as desktop popups.
## Notification sources
We will add new options to control emitting notifications.
1. Maintenance errors will by default notify the maintenance owner user, but this can be changed with `kopia maintenance set` flags. `--on-error-notify=none | default | username@hostname`
2. For `kopia snapshot create` we will add --on-error-notify=username@hostname
3. We will also add policy options and corresponding flags in `kopia policy set` to specify notification parameters for each snapshot source.
For example: `kopia policy set some-path --on-snapshot-error=notify --notify-user=someuser@somehost`
4. Later we will add additional configuration for notifying users about important software upgrades, etc.
Comments welcome, many of the ideas here are just initial thoughs to get the conversation started, not a complete design by any stretch.
If folks are interested in implementing parts of this design or the whole thing, I'll be happy to be help.
2 Likes
There is example for WARNING & ERROR but no example for NORMAL.SUCCESS or it may provide some simplest statistics also, like uploaded size, as well time taken to create particular snapshot?
As of now one have to parse output during snapshot and catching the last word “uploaded” to get number to be able to count and analyze snapshots pattern to predict future need for space on backend. While kopia repository status returning now Storage available: it isn’t what needed to predict space needed as well be able to predict best time for snapshoting by knowing how long approximately it would take time.