Different approach to handling multiple repositories

Feature Requests
1

Firstly, thank you so much for all the work on Kopia. I have been looking for a backup solution which is simple enough for end users and powerful enough for power users when required, and it looks like Kopia is it!

So far all my test have been good except for one thing which really threw me, and another which is a feature request:

  1. Please rename the “Disconnect” button under the Repository tab to something like “Delete repository definition”, because that’s what it does — it does not merely disconnect.
  2. TL;DR for the feature request (detail is below): repositories should be able to be marked as in/active, with a defined default for each/all. If repositories could have tags assigned, then it would be easy to de/activate sets of repositories, e.g. Client_1, Office, Home, Fast_connection.

Use case

The reason I clicked on “Disconnect” is because that is what I needed to do: disconnect from a specific repository and then later reconnect to it.

The reason is simple: I use my laptop both at work and at home. Each environment has its own repository/ies, and Kopia should not try in one environment to connect to any repositories from the other.If I cannot deactivate a repository, then two potential problems occur.

  1. If I need to connect to work from home via VPN, then Kopia sees the work SFTP repository and starts backing up work data over my very expensive and very limited connection from home. The same happens if I connect to a work Samba server and the repository definition which uses that sees it and also starts backing up, because it is made for LAN, not WAN use.
  2. I checked with Wireshark and Kopia continually tries to connect to all the defined SFTP repositories (I didn’t check other repository types). Yes, it it does so using an exponentially increasing timeout, but only up to a point (133 seconds) and then it starts all over again. Depending on the environment that can have my laptop blacklisted by a LAN server using something like fail2ban or by the egress firewall because it’s trying to get to another, non-work, subnet.

Feature request

[I am the only power user type person in my work environment, so this would need to work in the GUI as well as the CLI.]

Functionality

  1. Ability to mark repositories as in/active.
  2. Assign tags to repositories and be able to de/activate according to tag.
  3. Use repository names to name the related .config and .cache files, which would make it much less error-prone to use from the CLI.
  4. Separating log file names to include the repository name would also be helpful, though a bit of grep or awk is fine.

Implementation

On Linux the NetorkManager package has a workable tray icon approach:

  • Left-click is the user interface and brings up the list of pre-defined networks, showing the connectivity status of each. Clicking on one toggles its status.
  • Right-click is the admin interface and shows a list of actions such as editing/defining new connections, or showing connection information.

It would be helpful to colour each repository in that list according to its status, for example:

  • Green: Connected
  • Amber: Not connected, because I’m not supposed to be
  • Red: Should be connected, but can’t

I think that approach could be adapted for Kopia.

Once again, thank you so much for the work, and I hope that this feature request can be implemented, as I believe it will enhance Kopia’s potential use greatly.

2

I have written a bash script to help deal with the problems surrounding repositories which are unavailable to the UI.

If you want to code your own, here are the two relevant commands:

mv repository-<repo_name>.config repository-<repo_name>.config.INACTIVE

and

mv repository-<repo_name>.config.INACTIVE repository-<repo_name>.config

Of course, to be safe to use (this is about backups, after all) you need a lot of scaffolding, so here’s my script, which I called krepo and which is MIT-licenced. Share and enjoy.

#!/bin/bash

# ---

# The MIT License (MIT)
# 
# Copyright (c) 2024 John Botha
# 
# Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
# 
# The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
# 
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
# 

# ---

# This rests on the premise that Kopia ignores repository definitions with file names it sees as invalid. Thus we can hide repositories from Kopia so that unavailable ones aren't continually looked for on the network and don't clutter up the tray menu with useless entries.
#
# There is a fly in the ointment though: the file called "repository.config" must always exist. The reason I wrote this script is so that I can hide unavailable repositories, so to deal with this I made "repository.config" point to "$HOME/DONOTBACKUP/DUMMY-REPOSITORY/", which my standard ignore rules will ignore. I changed its description from "My Repository" to "DUMMY Repository", which helps minimise confusion.
#
# Real bummer: during testing I discovered that renaming "repository-XXX.config" to "INACTIVE.repository-XXX.config" would not hide it from Kopia, while renaming it to "repository-XXX.config.INACTIVE" would. It thus seems that its pattern matching is from the end. The reason this is a bummer is that if I could prepend "INACTIVE." then all the inactive repositories would be grouped together in a file listing, but now they're all over the place, making it more difficult to see at a glance.
#
# The script only works on repository files fitting this naming convention: "repository-*.config" and "repository-*.config.INACTIVE".

# ---

E_Help=11
E_Kopia=12;			T_Kopia="Kopia is running. Exiting."
E_Gone=13;			T_Gone=" does not exist. Exiting."

ConfDir="$HOME/.config/kopia"

# ---

ByeDie() {
  # Call with exit code to use, plus optional message.
  # If called with a message, display it on stdout. If in a terminal, wait for a key-press.
  STAGE_LEFT=$1
  1>&2 # All output to stderr from here to end
  shift
  # Display Text?
  [ $# -ge 1 ] && DT=1 && /usr/bin/echo "$@" || DT=0
  # Deal with being in a terminal
  [ -t 1 -a $STAGE_LEFT -ne 0 -a $DT -eq 1 ] \
  && read -n1 -sp "Press any key to exit..." JUNK && echo
  trap "" EXIT # Otherwise calling ByeDie() triggers the next trap (unexpected error)
  exit $STAGE_LEFT
} # ByeDie()

# This traps all errors, including those bypassing ByeDie.
trap \
'
  EC=$?
  [ $EC -eq 0 -o $EC -eq $E_Help ] \
  && ByeDie $EC \
  || ByeDie $EC Unexpected error caught: $EC
' EXIT

# ---

HelpBye() {
  cat <<EOH
SYNOPSIS
`basename $0` {activate|deactivate} {REPOSITORY_NAME}
`basename $0` {list}

Change the state of a Kopia repository (activate or deactivate) in $ConfDir. Alternatively, list the repositories and whether they are marked as INACTIVE.

To minimise the chance of disaster, no Kopia processes may be running when you de/activate a repository.
EOH
  exit $E_Help
} # HelpBye()

#---

ActivateRepository() {
  rfa="repository-$1.config"  # Repository file, active
  [ -s $rfa ] && exit 0  #  Already activated

  rfi="repository-$1.config.INACTIVE"  # Repository file, inactive
  [ ! -s $rfi ] && ByeDie $E_Gone Repository $1 $T_Gone  # Deactivated file does not exist.

  mv $rfi $rfa
} # ActivateRepository()

# ---

DeactivateRepository() {
  rfi="repository-$1.config.INACTIVE"  # Repository file, inactive
  [ -s $rfi ] && exit 0  # Already deactivated

  rfa="repository-$1.config"  # Repository file, active
  [ ! -f $rfa ] && ByeDie $E_Gone Reposotory $1 $T_Gone  # File does not exist

  mv $rfa $rfi
} # DeactivateRepository

# ---

ListRepositories() {
  for f in repository-*.config.INACTIVE repository-*.config
  do
    [ ! -e $f ] && continue  # Nothing to see here; move along.
    fn="`echo $f | cut -d - -f2`"
	if [[ "$fn" == *.INACTIVE ]] ; then
	  fn="`basename $fn .config.INACTIVE`"
	  echo $fn.INACTIVE
    else
	  basename $fn .config
	fi
  done
} # ListRepositories()

# ---

CanRun() {
  pgrep kopia >/dev/null 2>&1
  [ $? -eq 0 ] && ByeDie $E_Kopia $T_Kopia
  return 0
} # CanRun()

# ---

cd $ConfDir

case $1 in
  activate)
	CanRun && ActivateRepository $2
    ;;
  deactivate)
	CanRun && DeactivateRepository $2
    ;;
  list)
	ListRepositories
    ;;
  *)
	HelpBye
    ;;
esac