Synchronization: ERROR destination repository does not have a format blob

[Jan_Reinhardt]

I have learnt a lot about Kopia’s concepts by now. Still there is one thing Im not sure about: my KopiaUI holds connection to one repository. If I‘m saving all my snapshots to one remote machine, this is by design not a 3-2–1 backup strategy?
Kopia has no way to regularly backup to 2 different repositories, right? The way Kopia deals with this is by Synchronization, which fully syncs a complete (cloud-) repository with another (local) storage.

So when Im typing

kopia repository sync-to filesystem --path /repository/Online-Speicher --must-exist --delete

to get my synchronization going, Im getting this error message:

ERROR destination repository does not have a format blob

What am I doing wrong? Do I need to open up an empty repository at the destination?
Thank you, Jan

P.S. You cannot enable Synchronization in KopiaUI, only in Kopia CLI?
(= its hidden from newbie‘s eyes)

[dimejo]

@Jan_Reinhardt

If I‘m saving all my snapshots to one remote machine, this is by design not a 3-2–1 backup strategy?

Correct. The 3-2-1 rule advises to have two different types of media (or backends) for storage.

@Jan_Reinhardt

Kopia has no way to regularly backup to 2 different repositories, right?

KopiaUI can be connected to multiple repositories at the same time. When you right click on the Kopia icon in the system tray you will see an “Connect to Another Repository” option. This option allows yout to configure a 2nd (or 3rd or whatever) repository with completely independent settings.

@Jan_Reinhardt

What am I doing wrong? Do I need to open up an empty repository at the destination?

The sync-to command does not create a repository, it needs to exists.

[Jan_Reinhardt]

I’m running Kopia on a headless Linux machine, so there is no system tray icon. Just the KopiaUI in my webbrowser when i access remotely.

So what I managed to do now: I copied two files from the main repository to the other destination: kopia.repository.f and kopia.blobcfg.f

Then I typed kopia repository sync-to […] again, and things started magically working. 2,2 GB of repository data were synced.

Will this keep working automatically (= stay synced) with future snapshots?

[dimejo]

@Jan_Reinhardt

I’m running Kopia on a headless Linux machine, so there is no system tray icon. Just the KopiaUI in my webbrowser when i access remotely.

I assumed that you meant the GUI because the GUI binary is actually named KopiaUI.

The sync-to command can be great because it is rather easy to use but I’m personally not a big fan because it syncs “everything”. I prefer to have 2 separate repositories with different passwords and settings (i.e. policies) like compression, retention, snapshot-interval etc.

If you want to use 2 separate repositories at once you need to create 2 config files. I will try to explain:

When Kopia creates a repository (kopia repo create) it automatically connects (kopia repo connect) to this repository. When connecting to a repository, Kopia creates a configuration file (defaults to ~/.config/kopia/repository.config on Unix) which contains necessary information like repository type, repository location and cache location. Kopia is always using this configuration file unless you are using the --config-file flag.

Kopia aswell creates a file that contains the hashed password for this repository. The file needs to be in the same directory as the configuration file and named the same plus a kopia-password suffix (repository.config.kopia-password by default). If this password file is missing and you didn’t specify an environment variable, Kopia will ask for the repository password for every command.

Your initial repository should already be connected so you can rename this configuration file to a more distinct name:
rename repository repository_one ~/.config/kopia/repository.config*
Then create your second repository:
kopia repo create filesystem
Now you should have 4 files:

ls .config/kopia/
repository.config  repository.config.kopia-password  repository_one.config  repository_one.config.kopia-password

repository_one.config corresponds to your initial repository and repository.config corresponds to your new repository.

If you are now using kopia (e.g. kopia snap create) without using the --config-file flag, it will automatically use you new repository. By using the --config-file flag (e.g. kopia --config-file=~/.config/kopia/repository_one.config snap create), you can use your initial repository.

This should work for basically everything, even for sync-to:
kopia --config-file=~/.config/kopia/repository_one.config repo sync-to from-config ~/.config/kopia/repository.config

I hope this explanation makes sense and clarifies the general use of multiple repositories on the command line.

[Jan_Reinhardt]

@dimejo

I assumed that you meant the GUI because the GUI binary is actually named KopiaUI.

Yeah seems a bit confusing because when you access the Linux Kopia server via webbrowser the browser window also is named KopiaUI.

@dimejo

The sync-to command can be great because it is rather easy to use but I’m personally not a big fan because it syncs “everything”. I prefer to have 2 separate repositories with different passwords and settings (i.e. policies) like compression, retention, snapshot-interval etc.

If you want to use 2 separate repositories at once you need to create 2 config files. I will try to explain:

This is exactly how I want it to be for me as well

Thanks for all the stuff you wrote. I will give it a try and report back.
I am willing to fiddle around with command line stuff for initial setup, but once everything is set up, things need to work by themselves (= snapshotting to 2 different repos) and all I need to do is have an occasional look at the KopiaUI.

Cheers, Jan

P.S. one more thing about sync-to : when does Kopia perform the sync stuff? Right after a new snapshot was taken? Or at the time of the next maintenance run? (hourly quick maintenance vs daily full maintenance?)

[kapitainsky]

@Jan_Reinhardt

P.S. one more thing about sync-to : when does Kopia perform the sync stuff? Right after a new snapshot was taken? Or at the time of the next maintenance run? (hourly quick maintenance vs daily full maintenance?)

It is manual command. So whenever you run it:) Does not happen automatically - unless you schedule it using your OS means e.g. crontab.

[Jan_Reinhardt]

Ok here is what I found out. (Sadly, stuff is not working out for me.)
I’m running Kopia as a Docker container. When I set up the container, I specified a server password and a repository password.
So when I create a repository no repository.config.kopia-password is written. I only get the config file.
Then I managed to create two repos via the command line and to rename the config file, so I’ve got 2 config files in the right place. But as soon as I log in via the webbrowser in KopiaUI to dial in policy details, things get screwed over. KopiaUI overwrites (deletes) stuff in the /config folder, and I‘m back with my initial problems.

I suppose stuff would work if I only used KopiaCLI, but this is not the way I’d like to handle things. I am an intermediate (hobby) user, and backing up stuff is something I need to to, but this is too much hassle and too time consuming. I‘d like to configure the details (policies) via a GUI.

So… I guess I‘m back with the solution to go with an automated sync of just one repository. Which has its own fresh problem that I‘ll write about in the posting below.

Cheers, Jan

[Jan_Reinhardt]

@kapitainsky

It is manual command. So whenever you run it:) Does not happen automatically - unless you schedule it using your OS means e.g. crontab.

I would like to configure this sync command as a Kopia Snapshot Action / After Snapshot.
But actions are disabled in KopiaUI by default. So I checked the docs how to enable actions, and I changed "enableActions": true in the repository.config file.

But I did not manage to restart KopiaUI so the change would come into effect. How do I do this?
When I just disconnect and reconnect to the repo, the actions in the config file are switched back to „false“ again.

Guys, thanks for all the effort you‘re putting into helping me!

[kapitainsky]

Stop kopia → make changes to config file → start kopia.

Otherwise kopia overwrites things when you restart it.

[dimejo]

@Jan_Reinhardt

Then I managed to create two repos via the command line and to rename the config file, so I’ve got 2 config files in the right place. But as soon as I log in via the webbrowser in KopiaUI to dial in policy details, things get screwed over. KopiaUI overwrites (deletes) stuff in the /config folder, and I‘m back with my initial problems.

I just tested the commands I posted and realized that Kopia doesn’t expand the ~ (tilde) character and complains “Repository not configured”. Please try again by using a path without tilde character.

[Jan_Reinhardt]

ah this is is so easy, which made it so hard for me. works!

[Jan_Reinhardt]

So guys, I want to keep on trying. But I have spent many hours last week on trying, I need to change something in my data & backup strategy.

Instead of backing up to a filesystem within my home network, and then syncing elsewhere, my first repository shall be: Deutsche Telekom MagentaCloud, which is a WebDAV provider.

Unfortunately, when I try to set up a WebDAV repo using Kopia, stuff just crashes. I am providing login data as set up by Deutsche Telekom, but Kopia times out / cancels when setting up a repo.
Is this a known problem? Ive googled it, cant find any results of working / not working Kopia repositories at MagentaCloud.

I know Im a troubled guy… but then again, this is a troubleshooting support thread

Edit: This

[kapitainsky]

@Jan_Reinhardt

MagentaCloud

First make sure that this baby works. Try with some other webdav client (cyberduck works with any OS). Make sure you are using correct credentials for webdav access… as it is not necessarily the same as ones used to login to MagentaCloud web interface.

[Jan_Reinhardt]

done that already: I‘m accessing MagentaCloud with 4 other clients (official iPhone app, official iPad app, another filemanager app, Fritzbox router) and via Web login.

[dimejo]

Just tested on a Nextcloud Server (MagentaCLOUD is based on Nextcloud it seems) and it is working. But I had to use the URL displayed in the Nextcloud Files app, not the “default” WebDAV URL shown in the MagentaCLOUD FAQ.

[Jan_Reinhardt]

yes, MagentaCLOUD is based on Nextcloud for two years now.
But „the other nextcloud URL“ is not working on magentacloud:

[dimejo]

You need to replace test with your username:
https://magentacloud.de/remote.php/dav/files/USERNAME

If you want to use a subfolder to store your repository, you need to create that directory first and then prepend the directoryname:
https://magentacloud.de/remote.php/dav/files/USERNAME/DIRECTORYNAME

[Jan_Reinhardt]

Sadly, this doesnt work either:

[Jan_Reinhardt]

But when I use the default address published by Telekom, error message looks differently. I think the login authentication does work, but repository file creation then screws up:

[Jan_Reinhardt]

… but to possibly cut things short: is there any free webdav storage provider (5 GB free or anything) that Kopia is known to work with?
I will open an account and create a repository there for testing.