Hi,
Kopia seems to be hanging for me and not creating a repository correctly using the CLI on linux with the following command:
sudo kopia repository create s3 --bucket=<my-bucket> --endpoint=<my endpoint URL> --access-key=<my access key> --secret-access-key=<my secret key>
I know I’m using sudo so it will run as the root user and this is intentional to be certain it has permissions to access to all files in the backup path(s). But after running this command, it seems to connect and “create” the repository on the s3 server as there are then 3 files generated by kopia on the sever but does not complete the entire process. The command just hangs open indefinitely and never returns me to the command prompt.
The same occurs when I attempt:
kopia policy set --global --compression=pgzip
Am I doing something wrong? Or is it my s3 provider? Or some sort of permissions problem?
I’ve had this configured and run backups properly on this machine before using, I believe, the same commands as above and it’s worked. Further, I do have backups from a couple of different machines (windows desktop and laptop) configured through the gui on Windows and those seem to be running properly, so I don’t think it is necessarily the cloud storage provider.
Any help with this would be greatly appreciated. I love kopia. When it works, it’s performed beautifully. Not sure if anything has changed with the commands lately that could be causing this behavior. If 'm unable to figure this out, though, unfortunately I may need to find a different option, which I’d rather not do.
THank you, in advance.
Kind Regards,
Deckard
If all works on your other machines then create and configure this repo on other computer and only connect to it from one you have problems with - I doubt it is kopia issue - most likely something about your setup.
I’m sure I understand what you mean. I still need to be able to backup the problematic linux machine independently of those machines.
And, more to the point, what would cause those commands (which I understand to be the official commands according to the documentation) to simply hang open like that? It’s almost as though they’re waiting for something else, but they indicate nothing, and don’t exit with an error. They just sit therem unless I ctrl-C out of them.
I can also confirm that, after installing kopia on a different linux machine and attempting to create a repository with the exact same commands that I am met with the same results.
Setting up the same repository with same exact credentials all around through the GUI does still work on a Windows machine, however.
I also just found the “validate-provider” command and attempted that and it, too, hangs at “Writing blob (5000000 bytes)”.
I would think that maybe somehow my provider changed its API or something like that, or the buckets are somehow set as read-only, however, as I said, I can confirm that the same exact repository works just fine from a windows machine. There must be some new flag in the linux CLI command that I’m missing or something along those lines.
Anybody else having this same problem on a linux machine with a brand new install of Kopia v0.17.0?
Latest kopia with S3 destination on Linux. Zero issues.
$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 12 (bookworm)
Release: 12
Codename: bookworm
$ kopia repository create s3 --bucket=<my-bucket> --endpoint=<my endpoint URL> --access-key=<my access key> --secret-access-key=<my secret key>
Enter password to create new repository:
Re-enter password for verification:
Initializing repository with:
block hash: BLAKE2B-256-128
encryption: AES256-GCM-HMAC-SHA256
splitter: DYNAMIC-4M-BUZHASH
Connected to repository.
NOTICE: Kopia will check for updates on GitHub every 7 days, starting 24 hours after first use.
To disable this behavior, set environment variable KOPIA_CHECK_FOR_UPDATES=false
Alternatively you can remove the file "/home/kptsky/.config/kopia/repository.config.update-info.json".
Retention:
Annual snapshots: 3 (defined for this target)
Monthly snapshots: 24 (defined for this target)
Weekly snapshots: 4 (defined for this target)
Daily snapshots: 7 (defined for this target)
Hourly snapshots: 48 (defined for this target)
Latest snapshots: 10 (defined for this target)
Ignore identical snapshots: false (defined for this target)
Compression disabled.
To find more information about default policy run 'kopia policy get'.
To change the policy use 'kopia policy set' command.
NOTE: Kopia will perform quick maintenance of the repository automatically every 1h0m0s
and full maintenance every 24h0m0s when running as kptsky@debian12.
See https://kopia.io/docs/advanced/maintenance/ for more information.
NOTE: To validate that your provider is compatible with Kopia, please run:
$ kopia repository validate-provider
Most likely there is some problem with your linux environment.
You are indeed, correct,sir, in that it was not a problem with Kopia itself. Nor was it a problem with my linux environment.
After much investigation and haggling with my s3 provider’s support team, it turns out that, in fact, it was on their end. Of all things, the particular data center housing my s3 environment had literally run out storage space and they are in the process of upgrading. As such, writing to my s3 bucket on their servers would work on occasion and typically for very small files, but would otherwise fail. This is why I was getting what seemed to some semblance of things working at any one given point in time on any one given machine, but ultimately led to everything breaking down with any dataset of any significant size.
Of course, I was never warned of any maintenance taking place. God forbid these massive companies throw an extra few dollars and resources toward actual customer support. Their CEOs may just starve if they were to do so.
So, my apologies for wasting your time. By most indications, things seemed to be sort of working, but clearly were not in the ultimate sense.
Thank you for your attention to the matter, though. It is appreciated.
1 Like
WOW - this is indeed rare situation. Myself I always assume that my S3 provider works and would also seek problem with kopia or my system.
Thanks for reporting what was the problem. Good learning for anybody reading it.