Hi there, I’m new to Kopia I was looking for a tool to do backups to my personal PC and a excuse to start learning AWS.
I don’t want to use the UI if possible because the task I want to do is very simple:
I’m on windows 10 and wrote a simple PS script that run with windows task scheduler. The first time I ran kopia I had to connect to the S3 and put the credentials there and that worked like a charm once I figured out the correct permissions for the IAM account. The S3 is configured as GLACIER because my idea is to have a backup to use just in case of a disaster.
this is the content of the file. I managed to run it a few times but now I get this error and I’m totally lost.
ERROR open repository: unable to open repository: unable to create format manager: unable to read format blob: error getting kopia.repository blob: unable to complete GetBlob(kopia.repository,0,-1) despite 10 retries: The operation is not valid for the object’s storage class
Long story short - kopia does not support cold storage tier like AWS Glacier. If you want to use Glacier you need a backup program which knows how to handle it. There are free and paid options available.
The content of a repository basically is split into 2 parts:
Packs, that contain the actual data blobs
Index files with a list of all the files and a reference which pack contains which data.
The index files are the backbone of the repository and need to be accessed for every operation. Hence the index files can never be stored in a cold storage and the backup program needs to actively support this (e.g. rustic documentation).
IMHO cold storage makes more sense for a traditional backup, where you copy a directory and never access it until you want to restore.
Hi there, thank you. whay type of “low availability s3” could work with Kopia? what I understand from your answer is that index+data blob are stored in every single kopia file so I can’t ‘separate’ them to store index in a more common way and data blob as glacier right?
I use myself rustic (free) and Arq (paid). Both can utilize Glacier.
But unless you have really huge amount of data and know what you are doing I would think about using Glacier twice. It is not straightforward and their pricing model is complicated. If not careful you might end up paying more than you expect.
That’s a great advice. I have just jumped in GLACIER looking for a very cheap way of backing up data in case of a disaster with my desktop PC. What would be the most cost/effective way of doing this on your opinion?
(Given that would love to use kopia or rustic and a host the backup on AWS if possible)
Index and data blobs are stored in different files. The prefix of the file represents the content. See the documantation for more information:
Based on this knowledge you could build a solution (e.g. rclone with union backend) that somehow works but I would highly advise against it. Kopia is not aware of hot/cold storage and this can cause unexpected issues, especially when running maintenance.
About how much data are we talking? How important is this data to you? Do you have additional backups?
Looks like Kopia does support using Different Storage Classes out of the box (I discovered this just today) as long as the access is instant. So classes like “S3 Standard - Infrequent Access”, “S3 Express One Zone”, “S3 Glacier Instant Retrieval” and “S3 One Zone - Infrequent Access” should work for p blobs.