# Experimenting with rclone

**URL:** <https://kopia.discourse.group/t/experimenting-with-rclone/304>\
**Category:** Support\
**Created:** [March 6, 2021, 11:18pm UTC](https://kopia.discourse.group/t/experimenting-with-rclone/304 "2021-03-06T23:18:57Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![din](https://avatars.discourse-cdn.com/v4/letter/d/ccd318/32.png) [@din](https://kopia.discourse.group/u/din)\
**Post date:** [March 6, 2021, 11:18pm UTC](https://kopia.discourse.group/t/experimenting-with-rclone/304/1 "2021-03-06T23:18:57Z")

</div>

Hi,

I’m experimenting kopia with rclone to try to backup data to 1fichier, a very cheap file hosting provider that rclone can talk with:  
[https://rclone.org/fichier/](https://rclone.org/fichier/)

here is the result:

```
kopia repository create rclone --remote-path 1fichier:/din/kopia/test
Enter password to create new repository: 
Re-enter password for verification: 
Initializing repository with:
  block hash: BLAKE2B-256-128
  encryption: AES256-GCM-HMAC-SHA256
  splitter: DYNAMIC-4M-BUZHASH
kopia: error: cannot initialize repository: unable to write format blob: unable to write format blob: MOVE kopia.repository.f-5577006791947779410: 403, try --help

```

I checked on 1fichier console that the file was created in the right place.  
For what it’s worth, I used the same remote to do the same thing with restic and it worked.

Is there something I can try to solve this?

Thanks

---

<div class="post-metadata">

**Author:** ![din](https://avatars.discourse-cdn.com/v4/letter/d/ccd318/32.png) [@din](https://kopia.discourse.group/u/din)\
**Post date:** [March 11, 2021, 11:09pm UTC](https://kopia.discourse.group/t/experimenting-with-rclone/304/2 "2021-03-11T23:09:41Z")

</div>

I tried to run kopia with rclone to b2 backblaze and it works fine. So the issue seems to be storage provider specific. Maybe I should talk to the rclone guys. But can you think of a specific feature that kopia needs and that 1fichier could not provide ?

I use kopia 0.7.3 with rclone 1.53.3-1+b2 on debian bullseye

---

<div class="post-metadata">

**Author:** ![din](https://avatars.discourse-cdn.com/v4/letter/d/ccd318/32.png) [@din](https://kopia.discourse.group/u/din)\
**Post date:** [March 12, 2021, 1:57pm UTC](https://kopia.discourse.group/t/experimenting-with-rclone/304/3 "2021-03-12T13:57:35Z")

</div>

with the new 0.8.0-beta1 release:

```
./kopia repository create rclone --remote-path 1fichier:din/kopia/test2 --log-level=info
Enter password to create new repository: 
Re-enter password for verification: 
Initializing repository with:
  block hash: BLAKE2B-256-128
  encryption: AES256-GCM-HMAC-SHA256
  splitter: DYNAMIC-4M-BUZHASH
kopia: error: cannot initialize repository: unable to write format blob: unable to write format blob: unable to complete PutBlob(kopia.repository) despite 10 retries, try --help
```

---

<div class="post-metadata">

**Author:** ![din](https://avatars.discourse-cdn.com/v4/letter/d/ccd318/32.png) [@din](https://kopia.discourse.group/u/din)\
**Post date:** [March 13, 2021, 1:49pm UTC](https://kopia.discourse.group/t/experimenting-with-rclone/304/4 "2021-03-13T13:49:53Z")

</div>

Using kopia 0.7.3 and rclone 1.53.3-1+b2:

In order to bypass the MOVE, I created a local repo that I synched to its final online destination (1fichier) with rclone.  
I could then get a connection with:

`kopia repository connect rclone --remote-path 1fichier:din/kopia/test`

but then:

```
kopia snapshot create /data/test/dir/
 | 8 hashing, 0 hashed (23.1 KB), 0 cached (0 B), 0 uploaded (0 B), 0 errors, es / 0 hashing, 916 hashed (21.2 MB), 0 cached (0 B), 0 uploaded (0 B), 0 errors, estimated 22.1 MB (96.1%) 8s lefterror running maintenance: error updating maintenance schedule: MOVE kopia.maintenance.f-3916589616287113937: 403
kopia: error: unable to close repository: error flushing: error writing pending content: error writing pack content: error writing pack: can't save pack data content: MOVE p80/f7d/7acd2a050046839c9e42a65ce02.f-6334824724549167320: 403, try --help

```

Data is uploaded, but obviously, 1fichier doesn’t allow MOVE. I can think of two ways to work around this:

- One solution could be to backup locally and then sync online.

- Or: according to [Overview of cloud storage systems](https://rclone.org/overview/#move), rclone already has a procedure to deal with that issue. Would it be possible to use it with kopia?

---

<div class="post-metadata">

**Author:** ![jkowalski](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/jkowalski/32/3_2.png) [@jkowalski](https://kopia.discourse.group/u/jkowalski)\
**Post date:** [March 13, 2021, 2:20pm UTC](https://kopia.discourse.group/t/experimenting-with-rclone/304/5 "2021-03-13T14:20:00Z")

</div>

So the lack of MOVE would be a problem with current implementation, because move is used for atomic upload - Kopia will first upload a file named `something.tmp` and when done it will rename/move it to `something.f`. Only files ending with `.f` are treated as fully uploaded for safety reasons (in case the client died half-way through we don’t want to rely on partial contents).

Somebody could try to add an option in `PutBlobInPath` in `webdav_storage.go` to do direct upload to an `.f` file bypassing MOVE, but that would probably be unsafe - I’m not sure how rclone would handle atomicity of webdav uploads. If rclone webdav (which kopia uses as its abstract API) is translated as atomic upload to the 1fichier backend which is also atomic, this will in fact work and be safe.

---

<div class="post-metadata">

**Author:** ![jkowalski](https://yyz2.discourse-cdn.com/free1/user_avatar/kopia.discourse.group/jkowalski/32/3_2.png) [@jkowalski](https://kopia.discourse.group/u/jkowalski)\
**Post date:** [March 13, 2021, 2:22pm UTC](https://kopia.discourse.group/t/experimenting-with-rclone/304/6 "2021-03-13T14:22:41Z")

</div>

Actually, I took a look at rclone page which says:

```auto
1Fichier does not support modification times.

```

This is actually a dealbreaker today, since Kopia relies on modification times that follow wall clock time for safety of its operations.

---

<div class="post-metadata">

**Author:** ![din](https://avatars.discourse-cdn.com/v4/letter/d/ccd318/32.png) [@din](https://kopia.discourse.group/u/din)\
**Post date:** [March 13, 2021, 7:04pm UTC](https://kopia.discourse.group/t/experimenting-with-rclone/304/7 "2021-03-13T19:04:35Z")

</div>

End of story.

Thanks a lot for having a look! You’re doing a amazing job.