What is validate-provider doing?

Trying to setup with SFTP against a Hetzner Storage Box. They do not allow SSH (unless I enable it and then it will be on port 23, while SFTP stays put). I can connect manually via sftp.

What does this mean, and are there any special settings for this provider?

./kopia repository validate-provider
Opening 4 equivalent storage connections…
ERROR error closing active connection: error closing SFTP client: EOF
ERROR error closing active connection: error closing SFTP client: EOF
ERROR error closing active connection: error closing SFTP client: EOF
ERROR provider validation error: unable to open additional storage connections: unable to open storage connection: unable to open SFTP storage: error establishing connecting: unable to dial [uXXXXXXX.your-storagebox.de:22]: &ssh.ClientConfig{Config:ssh.Config{Rand:io.Reader(nil), RekeyThreshold:0x0, KeyExchanges:string(nil), Ciphers:string(nil), MACs:string(nil)}, User:“XXXXXXX”, Auth:ssh.AuthMethod{(ssh.passwordCallback)(0x17c2320)}, HostKeyCallback:(ssh.HostKeyCallback)(0x17be720), BannerCallback:(ssh.BannerCallback)(nil), ClientVersion:“”, HostKeyAlgorithms:string(nil), Timeout:0}: dial tcp xx.yy.zz.nn:22: connect: connection timed out

My guess is you are using a normal SSH key. Hetzner Storage Box requires a non-default SSH key for certain connections.

https://docs.hetzner.com/storage/storage-box/backup-space-ssh-keys

I’m not using ANY SSH keys, because I’m trying to use SFTP, and while they are similar, the Hetzner SFTP is on normal port 22 and ssh is on port 23. Where is the documentation that Kopia requires access to SSH when I specified SFTP? I’d also need to tell Kopia that ssh is on p23, not 22. How would I do that?

Key is optional.

Are you using cmd? Then all is documented (--port 23):

And if GUI then all is in front of your eyes:

This is “connect” information - and yes, I have read that.
I have also looked at https://kopia.io/docs/reference/command-line/common/repository-validate-provider/ which is the topic question. If you read that page, it says nothing about what it’s trying to “validate” and how. That is my question, because of the errors I got when I ran it.

Yeap. Another example of what is missing in docs:)

This is how it looks when it finishes successfully - it sort of explains what was validated:

$ kopia repository validate-provider
Opening 4 equivalent storage connections...
Validating storage capacity and usage
Validating blob list responses
Validating non-existent blob responses
Writing blob (5000000 bytes)
Validating conditional creates...
Validating list responses...
Validating partial reads...
Validating full reads...
Validating metadata...
Running concurrency test for 30s...
All good.
Cleaning up temporary data...

Thank you for that.

Still wondering why it is trying to use SSH (if it is…still not sure) when I am configuring a SFTP repository. As mentioned, above, this cloud provider allows SFTP via the usual port, but SSH is via 23.

Agree.

I would try ssh then… Unless you are ready to dive into source code you might never know:)

kopia author(s) are not really active here.

That is the problem. They are on two different ports. How do I configure the repository such that SFTP uses one port, but SSH (when required) uses a different port?

I will double-check the “create sftp” page.

I am a bit confused… You are talking about two different configurations. Either you access it using SFTP using username/password or using SSH keys. Two different configuration IMO.

But I do not have Hetzner account atm so can’t test. Maybe some other Hetzner user will shed some light here.

There is no “SSH” option to configure a repository. SFTP is an option.
username/password or SSH Keys can be used to connect via both protocols. SFTP (and SCP) are sub-protocols of SSH. BOTH require a username. The authentication of password vs private/public keys are two options to either connection.

SFTP has a set of valid commands. SSH is typically an interactive shell (not always), and has a different set of valid commands.

If I configure a SFTP repository, I would expect that it uses SFTP commands over the configured port. It seems kopia also wants to connect via SSH and issue commands that way…which seems to be an issue if the SSH port is different to the SFTP port (it is typically the same). The SSH commands and interaction do not seem documented, so this is a theory.

I have tried to re-create the repository with various “–ssh-command” and “–port” options and none will pass the “validate-provider”. (Some create the repository successfully, others do not even do that.)

1 Like

Thank you for clarification. It is rather terminology misunderstanding. But now we have this set.

SSH commnd(s) can be run by kopia by design and are even required in some cases. As per docs:

If the connection to SFTP server does not work, try adding --external which will launch an external ssh process that supports more connectivity options and may be needed for some hosts.

However it is optional. What “some hosts” means unfortunately is not explained:)