How to put nginx in front?

I’m trying to put nginx in front. I generated the cert with kopia and then added the crl and key to nginx. When I try and connect from the client I get…

# kopia repository connect server --url https://myurl:443   --server-cert-fingerprint ddcafd58b5943e015a2e9f4a4df849f1f2bf4b82346b4e416   
Connecting to server 'https://myurl:443' as 'user1@server1'...
Enter password to open repository:

failed to open repository: unable to establish session for purpose=: error establishing session: Session(): rpc error: code = Unavailable desc = connection closed
kopia: error: error connecting to API server: unable to establish session for purpose=: error establishing session: Session(): rpc error: code = Unavailable desc = connection closed, try --help
#

This is what I see in the nginx access log.

myurl:443 6.29.1.1 - - [25/Apr/2021:16:45:09 -0700] "PRI * HTTP/2.0" 400 173 "-" "-"

nginx config

server {
  listen 443;
  server_name myurl;

    ssl    on;
 

    ssl_certificate_key /var/lib/kopia/kopia-cert.key;
    ssl_certificate /var/lib/kopia/kopia-cert.cert;

  client_max_body_size 0;  # unlimited

  location / {
   proxy_pass https://localhost:51515;
  }
}

Kopia server uses gRPC which is based on http/2. Not sure exactly how this impacts nginx reverse proxying.

Thanks. Would not have figured this out without your tip. The below passes some basic tests.


server {
  listen 443 ssl http2;
  server_name myurl;

    ssl    on;
    ssl_certificate_key /var/lib/kopia/kopia-cert.key;
    ssl_certificate /var/lib/kopia/kopia-cert.cert;

  client_max_body_size 0;  # unlimited

  location / {
   grpc_pass grpcs://localhost:51515;
  }
}

BTW, would be great to document this on kopia.io, I’ll be happy to accept a PR:

The contents are here: kopia/site/content/docs at master · kopia/kopia · GitHub

I wanted to share my findings on using nginx:

  • the provided nginx example from @bbrendon works fine
  • you have to use grpcs. grpc does not work. This means the kopia server has to be started with a certificate (–insecure does not work). It can be a self-signed certificate, that does not matter.
  • you have to use nginx >= 1.16; with nginx 1.14 I get the following error
    upstream sent frame for closed stream 1 while reading upstream which is fixed with a recent nginx version (see here: #1792 (grpc module handles RST_STREAM(NO_ERROR) improperly on closed streams) – nginx)
    I tested and I got no error with nginx 1.18
  • You can use https://yourdomain.com:443 in kopia / kopia-ui to connect to the server
  • I did a quick test with backup, restore and policy modification in kopia-ui; everything worked
  • from time to time I can see an error in the logs:
    [error] 31#31: *15 upstream timed out (110: Connection timed out) while reading upstream... request: "POST /kopia_repository.KopiaRepository/Session HTTP/2.0
    I could not figure out why this occurs, but I couldn’t find anything not working either. Increasing proxy timeouts did not help - the error occurred before the timeout was reached.