Is there anywhere a tutorial for cluster building with self-signed certificates?

[maiorfi]

We have 3 nodes not reachable by internet (i.e. we can't use wizard procedure for let's encrypt certificates with challenge and auto-renewal) and a self-signed certificate.

What is the exact sequence of operation in order to build a cluster from scratch?

Until now, we tried many different ways, such as

• launching all three nodes in "wizard/initial" mode, following procedure for self-provided certificate on 1st node
• adding the other 2 nodes during second step of same procedure (also, it seems there is a bug in DNS name data entry widget, since it is not possible to enter a name different from hostname procedure is running on, i.e. 1st node in this case; we had to edit all settings.json files contained in .zip file generated by procedure)
• following "Continue cluster setup for new node" procedure on 2nd and 3rd nodes, uploading .zip file generated at the end of previous step anc choosing each time the right cluster node to "impersonate" in dropdown

Sadly, it seems that, after having followed such steps, each of 3 nodes has built a single node cluster, and there isn't any way to "break" the cluster, eventually making it possible to add other nodes manually.

Any hint?

[ayende]

Please note that you can use the Let's Encrypt model here, just need a DNS server for that.
The actual servers & hosts can be private.

For your scenario, shut down nodes A & B, delete the RavenData directory, start them and then add them via Manage Server > Cluster > Add Node

[maiorfi]

It would be great: we can manage a DNS for our domains easily. Should we use any specific DNS service in order to satisfy let's encrypt challenges (i.e. any DNS service certbot tool has plugins for) or any will work?
Is there any blog post or similar we can follow? It seems that standard wizard procedure (the one that takes advantage of ravendb community DNS) isn't suitable for private hosts. Am I wrong?

[ayende]

There are two options here:

• Use the default wizard, you'll get a domain such as *.maiorfi.ravendb.run (managed by RavenDB) - all of the certificates are on us. When you do the setup, provide the private ips that you want. Everything will work.
• Use your own DNS - in which case you can provide a certificate via certbot or anything you like. Then you manage it directly. In the setup wizard, choose "You own certificate", but for those kind of deployments, I would say to edit the settings.json directly. Since you'll want to ensure that RavenDB calls to certbot to update the certificate. You can do that using:
  https://ravendb.net/docs/article-page/5.3/csharp/server/configuration/security-configuration#security.certificate.load.exec

[maiorfi]

We're going to try with first way: what kind of settings.json should we start with? Should we use "Initial" setup mode with unprotected (i.e. http, not https) endpoints? When does Raven use ServerUrl and when PublicServerUrl during initial cluster formation? Also, is there any way to destroy a cluster without losing license registration (mandatory for cluster feature)?

[maiorfi]

Ok, we succeeded. As reference for the future, following points turned out to be the "crucial" ones:

On all nodes, start with a clean RavenData dir and with a settings.json like this one (here raven01, as well as for raven02 and raven03, are host names defined in order to resolve private IP on inter-node communications; we used /etc/hosts to map such hostnames with corresponding private IPs; also, we're quite not sure this is actually "crucial" for correct initial configuration):

{
 "DataDir": "RavenData",
 "License.Eula.Accepted": true,
 "Security.UnsecuredAccessAllowed": "PublicNetwork",
 "Setup.Mode": "Initial",
 "ServerUrl": "http://0.0.0.0:8080",
 "ServerUrl.Tcp": "tcp://0.0.0.0:38888",
 "PublicServerUrl": "http://raven01:8080",
 "PublicServerUrl.Tcp": "tcp://raven01:38888"
}

We followed "Let's Encrypt" procedure on initial wizard on node A. Here the important thing is to correctly define all 3 entries in cluster configuration, i.e. using private IP in "IP Address / Hostname" textbox.

At the end of such procedure, a .zip file is downloaded by browser. This .zip file contains on its root the pfx certificate that has to be imported on Current User Personal Certificates in order to authenticate both browser when accessing web Studio (on any cluster node) and client applications.

Same .zip file has to be used with wizard procedure "Continue cluster setup for new node" on nodes B and C, that have to be started with clear RavenData directory and with an initial settings.json like the one above.

Once completed such procedure on nodes B and C everything should work as expected.

Hope this will help!