(fleet) add openvpn-db installation

[dtapiacl]

• Install openvpn db instance.
• Performs authenticated pulls to help mitigate Docker Hub rate limits by associating the image pulls with a Docker Hub account.
• I currently have a working, tested instance running on Ruka with authenticated Docker pull.

[jhoblitt]

Why is the bitnami chart being used instead of mariadb-operator per https://rubin-obs.slack.com/archives/C07RXFUPVAA/p1745964809373419?thread_ts=1745857564.357769&cid=C07RXFUPVAA ?

This is deploying a single pod, which will go down when a node is drained, crashes, etc., which isn't acceptable for a production deployment. https://rubin-obs.slack.com/archives/C07RXFUPVAA/p1745966019358389?thread_ts=1745857564.357769&cid=C07RXFUPVAA

What is the backup mechanism for the database?

What is the relationship between this PR and lsst-it/lsst-control#1707 ?

[csilva-cl]

Why is the bitnami chart being used instead of mariadb-operator per https://rubin-obs.slack.com/archives/C07RXFUPVAA/p1745964809373419?thread_ts=1745857564.357769&cid=C07RXFUPVAA ?

It doesnt look there's an official mariadb-operator. Bitnami is the chart with more stars and it's being actively developed, there's also plenty of testing with this chart. I would leave it as it is.

[dtapiacl]

Why is the bitnami chart being used instead of mariadb-operator per https://rubin-obs.slack.com/archives/C07RXFUPVAA/p1745964809373419?thread_ts=1745857564.357769&cid=C07RXFUPVAA ?

I have a tested working instance in Ruka of the current helm chart. I'm sure there is stuff that needs to be ironed out, which I'm working on based on the feedback provided, but I would like to continue working with what I have at the moment, try to get this deployed as soon as possible, as we are running out of time. I

This is deploying a single pod, which will go down when a node is drained, crashes, etc., which isn't acceptable for a production deployment. https://rubin-obs.slack.com/archives/C07RXFUPVAA/p1745966019358389?thread_ts=1745857564.357769&cid=C07RXFUPVAA

I will set up a secondary replica count for this deployment based on values provided by the chart so a single pod is not deployed.

What is the backup mechanism for the database?

I'm going to use S3 backup method similar to what Carlos does with CNPG, I will be working on this today.

What is the relationship between this PR and lsst-it/lsst-control#1707 ?

This PR is needed as it deploys the DB in wich PR lsst-it/lsst-control#1707 depends on for the OVPN cluster setup. The cluster setup is done through the GUI of OVPN admin portal once VPN server is installed.

[jhoblitt]

Why is the bitnami chart being used instead of mariadb-operator per https://rubin-obs.slack.com/archives/C07RXFUPVAA/p1745964809373419?thread_ts=1745857564.357769&cid=C07RXFUPVAA ?

It doesnt look there's an official mariadb-operator. Bitnami is the chart with more stars and it's being actively developed, there's also plenty of testing with this chart. I would leave it as it is.

There is an official operator.

https://github.com/mariadb-operator/mariadb-operator

The main concern is that there are no replicas, failover, or backups.

[jhoblitt]

There are 3 things required for a production database.

• online replicas
• automated failover between those replicas and/or automation such that failover of any 2 k8s nodes doesn't take the database offline
• automated backups (and a tested and known working db restore procedure)

[dtapiacl]

Ok i have tested the replication offered by the chart and it is not what we need as it only offers replicas for the secondary pods in a master slave setup Primary still could fail. I will put this activity on hold for now this week and resume next week using the mariadb-operator suggested by @jhoblitt.

cc: @csilva-cl