Get MYSQL_PWD using an on-demand cluster query (PR 1 of 6) #342
Conversation
zzzeek
changed the title
zzzeek
force-pushed
the
OSPRH-14916-pr1
branch
2 times, most recently
from
f429f65 to
a91d023
Compare
|
/retest |
|
/recheck |
|
/test |
|
@zzzeek: The The following commands are available to trigger optional jobs: Use In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/test mariadb-operator-build-deploy-kuttl |
|
that last run failed in the openstack deploy step. everything succeeded except neutron that got stuck in db upgrade, it failed with: "sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1054, "Unknown column 'quotas.project_id' in 'field list'")" not really sure how that can happen |
|
/retest |
zzzeek
force-pushed
the
OSPRH-14916-pr1
branch
from
638bc6a to
a4dbf5a
Compare
zzzeek
force-pushed
the
OSPRH-14916-pr1
branch
from
a4dbf5a to
ee836ee
Compare
dciabrin
left a comment
dciabrin
left a comment
There was a problem hiding this comment.
Thanks for that.
The only problem I have with this review is that we currently use the db root user for probes, because we haven't got yet a clean way of creating alternative users (part of it will be resolved with your next review I think). So if we were to merge it as is, we'd end up doing two API calls for each probe sent to the galera pods, which is way too many.
I think we should discuss how to best cache that and only call the API when needed. I was thinking something along the lines of:
- we keep using root_auth any time we want to get root creds.
- internally, root_auth checks whether /root/.my.cnf exists. If it doesn't, it calls the API and generate that file with the current password.
- if it exists, it checks whether creds are still valid (e.g. by doing a mysqladmin ping). If not, do 2.
- if mysql is not running (i.e. when pod start), do not try to check creds validity, assume .my.cnf is valid and wait for the next probe to update it if needed.
templates/galera/bin/root_auth.sh
Outdated
|
|
||
| GALERA_INSTANCE="{{.galeraInstanceName}}" | ||
|
|
||
| # note jq is not installed in the galera image, macgyvering w/ python instead |
There was a problem hiding this comment.
Ack, I'll track the shortcoming in a separate Jira so we can fix that for good.
zzzeek
force-pushed
the
OSPRH-14916-pr1
branch
4 times, most recently
from
2467a94 to
4c0c5f5
Compare
zzzeek
force-pushed
the
OSPRH-14916-pr1
branch
3 times, most recently
from
a151658 to
f87d0bd
Compare
|
/lgtm |
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: lmiccini, zzzeek The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
zzzeek
force-pushed
the
OSPRH-14916-pr1
branch
from
f87d0bd to
c282e5d
Compare
In 1bb2318 the galera certs were regenerated with a three year expiry, but this did not include the CA expiration time, leading to failures again. this change updates that time as well and adds a script that can be used to regen the values.
In order to facilitate an in-place change to the name of the Secret that is referenced by a Galera instance for the mysql root password, rework the approach used by pods and shell scripts to no longer require the root secret name and/or password be passed by environment variable, instead using a pod-level cluster query to retrieve the current root password. The logic to retrieve this password is encapsulated into a single shell script that is present as a volume mount on running containers. This allows Job objects to be created with hashes that do not link to a specific Secret name, as well as to create StatefulSet objects that don't refer to this name. When the Secret name changes on a Galera instance for an in-place root password change, the hashes / CRs for these objects will remain unchanged. A subsequent change to the mariadb operator will add the ability to change the mysql root password of a Galera cluster using a dual-reference architecture where the "current" root secret will be part of <CR>/Status, while the secret referenced in <CR>/Spec will be the "new" root secret. When these two names differ, that will indicate an in-place password change should take place, as well as allowing the pre-existing root password to be available at the same time as the new one in order to do a root password change. The same architecture will be applied to a new class of "system" MariaDBAccount objects that are for use only by the Galera instance itself and do not have a link to any MariaDBDatabase CR. The Galera CR itself will no longer use osp-secret for the mysql root password nor will the secret be directly referenced from the Galera CR, instead referenced by a "system" MariaDBAccount CR which the Galera operator itself will create. 🤖 Generated with Claude Code (https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
zzzeek
force-pushed
the
OSPRH-14916-pr1
branch
from
c282e5d to
3527102
Compare
|
/lgtm |
openshift-merge-bot
bot
merged commit 966e316
into
openstack-k8s-operators:main
3 participants
In order to facilitate an in-place change to the name of the Secret that is referenced by a Galera instance for the mysql root password, rework
the approach used by pods and shell scripts to no longer require the root secret name and/or password be passed by environment variable, instead using a pod-level cluster query to retrieve the current root password. The logic to retrieve this password is encapsulated into a single shell script that is present as a volume mount on running containers.
This allows Job objects to be created with hashes that do not link to a specific Secret name, as well as to create StatefulSet objects that don't refer to this name. When the Secret name changes on a Galera instance for an in-place root password change, the hashes / CRs for these objects will remain unchanged.
A subsequent change to the mariadb operator will add the ability to change the mysql root password of a Galera cluster using a dual-reference architecture where
the "current" root secret will be part of /Status, while the secret referenced in /Spec will be the "new" root secret. When these two names differ, that will indicate an in-place password change should take place, as well as allowing the pre-existing root password to be available at the same time as the new one in order to do a root password
change. The same
architecture will be applied to a new class of "system" MariaDBAccount
objects that are for use only by the Galera instance itself
and do not have a link to any MariaDBDatabase CR. The
Galera CR itself will no longer use osp-secret
for the mysql root password nor will the secret be directly
referenced from the Galera CR, instead referenced by a
"system" MariaDBAccount CR which the Galera operator itself
will create.