[tlse] tls support for telemetry aodh #310

Deydra71 · 2024-02-19T14:03:57Z

Public/Internal service cert secrets and the CA bundle secret can be passed to configure httpd virtual hosts for tls termination. The certs get direct mounted to the appropriate place in etc/pki/tls/certs/%s.crt|key and a CA bundle to /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem . Job deployments for bootstrap/cron get the CA bundle added if configured.

Depends-On: openstack-k8s-operators/lib-common#428

Jira: https://issues.redhat.com/browse/OSPRH-4197

softwarefactory-project-zuul · 2024-02-19T14:04:11Z

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/openstack-k8s-operators/telemetry-operator for 310,6ca03b5a4075bcacd284d5b995483429a4fdbb94

softwarefactory-project-zuul · 2024-02-19T14:17:59Z

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/4795191de5ce4ac6846c8a4335e78376

❌ openstack-k8s-operators-content-provider FAILURE in 6m 35s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider

softwarefactory-project-zuul · 2024-02-19T16:09:29Z

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/56ed12fb47e94332926921972de87cbb

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 21m 50s
❌ podified-multinode-edpm-deployment-crc FAILURE in 1h 01m 09s

vyzigold

I know this is still WIP, but I took a brief look. So far it looks good except for that one comment I left.

I have a question regarding the CA certs mounted to the containers. We have a bug either in the operator or in aodh. Because of that aodh seems to be ignoring its configuration and it's always using public endpoint for keystone. So the question is if the CA certs are the same for internal and public endpoints? Will aodh be able to connect to public keystone endpoint after we're done with the TLS efforts?

vyzigold · 2024-02-20T08:15:39Z

api/v1beta1/autoscaling_types.go

+	// +kubebuilder:validation:Optional
+	// +operator-sdk:csv:customresourcedefinitions:type=spec
+	// TLS - Parameters related to the TLS
+	TLS tls.API `json:"tls,omitempty"`


I think it would make a bit more sense inside the Aodh struct above, since this is configuring TLS for aodh.

We are using two CAs - one for public and one for internal endpoints, and the combined-ca-bundle.pem file incorporates the CAs for both public and internal endpoints. There's also the possibility to provide service specific CA through ca.crt.

So, the aodh should be able to connect to the public keystone.

For more info --> https://github.com/openstack-k8s-operators/docs/blob/main/tls.md

cc: @stuggi

softwarefactory-project-zuul · 2024-02-20T10:03:22Z

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/dec7c0638eb044219cf33ef77c44d9b8

❌ openstack-k8s-operators-content-provider FAILURE in 6m 13s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider

softwarefactory-project-zuul · 2024-02-20T12:00:34Z

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/c7cb304147054a859737d3280795b691

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 23m 55s
❌ podified-multinode-edpm-deployment-crc FAILURE in 1h 00m 54s

Deydra71 · 2024-02-20T14:54:14Z

Currently the volumes and volume mounts in both ceilometer and aodh are not correctly created. Working on it.

softwarefactory-project-zuul · 2024-02-21T12:58:29Z

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/33039054729c4b7a896439cb1d4b975b

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 54m 49s
❌ podified-multinode-edpm-deployment-crc FAILURE in 1h 01m 45s
❌ telemetry-operator-multinode-autoscaling FAILURE in 1h 34m 21s

vyzigold · 2024-02-21T16:24:47Z

Looks like this is hitting the same issue as Emma's and mine PR, where the infra-operator resources are trying to pull incorrect images: https://logserver.rdoproject.org/10/310/15348e87af91e03038ba3320535d3553e9ace717/github-check/podified-multinode-edpm-deployment-crc/89a8845/controller/ci-framework-data/logs/openstack-k8s-operators-openstack-must-gather/namespaces/openstack/pods/memcached-0/memcached-0-describe

vyzigold · 2024-02-21T16:29:56Z

We had the same issue here: #284 . It should be fixed, so next run should get the CI further.

elfiesmelfie · 2024-02-21T17:01:18Z

We had the same issue here: #284 . It should be fixed, so next run should get the CI further.

The issue is resolved in openstack-k8s-operators/openstack-operator#643

Deydra71 · 2024-02-29T10:55:31Z

I believe you need to edit this file as well:

telemetry-operator/kuttl-test.yaml

Line 27 in 0d4f1ec

- tests/kuttl/suites/metricstorage/

. I hope that's enough to get them executed.

Yep, that was it. Thanks a lot!

Public/Internal service cert secrets and the CA bundle secret can be passed to configure httpd virtual hosts for tls termination. The certs are directly mounted to /var/lib/config-data/* and coppied to the appropriate place in etc/pki/tls/certs/%s.crt|key and a CA bundle to /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem . Job deployments for bootstrap/cron get the CA bundle added if configured. Signed-off-by: Veronika Fisarova <[email protected]> Depends-On: openstack-k8s-operators/lib-common#428

vyzigold · 2024-03-05T20:41:36Z

I read through the PR once again and I don't have anything to add. Thank you.
/approve
/lgtm

openshift-ci · 2024-03-05T20:41:45Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Deydra71, vyzigold

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [vyzigold]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Creates certs for k8s service of the service operator when spec.tls.endpoint.internal.enabled: true For a service like nova which talks to multiple service internal endpoints, this has to be set for each of them for, like: ~~~ customServiceConfig: | [keystone_authtoken] insecure = true [placement] insecure = true [neutron] insecure = true [glance] insecure = true [cinder] insecure = true ~~~ Depends-On: openstack-k8s-operators/lib-common#428 Depends-On: openstack-k8s-operators#620 Depends-On: openstack-k8s-operators/telemetry-operator#310 Depends-On: openstack-k8s-operators/telemetry-operator#327 Depends-On: openstack-k8s-operators/telemetry-operator#330 Signed-off-by: Veronika Fisarova <[email protected]>

Creates the telemetry aodh route and svc overrides. Creates certs for k8s service of the service operator when spec.tls.endpoint.internal.enabled: true For a service like nova which talks to multiple service internal endpoints, this has to be set for each of them for, like: ~~~ customServiceConfig: | [keystone_authtoken] insecure = true [placement] insecure = true [neutron] insecure = true [glance] insecure = true [cinder] insecure = true ~~~ Depends-On: openstack-k8s-operators/lib-common#428 Depends-On: openstack-k8s-operators#620 Depends-On: openstack-k8s-operators/telemetry-operator#310 Depends-On: openstack-k8s-operators/telemetry-operator#327 Depends-On: openstack-k8s-operators/telemetry-operator#330 Signed-off-by: Veronika Fisarova <[email protected]>

openshift-merge-robot added the needs-rebase label Feb 19, 2024

openshift-ci bot requested review from csibbitt and frenzyfriday February 19, 2024 14:04

Deydra71 added the do-not-merge/hold label Feb 19, 2024

Deydra71 removed request for csibbitt and frenzyfriday February 19, 2024 14:04

Deydra71 force-pushed the tls-support branch from 6ca03b5 to 12fd78e Compare February 19, 2024 14:10

openshift-merge-robot removed the needs-rebase label Feb 19, 2024

Deydra71 changed the title ~~[tls] Support for telemetry~~ [tlse] tls support for telemetry pod configuration Feb 19, 2024

Deydra71 changed the title ~~[tlse] tls support for telemetry pod configuration~~ [tlse] tls support for telemetry ceilometer, aodh Feb 19, 2024

Deydra71 force-pushed the tls-support branch from 12fd78e to c4ece80 Compare February 19, 2024 14:45

vyzigold reviewed Feb 20, 2024

View reviewed changes

Deydra71 requested review from olliewalsh and stuggi February 20, 2024 09:39

Deydra71 force-pushed the tls-support branch from c4ece80 to 159dc88 Compare February 20, 2024 09:55

Deydra71 force-pushed the tls-support branch from 159dc88 to a809063 Compare February 20, 2024 10:35

Deydra71 force-pushed the tls-support branch 2 times, most recently from 1f46715 to 15348e8 Compare February 21, 2024 11:02

Deydra71 force-pushed the tls-support branch 2 times, most recently from a03a641 to f10a866 Compare February 22, 2024 13:36

Deydra71 force-pushed the tls-support branch from b65f80b to 5a4f3c8 Compare February 29, 2024 10:44

Deydra71 force-pushed the tls-support branch from 5a4f3c8 to 16008db Compare March 1, 2024 09:07

Deydra71 changed the title ~~[tlse] tls support for telemetry ceilometer, aodh~~ [tlse] tls support for telemetry aodh Mar 1, 2024

Deydra71 force-pushed the tls-support branch 2 times, most recently from d22bef9 to dc095f8 Compare March 1, 2024 11:56

openshift-merge-robot added the needs-rebase label Mar 1, 2024

Deydra71 force-pushed the tls-support branch from dc095f8 to e8e3714 Compare March 4, 2024 15:55

openshift-merge-robot removed the needs-rebase label Mar 4, 2024

Deydra71 removed the do-not-merge/hold label Mar 4, 2024

Deydra71 requested a review from vyzigold March 4, 2024 17:32

openshift-ci bot assigned vyzigold Mar 5, 2024

openshift-ci bot added the lgtm label Mar 5, 2024

openshift-ci bot added the approved label Mar 5, 2024

openshift-merge-bot bot merged commit 2385f25 into openstack-k8s-operators:main Mar 5, 2024

Deydra71 mentioned this pull request Mar 7, 2024

[tlse] internal TLS support for telemetry aodh service openstack-k8s-operators/openstack-operator#695

Merged

[tlse] tls support for telemetry aodh #310

[tlse] tls support for telemetry aodh #310

Uh oh!

Conversation

Deydra71 commented Feb 19, 2024

Uh oh!

softwarefactory-project-zuul bot commented Feb 19, 2024

Uh oh!

softwarefactory-project-zuul bot commented Feb 19, 2024

Uh oh!

softwarefactory-project-zuul bot commented Feb 19, 2024

Uh oh!

vyzigold left a comment

Choose a reason for hiding this comment

Uh oh!

vyzigold Feb 20, 2024

Choose a reason for hiding this comment

Uh oh!

Deydra71 Feb 20, 2024

Choose a reason for hiding this comment

Uh oh!

Deydra71 Feb 20, 2024

Choose a reason for hiding this comment

Uh oh!

softwarefactory-project-zuul bot commented Feb 20, 2024

Uh oh!

softwarefactory-project-zuul bot commented Feb 20, 2024

Uh oh!

Deydra71 commented Feb 20, 2024

Uh oh!

softwarefactory-project-zuul bot commented Feb 21, 2024

Uh oh!

vyzigold commented Feb 21, 2024

Uh oh!

vyzigold commented Feb 21, 2024

Uh oh!

elfiesmelfie commented Feb 21, 2024

Uh oh!

Deydra71 commented Feb 29, 2024

Uh oh!

vyzigold commented Mar 5, 2024

Uh oh!

openshift-ci bot commented Mar 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants