Skip to content

Automatically create network policies to allow Prometheus scraping traffic for workloads annotated with prometheus.io/scrape#579

Merged
zohar7ch merged 7 commits intomainfrom
zohar7ch/auto-enable-metrics-collection-traffic
Apr 1, 2025
Merged

Automatically create network policies to allow Prometheus scraping traffic for workloads annotated with prometheus.io/scrape#579
zohar7ch merged 7 commits intomainfrom
zohar7ch/auto-enable-metrics-collection-traffic

Conversation

@zohar7ch
Copy link
Copy Markdown
Contributor

@zohar7ch zohar7ch commented Mar 18, 2025

Description

By setting intentsOperator.operator.automateThirdPartyNetworkPolicies and intentsOperator.operator.metricsScrapingServiceConfigs - Otterize can now automatically enable traffic from Prometheus server to any other workload marked with prometheus.io/scrape annotation.

You no longer have to create client-intents from the scraping server to your workloads.

References

Testing

Unit tests & e2e test were added

  • This change adds test coverage for new/changed/fixed functionality

Checklist

  • I have added documentation for new/changed functionality in this PR and in github.com/otterize/docs

@zohar7ch zohar7ch force-pushed the zohar7ch/auto-enable-metrics-collection-traffic branch 4 times, most recently from eb814ca to 63e4ba7 Compare March 23, 2025 09:45
@zohar7ch zohar7ch force-pushed the zohar7ch/auto-enable-metrics-collection-traffic branch 2 times, most recently from cbf0e64 to d92bbf9 Compare March 23, 2025 10:33
@zohar7ch zohar7ch marked this pull request as ready for review March 23, 2025 13:41
@zohar7ch zohar7ch changed the title Auto enable metrics collection traffic Automatically enable metrics scraping for workloads that are marked with the Prometheus scraping annotation Mar 24, 2025
@zohar7ch zohar7ch force-pushed the zohar7ch/auto-enable-metrics-collection-traffic branch 6 times, most recently from 2ba5a80 to 59c110c Compare March 24, 2025 16:56
@zohar7ch zohar7ch requested a review from omris94 March 24, 2025 16:57
@zohar7ch zohar7ch force-pushed the zohar7ch/auto-enable-metrics-collection-traffic branch 5 times, most recently from 796292b to 9001174 Compare March 24, 2025 20:20
@zohar7ch zohar7ch changed the title Automatically enable metrics scraping for workloads that are marked with the Prometheus scraping annotation Automatically create network policies to allow Prometheus scraping traffic for workloads annotated with prometheus.io/scrape Mar 25, 2025
@zohar7ch zohar7ch force-pushed the zohar7ch/auto-enable-metrics-collection-traffic branch from 8ccb754 to aa8b06d Compare March 25, 2025 09:28
@zohar7ch zohar7ch force-pushed the zohar7ch/auto-enable-metrics-collection-traffic branch from 884c50d to a1c5c51 Compare March 25, 2025 12:40
@zohar7ch zohar7ch force-pushed the zohar7ch/auto-enable-metrics-collection-traffic branch from a1c5c51 to 149ccb0 Compare March 30, 2025 11:41
zohar7ch added 7 commits April 1, 2025 16:33
To support the new "auto-enable metrics collection traffic" feature,
we need to obtain a new configuration. Since there is a strong
correlation between metrics collection traffic and external traffic — both
of which are actions intentionally performed by the developer in their
environment with the expectation that they will work with Otterize
— we use the same configuration value for both features.
Since marking a resource in Kubernetes for metrics scraping is a common
and may affect many resources, we aim to make the network policy more restrictive.
To achieve this, we obtain a configuration that identifies the metrics scraping
server. This allows us to create a network policy that only permits traffic between
the metrics scraping server and the deployment.
If we don't receive this configuration, we treat the situation as if automatic
enabling of metrics collection traffic is set to off.
We no longer use CLI for passing parameters to the operator (we use
environments variables), so it is OK to remove the old value
(AllowExternalTraffic), but there isn't any meaning in adding the new
flag
@zohar7ch zohar7ch force-pushed the zohar7ch/auto-enable-metrics-collection-traffic branch from 6be4468 to 9dfb822 Compare April 1, 2025 13:34
@zohar7ch zohar7ch merged commit c231e02 into main Apr 1, 2025
22 checks passed
@zohar7ch zohar7ch deleted the zohar7ch/auto-enable-metrics-collection-traffic branch April 1, 2025 13:53
@github-actions github-actions bot locked and limited conversation to collaborators Apr 1, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants