Skip to content

Commit da2ef4e

Browse files
Merge pull request kubernetes#1507 from wking/faster-update-PromQL-cache-warming
enhancements/update/targeted-update-edge-blocking: 1s between different PromQL
2 parents 56c322e + a92b100 commit da2ef4e

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

enhancements/update/targeted-update-edge-blocking.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -320,7 +320,7 @@ Additionally, the operator will continually re-evaluate the blocking conditional
320320
The timing of the evaluation and freshness are largely internal details, but to avoid [consuming excessive monitoring resources](#malicious-conditions) and because [the rules should be based on slowly-changing state](#clusters-moving-into-the-vulnerable-state-after-updating), the operator will handle polling with the following restrictions:
321321

322322
* The cluster-version operator will cache polling results for each query, so a single query which is used in evaluating multiple risks over multiple conditional update targets will only be evaluated once per round.
323-
* After evaluating a PromQL query, the cluster-version operator will wait at least 10 minutes before evaluating any PromQL.
323+
* After evaluating a PromQL query, the cluster-version operator will wait at least 1 second before evaluating any PromQL.
324324
This delay will not be persisted between operator restarts, so a crash-looping CVO may result in higher PromQL load.
325325
But a crash-looping CVO will also cause the `KubePodCrashLooping` alert to fire, which will summon the cluster administrator.
326326
* After evaluating a PromQL query, the cluster-version operator will wait at least an hour before evaluating that PromQL query again.
@@ -687,6 +687,8 @@ If the Prometheus implementation is not sufficiently hardened, malicious PromQL
687687
[Future monitoring improvements][mon-1772] might reduce the risk of expensive queries.
688688
And it's also possible to teach the cluster-version operator to reject PromQL that does not match expected patterns.
689689

690+
Attacks can also be mitigated by pointing ClusterVersion `spec.upstream` at an uncompromised update service, or by clearing `channel` to ask the CVO to not retrieve update recommendations at all.
691+
690692
#### Clusters without local Prometheus stacks
691693

692694
The Prometheus and monitoring stacks are fairly resource-intensive.

0 commit comments

Comments
 (0)