Skip to content

resource based throttling: reject only adhoc queries #6947

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

erlan-z
Copy link
Contributor

@erlan-z erlan-z commented Aug 6, 2025

What this PR does:
Exclude rules queries from resource-based query rejection

Description:
This change improves the resource-based query rejection mechanism by ensuring that only ad-hoc API queries are rejected when resource thresholds are breached. Queries originating from rule evaluations are now excluded from rejection.

Introduced RequestSource metadata to distinguish between API and ruler queries.

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@dosubot dosubot bot added the component/rules Bits & bobs todo with rules and alerts: the ruler, config service etc. label Aug 6, 2025
@erlan-z erlan-z force-pushed the resource-based-throttling-reject-only-addhoc-queries branch from 5e39a8b to 53cedc7 Compare August 6, 2025 16:22
@erlan-z erlan-z changed the title resource based throttling: reject only addhock queries resource based throttling: reject only adhoc queries Aug 6, 2025
@erlan-z erlan-z force-pushed the resource-based-throttling-reject-only-addhoc-queries branch from 53cedc7 to 47da956 Compare August 6, 2025 16:25
Copy link
Contributor

@justinjung04 justinjung04 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@@ -2262,7 +2263,7 @@ func (i *Ingester) trackInflightQueryRequest() (func(), error) {

i.maxInflightQueryRequests.Track(i.inflightQueryRequests.Inc())

if i.resourceBasedLimiter != nil {
if i.resourceBasedLimiter != nil && requestmeta.RequestSourceFromContext(ctx) == requestmeta.SourceApi {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: what do you think of adding helper functions in requestmeta itself? ie. requestmeta.RequestFromApi(ctx) and requestmeta.RequestFromRuler(ctx)

@erlan-z erlan-z force-pushed the resource-based-throttling-reject-only-addhoc-queries branch from 47da956 to 8bb3d4f Compare August 8, 2025 18:19
Copy link
Contributor

@justinjung04 justinjung04 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you

Comment on lines +7 to +10
const (
SourceApi = "api"
SourceRuler = "ruler"
)
Copy link
Member

@SungJin1212 SungJin1212 Aug 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about reusing tripperware.SourceAPI and tripperware.SourceRuler?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/rules Bits & bobs todo with rules and alerts: the ruler, config service etc. size/M
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants