-
Notifications
You must be signed in to change notification settings - Fork 151
IsNotFound errors are not handled gracefully in the scaler #1487
Copy link
Copy link
Labels
Description
Report
When HTTPScaledObjects are deployed rapidly (e.g. during batch deployments or performance testing), the scaler's controller-runtime informer cache may not have synced the new HTTPScaledObject by the time KEDA starts polling for metrics. This causes a cascade of errors: GetMetrics return hard errors, which kill the StreamIsActive gRPC stream, forcing KEDA to reconnect and retry producing many of ERROR-level log entries per deployment wave.
Expected Behavior
Scaler should gracefully handle IsNotFound errors.
Actual Behavior
Multiple errors are seen in the logs.
Steps to Reproduce the Problem
The timeline of the race condition:
- User deploys an HTTPScaledObject (e.g.
my-app-httpso) - KHA operator reconciles it and creates a KEDA ScaledObject
- KEDA immediately calls the external scaler's
StreamIsActive->IsActive->GetMetrics - The scaler does
e.reader.Get()which reads from the controller-runtime informer cache - The cache hasn't received the watch event yet ->
"HTTPScaledObject not found" - Hard error propagates up and terminates the gRPC stream
- KEDA reconnects, hits the same issue, producing error spam
Logs from KEDA HTTP Add-on
Full log
HTTP Add-on Version
0.12.2
Kubernetes Version
1.33
Platform
Google Cloud
Would you be open to contributing a fix?
Yes
Anything else?
No response
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
To Triage