Skip to content

IsNotFound errors are not handled gracefully in the scaler #1487

@Fedosin

Description

@Fedosin

Report

When HTTPScaledObjects are deployed rapidly (e.g. during batch deployments or performance testing), the scaler's controller-runtime informer cache may not have synced the new HTTPScaledObject by the time KEDA starts polling for metrics. This causes a cascade of errors: GetMetrics return hard errors, which kill the StreamIsActive gRPC stream, forcing KEDA to reconnect and retry producing many of ERROR-level log entries per deployment wave.

Expected Behavior

Scaler should gracefully handle IsNotFound errors.

Actual Behavior

Multiple errors are seen in the logs.

Steps to Reproduce the Problem

The timeline of the race condition:

  1. User deploys an HTTPScaledObject (e.g. my-app-httpso)
  2. KHA operator reconciles it and creates a KEDA ScaledObject
  3. KEDA immediately calls the external scaler's StreamIsActive -> IsActive -> GetMetrics
  4. The scaler does e.reader.Get() which reads from the controller-runtime informer cache
  5. The cache hasn't received the watch event yet -> "HTTPScaledObject not found"
  6. Hard error propagates up and terminates the gRPC stream
  7. KEDA reconnects, hits the same issue, producing error spam

Logs from KEDA HTTP Add-on

Full log

HTTP Add-on Version

0.12.2

Kubernetes Version

1.33

Platform

Google Cloud

Would you be open to contributing a fix?

Yes

Anything else?

No response

Metadata

Metadata

Assignees

Labels

Type

Projects

Status

To Triage

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions