Service binding rotation got stuck after creation failed with state CreateFailed

[xoronet]

Hi everyone,
we observed that the service binding rotation got stuck after one rotation attempt ended in a CreateFailed state. The CreateFailed state was cause by a temporary issue of the service provider's service broker. However, we would have expected that the rotation logic tries to resolve such an issue with further retries or continues at least with another attempt according to the configured rotation frequency.
We use the following configuration on our service binding:

  credentialsRotationPolicy:
    enabled: true
    rotatedBindingTTL: 168h
    rotationFrequency: 24h

With that, the rotation happens every day. After the service broker issue happened, the following days no rotation was executed.
Is this the expected behavior? Can we configure some kind of retry in case a new service binding ends up in CreateFailed state? Do you have advice how to monitor such a failure?
Thanks and best regards,
Clemens

[kerenlahav]

Hi @xoronet
errors should be retried, please specify the version you are using and attach the relevant logs
Thanks

[xoronet]

Hi @kerenlahav,

we use sap-btp-service-operator/controller:v0.9.3 as part of our Kyma Environment Instance.

I found some logs that could be relevant but not sure:

2025-11-27T06:00:56.086604310Z /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.21.0/pkg/internal/controller/controller.go:202
2025-11-27T06:00:56.086601110Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.1
2025-11-27T06:00:56.086598110Z /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.21.0/pkg/internal/controller/controller.go:300
2025-11-27T06:00:56.086595110Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem
2025-11-27T06:00:56.086591710Z /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.21.0/pkg/internal/controller/controller.go:340
2025-11-27T06:00:56.086588610Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler
2025-11-27T06:00:56.086585410Z /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.21.0/pkg/internal/controller/controller.go:119
2025-11-27T06:00:56.086582310Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile
2025-11-27T06:00:56.086578910Z /workspace/controllers/servicebinding_controller.go:203
2025-11-27T06:00:56.086572510Z github.com/SAP/sap-btp-service-operator/controllers.(*ServiceBindingReconciler).Reconcile
2025-11-27T06:00:56.086560710Z 2025-11-27T06:00:56Z ERROR controllers.ServiceBinding {"servicebinding": {"name":"XXX-identity-application-XXX-binding","namespace":"XXX"}, "correlation_id": "5baac5c3-a6dc-4be8-ac77-af59f8485c28", "XXX-identity-application-XXX-binding": "XXX", "error": "update binding is not allowed, this line should not be reached"}
2025-11-27T06:00:56.086481510Z 2025-11-27T06:00:56Z INFO controllers.ServiceBinding getting service instance named XXX-identity-application-XXX in namespace XXX for binding XXX-identity-application-XXX-binding in namespace XXX {"servicebinding": {"name":"XXX-identity-application-XXX-binding","namespace":"XXX"}, "correlation_id": "5baac5c3-a6dc-4be8-ac77-af59f8485c28", "XXX-identity-application-XXX-binding": "XXX"}

[kerenlahav]

thanks, what is the status of the failed binding?

[kerenlahav]

seems like the log is cut off in the middle please attach the full log, if there is sensitive data you can email it to me

[rakesh-garimella]

This issue is observed in multiple clusters. We see service binding status as failed. The message in conditions shows no error message but an empty space

'ServiceBinding create failed: '

[a-thaler]

For me I even have a stacktrace below the line:

2026-01-21T09:58:21Z	ERROR	controllers.ServiceBinding		{"servicebinding": {"name":"cap-java-hana-deployer-hana","namespace":"cap-java"}, "correlation_id": "892be892-e6c2-4621-b518-655b70d09d5e", "cap-java-hana-deployer-hana": "cap-java", "error": "update binding is not allowed, this line should not be reached"}
[github.com/SAP/sap-btp-service-operator/controllers.(*ServiceBindingReconciler).Reconcile](http://github.com/SAP/sap-btp-service-operator/controllers.(*ServiceBindingReconciler).Reconcile)
	/workspace/controllers/servicebinding_controller.go:224
[sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile](http://sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile)
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:216
[sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler](http://sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler)
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:461
[sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem](http://sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem)
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:421
[sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1](http://sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1)
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.22.4/pkg/internal/controller/controller.go:296

[kerenlahav]

@a-thaler can you please share the yaml for {"servicebinding": {"name":"cap-java-hana-deployer-hana","namespace":"cap-java"} (including the status)

[a-thaler]

This was the spec:

apiVersion: services.cloud.sap.com/v1
kind: ServiceBinding
metadata:
  name: cap-java-hana-deployer-hana
  namespace: cap-java
spec:
  externalName: cap-java-hana-deployer-hana
  secretKey: credentials
  secretName: cap-java-hana-deployer-hana
  serviceInstanceName: cap-java-hana

but I don't have the old status anymore as I fully wiped out the instance and the binding, waited some hours and re-applied. And now it works.

I think my scenario was not strictly related to the reported problem. I had a ServiceInstance which could not recover from a CreateFailed state without any detailed reason message. That's why I looked into the logs at some point.

[kerenlahav]

ok if it reproduces please attach the full resource, i have a suspicion and it will help to confirm it.

[hisarbalik]

Hi @kerenlahav, could you please share any updates on this issue when you have a moment?

[kerenlahav]

https://github.com/SAP/sap-btp-service-operator/releases/tag/v0.10.2