-
Notifications
You must be signed in to change notification settings - Fork 16.6k
Closed
Labels
area:Schedulerincluding HA (high availability) schedulerincluding HA (high availability) schedulerarea:corekind:bugThis is a clearly a bugThis is a clearly a bugneeds-triagelabel for new issues that we didn't triage yetlabel for new issues that we didn't triage yet
Description
Apache Airflow version
Other Airflow 2 version (please specify below)
If "Other Airflow 2 version" selected, which one?
2.10.4
What happened?
We deployed two schedulers and three workers in a cluster environment, and the task was interrupted in one execution, and the scheduler log showed that the task had failed, but the task was still queued in the metadatabase
What you think should happen instead?
I think it is airflow that produces scenes with inconsistent state in some scenes with high concurrency ...
How to reproduce
This problem is not easy to reproduce, we happened occasionally during production runs, but it was fine when the re-cleark allowed it
Operating System
Ubuntu 20.04.6 LTS
Versions of Apache Airflow Providers
2.10.4
Deployment
Other
Deployment details
using Helm and K8s for installation
Helm:v3.16.2+g13654a5
Kubernates: Client Version: v1.28.14 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: v1.28.14
Anything else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
area:Schedulerincluding HA (high availability) schedulerincluding HA (high availability) schedulerarea:corekind:bugThis is a clearly a bugThis is a clearly a bugneeds-triagelabel for new issues that we didn't triage yetlabel for new issues that we didn't triage yet