Skip to content

The Airflow task execution status is incorrect #55172

@coding-dragon520

Description

@coding-dragon520

Apache Airflow version

Other Airflow 2 version (please specify below)

If "Other Airflow 2 version" selected, which one?

2.10.4

What happened?

We deployed two schedulers and three workers in a cluster environment, and the task was interrupted in one execution, and the scheduler log showed that the task had failed, but the task was still queued in the metadatabase

What you think should happen instead?

I think it is airflow that produces scenes with inconsistent state in some scenes with high concurrency ...

How to reproduce

This problem is not easy to reproduce, we happened occasionally during production runs, but it was fine when the re-cleark allowed it

Operating System

Ubuntu 20.04.6 LTS

Versions of Apache Airflow Providers

2.10.4

Deployment

Other

Deployment details

using Helm and K8s for installation
Helm:v3.16.2+g13654a5
Kubernates: Client Version: v1.28.14 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: v1.28.14

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:Schedulerincluding HA (high availability) schedulerarea:corekind:bugThis is a clearly a bugneeds-triagelabel for new issues that we didn't triage yet

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions