AWS SQS message broker support status in Airflow 3.x with CeleryExecutor

[bun-paul-kwon]

Hello Airflow community,

We are currently using AWS MWAA (Celery Executor) and considering migrating to Airflow 3.0.2 on Kubernetes (AWS EKS).

• Image: apache/airflow:3.0.2
• Helm chart: Airflow 1.18.0

In our environment, we usually run 300–500 tasks via KubernetesPodOperator.
We have considered using KubernetesExecutor, but since it creates an additional worker pod for each task, this would likely result in a substantial increase in the total number of pods in our cluster. To avoid this scaling overhead, we are planning to continue with CeleryExecutor.

In addition, our broader goal is to reduce operational complexity by leveraging managed services for stateful components (such as the metadata database and the message broker) whenever possible.

Our main question is:
👉 Does Airflow still not officially support AWS SQS as a Message Broker for Celery Executor in Airflow 3.x?

In this 2022 discussion
, it was mentioned that AWS SQS was not officially supported. We would like to kindly confirm whether this is still the case in Airflow 3.0.2.

Thank you very much in advance for your clarification and support!

[potiuk]

Yes it does not. But if you would like to test and contribute it - you are most welcome. Generally speaking airflow has > 3500 contributors and usually those who need certain feature are most likely (and best) potential contributors of certain features - especialy in this case where it means that this all should be tested and executed in our CI - we have recently added localstack support for integrations, so - potentially sqs could be tested with it - generally adding such support would require adding integration tests that pass in our CI.

So - if you want to contribute or can find someone who could - it's all possible.