only two tasks can be in a running state

[OPSTime]

Self Checks

• I have read the Contributing Guide and Language Policy.
• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report, otherwise it will be closed.
• 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
• Please do not modify this template :) and fill in all the required fields.

Dify version

1.11.2

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

The concurrency capability is insufficient; in the same workflow, only two tasks can be in a running state simultaneously when invoked via webhook.

✔️ Expected Behavior

Increase the number of concurrent tasks

❌ Actual Behavior

Currently, only two tasks can run in parallel.

[fatelei]

you expect four ?, the two has success

[dosubot]

The concurrency limit for running tasks in your Dify self-hosted Docker setup is controlled by several environment variables. The most relevant are:

• NEXT_PUBLIC_BATCH_CONCURRENCY (default 5): controls how many batch tasks can run in parallel. You can increase this in your .env file and in docker-compose.yaml to raise the concurrency cap for webhook-invoked workflows. This variable was added specifically to make concurrency configurable in recent versions (details).
• CELERY_WORKER_AMOUNT, CELERY_MAX_WORKERS, CELERY_MIN_WORKERS, and CELERY_AUTO_SCALE: these control the number of Celery worker processes, which also affects how many tasks can be processed at once (see config, docker-compose).
• TENANT_ISOLATED_TASK_CONCURRENCY (default 1): may limit concurrency per tenant.
• MAX_PARALLEL_LIMIT (default 10): sets a global parallelism cap.

To increase concurrency, edit these variables in your .env and docker-compose.yaml, then restart your Docker services. For example, set NEXT_PUBLIC_BATCH_CONCURRENCY=5 or higher, and adjust Celery worker counts as needed.

Keep in mind that raising these limits will increase CPU, memory, and database/Redis load. Monitor your system resources after making changes to avoid overloading your server.

The observed limit of two concurrent tasks is likely due to the default values of these variables. Adjusting them as described should let you run more tasks in parallel via webhook.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[OPSTime]

you expect four ?, the two has success

Regardless of how much the client increases concurrency, when refreshing and viewing the workflow logs, there are always only two tasks simultaneously in the running state, preventing any improvement in processing speed.

[OPSTime]

The concurrency limit for running tasks in your Dify self-hosted Docker setup is controlled by several environment variables. The most relevant are:

• NEXT_PUBLIC_BATCH_CONCURRENCY (default 5): controls how many batch tasks can run in parallel. You can increase this in your .env file and in docker-compose.yaml to raise the concurrency cap for webhook-invoked workflows. This variable was added specifically to make concurrency configurable in recent versions (details).
• CELERY_WORKER_AMOUNT, CELERY_MAX_WORKERS, CELERY_MIN_WORKERS, and CELERY_AUTO_SCALE: these control the number of Celery worker processes, which also affects how many tasks can be processed at once (see config, docker-compose).
• TENANT_ISOLATED_TASK_CONCURRENCY (default 1): may limit concurrency per tenant.
• MAX_PARALLEL_LIMIT (default 10): sets a global parallelism cap.

To increase concurrency, edit these variables in your .env and docker-compose.yaml, then restart your Docker services. For example, set NEXT_PUBLIC_BATCH_CONCURRENCY=5 or higher, and adjust Celery worker counts as needed.

Keep in mind that raising these limits will increase CPU, memory, and database/Redis load. Monitor your system resources after making changes to avoid overloading your server.

The observed limit of two concurrent tasks is likely due to the default values of these variables. Adjusting them as described should let you run more tasks in parallel via webhook.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

   

Adjusted the settings but had no effect.

[s1mbah]

The concurrency limit for running tasks in your Dify self-hosted Docker setup is controlled by several environment variables. The most relevant are:

• NEXT_PUBLIC_BATCH_CONCURRENCY (default 5): controls how many batch tasks can run in parallel. You can increase this in your .env file and in docker-compose.yaml to raise the concurrency cap for webhook-invoked workflows. This variable was added specifically to make concurrency configurable in recent versions (details).
• CELERY_WORKER_AMOUNT, CELERY_MAX_WORKERS, CELERY_MIN_WORKERS, and CELERY_AUTO_SCALE: these control the number of Celery worker processes, which also affects how many tasks can be processed at once (see config, docker-compose).
• TENANT_ISOLATED_TASK_CONCURRENCY (default 1): may limit concurrency per tenant.
• MAX_PARALLEL_LIMIT (default 10): sets a global parallelism cap.

To increase concurrency, edit these variables in your .env and docker-compose.yaml, then restart your Docker services. For example, set NEXT_PUBLIC_BATCH_CONCURRENCY=5 or higher, and adjust Celery worker counts as needed.
Keep in mind that raising these limits will increase CPU, memory, and database/Redis load. Monitor your system resources after making changes to avoid overloading your server.
The observed limit of two concurrent tasks is likely due to the default values of these variables. Adjusting them as described should let you run more tasks in parallel via webhook.
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
   

Adjusted the settings but had no effect.

Same here - adjusting these parameters does not affect the processing speed of workflows with 2+ parallel branches.