bug: TPC-DS cluster timeout expired

[dqhl76]

Search before asking

• I had searched in the issues and found no similar issues.

Version

main

What's Wrong?

When rerun linux / sqllogic / cluster (tpcds, 4c, http)

0: query failed: Databend sqllogictests error: Code: 1002, Text = status: Cancelled, message: "Timeout expired", details: [], metadata: MetadataMap { headers: {} }

How to Reproduce?

https://github.com/databendlabs/databend/actions/runs/21709553448/job/62611402313

https://github.com/databendlabs/databend/actions/runs/21818052033/job/62953653537

Are you willing to submit PR?

• Yes I am willing to submit a PR!

[dqhl76]

The TPC-DS cluster CI is encountering timeout errors, likely causing by high concurrency related to the aggregate spiller. The semaphore is set to 48 when storage is fs. (CI is actually fs).

databend/src/query/service/src/pipelines/processors/transforms/aggregator/serde/transform_spill_reader.rs

Lines 241 to 247 in 3ecf0d4

	let _guard = semaphore.acquire().await;
	let instant = Instant::now();
	let data = operator
	.read_with(&location)
	.range(data_range)
	.await?
	.to_vec();

databend/src/query/settings/src/settings_default.rs

Lines 1613 to 1621 in 3ecf0d4

	fn spill_io_requests(num_cpus: u64) -> u64 {
	match GlobalConfig::try_get_instance() {
	None => std::cmp::min(num_cpus, 64),
	Some(conf) => match conf.storage.params.is_fs() {
	true => 48,
	false => std::cmp::min(num_cpus, 64),
	},
	}
	}

We plan to switch to the new aggregate spiller soon, which use sync spiller and limited to the num_cpus. So it will not have the problems. (I tested in this PR: #19436)

Let's keep this issue open until new implementation is enabled in default and the problem is verified as resolved