Questions about the running time slice of Tasks polled out by MultiLevelSplitQueue

[lancelly]

In the multi-level feedback scheduling mechanism of operating systems, tasks in lower-priority queues are executed for longer time slices. However, I noticed that in Trino's MultiLevelSplitQueue, each level's task queue, once selected, runs for the same duration of time slice. Is there any special consideration for this approach?

[hashhar]

While the time-slice is uniform the total time each level occupies on the CPU varies based on a fraction - controlled via levelScheduledTime in https://github.com/trinodb/trino/blob/master/core/trino-main/src/main/java/io/trino/execution/executor/timesharing/MultilevelSplitQueue.java.

I'd suggest you to see the "Resource Management" section in the original paper - https://trino.io/paper.html.
It explains the rationale.
The idea is that queries which use more CPU time are penalised to ensure the system favours interactive workloads. (That's a very large simplification but the paper expands on that).

[lancelly]

I understand your point now. Thank you very much for your answer!

[lancelly]

I have another question. The TimeSharingTaskHandle seems to treat the tasks derived from a single PlanFragment as a whole, and the cumulative execution time of these tasks leads to tasks belonging to the same PlanFragment entering a higher-level queue together. Although at the granularity of PlanFragment (SqlTaskExecution), the allocation of time slices seems reasonable, if a Query has many PlanFragments (I am just starting to learn Trino, so I'm not sure if this happens in Trino, and I am also curious about how many PlanFragments a Query can have under Trino's regular workload), it seems that the overall CPU utilization of the entire query might still be higher than expected. My question is, why doesn't Trino calculate the cumulative execution time of all Drivers derived from all PlanFragments of a Query distributed to the current node as the basis for determining the queue level for tasks, instead of only considering the Drivers derived from a single PlanFragment? What are the design reasons for this approach? The reasons I can think of are: 1. The current method is simpler in terms of coding. 2. If a query has many PlanFragments, then the concurrency for updating TimeSharingTaskHandle will be higher, which will have a greater impact on performance. 3. Without knowing the workload, considering the entire Query's execution time is more difficult than considering the execution time of a single PlanFragment.