Insights on Trino’s pagination logic

[kuripart]

Hi,

Opening this discussion from the slack question asked in #dev: https://trinodb.slack.com/archives/CP1MUNEUX/p1701371089507779

We are trying to understand how trino’s pagination logic calculates the total number of pages and the rows per page to be returned on the execution of a single query generally (irrespective of the connector used), eg: select * from abc;?

In the docs, it says:
A GET to the nextUri attribute returns the next batch of query results.

Would appreciate if someone could explain and give some stats on this calculation, along with details on what properties or limits that influence it. From our understanding it varies on the table or file size, and is random. Thanks.

[martint]

There's no calculation ahead of time since execution is pipelined and results are produced and sent to the client while the query is still running. There is a target payload size for the each response in the client protocol, which is controlled by the targetResultSize query parameter. It defaults to 1MB.

See:

• trino/core/trino-main/src/main/java/io/trino/server/protocol/ExecutingStatementResource.java

  Line 162 in ae7849e

  @QueryParam("targetResultSize") DataSize targetResultSize,

• trino/core/trino-main/src/main/java/io/trino/server/protocol/ExecutingStatementResource.java

  Lines 216 to 221 in ae7849e

  	if (targetResultSize == null) {
  	targetResultSize = DEFAULT_TARGET_RESULT_SIZE;
  	}
  	else {
  	targetResultSize = Ordering.natural().min(targetResultSize, MAX_TARGET_RESULT_SIZE);
  	}