Fix S3 lookup unbounded pagination with double call

[jorgee]

Problem

S3ObjectSummaryLookup.lookup() used an unbounded while(true) pagination loop that iterated through all objects sharing a given prefix (fetching 250 keys per page). On S3 buckets with large prefixes containing millions of objects, this caused excessive LIST API calls, high latency, and potential timeouts — just to check whether a single path exists.

Solution

Replace the unbounded loop with at most two bounded listObjects calls:

1. Call 1 — prefix(key), maxKeys(2): covers the common cases where the exact key or its first directory child appears within the first 2 lexicographic results.

2. Call 2 (fallback) — prefix(key + "/"), maxKeys(1): needed because S3 lists keys in lexicographic (UTF-8 byte) order, and characters like - (0x2D) and . (0x2E) sort before / (0x2F). This means sibling keys such as a-a/ and a.txt appear before a/ in the listing, potentially pushing the directory child outside Call 1's result window. Call 2 searches with prefix key/ directly, bypassing those siblings.

Example of the lexicographic ordering issue

Given keys a-a/file-3, a.txt, and a/file-1, S3 returns them as:

a-a/file-3   ← '-' (0x2D) < '/' (0x2F)
a.txt         ← '.' (0x2E) < '/' (0x2F)
a/file-1      ← '/' (0x2F) — the actual directory child

With maxKeys(2), Call 1 only sees a-a/file-3 and a.txt — neither matches. Call 2 with prefix a/ finds a/file-1, confirming that a is a directory.

Alternative to #6849

[netlify]

✅ Deploy Preview for nextflow-docs-staging canceled.

Name	Link
🔨 Latest commit	4d5fd24
🔍 Latest deploy log	https://app.netlify.com/projects/nextflow-docs-staging/deploys/699889e920365d00089931fa

[pditommaso]

Well done. Considering is really a tricky issue, I took the liberty to extend the docs/comment