-
Notifications
You must be signed in to change notification settings - Fork 129
Description
Issue Body
Environment
| Component | Version |
|---|---|
| Trino Gateway | 17 |
| Trino Server | 478 |
| Trino JDBC Driver | 478 |
| Java | 21 |
| Kubernetes | Yes (with Istio service mesh) |
Trino Spooling Configuration
protocol.spooling.enabled=true
protocol.spooling.shared-secret-key=<redacted>
protocol.spooling.retrieval-mode=COORDINATOR_PROXY
protocol.spooling.inlining.enabled=false
protocol.spooling.initial-segment-size=64MB
protocol.spooling.max-segment-size=128MB
Trino Gateway Configuration
extraWhitelistPaths:
- '/v1/.*'
- '/v2/.*'
- '.*'
- '/v1/spooled/download/.*'
config:
serverConfig:
node.environment: test
http-server.http.port: 8080
http-server.http.enabled: true
http-server.process-forwarded: true
Problem Description
When clients connect through Trino Gateway to a Trino cluster with spooling enabled, requests to /v1/spooled/download/... return HTTP 404. The gateway does not proxy these requests to the backend Trino cluster.
Direct connection to Trino: Works (spooling works correctly) Connection through Trino Gateway: Fails with 404 on spooled download
Workaround: Setting SET SESSION protocol_spooling_enabled = false or using JDBC URL parameter protocol=SPOOLING_V1_DISABLED resolves the issue, but disables spooling entirely.
Expected Behavior
Trino Gateway should proxy /v1/spooled/download/... requests to the appropriate backend Trino cluster, similar to how it handles /v1/statement/... requests.
Actual Behavior
The gateway returns 404 for /v1/spooled/download/... requests without forwarding them to the backend.
Evidence
Trino Gateway logs show successful routing for /v1/statement requests but no routing logs for requests: /v1/spooled/download/
2026-02-17T11:27:34.029Z INFO io.trino.gateway.ha.handler.RoutingTargetHandler Rerouting [https://1x.20x.2x.2xx:443/v1/statement/executing/...]--> [https://backend-trino-cluster/v1/statement/executing/...]
Istio proxy logs confirm the request reaches the gateway but returns 404:
{
"path": "/v1/spooled/download/",
"response_code": "404",
"response_flags": "-",
"response_code_details": "via_upstream",
"upstream_host": "10.5x.1xx.1xx:8080",
"upstream_service_time": "1"
}
The 404 is returned by the gateway itself (1ms response time, via_upstream pointing to gateway port 8080), not by the backend Trino cluster.
Client Error
java.io.IOException: Could not open segment for streaming, got error 'Not Found' with code 404
at io.trino.jdbc.$internal.client.OkHttpSegmentLoader.load(OkHttpSegmentLoader.java:67)
at io.trino.jdbc.$internal.client.spooling.SpooledSegmentIterator.load(SpooledSegmentIterator.java:64)[wrapped] java.io.UncheckedIOException: java.io.IOException: Could not open segment for streaming, got error 'Not Found' with code 404 at io.trino.jdbc.$internal.client.spooling.SpooledSegmentIterator.load(SpooledSegmentIterator.java:70) at io.trino.jdbc.$internal.client.spooling.SpooledSegmentIterator.computeNext(SpooledSegmentIterator.java:91) at io.trino.jdbc.$internal.client.spooling.SpooledSegmentIterator.computeNext(SpooledSegmentIterator.java:33) at io.trino.jdbc.$internal.guava.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at io.trino.jdbc.$internal.guava.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at io.trino.jdbc.$internal.client.spooling.SegmentsIterator.computeNext(SegmentsIterator.java:49) at io.trino.jdbc.$internal.client.spooling.SegmentsIterator.computeNext(SegmentsIterator.java:28) at io.trino.jdbc.$internal.guava.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at io.trino.jdbc.$internal.guava.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at io.trino.jdbc.$internal.client.CloseableLimitingIterator.computeNext(CloseableLimitingIterator.java:55) at io.trino.jdbc.$internal.guava.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at io.trino.jdbc.$internal.guava.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at io.trino.jdbc.$internal.client.StatementClientV1$HeartbeatingResultRows$1.computeNext(StatementClientV1.java:727) at io.trino.jdbc.$internal.client.StatementClientV1$HeartbeatingResultRows$1.computeNext(StatementClientV1.java:722) at io.trino.jdbc.$internal.guava.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at io.trino.jdbc.$internal.guava.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at io.trino.jdbc.AsyncResultIterator.lambda$new$1(AsyncResultIterator.java:76) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) [wrapped] java.sql.SQLException: Error fetching results at io.trino.jdbc.AbstractTrinoResultSet.next(AbstractTrinoResultSet.java:230) at io.trino.jdbc.TrinoResultSet.next(TrinoResultSet.java:33)
Steps to Reproduce
- Configure Trino cluster with
protocol.spooling.enabled=trueandprotocol.spooling.retrieval-mode=COORDINATOR_PROXY - Deploy Trino Gateway v17 pointing to this cluster
- Connect via JDBC through the gateway
- Execute any query that returns results (e.g.,
SHOW CATALOGS) - Observe 404 error when client attempts to fetch spooled results
Additional Context
- The
extraWhitelistPathsconfiguration includes but this doesn't appear to enable proxying for these paths/v1/spooled/download/.*