feat: use PascalCase for Query/Task / allow configOverrides for exchange manager

Author: dervoeti

deploy/helm/trino-operator/crds/crds.yaml

@@ -118,7 +118,7 @@ spec:
                           nullable: true
                           type: boolean
                         exchangeManager:
-                          description: Exchange manager configuration for spooling intermediate data during fault tolerant execution. Required when using `TASK` retry policy, optional for `QUERY` retry policy.
+                          description: Exchange manager configuration for spooling intermediate data during fault tolerant execution. Required when using `Task` retry policy, optional for `Query` retry policy.
                           nullable: true
                           oneOf:
                             - required:
@@ -128,6 +128,12 @@ spec:
                             - required:
                                 - local
                           properties:
+                            configOverrides:
+                              additionalProperties:
+                                type: string
+                              default: {}
+                              description: The `configOverrides` allow overriding arbitrary exchange manager properties.
+                              type: object
                             hdfs:
                               description: HDFS-based exchange manager.
                               properties:
@@ -344,7 +350,7 @@ spec:
                               type: integer
                           type: object
                         queryRetryAttempts:
-                          description: Maximum number of times Trino may attempt to retry a query before declaring it failed. Only applies to `QUERY` retry policy.
+                          description: Maximum number of times Trino may attempt to retry a query before declaring it failed. Only applies to `Query` retry policy.
                           format: uint32
                           minimum: 0.0
                           nullable: true
@@ -363,13 +369,13 @@ spec:
                           nullable: true
                           type: string
                         retryPolicy:
-                          description: The retry policy for fault tolerant execution. `QUERY` retries entire queries, `TASK` retries individual tasks. When set to `TASK`, an exchange manager must be configured.
+                          description: The retry policy for fault tolerant execution. `Query` retries entire queries, `Task` retries individual tasks. When set to `Task`, an exchange manager must be configured.
                           enum:
-                            - query
-                            - task
+                            - Query
+                            - Task
                           type: string
                         taskRetryAttemptsPerTask:
-                          description: Maximum number of times Trino may attempt to retry a single task before declaring the query failed. Only applies to `TASK` retry policy.
+                          description: Maximum number of times Trino may attempt to retry a single task before declaring the query failed. Only applies to `Task` retry policy.
                           format: uint32
                           minimum: 0.0
                           nullable: true

docs/modules/trino/examples/usage-guide/fault-tolerant-execution.yaml

@@ -11,7 +11,7 @@ spec:
       matchLabels:
         trino: trino-fault-tolerant
     faultTolerantExecution:
-      retryPolicy: TASK
+      retryPolicy: Task
       taskRetryAttemptsPerTask: 4
       retryInitialDelay: 10s
       retryMaxDelay: 60s

docs/modules/trino/pages/usage-guide/fault-tolerant-execution.adoc

@@ -23,20 +23,20 @@ To enable the feature, you need to configure it in your `TrinoCluster` resource
 spec:
   clusterConfig:
     faultTolerantExecution:
-      retryPolicy: QUERY  # <1>
+      retryPolicy: Query  # <1>
       queryRetryAttempts: 3  # <2>
 ----
-<1> The retry policy - either `QUERY` or `TASK`
-<2> Maximum number of times to retry a query (QUERY policy only)
+<1> The retry policy - either `Query` or `Task`
+<2> Maximum number of times to retry a query (Query policy only)
 
 == Retry policies
 
 The `retryPolicy` configuration property designates whether Trino retries entire queries or a query's individual tasks in the event of failure.
 
-=== QUERY retry policy
+=== Query retry policy
 
-A `QUERY` retry policy instructs Trino to automatically retry a query in the event of an error occurring on a worker node.
-A `QUERY` retry policy is recommended when the majority of the Trino cluster's workload consists of many small queries.
+A `Query` retry policy instructs Trino to automatically retry a query in the event of an error occurring on a worker node.
+A `Query` retry policy is recommended when the majority of the Trino cluster's workload consists of many small queries.
 
 By default, Trino does not implement fault tolerance for queries whose result set exceeds 32MB in size.
 This limit can be increased by modifying the `exchangeDeduplicationBufferSize` configuration property to be greater than the default value of `32MB`, but this results in higher memory usage on the coordinator.
@@ -47,28 +47,28 @@ This limit can be increased by modifying the `exchangeDeduplicationBufferSize` c
 spec:
   clusterConfig:
     faultTolerantExecution:
-      retryPolicy: QUERY
+      retryPolicy: Query
       queryRetryAttempts: 3
       exchangeDeduplicationBufferSize: 64MB  # Increased from default 32MB
 ...
 ----
 
-=== TASK retry policy
+=== Task retry policy
 
-A `TASK` retry policy instructs Trino to retry individual query tasks in the event of failure.
+A `Task` retry policy instructs Trino to retry individual query tasks in the event of failure.
 You **must** configure an exchange manager to use the task retry policy.
 This policy is recommended when executing large batch queries, as the cluster can more efficiently retry smaller tasks within the query rather than retry the whole query.
 
-IMPORTANT: A `TASK` retry policy is best suited for long-running queries, but this policy can result in higher latency for short-running queries executed in high volume.
-As a best practice, it is recommended to run a dedicated cluster with a `TASK` retry policy for large batch queries, separate from another cluster that handles short queries.
+IMPORTANT: A `Task` retry policy is best suited for long-running queries, but this policy can result in higher latency for short-running queries executed in high volume.
+As a best practice, it is recommended to run a dedicated cluster with a `Task` retry policy for large batch queries, separate from another cluster that handles short queries.
 There are tools that can help you achieve this by automatically routing queries based on certain criteria (such as query estimates or user) to different Trino clusters. Notable mentions are link:https://github.com/stackabletech/trino-lb[trino-lb {external-link-icon}^] and link:https://github.com/trinodb/trino-gateway[trino-gateway {external-link-icon}^].
 
 [source,yaml]
 ----
 spec:
   clusterConfig:
     faultTolerantExecution:
-      retryPolicy: TASK
+      retryPolicy: Task
       taskRetryAttemptsPerTask: 4
       exchangeManager:
         s3:
@@ -82,20 +82,20 @@ spec:
 == Exchange manager
 
 Exchange spooling is responsible for storing and managing spooled data for fault-tolerant execution.
-You can configure a filesystem-based exchange manager that stores spooled data in a specified location, such as AWS S3 and S3-compatible systems, Azure Blob Storage or HDFS.
+You can configure a filesystem-based exchange manager that stores spooled data in a specified location, such as AWS S3 and S3-compatible systems, HDFS, or local filesystem.
 
-NOTE: An exchange manager is required when using the `TASK` retry policy and optional for the `QUERY` retry policy.
+NOTE: An exchange manager is required when using the `Task` retry policy and optional for the `Query` retry policy.
 
 === S3-compatible storage
 
-You can use S3-compatible storage systems for exchange spooling, including AWS S3, MinIO, and Google Cloud Storage.
+You can use S3-compatible storage systems for exchange spooling, including AWS S3 and MinIO.
 
 [source,yaml]
 ----
 spec:
   clusterConfig:
     faultTolerantExecution:
-      retryPolicy: TASK
+      retryPolicy: Task
       exchangeManager:
         s3:
           baseDirectories:  # <1>
@@ -122,81 +122,7 @@ spec:
 <1> Multiple S3 buckets can be specified to distribute I/O load
 <2> S3 connection defined as a reference to an xref:concepts:s3.adoc[S3Connection] resource
 
-For Google Cloud Storage, you can use GCS buckets with S3 compatibility:
-
-[source,yaml]
-----
-spec:
-  clusterConfig:
-    faultTolerantExecution:
-      exchangeManager:
-        s3:
-          baseDirectories:
-            - "gs://my-gcs-bucket/trino-spooling"
-          connection:
-            inline:
-              host: storage.googleapis.com
-              port: 443
-              accessStyle: Path
-              credentials:
-                secretClass: gcs-hmac-credentials
-              tls:
-                verification:
-                  server:
-                    caCert:
-                      webPki: {}
-          gcsServiceAccountKey:
-            secretClass: "gcs-service-account-secret-class"
-            key: "service-account.json"
-----
-
-=== Azure Blob Storage
-
-You can configure Azure Blob Storage as the exchange spooling destination:
-
-[source,yaml]
-----
-spec:
-  clusterConfig:
-    faultTolerantExecution:
-      retryPolicy: TASK
-      exchangeManager:
-        azure:
-          baseDirectories:
-            - "abfs://[email protected]/exchange-spooling"
-          secretClass: azure-credentials  # <1>
-          key: connectionString  # <2>
-----
-<1> SecretClass providing the Azure connection string
-<2> Key name in the Secret that contains the connection string (defaults to `connectionString`)
-
-The Azure connection string should be provided via a SecretClass that refers to a Kubernetes Secret containing the Azure storage account connection string, like this:
-
-[source,yaml]
-----
-apiVersion: secrets.stackable.tech/v1alpha1
-kind: SecretClass
-metadata:
-  name: azure-credentials
-spec:
-  backend:
-    k8sSearch:
-      searchNamespace:
-        pod: {}
-----
-
-[source,yaml]
-----
-apiVersion: v1
-kind: Secret
-metadata:
-  name: azure-secret
-  labels:
-    secrets.stackable.tech/class: azure-credentials
-type: Opaque
-stringData:
-  connectionString: "DefaultEndpointsProtocol=https;AccountName=mystorageaccount;AccountKey=your_account_key;EndpointSuffix=core.windows.net"
-----
+For storage systems like Google Cloud Storage or Azure Blob Storage, you can use the S3-compatible configuration with `configOverrides` to provide the necessary exchange manager properties.
 
 === HDFS storage
 
@@ -207,7 +133,7 @@ You can configure HDFS as the exchange spooling destination:
 spec:
   clusterConfig:
     faultTolerantExecution:
-      retryPolicy: TASK
+      retryPolicy: Task
       exchangeManager:
         hdfs:
           baseDirectories:
@@ -229,7 +155,7 @@ A local directory can only be used for exchange in a distributed cluster if the
 spec:
   clusterConfig:
     faultTolerantExecution:
-      retryPolicy: TASK
+      retryPolicy: Task
       exchangeManager:
         local:
           baseDirectories:
@@ -286,7 +212,7 @@ When using connectors that do not explicitly support fault-tolerant execution, y
 
 == Example
 
-Here's an example of a Trino cluster with fault-tolerant execution enabled using the `TASK` retry policy and MinIO backed S3 as the exchange manager:
+Here's an example of a Trino cluster with fault-tolerant execution enabled using the `Task` retry policy and MinIO backed S3 as the exchange manager:
 
 [source,bash]
 ----