feat(presto-clp): Update docs and config generation to support reading archives from S3 and Presto split-filtering config. (#1228)

Author: anlowee

docs/src/user-docs/guides-using-presto.md

@@ -16,7 +16,7 @@ been merged into the main Presto repository so that you can use official Presto
 
 ## Requirements
 
-* [CLP][clp-releases] (clp-json) v0.4.0 or higher
+* [CLP][clp-releases] (clp-json) v0.5.0 or higher
 * [Docker] v28 or higher
 * [Docker Compose][docker-compose] v2.20.2 or higher
 * Python
@@ -31,7 +31,7 @@ Using Presto with CLP requires:
 
 ### Setting up CLP
 
-1. Follow the [quick-start](./quick-start/index.md) guide to download and extract the CLP package,
+1. Follow the [quick-start](quick-start/index.md) guide to download and extract the CLP package,
    but don't start the package just yet.
 2. Before starting the package, update the package's config as follows:
 
@@ -55,7 +55,15 @@ Using Presto with CLP requires:
     deployment infrastructure.
     :::
 
-3. Continue following the [quick-start](./quick-start/index.md#using-clp) guide to start CLP and
+3. If you'd like to store your compressed logs on S3, follow the
+   [using object storage](guides-using-object-storage/index.md) guide.
+
+   :::{note}
+   Currently, the Presto integration only supports the
+   [credentials](guides-using-object-storage/clp-config.md#credentials) authentication type.
+   :::
+
+4. Continue following the [quick-start](./quick-start/index.md#using-clp) guide to start CLP and
    compress your logs. A sample dataset that works well with Presto is [postgresql].
 
 ### Setting up Presto
@@ -78,17 +86,19 @@ Using Presto with CLP requires:
 
 4. Configure Presto to use CLP's metadata database as follows:
 
-    * Open and edit `coordinator/config-template/metadata-filter.json`.
+    * Open and edit `coordinator/config-template/split-filter.json`.
     * For each dataset you want to query, add a filter config of the form:
 
       ```json
       {
         "clp.default.<dataset>": [
           {
             "columnName": "<timestamp-key>",
-            "rangeMapping": {
-              "lowerBound": "begin_timestamp",
-              "upperBound": "end_timestamp"
+            "customOptions": {
+              "rangeMapping": {
+                "lowerBound": "begin_timestamp",
+                "upperBound": "end_timestamp"
+              }
             },
             "required": false
           }
@@ -108,7 +118,7 @@ Using Presto with CLP requires:
     docker compose up
     ```
 
-    * To use more than Presto worker, you can use the `--scale` option as follows:
+    * To use more than one Presto worker, you can use the `--scale` option as follows:
 
       ```bash
       docker compose up --scale presto-worker=<num-workers>
@@ -143,8 +153,20 @@ Each dataset in CLP shows up as a table in Presto. To show all available dataset
 SHOW TABLES;
 ```
 
+:::{note}
 If you didn't specify a dataset when compressing your logs in CLP, your logs will have been stored
-in the `default` dataset. To query the logs in this dataset:
+in the `default` dataset.
+:::
+
+To show all available columns in the `default` dataset:
+
+```sql
+DESCRIBE default;
+```
+
+If you wish to show the columns of a different dataset, replace `default` above.
+
+To query the logs in this dataset:
 
 ```sql
 SELECT * FROM default LIMIT 1;
@@ -164,11 +186,10 @@ The Presto CLP integration has the following limitations at present:
 * Nested fields containing special characters cannot be queried (see [y-scope/presto#8]). Allowed
   characters are alphanumeric characters and underscores. To get around this limitation, you'll
   need to preprocess your logs to remove any special characters.
-* Only logs stored on the filesystem, rather than S3, can be queried through Presto.
 
 These limitations will be addressed in a future release of the Presto integration.
 
-[clp-connector-docs]: https://docs.yscope.com/presto/connector/clp.html#metadata-filter-config-file
+[clp-connector-docs]: https://docs.yscope.com/presto/connector/clp.html#split-filter-config-file
 [clp-releases]: https://github.com/y-scope/clp/releases
 [docker-compose]: https://docs.docker.com/compose/install/
 [Docker]: https://docs.docker.com/engine/install/

tools/deployment/presto-clp/.gitignore

@@ -0,0 +1 @@
+/worker/config-template/clp.properties

tools/deployment/presto-clp/coordinator.env

@@ -1,5 +1,6 @@
 # clp.properties
 PRESTO_COORDINATOR_CLPPROPERTIES_METADATA_PROVIDER_TYPE=mysql
+PRESTO_COORDINATOR_CLPPROPERTIES_SPLIT_FILTER_PROVIDER_TYPE=mysql
 PRESTO_COORDINATOR_CLPPROPERTIES_SPLIT_PROVIDER=mysql
 
 # config.properties

tools/deployment/presto-clp/coordinator/config-template/clp.properties

@@ -6,4 +6,5 @@ clp.metadata-db-user=${PRESTO_COORDINATOR_CLPPROPERTIES_METADATA_DATABASE_USER}
 clp.metadata-db-password=${PRESTO_COORDINATOR_CLPPROPERTIES_METADATA_DATABASE_PASSWORD}
 clp.metadata-table-prefix=${PRESTO_COORDINATOR_CLPPROPERTIES_METADATA_TABLE_PREFIX}
 clp.split-provider-type=${PRESTO_COORDINATOR_CLPPROPERTIES_SPLIT_PROVIDER}
-clp.metadata-filter-config=/opt/presto-server/etc/metadata-filter.json
+clp.split-filter-provider-type=${PRESTO_COORDINATOR_CLPPROPERTIES_SPLIT_FILTER_PROVIDER_TYPE}
+clp.split-filter-config=/opt/presto-server/etc/split-filter.json