@@ -29,6 +29,7 @@ following contents, modifying the properties as appropriate:
2929 clp.metadata-db-name =clp_db
3030 clp.metadata-db-user =clp_user
3131 clp.metadata-db-password =clp_password
32+ clp.metadata-filter-config =/path/to/metadata-filter-config.json
3233 clp.metadata-table-prefix =clp_
3334 clp.split-provider-type =mysql
3435
@@ -68,6 +69,7 @@ Property Name Description
6869 database name is not specified in the URL.
6970``clp.metadata-db-password `` The password for the metadata database user. This option is required if
7071 ``clp.metadata-provider-type `` is set to ``mysql ``.
72+ ``clp.metadata-filter-config `` The absolute path of the metadata filter config file.
7173``clp.metadata-table-prefix `` A string prefix prepended to all metadata table names when querying the
7274 database. Useful for namespacing or avoiding collisions. This option is
7375 required if ``clp.metadata-provider-type `` is set to ``mysql ``.
@@ -94,6 +96,35 @@ If you prefer to use a different source--or the same source with a custom implem
9496implementations of the ``ClpMetadataProvider `` and ``ClpSplitProvider `` interfaces, and configure the connector
9597accordingly.
9698
99+ Metadata Filter Config File
100+ ----------------------------
101+
102+ The configuration file defines metadata filters for different scopes:
103+
104+ - **Catalog-level **: applies to all schemas and tables under the catalog.
105+ - **Schema-level **: applies to all tables under the specified catalog and schema.
106+ - **Table-level **: applies to the fully qualified ``catalog.schema.table ``.
107+
108+ Each scope maps to a list of filter definitions. Each filter includes:
109+
110+ - ``filterName ``: must match a column name in the table’s schema.
111+
112+ .. note ::
113+ Only numeric-type columns can currently be used as metadata filters.
114+
115+ - ``rangeMapping `` *(optional) *: specifies how the filter should be remapped when it targets metadata-only columns.
116+
117+ .. note ::
118+ This option is only valid if the column is numeric type.
119+
120+ For example, a condition like:
121+ ::
122+ "msg.timestamp" > 1234 AND "msg.timestamp" < 5678
123+ will be rewritten as:
124+ ::
125+ end_timestamp > 1234 AND begin_timestamp < 5678
126+ This ensures that metadata-based filtering produces a superset of the actual result.
127+
97128Data Types
98129----------
99130
0 commit comments