Running multiple Iceberg catalogs act as single catalog

[szisiu]

Environment:

• Ubuntu v22.04.4 LTS (host)
• Docker version v28.0.1, build 068a01e
• Containerized Trino v471 + HMS v3.x + MySQL v8.x + MinIO v2025-02

My containerized test deployment consists of 1 trino coordinator / 1 worker with iceberg connector, 1 minio service, 1 mysql and 1 hive metastore service (hms) .

Prepared and attached files to be used to reproduce the problem:

trino-docker-test.zip

Context:

• I need to keep some data in a separate catalogs and buckets, but still access those catalogs from within a single query to trino

• Created two or more iceberg connector files (eg iceberg1.properties, iceberg2.properties) witch some settings.

• Trino detects and load them correctly:

• Then ran a series of queries (MinIO bucket is created using docker):

CREATE SCHEMA iceberg1.testschema
      WITH ( 
		"location" = 's3a://testbucketone/testschema'
	);

CREATE TABLE iceberg1.testschema.testtbl (
    c1,
    c2
)
WITH (
    format = 'PARQUET'
)
AS VALUES
    (2021, 10000),
    (2022, 20000);

select * from iceberg1.testschema.testtbl;

• The output is:

• Running select * from iceberg1.testschema.testtbl; or select * from iceberg2.testschema.testtbl; outputs the same data.

• The data is placed in the expected bucket. Second one is empty.

• Dropping iceberg2.testschema removes the data from iceberg1 and iceberg2

• Tested different settings for Iceberg without luck and the similar approach with multiple mongodb connector configs and it worked without any issues.

If you need more info - let me know.

Thanks in advance.

[szisiu]

Experiencing the same issue when switched to Trino v474

[szisiu]

OK. I found for example in MongoDB connector docs (as well as in MySQL, Kafka, etc.) explicit statement that You can have as many catalogs as you need:

Is it possible to have this feature for the Iceberg connector?

[fischerc-rki]

You can use a single hms instance for different iceberg catalogs in trino. The following two steps must be taken (as far as I know):

• Create the catalog entries in the hms backing database (stackoverflow reference)

• Set the property hive.metastore.thrift.catalog-name in the properties file of the connector in trino

[szisiu]

thanks. while it might work it looks like an unofficial workaround.

[constantinfischer]

I wouldn't call it an unofficial workaround, since the docs regarding the thrift metastore property hive.metastore.thrift.catalog-name describe the interaction with the hms:

The term “Hive metastore catalog name” refers to the abstraction concept within Hive, enabling various systems to connect to distinct, independent catalogs stored in the metastore. By default, the catalog name in Hive metastore is set to “hive.” When this configuration property is left empty, the default catalog of the Hive metastore will be accessed.

If you dont specify a catalog-name, both iceberg catalogs in your example will use the default catalog "hive". When creating a schema or table in one iceberg catalog, it will also be captured by the other one as well.

Hence I don't see a reason why option 1 in your deployment architectures wouldn't work, in fact it is the setup in a proof of concept I have done. When setting up the catalog entries in the hms backing database, you specify the s3 path/bucket in which schemas will be materialized. Make sure to use hms >= 3.1 since in 3.0 there is bug in a unique constraint that prevents same names of schemas in different catalogs.

As a reference, #21502 adds the needed capabilities to trino.

Hope this helps!

[szisiu]

Sounds good. I need to investigate it in more detail. Thanks for the feedback.

[szisiu]

During investigation, I came up with the deployment architecures depicted below.

Assume we have 2 or more products (or any other required unit of separation).

Depending if a total data separation is mandatory and if data is accessible in a sql query ran against trino, we have:

With some modifications, you might got it useful - so sharing.

At the moment for various of reasong option 3 & 4 is the way to go for me.

It'd be nice, if trino w/ iceberg could support option no 1 (single HMS with multiple iceberg catalogs), maybe with some additional restrictions (using different buckets or different schema/table names).