Skip to content

Commit 91e314f

Browse files
committed
Address comments, split section
1 parent 6b07085 commit 91e314f

File tree

1 file changed

+61
-16
lines changed

1 file changed

+61
-16
lines changed

articles/hdinsight-aks/trino/trino-connect-to-metastore.md

Lines changed: 61 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -35,26 +35,62 @@ The following example covers the addition of Hive catalog and metastore database
3535

3636
**There are few important sections you need to add to your cluster ARM template to configure the Hive catalog and Hive metastore database:**
3737

38-
- `secretsProfile` with Azure Key Vault and list of secrets to be used in Trino cluster, required to connect to external Hive metastore.
39-
- `serviceConfigsProfiles` with Trino catalogs. For more information, see [Add catalogs to existing cluster](trino-add-catalogs.md).
40-
- `hive.metastore=hdi` catalog property indicating to use external Hive metastores database with in-cluster hive metastore service, for each catalog where hive metastore is required.
41-
- `config.properties` with external Hive metastore database parameters. Alternatively same parameters specified in `trinoProfile.catalogOptions.hive` section.
38+
### Metastore configuration
39+
Configure external Hive metastore database which will be used by the cluster in `config.properties` file:
40+
```json
41+
{
42+
"fileName": "config.properties",
43+
"values": {
44+
"hive.metastore.hdi.metastoreDbConnectionURL": "jdbc:sqlserver://mysqlserver1.database.windows.net;database=myhmsdb1;encrypt=true;trustServerCertificate=true;create=false;loginTimeout=30",
45+
"hive.metastore.hdi.metastoreDbConnectionUserName": "trinoadmin",
46+
"hive.metastore.hdi.metastoreDbConnectionPasswordSecret": "hms-db-pwd",
47+
"hive.metastore.hdi.metastoreWarehouseDir": "abfs://[email protected]/hive/warehouse"
48+
}
49+
}
50+
```
51+
| Property| Description| Example|
52+
|---|---|---|
53+
|hive.metastore.hdi.metastoreDbConnectionURL|JDBC connection string to database.|jdbc:sqlserver://mysqlserver1.database.windows.net;database=myhmsdb1;encrypt=true;trustServerCertificate=true;create=false;loginTimeout=30|
54+
|hive.metastore.hdi.metastoreDbConnectionUserName|SQL user name to connect to database.|trinoadmin|
55+
|hive.metastore.hdi.metastoreDbConnectionPasswordSecret|Secret referenceName configured in secretsProfile with password.|hms-db-pwd|
56+
|hive.metastore.hdi.metastoreWarehouseDir|ABFS URI to location in storage where data is stored.|`abfs://[email protected]/hive/warehouse`|
4257

58+
### Metastore authentication
59+
Configure authentication to external Hive metastore database by specifiying Azure Key Vault secrets to use in ARM template.
60+
> [!NOTE]
61+
> `referenceName` should match value provided in `hive.metastore.hdi.metastoreDbConnectionPasswordSecret`
62+
```json
63+
"secretsProfile": {
64+
"keyVaultResourceId": "/subscriptions/{USER_SUBSCRIPTION_ID}/resourceGroups/{USER_RESOURCE_GROUP}/providers/Microsoft.KeyVault/vaults/{USER_KEYVAULT_NAME}",
65+
"secrets": [
66+
{
67+
"referenceName": "hms-db-pwd",
68+
"type": "secret",
69+
"keyVaultObjectName": "hms-db-pwd"
70+
} ]
71+
},
72+
```
4373
| Property| Description| Example|
4474
|---|---|---|
4575
|secretsProfile.keyVaultResourceId|Azure resource ID string to Azure Key Vault where secrets for Hive metastore are stored.|/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/trino-rg/providers/Microsoft.KeyVault/vaults/trinoakv|
4676
|secretsProfile.secrets[*].referenceName|Unique reference name of the secret to use later in clusterProfile.|Secret1_ref|
4777
|secretsProfile.secrets[*].type|Type of object in Azure Key Vault, only “secret” is supported.|secret|
4878
|secretsProfile.secrets[*].keyVaultObjectName|Name of secret object in Azure Key Vault containing actual secret value.|secret1|
49-
|trinoProfile.catalogOptions.hive|List of Hive or iceberg or delta catalogs with parameters of external Hive metastore database, require parameters for each. To use external metastore database, catalog must be present in this list.
50-
|trinoProfile.catalogOptions.hive[*].catalogName|Name of Trino catalog configured in `serviceConfigsProfiles`, which configured to use external Hive metastore database.|hive1|
51-
|config.properties/hive.metastore.hdi.metastoreDbConnectionURL (or trinoProfile.catalogOptions.hive[*].metastoreDbConnectionURL)|JDBC connection string to database.|jdbc:sqlserver://mysqlserver1.database.windows.net;database=myhmsdb1;encrypt=true;trustServerCertificate=true;create=false;loginTimeout=30|
52-
|hive.metastore.hdi.metastoreDbConnectionUserName (or trinoProfile.catalogOptions.hive[*].metastoreDbConnectionUserName)|SQL user name to connect to database.|trinoadmin|
53-
|hive.metastore.hdi.metastoreDbConnectionPasswordSecret (or trinoProfile.catalogOptions.hive[*].metastoreDbConnectionPasswordSecret)|Secret referenceName configured in secretsProfile with password.|hms-db-pwd|
54-
|hive.metastore.hdi.metastoreWarehouseDir (or trinoProfile.catalogOptions.hive[*].metastoreWarehouseDir)|ABFS URI to location in storage where data is stored.|`abfs://[email protected]/hive/warehouse`|
5579

56-
To configure external Hive metastore to an existing Trino cluster, add the required sections in your cluster ARM template by referring to the following example:
80+
### Catalog configuration
81+
In order for a Trino catalog to use external Hive metastore it should specify `hive.metastore=hdi` property. For more information, see [Add catalogs to existing cluster](trino-add-catalogs.md):
82+
```
83+
{
84+
"fileName": "hive1.properties",
85+
"values": {
86+
"connector.name": "hive",
87+
"hive.metastore": "hdi"
88+
}
89+
}
90+
```
5791

92+
### Complete example
93+
To configure external Hive metastore to an existing Trino cluster, add the required sections in your cluster ARM template by referring to the following example:
5894

5995
```json
6096
{
@@ -91,10 +127,10 @@ To configure external Hive metastore to an existing Trino cluster, add the requi
91127
{
92128
"fileName": "config.properties",
93129
"values": {
94-
"hive.metastore.hdi.metastoreDbConnectionURL": "jdbc:sqlserver://mysqlserver1.database.windows.net;database=myhmsdb1;encrypt=true;trustServerCertificate=true;create=false;loginTimeout=30",
95-
"hive.metastore.hdi.metastoreDbConnectionUserName": "trinoadmin",
96-
"hive.metastore.hdi.metastoreDbConnectionPasswordSecret": "hms-db-pwd",
97-
"hive.metastore.hdi.metastoreWarehouseDir": "abfs://[email protected]/hive/warehouse"
130+
"hive.metastore.hdi.metastoreDbConnectionURL": "jdbc:sqlserver://mysqlserver1.database.windows.net;database=myhmsdb1;encrypt=true;trustServerCertificate=true;create=false;loginTimeout=30",
131+
"hive.metastore.hdi.metastoreDbConnectionUserName": "trinoadmin",
132+
"hive.metastore.hdi.metastoreDbConnectionPasswordSecret": "hms-db-pwd",
133+
"hive.metastore.hdi.metastoreWarehouseDir": "abfs://[email protected]/hive/warehouse"
98134
}
99135
}
100136
]
@@ -139,9 +175,18 @@ select * from hive1.schema1.tpchorders limit 100;
139175
```
140176

141177
## Alternative configuration
142-
Alternatively external Hive metastore database parameters can be specified in trinoProfile together with `hive.metastore=hdi` catalog property:
178+
Alternatively external Hive metastore database parameters can be specified in `trinoProfile.catalogOptions.hive` together with `hive.metastore=hdi` catalog property:
143179

180+
| Property| Description| Example|
181+
|---|---|---|
182+
|trinoProfile.catalogOptions.hive|List of Hive or iceberg or delta catalogs with parameters of external Hive metastore database, require parameters for each. To use external metastore database, catalog must be present in this list.
183+
|trinoProfile.catalogOptions.hive[*].catalogName|Name of Trino catalog configured in `serviceConfigsProfiles`, which configured to use external Hive metastore database.|hive1|
184+
|trinoProfile.catalogOptions.hive[*].metastoreDbConnectionURL|JDBC connection string to database.|jdbc:sqlserver://mysqlserver1.database.windows.net;database=myhmsdb1;encrypt=true;trustServerCertificate=true;create=false;loginTimeout=30|
185+
|trinoProfile.catalogOptions.hive[*].metastoreDbConnectionUserName|SQL user name to connect to database.|trinoadmin|
186+
|trinoProfile.catalogOptions.hive[*].metastoreDbConnectionPasswordSecret|Secret referenceName configured in secretsProfile with password.|hms-db-pwd|
187+
|trinoProfile.catalogOptions.hive[*].metastoreWarehouseDir|ABFS URI to location in storage where data is stored.|`abfs://[email protected]/hive/warehouse`|
144188

189+
### Complete example
145190
```json
146191
{
147192
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",

0 commit comments

Comments
 (0)