Skip to content

Commit 0de0ed0

Browse files
HariGS-DBHariGS-DB
authored
Added documentation for PrincipalACl migration and delete-missing-principal cmd (#2552)
1 parent 1325a46 commit 0de0ed0

File tree

1 file changed

+51
-14
lines changed

1 file changed

+51
-14
lines changed

‎README.md‎

Lines changed: 51 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,7 @@ See [contributing instructions](CONTRIBUTING.md) to help improve this project.
9696
* [Access for AWS S3 Buckets](#access-for-aws-s3-buckets)
9797
* [Access for Azure Storage Accounts](#access-for-azure-storage-accounts)
9898
* [`create-missing-principals` command (AWS Only)](#create-missing-principals-command-aws-only)
99+
* [`delete-missing-principals` command (AWS Only)](#delete-missing-principals-command-aws-only)
99100
* [`create-uber-principal` command](#create-uber-principal-command)
100101
* [`migrate-credentials` command](#migrate-credentials-command)
101102
* [`validate-external-locations` command](#validate-external-locations-command)
@@ -1238,6 +1239,16 @@ Two optional parameter are available for this command:
12381239

12391240
[[back to top](#databricks-labs-ucx)]
12401241

1242+
## `delete-missing-principals` command (AWS Only)
1243+
```bash
1244+
databricks labs ucx delete-missing-principals --aws-profile <aws_profile>
1245+
```
1246+
This command helps to delete the IAM role created by UCX. It lists all the IAM Roles generated by the principal-prefix-access
1247+
command and allows user to select multiple roles to delete. It also checks if selected roles are mapped to any storage credentials
1248+
and asks for confirmation from user. Once confirmed, it deletes the role and its associated inline policy.
1249+
1250+
[[back to top](#databricks-labs-ucx)]
1251+
12411252
## `create-uber-principal` command
12421253

12431254
```text
@@ -1313,6 +1324,15 @@ Once the [`assessment` workflow](#assessment-workflow) finished successfully, an
13131324
run this command to have Unity Catalog external locations created. The candidate locations to be created are extracted from guess_external_locations
13141325
task in the assessment job. You can run [`validate-external-locations` command](#validate-external-locations-command) to check the candidate locations.
13151326

1327+
**Location ACLs:**
1328+
`migrate-locations` command applies any location ACL from existing cluster.
1329+
For Azure, it checks if there are any interactive cluster or SQL warehouse
1330+
which has service principals configured to access storage. It maps the storage account to the external location created and grants `CREATE_EXTERNAL_TABLE`,
1331+
`CREATE_EXTERNAL_VOLUME` and `READ_FILES` permission on the location to all the user who have access to the interactive cluster or SQL warehouse
1332+
For AWS, it checks any instance profiles mapped to the interactive cluster or SQL warehouse. It checks the mapping of instance profiles to the bucket. It then
1333+
maps the bucket to the external locations created and grants `CREATE_EXTERNAL_TABLE`, `CREATE_EXTERNAL_VOLUME` and `READ_FILES` permission on the location to all the user who have access to the interactive cluster
1334+
or SQL warehouse
1335+
13161336
Once you're done with this command, proceed to the [`create-table-mapping` command](#create-table-mapping-command).
13171337

13181338
[[back to top](#databricks-labs-ucx)]
@@ -1376,7 +1396,14 @@ databricks labs ucx create-catalogs-schemas
13761396
```
13771397
After [`create-table-mapping` command](#create-table-mapping-command) is executed, you can run this command to have the required UC catalogs and schemas created.
13781398
This command is supposed to be run before migrating tables to UC using [table migration process](#Table-Migration).
1379-
1399+
Catalog & Schema ACL:
1400+
`create-catalogs-schemas` command also applies any catalog and schema ACL from existing clusters.
1401+
For Azure it checks if there are any interactive cluster or sql warehouse which has service principals configured to access storage.
1402+
It maps the storage account to the tables which has external location on those storage account created and grants `USAGE` access to
1403+
the schema and catalog if at least one such table is migrated to it.
1404+
For AWS, it checks any instance profiles mapped to the interactive cluster or sql warehouse. It checks the mapping of instance profiles
1405+
to the bucket. It then maps the bucket to the tables which has external location on those bucket created and grants `USAGE` access to
1406+
the schema and catalog if at least one such table is migrated to it.
13801407
[[back to top](#databricks-labs-ucx)]
13811408

13821409
## `migrate-tables` command
@@ -1390,6 +1417,15 @@ Anytime after [`create-table-mapping` command](#create-table-mapping-command) is
13901417
This command kicks off the [table migration](#Table-Migration) process. It triggers the `migrate-tables` workflow,
13911418
and if there are HiveSerDe tables detected, prompt whether to trigger the `migrate-external-hiveserde-tables-in-place-experimental` workflow.
13921419

1420+
Table and View ACL:
1421+
`migrate-tables` command also applies any table and view ACL from existing clusters.
1422+
For Azure it checks if there are any interactive cluster or sql warehouse which has service principals configured to access storage.
1423+
It maps the storage account to the tables which has external location on those storage account created and grants either `SELECT` permission if
1424+
the service principal only has read access on the storage account and `ALL_PRIVILEGES` if the service principal has write access on the storage account
1425+
For AWS, it checks any instance profiles mapped to the interactive cluster or sql warehouse. It checks the mapping of instance profiles
1426+
to the bucket. It then maps the bucket to the tables which has external location on those bucket created and grants either `SELECT` permission if
1427+
the instance profile only has read access on the bucket and `ALL_PRIVILEGES` if the instance profile has write access on the bucket.
1428+
13931429
[[back to top](#databricks-labs-ucx)]
13941430

13951431
## `revert-migrated-tables` command
@@ -1814,20 +1850,21 @@ run the commands that need cloud administrator rights.
18141850

18151851
Admin privileges required for commands:
18161852

1817-
| **CLI command** | **Admin privileges** |
1818-
|----|----|
1819-
| [<u>install</u>](#install-ucx) | Workspace Admin |
1853+
| **CLI command** | **Admin privileges** |
1854+
|--------------------------------------------------------------------------------------------------|----|
1855+
| [<u>install</u>](#install-ucx) | Workspace Admin |
18201856
| [<u>account install</u>](#advanced-installing-ucx-on-all-workspaces-within-a-databricks-account) | Account Admin |
1821-
| [<u>create-account-groups</u>](#create-account-groups-command) | Account Admin |
1822-
| [<u>validate-groups-membership</u>](#validate-groups-membership-command) | Account Admin |
1823-
| [<u>create-uber-principal</u>](#create-uber-principal-command) | Cloud Admin |
1824-
| [<u>principal-prefix-access</u>](#principal-prefix-access-command) | Cloud Admin |
1825-
| [<u>create-missing-principals</u>](#create-missing-principals-command-aws-only) | Cloud Admin |
1826-
| [<u>migrate-credentials</u>](#migrate-credentials-command) | Cloud Admin, Account Admin / Metastore Admin / CREATE STORAGE CREDENTIAL privilege |
1827-
| [<u>migrate-location</u>](#migrate-locations-command) | Metastore Admin / CREATE EXTERNAL LOCATION privilege |
1828-
| [<u>create-catalogs-schemas</u>](#create-catalogs-schemas-command) | Metastore Admin / CREATE CATALOG privilege |
1829-
| [<u>sync-workspace-info</u>](#sync-workspace-info-command) | Account Admin |
1830-
| [<u>manual-workspace-info</u>](#manual-workspace-info-command) | Workspace Admin |
1857+
| [<u>create-account-groups</u>](#create-account-groups-command) | Account Admin |
1858+
| [<u>validate-groups-membership</u>](#validate-groups-membership-command) | Account Admin |
1859+
| [<u>create-uber-principal</u>](#create-uber-principal-command) | Cloud Admin |
1860+
| [<u>principal-prefix-access</u>](#principal-prefix-access-command) | Cloud Admin |
1861+
| [<u>create-missing-principals</u>](#create-missing-principals-command-aws-only) | Cloud Admin |
1862+
| [<u>delete-missing-principals</u>](#delete-missing-principals-command-aws-only) | Cloud Admin |
1863+
| [<u>migrate-credentials</u>](#migrate-credentials-command) | Cloud Admin, Account Admin / Metastore Admin / CREATE STORAGE CREDENTIAL privilege |
1864+
| [<u>migrate-location</u>](#migrate-locations-command) | Metastore Admin / CREATE EXTERNAL LOCATION privilege |
1865+
| [<u>create-catalogs-schemas</u>](#create-catalogs-schemas-command) | Metastore Admin / CREATE CATALOG privilege |
1866+
| [<u>sync-workspace-info</u>](#sync-workspace-info-command) | Account Admin |
1867+
| [<u>manual-workspace-info</u>](#manual-workspace-info-command) | Workspace Admin |
18311868

18321869
[[back to top](#databricks-labs-ucx)]
18331870

0 commit comments

Comments
 (0)