Skip to content

Commit 6075fec

Browse files
authored
Add migrate-locations command (#1016)
## Changes Add cli command `migrate_locations` to create UC external location. ### Linked issues <!-- DOC: Link issue with a keyword: close, closes, closed, fix, fixes, fixed, resolve, resolves, resolved. See https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword --> #100 #1006 ### Functionality - [ ] added relevant user documentation - [ ] added new CLI command - [ ] modified existing command: `databricks labs ucx ...` - [ ] added a new workflow - [ ] modified existing workflow: `...` - [ ] added a new table - [ ] modified existing table: `...` ### Tests <!-- How is this tested? Please see the checklist below and also describe any other relevant tests --> - [ ] manually tested - [ ] added unit tests - [ ] added integration tests - [ ] verified on staging environment (screenshot attached)
1 parent 13be222 commit 6075fec

File tree

4 files changed

+60
-1
lines changed

4 files changed

+60
-1
lines changed

README.md

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -476,12 +476,25 @@ databricks labs ucx validate-external-locations
476476
```
477477

478478
Once the [`assessment` workflow](#assessment-workflow) finished successfully, [storage credentials](#migrate-credentials-command) are configured,
479-
run this command to ensure the relevant Unity Catalog external locations are created if they are missing.
479+
run this command to validate and report the missing Unity Catalog external locations to be created.
480480

481481
This command validates and provides mapping to external tables to external locations, also as Terraform configurations.
482482

483483
[[back to top](#databricks-labs-ucx)]
484484

485+
486+
## `migrate-locations` command
487+
488+
```text
489+
databricks labs ucx migrate-locations
490+
```
491+
492+
Once the [`assessment` workflow](#assessment-workflow) finished successfully, and [storage credentials](#migrate-credentials-command) are configured,
493+
run this command to have Unity Catalog external locations created. The candidate locations to be created are extracted from guess_external_locations
494+
task in the assessment job. You can run `validate_external_locations` command to check the candidate locations.
495+
496+
[[back to top](#databricks-labs-ucx)]
497+
485498
## `create-table-mapping` command
486499

487500
```text

labs.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -133,3 +133,6 @@ commands:
133133
flags:
134134
- name: workspace_ids
135135
description: List of workspace IDs to create account groups from.
136+
137+
- name: migrate_locations
138+
description: Create UC external locations based on the output of guess_external_locations assessment task.

src/databricks/labs/ucx/cli.py

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
from databricks.labs.ucx.assessment.aws import AWSResourcePermissions
1515
from databricks.labs.ucx.azure.access import AzureResourcePermissions
1616
from databricks.labs.ucx.azure.credentials import ServicePrincipalMigration
17+
from databricks.labs.ucx.azure.locations import ExternalLocationsMigration
1718
from databricks.labs.ucx.config import WorkspaceConfig
1819
from databricks.labs.ucx.framework.crawlers import StatementExecutionBackend
1920
from databricks.labs.ucx.hive_metastore import ExternalLocations, TablesCrawler
@@ -344,5 +345,22 @@ def create_uber_principal(w: WorkspaceClient, subscription_id: str):
344345
return
345346

346347

348+
@ucx.command
349+
def migrate_locations(w: WorkspaceClient):
350+
"""This command creates UC external locations. The candidate locations to be created are extracted from guess_external_locations
351+
task in the assessment job. You can run validate_external_locations command to check the candidate locations. Please make sure
352+
the credentials haven migrated before running this command. The command will only create the locations that have corresponded UC Storage Credentials.
353+
"""
354+
if w.config.is_azure:
355+
logger.info("Running migrate_locations for Azure")
356+
installation = Installation.current(w, 'ucx')
357+
service_principal_migration = ExternalLocationsMigration.for_cli(w, installation)
358+
service_principal_migration.run()
359+
if w.config.is_aws:
360+
logger.error("migrate_locations is not yet supported in AWS")
361+
if w.config.is_gcp:
362+
logger.error("migrate_locations is not yet supported in GCP")
363+
364+
347365
if __name__ == "__main__":
348366
ucx()

tests/unit/test_cli.py

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
installations,
1919
manual_workspace_info,
2020
migrate_credentials,
21+
migrate_locations,
2122
move,
2223
open_remote_config,
2324
principal_prefix_access,
@@ -357,3 +358,27 @@ def test_create_master_principal(ws):
357358
with patch("databricks.labs.blueprint.tui.Prompts.question", return_value=True):
358359
with pytest.raises(ValueError):
359360
create_uber_principal(ws, subscription_id="12")
361+
362+
363+
def test_migrate_locations_azure(ws):
364+
ws.config.is_azure = True
365+
ws.config.is_aws = False
366+
ws.config.is_gcp = False
367+
migrate_locations(ws)
368+
ws.external_locations.list.assert_called()
369+
370+
371+
def test_migrate_locations_aws(ws, caplog):
372+
ws.config.is_azure = False
373+
ws.config.is_aws = True
374+
ws.config.is_gcp = False
375+
migrate_locations(ws)
376+
assert "migrate_locations is not yet supported in AWS" in caplog.messages
377+
378+
379+
def test_migrate_locations_gcp(ws, caplog):
380+
ws.config.is_azure = False
381+
ws.config.is_aws = False
382+
ws.config.is_gcp = True
383+
migrate_locations(ws)
384+
assert "migrate_locations is not yet supported in GCP" in caplog.messages

0 commit comments

Comments
 (0)