Exporter: Support for some Unity Catalog resources (#3242)

* Exporter: initial support for `databricks_catalog`, `databricks_schema` and `databricks_volume`

Initial support for exporting of `databricks_catalog`, `databricks_schema` and
`databricks_volume`.  Code currently emits not supported resources, such as,
`databricks_grants`, `databricks_connection`, etc. - support for them will be in the
subsequent PRs.

Also, includes a bit of refactoring of the repeatable code for emitting in the incremental
mode.

Tests will be added separately when we get support for more resources, such as grants, etc.

* Using auxiliary functions in more places

* Added support for `databricks_grants`

* Added support for `databricks_connection`

* Add support for `databricks_share`

* Add support for `databricks_recipient`

* Fix tests

* Add `databricks_registered_model`

* A bit of code refactoring & adding dependencies

* `azure_service_principal.client_secret` is a variable...

* Fix crash when handling variables for resource without `Name` function

* Added support for `databricks_metastore` and `databricks_metastore_assignment`

* Add Ignore function for `databricks_external_location` to ignore automatically created locations

* Don't emit default external location during listing

* Improve reference matching

* Improve reference matching for shares & grants

* Add listing to UC connections, add matching by name to UC objects

* Update docs & fix test

* Refactoring, code coverage improvements

* More improvements for code coverage

* More code coverage

* more coverage

* Support for `databricks_catalog_workspace_binding` (implemented by George & Uma)

Author: alexott

docs/guides/experimental-exporter.md

@@ -79,12 +79,20 @@ Services are just logical groups of resources used for filtering and organizatio
 * `sql-endpoints` - **listing** [databricks_sql_endpoint](../resources/sql_endpoint.md) along with [databricks_sql_global_config](../resources/sql_global_config.md).
 * `sql-queries` - **listing** [databricks_sql_query](../resources/sql_query.md).
 * `storage` - only [databricks_dbfs_file](../resources/dbfs_file.md) referenced in other resources (libraries, init scripts, ...) will be downloaded locally and properly arranged into terraform state.
-* `uc-artifact-allowlist` - exports [databricks_artifact_allowlist](../resources/artifact_allowlist.md) resources for Unity Catalog Allow Lists attached to the current metastore.
-* `uc-system-schemas` - exports [databricks_system_schema](../resources/system_schema.md) resources for the UC metastore of the current workspace.
-* `uc-storage-credentials` - exports [databricks_storage_credential](../resources/storage_credential) resources on workspace or account level.
-* `uc-external-locations` - exports [databricks_external_location[(../resources/external_location) resource.
+* `uc-artifact-allowlist` - **listing** exports [databricks_artifact_allowlist](../resources/artifact_allowlist.md) resources for Unity Catalog Allow Lists attached to the current metastore.
+* `uc-catalogs` - **listing** [databricks_catalog](../resources/catalog.md) and [databricks_catalog_workspace_binding](../resources/catalog_workspace_binding.md)
+* `uc-connections` - **listing** [databricks_connection](../resources/connection.md).  *Please note that because API doesn't return sensitive fields, such as, passwords, tokens, ..., the generated `options` block could be incomplete!*
+* `uc-grants` -  [databricks_grants](../resources/grants.md)
+* `uc-metastores` - **listing** [databricks_metastore](../resources/metastore.md) and [databricks_metastore_assignment](../resource/metastore_assignment.md) (only on account-level).  *Please note that when using workspace-level configuration, only metastores from the workspace's region are listed!*
+* `uc-models` - [databricks_registered_model](../resources/registered_model.md)
+* `uc-schemas` -  [databricks_schema](../resources/schema.md)
+* `uc-shares` - **listing** [databricks_share](../resources/share.md) and [databricks_recipient](../resources/recipient.md)
+* `uc-system-schemas` - **listing** exports [databricks_system_schema](../resources/system_schema.md) resources for the UC metastore of the current workspace.
+* `uc-storage-credentials` - **listing** exports [databricks_storage_credential](../resources/storage_credential) resources on workspace or account level.
+* `uc-external-locations` - **listing** exports [databricks_external_location[(../resources/external_location) resource.
+* `uc-volumes` -  [databricks_volume](../resources/volume.md)
 * `users` - [databricks_user](../resources/user.md) and [databricks_service_principal](../resources/service_principal.md) are written to their own file, simply because of their amount. If you use SCIM provisioning, migrating workspaces is the only use case for importing `users` service.
-* `workspace` - [databricks_workspace_conf](../resources/workspace_conf.md) and [databricks_global_init_script](../resources/global_init_script.md)
+* `workspace` - **listing** [databricks_workspace_conf](../resources/workspace_conf.md) and [databricks_global_init_script](../resources/global_init_script.md)
 
 ## Secrets
 
@@ -109,10 +117,14 @@ Exporter aims to generate HCL code for most of the resources within the Databric
 | --- | --- | --- |
 | [databricks_access_control_rule_set](../resources/access_control_rule_set.md) | Yes | No |
 | [databricks_artifact_allowlist](../resources/artifact_allowlist.md) | Yes | No |
+| [databricks_catalog](../resources/catalog.md) | Yes | Yes |
 | [databricks_cluster](../resources/cluster.md) | Yes | No |
 | [databricks_cluster_policy](../resources/cluster_policy.md) | Yes | No |
+| [databricks_connection](../resources/connection.md) | Yes | Yes |
 | [databricks_dbfs_file](../resources/dbfs_file.md) | Yes | No |
+| [databricks_external_location](../resources/external_location.md) | Yes | Yes |
 | [databricks_global_init_script](../resources/global_init_script.md) | Yes | Yes |
+| [databricks_grants](../resources/grants.md) | Yes | No |
 | [databricks_group](../resources/group.md) | Yes | No |
 | [databricks_group_instance_profile](../resources/group_instance_profile.md) | Yes | No |
 | [databricks_group_member](../resources/group_member.md) | Yes | No |
@@ -122,20 +134,26 @@ Exporter aims to generate HCL code for most of the resources within the Databric
 | [databricks_ip_access_list](../resources/ip_access_list.md) | Yes | Yes |
 | [databricks_job](../resources/job.md) | Yes | No |
 | [databricks_library](../resources/library.md) | Yes\* | No |
-| [databricks_mlflow_model](../resources/mlflow_model.md) | No | No |
+| [databricks_metastore](../resources/metastore.md) | Yes | Yes |
+| [databricks_metastore_assignment](../resources/metastore_assignment.md) | Yes | No |
 | [databricks_mlflow_experiment](../resources/mlflow_experiment.md) | No | No |
+| [databricks_mlflow_model](../resources/mlflow_model.md) | No | No |
 | [databricks_mlflow_webhook](../resources/mlflow_webhook.md) | Yes | Yes |
 | [databricks_model_serving](../resources/model_serving) | Yes | Yes |
 | [databricks_notebook](../resources/notebook.md) | Yes | Yes |
 | [databricks_obo_token](../resources/obo_token.md) | Not Applicable | No |
 | [databricks_permissions](../resources/permissions.md) | Yes | No |
 | [databricks_pipeline](../resources/pipeline.md) | Yes | Yes |
+| [databricks_recipient](../resources/recipient.md) | Yes | Yes |
+| [databricks_registered_model](../resources/registered.md) | Yes | Yes |
 | [databricks_repo](../resources/repo.md) | Yes | No |
+| [databricks_schema](../resources/schema.md) | Yes | Yes |
 | [databricks_secret](../resources/secret.md) | Yes | No |
 | [databricks_secret_acl](../resources/secret_acl.md) | Yes | No |
 | [databricks_secret_scope](../resources/secret_scope.md) | Yes | No |
 | [databricks_service_principal](../resources/service_principal.md) | Yes | No |
 | [databricks_service_principal_role](../resources/service_principal_role.md) | Yes | No |
+| [databricks_share](../resources/share.md) | Yes | Yes |
 | [databricks_sql_alert](../resources/sql_alert.md) | Yes | Yes |
 | [databricks_sql_dashboard](../resources/sql_dashboard.md) | Yes | Yes |
 | [databricks_sql_endpoint](../resources/sql_endpoint.md) | Yes | No |
@@ -144,11 +162,13 @@ Exporter aims to generate HCL code for most of the resources within the Databric
 | [databricks_sql_query](../resources/sql_query.md) | Yes | Yes |
 | [databricks_sql_visualization](../resources/sql_visualization.md) | Yes | Yes |
 | [databricks_sql_widget](../resources/sql_widget.md) | Yes | Yes |
+| [databricks_storage_credential](../resources/storage_credential.md) | Yes | Yes |
 | [databricks_system_schema](../resources/system_schema.md) | Yes | No |
 | [databricks_token](../resources/token.md) | Not Applicable | No |
 | [databricks_user](../resources/user.md) | Yes | No |
 | [databricks_user_instance_profile](../resources/user_instance_profile.md) | No (Deprecated) | No |
 | [databricks_user_role](../resources/user_role.md) | Yes | No |
+| [databricks_volume](../resources/volume.md) | Yes | Yes |
 | [databricks_workspace_conf](../resources/workspace_conf.md) | Yes (partial) | No |
 | [databricks_workspace_file](../resources/workspace_file.md) | Yes | Yes |