[Documentation] Unified Provider: One provider to manage both workspace and account level resources

NEXT_CHANGELOG.md

@@ -15,6 +15,7 @@
 * Document `table_update` trigger in `databricks_job` resource ([#5107](https://github.com/databricks/terraform-provider-databricks/pull/5107))
 * Document new attributes in `databricks_app` resource and data sources ([#5108](https://github.com/databricks/terraform-provider-databricks/pull/5108))
 * Document `git_email` in `databricks_git_credential` resource ([#5099](https://github.com/databricks/terraform-provider-databricks/pull/5099))
+* Document Unified Terraform Provider ([#5122](https://github.com/databricks/terraform-provider-databricks/pull/5122))
 
 ### Exporter
 

docs/guides/unified-provider.md

@@ -0,0 +1,121 @@
+# Unified Terraform Provider
+[![Public Beta](https://img.shields.io/badge/Release_Stage-Public_Beta-orange)](https://docs.databricks.com/aws/en/release-notes/release-types)
+
+## Introduction
+
+Unified Terraform Provider allows you to manage workspace-level resources and data sources through an account-level provider. This significantly simplifies Terraform configurations and resource management, as only one provider is needed per account.
+
+**Note:** This feature is in [Public Beta](https://docs.databricks.com/aws/en/release-notes/release-types). If you experience any issues, please refer to the [Reporting Issues](#reporting-issues) section below.
+
+## Usage
+
+To manage a workspace-level resource through the account provider, specify the `provider_config` at the resource level with the `workspace_id` of the workspace the resource belongs to.
+
+Depending on the internal implementation of the resource or data source, `provider_config` can be either a block or an attribute. For details, please refer to the documentation of the specific resource or data source.
+
+### Block
+```hcl
+resource "workspace_level_resource" "this" {
+    provider_config {
+        workspace_id = "12345"
+    }
+    ...
+}
+```
+
+### Attribute
+```hcl
+resource "workspace_level_resource" "this" {
+    provider_config = {
+        workspace_id = "12345"
+    }
+    ...
+}
+```
+
+**Note:** This feature is being rolled out incrementally. Some resources do not yet support the unified provider. Please check the resource-specific documentation to see if the `provider_config` attribute or block is available.
+
+## Migrating to Unified Provider
+
+If you are currently managing both workspace-level and account-level resources and data sources, you likely have multiple provider configurations that you specify for each resource using aliases. For example:
+```hcl
+// Define an account provider with alias
+provider "databricks" {
+    alias         = "account"
+    host          = var.account_host
+    account_id    = var.account_id
+    client_id     = var.account_client_id
+    client_secret = var.account_client_secret
+}
+
+// Use the account provider for account-level resources
+resource "databricks_account_federation_policy" "this" {
+    provider = databricks.account
+    policy_id = "my-policy"
+    oidc_policy = {
+        issuer = "https://myidp.example.com"
+        subject_claim = "sub"
+    }
+}
+
+// Define a workspace provider with alias
+provider "databricks" {
+    alias           = "workspace"
+    host            = var.workspace_host
+    client_id       = var.workspace_client_id
+    client_secret   = var.workspace_client_secret
+}
+
+// Use the workspace provider for workspace-level resources
+resource "databricks_workspace_level_resource" "this" {
+    provider = databricks.workspace
+    name = "resource name"
+}
+```
+
+To migrate to the unified provider, you can remove the workspace-level provider and specify the `workspace_id` in the `provider_config` attribute or block instead. For example:
+
+```hcl
+// Define an account provider
+provider "databricks" {
+    host          = var.account_host
+    account_id    = var.account_id
+    client_id     = var.account_client_id
+    client_secret = var.account_client_secret
+}
+
+// Create an account-level resource
+resource "databricks_account_federation_policy" "this" {
+    policy_id = "my-policy"
+    oidc_policy = {
+        issuer = "https://myidp.example.com"
+        subject_claim = "sub"
+    }
+}
+
+// Create a workspace under the account
+resource "databricks_mws_workspaces" "this" {
+    account_id   = var.account_id
+    aws_region   = "us-east-1"
+    compute_mode = "SERVERLESS"
+}
+
+// Create a workspace-level resource using provider_config
+resource "databricks_workspace_level_resource" "this" {
+    provider_config {
+        workspace_id = databricks_mws_workspaces.this.workspace_id
+    }
+    name = "resource name"
+}
+```
+
+## Limitations
+
+There are some limitations to this feature that we plan to address in the near future:
+
+1. Databricks CLI and Azure CLI authentication methods are not currently supported
+2. Some resources do not yet support the unified provider. Please refer to the documentation for each resource or data source to check if they support the `provider_config` attribute or block.
+
+## Reporting Issues
+
+This feature is in Public Beta. If you encounter any issues, please report them on [GitHub Issues](https://github.com/databricks/terraform-provider-databricks/issues) with the `Unified Provider` label.

docs/index.md

@@ -173,7 +173,7 @@ There are currently a number of supported methods to [authenticate](https://docs
 !> **Warning** Please be aware that hard coding any credentials in plain text is not something that is recommended. We strongly recommend using a Terraform backend that supports encryption. Please use [environment variables](#environment-variables), `~/.databrickscfg` file, encrypted `.tfvars` files or secret store of your choice (Hashicorp [Vault](https://www.vaultproject.io/), AWS [Secrets Manager](https://aws.amazon.com/secrets-manager/), AWS [Param Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html), Azure [Key Vault](https://azure.microsoft.com/en-us/services/key-vault/))
 
 ### Authenticating with GitHub OpenID Connect (OIDC)
-The arguments `host` and `client_id` are used for the authentication which maps to the `github-oidc` authentication type. 
+The arguments `host` and `client_id` are used for the authentication which maps to the `github-oidc` authentication type.
 
 These can be declared in the provider block or set in the environment variables `DATABRICKS_HOST` and `DATABRICKS_CLIENT_ID` respectively. Example:
 
@@ -197,10 +197,11 @@ provider "databricks" {
   account_id = var.account_id
 }
 ```
+Note: Some workspace level resources can be managed through account provider. please see the [unified provider](./guides/unified-provider.md) for more details.
 
 ### Authenticating with Databricks CLI
 
-The provider can authenticate using the Databricks CLI. After logging in with the `databricks auth login` command to your account or workspace, you only need to specify the name of the profile in your provider configuration. Terraform will automatically read and reuse the cached OAuth token to interact with the Databricks REST API. See [the user-to-machine authentication guide](https://docs.databricks.com/aws/en/dev-tools/cli/authentication#oauth-user-to-machine-u2m-authentication) for more details. 
+The provider can authenticate using the Databricks CLI. After logging in with the `databricks auth login` command to your account or workspace, you only need to specify the name of the profile in your provider configuration. Terraform will automatically read and reuse the cached OAuth token to interact with the Databricks REST API. See [the user-to-machine authentication guide](https://docs.databricks.com/aws/en/dev-tools/cli/authentication#oauth-user-to-machine-u2m-authentication) for more details.
 
 You can specify a [CLI connection profile](https://docs.databricks.com/aws/en/dev-tools/cli/profiles) through `profile` parameter or `DATABRICKS_CONFIG_PROFILE` environment variable:
 
@@ -230,7 +231,9 @@ provider "databricks" {
 }
 ```
 
-To create resources at both the account and workspace levels, you can create two providers as shown below:
+To create resources at both the account and workspace levels, you can create two providers as shown below.
+
+Note: Some workspace level resources can be managed through account provider. please see the [unified provider](./guides/unified-provider.md) for more details.
 
 ``` hcl
 provider "databricks" {
@@ -275,9 +278,11 @@ provider "databricks" {
 
 ### Authenticating with Workload Identity Federation (WIF)
 
-Workload Identity Federation can be used to authenticate Databricks from automated workflows. This is done through the tokens issued by the automation environment. For more details on environment variables regarding the specific environments, please see: https://docs.databricks.com/aws/en/dev-tools/auth/oauth-federation-provider. 
+Workload Identity Federation can be used to authenticate Databricks from automated workflows. This is done through the tokens issued by the automation environment. For more details on environment variables regarding the specific environments, please see: https://docs.databricks.com/aws/en/dev-tools/auth/oauth-federation-provider.
+
+To create resources at both the account and workspace levels, you can create two providers as shown below.
 
-To create resources at both the account and workspace levels, you can create two providers as shown below:
+Note: Some workspace level resources can be managed through account provider. please see the [unified provider](./guides/unified-provider.md) for more details.
 
 Workspace level provider:
 ```hcl
@@ -300,7 +305,36 @@ provider "databricks" {
 }
 ```
 
-Note: `auth_type` for Github Actions would be "github-oidc". For more details, please see the document linked above. 
+Note: `auth_type` for Github Actions would be "github-oidc". For more details, please see the document linked above.
+
+### Authenticating with Unified Provider
+Unified Provider allows management of workspace level terraform resources through account level provider. You can specify the `provider_config` block or attribute depending on the resource with the `workspace_id` that the resource will belong to. For more details, please see the [documentation](./guides/unified-provider.md).
+
+
+Example:
+```hcl
+// Create an account level provider
+provider "databricks" {
+  host          = var.account_host
+  client_id     = var.client_id
+  client_secret = var.client_secret
+  account_id    = var.account_id
+}
+
+// Create a workspace under the account
+resource "databricks_mws_workspaces" "this" {
+  ...
+}
+
+// Create a workspace level resource under the workspace above
+resource "databricks_workspace_level_resource" "this" {
+  provider_config = {
+    workspace_id = databricks_mws_workspaces.this.workspace_id
+  }
+  ...
+}
+
+```
 
 
 ## Special configurations for Azure
@@ -351,7 +385,7 @@ resource "databricks_user" "my-user" {
 }
 ```
 
-Follow the [Configuring OpenID Connect in Azure](https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/configuring-openid-connect-in-azure). You can then use the Azure service principal to authenticate in databricks. 
+Follow the [Configuring OpenID Connect in Azure](https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/configuring-openid-connect-in-azure). You can then use the Azure service principal to authenticate in databricks.
 
 
 There are `ARM_*` environment variables provide a way to share authentication configuration using the `databricks` provider alongside the [`azurerm` provider](https://registry.terraform.io/providers/hashicorp/azurerm/latest).
@@ -462,7 +496,7 @@ To make Databricks Terraform Provider generally available, we've moved it from [
 You should have [`.terraform.lock.hcl`](https://github.com/databrickslabs/terraform-provider-databricks/blob/v0.6.2/scripts/versions-lock.hcl) file in your state directory that is checked into source control. terraform init will give you the following warning.
 
 ```text
-Warning: Additional provider information from registry 
+Warning: Additional provider information from registry
 
 The remote registry returned warnings for registry.terraform.io/databrickslabs/databricks:
 - For users on Terraform 0.13 or greater, this provider has moved to databricks/databricks. Please update your source in required_providers.