Added databricks_service_principal and databricks_service_principals data resources (#1370)

* return `application_id` from `databricks_current_user` if that user is a Service Principal
* add `databricks_service_principal` data source based on `application_id`
* add `databricks_service_principals` data source based on `display_name`

Co-authored-by: Ron DeFreitas <[email protected]>

Author: nkvuong

docs/data-sources/current_user.md

@@ -56,6 +56,7 @@ output "job_url" {
 Data source exposes the following attributes:
 
 * `id` -  The id of the calling user.
+* `application_id` - Application ID of the [service principal](../resources/service_principal.md) if the currently logged-in user is a service principal, e.g. `11111111-2222-3333-4444-555666777888`
 * `external_id` - ID of the user in an external identity provider.
 * `user_name` - Name of the [user](../resources/user.md), e.g. `[email protected]`.
 * `home` - Home folder of the [user](../resources/user.md), e.g. `/Users/[email protected]`.

docs/data-sources/service_principal.md

@@ -0,0 +1,58 @@
+---
+subcategory: "Security"
+---
+
+# databricks_service_principal Data Source
+
+-> **Note** If you have a fully automated setup with workspaces created by [databricks_mws_workspaces](../resources/mws_workspaces.md) or [azurerm_databricks_workspace](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/databricks_workspace), please make sure to add [depends_on attribute](../index.md#data-resources-and-authentication-is-not-configured-errors) in order to prevent _authentication is not configured for provider_ errors.
+
+Retrieves information about [databricks_service_principal](../resources/service_principal.md).
+
+## Example Usage
+
+Adding service principal `11111111-2222-3333-4444-555666777888` to administrative group
+
+```hcl
+data "databricks_group" "admins" {
+  display_name = "admins"
+}
+
+data "databricks_service_principal" "spn" {
+  application_id = "11111111-2222-3333-4444-555666777888"
+}
+
+resource "databricks_group_member" "my_member_a" {
+  group_id  = data.databricks_group.admins.id
+  member_id = data.databricks_service_principal.spn.id
+}
+```
+
+## Argument Reference
+
+Data source allows you to pick service principals by the following attributes
+
+- `application_id` - (Required) ID of the service principal. The service principal must exist before this resource can be retrieved.
+
+## Attribute Reference
+
+Data source exposes the following attributes:
+
+- `sp_id` - The id of the service principal.
+- `external_id` - ID of the service principal in an external identity provider.
+- `display_name` - Display name of the [service principal](../resources/service_principal.md), e.g. `Foo SPN`.
+- `home` - Home folder of the [service principal](../resources/service_principal.md), e.g. `/Users/11111111-2222-3333-4444-555666777888`.
+- `repos` - Repos location of the [service principal](../resources/service_principal.md), e.g. `/Repos/11111111-2222-3333-4444-555666777888`.
+- `active` - Whether service principal is active or not.
+
+## Related Resources
+
+The following resources are used in the same context:
+
+* [End to end workspace management](../guides/passthrough-cluster-per-user.md) guide
+* [databricks_current_user](current_user.md) data to retrieve information about [databricks_user](../resources/user.md) or [databricks_service_principal](../resources/service_principal.md), that is calling Databricks REST API.
+* [databricks_group](../resources/group.md) to manage [groups in Databricks Workspace](https://docs.databricks.com/administration-guide/users-groups/groups.html) or [Account Console](https://accounts.cloud.databricks.com/) (for AWS deployments).
+* [databricks_group](group.md) data to retrieve information about [databricks_group](../resources/group.md) members, entitlements and instance profiles.
+* [databricks_group_instance_profile](../resources/group_instance_profile.md) to attach [databricks_instance_profile](../resources/instance_profile.md) (AWS) to [databricks_group](../resources/group.md).
+* [databricks_group_member](../resources/group_member.md) to attach [users](../resources/user.md) and [groups](../resources/group.md) as group members.
+* [databricks_permissions](../resources/permissions.md) to manage [access control](https://docs.databricks.com/security/access-control/index.html) in Databricks workspace.
+* [databricks_service principal](../resources/service_principal.md) to manage [service principals](../resources/service_principal.md)

docs/data-sources/service_principals.md

@@ -0,0 +1,59 @@
+---
+subcategory: "Security"
+---
+
+# databricks_service_principals Data Source
+
+-> **Note** If you have a fully automated setup with workspaces created by [databricks_mws_workspaces](../resources/mws_workspaces.md) or [azurerm_databricks_workspace](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/databricks_workspace), please make sure to add [depends_on attribute](../index.md#data-resources-and-authentication-is-not-configured-errors) in order to prevent _authentication is not configured for provider_ errors.
+
+Retrieves `application_ids` of all [databricks_service_principal](../resources/service_principal.md) based on their `display_name`
+
+## Example Usage
+
+Adding all service principals of which display name contains `my-spn` to admin group
+
+```hcl
+data "databricks_group" "admins" {
+  display_name = "admins"
+}
+
+data "databricks_service_principals" "spns" {
+  display_name = "my-spn"
+}
+
+data "databricks_service_principal" "spn" {
+  for_each       = toset(data.databricks_service_principals.spns.application_ids)
+  application_id = each.value
+}
+
+resource "databricks_group_member" "my_member_spn" {
+  for_each  = toset(data.databricks_service_principals.spns.application_ids)
+  group_id  = data.databricks_group.admins.id
+  member_id = data.databricks_service_principal.spn[each.value].sp_id
+}
+```
+
+## Argument Reference
+
+Data source allows you to pick service principals by the following attributes
+
+- `display_name_contains` - (Optional) Only return [databricks_service_principal](databricks_service_principal.md) display name that match the given name string
+
+## Attribute Reference
+
+Data source exposes the following attributes:
+
+- `application_ids` - List of `application_ids` of service principals Individual service principal can be retrieved using [databricks_service_principal](databricks_service_principal.md) data source
+
+## Related Resources
+
+The following resources are used in the same context:
+
+* [End to end workspace management](../guides/passthrough-cluster-per-user.md) guide
+* [databricks_current_user](current_user.md) data to retrieve information about [databricks_user](../resources/user.md) or [databricks_service_principal](../resources/service_principal.md), that is calling Databricks REST API.
+* [databricks_group](../resources/group.md) to manage [groups in Databricks Workspace](https://docs.databricks.com/administration-guide/users-groups/groups.html) or [Account Console](https://accounts.cloud.databricks.com/) (for AWS deployments).
+* [databricks_group](group.md) data to retrieve information about [databricks_group](../resources/group.md) members, entitlements and instance profiles.
+* [databricks_group_instance_profile](../resources/group_instance_profile.md) to attach [databricks_instance_profile](../resources/instance_profile.md) (AWS) to [databricks_group](../resources/group.md).
+* [databricks_group_member](../resources/group_member.md) to attach [users](../resources/user.md) and [groups](../resources/group.md) as group members.
+* [databricks_permissions](../resources/permissions.md) to manage [access control](https://docs.databricks.com/security/access-control/index.html) in Databricks workspace.
+* [databricks_service principal](../resources/service_principal.md) to manage [service principals](../resources/service_principal.md)

provider/provider.go

@@ -50,6 +50,8 @@ func DatabricksProvider() *schema.Provider {
 			"databricks_notebook":                workspace.DataSourceNotebook(),
 			"databricks_notebook_paths":          workspace.DataSourceNotebookPaths(),
 			"databricks_schemas":                 catalog.DataSourceSchemas(),
+			"databricks_service_principal":       scim.DataSourceServicePrincipal(),
+			"databricks_service_principals":      scim.DataSourceServicePrincipals(),
 			"databricks_spark_version":           clusters.DataSourceSparkVersion(),
 			"databricks_tables":                  catalog.DataSourceTables(),
 			"databricks_views":                   catalog.DataSourceViews(),

scim/data_service_principal.go

@@ -0,0 +1,41 @@
+package scim
+
+import (
+	"context"
+	"fmt"
+
+	"github.com/databrickslabs/terraform-provider-databricks/common"
+	"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
+)
+
+// DataSourceServicePrincipal returns information about the spn specified by the application_id
+func DataSourceServicePrincipal() *schema.Resource {
+	type spnData struct {
+		ApplicationID string `json:"application_id,omitempty" tf:"computed"`
+		DisplayName   string `json:"display_name,omitempty" tf:"computed"`
+		SpID          string `json:"sp_id,omitempty" tf:"computed"`
+		Home          string `json:"home,omitempty" tf:"computed"`
+		Repos         string `json:"repos,omitempty" tf:"computed"`
+		Active        bool   `json:"active,omitempty" tf:"computed"`
+		ExternalID    string `json:"external_id,omitempty" tf:"computed"`
+	}
+	return common.DataResource(spnData{}, func(ctx context.Context, e interface{}, c *common.DatabricksClient) error {
+		response := e.(*spnData)
+		spnAPI := NewServicePrincipalsAPI(ctx, c)
+		spList, err := spnAPI.filter(fmt.Sprintf("applicationId eq '%s'", response.ApplicationID))
+		if err != nil {
+			return err
+		}
+		if len(spList) == 0 {
+			return fmt.Errorf("cannot find SP with ID %s", response.ApplicationID)
+		}
+		sp := spList[0]
+		response.DisplayName = sp.DisplayName
+		response.Home = fmt.Sprintf("/Users/%s", sp.ApplicationID)
+		response.Repos = fmt.Sprintf("/Repos/%s", sp.ApplicationID)
+		response.ExternalID = sp.ExternalID
+		response.Active = sp.Active
+		response.SpID = sp.ID
+		return nil
+	})
+}

scim/data_service_principal_test.go

@@ -0,0 +1,87 @@
+package scim
+
+import (
+	"testing"
+
+	"github.com/databrickslabs/terraform-provider-databricks/qa"
+	"github.com/stretchr/testify/require"
+)
+
+func TestDataServicePrincipalReadByAppId(t *testing.T) {
+	qa.ResourceFixture{
+		Fixtures: []qa.HTTPFixture{
+			{
+				Method:   "GET",
+				Resource: "/api/2.0/preview/scim/v2/ServicePrincipals?filter=applicationId%20eq%20%27abc%27",
+				Response: UserList{
+					Resources: []User{
+						{
+							ID:            "abc",
+							DisplayName:   "Example Service Principal",
+							Active:        true,
+							ApplicationID: "abc",
+							Groups: []ComplexValue{
+								{
+									Display: "admins",
+									Value:   "4567",
+								},
+								{
+									Display: "ds",
+									Value:   "9877",
+								},
+							},
+						},
+					},
+				},
+			},
+		},
+		Resource:    DataSourceServicePrincipal(),
+		HCL:         `application_id = "abc"`,
+		Read:        true,
+		NonWritable: true,
+		ID:          "_",
+	}.ApplyAndExpectData(t, map[string]interface{}{
+		"sp_id":          "abc",
+		"application_id": "abc",
+		"display_name":   "Example Service Principal",
+		"active":         true,
+		"home":           "/Users/abc",
+		"repos":          "/Repos/abc",
+	})
+}
+
+func TestDataServicePrincipalReadNotFound(t *testing.T) {
+	_, err := qa.ResourceFixture{
+		Fixtures: []qa.HTTPFixture{
+			{
+				Method:   "GET",
+				Resource: "/api/2.0/preview/scim/v2/ServicePrincipals?filter=applicationId%20eq%20%27abc%27",
+				Response: UserList{},
+			},
+		},
+		Resource:    DataSourceServicePrincipal(),
+		HCL:         `application_id = "abc"`,
+		Read:        true,
+		NonWritable: true,
+		ID:          "_",
+	}.Apply(t)
+	require.Error(t, err, err)
+}
+
+func TestDataServicePrincipalReadError(t *testing.T) {
+	_, err := qa.ResourceFixture{
+		Fixtures: []qa.HTTPFixture{
+			{
+				Method:   "GET",
+				Resource: "/api/2.0/preview/scim/v2/ServicePrincipals?filter=applicationId%20eq%20%27abc%27",
+				Status:   500,
+			},
+		},
+		Resource:    DataSourceServicePrincipal(),
+		HCL:         `application_id = "abc"`,
+		Read:        true,
+		NonWritable: true,
+		ID:          "_",
+	}.Apply(t)
+	require.Error(t, err, err)
+}

scim/data_service_principals.go

@@ -0,0 +1,35 @@
+package scim
+
+import (
+	"context"
+	"fmt"
+	"sort"
+
+	"github.com/databrickslabs/terraform-provider-databricks/common"
+	"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
+)
+
+// DataSourceServicePrincipals searches for service principals based on display_name
+func DataSourceServicePrincipals() *schema.Resource {
+	type spnsData struct {
+		DisplayNameContains string   `json:"display_name_contains,omitempty" tf:"computed"`
+		ApplicationIDs      []string `json:"application_ids,omitempty" tf:"computed,slice_set"`
+	}
+	return common.DataResource(spnsData{}, func(ctx context.Context, e interface{}, c *common.DatabricksClient) error {
+		response := e.(*spnsData)
+		spnAPI := NewServicePrincipalsAPI(ctx, c)
+
+		spList, err := spnAPI.filter(fmt.Sprintf("displayName co '%s'", response.DisplayNameContains))
+		if err != nil {
+			return err
+		}
+		if len(spList) == 0 {
+			return fmt.Errorf("cannot find SPs with display name containing %s", response.DisplayNameContains)
+		}
+		for _, sp := range spList {
+			response.ApplicationIDs = append(response.ApplicationIDs, sp.ApplicationID)
+		}
+		sort.Strings(response.ApplicationIDs)
+		return nil
+	})
+}