Skip to content

Commit f350081

Browse files
shreyas-goenkaclaude
shreyas-goenka
and
claude
authored
direct: Add dashboards support (#3725)
## Summary This PR adds support for **Databricks Lakeview Dashboards** in the direct deployment mode (`DATABRICKS_BUNDLE_ENGINE=direct-exp`). Dashboards can now be deployed, updated, and deleted directly via API calls without using Terraform. ## Key Features ### 1. **Full Dashboard Lifecycle Management** - **Create**: Creates draft dashboards and automatically publishes them with warehouse and credential settings - **Update**: Updates both dashboard content and publish settings (warehouse_id, embed_credentials) - **Delete**: Safely trashes dashboards with idempotent handling - **Refresh**: Fetches dashboard state from both draft and published endpoints in parallel ### 2. **Smart Field Change Detection** The implementation uses `FieldTriggers` to determine the correct action for each field change: #### Update Actions (in-place modifications): - `display_name` - Dashboard name changes - `warehouse_id` - SQL warehouse assignment changes - `embed_credentials` - Credential embedding toggle - `serialized_dashboard` - Dashboard content/layout changes #### Recreate Actions (delete + create): - `parent_path` - Moving dashboard to different folder requires recreation #### Skip Actions (ignored for diff): - Output-only fields: `dashboard_id`, `path`, `create_time`, `update_time`, `lifecycle_state` - `etag` - Skipped for local changes (user edits), but triggers updates for remote drift detection - `serialized_dashboard` - Skipped for remote drift (etag is sufficient) ### 3. **Advanced Capabilities** **Parallel API Calls**: Refresh operations fetch both draft and published dashboard states concurrently using goroutines for better performance. **Automatic Parent Directory Creation**: If the parent directory doesn't exist, it's automatically created before dashboard creation. **Idempotent Delete**: Handles already-trashed dashboards gracefully by checking the dashboard state on 403 errors. **embed_credentials Edge Case**: Unlike Terraform, this implementation properly detects changes to the `embed_credentials` field by fetching published dashboard state separately. **Serialized Dashboard Flexibility**: Supports both string and map/object formats for `serialized_dashboard` in bundle configuration. ### 4. **Comprehensive Test Coverage** Added 5 new acceptance test suites testing field change behavior: - `change-name` - Tests display_name updates - `change-warehouse-id` - Tests warehouse assignment updates - `change-embed-credentials` - Tests credential embedding toggle - `change-serialized-dashboard` - Tests dashboard content updates - `change-parent-path` - Tests parent path changes trigger recreate Each test verifies: - Correct API methods (POST/PATCH/DELETE) are called - Dashboard ID changes (or doesn't change) as expected - Field values are properly updated in remote state ## Testing Existing and new unit and integration tests. --------- Co-authored-by: Claude <[email protected]>
1 parent a7e9ebd commit f350081

File tree

90 files changed

+1710
-74
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

90 files changed

+1710
-74
lines changed

‎acceptance/.gitattributes‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
# For example, an additional \r character in windows changes the upload payload if you are
44
# uploading the file's content to a workspace.
55
*.txt text eol=lf
6+
*.lvdash.json text eol=lf
67

78
# The out.test.toml file is autogenerated based on the merged test.toml view.
89
out.test.toml linguist-generated=true

‎acceptance/bundle/generate/dashboard-inplace/output.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ Deployment complete!
1919
"lifecycle_state":"ACTIVE",
2020
"parent_path":"/Users/[USERNAME]/.bundle/dashboard update inplace/default/resources",
2121
"path":"/Users/[USERNAME]/.bundle/dashboard update inplace/default/resources/test dashboard.lvdash.json",
22-
"serialized_dashboard":"{\"a\":\"b\"}",
22+
"serialized_dashboard":"{\"a\":\"b\"}\n",
2323
"update_time":"[TIMESTAMP]",
2424
"warehouse_id":"my-warehouse-1234"
2525
}

‎acceptance/bundle/generate/dashboard-inplace/test.toml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,8 @@ EnvMatrix.DATABRICKS_BUNDLE_ENGINE = ["terraform"] # dashboards not supported y
33
[[Repls]]
44
Old = "[0-9a-f]{32}"
55
New = "[DASHBOARD_ID]"
6+
7+
[[Repls]]
8+
Old = '2\d\d\d-\d\d-\d\d(T| )\d\d:\d\d:\d\d(\.\d+(Z|\+\d\d:\d\d)?)?Z'
9+
New = "[TIMESTAMP]"
10+
Order = 9

‎acceptance/bundle/refschema/out.fields.txt‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -415,6 +415,30 @@ resources.clusters.*.permissions.permissions[*].group_name string ALL
415415
resources.clusters.*.permissions.permissions[*].permission_level iam.PermissionLevel ALL
416416
resources.clusters.*.permissions.permissions[*].service_principal_name string ALL
417417
resources.clusters.*.permissions.permissions[*].user_name string ALL
418+
resources.dashboards.*.create_time string ALL
419+
resources.dashboards.*.dashboard_id string ALL
420+
resources.dashboards.*.display_name string ALL
421+
resources.dashboards.*.embed_credentials bool ALL
422+
resources.dashboards.*.etag string ALL
423+
resources.dashboards.*.file_path string INPUT
424+
resources.dashboards.*.id string INPUT
425+
resources.dashboards.*.lifecycle resources.Lifecycle INPUT
426+
resources.dashboards.*.lifecycle.prevent_destroy bool INPUT
427+
resources.dashboards.*.lifecycle_state dashboards.LifecycleState ALL
428+
resources.dashboards.*.modified_status string INPUT
429+
resources.dashboards.*.parent_path string ALL
430+
resources.dashboards.*.path string ALL
431+
resources.dashboards.*.permissions []resources.DashboardPermission INPUT
432+
resources.dashboards.*.permissions[*] resources.DashboardPermission INPUT
433+
resources.dashboards.*.permissions[*].group_name string INPUT
434+
resources.dashboards.*.permissions[*].level resources.DashboardPermissionLevel INPUT
435+
resources.dashboards.*.permissions[*].service_principal_name string INPUT
436+
resources.dashboards.*.permissions[*].user_name string INPUT
437+
resources.dashboards.*.serialized_dashboard any ALL
438+
resources.dashboards.*.serialized_dashboard string ALL
439+
resources.dashboards.*.update_time string ALL
440+
resources.dashboards.*.url string INPUT
441+
resources.dashboards.*.warehouse_id string ALL
418442
resources.database_catalogs.*.create_database_if_not_exists bool ALL
419443
resources.database_catalogs.*.database_instance_name string ALL
420444
resources.database_catalogs.*.database_name string ALL

‎acceptance/bundle/resources/dashboards/change-embed-credentials/dash.lvdash.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{}

‎acceptance/bundle/resources/dashboards/change-embed-credentials/databricks.yml.tmpl‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
bundle:
2+
name: change-embed-credentials-$UNIQUE_NAME
3+
4+
resources:
5+
dashboards:
6+
my_dashboard:
7+
display_name: test dashboard
8+
file_path: ./dash.lvdash.json
9+
warehouse_id: $TEST_DEFAULT_WAREHOUSE_ID
10+
embed_credentials: $EMBED_CREDENTIALS

‎acceptance/bundle/resources/dashboards/change-embed-credentials/out.patch.requests.direct.txt‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{
2+
"method": "PATCH",
3+
"path": "/api/2.0/lakeview/dashboards/[DASHBOARD_ID]",
4+
"body": {
5+
"display_name": "test dashboard",
6+
"parent_path": "/Workspace/Users/[USERNAME]/.bundle/change-embed-credentials-[UNIQUE_NAME]/default/resources",
7+
"serialized_dashboard": "{}\n",
8+
"warehouse_id": "[TEST_DEFAULT_WAREHOUSE_ID]"
9+
}
10+
}

‎acceptance/bundle/resources/dashboards/change-embed-credentials/out.patch.requests.terraform.txt‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
{
2+
"method": "PATCH",
3+
"path": "/api/2.0/lakeview/dashboards/[DASHBOARD_ID]",
4+
"body": {
5+
"create_time": "[TIMESTAMP]",
6+
"dashboard_id": "[DASHBOARD_ID]",
7+
"display_name": "test dashboard",
8+
"etag": [ETAG],
9+
"lifecycle_state": "ACTIVE",
10+
"parent_path": "/Users/[USERNAME]/.bundle/change-embed-credentials-[UNIQUE_NAME]/default/resources",
11+
"path": "/Users/[USERNAME]/.bundle/change-embed-credentials-[UNIQUE_NAME]/default/resources/test dashboard.lvdash.json",
12+
"serialized_dashboard": "{}\n",
13+
"update_time": "[TIMESTAMP]",
14+
"warehouse_id": "[TEST_DEFAULT_WAREHOUSE_ID]"
15+
}
16+
}

‎acceptance/bundle/resources/dashboards/change-embed-credentials/out.post.requests.txt‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
{
2+
"method": "POST",
3+
"path": "/api/2.0/lakeview/dashboards",
4+
"body": {
5+
"display_name": "test dashboard",
6+
"parent_path": "/Workspace/Users/[USERNAME]/.bundle/change-embed-credentials-[UNIQUE_NAME]/default/resources",
7+
"serialized_dashboard": "{}\n",
8+
"warehouse_id": "[TEST_DEFAULT_WAREHOUSE_ID]"
9+
}
10+
}
11+
{
12+
"method": "POST",
13+
"path": "/api/2.0/workspace/mkdirs",
14+
"body": {
15+
"path": "/Workspace/Users/[USERNAME]/.bundle/change-embed-credentials-[UNIQUE_NAME]/default/resources"
16+
}
17+
}
18+
{
19+
"method": "POST",
20+
"path": "/api/2.0/lakeview/dashboards",
21+
"body": {
22+
"display_name": "test dashboard",
23+
"parent_path": "/Workspace/Users/[USERNAME]/.bundle/change-embed-credentials-[UNIQUE_NAME]/default/resources",
24+
"serialized_dashboard": "{}\n",
25+
"warehouse_id": "[TEST_DEFAULT_WAREHOUSE_ID]"
26+
}
27+
}
28+
{
29+
"method": "POST",
30+
"path": "/api/2.0/lakeview/dashboards/[DASHBOARD_ID]/published",
31+
"body": {
32+
"embed_credentials": false,
33+
"warehouse_id": "[TEST_DEFAULT_WAREHOUSE_ID]"
34+
}
35+
}
36+
{
37+
"method": "POST",
38+
"path": "/api/2.0/lakeview/dashboards/[DASHBOARD_ID]/published",
39+
"body": {
40+
"embed_credentials": true,
41+
"warehouse_id": "[TEST_DEFAULT_WAREHOUSE_ID]"
42+
}
43+
}

‎acceptance/bundle/resources/dashboards/change-embed-credentials/out.test.toml‎

Lines changed: 6 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)