v1.40.0

New Features and Improvements

• Added enable_predictive_optimization parameter for databricks_catalog and databricks_schema resources. (#3416).
• Added deployment field to the databricks_pipeline resource (#3428).
• Added lookup by name or region to databricks_metastore data source (#3419).
• Added zone_id to gcp_attributes of databricks_instance_pool (#3426).
• Added support for SQL Notebooks execution in databricks_job (#3420).
• Support adding and removing columns in databricks_sql_table (#3381).
• Fixed databricks_entitlement Update & Delete (#3434).
• Fixed databricks_mws_private_access_settings creation (#3439).
• Fixed databricks_user and databricks_service_principal integration tests (#3469).
• Fixed suppress_diff for webhook notifications at the task level in databricks_job (#3441).
• Only suppress diff for run_if when it is ALL_SUCCESS (#3454).
• Used Account client in the databricks_aws_crossaccount_policy data source (#3423).
• Added data sources databricks_external_locations and databricks_external_location (#3464).
• Added environment and environment_key to job settings (#3461).
• Added managed properties (#3465).

Documentation Changes

• Use correct provider alias in AWS E2 provisioning guide (#3425).
• Document pricing_tier of databricks_mws_workspaces resource (#3433).
• Updated documentation for Delta Sharing resources; refactoring relevant resources & data sources (#3466).
• Added description for new fields in databricks_lakehouse_monitor
• Update about using Databricks-managed service principals in all clouds (#3482)

Exporter

• Decrease a need for get-status calls for directories when listing workspace objects (#3470).
• Don't emit permissions and grants if object is going to be ignored (#3417).
• Don't generate references to ignored resources (#3418).
• Generate resource data from the object status obtained during the listing operation (#3460).
• Improve emitting of directories (#3462).
• Skip listing of jobs deployed with DABs (#3445).

Internal Changes

• Migrated databricks_cluster resource to Go SDK (#3376).
• Used ClusterSpec in databricks_cluster resource (#3459).
• Bump Go SDK to v0.38.0 (#3458).
• Bump golang.org/x/mod from 0.16.0 to 0.17.0 (#3435).
• Clean up integration tests and expose helper functions for identifying cloud (#3438).
• Fix integration tests for databricks_entitlements (#3467).
• Switch to use structs from Go SDK for databricks_recipient resource (#3400).
• Make libraries in resource_job slice instead of set (#3398).
• Fixed TestUcAccJobRunAsMutations
• Ensure tests are only run in the selected environment (#3476)
• Use fixture cluster for databricks_sql_permissions integration test (#3483)
• Improve integration tests for databricks_model_serving (#3479)