Merge branch 'main' into gkiko10/JOBS-19305

Author: renaudhartert-db

.codegen/_openapi_sha

@@ -1 +1 @@
-cf9c61453990df0f9453670f2fe68e1b128647a2
+5285ce76f81314f342c1702d5c2ad4ef42488781

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,12 +1,28 @@
-## Changes
-<!-- Summary of your changes that are easy to understand -->
+## What changes are proposed in this pull request?
 
-## Tests
-<!-- 
-How is this tested? Please see the checklist below and also describe any other relevant tests 
--->
+Provide the readers and reviewers with the information they need to understand
+this PR in a comprehensive manner. 
 
-- [ ] `make test` run locally
-- [ ] `make fmt` applied
-- [ ] relevant integration tests applied
+Specifically, try to answer the two following questions:
 
+- **WHAT** changes are being made in the PR? This should be a summary of the 
+  major changes to allow the reader to quickly understand the PR without having
+  to look at the code. 
+- **WHY** are these changes needed? This should provide the context that the 
+  reader might be missing. For example, were there any decisions behind the 
+  change that are not reflected in the code itself? 
+
+The “why part” is the most important of the two as it usually cannot be 
+inferred from the code itself. A well-written PR description will help future
+developers (including your future self) to know how to interact and update your
+code.
+
+## How is this tested?
+
+Describe any tests you have done; especially if test tests are not part of
+the unit tests (e.g. local tests).
+
+**ALWAYS ANSWER THIS QUESTION:** Answer with "N/A" if tests are not applicable
+to your PR (e.g. if the PR only modifies comments). Do not be afraid of 
+answering "Not tested" if the PR has not been tested. Being clear about what 
+has been done and not done provides important context to the reviewers. 

.github/workflows/external-message.yml

@@ -11,81 +11,22 @@ on:
     branches:
       - main
 
-
 jobs:
   comment-on-pr:
     runs-on: ubuntu-latest
     permissions:
       pull-requests: write
 
     steps:
-      # NOTE: The following checks may not be accurate depending on Org or Repo settings. 
-      - name: Check user and potential secret access
-        id: check-secrets-access
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-        run: |
-          USER_LOGIN="${{ github.event.pull_request.user.login }}"
-          REPO_OWNER="${{ github.repository_owner }}"
-          REPO_NAME="${{ github.event.repository.name }}"
-          
-          echo "Pull request opened by: $USER_LOGIN"
-          
-          # Check if PR is from a fork
-          IS_FORK=$([[ "${{ github.event.pull_request.head.repo.full_name }}" != "${{ github.repository }}" ]] && echo "true" || echo "false")
-          
-          HAS_ACCESS="false"
-          
-          # Check user's permission level on the repository
-          USER_PERMISSION=$(gh api repos/$REPO_OWNER/$REPO_NAME/collaborators/$USER_LOGIN/permission --jq '.permission')
-          
-          if [[ "$USER_PERMISSION" == "admin" || "$USER_PERMISSION" == "write" ]]; then
-            HAS_ACCESS="true"
-          elif [[ "$USER_PERMISSION" == "read" ]]; then
-            # For read access, we need to check if the user has been explicitly granted secret access
-            # This information is not directly available via API, so we'll make an assumption
-            # that read access does not imply secret access
-            HAS_ACCESS="false"
-          fi
-          
-          # Check if repo owner is an organization
-          IS_ORG=$(gh api users/$REPO_OWNER --jq '.type == "Organization"')
-          
-          if [[ "$IS_ORG" == "true" && "$HAS_ACCESS" == "false" ]]; then
-            # Check if user is a member of any team with write or admin access to the repo
-            TEAMS_WITH_ACCESS=$(gh api repos/$REPO_OWNER/$REPO_NAME/teams --jq '.[] | select(.permission == "push" or .permission == "admin") | .slug')
-            for team in $TEAMS_WITH_ACCESS; do
-              IS_TEAM_MEMBER=$(gh api orgs/$REPO_OWNER/teams/$team/memberships/$USER_LOGIN --silent && echo "true" || echo "false")
-              if [[ "$IS_TEAM_MEMBER" == "true" ]]; then
-                HAS_ACCESS="true"
-                break
-              fi
-            done
-          fi
-          
-          # If it's a fork, set HAS_ACCESS to false regardless of other checks
-          if [[ "$IS_FORK" == "true" ]]; then
-            HAS_ACCESS="false"
-          fi
-          
-          echo "has_secrets_access=$HAS_ACCESS" >> $GITHUB_OUTPUT
-          if [[ "$HAS_ACCESS" == "true" ]]; then
-            echo "User $USER_LOGIN likely has access to secrets"
-          else
-            echo "User $USER_LOGIN likely does not have access to secrets"
-          fi
-
-
       - uses: actions/checkout@v4
 
       - name: Delete old comments
-        if: steps.check-secrets-access.outputs.has_secrets_access != 'true'
         env:
            GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
             # Delete previous comment if it exists
             previous_comment_ids=$(gh api "repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/comments" \
-              --jq '.[] | select(.body | startswith("<!-- INTEGRATION_TESTS -->")) | .id')
+              --jq '.[] | select(.body | startswith("<!-- INTEGRATION_TESTS_MANUAL -->")) | .id')
             echo "Previous comment IDs: $previous_comment_ids"
             # Iterate over each comment ID and delete the comment
             if [ ! -z "$previous_comment_ids" ]; then
@@ -96,14 +37,15 @@ jobs:
             fi
 
       - name: Comment on PR
-        if: steps.check-secrets-access.outputs.has_secrets_access != 'true'
         env:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           COMMIT_SHA: ${{ github.event.pull_request.head.sha }}
         run: |
           gh pr comment ${{ github.event.pull_request.number }} --body \
-          "<!-- INTEGRATION_TESTS -->
-          Run integration tests manually:
+          "<!-- INTEGRATION_TESTS_MANUAL -->
+          If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:
+          
+          Trigger:
           [go/deco-tests-run/sdk-py](https://go/deco-tests-run/sdk-py)
 
           Inputs:

.github/workflows/integration-tests.yml

@@ -12,20 +12,21 @@ jobs:
   check-token:
     name: Check secrets access
     runs-on: ubuntu-latest
+    environment: "test-trigger-is"
     outputs:
       has_token: ${{ steps.set-token-status.outputs.has_token }}
     steps:
-      - name: Check if GITHUB_TOKEN is set
+      - name: Check if DECO_WORKFLOW_TRIGGER_APP_ID is set
         id: set-token-status
         run: |
-          if [ -z "${{ secrets.GITHUB_TOKEN }}" ]; then
-            echo "GITHUB_TOKEN is empty. User has no access to tokens."
-            echo "::set-output name=has_token::false"
-          else
-            echo "GITHUB_TOKEN is set. User has no access to tokens."
-            echo "::set-output name=has_token::true"
-          fi
-
+            if [ -z "${{ secrets.DECO_WORKFLOW_TRIGGER_APP_ID }}" ]; then
+              echo "DECO_WORKFLOW_TRIGGER_APP_ID is empty. User has no access to secrets."
+              echo "::set-output name=has_token::false"
+            else
+              echo "DECO_WORKFLOW_TRIGGER_APP_ID is set. User has access to secrets."
+              echo "::set-output name=has_token::true"
+            fi
+    
   trigger-tests:
     name: Trigger Tests
     runs-on: ubuntu-latest

CHANGELOG.md

@@ -1,5 +1,62 @@
 # Version changelog
 
+## [Release] Release v0.37.0
+
+### Bug Fixes
+
+ * Correctly generate classes with nested body fields ([#808](https://github.com/databricks/databricks-sdk-py/pull/808)).
+
+
+### Internal Changes
+
+ * Add `cleanrooms` package ([#806](https://github.com/databricks/databricks-sdk-py/pull/806)).
+ * Add test instructions for external contributors ([#804](https://github.com/databricks/databricks-sdk-py/pull/804)).
+ * Always write message for manual test execution ([#811](https://github.com/databricks/databricks-sdk-py/pull/811)).
+ * Automatically trigger integration tests on PR ([#800](https://github.com/databricks/databricks-sdk-py/pull/800)).
+ * Better isolate ML serving auth unit tests ([#803](https://github.com/databricks/databricks-sdk-py/pull/803)).
+ * Move templates in the code generator ([#809](https://github.com/databricks/databricks-sdk-py/pull/809)).
+
+
+### API Changes:
+
+ * Added [w.aibi_dashboard_embedding_access_policy](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/settings/aibi_dashboard_embedding_access_policy.html) workspace-level service and [w.aibi_dashboard_embedding_approved_domains](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/settings/aibi_dashboard_embedding_approved_domains.html) workspace-level service.
+ * Added [w.credentials](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/credentials.html) workspace-level service.
+ * Added `app_deployment` field for `databricks.sdk.service.apps.CreateAppDeploymentRequest`.
+ * Added `app` field for `databricks.sdk.service.apps.CreateAppRequest`.
+ * Added `app` field for `databricks.sdk.service.apps.UpdateAppRequest`.
+ * Added `table` field for `databricks.sdk.service.catalog.CreateOnlineTableRequest`.
+ * Added `azure_aad` field for `databricks.sdk.service.catalog.GenerateTemporaryTableCredentialResponse`.
+ * Added `full_name` field for `databricks.sdk.service.catalog.StorageCredentialInfo`.
+ * Added `dashboard` field for `databricks.sdk.service.dashboards.CreateDashboardRequest`.
+ * Added `schedule` field for `databricks.sdk.service.dashboards.CreateScheduleRequest`.
+ * Added `subscription` field for `databricks.sdk.service.dashboards.CreateSubscriptionRequest`.
+ * Added `warehouse_id` field for `databricks.sdk.service.dashboards.Schedule`.
+ * Added `dashboard` field for `databricks.sdk.service.dashboards.UpdateDashboardRequest`.
+ * Added `schedule` field for `databricks.sdk.service.dashboards.UpdateScheduleRequest`.
+ * Added `page_token` field for `databricks.sdk.service.oauth2.ListServicePrincipalSecretsRequest`.
+ * Added `next_page_token` field for `databricks.sdk.service.oauth2.ListServicePrincipalSecretsResponse`.
+ * Added `connection_name` field for `databricks.sdk.service.pipelines.IngestionGatewayPipelineDefinition`.
+ * Added `is_no_public_ip_enabled` field for `databricks.sdk.service.provisioning.CreateWorkspaceRequest`.
+ * Added `external_customer_info` and `is_no_public_ip_enabled` fields for `databricks.sdk.service.provisioning.Workspace`.
+ * Added `last_used_day` field for `databricks.sdk.service.settings.TokenInfo`.
+ * Changed `create()` method for [w.apps](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/apps.html) workspace-level service with new required argument order.
+ * Changed `execute_message_query()` method for [w.genie](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/genie.html) workspace-level service . New request type is `databricks.sdk.service.dashboards.GenieExecuteMessageQueryRequest` dataclass.
+ * Changed `execute_message_query()` method for [w.genie](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/genie.html) workspace-level service to type `execute_message_query()` method for [w.genie](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/genie.html) workspace-level service.
+ * Changed `create()`, `create_schedule()`, `create_subscription()` and `update_schedule()` methods for [w.lakeview](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/lakeview.html) workspace-level service with new required argument order.
+ * Removed [w.clean_rooms](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/clean_rooms.html) workspace-level service.
+ * Removed `deployment_id`, `mode` and `source_code_path` fields for `databricks.sdk.service.apps.CreateAppDeploymentRequest`.
+ * Removed `description`, `name` and `resources` fields for `databricks.sdk.service.apps.CreateAppRequest`.
+ * Removed `description` and `resources` fields for `databricks.sdk.service.apps.UpdateAppRequest`.
+ * Removed `name` and `spec` fields for `databricks.sdk.service.catalog.CreateOnlineTableRequest`.
+ * Removed `display_name`, `parent_path`, `serialized_dashboard` and `warehouse_id` fields for `databricks.sdk.service.dashboards.CreateDashboardRequest`.
+ * Removed `cron_schedule`, `display_name` and `pause_status` fields for `databricks.sdk.service.dashboards.CreateScheduleRequest`.
+ * Removed `subscriber` field for `databricks.sdk.service.dashboards.CreateSubscriptionRequest`.
+ * Removed `display_name`, `etag`, `serialized_dashboard` and `warehouse_id` fields for `databricks.sdk.service.dashboards.UpdateDashboardRequest`.
+ * Removed `cron_schedule`, `display_name`, `etag` and `pause_status` fields for `databricks.sdk.service.dashboards.UpdateScheduleRequest`.
+ * Removed `prev_page_token` field for `databricks.sdk.service.jobs.Run`.
+
+OpenAPI SHA: 5285ce76f81314f342c1702d5c2ad4ef42488781, Date: 2024-11-04
+
 ## [Release] Release v0.36.0
 
 ### Breaking Changes