Skip to content

Conversation

@ilia-db
Copy link
Contributor

@ilia-db ilia-db commented Oct 27, 2025

Changes

Based on #3775

Make a simple connection with "echo 'Hello'" and check that databricks ssh connect output has it.

SSH tunnel requires linux releases to be present, so it can upload them to the workspace for the ssh server to use. Added the build step directly to the acceptance_test, similar to how we build the CLI itself there. Initially I've modified our runner to install goreleaser and running make snapshot-release, but it takes over 10m on linux machines. Current build step only adds ~1m to the test time.

@ilia-db ilia-db force-pushed the ssh-tunnel-integ-tests branch from d307bfa to a988e53 Compare October 27, 2025 12:42
@ilia-db ilia-db force-pushed the ssh-tunnel-integ-tests-followup branch from 24d5662 to eb3c747 Compare October 27, 2025 12:44
@ilia-db ilia-db temporarily deployed to test-trigger-is October 27, 2025 12:44 — with GitHub Actions Inactive
@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Oct 27, 2025

Run: 19231904029

Env 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip
💚​ aws linux 1 1 357 601
💚​ aws windows 1 1 358 600
💚​ aws-ucws linux 1 1 492 492
💚​ aws-ucws windows 1 1 493 491
💚​ azure linux 1 1 357 600
💚​ azure windows 1 1 358 599
💚​ azure-ucws linux 1 1 488 491
💚​ azure-ucws windows 1 1 489 490
🔄​ gcp linux 4 1 1 352 602
🔄​ gcp windows 6 1 1 351 601
12 failing tests:
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
TestAccept/bundle/resources/experiments/basic ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
TestAccept/bundle/resources/experiments/basic/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
TestAccept/bundle/resources/pipelines/auto-approve ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
TestAccept/bundle/resources/pipelines/auto-approve/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
TestAccept/bundle/run/app-with-job 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
TestAccept/bundle/templates/default-python/combinations/classic ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=yes/PY=yes ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
TestAccept/bundle/templates/default-python/integration_classic ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct/UV_PYTHON=3.13 ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.11 ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.13 ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f

}

func CreateReleaseArtifact(t *testing.T, cwd, releasesDir, osName, arch string) {
tempBuildDir := filepath.Join(releasesDir, "tmp_"+arch)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this directory called tmp_ ? Is it cleaned up?

This is similar to BuildCLI() and there we build into the same directory, it's faster on second run when file is already present and nothing changed.

Can we re-use BuildCLI() function here? This will reuse build cache. When run on linux it will also re-use build artifact (even if you call go build for second time, it'll be faster).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It make sense to re-use BuildCLI, added that in the new revision

zipName := fmt.Sprintf("databricks_cli_%s_%s.zip", osName, arch)
zipPath := filepath.Join(releasesDir, zipName)

zipFile, err := os.Create(zipPath)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we skip zipping if its mtime > mtime of the binary?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on my testing go build modifies the binaries all the time, even if it skips the actual compilation phase. I'm re-creating the archives all the time because of it, but because the builds themselves are now cached, it's still pretty fast on subsequent runs

But this time also store private client key in the secrets scope.
@ilia-db ilia-db force-pushed the ssh-tunnel-integ-tests-followup branch from eb3c747 to f813e3f Compare October 31, 2025 10:31
@ilia-db ilia-db temporarily deployed to test-trigger-is October 31, 2025 10:31 — with GitHub Actions Inactive
@ilia-db ilia-db temporarily deployed to test-trigger-is November 6, 2025 12:30 — with GitHub Actions Inactive
@ilia-db ilia-db force-pushed the ssh-tunnel-integ-tests-followup branch from ad6d8e6 to 20e1607 Compare November 6, 2025 12:50
@ilia-db ilia-db temporarily deployed to test-trigger-is November 6, 2025 12:50 — with GitHub Actions Inactive
@ilia-db ilia-db requested a review from denik November 6, 2025 16:24
@ilia-db ilia-db force-pushed the ssh-tunnel-integ-tests-followup branch from 20e1607 to e7136fc Compare November 10, 2025 11:36
Base automatically changed from ssh-tunnel-integ-tests to main November 10, 2025 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants