Skip to content

direct: type checks for path validity#3504

Merged
denik merged 10 commits intomainfrom
denik/structaccess-type
Aug 28, 2025
Merged

direct: type checks for path validity#3504
denik merged 10 commits intomainfrom
denik/structaccess-type

Conversation

@denik
Copy link
Contributor

@denik denik commented Aug 27, 2025

Changes

  • New function structaccess.Validate for type checking the path against the type.
  • Make use of it during planning to show better error messages (schema mismatch vs field not provided).

Why

I'm going to add lookups in remote state. Remote state is optional / needs a request to backend. Type checking the path allows checking if reference is valid for remote state without actually having the state.

Tests

New acceptance test to show value-based non-existent field.

More non-regression test that cover different types of access and show inconsistencies between terraform and direct.

@denik denik temporarily deployed to test-trigger-is August 27, 2025 18:32 — with GitHub Actions Inactive
@denik denik changed the title direct: static checks for path validity direct: type checks for path validity Aug 27, 2025
@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Aug 27, 2025

Run: 17296614378

Env ✅​pass 🔄​flaky 🙈​skip
✅​ aws linux 308 503
✅​ aws windows 309 502
🔄​ aws-ucws linux 414 10 399
🔄​ aws-ucws windows 415 10 398
🔄​ azure linux 306 2 502
🔄​ azure windows 305 4 501
✅​ azure-ucws linux 424 398
✅​ azure-ucws windows 425 397
✅​ gcp linux 307 504
✅​ gcp windows 308 503
16 failing tests:
Test Name aws-ucws linux aws-ucws windows azure linux azure windows
TestAccept/bundle/deploy/dashboard/detect-change 🔄​flaky 🔄​flaky ✅​pass ✅​pass
TestAccept/bundle/deploy/dashboard/generate_inplace 🔄​flaky 🔄​flaky ✅​pass ✅​pass
TestAccept/bundle/deploy/dashboard/nested-folders 🔄​flaky 🔄​flaky ✅​pass ✅​pass
TestAccept/bundle/deploy/dashboard/simple 🔄​flaky 🔄​flaky ✅​pass ✅​pass
TestAccept/bundle/deploy/dashboard/simple_outside_bundle_root 🔄​flaky 🔄​flaky ✅​pass ✅​pass
TestAccept/bundle/deploy/dashboard/simple_syncroot 🔄​flaky 🔄​flaky ✅​pass ✅​pass
TestAccept/bundle/deployment/bind/dashboard 🔄​flaky 🔄​flaky ✅​pass ✅​pass
TestAccept/bundle/deployment/bind/dashboard/recreation 🔄​flaky 🔄​flaky ✅​pass ✅​pass
TestAccept/bundle/templates/default-python/combinations/classic ✅​pass ✅​pass ✅​pass 🔄​flaky
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=direct-exp/DLT=yes/NBOOK=yes/PY=no ✅​pass ✅​pass 🔄​flaky ✅​pass
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=yes/NBOOK=no/PY=no ✅​pass ✅​pass ✅​pass 🔄​flaky
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_CLI_DEPLOYMENT=terraform/DLT=yes/NBOOK=yes/PY=yes ✅​pass ✅​pass 🔄​flaky ✅​pass
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=direct-exp/UV_PYTHON=3.12 ✅​pass ✅​pass ✅​pass 🔄​flaky
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_CLI_DEPLOYMENT=direct-exp/UV_PYTHON=3.13 ✅​pass ✅​pass ✅​pass 🔄​flaky
TestDashboardAssumptions_WorkspaceImport 🔄​flaky 🔄​flaky ✅​pass ✅​pass
TestFetchRepositoryInfoAPI_FromRepo 🔄​flaky 🔄​flaky ✅​pass ✅​pass

Base automatically changed from denik/structaccess-path to main August 28, 2025 07:59
@denik denik force-pushed the denik/structaccess-type branch from dfc22cd to 948cf8b Compare August 28, 2025 08:06
@denik denik temporarily deployed to test-trigger-is August 28, 2025 08:06 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is August 28, 2025 08:49 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is August 28, 2025 09:14 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is August 28, 2025 09:26 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is August 28, 2025 09:32 — with GitHub Actions Inactive
// Fields marked readonly/internal are considered inaccessible.
btag := structtag.BundleTag(sf.Tag.Get("bundle"))
if btag.Internal() || btag.ReadOnly() {
return fmt.Errorf("%s: field %q not found in %s", newPrefix, c.Key(), owner.String())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we be more specific here and say that we're trying to access internal field?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a good idea. However, this code is actually unused -- bundle tags filtered out earlier, so I remove this here: 7131d9c

The reason earlier code filters it out and not doing hard error is because we might still have success later with non-internal field.

// Fields marked readonly/internal are considered inaccessible.
btag := structtag.BundleTag(sf.Tag.Get("bundle"))
if btag.Internal() || btag.ReadOnly() {
return fmt.Errorf("%s: field %q not found in %s", newPrefix, c.Key(), owner.String())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, why can't we use ReadOnly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently it is only used for "id" which is handled separately.

@@ -0,0 +1,8 @@
resources:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like TF fails to deploy with all of ingestion_definition,ingestion_definition.0.objects must be specified but direct is not, why is that? and is it intended? TF error looks valid though

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a validation that TF does. In direct we don't do any validations (but important ones we should add to initialize phase). I added objects so that it passes TF validation -- 2274f77

@denik denik temporarily deployed to test-trigger-is August 28, 2025 11:27 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is August 28, 2025 12:59 — with GitHub Actions Inactive
@denik denik requested a review from andrewnester August 28, 2025 13:07
@denik denik added this pull request to the merge queue Aug 28, 2025
Merged via the queue into main with commit 125a368 Aug 28, 2025
13 checks passed
@denik denik deleted the denik/structaccess-type branch August 28, 2025 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants