Create a registry package for products to self-register resources #16244

SirGitsalot · 2026-01-28T00:19:03Z

This is part of the Bazel work, which includes much more modularized builds. Right now the MM compiler can only generate a provider or integration tests that include every product, resource, and data source. The main reason for that is how these are cataloged in the provider: they are imported at compile time and statically registered. In the future, they will be discovered at runtime and dynamically registered.

This PR also switches MMv1 resource discovery to use the registry. It's still static compile-time registration, but with a step toward the aforementioned dynamic registration. Handwritten and DCL resources will come in subsequent PRs, and then the switch to fully dynamic.

The registry ended up in its own package due to circular dependencies. In the future it can probably be moved into the provider package, but not for now as static registration means that the provider package depends on product packages, and product packages will always depend on the provider package.

Probably the most important thing to check is the diff in the generated versions of google[-beta]/provider/provider_mmv1_resources.go. The assignments in generatedIAMDatasources and generatedResources should match 1:1, with the right hand side now coming from the registry.

…ources

modular-magician · 2026-01-28T00:36:50Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 963 files changed, 12777 insertions(+), 1105 deletions(-))
google-beta provider: Diff ( 1061 files changed, 14233 insertions(+), 1244 deletions(-))
terraform-google-conversion: Diff ( 102 files changed, 2856 insertions(+))

modular-magician · 2026-01-28T02:54:25Z

Tests analytics

Total tests: 5967
Passed tests: 5332
Skipped tests: 633
Affected tests: 2

Click here to see the affected service packages

All service packages are affected

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccAccessContextManager__service_perimeter_dry_run_egress_policy

TestAccAccessContextManager__service_perimeter_dry_run_ingress_policy

Get to know how VCR tests work

modular-magician · 2026-01-28T03:12:08Z

🟢 Tests passed during RECORDING mode:
TestAccAccessContextManager__access_level [Debug log]
TestAccAccessContextManager__access_level_condition [Debug log]
TestAccAccessContextManager__access_level_custom [Debug log]
TestAccAccessContextManager__access_level_full [Debug log]
TestAccAccessContextManager__access_levels [Debug log]
TestAccAccessContextManager__access_policy [Debug log]
TestAccAccessContextManager__access_policy_scoped [Debug log]
TestAccAccessContextManager__authorized_orgs_desc [Debug log]
TestAccAccessContextManager__service_perimeter [Debug log]
TestAccAccessContextManager__service_perimeter_dry_run_egress_policy [Debug log]
TestAccAccessContextManager__service_perimeter_dry_run_ingress_policy [Debug log]
TestAccAccessContextManager__service_perimeter_update [Debug log]
TestAccAccessContextManager__service_perimeters [Debug log]

🔴 Tests failed when rerunning REPLAYING mode:
TestAccAccessContextManager__service_perimeter_dry_run_egress_policy [Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.

🟢 All tests passed!

View the build log or the debug log for each test

melinath

Overall makes sense / looks good. A few questions & change requests below.

melinath · 2026-01-28T18:49:11Z

mmv1/third_party/terraform/registry/registry.go

+}
+
+type registeredProducts struct {
+	sync.RWMutex


This should be a pointer to avoid copying the mutex per https://pkg.go.dev/sync#Mutex: "A Mutex must not be copied after first use."

Suggested change

sync.RWMutex

*sync.RWMutex

I'm not sure whether we can achieve that like this... possibly this needs to be a real field on the struct & Lock/Unlock callers need to access the mutex directly.

I changed the instantiations to be pointers. They're package-private types and their instantiations are package-private, which also prevents copying.

mmv1/third_party/terraform/registry/registry.go

melinath · 2026-01-28T19:06:41Z

mmv1/third_party/terraform/registry/registry.go

+}
+
+// Schema is used to configure a resource or data source within the registry.
+type Schema struct {


Why does this use a single Schema struct w/ SchemaType instead of having different structs per registerable type? It feels like resource/datasource are handled sufficiently differently that it could be easier to use if they were kept separate. (I don't feel very strongly about this, just curious about the reasoning.)

I thought about it but ultimately decided that since this is just a small wrapper around the Terraform schema type, and that type handles both data sources and resources that this wrapper would too.

melinath · 2026-01-28T19:14:14Z

mmv1/templates/terraform/iam_policy.go.tmpl

    _ = schema.Noop
 )

+func init() {


it looks like this is ending up in TGC, but the other registration blocks aren't. This seems a little surprising. Might cause problems?

In theory it's harmless, but I've modified the template to avoid it on TGC.

…gnore_schema_changes(dataPolicies) is defined (#15006)

…playing (#16104)

…ility policy block. (#15916)

…6013) Co-authored-by: Mike Prieto <[email protected]> Co-authored-by: Shuya Ma <[email protected]> Co-authored-by: jgforsberg <[email protected]> Co-authored-by: shang-jason <[email protected]> Co-authored-by: Zhenhua Li <[email protected]> Co-authored-by: Taneli Leppä <[email protected]> Co-authored-by: Hank Freund <[email protected]> Co-authored-by: Ning Kang <[email protected]> Co-authored-by: Ning Kang <[email protected]> Co-authored-by: nshah-google <[email protected]> Co-authored-by: Riley Karson <[email protected]> Co-authored-by: diegomguzman-google <[email protected]> Co-authored-by: Nandini Agrawal <[email protected]> Co-authored-by: Max Portocarrero CI&T <[email protected]> Co-authored-by: Ricardo Godoy <[email protected]> Co-authored-by: sushilchaskar28 <[email protected]> Co-authored-by: Chi-Pin <[email protected]> Co-authored-by: Björn <[email protected]> Co-authored-by: Rahul Jaideoka <[email protected]> Co-authored-by: shivangd <[email protected]> Co-authored-by: shubhechha-google <[email protected]> Co-authored-by: Andras Kerekes <[email protected]> Co-authored-by: Luca Prete <[email protected]> Co-authored-by: Stephane Charite <[email protected]> Co-authored-by: Thomas Rodgers <[email protected]> Co-authored-by: sreemoyee-basu <[email protected]> Co-authored-by: Nicolas Dumazet <[email protected]>

Co-authored-by: rlapin-pl <[email protected]>

Co-authored-by: Shuya Ma <[email protected]>

Co-authored-by: Ramon Vermeulen <[email protected]>

Co-authored-by: Riley Karson <[email protected]>

… resources (#16227)

…date (#16221)

modular-magician · 2026-01-28T20:45:41Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 964 files changed, 12797 insertions(+), 1106 deletions(-))
google-beta provider: Diff ( 1063 files changed, 14264 insertions(+), 1246 deletions(-))
terraform-google-conversion: Diff ( 102 files changed, 2856 insertions(+))

modular-magician · 2026-01-28T22:40:07Z

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 964 files changed, 12797 insertions(+), 1106 deletions(-))
google-beta provider: Diff ( 1063 files changed, 14264 insertions(+), 1246 deletions(-))
terraform-google-conversion: Diff ( 102 files changed, 2856 insertions(+))

modular-magician · 2026-01-28T23:04:03Z

Tests analytics

Total tests: 5977
Passed tests: 5343
Skipped tests: 633
Affected tests: 1

Click here to see the affected service packages

All service packages are affected

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccAccessContextManager__service_perimeter_dry_run_egress_policy

Get to know how VCR tests work

modular-magician · 2026-01-28T23:21:30Z

🟢 Tests passed during RECORDING mode:
TestAccAccessContextManager__access_level [Debug log]
TestAccAccessContextManager__access_level_condition [Debug log]
TestAccAccessContextManager__access_level_custom [Debug log]
TestAccAccessContextManager__access_level_full [Debug log]
TestAccAccessContextManager__access_levels [Debug log]
TestAccAccessContextManager__access_policy [Debug log]
TestAccAccessContextManager__access_policy_scoped [Debug log]
TestAccAccessContextManager__authorized_orgs_desc [Debug log]
TestAccAccessContextManager__service_perimeter [Debug log]
TestAccAccessContextManager__service_perimeter_dry_run_egress_policy [Debug log]
TestAccAccessContextManager__service_perimeter_dry_run_ingress_policy [Debug log]
TestAccAccessContextManager__service_perimeter_update [Debug log]
TestAccAccessContextManager__service_perimeters [Debug log]

🔴 Tests failed when rerunning REPLAYING mode:
TestAccAccessContextManager__service_perimeter_dry_run_egress_policy [Error message] [Debug log]
TestAccAccessContextManager__service_perimeter_dry_run_ingress_policy [Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.

🟢 All tests passed!

View the build log or the debug log for each test

modular-magician · 2026-01-29T00:57:19Z

Tests analytics

Total tests: 5980
Passed tests: 5346
Skipped tests: 633
Affected tests: 1

Click here to see the affected service packages

All service packages are affected

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccAccessContextManager__service_perimeter_dry_run_egress_policy

Get to know how VCR tests work

modular-magician · 2026-01-29T01:15:00Z

🟢 Tests passed during RECORDING mode:
TestAccAccessContextManager__access_level [Debug log]
TestAccAccessContextManager__access_level_condition [Debug log]
TestAccAccessContextManager__access_level_custom [Debug log]
TestAccAccessContextManager__access_level_full [Debug log]
TestAccAccessContextManager__access_levels [Debug log]
TestAccAccessContextManager__access_policy [Debug log]
TestAccAccessContextManager__access_policy_scoped [Debug log]
TestAccAccessContextManager__authorized_orgs_desc [Debug log]
TestAccAccessContextManager__service_perimeter [Debug log]
TestAccAccessContextManager__service_perimeter_dry_run_egress_policy [Debug log]
TestAccAccessContextManager__service_perimeter_dry_run_ingress_policy [Debug log]
TestAccAccessContextManager__service_perimeter_update [Debug log]
TestAccAccessContextManager__service_perimeters [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.

🟢 All tests passed!

View the build log or the debug log for each test

SirGitsalot added 2 commits January 27, 2026 15:53

Create a registry package for products to self-register supported res…

203b977

…ources

Update template for changes to the Product type

37af482

modular-magician added the service/terraform label Jan 28, 2026

SirGitsalot requested a review from melinath January 28, 2026 18:17

SirGitsalot marked this pull request as ready for review January 28, 2026 18:17

melinath requested changes Jan 28, 2026

View reviewed changes

shuyama1 and others added 21 commits January 28, 2026 12:08

Temporarily skip 100% failing tests in VCR (Part 1) (#16091)

5ea688d

Temporarily skip 100% failing tests in VCR (Part 2) (#16092)

cbfa9b7

Temporarily skip 100% failing tests in VCR (Part 3) (#16093)

e8653c4

Adding 400g interconnect attachment bandwidth (#16089)

c0316a0

modify loader to add extra fields after loading (#16082)

59521d3

Adding data source to list compute routers (#16027)

fe4e551

Fixed meta yaml formatting (#16100)

99d3169

Remove post-startup-script key from protected metadata for EUC (#15998)

3161388

Multicast domain activation (#15896)

5d64c0e

minor(bigquery_table): Using merge and update for dataPolicies if i…

83fc167

…gnore_schema_changes(dataPolicies) is defined (#15006)

Fix TestAccDataFusionInstance_dataFusionInstanceCmekExample in VCR Re…

532e60a

…playing (#16104)

Add RestoreWorkload resource with custom create and delete (#16080)

3c88bea

change the prefix of folders (#16098)

d7bc5a0

Made meta.yaml tests raise parsing errors and enforce accuracy (#16102)

4e5e89d

feat(compute) - Fix terraform provider crush on empty instance flexib…

f1e4698

…ility policy block. (#15916)

Add IpCollection to the Address resource (#16070)

716a254

Wait for compute api enablement (#16108)

d1bf509

Ensure SQL operations complete before reporting errors (#16059)

68ab0bb

fix TestAccWorkbenchInstance_updateBootDisk (#16109)

44d164f

Implement igmp_query field for compute instance resources (#15926)

6de925a

rileykarson and others added 23 commits January 28, 2026 12:08

Fix devsite link in DialogflowCX ToolVersion (#16210)

1a03c2e

Revert "Display nat ip (#15675)" (#16114)

aafc59c

Modify the redis cluster update tests (#16218)

a985e6a

Fix single product gen (#16217)

700307c

Fix update composer v2 image (#16224)

009b77d

Co-authored-by: rlapin-pl <[email protected]>

Add shuyama1 vacation (#16206)

c020552

bump VCR timeout (#16231)

1425d5d

tgc-revival: generate handwritten tests by default (#16220)

9c56207

tgc-revival: fallback to import format to get IdentityParam (#16235)

5c61463

Unused datasource in tag test (#16229)

25ce42f

fix(container): fix a bug in GKE blue green upgrade policy (#16239)

08863f2

Add skip validations in vmware cluster (#16135)

81bbfe6

Co-authored-by: Shuya Ma <[email protected]>

Ignore IAM condition fields in missing test detector (#16234)

9abfa87

Split nightly test data ingestion from ticket creation (#16240)

8d018ec

feat(logging): Add Cloud Logging Saved Query resource (#16182)

fb7d986

Co-authored-by: Ramon Vermeulen <[email protected]>

Dataplex data asset (#16106)

9515ceb

Co-authored-by: Riley Karson <[email protected]>

Fix test-failure-ticket (#16249)

8b83e68

datafusion: added patch_revision field to google_data_fusion_instance…

489d490

… resources (#16227)

Allow PSC Service Attachment to update target_service as a mutable up…

cd37070

…date (#16221)

Removed ReadProperties method (#16219)

81831ea

Rebase and s/BasePath/BaseUrl/

aa65326

Make registry singletons pointers

57a19ca

Panic if duplicate registrations are attmpted

e38ac1e

Do not perform IAM registration with TGC

2d8b43f

Create a registry package for products to self-register resources #16244

Are you sure you want to change the base?

Create a registry package for products to self-register resources #16244

Conversation

SirGitsalot commented Jan 28, 2026

Uh oh!

modular-magician commented Jan 28, 2026

Diff report

Uh oh!

modular-magician commented Jan 28, 2026

Tests analytics

Action taken

Uh oh!

modular-magician commented Jan 28, 2026

Uh oh!

melinath left a comment

Choose a reason for hiding this comment

Uh oh!

melinath Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

SirGitsalot Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

melinath Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

SirGitsalot Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

melinath Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

SirGitsalot Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

modular-magician commented Jan 28, 2026

Diff report

Uh oh!

modular-magician commented Jan 28, 2026

Diff report

Uh oh!

modular-magician commented Jan 28, 2026

Tests analytics

Action taken

Uh oh!

modular-magician commented Jan 28, 2026

Uh oh!

modular-magician commented Jan 29, 2026

Tests analytics

Action taken

Uh oh!

modular-magician commented Jan 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants