Made meta.yaml tests raise parsing errors and enforce accuracy

[melinath]

This PR refactors how the meta.yaml files are processed in tests so that we're able to catch errors in loading the metadata (for example parsing errors like the one fixed by #16100).

I wasn't sure how much of the logged errors are a contract that TGC's later parsing requires; I tried to keep the output as similar as possible.

I removed the "statistic" approach since at this point we need to expect 100% correctness in meta.yaml files.

Note: There are some metadata issues that will need to be fixed before this can be merged. I'll fix them in a follow-on commit, but for now they can serve to demonstrate that the logic works.

Primarily want review from @zli82016 on this as TGC owner, but cc @ScottSuarez if you want to verify that it all looks correct.

Release Note Template for Downstream PRs (will be copied)

See Write release notes for guidance.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 376 insertions(+), 250 deletions(-))
google-beta provider: Diff ( 4 files changed, 376 insertions(+), 249 deletions(-))

[melinath]

@zli82016 I'm done with this PR for now - let me know if it looks good from the TGC perspective, thanks!

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 5 files changed, 399 insertions(+), 253 deletions(-))
google-beta provider: Diff ( 5 files changed, 399 insertions(+), 252 deletions(-))

[modular-magician]

Tests analytics

Total tests: 5901
Passed tests: 5277
Skipped tests: 621
Affected tests: 3

Click here to see the affected service packages

All service packages are affected

Action taken

Found 3 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

• TestAccAccessContextManager__service_perimeter_dry_run_egress_policy
• TestAccAccessContextManager__service_perimeter_dry_run_ingress_policy
• TestAccDataFusionInstance_dataFusionInstanceCmekExample

Get to know how VCR tests work

[zli82016]

Tested the PR locally and the test metadata doesn't change. The logged errors are not parsed in the TGC workflow.

[modular-magician]

🟢 Tests passed during RECORDING mode:
TestAccAccessContextManager__access_level [Debug log]
TestAccAccessContextManager__access_level_condition [Debug log]
TestAccAccessContextManager__access_level_custom [Debug log]
TestAccAccessContextManager__access_level_full [Debug log]
TestAccAccessContextManager__access_levels [Debug log]
TestAccAccessContextManager__access_policy [Debug log]
TestAccAccessContextManager__access_policy_scoped [Debug log]
TestAccAccessContextManager__authorized_orgs_desc [Debug log]
TestAccAccessContextManager__service_perimeter [Debug log]
TestAccAccessContextManager__service_perimeter_dry_run_egress_policy [Debug log]
TestAccAccessContextManager__service_perimeter_dry_run_ingress_policy [Debug log]
TestAccAccessContextManager__service_perimeter_update [Debug log]
TestAccAccessContextManager__service_perimeters [Debug log]
TestAccDataFusionInstance_dataFusionInstanceCmekExample [Debug log]

🔴 Tests failed when rerunning REPLAYING mode:
TestAccAccessContextManager__service_perimeter_dry_run_ingress_policy [Error message] [Debug log]
TestAccDataFusionInstance_dataFusionInstanceCmekExample [Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.

---

🟢 All tests passed!

View the build log or the debug log for each test

[modular-magician]

Tests analytics

Total tests: 5900
Passed tests: 5276
Skipped tests: 622
Affected tests: 2

Click here to see the affected service packages

All service packages are affected

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

• TestAccAccessContextManager__service_perimeter_dry_run_egress_policy
• TestAccDataFusionInstance_dataFusionInstanceCmekExample

Get to know how VCR tests work

[modular-magician]

🟢 Tests passed during RECORDING mode:
TestAccAccessContextManager__access_level [Debug log]
TestAccAccessContextManager__access_level_condition [Debug log]
TestAccAccessContextManager__access_level_custom [Debug log]
TestAccAccessContextManager__access_level_full [Debug log]
TestAccAccessContextManager__access_levels [Debug log]
TestAccAccessContextManager__access_policy [Debug log]
TestAccAccessContextManager__access_policy_scoped [Debug log]
TestAccAccessContextManager__authorized_orgs_desc [Debug log]
TestAccAccessContextManager__service_perimeter [Debug log]
TestAccAccessContextManager__service_perimeter_dry_run_egress_policy [Debug log]
TestAccAccessContextManager__service_perimeter_dry_run_ingress_policy [Debug log]
TestAccAccessContextManager__service_perimeter_update [Debug log]
TestAccAccessContextManager__service_perimeters [Debug log]
TestAccDataFusionInstance_dataFusionInstanceCmekExample [Debug log]

🔴 Tests failed when rerunning REPLAYING mode:
TestAccAccessContextManager__service_perimeter_dry_run_egress_policy [Error message] [Debug log]
TestAccAccessContextManager__service_perimeter_dry_run_ingress_policy [Error message] [Debug log]
TestAccDataFusionInstance_dataFusionInstanceCmekExample [Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.

---

🟢 All tests passed!

View the build log or the debug log for each test