feat(bridge): add provider inconsistency detection feature

[rshade]

Implements the Provider Inconsistency Detection feature, which detects and reports when upstream Terraform providers produce inconsistent results after apply operations. This feature helps users identify bugs in upstream providers by comparing planned state with actual state.

Feature Details

• Added three environment variables to control the feature:

  • PULUMI_DETECT_INCONSISTENT_APPLY: Enables/disables detection (default: false)
  • PULUMI_DETECT_INCONSISTENT_APPLY_DETAIL: Controls detail level (normal, debug, trace)
  • PULUMI_DETECT_INCONSISTENT_APPLY_RESOURCES: Limits to specific resources (comma-separated list)

• Implemented a comprehensive filtering system:

  • Common filters for timestamps, IDs, etags, and other frequently changing attributes
  • Provider-specific filters for AWS, GCP, and Azure resources
  • Type conversion handling (e.g., handling "5" vs 5, boolean representation differences)
  • Path-based filtering to ignore known problematic attributes

• Added robust state comparison logic:

  • Support for both SDK and Plugin Framework providers
  • Performance optimizations for large resources (sampling approach for complex objects)
  • Early exit paths to minimize overhead when differences are detected
  • Depth-limited recursion to handle deeply nested objects

• Integrated detection into resource Create and Update methods:

  • Implementation in both SDKv2 and Plugin Framework provider code paths
  • Non-intrusive design that only logs warnings without affecting resource operations
  • Only runs after successful operations to maintain backward compatibility

The feature is completely opt-in and only enabled via environment variables, ensuring no behavior changes for existing users. When enabled, it provides valuable debugging information to help identify inconsistencies in provider behavior that might cause issues with resource management.

Closes #2413

[Copilot]

Pull Request Overview

This PR implements a new provider inconsistency detection feature to help identify discrepancies between the planned and actual state from Terraform providers after apply operations. Key changes include:

• Adding environment variables and configuration logic for controlling inconsistency detection.
• Implementing detection logic and filters for both SDK and Plugin Framework providers.
• Integrating detection into resource Create and Update flows in both tfbridge and PF provider implementations.

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
pkg/tfbridge/providerinfo.go	Adds licensing header and package declaration.
pkg/tfbridge/provider.go	Integrates inconsistency detection into Create and Update methods.
pkg/tfbridge/inconsistency_filter.go	Introduces filtering for known attribute inconsistencies.
pkg/tfbridge/inconsistency_detector.go	Implements detection logic comparing planned vs. actual state.
pkg/tfbridge/inconsistency_config.go	Adds configuration parsing for inconsistency detection.
pkg/pf/tfbridge/provider_update.go	Adds detection logic to the PF provider update path.
pkg/pf/tfbridge/provider_create.go	Adds detection logic to the PF provider create path.
pkg/pf/tfbridge/inconsistency_detector.go	Implements PF-specific inconsistency detection using recursive diff.

[Copilot]

Pull Request Overview

Introduces a feature to detect and report when Terraform providers yield unexpected state changes by comparing planned vs applied state and filtering known benign differences.

• Adds a filter framework for ignoring common and provider-specific attribute/value inconsistencies
• Implements core detection logic, state comparison, and reporting with configurable detail levels
• Hooks the detection into SDKv2 Plugin Framework provider Create/Update paths

Reviewed Changes

Copilot reviewed 31 out of 31 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
pkg/tfbridge/inconsistency_filter.go	Implements attribute/value filters for known provider inconsistencies
pkg/tfbridge/inconsistency_detector.go	Core logic to compare planned vs applied state and collect discrepancies
pkg/tfbridge/inconsistency_config.go	Parses environment variables to configure detection behavior
pkg/pf/tfbridge/provider_update.go	Integrates detection into PF provider UpdateWithContext
pkg/pf/tfbridge/provider_create.go	Integrates detection into PF provider CreateWithContext

Comments suppressed due to low confidence (3)

pkg/pf/tfbridge/provider_update.go:118

• Calling detectAndReportPFInconsistencies here refers to an undefined function in this package; it should be qualified (e.g., tfbridge.DetectAndReportPFInconsistencies) or imported from the correct module to avoid a compile error.

if err := detectAndReportPFInconsistencies(

pkg/pf/tfbridge/provider_create.go:110

• Calling detectAndReportPFInconsistencies here refers to an undefined function in this package; it should be qualified (e.g., tfbridge.DetectAndReportPFInconsistencies) or imported from the correct module to avoid a compile error.

if err := detectAndReportPFInconsistencies(

pkg/tfbridge/inconsistency_detector.go:199

• The expression len(plannedAttrs)+len(actualAttrs)-MaxReportedDifferences may not reflect the actual number of skipped differences, resulting in an inaccurate message. Consider tracking the real count of hidden discrepancies for a precise truncation notice.

"and %d more differences (truncated)",

[codecov]

Codecov Report

Attention: Patch coverage is 8.95141% with 712 lines in your changes missing coverage. Please review.

Project coverage is 63.71%. Comparing base (3c1fafe) to head (900dc5f).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
pkg/pf/tfbridge/inconsistency_detector.go	1.85%	264 Missing ⚠️
pkg/tfbridge/inconsistency_filter.go	9.78%	212 Missing ⚠️
pkg/tfbridge/inconsistency_detector.go	0.00%	155 Missing ⚠️
pkg/tfbridge/inconsistency_config.go	29.68%	43 Missing and 2 partials ⚠️
pkg/tfbridge/provider.go	0.00%	22 Missing and 2 partials ⚠️
pkg/pf/tfbridge/provider_create.go	61.11%	5 Missing and 2 partials ⚠️
pkg/pf/tfbridge/provider_update.go	64.28%	4 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3071      +/-   ##
==========================================
- Coverage   68.59%   63.71%   -4.89%     
==========================================
  Files         335      319      -16     
  Lines       43423    42941     -482     
==========================================
- Hits        29788    27359    -2429     
- Misses      11956    13819    +1863     
- Partials     1679     1763      +84     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[t0yv0]

hi @rshade - can you give some context on the desired functionality here? Thanks so much.

At a glace on what is requested in :
#2413

The inconsistency of TF planned and applied values is already a feature of TF provider frameworks.

I believe this is connected to

pulumi-terraform-bridge/pkg/vendored/opentofu/providers/provider.go

Line 349 in c16e1ab

LegacyTypeSystem bool

LegacyTypeSystem flag on the protocol.

Some providers such as pulumi-aws cannot guarantee consistency and need the LegacyTypeSystem turned on. It appears that possibly the bridge over-eagerly sets LegacyTypeSystem for providers that do not need it and can succeed with aggressive TF consistency checks turned on. Could the fix be along the lines of tweaking LegacyTypeSystem setting and/or perhaps tweaking error recognition and reporting?