Skip to content

feat(resources): Add JSONata propertyTransforms for diff suppression#2729

Merged
corymhall merged 4 commits intomasterfrom
propertyTransforms
Feb 17, 2026
Merged

feat(resources): Add JSONata propertyTransforms for diff suppression#2729
corymhall merged 4 commits intomasterfrom
propertyTransforms

Conversation

@rshade
Copy link
Contributor

@rshade rshade commented Dec 22, 2025

Implements CloudFormation-compatible property transforms using JSONata expressions. These transforms normalize cloud provider responses to prevent spurious diffs during refresh/update cycles.

Key changes:

  • Extract propertyTransforms from CloudFormation schemas during code generation
  • Add JSONata evaluation engine (blues/jsonata-go) for runtime transforms
  • Apply transforms during diff calculation to normalize values before comparison
  • Support all JSONata functions from CF schemas ($lowercase, $uppercase, $join, etc.)

75 resources now have propertyTransforms including:

  • RDS: $lowercase(DBClusterIdentifier), $lowercase(Engine)
  • EFS: $uppercase() for replicationOverwriteProtection enum normalization
  • Various AWS services with identifier case normalization

Includes E2E tests validating:

  • EFS DISABLED→REPLICATING enum transition handling
  • RDS UPPERCASE→lowercase identifier normalization

@github-actions
Copy link
Contributor

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

@codecov
Copy link

codecov bot commented Dec 22, 2025

Codecov Report

❌ Patch coverage is 82.67148% with 48 lines in your changes missing coverage. Please review.
✅ Project coverage is 53.98%. Comparing base (96c10ef) to head (070d128).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
provider/pkg/resources/suppress_diff.go 79.12% 11 Missing and 8 partials ⚠️
provider/pkg/resources/jsonata_transform.go 86.56% 12 Missing and 6 partials ⚠️
provider/pkg/schema/gen.go 84.61% 3 Missing and 3 partials ⚠️
provider/pkg/provider/provider.go 61.53% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2729      +/-   ##
==========================================
+ Coverage   52.92%   53.98%   +1.05%     
==========================================
  Files          50       51       +1     
  Lines        6146     6391     +245     
==========================================
+ Hits         3253     3450     +197     
- Misses       2620     2651      +31     
- Partials      273      290      +17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@rshade rshade force-pushed the propertyTransforms branch 2 times, most recently from 4f74f73 to 108e70e Compare December 23, 2025 15:05
@rshade rshade marked this pull request as ready for review December 23, 2025 15:06
Copilot AI review requested due to automatic review settings December 23, 2025 15:06
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements CloudFormation-compatible property transforms using JSONata expressions to prevent spurious diffs during resource refresh/update cycles. The implementation extracts transform specifications from CloudFormation schemas during code generation and applies them at runtime during diff calculation to normalize AWS response values before comparison.

Key Changes:

  • Added JSONata evaluation engine (blues/jsonata-go v1.5.4) for runtime property value transformations
  • Implemented schema extraction logic to read propertyTransform specifications from CloudFormation schemas during code generation
  • Integrated transform-based diff suppression into the existing SuppressAWSManagedDiffs flow, affecting 75 AWS resources

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
provider/pkg/resources/jsonata_transform.go Core JSONata transform engine with caching, path matching, and evaluation logic
provider/pkg/resources/suppress_diff.go Integration of property transforms into diff suppression with recursive object/array handling
provider/pkg/schema/gen.go Extraction of propertyTransform specs from CloudFormation schemas and CFN-to-SDK path conversion
provider/pkg/schema/resource_props_test.go Unit tests for CFN path conversion and transform extraction
provider/pkg/resources/jsonata_transform_test.go Unit tests for path matching, caching, evaluation, and value comparison
provider/pkg/resources/jsonata_integration_test.go Integration tests covering RDS, EC2, EFS transforms with realistic scenarios
provider/pkg/provider/provider_2e2_test.go E2E test for RDS lowercase transforms (skipped by default due to 15-20min runtime)
provider/pkg/provider/testdata/rds-lowercase-transforms/Pulumi.yaml Test infrastructure definition for RDS E2E testing
provider/pkg/metadata/metadata.go Added PropertyTransforms field to CloudAPIResource struct
provider/go.mod & provider/go.sum Added blues/jsonata-go v1.5.4 dependency
provider/cmd/pulumi-resource-aws-native/metadata.json Added propertyTransforms for 75 resources across various AWS services

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@rshade rshade force-pushed the propertyTransforms branch 4 times, most recently from 94c2f97 to 4f1e4d4 Compare December 23, 2025 21:47
Copy link
Member

@corymhall corymhall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't been able to review the entire PR yet, but here are my comments so far. Most of it is just related to re-using existing functionality rather than re-writing the logic.

@rshade rshade force-pushed the propertyTransforms branch from 4f1e4d4 to 3a3f6ae Compare December 29, 2025 21:52
@rshade rshade force-pushed the propertyTransforms branch 3 times, most recently from 5c60380 to 56fe644 Compare December 30, 2025 15:26
@rshade rshade requested a review from Copilot December 30, 2025 15:37
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 8 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@rshade rshade force-pushed the propertyTransforms branch from 56fe644 to 4eb89fd Compare December 30, 2025 17:19
…ression

Implements CloudFormation-compatible property transforms using JSONata
expressions. These transforms normalize cloud provider responses to prevent
spurious diffs during refresh/update cycles.

Key changes:
- Extract propertyTransforms from CloudFormation schemas during code generation
- Add JSONata evaluation engine (blues/jsonata-go) for runtime transforms
- Apply transforms during diff calculation to normalize values before comparison
- Support all JSONata functions from CF schemas ($lowercase, $uppercase, $join, etc.)

75 resources now have propertyTransforms including:
- RDS: $lowercase(DBClusterIdentifier), $lowercase(Engine)
- EFS: $uppercase() for replicationOverwriteProtection enum normalization
- Various AWS services with identifier case normalization

Includes E2E tests validating:
- EFS DISABLED→REPLICATING enum transition handling
- RDS UPPERCASE→lowercase identifier normalization
@rshade rshade force-pushed the propertyTransforms branch from 4eb89fd to 4b205a9 Compare January 5, 2026 17:13
@pgavlin
Copy link
Member

pgavlin commented Feb 3, 2026

Related to #2686

@corymhall corymhall enabled auto-merge (squash) February 17, 2026 14:36
@corymhall corymhall merged commit b3603e1 into master Feb 17, 2026
20 checks passed
@corymhall corymhall deleted the propertyTransforms branch February 17, 2026 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants