Skip to content

Conversation

c-warren
Copy link
Contributor

@c-warren c-warren commented Sep 4, 2025

What changed?

Adds fuzz tests to all thrift->proto->thrift round trip mappers.

Why?

When mapping from idl to another there are multiple scenarios:

  1. neither mapper implements a field
  2. the mapper implementations are incompatible
  3. both mappers implement a field correctly
  4. both mappers implement conversion incorrectly but in a reversible way (e.g they both do a rot13 of an [a-z] string)

Our existing tests test for scenarios 2 and 3, but don't test for 1 - if someone adds a new field and omits it from the mappers + the test data, it will never be tested for. This PR adds a fuzz test which randomly assigns data to all fields (including nils), which ensures that unmapped data will be tested for without requiring the developer to check it (and keeps the tests for 2 & 3).

⚠️ This is a huge PR and I'm not sure it's particularly readable. I'd recommend starting at runFuzzTest to get the general idea of how it has been implemented. In particular, FuzzOptions should allow a developer to:

  • intentionally exclude fields that shouldn't be mapped by a specific mapper
  • add custom fuzz functions for abnormal fields (like enums, durations, timestamps if required)

However I will say - it's added 2000 lines to this file which makes it borderline unreadable (maybe not borderline).

How did you test it?

It's all unit tests.

Potential risks

N/A

Copy link

codecov bot commented Sep 4, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.50%. Comparing base (b6fc775) to head (5c04f9d).

❌ Your project check has failed because the head coverage (82.50%) is below the target coverage (85.00%). You can increase the head coverage or adjust the target coverage.
see 2 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b6fc775...5c04f9d. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

},
ExcludedFields: []string{
// Exclude nested fields that have complex issues like in DescribeDomainResponse
"Domain.ActiveClusters", // Nil pointer dereference in mapper conversion
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a bug or a problem in data representation?

Copy link
Member

@davidporter-id-au davidporter-id-au left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This strikes me as really good work, very useful.

Re the todos / fields that couldn't be mapped, I'd request a clarification / distinction between the reasons for finding that fields can't be mapped, something along the lines of:

  • This field isn't mapped because it's not relevant and we don't care. it's known, understood and safe to ignore.
  • This field isn't mapped, the reason is not understood or known or not investigated yet.
  • A clear bug / failure to handle nils / failure to map / a known bug

and so fourth, to be Rumsfeldian for a second just a means for the reader to differentiate between the following dimensions:

  • investigated / not investigated
  • bug / not a bug / limitation of the mapper / not added because too much work

Other than requesting clarity on the comments, I have no other concerns. And, to be clear, am not asking for further investigations or effort (that's totally fine to push to future work). Just a clear indication of what is currently known

Copy link
Member

@natemort natemort left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for all your work on this!

@c-warren c-warren merged commit 73d0e88 into cadence-workflow:master Sep 9, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants