feat: add preflight checks for transform#736
Open
katie-perry wants to merge 2 commits intostagingfrom
Open
Conversation
katie-perry
added
the
chore
katie-perry
commented
Feb 26, 2026
| "Known symptom: AttributeError involving `ValidationInfo` and " | ||
| "`warnings`.\n" | ||
| "Try pinning pydantic below 2.12 in this environment, e.g.:\n" | ||
| ' pip install "pydantic<2.12"' |
Contributor
Author
There was a problem hiding this comment.
this is the weird issue I ran into yesterday that I talked about in the description. I don't really like having this specific in here (I'll likely end up removing it) but left it in for now to call attention to this - should we be pinning a specific pydantic version between the metakb and normalizer repos? The difference in versions I had between metakb and variation normalizer was calling a failure. Matching my metakb pydantic version to the one in var-norm fixed it.
I'm wondering if I should've looked more into it and created a discussion or a ticket...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I was having some issues yesterday when I was trying to run
transformand lost more time than I'd like to admit because I didn't realize I had some environment variables misconfigured (or forgot to set them at all). The transform will fully run, taking a few minutes, leading a dev to believe it's working properly, just to arrive at the end with an empty db and silent failures.This code was debug code for myself, but I decided to open this PR in case it might be helpful to add to the cli. Everything should work the same as before (I think. I didn't verify with the OG way of having the normalizers stood up on a local ddb instance) but with initial checks and probes before moving on.
One thing I'm not super sure on is the pydantic version issue. It seems super niche to be including in the code like this, so I might end up taking that out. I'll comment on it to bring attention.
I'm also unsure about exposing the errors - it seems like there's existing patterns in place to silence/log errors. When I was trying this out yesterday, though, I didn't see any errors logged to metakb.log when I was running the transform.