Skip to content

add prefix option to DataTransformMDB class#411

Merged
DylanWelzel merged 1 commit into1.1.xfrom
datatransform-prefix
Nov 13, 2025
Merged

add prefix option to DataTransformMDB class#411
DylanWelzel merged 1 commit into1.1.xfrom
datatransform-prefix

Conversation

@DylanWelzel
Copy link
Copy Markdown
Contributor

This pull request improves the handling of identifier prefixes in the DataTransformMDB class by allowing prefixes to be defined at the node level in the transformation graph and ensuring they are automatically applied during data transformation. The changes centralize and standardize how prefixes are managed, making the codebase more robust and easier to configure.

Prefix handling enhancements:

  • Added support for defining identifier prefixes as node attributes in the graph, with automatic prefix application when converting identifiers to nodes.
  • Introduced the _apply_prefix method to encapsulate prefix logic, ensuring identifiers are correctly formatted with their configured prefixes.

Integration of prefix logic in data transformation:

  • Updated the _copy method to use _apply_prefix, applying prefixes to _id fields when copying identifiers.
  • Modified the _build_hit_miss_lsts function to apply prefixes to _id fields when constructing new documents, using the new _apply_prefix method.
  • Updated the call to _copy in key_lookup_batch to pass the output type, enabling correct prefix application during batch lookups.

@DylanWelzel DylanWelzel merged commit 2d84b08 into 1.1.x Nov 13, 2025
6 of 7 checks passed
@DylanWelzel DylanWelzel deleted the datatransform-prefix branch November 13, 2025 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants