feat(pipeline): add CRTDL preprocessing for DIMP enrichment#122
Merged
feat(pipeline): add CRTDL preprocessing for DIMP enrichment#122
Conversation
2f1498c to
4b1dcd7
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #122 +/- ##
==========================================
- Coverage 84.78% 84.78% -0.01%
==========================================
Files 48 50 +2
Lines 5173 5514 +341
==========================================
+ Hits 4386 4675 +289
- Misses 598 632 +34
- Partials 189 207 +18 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
bbcfcd7 to
67e7f9a
Compare
43a59b7 to
462fe4d
Compare
juliangruendner
requested changes
Feb 3, 2026
Contributor
juliangruendner
left a comment
There was a problem hiding this comment.
this does not seem to do anything -
when using this enrichment file:
[
{
"group_reference": "https://www.medizininformatik-initiative.de/fhir/core/modul-fall/StructureDefinition/KontaktGesundheitseinrichtung",
"add_group_if_not_exists": true,
"attributes_to_add": [
{
"attributeRef": "Encounter.class",
"mustHave": false
},
{
"attributeRef": "Encounter.status",
"mustHave": false
},
{
"attributeRef": "Encounter.serviceType",
"mustHave": false
}
]
}
]the group does not appear in the enriched CRTDL
b5ab1a4 to
c324ed1
Compare
784922a to
efa34b4
Compare
2d5fb2b to
fd028ad
Compare
558c077 to
f57469f
Compare
Contributor
juliangruendner
left a comment
There was a problem hiding this comment.
the following enrichment config does not lead to the group being added if it does not exist in the CRTDL
[
{
"groupReference": "https://www.medizininformatik-initiative.de/fhir/core/modul-person/StructureDefinition/PatientPseudonymisiert",
"addGroupIfNotExists": true,
"attributesToAdd": [
{
"attributeRef": "Patient.identifier",
"mustHave": false
}
]
},
{
"groupReference": "https://www.medizininformatik-initiative.de/fhir/core/modul-fall/StructureDefinition/KontaktGesundheitseinrichtung",
"addGroupIfNotExists": true,
"attributesToAdd": [
{
"attributeRef": "Encounter.identifier",
"mustHave": false
}
]
}
]36b4e20 to
0ded055
Compare
0ded055 to
935806a
Compare
juliangruendner
previously approved these changes
Mar 11, 2026
Add capability to enrich CRTDL files with additional attributes required by DIMP (pseudonymization) before sending them to TORCH. This addresses the need for identifiers like Patient.identifier:PseudonymisierterIdentifier and Encounter.identifier:Aufnahmenummer that are not part of research queries but needed for pseudonymization. Key features: - Add/update attributes in existing CRTDL groups by groupReference - Create new groups when addGroupIfNotExists is true - Resolve linkedGroups profile URLs to group IDs - Support both external JSON file and inline YAML configuration - Save enriched CRTDL to job directory for debugging/auditing Files created: - internal/models/crtdl_preprocessing.go: Config and enrichment models - internal/services/crtdl_preprocessor.go: Pure enrichment functions - tests/unit/crtdl_preprocessor_test.go: 20 comprehensive unit tests Closes #75
935806a to
f3cdaf3
Compare
juliangruendner
approved these changes
Mar 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add capability to enrich CRTDL files with additional attributes required by DIMP (pseudonymization) before sending them to TORCH.
groupReferenceaddGroupIfNotExists: truelinkedGroupsprofile URLs to group IDsFiles created:
internal/models/crtdl_preprocessing.go: Config and enrichment modelsinternal/services/crtdl_preprocessor.go: Pure enrichment functionstests/unit/crtdl_preprocessor_test.go: 20 comprehensive unit testsCloses #75
Test plan
make test)