Skip to content

Refactored ID mapping to use databricks jdbc template instead (#1571)#1576

Merged
ao508 merged 4 commits intomasterfrom
dev-databricks-conn
Jun 27, 2025
Merged

Refactored ID mapping to use databricks jdbc template instead (#1571)#1576
ao508 merged 4 commits intomasterfrom
dev-databricks-conn

Conversation

@ao508
Copy link
Collaborator

@ao508 ao508 commented Jun 25, 2025

Migrate to Databricks from CRDB mapping table

Briefly describe changes proposed in this pull request:

  • Refactored model, persistence, and service layers with changes related to migrating to using databricks
    • updated ID mapping model with new column names
  • Removed references to CRDB ID mapping tables
  • Fixed/updated unit tests affected by incoming changes to the ID lookup
  • updated README doc with java arg necessary for databricks dependency as java21 isn't fully supported yet but there's a workaround provided by Databricks to help in the meantime.

Crossing T's and dotting I's

Please follow these checklists to help prevent any unexpected issues from being introduced by the changes in this pull request. If an item does not apply then indicate so by surrounding the line item with ~~ to strikethrough the text. See basic writing and formatting syntax for more information.

I. Web service and data model checklist

Please follow these checks if any changes were made to any classes in the web, service, or persistence layers.

Code checks:

  • Endpoints were tested to ensure their integrity.
  • Screenshots have been provided to demonstrate changes made to the response body JSON schema and/or swagger page.
  • Unit tests were updated in relation to updates to the mocked test data.

II. Neo4j models and database schema checklist:

  • Neo4j persistence models were changed.
  • The graph database produces the expected changes to models, relationships, and/or property names. [provide screenshot of updated elements in graph db below]

Screenshot: New Patient node imported during a clinical sample import has both a CMO ID and DMP ID PatientAlias
image

III. Message handlers checklist:

  • Changes in this PR affect the workflow of incoming messages.
  • Messages are following the expected workflow when published to the topic(s) changed or introduced in this pull request. (confirmed by testing the request-reply PATIENT_ID_MAPPING topic)
  • Unit tests were added to ensure messages are handled as expected.

If no unit tests were updated or added, then please explain why: Updated existing tests to ensure they still work as expected.

Please describe how the workflow and messaging was tested/simulated:

Describe your testing environment:

  • NATS [local, local docker, dev server, production]
  • Neo4j [local, local docker, dev server, production]
  • SMILE Server [local, local docker, dev server, production]
  • Message publishing simulation [nats cli, docker nats cli, smile publisher tool, other (describe below)]

IV. Configuration and/or permissions checklist:

  • New topics were introduced.
  • The topics and appropriate permissions were updated in smile-configuration. pending merge
  • If applicable, a new account was set up and the account credentials and keys are checked into smile-configuration.
  • Account credentials and keys were shared with the appropriate parties.

General checklist:

  • All requested changes and comments have been resolved.
  • The commit log is comprehensible. It follows 7 rules of great commit messages. For most PRs a single commit should suffice, in some cases multiple topical commits can be useful. During review it is ok to see tiny commits (e.g. Fix reviewer comments), but right before the code gets merged to master or rc branch, any such commits should be squashed since they are useless to the other developers. Definitely avoid merge commits, use rebase instead.

ao508 added 4 commits June 24, 2025 16:25
Signed-off-by: Angelica Ochoa <15623749+ao508@users.noreply.github.com>
Signed-off-by: Angelica Ochoa <15623749+ao508@users.noreply.github.com>
Signed-off-by: Angelica Ochoa <15623749+ao508@users.noreply.github.com>
Signed-off-by: Angelica Ochoa <15623749+ao508@users.noreply.github.com>
Copy link
Contributor

@qu8n qu8n left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything seems clear. Thank you!

@ao508 ao508 merged commit e4ba269 into master Jun 27, 2025
2 checks passed
@ao508 ao508 deleted the dev-databricks-conn branch June 27, 2025 18:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants