Skip to content

Conversation

@suddendust
Copy link
Contributor

@suddendust suddendust commented Sep 22, 2025

Description

This PR adds this method DocumentType getDocumentType(); to Document.java which returns either FLAT or NESTED. This information can be used by library users to make certain decisions, like in entity-service, we can decide the kind of getter to use based on whether the doc is nested or flat.

Testing

I have added some new test cases to JsonDocumentTest.

Checklist:

  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • Any dependent changes have been merged and published in downstream modules

@codecov
Copy link

codecov bot commented Sep 22, 2025

Codecov Report

❌ Patch coverage is 90.32258% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.34%. Comparing base (107cda0) to head (9c474db).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...va/org/hypertrace/core/documentstore/Document.java 0.00% 1 Missing ⚠️
...rg/hypertrace/core/documentstore/JSONDocument.java 93.33% 0 Missing and 1 partial ⚠️
.../documentstore/postgres/PostgresQueryExecutor.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main     #233      +/-   ##
============================================
+ Coverage     78.49%   79.34%   +0.84%     
- Complexity     1069     1072       +3     
============================================
  Files           206      208       +2     
  Lines          5237     5262      +25     
  Branches        450      451       +1     
============================================
+ Hits           4111     4175      +64     
+ Misses          818      770      -48     
- Partials        308      317       +9     
Flag Coverage Δ
integration 79.34% <90.32%> (+0.84%) ⬆️
unit 56.80% <87.09%> (+1.00%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@suddendust suddendust changed the title [Draft] Make JsonDocument Underlying Storage Type Aware (Document store / SQL store) Make JsonDocument Underlying Storage Type Aware (Document store / SQL store) Sep 22, 2025
Copy link
Contributor

@suresh-prakash suresh-prakash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for keeping this PR short and sweet. 🙂

Comment on lines 4 to 5
SQL_STORE,
DOCUMENT_STORE
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SQL_STORE and DOCUMENT_STORE represent the underlying store types, not exactly the document types.

Can we go with something like
FLAT_STRUCTURE,
NESTED_STRUCTURE,
?

Or, even simply, FLAT, NESTED so that we can call them Flat document and Nested document in our future discussions.

Copy link
Contributor Author

@suddendust suddendust Sep 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea behind adding this attribute is to identify whether the document contains annotated values (valueList, valueMap, etc.) in the attributes field/column or not. For documents containing just one column, does it make sense to call them nested or flat? Do you think smth like PLAIN and ANNOTATED would make more sense?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The nesting of valueList, valueMap, etc. is not handled by the document-store itself. It is something maintained by the clients. So, in a true-sense the annotation is the responsibility of the clients and the document-store itself is agnostic of that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, makes sense. Using FLAT and NESTED now.

return new JSONDocument(objectNode);
}

public static JSONDocument errorDocument(String message, DocumentType documentType) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please explain why this is significant?

Copy link
Contributor Author

@suddendust suddendust Sep 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning the type of an error json doc indicates that that error was encountered while generating a doc of this particular type. Callers can use this information in future.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's not needed for feature parity, I'd keep this change separate (and probably defer it until it's needed).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.


private static ObjectMapper mapper = new ObjectMapper();
private JsonNode node;
private DocumentType documentType;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid issues like NullPointerException, I'd default it to NESTED.

JsonNode jsonNode = MAPPER.readTree(jsonString);
JsonNode decodedJsonNode = recursiveClone(jsonNode, MongoUtils::decodeKey, identity());
return new JSONDocument(decodedJsonNode);
return new JSONDocument(decodedJsonNode, DocumentType.DOCUMENT_STORE);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once you start having a default, these changes are redundant.

}
}
return new JSONDocument(MAPPER.writeValueAsString(jsonNode));
return new JSONDocument(MAPPER.writeValueAsString(jsonNode), DocumentType.SQL_STORE);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same everywhere.

suresh-prakash
suresh-prakash previously approved these changes Sep 24, 2025
suresh-prakash
suresh-prakash previously approved these changes Sep 24, 2025
@suresh-prakash suresh-prakash merged commit 1f1876c into hypertrace:main Sep 24, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants