Skip to content

Conversation

@Jackie-Jiang
Copy link
Contributor

Within the routing table, instance ids (server ids) are repeated under each segment's routing entry. We explicitly intern them when reading them from InstanceConfig or IS/EV to reduce the memory footprint

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes memory usage in routing tables by interning instance IDs (server IDs) that are frequently repeated across segment routing entries. The changes ensure instance IDs are interned when reading from InstanceConfig, IdealState, and ExternalView to reduce memory footprint.

Changes:

  • Modified ServerInstance constructor to accept a pre-interned instance ID parameter
  • Added string interning for instance IDs and states in IdealState and ExternalView ZNRecords
  • Updated instance config processing to intern instance IDs before creating ServerInstance objects

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
pinot-core/src/main/java/org/apache/pinot/core/transport/ServerInstance.java Refactored constructor to accept pre-interned instanceId parameter; preserved backward compatibility with @VisibleForTesting constructor
pinot-broker/src/main/java/org/apache/pinot/broker/routing/manager/BaseBrokerRoutingManager.java Added internMapFields() method to intern instance IDs and states; integrated interning into IdealState/ExternalView loading and instance config processing

@codecov-commenter
Copy link

codecov-commenter commented Jan 30, 2026

Codecov Report

❌ Patch coverage is 84.21053% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.18%. Comparing base (77a3dd6) to head (6ee8471).

Files with missing lines Patch % Lines
...rg/apache/pinot/core/transport/ServerInstance.java 71.42% 1 Missing and 1 partial ⚠️
...oker/routing/manager/BaseBrokerRoutingManager.java 91.66% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #17605      +/-   ##
============================================
+ Coverage     63.15%   63.18%   +0.03%     
  Complexity     1479     1479              
============================================
  Files          3173     3173              
  Lines        189917   189927      +10     
  Branches      29064    29064              
============================================
+ Hits         119935   120004      +69     
+ Misses        60658    60598      -60     
- Partials       9324     9325       +1     
Flag Coverage Δ
custom-integration1 100.00% <ø> (ø)
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 0.00% <ø> (ø)
java-11 63.15% <84.21%> (+0.01%) ⬆️
java-21 63.12% <84.21%> (+7.63%) ⬆️
temurin 63.18% <84.21%> (+0.03%) ⬆️
unittests 63.18% <84.21%> (+0.03%) ⬆️
unittests1 55.54% <57.14%> (+0.01%) ⬆️
unittests2 34.03% <84.21%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Jackie-Jiang
Copy link
Contributor Author

Seems the change is adding CPU overhead, and string deduplication can be achieved by XX:+UseStringDeduplication JVM option. Closing it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants