-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[fix][ml] Fix issues in estimateEntryCountBySize #24089
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[fix][ml] Fix issues in estimateEntryCountBySize #24089
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #24089 +/- ##
============================================
+ Coverage 73.57% 74.16% +0.59%
+ Complexity 32624 32070 -554
============================================
Files 1877 1864 -13
Lines 139502 144334 +4832
Branches 15299 16463 +1164
============================================
+ Hits 102638 107052 +4414
+ Misses 28908 28824 -84
- Partials 7956 8458 +502
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
|
There seem to be other gaps in the estimation logic. I'll revisit this in this PR in further commits. |
managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java
Show resolved
Hide resolved
214a2a2 to
dd2d0b4
Compare
(cherry picked from commit a44b2cf)
(cherry picked from commit a44b2cf)
(cherry picked from commit a44b2cf)
|
follow up in #24125, please review |
(cherry picked from commit a44b2cf)
Motivation
The changes introduced in PR #23931 caused a regression in the entry count estimation logic for branch-3.0. While PR #24055 attempted to address this regression, it did not fully resolve the issues, leaving Pulsar's branch-3.0 in a broken state. A
NullPointerExceptioncould occur when usingReadOnlyManagedLedgerImplinstances, as thecurrentLedgeris null in these cases.ReadOnlyManagedLedgerImplis used in branch-3.0 by PulsarSQL. In master branch, it's currently unused, but since the code exists, it makes sense to fix this bug in the master branch. This PR also contains other improvements.A problem was also detected in using ManagedLedgerImpl.getNextValidPosition method to find the next valid position. This method is relatively costly and takes the current LAC into account which doesn't make much sense just for estimation purposes.
Modifications
currentLedgerto handleReadOnlyManagedLedgerImplinstances properlyEntryCountEstimatorclass to improve testability and code claritylongtointto match the actual usage in the codebaseEntryCountEstimatorTestthat validate various scenarios.Documentation
docdoc-requireddoc-not-neededdoc-complete