✨ feat(checksum calculator): Add cache to remote checksum calculator#1481
✨ feat(checksum calculator): Add cache to remote checksum calculator#1481
Conversation
|
@algomaster99 would love a review! Quite low hanging apple in terms of speedup to complexity ratio |
6d0e583 to
f0c71d3
Compare
There was a problem hiding this comment.
Pull request overview
This PR adds caching functionality to the RemoteChecksumCalculator to significantly improve performance when generating lockfiles in remote mode. The change addresses issue #1476, which reported extremely slow execution times (around 20 minutes) when generating lockfiles with remote checksums. According to the PR description, the implementation reduces lockfile generation time from approximately 20 minutes to around 5 minutes by caching previously fetched checksums and repository information.
Changes:
- Added two
ConcurrentHashMapcaches to store checksums and repository information - Implemented cache lookup logic in both checksum calculation and repository resolution methods
- Cache uses artifact coordinates as keys and stores empty strings/sentinel values for failed lookups
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
...src/main/java/io/github/chains_project/maven_lockfile/checksum/RemoteChecksumCalculator.java
Show resolved
Hide resolved
...src/main/java/io/github/chains_project/maven_lockfile/checksum/RemoteChecksumCalculator.java
Outdated
Show resolved
Hide resolved
|
The cache implemented here is useful if we compute checksum of a dependency multiple times. For example, we can see I suggest computing performance improvement over spoon just to know what actually is slowing down remote checksum calculator. |
|
I will run tests over spoon, my plan is to implement parallel connections in another PR to separate the functionality. Adding parallellism introduces quite a bit of complexity so I thought it would be a good idea to start with this easier change which already gets huge improvements. |
|
Here are my spoon tests. Without caching: With caching: |
|
Oh this is very nice!
Sounds good, although 1 minute is also not too bad :) |
4b9c931 to
e87ca10
Compare
@LogFlames could you share the exact command to reproduce this? I feel Even with |
Adds cache to remote checksum calculator. Part of solution for #1476.
Reduces time to generate lockfile for
maven-lockfilein remote mode (plugins included) from:To