[ICRDMA] Chunk reclaimation feature for slot mem pool. Change limit. EXT-1782 #31708
Conversation
The original implementation of slot mem pool has no ability to reuse chunk for slots size X if it was split for slots size Y. It can cause the situation when we have no abbility to allocate memory it all chunks was splited for wirkset with different size. The proposed change track usage of chunk and allows to reuse it for any slot size if use count for chunk become 0. This commit also changes limit so allocate object up to 32MB are allowed. Two improvement should be implemented in a future work: - Priority to alocate slots by use count of underlying chunk. This heuristic increases probability of success reclaimation - Varied chunk size. No need to allocate 32Mb chunk to split it in to 4K slots. It also increases probability of success reclaimation and improve perfomance of it
Run Extra TestsRun additional tests for this PR. You can customize:
|
|
🟢 |
|
⚪
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ ⚪ Ya make output | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ ⚪ Ya make output | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
BackportTo backport this PR, click the button next to the target branch and then click "Run workflow" in the Run Actions UI.
|
dcherednik
changed the title
…XT-1782 (ydb-platform#31708) The original implementation of slot mem pool has no ability to reuse chunk for slots size X if it was split for slots size Y. It may cause the situation when we have no ability to allocate memory if all chunks was split for workset with different sizes. The proposed change track usage of chunk and allows to reuse it for any slot size if use count for the chunk becomes 0. This commit also changes limit so allocate object up to 32MB are allowed. Two improvement should be implemented in a future work: * Priority to allocate slots by use count of underlying chunk. This heuristic increases probability of success reclaimation * Varied chunk size. No need to use 32Mb chunk to split it in to 4K slots. It also increases probability of success reclaimation and improve performance of it
…XT-1782 (#31708) The original implementation of slot mem pool has no ability to reuse chunk for slots size X if it was split for slots size Y. It may cause the situation when we have no ability to allocate memory if all chunks was split for workset with different sizes. The proposed change track usage of chunk and allows to reuse it for any slot size if use count for the chunk becomes 0. This commit also changes limit so allocate object up to 32MB are allowed. Two improvement should be implemented in a future work: * Priority to allocate slots by use count of underlying chunk. This heuristic increases probability of success reclaimation * Varied chunk size. No need to use 32Mb chunk to split it in to 4K slots. It also increases probability of success reclaimation and improve performance of it
3 participants
Changelog entry
The original implementation of slot mem pool has no ability to reuse chunk for slots size X if it was split for slots size Y. It can cause the situation when we have no abbility to allocate memory if all chunks was split for workset with different size.
The proposed change track usage of chunk and allows to reuse it for any slot size if use count for chunk become 0.
This commit also changes limit so allocate object up to 32MB are allowed.
Two improvement should be implemented in a future work:
...
Changelog category
Description for reviewers
...