Skip to content

Make Mmapper a struct, and existing impls storages #1345

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

wks
Copy link
Collaborator

@wks wks commented Jul 19, 2025

We make Mmapper a struct instead of trait. It will be solely responsible for transitioning the map states.

Existing implementations, including ByteMapMmapper and TwoLevelMmapper, become the storage backends of Mmapper, namely ByteMapStateStorage and TwoLevelStateStorage. They are only responsible for maintaining the (one-level or two-level) array of Atomic<MapState>, and provide high-level functions for bulk-setting and bulk-transitioning map states.

Methods of MapState are moved to Mmapper. All bulk-setting operations can call mmap or mprotect on contiguous range of chunks instead of one chunk at a time, regardless whether using ByteMapStateStorage or TwoLevelStateStorage.

DRAFT: This is a very aggressive refactoring over the existing Mmapper framework. We need to

  • Fix Mmapper granularity: chunk or page? #1344 in this PR if not already addressed by another PR.
  • Make sure it works on both 32-bit and 64-bit machines.
  • Remove unused methods.
  • Rename variables, especially those that mention "mmapper", "fragmented", etc.
  • Performance evaluation, especially whether we need a specialized fast path for single-chunk modification, compared to the iterator-based "bulk transition" mechanism.
  • Find a way to allow dynamic selection of ByteMapMmapper and TwoLevelMmapper. This may involve a level of dyn somewhere. Make sure the performance is OK. It should be because it is supposed to be much cheaper than mmap and mprotect system calls.

wks added 2 commits July 20, 2025 01:53
We make Mmapper a struct instead of trait.  It will be solely
responsible for transitioning the map states.

Existing implementations, including `ByteMapMmapper` and
`TwoLevelMmapper`, become the storage backends of `Mmapper`.  They are
only responsible for maintaining the (one-level or two-level) array of
`Atomic<MapState>`, and provide high-level functions for bulk-setting
and bulk-transitioning map states.

Methods of `MapState` are moved to `Mmapper`.
They are now only responsible for the storage, not mmapping.
@wks
Copy link
Collaborator Author

wks commented Jul 19, 2025

If this refactoring is successful, it may shed some light on the old issue #587. It demonstrates a way to transform Java-style OOP with abstract classes and inheritance into Rust-style data structure composition, where former sub-classes become "backends" (ByteMapStateStorage and TwoLevelStateStorage) of the "main data type" (Mmapper) that performs the actual work. We may refactor other types in this way, such as

  • VMMap and the backends Map32 and Map64
  • FreeList and the backends IntArrayFreeList and RawMemoryFreeList
  • Space as a struct (with most fields from BaseSpace), and make each concrete space a "backend".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant