Skip to content

alloc-ng.org

Latest commit

 

History

History
64 lines (62 loc) · 3.04 KB

alloc-ng.org

File metadata and controls

64 lines (62 loc) · 3.04 KB

The alloc-ng branch of the git repository contains the prototype for a next-generation allocator API. This new API is lower-level than the existing API, but the design is considerably more flexible and offers many potential performance improvements.

Goals

See file:../src/yu_alloc.h

Fast, 64-bit optimized, virtual-memory-aware allocator

Explicit control over virtual memory pages

Embrace over-reserving addresses

64-bit address space is almost unlimited. Reserving the entire object heap of addresses is feasible if we don’t have to commit it.

Minimize allocations

Have an explicit API to get the real usable size of an allocation. Care must be taken not to unnecessarily expose implementation details. However, this could avoid unnecessary calls to allocator functions in a number of circumstances

Progress

Spec out allocator API

See file:alloc-ng-specv2.org

[#B] Platform abstraction layer for virtual memory

See file:../src/platform.h
  • [X] POSIX (mmap+posix_madvise) implementation
  • [X] Linux (mmap+madvise) implementation Linux has a little more flexibility than the pure POSIX API.
  • [X] Win32 (VirtualAlloc) implementation Untested

[#A] Test suite for new features

  • [X] Allocated/usable size functions
  • [X] Reserving/releasing virtual memory
  • [X] Committing/decommitting virtual memory
  • [X] Both at once
  • [X] Reserving roughly at a fixed address
  • [X] Reserving exactly at a fixed address
  • [X] Context management of pages

[#A] Provide a wrapper for system malloc()

Can use system malloc() to implement bookkeeping for managing pages.

[#C] Provide a debug allocator

Probably based on dmalloc. Requires implementing ‘extras’, especially since dmalloc does not support aligned_alloc. The existing debug_alloc may be of use. However, a true debug_alloc will have to track page-level allocations as well. Shimming this on top of dmalloc may not work, since subsystems expect to be able to reserve large address spaces. Might be OK on Linux with its lazy committing policy.

[#C] Provide a higher-performing allocator wrapping jemalloc APIs directly

jemalloc provides more control like aligned realloc and getting usable size. This does require using experimental jemalloc APIs, but should be much more efficient than shimming the functionality.

[#A] Update existing subsystems to new allocator API

Changes in most cases should be relatively minimal.
  • [X] Test framework (test/test.h)
  • [X] yu_buf, yu_str
  • [X] test_alloc
  • [X] Object arenas
  • [X] Generic data structures

[#B] Update existing subsystems to make efficient use of new features

Arenas in particular can make efficient use of reserving huge chunks of addresses without actually committing. Part of the reason for the whole redesign is to improve garbage collector performance.