Allocating/using shared memory

[jakirkham]

It would be very useful to have a way to allocate typed memory blocks in shared memory (either CPU or GPU). This is helpful in cases where multiple processes are used for parallelism and need fast access to read from (and possibly write to) memory shared across processes and when the memory usage is large to keep from going over the memory limit.

Some use cases for shared memory:

• https://pytorch.org/docs/stable/notes/multiprocessing.html#multiprocessing-best-practices
• https://arrow.apache.org/blog/2017/08/08/plasma-in-memory-object-store/
• https://mxnet.incubator.apache.org/architecture/note_memory.html

Some strategies for allocating shared memory:

• http://pubs.opengroup.org/onlinepubs/009695399/functions/shmget.html
• http://pubs.opengroup.org/onlinepubs/009604599/functions/shm_open.html
• https://docs.microsoft.com/en-us/windows/desktop/memory/creating-named-shared-memory
• https://devblogs.nvidia.com/using-shared-memory-cuda-cc/