How to improve shared memory utilization in dynamic mode?

[AmazingPP]

When using Iceoryx2's dynamic data mode, how can overall shared memory utilization be improved? In scenarios where subscriber-max-buffer-size is large and the payload size varies greatly, it's easy to encounter memory fragmentation or inefficient allocation. Are there any recommended strategies or best practices for optimization?

I would like to know:

1. Are there any parameters or strategies that can efficiently handle large payload size variations?
2. Is there a mechanism for dynamically reclaiming, reusing, or compressing memory segments?
3. Beyond allocation strategy, are there any practical experiences you can share?

Looking forward to everyone's suggestions and experience sharing!

[elfenpiff]

@AmazingPP

it's easy to encounter memory fragmentation or inefficient allocation

iceoryx2 pre-allocates memory, even in the fully dynamic case. In this case, it starts with the hinted size and if this does not suffice it pre-allocates a larger segment. Every segment is handled by a pool allocator with the same bucket size, so memory fragmentation and inefficient allocation (performance-wise) should never be a problem.

Are there any parameters or strategies that can efficiently handle large payload size variations?

What do you mean here? That you send a sample that has a size of 1GB and then a sample of 1 Byte, so you have huge size differences? If so I would recommend to split it up into 2 services if possible. Iceoryx2 always preallocates the worst case size, so if you have rarely 1GB samples and often only 1 byte samples then you might waste memory - on the other hand, with this strategy iceoryx2 can guarantee you that it never runs out of memory. And since we are also aiming for mission-critical systems, it is an important use case.

But on our roadmap, we also have a buddy allocator, which would be much more memory efficient, but it would be significantly slower, introduces fragmentation issues, and your system might get into a state where there is no more communication via this service possible since there is no more memory available.
FYI: If you are interested in this feature it would require funding.

Is there a mechanism for dynamically reclaiming, reusing, or compressing memory segments?

No, but you could serialize and compress your payload.

Beyond allocation strategy, are there any practical experiences you can share?

It would be helpful if you would be able to share your use-case/problem or the project you are working on and then we can provide you with some practical hands on tips or best practices.

[AmazingPP]

Hi @elfenpiff , thanks a lot for your detailed reply! 🙏

I think I misunderstood some parts of the memory model in iceoryx2. I originally thought that in dynamic mode, memory chunks were allocated and freed more like a heap allocator, which might lead to fragmentation or inefficient reuse. Now I understand from your explanation that each segment is preallocated and managed by a pool allocator with fixed bucket sizes, so fragmentation is not actually an issue. That clarifies a lot!

For context, my use case is in network security monitoring (NSM): I need to transmit protocol metadata such as HTTP response bodies, where payload sizes vary a lot (most small, but occasionally very large).

As you suggested, splitting into multiple services makes sense. My real question is: 👉 when payload size distribution is highly skewed, are there recommended strategies (besides splitting services) to balance memory efficiency and reliability?

Thanks again for your detailed reply — it helps me align my use case with iceoryx2’s model.

[elfenpiff]

@AmazingPP always happy to support the community :) !

As you suggested, splitting into multiple services makes sense. My real question is: 👉 when payload size distribution is highly skewed, are there recommended strategies (besides splitting services) to balance memory efficiency and reliability?

Here I would recommend using a configurable buddy allocator as the underlying allocator. As I mentioned before, it is planned, but we are still looking for funding, so I cannot provide you with a due date here. This could be much more memory efficient, and as a fallback, one could always try to allocate a larger data segment when the buddy allocator is fragmented or the optimistic low memory size estimations turn out to be wrong.
But the big downside of such an approach is that, in theory, it could lead to a system that runs out of communication memory. Depending on the system, it could be a theoretical problem. Still, then you would also require the proper tooling to monitor the average and worst-case memory consumption in your system to get a feeling for it.

We are currently implementing that kind of tooling, which we want to release at the end of September. With it, you are able to take a look at the communication graph, see the full resource (CPU, Memory, IO) load of all iceoryx2 endpoints, and based on that, you could start such an optimization.

[AmazingPP]

Thanks for the update! I'm glad to hear the buddy allocator is in the works — it sounds promising for improving memory efficiency, though I understand the potential risks of fragmentation and running out of memory.

Looking forward to the new tooling at the end of September to help monitor memory usage and optimize further. I’ll keep monitoring the system in the meantime.

Thanks again for the info!