Operating memory research

[RaynaldM]

Before release 23.0.0 we regularly had a few Out Of Memory, which were very difficult to track down. We had found 2 probable causes.
It's not much, but in a high-traffic context, small streams make big rivers.
But the hunt is not finish...

Update by Maintainer on February 19

As a team, we will endlessly discuss any memory issues, problems, and practices related to memory consumption.
Since .NET was designed to address memory consumption & management issues by Garbage Collector, I would say that in this topic we will be looking for potential design issues when copying, forwarding, scanning, computing large blocks of request/response data.
So expect big problems just with the design of features.

[raman-m]

For more lucky hunting I would say you could add more logger points...
I agree OOM problems cannot be detected easily now because Ocelot has no memory consuming monitor, indicators and memory events...

[raman-m]

@ggnaegi and @RaynaldM
Any ideas on how to monitor such cases and memory management?
Seems we need definitely to design some lite memory controller/monitor...

[ggnaegi]

@ggnaegi and @RaynaldM
Any ideas on how to monitor such cases and memory management?
Seems we need definitely to design some lite memory controller/monitor...

Metrics?

[raman-m]

Does our lovely Bla-bla gateway have some metrics/indicators or monitoring features?
We could start from general metrics for memory consumption, but I'm afraid that will be not easy... Seems it requires a refactoring of all Ocelot core... 🤔
I believe for the first time we can read consumed memory on app domain level... That should be enough...
More precise metrics/indicators require big refactoring road/milestone...

[RaynaldM]

Our experience with metrics is currently limited to measuring response times, the number of active requests (requests but not yet responses), the number of requests per second and error counts.
As far as OOMs are concerned, we see them in the logs, but with a slight delay, and it's quite difficult to trace the thread of events that generated them. In many cases, an OOM is triggered on a method, but it is merely the victim of a leak elsewhere.
And to top it all off, OOMs happen quite randomly.
I think it's better to concentrate on improving Ocelot gently and punctually, whenever we recognize a problematic piece of code (and there really are lots of places where we can improve it).

[RaynaldM]

As you can see, memory consumption varies greatly from pod to pod, due to the diversity of operations to be carried out.

It varies from 150 MB to 1.2 GB.

[ks1990cn]

@RaynaldM Intresting, how you are measuring these graphs? Is it something on Production with azure/aws, which tool it is?

What about just finding through visual studio profilers, is it effective enough (Cause it will not be able to replicate high traffic cases)?

Just gone through statement and trying to understand, what approach you are following.

https://learn.microsoft.com/en-us/visualstudio/profiling/profiling-feature-tour?view=vs-2022

[ks1990cn]

Here we can see from 50s to 1:40min we got 2 times allocations, one before GC and after GC. This is done with 100 Virtual Users load for 5 mins.
We can through VS revisit our codes where it makes more allocations?

[ks1990cn]

.Net 8 have new feature for realtime GC monitoring, .Net Aspire.

https://learn.microsoft.com/en-us/dotnet/aspire/get-started/aspire-overview
https://www.youtube.com/watch?v=DORZA_S7f9w
https://github.com/dotnet/eShop

Can we give user option to integrate?

[raman-m]

The quote:

.NET Aspire is an opinionated, cloud ready stack for building observable, production ready, distributed applications.

@ks1990cn
Does it mean this technology (GC monitoring) is available in "clouds", probably in Azure environment only?

Can we give user option to integrate?

I'm not sure on integration. I have to ask the team... This is a general idea to discuss, and nothing more now.

@RaynaldM @ggnaegi
What about .NET Aspire ?
To be fair, I've heard about it for the 1st time. My concern is that .NET Aspire is

...delivered through a collection of NuGet packages that handle specific cloud-native concerns.

So, this is extra .NET packages, probably not a part of .NET SDK 8...

[ks1990cn]

https://github.com/dotnet/eShop

This is an example. Idea - they want in future for aws too, currently its supporting azure

[raman-m]

How can I monitor memory consumption without any clouds?
Is it possible to monitor self-hosted asp.net app in console & Kestrel?
🆗 I will review https://github.com/dotnet/eShop repository later to have better understanding.

[ggnaegi]

@raman-m @RaynaldM We should try to improve the overall application performance, but the last changes had a big impact on OOMs. I think this issue isn't specific enough, maybe we should convert it to a discussion?

[raman-m]

Converted!