Memory leak

[panterlo]

Describe the bug
A clear and concise description of what the bug is.

A memory leak seems to be present.

using var session = _memgraphDriver.AsyncSession(conf => conf.WithDatabase("ABC"));
var cursor = await session.RunAsync(query);
var allRecords = await cursor.ToListAsync(); // Causes memory leak
await session.CloseAsync();

To Reproduce
Steps to reproduce the behavior:

1. Add the following code with a query that returns a few thousand nodes

using var session = _memgraphDriver.AsyncSession(conf => conf.WithDatabase("ABC"));
var cursor = await session.RunAsync(query);
var allRecords = await cursor.ToListAsync(); // Causes memory leak
await session.CloseAsync();
// Optional - doesn't not affect memory leak await session.DisposeAsync();
// Optional - doesn't not affect memory leak await _memgraphDriver.DisposeAsync();

2. Run the method several times inspecting the memory growth
   Memory will continue to grow for each method call

Expected behavior
A clear and concise description of what you expected to happen.

Memory will quickly grow as in our query example it grows with 400 MB per method call.

Version Info (please complete the following information):

• .NET Version: .NET Core 8.0
• .NET Driver Version 5.28.3 (also tested earlier versions down to 5.27.0)

Additional context
Add any other context about the problem here.

[RichardIrons-neo4j]

Hi @panterlo,

When you call ToListAsync() on a cursor, it consumes the entire result set, creating an IRecord for each record returned and storing them all in a list. Until this list is garbage collected, the memory allocated for all these records will not be freed. If you'd like to reduce memory usage, you could do something like this:

await using var session = _memgraphDriver.AsyncSession(conf => conf.WithDatabase("ABC"));
var cursor = await session.RunAsync(query);
await foreach(IRecord record in cursor)
{
    // do some processing with the record
}

In this way, only one record needs to be in memory at a time, and they will be garbage collected much more frequently. Let me know if you have any questions.

[panterlo]

Thanks, we have tried that and same memory leak exists with

IResultCursor cursor = await tx.RunAsync(query);
while (await cursor.FetchAsync() == true)
{
    IRecord current = cursor.Current;
}

Memory grows identically in both scenarios so something isn't properly disposed.

[panterlo]

I should have said that it doesn't matter - you can just run the code without touching the IRecord and notice how the memory grows for each method call. In the case above the result is significant so it increases roughly 400 MB per method call and after just 10 calls the memory have grown to over 4 GB.

[AndyHeap-NeoTech]

We'll re-open and see if we can reproduce. Is there any further information you can give us about what is being done with the records/list (if anything) after you have gotten them?

[panterlo]

Hi Andy,

The code only consists of these four lines, nothing more:

using var driver = GraphDatabase.Driver(...);
using var session = driver.AsyncSession(conf => conf.WithDatabase("ABC"));
var cursor = await session.RunAsync(query);
var allRecords = await cursor.ToListAsync();

Query returns a few thousand nodes and relationships, we tried a very small query and indeed it also increase memory every method call (but harder to notice).

[AndyHeap-NeoTech]

Hi Panterio,

I've spent some time digging into this and am unable to reproduce what you are seeing. Using the following code and connecting to a convenient neo4j db that we have online. Each query is returning 450MB+ of data (I'm unsure of the exact amount).

using Neo4j.Driver;

var driverInstance = GraphDatabase.Driver(new Uri("neo4j+s://demo.neo4jlabs.com"),
    AuthTokens.Basic("goodreads", "goodreads"));

var queryString = "MATCH(b:Book) RETURN b";

for (int i = 0; i < 5; i++)
{
    using var session = driverInstance.AsyncSession(conf => conf.WithDatabase("goodreads"));
    var cursor = await session.RunAsync(queryString);
    var allRecords = await cursor.ToListAsync();
}     

I am seeing the following memory usage:

You can see the garbage collection occurring as expected. I have also tried the above code using driver ExecutableQuery and ExecuteRead/Write interfaces with FetchAsync and ToListAsync and all produce the same memory graph.