Unexpectedly high memory usage

[Artamus]

What are you trying to achieve?

I have a grpc-swift-2 binary against which I am running some load tests with k6. As the VUs (concurrent connections) ramp up, the memory usage balloons - at 1000 concurrent connections the program consumes ~1200M memory (according to htop).

I was able to reproduce this behaviour with the route-guide code example and the following k6 script:

import { Client, StatusOK } from "k6/net/grpc";
import { check, sleep } from "k6";

export const options = {
  vus: 1000,
  duration: "5m",
};

const client = new Client();
client.load(["Sources/Protos/routeguide"], "route_guide.proto");

export default () => {
  client.connect("127.0.0.1:31415", { reflect: false, plaintext: true });

  const data = { latitude: 407_838_351, longitude: -746_143_763 };
  const response = client.invoke("routeguide.RouteGuide/GetFeature", data);

  check(response, {
    "status is OK": (r) => r && r.status === StatusOK,
  });

  client.close();
};

What have you tried so far?

Not many ideas on what to try. Mostly compared to other solutions I have at the ready to sanity check whether this memory usage could be normal (but an equivalent Vapor program to the one I was originally testing was consuming only ~70 MiB of memory).

I'm running this on Ubuntu 22 on WSL2. I compiled the program in release mode.
Swift version 6.0.2 (swift-6.0.2-RELEASE)
Target: x86_64-unknown-linux-gnu

[gjcairo]

Hey @Artamus - thanks for filing this. We'll have to take a closer look before we can provide a meaningful update.

[Artamus]

FWIW I also managed to reproduce this on an M2 Macbook Pro, where at least according to htop the memory usage with the provided load testing script hit ~3GiB 😬.
swift --version output is

swift-driver version: 1.120.5 Apple Swift version 6.1.2 (swiftlang-6.1.2.1.2 clang-1700.0.13.5)
Target: arm64-apple-macosx15.0

[glbrntt]

Would you be able to re-run that on macOS under Instruments with the Allocations instrument and share the trace? That'd make it much easier to look into.

Mostly compared to other solutions I have at the ready to sanity check whether this memory usage could be normal (but an equivalent Vapor program to the one I was originally testing was consuming only ~70 MiB of memory).

Could you share what you used to test Vapor here as well? Just want to validate that we are comparing like for like.

[Artamus]

Would you be able to re-run that on macOS under Instruments with the Allocations instrument and share the trace? That'd make it much easier to look into.

The trace files are quite large, so I couldn't upload them inline. I've uploaded them to my Google Drive for now, but let me know if you'd like me to use some alternative platform.
Just do specify - this trace is for the RouteGuide gRPC test application.

Could you share what you used to test Vapor here as well? Just want to validate that we are comparing like for like.

Sure thing. The repo can be found here, with the vapor and grpc subprojects separate. They should be equivalent (I know the gRPC one's doing some extra database calls on purpose to match the fact that I used Fluent in the Vapor one).

[glbrntt]

The trace files are quite large, so I couldn't upload them inline. I've uploaded them to my Google Drive for now, but let me know if you'd like me to use some alternative platform. Just do specify - this trace is for the RouteGuide gRPC test application.

Thanks for providing that. As far as I can tell, peak memory use here is ~45 MiB. Was this where you were expecting 3GiB?

[Artamus]

Hmm, I ran it again with Instrumentation for a bit longer this time, but I don't really know how to read those traces, so I can't really cross-check what I'm seeing with htop easily. It also doesn't seem to do nearly as much req/s with Instrumentation on, so perhaps I need to get a longer trace? Perhaps it would help if I ran k6 for a while and then started gathering a trace?

[Artamus]

Here's another trace where I let it get higher than 100 MiB at least: https://drive.google.com/file/d/191wfaHlRZVvj-U48lubr0DLQGctRt0Nu/view?usp=sharing