Native container image memory settings

[matJC]

Hello!

I'm working on tuning a native container image, in particular around memory usage. I'm building these images via jib.

Am I right in understanding that I can only set memory settings like -XX:MaximumHeapSizePercent or -Xmx at build time using quarkus.jib.native-arguments? Is it the case that these settings must be baked into the native image? It sure would be helpful for more rapid tuning if we didn't have to rebake full native builds to try out settings.

Hopefully I'm just overlooking the right way.

Thanks all!

[quarkus-bot[bot]]

/cc @geoand (jib), @zakkak (native-image)

[zakkak]

Am I right in understanding that I can only set memory settings like -XX:MaximumHeapSizePercent or -Xmx at build time using quarkus.jib.native-arguments?

Yes, that's my understanding as well although I am not that familiar with native container image builds.

Is it the case that these settings must be baked into the native image?

No, they just need to be baked in the native container image.

It sure would be helpful for more rapid tuning if we didn't have to rebake full native builds to try out settings.

Yes, that makes sense. I am afraid it's not possible at the moment though.

[matJC]

Thanks for responding @zakkak 👍

No, they just need to be baked in the native container image.

The terminology is so close, I had a double take on this one, but I see the distinction between the native image and the native container image. Graal's docs indeed make mention of which settings are available at run time w.r.t. the native image (I would probably have called this the native binary to avoid the terminology collision with container images, but I digress..).

Yes, that makes sense. I am afraid it's not possible at the moment though.

Call this a feature request 😃
I'll start baking these settings into the native container image and will report back on my level of success.

Thanks for your great work on this project. I've been digging into GraalVM from a few vectors at this point and can appreciate the challenge you and your team have embarked on here. Best of luck with it 🙌

[zakkak]

The terminology is so close, I had a double take on this one

Yes that's frustrating, I copied the "native container image" term from your question in order to avoid this and I still failed :)

I'll start baking these settings into the native container image and will report back on my level of success.

To my understanding you are already baking them in. For your experiments and tuning please note that you can still test different options by passing flags to the docker run command without having to rebuild anything. e.g.:

no options passed to podman run (native container image compiled with -Dquarkus.jib.native-arguments=-Xmx1g)

❯ podman run --rm localhost/zakkak/cli:1.0.0-SNAPSHOT 
__  ____  __  _____   ___  __ ____  ______ 
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2023-11-30 15:49:12,895 INFO  [io.quarkus] (main) cli 1.0.0-SNAPSHOT native (powered by Quarkus 3.6.0) started in 0.003s. 
2023-11-30 15:49:12,895 INFO  [io.quarkus] (main) Profile prod activated. 
2023-11-30 15:49:12,895 INFO  [io.quarkus] (main) Installed features: [cdi, picocli, smallrye-context-propagation]
Hello picocli, go go commando!
max Memory = 1 GB
2023-11-30 15:49:12,896 INFO  [io.quarkus] (main) cli stopped in 0.000s

-Xmx10g passed to podman run (native container image compiled with -Dquarkus.jib.native-arguments=-Xmx1g)

__  ____  __  _____   ___  __ ____  ______ 
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2023-11-30 15:49:17,741 INFO  [io.quarkus] (main) cli 1.0.0-SNAPSHOT native (powered by Quarkus 3.6.0) started in 0.003s. 
2023-11-30 15:49:17,741 INFO  [io.quarkus] (main) Profile prod activated. 
2023-11-30 15:49:17,741 INFO  [io.quarkus] (main) Installed features: [cdi, picocli, smallrye-context-propagation]
Hello picocli, go go commando!
max Memory = 10 GB
2023-11-30 15:49:17,741 INFO  [io.quarkus] (main) cli stopped in 0.000s

Thanks for your great work on this project. I've been digging into GraalVM from a few vectors at this point and can appreciate the challenge you and your team have embarked on here. Best of luck with it 🙌

Thank you for your kind words.

[matJC]

Ok great -- this is effectively what I was looking for. Thanks!

It took me a bit to realize how you were passing the flags. I was originally trying to do it via env vars, but the method is to pass container entrypoint args like -Xmx512m, etc. 👍

[zakkak]

Uh, yes I could have been more verbose on that. For future reference, typically when deploying a "native container image" we just do podman run my-native-container-image (you can use docker where I use podman), this will essentially use the entrypoint ./application plus any flags we set through quarkus.jib.native-arguments. So for the example above it would be:

$ podman inspect --format='{{.Config.Entrypoint}}' localhost/zakkak/cli:1.0.0-SNAPSHOT
[./application -Xmx1g]

Now if instead of podman run my-native-container-image we run podman run my-native-container-image -Xmx10g this will result in ./application -Xmx2g -Xmx10g to be run. Since mx can only have one value, when passing it multiple times the application (just like the JVM) will only store the latest value, which how you are able to "override" it and do experiments. Unfortunately this is not possible for all flags, but in most cases it should suffice.