Skip to content

Commit da4251b

Browse files
committed
Add tip for native mode for Llama3
1 parent c10b478 commit da4251b

File tree

1 file changed

+11
-0
lines changed

1 file changed

+11
-0
lines changed

docs/modules/ROOT/pages/llama3.adoc

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,17 @@ WARNING: Models are huge, so make sure you have enough disk space.
3939
4040
NOTE: Due to model's large size, pulling them can take time
4141
42+
=== Native mode
43+
44+
Currently, Llama3.java only works in native mode with Early Access version's of Oracle GraalVM 24 (which can be easily downloaded with https://sdkman.io[SDKMan]).
45+
46+
To achieve the best performance in native mode, it is suggested to configure the application with the following:
47+
48+
[source,properties,subs=attributes+]
49+
----
50+
quarkus.native.additional-build-args=-O3,-march=native
51+
----
52+
4253
== Using Llama3.java
4354
4455
To let Llama3.java running inference on your models, add the following dependency into your project:

0 commit comments

Comments
 (0)