Skip to content

Release v3.0.0 #357

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
amakropoulos merged 98 commits into from
Jan 12, 2026
Merged

Release v3.0.0 #357

amakropoulos merged 98 commits into from
Jan 12, 2026

Conversation

@amakropoulos
Copy link
Collaborator

@amakropoulos amakropoulos commented Aug 13, 2025 •
edited
Loading

Rewrite the LLM backend, LlamaLib, as a standalone C++/C# library and adaptation of LLMUnity.

Features:

  • implement LlamaLib as object-oriented C++/C# library
  • update llama.cpp to b7664
  • fix Vulkan GPU backend
  • Android 16kb support
  • fix iOS - xcode building
  • fix RAG functionality for iOS
  • polish samples
  • optimise streaming functionality and implement callbacks on the C++ end
  • remove chat templates from LLMUnity, use the llama.cpp templating
  • implement property checks
  • common handlng for both json and gbnf grammars
  • simplify integration of tinyBLAS (light GPU backend for Nvidia GPUs)
  • transition of client / server functionality in LlamaLib

Issues:

amakropoulos and others added 22 commits August 13, 2025 19:00
@amakropoulos
WIP
fdc53bf
@amakropoulos
Merge fdc53bf into 891497b
bda6dff
@amakropoulos
update VERSION
0f94f6f
@amakropoulos
working setup
728babc
@amakropoulos
fix platform paths on postprocess
ddc67c6
@amakropoulos
initial working LLM with LlamaLib
ff1afde
@amakropoulos
add NewtonSoft JSON dependency
00a76ec
@amakropoulos
add LlamaLib
bd1ec6d
@amakropoulos
implementation of LLMAgent methods (untested)
bc74831
@amakropoulos
allow empty templates
8118aa5
@amakropoulos
small fixes mainly on the ChatAsync
05caeea
@amakropoulos
use ChatAsync instead of Chat
23282d4
@amakropoulos
move completion parameters to LLMCaller
8762162
@amakropoulos
rename LLMCaller to LLMClient
42ae20f
@amakropoulos
simplify grammar by keeping the grammar value instead of the file
5c62cec
@amakropoulos
remove unused completion parameters
f778fb9
@amakropoulos
getters/setters for variables needing more actions
567ad3f
@amakropoulos
remove not needed classes
958f8b9
@amakropoulos
general improvements and small fixes
2614d20
@amakropoulos
more changes
18d1d1b
@amakropoulos
more improvements
8f7f869
@amakropoulos
check callback target for destruction
c715ccf
Draco18s
Draco18s suggested changes Sep 14, 2025
View reviewed changes
Copy link

@Draco18s Draco18s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Downloaded this to see if it solves any of the problems I've been having.

Some issues:

  • Sample chat bot scene has a missing script on the LLMCharacter object and the ChatBot monobehavior has no reference for its LLMAgent, so that will throw an NRE
  • LLMAgent.Warmup calls ChatAsync before the Lama llmAgent.llmAgent field is set (by SetupLLMClient), throwing a NRE, even if not using a Lama based model.
  • Using a coroutine to wait for the field to be non-null before calling Warmup just straight up crashes the Unity editor.
  • Ended up just waiting for the field to be non-null and calling WarmupCallback directly instead.
  • Whether or not the response is streamed has gone missing. Seems to always be in stream mode, chatbot demo doesn't populate the chat bubble, just constantly replaces with the next token.

@amakropoulos
show server architecture in the command
102eb53
@amakropoulos
update LlamaLib
b60de0e
@amakropoulos
move tests to Editor instead of Runtime, adapt tests for windows
5af2577
@amakropoulos
fix and polish samples
9f409e1
@amakropoulos
check for null before competion
fc3b214
@amakropoulos
auto-indentation
06da6c5
@amakropoulos
bring back the legacy input / event system
e865b10
@amakropoulos
adapt unit tests
615279e
@amakropoulos
implement auto-setup of EventSystem for both legacy and latest Unity …
11ecc69 
…versions
@amakropoulos
handle both input systems at the same time
fbf0aa9
@amakropoulos
remove secondar contentSizeFitter
004ff0f
@amakropoulos
show error if a LLM is used for embedding
ada88cf
@amakropoulos
update Windows unit tests
1937328
@amakropoulos
remove exit button from mobile demo
913a98d
@amakropoulos
use __Internal only in iOS Player
4cf6e69
@amakropoulos
fix iOS/visionOS plugin filenames
05c8b21
@amakropoulos
get processor type outside async
593c9b4
@amakropoulos
add search paths to xcode project
d576373
@amakropoulos
add cublas toggle in setup settings
25202be
@amakropoulos
add cublas toggle only in LLM
0ebe145
@amakropoulos
add logo
827961a
@amakropoulos
select (down)loaded model
09eb29b
@amakropoulos
add logo meta
e68192a
@amakropoulos
update images
ef2eca2
@amakropoulos
remove redundant variables
7abb5df
@amakropoulos
fix update tooltips script and update them
fc1d441
@amakropoulos
update readme
fd76b9c
@amakropoulos
guard VisionOS for Unity 2021 compatibility
e691523
@amakropoulos
add gemma-embedding to embedding architectures
9bde194
@amakropoulos amakropoulos merged commit 3d478a6 into main Jan 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@Draco18s Draco18s Draco18s requested changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

4 participants

@amakropoulos @Leuconoe @Draco18s