Skip to content

Speculative partial fix for #4727 #1291

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
copybara-service wants to merge 959 commits into from
Closed

Speculative partial fix for #4727 #1291

copybara-service wants to merge 959 commits into from

Conversation

@copybara-service
Copy link
Contributor

@copybara-service copybara-service bot commented Jan 27, 2026

Speculative partial fix for #4727

We need to pick up commit bazelbuild/apple_support@44c43c7 in every dependency of LiteRT LM.

gcarranza-1 and others added 30 commits November 7, 2025 13:49
@gcarranza-1 @copybara-github
Remove usages of deprecated GpuOptions function overloads.
04b0671 
LiteRT-LM-PiperOrigin-RevId: 829563774
@ai-edge-bot @copybara-github
Introduce TaskState::kMaxNumTokensReached to signal exceeding token…
c0780bf 
… limits.

LiteRT-LM-PiperOrigin-RevId: 829572476
@whhone @copybara-github
[litertlm] add example/Main.kt to build and run Kotlin API with Bazel
9ace482 
1. provide a real example of using the Kotlin API
1. make it very simple to try out the Kotlin API from source

**To run it with bazel**:
```
bazel run -c opt //kotlin/java/com/google/ai/edge/litertlm/example:main -- <abs_model_path>
```

**To build a standalone binary and run**:
```
bazel build -c opt //kotlin/java/com/google/ai/edge/litertlm/example:main_deploy.jar

./bazel-bin/kotlin/java/com/google/ai/edge/litertlm/example/main_deploy.jar -- <abs_model_apth>
```

LiteRT-LM-PiperOrigin-RevId: 829573268
@hheydary @copybara-github
Create fake model for testing.
e766268 
LiteRT-LM-PiperOrigin-RevId: 829591297
@gcarranza-1 @copybara-github
Remove unused litert_qualcomm_options C header.
108d41e 
LiteRT-LM-PiperOrigin-RevId: 829596495
@ai-edge-bot @copybara-github
Refactor prefill, decode and scoring logic into a new tasks.cc.
1424258 
LiteRT-LM-PiperOrigin-RevId: 829609532
@ai-edge-bot @copybara-github
Refactor: Pass litert::TensorBuffer by value in pipeline and tasks.
2f4f7a7 
LiteRT-LM-PiperOrigin-RevId: 829625321
@matthewchan-g @copybara-github
Improve tool response formatting in Gemma3DataProcessor.
daf2d40 
LiteRT-LM-PiperOrigin-RevId: 829635897
@whhone @copybara-github
[litertlm] minor fix to example/BUILD
c96af67 
1. add jvm_flags to get rid of the warning message
2. fix typo in bazel command

LiteRT-LM-PiperOrigin-RevId: 829699946
@whhone @copybara-github
[litertlm] simplify sendMessage response handling
392b4ab 
Printing the message as string is a very common use case. Make it super simple.

If there is image / audio output in the future, then the toString() of them is something like: `Content(AudioFile("/path/to/file"))`

LiteRT-LM-PiperOrigin-RevId: 829877174
@whhone @copybara-github
[litertlm] minor improvement for creating Message inline
8dd50f5 
Extends the `Message.of(content)` to `Message.of(content1, content2, content3, ...)`. It still works with only one content.

LiteRT-LM-PiperOrigin-RevId: 830494523
@tf-marissaw @copybara-github
Do not prefer textures on Apple devices
296e5c4 
Apple devices perform better with Buffers.

LiteRT-LM-PiperOrigin-RevId: 830924175
@whhone @copybara-github
[litertlm] Windows JVM build
fcd8640 
Add `__declspec(dllexport)` when building for Windows so the .dll has the exported symbols. https://learn.microsoft.com/en-us/cpp/cpp/using-dllimport-and-dllexport-in-cpp-classes

LiteRT-LM-PiperOrigin-RevId: 830991930
@whhone @copybara-github
[litertlm] Add sendMessageAsync that gets response as Flow
925b917 
See Main.kt :-)

This `Flow` approach is also faster than the callback approach because it does not wait for the `onMessage()` logic.

LiteRT-LM-PiperOrigin-RevId: 831131771
@tf-marissaw @copybara-github
Update dependencies of litert_lm
8d685d3 
LiteRT-LM-PiperOrigin-RevId: 831181727
@terryheo @copybara-github
Update dependencies of litert_lm
4cdbc62 
LiteRT-LM-PiperOrigin-RevId: 831194431
@terryheo @copybara-github
Update litert_lm with new way of setting GpuOptions
33cae51 
Used Options::GetGpuOptions().
The new Create() method takes non-const reference to Options object.

LiteRT-LM-PiperOrigin-RevId: 831216121
@ai-edge-bot @copybara-github
internal change and clean up.
baebde2 
LiteRT-LM-PiperOrigin-RevId: 831627364
@ai-edge-bot @copybara-github
Internal change and clean up.
efd0c0d 
LiteRT-LM-PiperOrigin-RevId: 831637934
@ai-edge-bot @copybara-github
Snap the litert dependency to the latest
824d223 
LiteRT-LM-PiperOrigin-RevId: 831948857
@terryheo @copybara-github
litert_lm: Apply recent LiteRt CompiledModel::Create() API change
75111c2 
The new API doesn't require Model but create it internally with model_filename
or model_buffer.
Also users can get Signature from CompiledModel directly.

LiteRT-LM-PiperOrigin-RevId: 832038920
@whhone @copybara-github
[litertlm] set the log level to "at least error" in JNI
0d85d9a 
LiteRT-LM-PiperOrigin-RevId: 832039059
@gcarranza-1 @copybara-github
Remove LiteRT LM usages of LITERT_HOST_MEMORY_BUFFER_ALIGNMENT.
7df3610 
LiteRT-LM-PiperOrigin-RevId: 832310477
@ai-edge-bot @copybara-github
Add vision/audio backend support in engine settings creation, and add…
4aeacba 
… cache directory settings to LiteRT-LM C

LiteRT-LM-PiperOrigin-RevId: 832381402
@ai-edge-bot @copybara-github
Add ExecutionManager for managing LLM inference tasks.
8438a60 
LiteRT-LM-PiperOrigin-RevId: 832406710
@ai-edge-bot @copybara-github
Move tensor buffer creation for scoring into the worker thread.
46650bb 
LiteRT-LM-PiperOrigin-RevId: 832460497
@yishuangP @copybara-github
Implement CreateConstraint in Gemma3DataProcessor. Currently the me…
3073b2b 
…thod returns null.

LiteRT-LM-PiperOrigin-RevId: 832500201
@gcarranza-1 @copybara-github
Refactor LiteRT LM usages of TensorBuffer creation to use CreateManag…
53e7415 
…edHostMemory when environment is not needed.

LiteRT-LM-PiperOrigin-RevId: 832519936
@whhone @copybara-github
[litertlm] update Kotlin API doc
a328771 
- Add sample code for a chat app and animation demo. And a Bazel command to quickly try it.
- Mention JVM support, in additional to the original Android support.

LiteRT-LM-PiperOrigin-RevId: 832701364
@ai-edge-bot @copybara-github
Internal code change.
66adc81 
LiteRT-LM-PiperOrigin-RevId: 833396367
hheydary and others added 20 commits January 22, 2026 09:24
@hheydary @copybara-github
Fix dangling reference issue.
7bf2787 
LiteRT-LM-PiperOrigin-RevId: 859642028
@protobird-git @copybara-github
Don't clear KV cache before prefill if requested.
4e6fc63 
When context length is big, it increase init time noticeably, e.g. 8s when context length = 32k for gemma3-1b.

LiteRT-LM-PiperOrigin-RevId: 859666933
@protobird-git @copybara-github
Add a flag to en/disable sampler to handle decode input tensors
b9aea00 
LiteRT-LM-PiperOrigin-RevId: 859770885
@whhone @copybara-github
Internal change
dd3d5e4 
LiteRT-LM-PiperOrigin-RevId: 859859182
@whhone @copybara-github
[litertlm] Add a Kotlin example of tool calling
b01f42e 
LiteRT-LM-PiperOrigin-RevId: 859875851
@ai-edge-bot @copybara-github
Internal change
7042b19 
LiteRT-LM-PiperOrigin-RevId: 859999259
@ai-edge-bot @copybara-github
Internal changes and clean up.
50906b5 
LiteRT-LM-PiperOrigin-RevId: 860120569
@protobird-git @copybara-github
Disable gpu_madvise_original_shared_tensors when magic numbers are used.
150f974 
The given option is true by default and used to set the mmap'ed memory for
shared weights are swapped out to reduce memory footprint.
When memory is swapped out, all the temporary changes made by magic numbers
are reverted. So, when magic numbers are used, the give flags must be disabled.

LiteRT-LM-PiperOrigin-RevId: 860150634
@advaitjain @copybara-github
Change default backend to CPU in litert_lm_advanced_main.cc.
7cfdd0e 
LiteRT-LM-PiperOrigin-RevId: 860170788
@matthewchan-g @copybara-github
Implement LlgConstraintProvider and ExternalConstraintProvider.
2847d3b 
LiteRT-LM-PiperOrigin-RevId: 860216677
@ai-edge-bot @copybara-github
Introduce LlmContext, configs and state definitions.
4ed3ca8 
LiteRT-LM-PiperOrigin-RevId: 860278850
@whhone @copybara-github
[litertlm] Update the instruction of setting hugging face token
cef9a49 
We will remove the `--hk_token`. The environment is the only way to set the token.

LiteRT-LM-PiperOrigin-RevId: 860319701
@ai-edge-bot @copybara-github
Encapsulate executor state within LlmContext.
e20eab4 
LiteRT-LM-PiperOrigin-RevId: 860320494
@protobird-git @copybara-github
Add force_f32 option in go & C api for better quality on gpu backend
3eecf9b 
LiteRT-LM-PiperOrigin-RevId: 860635677
@protobird-git @copybara-github
Update dependencies of litert_lm
83c6992 
- Add TensorBuffer Clear method
- Replace @local_xla with @xla

LiteRT-LM-PiperOrigin-RevId: 860680613
@protobird-git @copybara-github
Use TensorBuffer::Clear() instead of ZeroTensorBuffer()
d1d4d86 
LiteRT-LM-PiperOrigin-RevId: 861221697
@fengwuyao @copybara-github
Reverts 374f764
dae6a75 
LiteRT-LM-PiperOrigin-RevId: 861230859
@tf-marissaw @copybara-github
Add Program Cache fd support to LiteRT LM
ed7a8f7 
This is similar to the existing weight cache support that
XNNPack uses.

LiteRT-LM-PiperOrigin-RevId: 861245691
@matthewchan-g @copybara-github
Add constrained decoding to Conversation API, enabling support for ex…
d20033a 
…ternal constraints and LLGuidance.

LiteRT-LM-PiperOrigin-RevId: 861255750
@yichunk @copybara-github
Add support_single_turn capability to prompt_template, which is ena…
f794979 
…bled by checking whether there is corresponding `is_appending_to_prefill` in the jinja template. Templates with such capability can support multi-prefill.

LiteRT-LM-PiperOrigin-RevId: 861409419
@copybara-service copybara-service bot force-pushed the litert_lm_pr_861425587 branch from fccfb75 to c9066b1 Compare January 27, 2026 01:48
ai-edge-bot and others added 5 commits January 26, 2026 18:46
@ai-edge-bot @copybara-github
Log the input.
1be73b5 
LiteRT-LM-PiperOrigin-RevId: 861449983
@ai-edge-bot @copybara-github
Internal change
f1f1a9e 
LiteRT-LM-PiperOrigin-RevId: 861460877
@protobird-git
Use LFS (Large File Storage) to prebuilt binaries (#1294)
986919a 
* Use LFS (Large File Storage) to prebuilt binaries

* Enable LFS in workflows
@protobird-git @copybara-github
Sync with github to support LFS (Large File Storage) for prebuilt bin…
014c177 
…aries

LiteRT-LM-PiperOrigin-RevId: 861722316
@dsharletg @copybara-github
Speculative partial fix for #4727
a1fd171 
We need to pick up commit bazelbuild/apple_support@44c43c7 in every dependency of LiteRT LM.

LiteRT-LM-PiperOrigin-RevId: 861425587
@copybara-service copybara-service bot force-pushed the litert_lm_pr_861425587 branch from c9066b1 to a1fd171 Compare January 27, 2026 18:41
@protobird-git protobird-git deleted the litert_lm_pr_861425587 branch February 1, 2026 03:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.