Skip to content

[swift] add module omnivlm #11171

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 80 commits into from
Closed

[swift] add module omnivlm #11171

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
5f81588
support ggml
zhiyuan8 Sep 10, 2024
3a35526
update README after renaming GGML
zhiyuan8 Sep 10, 2024
4a29bca
update vulkan target name
zhiyuan8 Oct 23, 2024
c7b912b
support omni-audio
zhiyuan8 Nov 3, 2024
f0d1c4f
enable qwen2-audio work E2E
zhiyuan8 Nov 3, 2024
9e67ef7
remove uneccesary build and rename shared lib
zhiyuan8 Nov 3, 2024
4bdc70a
update to C++17 for compilation
zhiyuan8 Nov 3, 2024
d277c67
add omni-vlm examples (C++ & python)
Nov 4, 2024
995baef
Disable cxxabi.h dependency on Windows
zhiyuan8 Nov 4, 2024
a4747b2
fix error on windows qwen2-audio/whisper.cpp:9935:38: error: '_O_BINA…
zhiyuan8 Nov 4, 2024
6f1ed6e
Adding #include <io.h> & <fcntl.h>
zhiyuan8 Nov 4, 2024
1419681
disable <cxxabi.h> for MSC_VER
zhiyuan8 Nov 4, 2024
d42e037
remove C++20 style
zhiyuan8 Nov 4, 2024
05853eb
remove C++20 syntax
zhiyuan8 Nov 4, 2024
91b3caf
Merge pull request #6 from NexaAI/master-release-audio-lm
zhiyuan8 Nov 5, 2024
d6c0627
Merge pull request #7 from NexaAI/master
zhiyuan8 Nov 5, 2024
983b462
Merge pull request #8 from NexaAI/weili/master-release
zhiyuan8 Nov 5, 2024
b535cd9
Merge pull request #9 from NexaAI/master
zhiyuan8 Nov 5, 2024
38c6fa3
enable lib to be exported in nexa SDK
zhiyuan8 Nov 5, 2024
22da7bc
add returned string (pure c const char* type) for omni-vlm inference api
Nov 6, 2024
5574bda
Merge pull request #10 from NexaAI/weili/master-release
zhiyuan8 Nov 6, 2024
b24a409
add returned string (const char*) for qwen2 audio
Nov 6, 2024
6a4cf0b
Merge pull request #11 from NexaAI/weili/master-release
zhiyuan8 Nov 6, 2024
5edadff
add returned string type (const char*) for nexa-omni-audio
Nov 7, 2024
20b9f02
Merge pull request #12 from NexaAI/weili/master-release
zhiyuan8 Nov 7, 2024
3dfac78
add returned string type (const char*) for nexa-omni-audio
Nov 7, 2024
df5841b
Merge pull request #13 from NexaAI/weili/master-release
zhiyuan8 Nov 7, 2024
86c2233
add submodule llava for android
Nov 8, 2024
400fc2a
add one more model
Nov 8, 2024
b17684e
add include llava.h
Nov 8, 2024
16c2247
remove redundant omni-vlm-v2/ folder, all omni-vlm examples will be a…
Nov 8, 2024
3d9c63a
remove omni-vlm-v2/
Nov 8, 2024
eb6d546
update README.md
Nov 8, 2024
8c41728
Merge pull request #15 from NexaAI/weili/master-release
zhiyuan8 Nov 8, 2024
d5df536
Merge pull request #14 from NexaAI/teliu/android/dev
zhiyuan8 Nov 8, 2024
ecfe0b4
changed download models and nlen
zhycheng614 Nov 8, 2024
667a6d9
Merge pull request #16 from NexaAI/perry/android-dev
zhycheng614 Nov 8, 2024
d04e354
fix OCR template error.
Nov 9, 2024
21bc833
Merge pull request #17 from NexaAI/weili/master-release
zhiyuan8 Nov 9, 2024
6f0e8c3
Create CODEOWNERS
zhiyuan8 Nov 10, 2024
7cf07df
reset model in every inerence step to avoid nosense output.
Nov 11, 2024
362bdf3
Merge pull request #18 from NexaAI/weili/master-release
zhiyuan8 Nov 11, 2024
5f2d958
Merge pull request #19 from NexaAI/master
zhiyuan8 Nov 11, 2024
55953d3
[omni-vlm] fixed the segmentation fault issue in nano-vlm-instruct(WIP,
Nov 12, 2024
82dbdbd
Merge pull request #20 from NexaAI/weili/master-release
zhiyuan8 Nov 12, 2024
89bcf5a
Merge pull request #21 from NexaAI/master
zhiyuan8 Nov 12, 2024
4e80184
fix vulkan build bug for external build
Davidqian123 Nov 12, 2024
98297af
Merge pull request #22 from NexaAI/david/vulkan2
zhiyuan8 Nov 12, 2024
bb33473
Merge pull request #23 from NexaAI/david/vulkan2
zhiyuan8 Nov 12, 2024
fc25544
[memory leakage] fixed a leakage by projector free
Nov 14, 2024
b9845b4
Merge pull request #24 from NexaAI/weili/master-release
zhiyuan8 Nov 14, 2024
aad0167
audio embedding free() (but still memory leakage detected)
Nov 14, 2024
8e2e630
fix mem leakage based on leaks tool (still WIP)
Nov 14, 2024
e4ca946
free omni_ctx heap malloc space in omni_free() api
Nov 15, 2024
25190fe
Merge pull request #25 from NexaAI/weili/master-release
zhiyuan8 Nov 15, 2024
fe792d6
Merge pull request #26 from NexaAI/master
zhiyuan8 Nov 15, 2024
fd2c582
remove reference interface from extern C in qwen2audio examples
Nov 21, 2024
7589158
expose omni_context_params struct
Nov 21, 2024
bbf1aaa
Merge remote-tracking branch 'origin' into zack/vlm
zhiyuan8 Nov 22, 2024
460212a
change template for inference
zhiyuan8 Nov 22, 2024
fe8c7b4
revert CMakeList
zhiyuan8 Nov 22, 2024
43f41a4
Merge pull request #28 from NexaAI/zack/vlm
zhiyuan8 Nov 22, 2024
3479f51
udpate prompt template in wrapper
Davidqian123 Nov 22, 2024
809db95
ugrade to llama.cpp 74d73dc
TerryT9 Dec 2, 2024
a2c5305
merge from master
TerryT9 Dec 2, 2024
661b3f7
Merge pull request #31 from NexaAI/teliu/dev
zhiyuan8 Dec 2, 2024
0b15d2d
fix conficts (#32)
TerryT9 Dec 2, 2024
71b563e
Merge branch 'weili/dev' of github.com:NexaAI/llama.cpp into weili/dev
Dec 3, 2024
97267e6
bug fix in common-nexa.cpp
liwiii Dec 3, 2024
be54cb0
bug fix
Dec 3, 2024
ca7e8ef
fix clip_n_patch() allocation size error for 81-series omni-vlm models
Dec 3, 2024
07c7ff3
Merge branch 'weili/dev' of github.com:NexaAI/llama.cpp into weili/dev
Dec 3, 2024
b86cded
remove iostream header
Dec 3, 2024
b2958b3
Merge pull request #33 from NexaAI/weili/dev
zhiyuan8 Dec 3, 2024
64a6001
update omni audio cmake
TerryT9 Dec 3, 2024
5962b50
Update omni-audio cmake content to make it static (#36)
TerryT9 Dec 10, 2024
1487d32
Add streaming for omnivlm (#39)
TerryT9 Jan 6, 2025
d3eeeae
support omnivlm for ios
TerryT9 Jan 10, 2025
6177770
add swift test cases
TerryT9 Jan 10, 2025
91ab9ed
update test case
TerryT9 Jan 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
24 changes: 0 additions & 24 deletions .clang-tidy
View file
Open in desktop

This file was deleted.

22 changes: 0 additions & 22 deletions .devops/cloud-v-pipeline
View file
Open in desktop

This file was deleted.

33 changes: 0 additions & 33 deletions .devops/full-cuda.Dockerfile
View file
Open in desktop

This file was deleted.

50 changes: 0 additions & 50 deletions .devops/full-rocm.Dockerfile
View file
Open in desktop

This file was deleted.

25 changes: 0 additions & 25 deletions .devops/full.Dockerfile
View file
Open in desktop

This file was deleted.

44 changes: 0 additions & 44 deletions .devops/llama-cli-cann.Dockerfile
View file
Open in desktop

This file was deleted.

37 changes: 0 additions & 37 deletions .devops/llama-cli-cuda.Dockerfile
View file
Open in desktop

This file was deleted.

28 changes: 0 additions & 28 deletions .devops/llama-cli-intel.Dockerfile
View file
Open in desktop

This file was deleted.

45 changes: 0 additions & 45 deletions .devops/llama-cli-rocm.Dockerfile
View file
Open in desktop

This file was deleted.

27 changes: 0 additions & 27 deletions .devops/llama-cli-vulkan.Dockerfile
View file
Open in desktop

This file was deleted.

23 changes: 0 additions & 23 deletions .devops/llama-cli.Dockerfile
View file
Open in desktop

This file was deleted.

Loading
Loading