Skip to content

Dev refactoring #9662

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 127 commits into from
Closed

Dev refactoring #9662

wants to merge 127 commits into from

Conversation

ykhrustalev
Copy link
Contributor

QNN example for Android

zhou.weiguo and others added 30 commits April 24, 2024 16:28
@jeffzhou2000
ggml: add Qualcomm QNN(Qualcomm Neural Network,aka Qualcomm AI Engine…
b0c3013 
… Direct) backend
@jeffzhou2000
ggml: add Qualcomm QNN(Qualcomm Neural Network,aka Qualcomm AI Engine…
d325088 
… Direct) backend
@jeffzhou2000
rebase
c75817b
@jeffzhou2000
refine ggml-qnn-ut program and script to make reviewers happy
9c872cb
@jeffzhou2000
review: replace external declaration with NDK header file
926a866
@jeffzhou2000
add supportive of quantize data type Q8_0
dd29834
@jeffzhou2000
review: remove unused QNN helper functions
f4c5303
@jeffzhou2000
ggml-qnn: remove static global vars to support multi-instance simulta…
2fab33d 
…neously
@jeffzhou2000
review: remove static global vars to support multi-instance simultane…
94ee775 
…ously and thread safe
@jeffzhou2000
review: put qnn's internal log inside preprocessor diretive
5d691c6
@jeffzhou2000
review: code format using clang-format + manually modification accord…
fdf0272 
…ing to review comments
@jeffzhou2000
review: fix a memory leak introduced by review modification which exp…
3e8b61f 
…lained in https://github.com/zhouwg/llama.cpp/pull/1
@jeffzhou2000
npu: probe htp info and capacity of rpc ion memory
d38d4a6
@jeffzhou2000
ggml-qnn: refine source code of ggml-qnn.cpp to make reviewer more happy
5f8cfe4
@jeffzhou2000
ggml-qnn: refine ggml inference using QNN NPU
5269e08
@jeffzhou2000
ggml-qnn: refine ggml inference using QNN NPU
faaa86b
@jeffzhou2000
review: make a MVP(Minimum Viable PR) style PR in upstream
5598fbd
@chraac
init the test array with const values
5e18cdc
@chraac
add ggml_qnn_tensor_binder
6c68adc
@chraac
use tensor wrapper in add
37bb926
@chraac
use tensor wrapper in matmul
36e41a1
@chraac
use ggml_qnn_tensor_reader for output tensor
a5679dd
@chraac
use ggml_qnn_tensor_writer for all parameters
5fe7b87
@chraac
rename
9456bba
@chraac
fix todo
65a14d9
@chraac
make the constant condition first
aeef0c6
@chraac
remove TODO
dfe159f
@chraac
split logger function, tensors and backend from main qnn source
9932062
@chraac
remove reference of g_qnn_mgr in qnn_instance
3c491a3
@chraac
fix compiling error
3fe07eb
chraac and others added 25 commits July 26, 2024 22:52
@chraac
refactoring: split qnn rpc buffer into dedicated class
ee305cc
@chraac
fix: try fix error in 2nd run by appending dimension into graph key
47735cb
@chraac
fix: suppress warning
be9a8c7
@chraac
refactoring: opt graph key gen
18aa665
@chraac
refactoring: remove dup code
2c73791
@chraac
refactoring: remove unused get_rpcmem_from_memhandle func
ccfec70
@chraac
feat: add error string for QnnOpPackage_Error_t
867c91b
@chraac
refactoring: move forward and supports_op into ops file
5da73f8
@chraac
feat: check if dims equal for add
e0c9b34 
looks qnn add can only applied to matrix with equal dimensions
@chraac
refactoring: remove internal functions, use op table directly
8ab1f15
@chraac
refactoring: print the name of unsupport op
e33b5c9
@chraac
refactoring: improve tensor print
1f9d2a7
@chraac
Merge branch 'master' into dev-refactoring
5ecbeb5 
# Conflicts:
#	CMakeLists.txt
#	ggml/src/CMakeLists.txt
#	ggml/src/ggml-backend.c
#	src/llama.cpp
@chraac
refactoring: set the default qnn lib search path at CMakeLists.txt by…
6da8294 
… GGML_QNN_DEFAULT_LIB_SEARCH_PATH
@chraac
refactoring: add convient macro to disable copy and move of class
9a5f802
@chraac
feat: add ggml_qnn_op_config for handle different op
74eb05a
@chraac
Merge remote-tracking branch 'origin/master' into dev-refactoring
6cc7432
@chraac
fix: try fix the tensor rank of mul mat
47f6e02
@chraac
Merge branch 'master' into dev-refactoring
5ea980d
@myan-o
Fixed a bug where debug code was included in the release, resulting i… (
dedadf2 
#1)

* Fixed a bug where debug code was included in the release, resulting in an undefined function error.

* Change the path of the QNN library when building in termux environment

* Revert "Change the path of the QNN library when building in termux environment"

This reverts commit c6e26a3.

* Changed so that GGML_QNN_DEFAULT_LIB_SEARCH_PATH can be set from command line arguments
@chraac
Merge branch 'master' into dev-refactoring
6bee798
@chraac
Merge branch 'master' into dev-refactoring
c9be2ba
@chraac
Merge branch 'master' into dev-refactoring
67e8af7 
# Conflicts:
#	src/llama.cpp
@chraac
fix compiling error
481cb3a
@chraac
Merge branch 'master' into dev-refactoring
b0b75d4
@github-actions github-actions bot added build Compilation issues ggml changes relating to the ggml tensor library for machine learning labels Sep 26, 2024
@Francis235
Copy link

Hi, I want to know how is llama.cpp's support for QNN backend now? Is there any specific usage tutorial?

@ggerganov ggerganov closed this Sep 30, 2024
@ggerganov
Copy link
Member

There is no plan to merge this backend.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

build Compilation issues ggml changes relating to the ggml tensor library for machine learning

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@ykhrustalev @Francis235 @ggerganov @jeffzhou2000 @chraac @myan-o