Commit ff27f80
authored
ggml: initial IBM zDNN backend (#14975)
* ggml-zdnn: inital backend impl
Signed-off-by: Aaron Teo <[email protected]>
ggml-zdnn: temp change z17 to arch15
Signed-off-by: Aaron Teo <[email protected]>
ggml-zdnn: fix build bugs
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: tensor->extra logging check
Signed-off-by: Aaron Teo <[email protected]>
ggml-zdnn: add layout name mapping, ztensor information
Signed-off-by: Aaron Teo <[email protected]>
ggml-zdnn: separate logging into its own line
Signed-off-by: Aaron Teo <[email protected]>
ggml-zdnn: add shape comparison
Signed-off-by: Aaron Teo <[email protected]>
ggml-zdnn: add ggml_tensor shape log
Signed-off-by: Aaron Teo <[email protected]>
ggml-zdnn: fix incorrect shape logging
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add output buffer check
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: run compute and store into tensor->extra
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add set_tensor
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add more loggers
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: update set_tensor logging to check only for matmul
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: last working matmul version
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add comments to prevent accidentally deleting lines
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: support op out_prod
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: update op out_prod to use tensor->extra
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: rewrite the backend implementation
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: bugfix new impl
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: fix compiler warnings and bugfixes
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: test ztensor finding in init_tensor
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: implement at least 1 op to test
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: assign tensor->extra to buffer
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add check for view tensors to prevent init_tensor
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: rework init_tensor to create new buffers
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: switch to std vector instead of array
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: switch buffers back and set to arbitrary number
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: impl init_tensor
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: update supports_op matmul matrix
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: fix incorrect ztensor shape, reduce memory padding
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: code clean up
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: impl matmul
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: fix compiler error missing type
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: fix missing data transform call
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add bias init_tensor
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: tighten memory usage, change string allocation
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add bias ztensor and data free
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add bias data transform
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add more debug info for extra buffer transform
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add logger to check if mat mul ops go through set_tensor
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: activate bias transform in matmul
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: move weights transform into mulmat
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add more safeguards in matmul
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: fix sequencing of transforms
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: bugfix transform ztensor vs origtensor
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: figure out why sigtrap is happening
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: fix sigsegv
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: move everything back to local declaration
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: move bias data to local also
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: bring back working matmul
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: rewrite into mre
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: fix missing vector import
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: fix missing vector import in header
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: attempt to fix sigsegv
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: fix missing load tensor
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: fix invalid ztensor buffer release
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add logging to debug free buffer
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: remove free_buffer debug info
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add parmblkformat detections
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add nnpa installed detection
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add zdnn_init call for static libs
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add init_tensor
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: attempt at fixing invalid buffer
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: switch to using deque to fix pointer deref problem
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add weights logging to check
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: attempt to use unique ptr
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add tensor to pre_tfm_desc logging
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add inputs logging
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: disable op_none initialisation for testing
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: fix missing return from init_tensor
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: load ztensors in cgraph exec
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: work on moving output ztensor as well
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: disable logging and breakpoints for full test
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: attempt at manually changing the layout
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: attempt at using default nwhc format instead
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: disable global load ztensor for now
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: fix errorenous output load tensor
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: add guards to prevent loading ztensor if transformed
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: code cleanup
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: bring load ztensor back to init routine
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: code clean up
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: fix ztensor deallocation abort
stabilise ggml <-> zdnn api
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: clean up matmul selection
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: clean up project structure
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: update documentation, prepare for upstream
Signed-off-by: Aaron Teo <[email protected]>
* chore: add codeowners
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: disable batched matmul
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: attempt at fixing tensor views during matmul
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: deny all view tensors directly
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: fix pr comments
Signed-off-by: Aaron Teo <[email protected]>
* docs: update ops docs for zdnn
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: redo test-backend-ops for ops.md
Signed-off-by: Aaron Teo <[email protected]>
* ggml-zdnn: fix typo in build-s390x.md
Signed-off-by: Aaron Teo <[email protected]>
* codeowners: remove taronaeo for now
Signed-off-by: Aaron Teo <[email protected]>
* Revert "codeowners: remove taronaeo for now"
This reverts commit 411ea4e.
* ggml-zdnn: remove unused ggml_zdnn macro
Signed-off-by: Aaron Teo <[email protected]>
---------
Signed-off-by: Aaron Teo <[email protected]>1 parent d3248d9 commit ff27f80
File tree
15 files changed
+9265
-102
lines changed- .github
- ISSUE_TEMPLATE
- docs
- ops
- ggml
- include
- src
- ggml-cpu
- ggml-zdnn
15 files changed
+9265
-102
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
43 | | - | |
| 43 | + | |
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
45 | | - | |
| 45 | + | |
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
25 | 30 | | |
26 | 31 | | |
27 | 32 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
| 13 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
76 | 76 | | |
77 | 77 | | |
78 | 78 | | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
79 | 96 | | |
80 | 97 | | |
81 | 98 | | |
| |||
145 | 162 | | |
146 | 163 | | |
147 | 164 | | |
148 | | - | |
| 165 | + | |
149 | 166 | | |
150 | 167 | | |
151 | 168 | | |
152 | | - | |
| 169 | + | |
153 | 170 | | |
154 | | - | |
| 171 | + | |
155 | 172 | | |
156 | | - | |
| 173 | + | |
157 | 174 | | |
158 | 175 | | |
159 | 176 | | |
| |||
229 | 246 | | |
230 | 247 | | |
231 | 248 | | |
232 | | - | |
233 | | - | |
234 | | - | |
235 | | - | |
236 | | - | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
237 | 255 | | |
238 | 256 | | |
239 | 257 | | |
| |||
242 | 260 | | |
243 | 261 | | |
244 | 262 | | |
245 | | - | |
| 263 | + | |
246 | 264 | | |
247 | 265 | | |
248 | 266 | | |
| |||
273 | 291 | | |
274 | 292 | | |
275 | 293 | | |
276 | | - | |
| 294 | + | |
0 commit comments