Enable automatic endianness conversion, SIMD & MASS support #4
Conversation
…k log. 92% tests passing!
buildenv
Outdated
| # rm -f "llama" | ||
| # ln -s "llama.cpp" "llama" | ||
| # ln -s "llama.cpp" $ZOPEN_NAME | ||
| export ZOPEN_SKIP_ZOSLIB_ENV_HOOK=1 |
There was a problem hiding this comment.
Do you have more info on why this was needed?
There was a problem hiding this comment.
Charan tried this to restrict clang cmake flags while integrating SIMD / MASS.
It's not required and I verified this by removing it, building afresh and testing all the functionalities.
I have removed it in the latest commit.
| disabled={isTyping} | ||
| > | ||
| ⤠| ||
| ⤠|
There was a problem hiding this comment.
Is this change intentional?
There was a problem hiding this comment.
No, it's some kind of encoding issue with the send symbol on the demo website (I have attached a picture of it)
We weren't able to git restore the file to its original state, and it seemed to be a one-time issue with encoding that character so we just committed it for now.
patches/repack.cpp.patch
Outdated
| // instance for IQ4 | ||
| static const ggml::cpu::repack::tensor_traits<block_iq4_nl, 4, 4, GGML_TYPE_Q8_0> iq4_nl_4x4_q8_0; | ||
|
|
||
| - if (cur->type == GGML_TYPE_Q4_0) { |
There was a problem hiding this comment.
Perhaps we can guard this with #ifndef __MVS__ instead of commenting it?
There was a problem hiding this comment.
Yep, that's a much better approach, I have integrated this change in the latest commit. Thanks!
2 participants
Summary
This PR adds support for automatically converting models to the platform's native endianness at runtime and support for some z/OS specific architectural features.
Changes Made
Build Configuration (buildenv):
ZOPEN_CONFIGURE_OPTSPatches
Endianness Changes
__builtin_bswapfunctions in the compiler.SIMD Changes
vec_xl,vec_xst,vec_reve,vec_extract,vec_insertandvec_splatto manipulate vectorized data.MASS changes
Testing
Ran llama.cpp's test suite - 92.11% of tests are passing now: