Is the LLAMA_NATIVE flag enabled by default? And now how to determine which instructions are used?

[Folko-Ven]

Should I specify -DLLAMA_NATIVE=ON/LLAMA_NATIVE=1 when building? And how to determine which instructions are used?
In earlier versions, I saw what instructions are included during assembly during model loading, but now I don't see anything like that.

@netrunnereve , Sorry to bother you, could you please resolve my doubts.

[KerfuffleV2]

I'm not who you asked, but there's no LLAMA_NATIVE option in Makefile builds so you don't need to worry about that.

As for turning it on in cmake, it just tells the compiler to use the features that are available on the machine that's compiling. If you only plan on running the executable on the same machine you compiled it on, then turning on native shouldn't have a disadvantage. If you're going to copy it somewhere else, publish it for people to download, etc then if the system it eventually runs on doesn't support the same features as the one it was compiled on then you'll run into issues.

[KerfuffleV2]

But how can I confirm with which sets of instruction the file was compiled as a result?

As far as I know, that option just passes -march=native to the compiler, essentially saying: "Okay, compiler, use whatever features you want hat are available on this host. I trust your judgement."

So basically, whatever the compiler supports and chooses to use.

[Folko-Ven]

Ah, I meant that in older versions, after loading the model, it was written in big green letters what is used and what is not, well, in any case, it's not so important since I mostly use gcc (I don't know why, but it works faster than cmake for me).
P.S. I didn't even know that compilers become so smart.

[slaren]

Do you mean the "system info" line? That is still printed.

system_info: n_threads = 16 / 32 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 |

[KerfuffleV2]

Ahh, there's actually some weird stuff (in my opinion, anyway) with the log system. That output still gets produced, if I understand what you're talking about:

system_info: n_threads = 8 / 16 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | 

However, if you compile with logging, you won't see it on the console anymore. If you compile without logging, then it goes to the console. However, the cmake configuration here doesn't let you turn off logging, so...

[Folko-Ven]

Oh yes, this is it! I compiled with LLAMA_DISABLE_LOGS=1 and system_info is back. Thank you very much!
Well, probably if I use cmake with llama_native I'll have to trust the compiler. :)
Thanks again for your time.

[netrunnereve]

Looks like the others have already answered you but basically LLAMA_NATIVE=ON in CMake will detect what instructions are supported by your CPU and build the program using only those instructions. That's the default. If you set it to off it'll use a preset selection of instructions which you can alter with additional flags to match the machines you want to run it on.

[Folko-Ven]

Yes, I've already figured it out, but thanks a lot for the answer anyway.