Skip to content

Conversation

@sandrohanea
Copy link
Contributor

I want to build Whisper Runtimes for machines without AVX on an AVX-enabled host with Windows OS. This is for providing the runtimes for Whisper.net for users without AVX,

Because the checks if AVX, AVX2, etc are executed regardless of if the options are provided or not and updated it anyway, I cannot build these runtimes for Whisper.net.Runtime.NoAvx:

See: sandrohanea/whisper.net#239

I checked the build and indeed, even if I set -DGGML_AVX=OFF it will still be set to true because it is available on the host machine:
https://github.com/sandrohanea/whisper.net/actions/runs/11572290952/job/32211959849
image

This PR is addressing this issue to run those checks only if the option is already true, allowing it to be overridden to false if the capability is not found on the machine.

It also changes the default for MSVC to have INS_ENB by default on (to keep the same behaviour if building on an AVX enabled host)

This configuration will allow:

  • building AVX-enabled library on AVX-enabled host

  • building NotAVX-enabled library on AVX-enabled host

  • building NotAVX-enabled library on NotAVX-enabled host

  • Building AVX-enabled library by default on AVX-Enabled Host

  • Building NotAVX-enabled library by default on NotAVX-Enabled Host


# instruction set specific
if (GGML_NATIVE OR NOT GGML_NATIVE_DEFAULT)
if (NOT MSVC AND (GGML_NATIVE OR NOT GGML_NATIVE_DEFAULT))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand why the NOT MSVC check is necessary here. Can you clarify further and maybe an example of when it makes a difference?

Also, are you utilizing the CMAKE_CROSSCOMPILING variable in your builds?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am using:
cmake -S . -B build/win-x64 -A x64 (for building x64 Windows version with MSVC) without options in env variables (except the path for msvc, etc).

This will produce:
CMAKE_CROSSCOMPILING: FALSE
GGML_NATIVE_DEFAULT: ON
GGML_NATIVE: ON
INS_ENV: OFF

Before checking the usage of AVX and setting it to false only if it is not available on the machine (without setting it to true otherwise):
image

The above configuration would produce an AVX-enabled library on AVX-enabled host.

After adding that check, the default will now be OFF.
It's not a problem for me being only the default and still will be able to override it, but I don't think it is expected for others.

I was trying to allow all of this:

  • building AVX-enabled library on AVX-enabled host

  • building NotAVX-enabled library on AVX-enabled host

  • building NotAVX-enabled library on NotAVX-enabled host

  • Building AVX-enabled library by default on AVX-Enabled Host

  • Building NotAVX-enabled library by default on NotAVX-Enabled Host

if removing that default check, the 4th will be true:

  • Building NotAVX-enabled library by default on AVX-Enabled Host

Your call on this one, I can change the default however you say.

@sandrohanea sandrohanea requested a review from ggerganov October 30, 2024 10:05
@slaren
Copy link
Member

slaren commented Oct 30, 2024

The AVX options are intended to be used for cross-compiling to a specific instruction set independently of the host computer. If you don't want this behavior, you need to disable GGML_NATIVE.

@sandrohanea
Copy link
Contributor Author

Got it, I misunderstood it and was not disabling GGML_NATIVE.

If disabling GGML_NATIVE I am able to build both AVX-Enabled Libs and Not-AVX-Enabled Libs using one single host (that have AVX).

Thank you for the answers and your time @slaren , @ggerganov (will close this now).

@sandrohanea sandrohanea deleted the sandro/fix-noavx branch October 30, 2024 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants