Skip to content

Conversation

polarathene
Copy link
Contributor

@polarathene polarathene commented Aug 5, 2025

This PR addresses feedback from another PR I gave:

  • APT: Use DEB822 .sources format instead of deprecated .list format.
  • Use Dockerfile HereDoc syntax.
  • Use -qq (implies -y) for apt-get commands, and ensure consistent cleanup via rm is used in related RUN steps.

NOTE: I am not familiar with oneAPI or supporting it.

As this feature is not part of the official mistral.rs Dockerfile (that it was claimed to have been imported from), there is a libomp-dev runtime dependency (since the original Dockerfile contribution at mistral.rs), but not one associated to oneAPI as has been added into the builder stage (presumably just for headers and symlink, at which point I wonder if the original runtime stage package had ever been required previously 🤷‍♂️).

Since the package has existed in the runtime stage from the beginning I've avoided shifting it into the MKL associated RUN or adding commentary, as I do not know if that is used by anything else?


Regarding size of this dependency

Personally I am against monolithic images and mixing concerns from a maintenance and user perspectives. Mixing the MKL/oneAPI support with CUDA seems unnecessary and is definitely wasteful to users interested in building the image that aren't using either of these heavy dependencies.

Installing the intel-oneapi-hpc-toolkit package appears to add approx 12GB of weight, which in this case will be double due to it being additive to both build and runtime stages. It could possibly be optimized there but I don't have the hardware available to investigate.

CUDA is of similar weight for it's build image, but the runtime image is notably slimmer by comparison.

I presently lack a system with the disk space to build the full Dockerfile, but I assume it's rather large. The test Dockerfile I used spent the bulk of it's build time just installing the dep (500 secs, about 80% of build time), and that is with no builder stage involved.


I'd encourage separate image variants for such going forward.

The existing support was snuck in despite the contributor having already been advised for their related contribution at mistral.rs to add mkl feature support as a separate image variant.

- APT: Use DEB822 `.sources` format instead of deprecated `.list` format.
- Use Dockerfile HereDoc syntax.
- Use `-qq` (implies `-y`) for `apt-get` commands, and ensure consistent cleanup via `rm` is used in related `RUN` steps.
@polarathene polarathene changed the title refactor: Dockerfile - MKL oneAPI support refactor: Dockerfile - MKL oneAPI support - Use DEB822 .sources Aug 5, 2025
@polarathene
Copy link
Contributor Author

Potential compatibility concerns for non-Intel or non-AMD64 hardware

Additionally since this contribution via #220 didn't actually go through any feedback regarding testing, it is unclear how well implemented/supported it is beyond the single user that contributed it (without much context to their usage/testing with it, given their primary input from where it was added has been focused around CUDA).

The TEI project for example has a similar addition in their primary Dockerfile, but involves a patched lib used with LD_PRELOAD at runtime which is cited for why they copy of all these related oneapi MKL libs, whilst not installing any related package separately in their runtime image.

Chances are that package added to the runtime image isn't entirely required in the runtime image, but the contributor did not bother to consider this or it's impact on others 🤷‍♂️

That said... the TEI project has a separate Intel optimized image variant, and this separate MKL addition to their default Dockerfile lacked context as to why that decision was made.

As a result this prevents compatibility when building for ARM64, as presumably the package is only supported for Intel CPUs on x86_64 / AMD64.

I can confirm that the package this PR is refactoring the install of is not supported on ARM64 either:

40.31 Reading package lists...
150.3 + apt-get install -qq intel-oneapi-hpc-toolkit
286.6 E: Unable to locate package intel-oneapi-hpc-toolkit

Another report related to the TEI reference is their image is incompatible with AMD CPUs, which seems feasible if the build is relying on MKL hardware support at runtime due to the build relying on Intel optimizations 😅 So not even AMD64 users will have a good time?

I'm not able to check that on an AMD system at present, but it might be worth confirming as TEI is also using Candle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant