Skip to content

WIP: Reintegrate MiGraphX support#249

Draft
dfgHiatus wants to merge 1 commit intomainfrom
feat/migraphx-for-all
Draft

WIP: Reintegrate MiGraphX support#249
dfgHiatus wants to merge 1 commit intomainfrom
feat/migraphx-for-all

Conversation

@dfgHiatus
Copy link
Copy Markdown
Collaborator

This is a similar issue to #247. Similarly, instead of statically bundling the ONNX Runtime within the application we instruct the application to look at the user's system runtime(s).

(Please note that this affects Linux only)

Pros:

  • Hardware acceleration on Linux!

Cons:

  • This closes off the door for proper CUDA support - MiGraphX is mutually exclusive with it.
  • The app has to convert our ONNX model to a MiGraphX format, hitching for a few seconds every time. While there is a way to save this graph to disk and load that into the future, I was not able to get it to work myself: https://onnxruntime.ai/docs/execution-providers/MIGraphX-ExecutionProvider.html#session-variables
  • This PR neuters the app's ability to run inference by itself. This is not friendly to anyone who just wants something that will work out of the box, with zero configuration whatsoever. You will need to install the onnxruntime for your system, and possibly (big!):
    • rocm-hip-sdk
    • migraphx
    • onnxruntime-rocm
    • ...And so on

This points to a bigger problem. A lot of these execution providers are mutually exclusive, even if we were to automate the build process for them it still falls on the end user to install the proper runtimes for their system, which may vary greatly.

Either way, I am putting it up here if any eager Linux users want to toy around with acceleration on device. Good luck, and have fun.

@dfgHiatus dfgHiatus self-assigned this Mar 22, 2026
@dfgHiatus dfgHiatus added enhancement New feature or request help wanted Extra attention is needed labels Mar 22, 2026
@exuvo
Copy link
Copy Markdown

exuvo commented Mar 23, 2026

I think you are overcomplicating things. Just let linux users build from source, they are used to it and most of the time the user repositories already have the build files there so it is simple to do too. One binary fits all clearly does not work well with how onnx is made.

I already have HW acceleration on arch linux by the AUR packaging removing the shipped libonnxruntime to let it use system ones. In my case onnxruntime-opt-rocm.

rm "${_publishdir}/libonnxruntime"*

I think that is an acceptable step to have if the user wants to use a specific target runtime instead of the default CPU one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request help wanted Extra attention is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants