[Feature] Mixed Hardware Acceleration Capabiltity Expanded Microservices/Machine Learning Container Environment

[tbelway]

I have searched the existing feature requests to make sure this is not a duplicate request.

• Yes

The feature

Currently the video transcoding settings are configured in the administration GUI, where the Acceleration API can be chosen amongst a list including NVENC, Quick Sync, VAAPI, RKMPP and Disabled. However, it is possible one will have a mixed environment with, for example, an NVIDIA GPU passed through to one box, an intel CPU with iGPU on another box, and an AMD CPU on another.

It would be beneficial to allow for granular environment settings per container, so that you can have multiple microservices/ml nodes operating on these different environments and not have the transaction fail due to the GUI setting having NVENC, when it is trying to run on a box with Quick Sync or no possible hardware acceleration.

I would propose the option be set in GUI with a container level environment variable to override allowing for the primary instance having, for example, NVENC, and then each other instance can toggle between HWA being disables or enabled with a different HWA configuration.

:edit:

Proposed Requirements:

1. Either Partial Config Modification or Environment Variables
   1a) If partial config modification, should take any undeclared values from database and not from default values
2. If Partial Config is detected, should not overwrite current values in DB
3. machine learning URL should accommodates multiple values, perhaps using an array instead of a string
   3a) Need a way to to handle machine learning designation, either through redis or postgres?

Platform

• Server
• Web
• Mobile

[mertalev]

You can use config files to specify different transcoding settings in each container.

[tbelway]

Yup, I'm rerunning smart search now with the cuda image. It'll be a few days for this to complete... I have 1.5TB of photos.

That being said, if this feature is implemented already then... what exactly is the correct way to do it? Clearly it's not as simple as outlined in this discussion...

I'd be happy to test as much as needed to g et it working, and I can even assist with the documentation.

:edit:

Well, holy crap, I guess you guys really optimized the CUDA ML. It's already at only 26k remaining! That means it'll probably be done in under an hour...

[tbelway]

So then, in order to launch multiple ML and microservices machines, what parameters need to defined?
Should we reopen this feature request?

[mertalev]

I reopened the FR.

Any non-default settings you want to apply to all jobs should be specified in each config file, including the CLIP model. Settings that should be different per instance, like transcoding, can have different values.

[tbelway]

OK, so the work around until the FR has traction is full configs then :)

:edit:

Tested, at the very least the ML does not function in this way, the CPU instance is not getting any work at all. I assume it has to do with the ML API pointer not sharing a URL.
Microservices however is functioning as expected.

[tbelway]

Looks like today's release addresses this at least for hte most part :D

[tbelway]

Given the capabilities now exist, should close this discussion.