ANE support

[ssrp7sz4zq-hash]

Hi Nagadomi,

Currently, IW3 uses GPU during conversion with apple silicon. If u can add support for models that utilize Apple Neural Engine, that would be more faster or convert in real time.

Thanks

[nagadomi]

Hi,
I have no motivation to work on this, since I don't own any Apple devices and renting a macOS instance on EC2 for testing would cost about $50 per day.

However, if it's just about replacing the loaded model with a CoreML .mlpackage wrapper that has the same interface, it shouldn't be that difficult.
Someone with enough motivation could probably do it.

[ssrp7sz4zq-hash]

Hi, I have no motivation to work on this, since I don't own any Apple devices and renting a macOS instance on EC2 for testing would cost about $50 per day.

However, if it's just about replacing the loaded model with a CoreML .mlpackage wrapper that has the same interface, it shouldn't be that difficult. Someone with enough motivation could probably do it.

Do you have any guidance to replace the model with .mlpackage?

Thanks in advance!

[nagadomi]

I only tested converting DepthAnything V1/V2/V3 to CoreML. https://gist.github.com/nagadomi/e1b8023cb9fa57e278d4bc10fc0beb60
The conversion succeeded with a fixed input size but failed to support variable input sizes.
I checked Apple's publicly released coreml-depth-anything-v2-small, and it also uses a fixed 518×518 input size.

If it can't handle variable resolutions, the issue is more complex.
It might be practical to have nunif.models.utils.compile_model return a wrapper model that performs dynamic CoreML model conversion, and use it with the torch.compile option.

Well, in any case, I won't know the exact procedure without actually investigating and experimenting.
I can answer specific questions related to this repository.

[nagadomi]

For iw3.desktop, if perfection isn't a priority, fixed resolutions such as 280, 392, or 518 with aspect ratios like 16:10 or 1:1 should be sufficient. The batch size is always 1.
In that case, it's just a matter of adding a CoreML-specific model such as Any_V2_S_coreml_392x392, which is enabled when sys.platform == "darwin".

[nagadomi]

It is not ANE, but autocast (FP16) has been enabled on MPS in the dev branch. #643
I have not tested it on MPS, but performance may improve significantly.