Support Granite 4 models

[Christian24]

Description of the bug:

Hello,

The Granite 4 models by IBM seem to be especially great at low parameter counts. It would be great to have them supported in the LiteRT ecosystem.

Actual vs expected behavior:

No response

Any other information you'd like to share?

No response

[sourcelite]

Hi @Christian24, welcome here and thanks for this feature request. True, both the Granite 3.0 and 4.0 series are great coz they are all rounder and also great function calling, maybe make a great model for on-device accelerator. Let us check the possibility.

just curious - maybe this info be helpful to us.
1.] what Granite model series (traditional or Hybrid H model series) you using ? Granite has few very well known flavors. Granite-4.0-H-350M, H-1B, or the H-Small MoE using any Hybrid Mamba-2 layers ?, these are traditional Granite-4.0-Micro 3B). these are standard dense transformers. traditional are better for NPU acceleration, hybrid needs to map new mamba 2 kernal to NPU.
2.] what is your target hardware ? android/iOS/web
general info I have is -
Android (Qualcomm/Mediatek NPU): requires 4-bit/8-bit quantisation and AOT compilation
iOS (Metal/CoreML): requires specific FP16 optimization
Web (WebGPU/ML Drift): requires the new LiteRT WebGPU backend

@aravindmurali22 can we mark this as feature request and any further steps ?

[Christian24]

1. The normal Granite 4 model in 350M Parameters. Currently mostly technology demo. I might switch to 1B parameters later, but so far 350 M seems to work great.
2. Web / WebAI, so browser

[Christian24]

@sourcelite I did submit a feature request, but the template looks more like a bug template. Maybe the configuration is wrong? I just checked. It seems the template for feature requests is indeed a bug template.

[aravindmurali22]

Hi @g-davidmassoud could you please take a look into this? Thanks!

[g-davidmassoud]

We have Granite 4 on the roadmap but I can't share any timelines for completion.

[aravindmurali22]

@g-davidmassoud , thanks for the response. In this case we can keep this issue open to track this feature request. Thanks.