model : support vision LiquidAI LFM2-VL family

[tdakhran]

PR is based on ngxson#28. Huge thanks to @ngxson for bootstrap!

Add support for LFM2-VL vision models from LiquidAI.
Checkpoints are available on HF

• LFM2-VL-1.6B
• LFM2-VL-450M

LFM2-VL is a dynamic image resolution model, and support for dynamic resolution is implemented.
It uses siglip2 naflex, which does interpolation of positional embedding, which is implemented in the resize_position_embeddings function.
The preprocessor calculates the optimal image size (smart resize), then resizes and pads the input image.

Tested with all combinations of the following params

• backends: CPU, CUDA
• image resolutions: 256x256, 277x512, 512x277, 512x384, 512x512
• quantization(backbone/mmproj): F32/F32, Q4_0/Q8_0

Sample output for the image below with prompt describe image in one sentence

main: loading model: /data/playground/vlm2/LFM2-VL-450M/LFM2-VL-450M-Q4_0.gguf
encoding image slice...
image slice encoded in 117 ms
decoding image batch 1/1, n_tokens_batch = 64
image decoded (batch 1/1) in 49 ms

The image features a woman wearing a stylish hat adorned with blue feathers, set against a warm, orange-toned background.


llama_perf_context_print:        load time =     198.18 ms
llama_perf_context_print: prompt eval time =     202.74 ms /    78 tokens (    2.60 ms per token,   384.73 tokens per second)
llama_perf_context_print:        eval time =     163.87 ms /    27 runs   (    6.07 ms per token,   164.76 tokens per second)
llama_perf_context_print:       total time =     440.29 ms /   105 tokens

test log with timings.

[ngxson]

Very cool! Thanks for taking time to test it.

I'll deploy a GGUF on ggml-org for testing purpose.

We can merge after you resolve the comment of @CISC

[CISC]

I just learned the hard way that CUDA IM2COL doesn't support BF16. :)