update readme and upload models

Author: foldl

README.md

@@ -13,6 +13,7 @@ pure C++ implementation based on [@ggerganov](https://github.com/ggerganov)'s [g
 
 **What's New:**
 
+* 2025-06-03: Kimi-VL
 * 2025-05-28: Gemma3 fully supported
 * 2025-05-23: [I can see](./docs/multimodal.md): Fuyu
 * 2025-05-21: Re-quantization when loading (e.g. `--re-quantize q4_k`)

docs/gpu.md

@@ -51,6 +51,15 @@ The full format of `-ngl` is `-ngl [id:]layer_specs[;id:layer_specs]..`. `id` is
 Suppose device 0 is GPU, and device 1 is CPU, `-ngl 1:5;0:10` will put the first 5 layers to CPU, the next 10 layers to GPU,
 and all other layers to CPU as default.
 
+You can use `-mgl` (`--model_gpu_layers`) to specify number of layers of a specific model to be deployed to different backend devices.
+`-mgl MODEL N`, in which `N` shares the same syntax as `-ngl` and `MODEL` can be
+
+* `main`: the main model.
+* `vis`: the vision accessory model (which typically project images/videos into LLM).
+* `any`: any model.
+
+`-ngl N` is equivalent to `-mgl any N`.
+
 Tip: Use `--show_devices` to check all available devices and `--show` to check basic hyper parameters of a model.
 
 ## Known issues

docs/models.md

@@ -297,6 +297,9 @@ Please use `--format completion` for these models.
 
     Note: Only download `tokenizer.model` and DO NOT download `tokenizer.json` when converting.
 
+* Kimi (`KimiVLForConditionalGeneration`)
+    * [x] VL: [A3B-Instruct](https://huggingface.co/moonshotai/Kimi-VL-A3B-Instruct/tree/7a3c132a7b0f1f1677f5a72f258bd3afded7d357), [A3B-Thinking](https://huggingface.co/moonshotai/Kimi-VL-A3B-Thinking/commit/16681d8ac24e505088698e4e34ea494dd6e24400)
+
 ## RAG Models
 
 * Text Embedding (`XLMRobertaModel`)

scripts/models.json

@@ -2776,5 +2776,25 @@
                 }
             }
         }
+    },
+    "kimi-vl": {
+        "brief": "Kimi-VL, an efficient open-source Mixture-of-Experts (MoE) vision-language model (VLM) that offers advanced multimodal reasoning, long-context understanding, and strong agent capabilities.",
+        "default": "a3b-instruct",
+        "license": "MIT",
+        "variants": {
+            "a3b-instruct": {
+                "default": "q8",
+                "quantized": {
+                    "q8": {
+                        "size": 17566398608,
+                        "url": "chatllm_quantized_kimi-vl/kimi-vl.bin"
+                    },
+                    "q4_1": {
+                        "size": 10447149072,
+                        "url": "chatllm_quantized_kimi-vl/kimi-vl-q4_1.bin"
+                    }
+                }
+            }
+        }
     }
 }