Skip to content

Commit 9d048b3

Browse files
authored
Refactor web-llm to support unified model artifact config (#91)
What does this PR do? 1. make mlc-llm a submodule of web-llm, and only keep the logic of web in web-llm, so we don't need to have multiple copies of code. 2. web-llm now takes in params , tokenizer, and runtime configurations from a folder in the following structure, so different models could be handled in the same format. ``` /mlc-llm-config.json /tokenizer.json /ndarray-cache.json /param_0.bin ... Schema of mlc-llm-config.json { # root to model artifacts, default to . (same as this file) # all artifacts are in the same folder as model_url "model_url": ".", # model lib name, used to look up lib in system or dylib "model_lib": "vicuna-v1-7b-q4f16_0", # model identifier, used to specify local directory "local_id": "myllama-v1-7b-q4f16_0", # list of tokenizer files, relative to model_url "tokenizer_files": ["tokenizer.model"], # ndarray files are implicitly in the same as model_url # the lists can be found in ndarray-cache.json # other parameters "conv_template": "vicuna-v1.1", "temperature": 0.9 , .. } ```
1 parent bbe768e commit 9d048b3

24 files changed

+110
-4760
lines changed

.gitmodules

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
11
[submodule "3rdparty/sentencepiece-js"]
22
path = 3rdparty/sentencepiece-js
33
url = https://github.com/tqchen/sentencepiece-js
4+
[submodule "mlc-llm"]
5+
path = mlc-llm
6+
url = https://github.com/mlc-ai/mlc-llm

build.py

Lines changed: 0 additions & 200 deletions
This file was deleted.

0 commit comments

Comments
 (0)