Skip to content

Commit f6499bd

Browse files
authored
Merge pull request #5 from codefuse-ai/modelcache_dev
Adjusting the architecture diagram in the readme file
2 parents ab19af0 + f46615b commit f6499bd

File tree

5 files changed

+2
-2
lines changed

5 files changed

+2
-2
lines changed

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -89,7 +89,7 @@ res = requests.post(url, headers=headers, json=json.dumps(data))
8989
## Articles
9090
Coming soon...
9191
## modules
92-
![modelcache modules](docs/modelcache_modules_en.png)
92+
![modelcache modules](docs/modelcache_modules_20231114.png)
9393
## Function-Comparison
9494
In terms of functionality, we have made several changes to the git repository. Firstly, we have addressed the network issues with huggingface and enhanced the inference speed by introducing local inference capabilities for embeddings. Additionally, considering the limitations of the SqlAlchemy framework, we have completely revamped the module responsible for interacting with relational databases, enabling more flexible database operations. In practical scenarios, LLM products often require integration with multiple users and multiple models. Hence, we have added support for multi-tenancy in the ModelCache, while also making preliminary compatibility adjustments for system commands and multi-turn dialogue.
9595

README_CN.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -89,7 +89,7 @@ res = requests.post(url, headers=headers, json=json.dumps(data))
8989
## 文章
9090
敬请期待
9191
## 架构大图
92-
![modelcache modules](docs/modelcache_modules.png)
92+
![modelcache modules](docs/modelcache_modules_20231114.png)
9393
## 功能对比
9494
功能方面,为了解决huggingface网络问题并提升推理速度,增加了embedding本地推理能力。鉴于SqlAlchemy框架存在一些限制,我们对关系数据库交互模块进行了重写,以更灵活地实现数据库操作。在实践中,大型模型产品需要与多个用户和多个模型对接,因此在ModelCache中增加了对多租户的支持,同时也初步兼容了系统指令和多轮会话。
9595
<html>

docs/modelcache_modules.png

-455 KB
Binary file not shown.

docs/modelcache_modules_20231114.png

740 KB
Loading

docs/modelcache_modules_en.png

-472 KB
Binary file not shown.

0 commit comments

Comments
 (0)