Skip to content

Commit ad69961

Browse files
authored
Merge pull request #10 from codefuse-ai/modelcache_localDB_dev
update readme file with the latest news; update requirements;
2 parents 48ad5bb + 2e02694 commit ad69961

File tree

3 files changed

+4
-1
lines changed

3 files changed

+4
-1
lines changed

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ Codefuse-ModelCache
2424
- [Core-Features](#Core-Features)
2525
- [Acknowledgements](#Acknowledgements)
2626
## news
27+
[2023.11.20] codefuse-ModelCache has integrated local storage, such as sqlite and faiss, providing users with the convenience of quickly initiating tests.
2728
[2023.08.26] codefuse-ModelCache...
2829
### Introduction
2930
Codefuse-ModelCache is a semantic cache for large language models (LLMs). By caching pre-generated model results, it reduces response time for similar requests and improves user experience. <br />This project aims to optimize services by introducing a caching mechanism. It helps businesses and research institutions reduce the cost of inference deployment, improve model performance and efficiency, and provide scalable services for large models. Through open-source, we aim to share and exchange technologies related to large model semantic cache.

README_CN.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ Codefuse-ModelCache
1010
<p>
1111
<b>中文</b> |
1212
<a href="https://github.com/codefuse-ai/CodeFuse-ModelCache/blob/main/README.md">English</a>
13-
<p>
13+
</p>
1414
</h4>
1515
</div>
1616

@@ -23,6 +23,7 @@ Codefuse-ModelCache
2323
- [架构大图](#架构大图)
2424
- [核心功能](#核心功能)
2525
## 新闻
26+
[2023.11.20] codefuse-ModelCache增加本地存储能力, 适配了嵌入式数据库sqlite、faiss,方便用户快速启动测试。
2627
[2023.10.31] codefuse-ModelCache...
2728
## 项目简介
2829
Codefuse-ModelCache 是一个开源的大模型语义缓存系统,通过缓存已生成的模型结果,降低类似请求的响应时间,提升用户体验。该项目从服务优化角度出发,引入缓存机制,在资源有限和对实时性要求较高的场景下,帮助企业和研究机构降低推理部署成本、提升模型性能和效率、提供规模化大模型服务。我们希望通过开源,分享交流大模型语义Cache的相关技术。

requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,4 @@ PyMySQL==1.1.0
99
Requests==2.31.0
1010
torch==2.1.0
1111
transformers==4.34.1
12+
faiss-cpu==1.7.4

0 commit comments

Comments
 (0)