Skip to content

Commit 4f5d796

Browse files
peng3307165peng3307165
committed
Added feature comparison in the README.
1 parent 1096323 commit 4f5d796

File tree

2 files changed

+288
-0
lines changed

2 files changed

+288
-0
lines changed

‎README.md

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,151 @@ res = requests.post(url, headers=headers, json=json.dumps(data))
9090
Coming soon...
9191
## modules
9292
![modelcache modules](docs/modelcache_modules_en.png)
93+
## Function-Comparison
94+
In terms of functionality, we have made several changes to the git repository. Firstly, we have addressed the network issues with huggingface and enhanced the inference speed by introducing local inference capabilities for embeddings. Additionally, considering the limitations of the SqlAlchemy framework, we have completely revamped the module responsible for interacting with relational databases, enabling more flexible database operations. In practical scenarios, LLM products often require integration with multiple users and multiple models. Hence, we have added support for multi-tenancy in the ModelCache, while also making preliminary compatibility adjustments for system commands and multi-turn dialogue.
95+
96+
<html>
97+
<head>
98+
<style>
99+
table, th, td {
100+
border-collapse: collapse;
101+
text-align: left;
102+
padding: 10px;
103+
margin-left: 20px;
104+
margin-right: 20px;
105+
}
106+
.checkmark {
107+
font-size: 24px;
108+
}
109+
110+
</style>
111+
</head>
112+
<body>
113+
114+
<table>
115+
<tr>
116+
<th rowspan="2">Module</th>
117+
<th rowspan="2">Function</th>
118+
119+
</tr>
120+
<tr>
121+
<th>ModelCache</th>
122+
<th>GPTCache</th>
123+
</tr>
124+
<tr>
125+
<td rowspan="2">Basic Interface</td>
126+
<td>Data query interface</td>
127+
<td class="checkmark">&#9745; </td>
128+
<td class="checkmark">&#9745; </td>
129+
</tr>
130+
<tr>
131+
<td>Data writing interface</td>
132+
<td class="checkmark">&#9745; </td>
133+
<td class="checkmark">&#9745; </td>
134+
</tr>
135+
<tr>
136+
<td rowspan="3">Embedding</td>
137+
<td>Embedding model configuration</td>
138+
<td class="checkmark">&#9745; </td>
139+
<td class="checkmark">&#9745; </td>
140+
</tr>
141+
<tr>
142+
<td>Large model embedding layer</td>
143+
<td class="checkmark">&#9745; </td>
144+
<td></td>
145+
</tr>
146+
<tr>
147+
<td>BERT model long text processing</td>
148+
<td class="checkmark">&#9745; </td>
149+
<td></td>
150+
</tr>
151+
<tr>
152+
<td rowspan="2">Large model invocation</td>
153+
<td>Decoupling from large models</td>
154+
<td class="checkmark">&#9745; </td>
155+
<td></td>
156+
</tr>
157+
<tr>
158+
<td>Local loading of embedding model</td>
159+
<td class="checkmark">&#9745; </td>
160+
<td></td>
161+
</tr>
162+
<tr>
163+
<td rowspan="2">Data isolation</td>
164+
<td>Model data isolation</td>
165+
<td class="checkmark">&#9745; </td>
166+
<td class="checkmark">&#9745; </td>
167+
</tr>
168+
<tr>
169+
<td>Hyperparameter isolation</td>
170+
<td></td>
171+
<td></td>
172+
</tr>
173+
<tr>
174+
<td rowspan="3">Databases</td>
175+
<td>MySQL</td>
176+
<td class="checkmark">&#9745; </td>
177+
<td class="checkmark">&#9745; </td>
178+
</tr>
179+
<tr>
180+
<td>Milvus</td>
181+
<td class="checkmark">&#9745; </td>
182+
<td class="checkmark">&#9745; </td>
183+
</tr>
184+
<tr>
185+
<td>OceanBase</td>
186+
<td class="checkmark">&#9745; </td>
187+
<td></td>
188+
</tr>
189+
<tr>
190+
<td rowspan="3">Session management</td>
191+
<td>Single-turn dialogue</td>
192+
<td class="checkmark">&#9745; </td>
193+
<td class="checkmark">&#9745; </td>
194+
</tr>
195+
<tr>
196+
<td>System commands</td>
197+
<td class="checkmark">&#9745; </td>
198+
<td></td>
199+
</tr>
200+
<tr>
201+
<td>Multi-turn dialogue</td>
202+
<td class="checkmark">&#9745; </td>
203+
<td></td>
204+
</tr>
205+
<tr>
206+
<td rowspan="2">Data management</td>
207+
<td>Data persistence</td>
208+
<td class="checkmark">&#9745; </td>
209+
<td class="checkmark">&#9745; </td>
210+
</tr>
211+
<tr>
212+
<td>One-click cache clearance</td>
213+
<td class="checkmark">&#9745; </td>
214+
<td></td>
215+
</tr>
216+
<tr>
217+
<td rowspan="2">Tenant management</td>
218+
<td>Support for multi-tenancy</td>
219+
<td class="checkmark">&#9745; </td>
220+
<td></td>
221+
</tr>
222+
<tr>
223+
<td>Milvus multi-collection capability</td>
224+
<td class="checkmark">&#9745; </td>
225+
<td></td>
226+
</tr>
227+
<tr>
228+
<td>Other</td>
229+
<td>Long-short dialogue distinction</td>
230+
<td class="checkmark">&#9745; </td>
231+
<td></td>
232+
</tr>
233+
</table>
234+
235+
</body>
236+
</html>
237+
93238
## Core-Features
94239
In ModelCache, we adopted the main idea of GPTCache, includes core modules: adapter, embedding, similarity, and data_manager. The adapter module is responsible for handling the business logic of various tasks and can connect the embedding, similarity, and data_manager modules. The embedding module is mainly responsible for converting text into semantic vector representations, it transforms user queries into vector form.The rank module is used for sorting and evaluating the similarity of the recalled vectors. The data_manager module is primarily used for managing the database. In order to better facilitate industrial applications, we have made architectural and functional upgrades as follows:
95240

‎README_CN.md

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,149 @@ res = requests.post(url, headers=headers, json=json.dumps(data))
9090
敬请期待
9191
## 架构大图
9292
![modelcache modules](docs/modelcache_modules.png)
93+
## 功能对比
94+
功能方面，为了解决huggingface网络问题并提升推理速度，增加了embedding本地推理能力。鉴于SqlAlchemy框架存在一些限制，我们对关系数据库交互模块进行了重写，以更灵活地实现数据库操作。在实践中，大型模型产品需要与多个用户和多个模型对接，因此在ModelCache中增加了对多租户的支持，同时也初步兼容了系统指令和多轮会话。
95+
<html>
96+
<head>
97+
<style>
98+
table, th, td {
99+
border-collapse: collapse;
100+
text-align: left;
101+
padding: 8px;
102+
margin-left: 20px;
103+
margin-right: 20px;
104+
}
105+
.checkmark {
106+
font-size: 24px;
107+
}
108+
109+
</style>
110+
</head>
111+
<body>
112+
113+
<table>
114+
<tr>
115+
<th rowspan="2">模块</th>
116+
<th rowspan="2">功能</th>
117+
118+
</tr>
119+
<tr>
120+
<th>ModelCache</th>
121+
<th>GPTCache</th>
122+
</tr>
123+
<tr>
124+
<td rowspan="2">基础接口</td>
125+
<td>数据查询接口</td>
126+
<td class="checkmark">&#9745; </td>
127+
<td class="checkmark">&#9745; </td>
128+
</tr>
129+
<tr>
130+
<td>数据写入接口</td>
131+
<td class="checkmark">&#9745; </td>
132+
<td class="checkmark">&#9745; </td>
133+
</tr>
134+
<tr>
135+
<td rowspan="3">Embedding</td>
136+
<td>embedding模型配置</td>
137+
<td class="checkmark">&#9745; </td>
138+
<td class="checkmark">&#9745; </td>
139+
</tr>
140+
<tr>
141+
<td>大模型embedding层</td>
142+
<td class="checkmark">&#9745; </td>
143+
<td></td>
144+
</tr>
145+
<tr>
146+
<td>bert模型长文本处理</td>
147+
<td class="checkmark">&#9745; </td>
148+
<td></td>
149+
</tr>
150+
<tr>
151+
<td rowspan="2">Large model invocation</td>
152+
<td>是否与大模型解耦</td>
153+
<td class="checkmark">&#9745; </td>
154+
<td></td>
155+
</tr>
156+
<tr>
157+
<td>embeddingg模型本地加载</td>
158+
<td class="checkmark">&#9745; </td>
159+
<td></td>
160+
</tr>
161+
<tr>
162+
<td rowspan="2">数据隔离</td>
163+
<td>模型数据隔离</td>
164+
<td class="checkmark">&#9745; </td>
165+
<td class="checkmark">&#9745; </td>
166+
</tr>
167+
<tr>
168+
<td>超参数隔离</td>
169+
<td></td>
170+
<td></td>
171+
</tr>
172+
<tr>
173+
<td rowspan="3">数据库</td>
174+
<td>MySQL</td>
175+
<td class="checkmark">&#9745; </td>
176+
<td class="checkmark">&#9745; </td>
177+
</tr>
178+
<tr>
179+
<td>Milvus</td>
180+
<td class="checkmark">&#9745; </td>
181+
<td class="checkmark">&#9745; </td>
182+
</tr>
183+
<tr>
184+
<td>OceanBase</td>
185+
<td class="checkmark">&#9745; </td>
186+
<td></td>
187+
</tr>
188+
<tr>
189+
<td rowspan="3">会话管理</td>
190+
<td>单轮回话</td>
191+
<td class="checkmark">&#9745; </td>
192+
<td class="checkmark">&#9745; </td>
193+
</tr>
194+
<tr>
195+
<td>system指令</td>
196+
<td class="checkmark">&#9745; </td>
197+
<td></td>
198+
</tr>
199+
<tr>
200+
<td>多轮回话</td>
201+
<td class="checkmark">&#9745; </td>
202+
<td></td>
203+
</tr>
204+
<tr>
205+
<td rowspan="2">数据管理</td>
206+
<td>数据持久化</td>
207+
<td class="checkmark">&#9745; </td>
208+
<td class="checkmark">&#9745; </td>
209+
</tr>
210+
<tr>
211+
<td>一键清空缓存</td>
212+
<td class="checkmark">&#9745; </td>
213+
<td></td>
214+
</tr>
215+
<tr>
216+
<td rowspan="2">租户管理</td>
217+
<td>支持多租户（多模型）</td>
218+
<td class="checkmark">&#9745; </td>
219+
<td></td>
220+
</tr>
221+
<tr>
222+
<td>milvus多表能力</td>
223+
<td class="checkmark">&#9745; </td>
224+
<td></td>
225+
</tr>
226+
<tr>
227+
<td>其他</td>
228+
<td>长短对话区分能力</td>
229+
<td class="checkmark">&#9745; </td>
230+
<td></td>
231+
</tr>
232+
</table>
233+
234+
</body>
235+
</html>
93236
## 核心功能
94237
在ModelCache中，沿用了GPTCache的主要思想，包含了一系列核心模块：adapter、embedding、similarity和data_manager。adapter模块主要功能是处理各种任务的业务逻辑，并且能够将embedding、similarity、data_manager等模块串联起来；embedding模块主要负责将文本转换为语义向量表示，它将用户的查询转换为向量形式，并用于后续的召回或存储操作；rank模块用于对召回的向量进行相似度排序和评估；data_manager模块主要用于管理数据库。同时，为了更好的在工业界落地，我们做了架构和功能上的升级，如下：
95238

0 commit comments

Comments
 (0)