Skip to content

Commit 386d1b3

Browse files
authored
Merge pull request #13 from codefuse-ai/hmemory_dev
Hmemory dev
2 parents e63eec2 + 583667c commit 386d1b3

File tree

123 files changed

+7728
-1500
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

123 files changed

+7728
-1500
lines changed

.gitignore

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,4 +15,6 @@ build
1515
*egg-info
1616
dist
1717
.ipynb_checkpoints
18-
zdatafront*
18+
zdatafront*
19+
*antgroup*
20+
*ipynb

Dockerfile

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
From python:3.9.18-bookworm
1+
FROM python:3.9-bookworm
22

33
WORKDIR /home/user
44

@@ -11,10 +11,10 @@ COPY ./requirements.txt /home/user/docker_requirements.txt
1111
# RUN service inetutils-inetd start
1212
# service inetutils-inetd status
1313

14-
RUN wget https://oss-cdn.nebula-graph.com.cn/package/3.6.0/nebula-graph-3.6.0.ubuntu1804.amd64.deb
15-
RUN dpkg -i nebula-graph-3.6.0.ubuntu1804.amd64.deb
14+
# RUN wget https://oss-cdn.nebula-graph.com.cn/package/3.6.0/nebula-graph-3.6.0.ubuntu1804.amd64.deb
15+
# RUN dpkg -i nebula-graph-3.6.0.ubuntu1804.amd64.deb
1616

1717
RUN pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
18-
RUN pip install -r /home/user/docker_requirements.txt
18+
RUN pip install -r /home/user/docker_requirements.txt --retries 5 --timeout 120
1919

2020
CMD ["bash"]

README.md

Lines changed: 46 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,12 @@
22
<a href="README_zh.md">中文</a>&nbsp | &nbsp<a>English&nbsp </a>
33
</p>
44

5-
# <p align="center">CodeFuse-muAgent: A Multi-Agent FrameWork For Faster Build Agents</p>
5+
# <p align="center">CodeFuse-muAgent: An Innovative Agent Framework Driven By KG Engine</p>
66

77
<p align="center">
88
<a href="README.md"><img src="https://img.shields.io/badge/文档-中文版-yellow.svg" alt="ZH doc"></a>
99
<a href="README_en.md"><img src="https://img.shields.io/badge/document-English-yellow.svg" alt="EN doc"></a>
10-
<img src="https://img.shields.io/github/license/codefuse-ai/CodeFuse-muAgent" alt="License">
10+
<a href="LICENSE.md"><img src="https://img.shields.io/badge/license-Apache%202.0-yellow" alt="License">
1111
<a href="https://github.com/codefuse-ai/CodeFuse-muAgent/issues">
1212
<img alt="Open Issues" src="https://img.shields.io/github/issues-raw/codefuse-ai/CodeFuse-muAgent" />
1313
</a>
@@ -17,100 +17,73 @@
1717

1818

1919
## 🔔 News
20-
- [2024.04.01] codefuse-muagent is now open source, featuring functionalities such as knowledge base, code library, tool usage, code interpreter, and more
20+
- [2024.04.01] codefuse-muAgent is now open source, featuring functionalities such as knowledge base, code library, tool usage, code interpreter, and more
21+
- [2024.09.05] we release muAgent v2.0 - EKG: An Innovative Agent Framework Driven By KG Engine.
22+
23+
2124

2225
## 📜 Contents
2326
- [🤝 Introduction](#-Introduction)
2427
- [🚀 QuickStart](#-QuickStart)
25-
- [🧭 Key Technologies](#-Key-Technologies)
28+
- [🧭 Features](#-Features)
29+
- [🤗 Contribution](#-Contribution)
2630
- [🗂 Miscellaneous](#-Miscellaneous)
2731
- [📱 Contact Us](#-Contact-Us)
2832

2933

3034
## 🤝 Introduction
31-
Developed by the Ant CodeFuse Team, CodeFuse-muAgent is a Multi-Agent framework whose primary goal is to streamline the Standard Operating Procedure (SOP) orchestration for agents. muagent integrates a rich collection of toolkits, code libraries, knowledge bases, and sandbox environments, enabling users to rapidly construct complex Multi-Agent interactive applications in any field. This framework allows for the efficient execution and handling of multi-layered and multi-dimensional complex tasks.
3235

33-
![](docs/resources/agent_runtime.png)
36+
A brand new Agent Framework driven by LLM and EKG(Eventic Knowledge Graph, Industry Knowledge Carrier),collaboratively utilizing MultiAgent, FunctionCall, CodeInterpreter, etc. Through canvas-based drag-and-drop and simple text writing, the large language model can assists you in executing various complex SOP under human guidance. It is compatbile with existing frameworks on the market and can achieve four core differentiating technical functions: Complex Reasoning, Online Collaboration, Human Interaction, Knowledge On-demand.
37+
This framework has been validated in multiple complex DevOps scenarios within Ant Group. At the sametime, come and experience the Undercover game we quickly built!
3438

35-
## 🚀 快速使用
36-
For complete documentation, see: [CodeFuse-muAgent](https://codefuse-ai.github.io/docs/api-docs/MuAgent/overview/multi-agent)
37-
For more [demos](https://codefuse-ai.github.io/docs/api-docs/MuAgent/connector/customed_examples)
39+
![](docs/resources/ekg-arch-en.webp)
3840

39-
1. Installation
40-
```
41-
pip install codefuse-muagent
42-
```
41+
## 🚀 QuickStart
42+
For complete documentation, see: [CodeFuse-muAgent](https://codefuse.ai/docs/api-docs/MuAgent/overview/multi-agent)
43+
For more [demos](https://codefuse.ai/docs/api-docs/MuAgent/connector/customed_examples)
4344

44-
2. Code answer Prepare related llm and embedding model configurations
45-
```
46-
import os
47-
48-
# set your config
49-
api_key = ""
50-
api_base_url= ""
51-
model_name = ""
52-
embed_model = ""
53-
embed_model_path = ""
54-
55-
from muagent.llm_models.llm_config import EmbedConfig, LLMConfig
56-
from muagent.connector.phase import BasePhase
57-
from muagent.connector.schema import Message, Memory
58-
from muagent.codechat.codebase_handler.codebase_handler import CodeBaseHandler
59-
60-
llm_config = LLMConfig(
61-
model_name=model_name, api_key=api_key, api_base_url=api_base_url, temperature=0.3
62-
)
63-
64-
embed_config = EmbedConfig(
65-
embed_engine="model", embed_model=embed_model, embed_model_path=embed_model_path
66-
)
67-
```
6845

69-
<br>
46+
### EKG Services
7047

71-
Initialize the codebase
72-
```
73-
from muagent.base_configs.env_config import CB_ROOT_PATH
74-
codebase_name = 'client_local'
75-
code_path = "D://chromeDownloads/devopschat-bot/client_v2/client"
76-
77-
cbh = CodeBaseHandler(
78-
codebase_name, code_path, crawl_type='dir', use_nh=use_nh,local_graph_path=CB_ROOT_PATH,
79-
llm_config=llm_config, embed_config=embed_config
80-
)
81-
cbh.import_code(do_interpret=do_interpret)
48+
```bash
49+
# use ekg services only three steps
50+
# step1. git clone
51+
git clone https://github.com/codefuse-ai/CodeFuse-muAgent.git
52+
53+
# step2.
54+
cd CodeFuse-muAgent
55+
56+
# step3. start all container services, it might cost some time
57+
docker-compose up -d
8258
```
8359

84-
<br>
60+
The current image version includes only the basic EKG service. We expect to provide a front-end interface and back-end interaction services in September.
61+
62+
To Be Continued!
63+
64+
8565

86-
Start codebase Q&A
66+
### SDK
67+
We also provide a version of the SDK for using muagent.
68+
1. Installation
8769
```
88-
#
89-
phase_name = "codeChatPhase"
90-
phase = BasePhase(
91-
phase_name, embed_config=embed_config, llm_config=llm_config,
92-
)
93-
#
94-
query_content = "what does the remove' function?"
95-
query = Message(
96-
role_name="user", role_type="human", input_query=query_content,
97-
code_engine_name=codebase_name, score_threshold=1.0, top_k=3, cb_search_type="tag",
98-
local_graph_path=CB_ROOT_PATH, use_nh=False
99-
)
100-
output_message3, output_memory3 = phase.step(query)
101-
print(output_memory3.to_str_messages(return_all=True, content_key="parsed_output_list"))
70+
pip install codefuse-muagent
10271
```
10372

104-
## Key Technologies
73+
2. Code answer Prepare related llm and embedding model configurations
74+
you can see [docs](https://codefuse.ai/docs/api-docs/MuAgent/connector/customed_examples) and [~/examples](https://github.com/codefuse-ai/CodeFuse-muAgent/tree/main/examples)
75+
76+
10577

106-
- Agent Base:Four fundamental Agent types are constructed – BaseAgent, ReactAgent, ExecutorAgent, SelectorAgent, supporting basic activities across various scenarios
107-
- Communication: Information transmission between Agents is accomplished through Message and Parse Message entities, interacting with Memory Manager and managing memories in the Memory Pool
108-
- Prompt Manager: Customized Agent Prompts are automatically assembled with the aid of Role Handler, Doc/Tool Handler, Session Handler, and Customized Handler
109-
- Memory Manager: Facilitates the management of chat history storage, information compression, and memory retrieval, culminating in storage within databases, local systems, and vector databases via the Memory Pool
110-
- Component: Auxiliary ecosystem components to construct Agents, including Retrieval, Tool, Action, Sandbox, etc.
111-
- Customized Model: Supports the integration of private LLM and Embedding models
78+
## 🧭 Features
79+
- **EKG Builder**:Through the design of virtual teams, scene intentions, and semantic nodes, you can experience the differences between online and local documentation, or annotated versus unannotated code handover. For a vast amount of existing documents (text, diagrams, etc.), we support intelligent parsing, which is available for one-click import.
80+
- **EKG Assets**:Through comprehensive KG Schema design—including Intention Nodes, Workflow Nodes, Tool Nodes, and Character Nodes—we can meet various SOP Automation requirements. The inclusion of Tool Nodes in the KG enhances the accuracy of tool selection and parameter filling. Additionally, the incorporation of Characters (whether human or agents) in the KG allows for human-involved process advancement, making it flexible for use in multiplayer text-based games.
81+
- **EKG Reasoning**:Compared to purely model-based or entirely fix-flow Reasoning, our framework allows LLM to operate under human guidance-flexibility, control, and enabling exploration in unknown scenarios. Additionally, successful exploration experiences can be summarized and documented into KG, minimizing detours for similar issues.
82+
- **Diagnose**:After KG editing, visual interface allows for quick debugging, and successful Execution path configurations will be automatically documented, which reduces model interactions, accelerates inference, and minimizes LLM Token costs. Additionally, during online execution, we provide comprehensive end-to-end visual monitoring.
83+
- **Memory**:Unified message pooling design supports categorized message delivery and subscription based on the needs of different scenarios, like multi-agent. Additionally, through message retrievel, rerank and distillation, it facilitates long-context handling, improving the overall question-answer quality.
84+
- **ActionSpace**:Adhering to Swagger protocol, we provide tool registration, tool categorization, and permission management, facilitating LLM Function Calling. We offer a secure and trustworthy code execution environment, and ensuring precise code generation to meet the demands of various scenarios, including visual plot, numerical calculations, and table editing.
11285

113-
## Contribution
86+
## 🤗 Contribution
11487
We are deeply grateful for your interest in the Codefuse project and warmly welcome any suggestions, opinions (including criticism), comments, and contributions.
11588

11689
Feel free to raise your suggestions, opinions, and comments directly through GitHub Issues. There are numerous ways to participate in and contribute to the Codefuse project: code implementation, writing tests, process tool improvements, documentation enhancements, etc.

README_zh.md

Lines changed: 33 additions & 71 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,12 @@
22
<a>中文</a>&nbsp | &nbsp<a href="README.md">English&nbsp </a>
33
</p>
44

5-
# <p align="center">CodeFuse-muAgent: A Multi-Agent FrameWork For Faster Build Agents</p>
5+
# <p align="center">CodeFuse-muAgent: An Innovative Agent Framework Driven By KG Engine</p>
66

77
<p align="center">
88
<a href="README_zh.md"><img src="https://img.shields.io/badge/文档-中文版-yellow.svg" alt="ZH doc"></a>
99
<a href="README.md"><img src="https://img.shields.io/badge/document-English-yellow.svg" alt="EN doc"></a>
10-
<img src="https://img.shields.io/github/license/codefuse-ai/CodeFuse-muAgent" alt="License">
10+
<a href="LICENSE.md"><img src="https://img.shields.io/badge/license-Apache%202.0-yellow" alt="License">
1111
<a href="https://github.com/codefuse-ai/CodeFuse-muAgent/issues">
1212
<img alt="Open Issues" src="https://img.shields.io/github/issues-raw/codefuse-ai/CodeFuse-muAgent" />
1313
</a>
@@ -18,104 +18,66 @@
1818

1919
## 🔔 更新
2020
- [2024.04.01] CodeFuse-muAgent 开源,支持知识库、代码库、工具使用、代码解释器等功能
21+
- [2024.09.05] 发布 muAgent v2.0 - EKG:一款由知识图谱引擎驱动的创新代理框架
2122

2223
## 📜 目录
2324
- [🤝 介绍](#-介绍)
2425
- [🚀 快速使用](#-快速使用)
2526
- [🧭 关键技术](#-关键技术)
27+
- [🤗 贡献指南](#-贡献指南)
2628
- [🗂 其他](#-其他)
2729
- [📱 联系我们](#-联系我们)
2830

2931

3032
## 🤝 介绍
31-
CodeFuse-muAgent 是蚂蚁CodeFuse团队开发的Mulit Agent框架,其核心宗旨在于简化agents的标准操作程序(SOP)编排流程。muagent整合了一系列丰富的工具库、代码库、知识库以及沙盒环境,可支撑用户在任何领域场景都能依托muagent迅速搭建起复杂的多Agent交互应用。通过这一框架,用户能够高效地执行和处理多层次、多维度的复杂任务
33+
全新体验的 Agent 框架,将KG从知识获取来源直接升级为Agent编排引擎!基于 LLM+ EKG(Eventic Knowledge Graph 行业知识承载)驱动,协同 MultiAgent、FunctionCall、CodeInterpreter等技术,通过画布式拖拽、轻文字编写,让大模型在人的经验指导下帮助你实现各类复杂 SOP 流程。兼容现有市面各类 Agent 框架,同时可实现复杂推理、在线协同、人工交互、知识即用四大核心差异技术功能。这套框架目前在蚂蚁集团内多个复杂DevOps场景落地验证,同时来体验下我们快速搭建的谁是卧底游戏吧
3234

33-
![](docs/resources/agent_runtime.png)
35+
36+
![](docs/resources/ekg-arch-zh.webp)
3437

3538

3639
## 🚀 快速使用
37-
完整文档见:[CodeFuse-muAgent](https://codefuse-ai.github.io/zh-CN/docs/api-docs/MuAgent/overview/multi-agent)
38-
更多[demo](https://codefuse-ai.github.io/zh-CN/docs/api-docs/MuAgent/connector/customed_examples)
40+
完整文档见:[CodeFuse-muAgent](https://codefuse.ai/zh-CN/docs/api-docs/MuAgent/overview/multi-agent)
41+
更多[demo](https://codefuse.ai/zh-CN/docs/api-docs/MuAgent/connector/customed_examples)
3942

40-
1. 安装
41-
```
42-
pip install codefuse-muagent
43-
```
43+
### EKG服务
4444

45-
2. code answer
45+
```bash
46+
# 使用我们的EKG服务只需要三步!(beta版本,需要将本地代码打包到容器中)
4647

47-
准备相关llm 和embedding model 配置
48-
```
49-
import os
50-
51-
# set your config
52-
api_key = ""
53-
api_base_url= ""
54-
model_name = ""
55-
embed_model = ""
56-
embed_model_path = ""
57-
58-
from muagent.llm_models.llm_config import EmbedConfig, LLMConfig
59-
from muagent.connector.phase import BasePhase
60-
from muagent.connector.schema import Message, Memory
61-
from muagent.codechat.codebase_handler.codebase_handler import CodeBaseHandler
62-
63-
llm_config = LLMConfig(
64-
model_name=model_name, api_key=api_key, api_base_url=api_base_url, temperature=0.3
65-
)
66-
67-
embed_config = EmbedConfig(
68-
embed_engine="model", embed_model=embed_model, embed_model_path=embed_model_path
69-
)
70-
```
48+
# 第一步. 加载代码
49+
git clone https://github.com/codefuse-ai/CodeFuse-muAgent.git
7150

72-
<br>
51+
# 第二步.
52+
cd CodeFuse-muAgent
7353

74-
初始化代码库
54+
# 第三步. 启动所有容器服务,EKG基础镜像构建需要花费点时间
55+
docker-compose up -d
7556
```
76-
# initialize codebase
77-
from muagent.base_configs.env_config import CB_ROOT_PATH
7857

79-
codebase_name = 'client_local'
80-
code_path = "D://chromeDownloads/devopschat-bot/client_v2/client"
58+
当前镜像版本仅包含了EKG基础服务。我们将会在9月底提供前端交互和后端交互的镜像服务。
8159

82-
cbh = CodeBaseHandler(
83-
codebase_name, code_path, crawl_type='dir', use_nh=use_nh,local_graph_path=CB_ROOT_PATH,
84-
llm_config=llm_config, embed_config=embed_config
85-
)
86-
cbh.import_code(do_interpret=do_interpret)
87-
```
88-
89-
<br>
60+
敬请期待!
9061

91-
开始代码库问答
62+
### SKD版本
63+
1. 安装
9264
```
93-
#
94-
phase_name = "codeChatPhase"
95-
phase = BasePhase(
96-
phase_name, embed_config=embed_config, llm_config=llm_config,
97-
)
98-
99-
query_content = "remove 可以做什么?"
100-
query = Message(
101-
role_name="user", role_type="human", input_query=query_content,
102-
code_engine_name=codebase_name, score_threshold=1.0, top_k=3, cb_search_type="tag",
103-
local_graph_path=CB_ROOT_PATH, use_nh=False
104-
)
105-
output_message3, output_memory3 = phase.step(query)
106-
print(output_memory3.to_str_messages(return_all=True, content_key="parsed_output_list"))
65+
pip install codefuse-muagent
10766
```
10867

68+
2. 代码问答和相关配置,可以看 [docs](https://codefuse.ai/docs/api-docs/MuAgent/connector/customed_examples) 和代码示例 [examples](https://github.com/codefuse-ai/CodeFuse-muAgent/tree/main/examples)
69+
70+
10971
## 🧭 关键技术
11072

111-
- Agent Base:构建了四种基本的Agent类型BaseAgent、ReactAgent、ExecutorAgent、SelectorAgent,支撑各种场景的基础活动
112-
- Communication:通过Message和Parse Message 实体完成Agent间的信息传递,并与Memory Manager交互再Memory Pool完成记忆管理
113-
- Prompt Manager:通过Role Handler、Doc/Tool Handler、Session Handler、Customized Handler,来自动化组装Customized 的Agent Prompt
114-
- Memory Manager: 用于支撑 chat history 的存储管理、信息压缩、记忆检索等管理,最后通过Memory Pool在数据库、本地、向量数据库中完成存储
115-
- Component:用于构建Agent的辅助生态组件,包括Retrieval、Tool、Action、Sandbox等
116-
- Customized Model:支持私有化的LLM和Embedding的接入
73+
- **图谱构建**:通过虚拟团队构建、场景意图划分,让你体验在线文档VS本地文档的差别;同时,文本语义输入的节点使用方式,让你感受有注释代码VS无注释代码的差别,充分体现在线协同的优势;面向海量存量文档(通用文本、流程画板等),支持文本智能解析、一键导入
74+
- **图谱资产**:通过场景意图、事件流程、统一工具、组织人物四部分的统一图谱设计,满足各类SOP场景所需知识承载;工具在图谱的纳入进一步提升工具选择、参数填充的准确性,人物/智能体在图谱的纳入,让人可加入流程的推进,可灵活应用于多人文本游戏
75+
- **图谱推理**:相比其他Agent框架纯模型推理、纯人工编排的推理模式,让大模型在人的经验/设计指导下做事,灵活、可控,同时面向未知局面,可自由探索,同时将成功探索经验总结、图谱沉淀,面向相似问题,少走弯路;整体流程唤起支持平台对接(规则配置)、语言触发,满足各类诉求
76+
- **调试运行**:图谱编辑完成后,可视调试,快速发现流程错误、修改优化,同时面向调试成功路径,关联配置自动沉淀,减少模型交互、模型开销,加速推理流程;此外,在线运行中,我们提供全链路可视化监控
77+
- **记忆管理**:统一消息池设计,支持各类场景所需分门别类消息投递、订阅,隔离且互通,便于多Agent场景消息管理使用;同时面向超长上下文,支持消息检索、排序、蒸馏,提升整体问答质量
78+
- **操作空间**:遵循Swagger协议,提供工具注册、权限管理、统一分类,方便LLM在工具调用中接入使用;提供安全可信代码执行环境,同时确保代码精准生成,满足可视绘图、数值计算、图表编辑等各类场景诉求
11779

118-
## 贡献指南
80+
## 🤗 贡献指南
11981
非常感谢您对 Codefuse 项目感兴趣,我们非常欢迎您对 Codefuse 项目的各种建议、意见(包括批评)、评论和贡献。
12082

12183
您对 Codefuse 的各种建议、意见、评论可以直接通过 GitHub 的 Issues 提出。

0 commit comments

Comments
 (0)