diff --git a/.cursor/rules/english_comments.mdc b/.cursor/rules/english_comments.mdc new file mode 100644 index 000000000..b40ebe110 --- /dev/null +++ b/.cursor/rules/english_comments.mdc @@ -0,0 +1,122 @@ +--- +description: English Comments Rule +alwaysApply: false +--- +# English Comments Rule + +## English-Only Comments Requirement + +All code comments MUST be written in English. No Chinese, Japanese, Korean, or other non-English languages are allowed in comments. + +### ✅ Correct Pattern: +```python +# Initialize the processor with default settings +processor = DataProcessor() + +# Cache inspector for 60 seconds +self._inspector_ttl = 60 + +# Ensure current application configuration is correct +if not self._validate_config(): + raise ConfigurationError("Invalid configuration") + +# Concurrently fetch all task details asynchronously +tasks = await self._fetch_all_tasks_async() + +# Lazy load CLIP model +self._clip_model = None +``` + +### ❌ Forbidden Pattern: +```python +# 初始化处理器 +processor = DataProcessor() + +# inspector缓存时间，秒 +self._inspector_ttl = 60 + +# 确保当前应用配置正确 +if not self._validate_config(): + raise ConfigurationError("Invalid configuration") + +# 并发异步获取所有任务详情 +tasks = await self._fetch_all_tasks_async() + +# 延迟加载CLIP模型 +self._clip_model = None +``` + +## Architecture Rules + +1. **English-Only Comments**: All comments must be written in English, regardless of the target audience or deployment region. + +2. **Code Documentation**: + - Function docstrings must be in English + - Inline comments must be in English + - TODO comments must be in English + - FIXME comments must be in English + +3. **Variable and Function Names**: + - Variable names can be in English or follow existing naming conventions + - Function names should be descriptive in English + - Class names should be in English + +4. **String Literals**: + - User-facing strings can be localized + - Error messages can be localized + - Comments and documentation must remain in English + +## File Structure Requirements + +- All `.py` files: English comments only +- All `.js`/`.ts` files: English comments only +- All `.md` files: English documentation only +- All `.yaml`/`.yml` files: English comments only + +## Migration Checklist + +When refactoring existing code: +1. ✅ Identify all non-English comments +2. ✅ Translate comments to English while preserving meaning +3. ✅ Update docstrings to English +4. ✅ Ensure inline comments are in English +5. ✅ Verify all TODO/FIXME comments are in English +6. ✅ Update any related documentation + +## Validation + +Before committing changes: +- No Chinese characters in comments +- No Japanese characters in comments +- No Korean characters in comments +- All comments are in English +- All docstrings are in English +- All TODO/FIXME comments are in English + +## Examples of Refactoring + +### Before: +```python +# 解析输入URL +def parse_url(url: str) -> str: + """解析并验证URL格式""" + # 检查URL是否有效 + if not url.startswith('http'): + raise ValueError("无效的URL格式") + return url +``` + +### After: +```python +# Parse input URL +def parse_url(url: str) -> str: + """Parse and validate URL format""" + # Check if URL is valid + if not url.startswith('http'): + raise ValueError("Invalid URL format") + return url +``` +description: +globs: +alwaysApply: false +--- diff --git a/.cursor/rules/environment_variable.mdc b/.cursor/rules/environment_variable.mdc new file mode 100644 index 000000000..6c4b2fb43 --- /dev/null +++ b/.cursor/rules/environment_variable.mdc @@ -0,0 +1,69 @@ +--- +description: Environment Variable Management Rules +alwaysApply: false +--- +# Environment Variable Management Rules + +## Environment Variable Access Pattern + +All environment variable access MUST go through `backend/consts/const.py`. Direct `os.getenv()` or `os.environ.get()` calls are NOT allowed in any other files. + +### ✅ Correct Pattern: +```python +# In backend/consts/const.py +APPID = os.getenv("APPID", "") +TOKEN = os.getenv("TOKEN", "") + +# In other files +from consts.const import APPID, TOKEN +``` + +### ❌ Forbidden Pattern: +```python +# Direct environment variable access - NOT ALLOWED +import os +appid = os.getenv("APPID") +token = os.environ.get("TOKEN") +``` + +## Architecture Rules + +1. **Single Source of Truth**: `backend/consts/const.py` is the ONLY place where `os.getenv()` or `os.environ.get()` should be called. + +2. **SDK Layer Dependencies**: SDK modules (in `sdk/` directory) should NOT directly access environment variables. Instead: + - Accept configuration parameters through constructors + - Remove `from_env()` methods from config classes + - Pass required values from higher-level modules + +3. **Configuration Classes**: + - Remove `@classmethod from_env()` methods + - Use `__init__()` with required parameters + - Keep reasonable defaults for technical configurations + +4. **Service Initialization**: Services should read from `const.py` and pass configurations to SDK modules. + +## File Structure Requirements + +- `backend/consts/const.py`: Centralized environment variable management +- `backend/apps/`: Import from `consts.const` and pass to SDK modules +- `backend/services/`: Import from `consts.const` for service configuration +- `sdk/`: Accept configuration through parameters, no direct env access + +## Migration Checklist + +When modifying environment variable access: +1. ✅ Add new variables to `backend/consts/const.py` +2. ✅ Update `.env.example` with new variables +3. ✅ Remove `os.getenv()` calls from target files +4. ✅ Import from `consts.const` in backend files +5. ✅ Pass configuration as parameters to SDK modules +6. ✅ Remove `from_env()` methods from config classes +7. ✅ Update service constructors to read from `const.py` + +## Validation + +Before committing changes: +- No `os.getenv()` calls outside `const.py` +- No `from_env()` methods in config classes +- All environment variables defined in `const.py` +- SDK modules accept configuration through parameters \ No newline at end of file diff --git a/.github/workflows/deploy-docs.yml b/.github/workflows/deploy-docs.yml new file mode 100644 index 000000000..39ddbca56 --- /dev/null +++ b/.github/workflows/deploy-docs.yml @@ -0,0 +1,84 @@ +# 构建 VitePress 站点并将其部署到 GitHub Pages 的工作流程 +# +name: Deploy VitePress Docs to Pages + +on: + # 在针对 `develop` 分支的推送上运行 + push: + branches: [develop] + # 只有当doc目录下的文件发生变化时才触发 + paths: + - 'doc/**' + + # 允许从 Actions 选项卡手动运行此工作流程 + workflow_dispatch: + +# 设置 GITHUB_TOKEN 的权限，以允许部署到 GitHub Pages +permissions: + contents: read + pages: write + id-token: write + +# 只允许同时进行一次部署，跳过正在运行和最新队列之间的运行队列 +# 但是，不要取消正在进行的运行，因为我们希望允许这些生产部署完成 +concurrency: + group: pages + cancel-in-progress: false + +jobs: + # 构建工作 + build: + runs-on: ubuntu-latest + defaults: + run: + working-directory: ./doc + steps: + - name: Checkout + uses: actions/checkout@v4 + with: + fetch-depth: 0 # 如果未启用 lastUpdated，则不需要 + + - name: Setup Node + uses: actions/setup-node@v4 + with: + node-version: 20 + cache: npm + cache-dependency-path: ./doc/package-lock.json + + - name: Setup Pages + uses: actions/configure-pages@v4 + + - name: Install dependencies + run: npm ci + + - name: Temporarily modify base path for GitHub Pages + run: | + # 创建一个临时的config文件，将base从'/doc/'改为'/nexent/'用于GitHub Pages + sed 's|base: '"'"'/doc/'"'"'|base: '"'"'/nexent/'"'"'|g' docs/.vitepress/config.mts > docs/.vitepress/config.github.mts + mv docs/.vitepress/config.mts docs/.vitepress/config.original.mts + mv docs/.vitepress/config.github.mts docs/.vitepress/config.mts + + - name: Build with VitePress + run: npm run docs:build + + - name: Restore original config + run: | + mv docs/.vitepress/config.original.mts docs/.vitepress/config.mts + + - name: Upload artifact + uses: actions/upload-pages-artifact@v3 + with: + path: doc/docs/.vitepress/dist + + # 部署工作 + deploy: + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + needs: build + runs-on: ubuntu-latest + name: Deploy + steps: + - name: Deploy to GitHub Pages + id: deployment + uses: actions/deploy-pages@v4 \ No newline at end of file diff --git a/AUTHORS b/AUTHORS index 792fc0ac6..570ffe1ea 100644 --- a/AUTHORS +++ b/AUTHORS @@ -1,8 +1,21 @@ Nexent Team: + +Team Manager / Product Manager: Shuangrui Chen @Phinease + +Senior System Engineer: Simeng Bian @Simeng Bian -Tao Liu @ liutao12138 +Tao Liu @liutao12138 + +Development Group Leader: Jingyuan Li @ljy65535 + +Developer: +Yichen Xia @Jasonxia007 Mingchen Wan @WMC001 -Yichen Xia @ Jasonxia007 -Yu Lin @ linsensen222 +Yu Lin @linsensen222 +Wenqi Bai @Bavichi +Feiyang Xiang @feixiangkong + +Operations Manager: +Chenxue Jia @Davina-jcx diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 02cc03cf9..146080442 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -5,7 +5,7 @@ Thank you for considering contributing to Nexent! From code to docs to sharing your experience, every bit helps make Nexent better for everyone. It also helps us if you share Nexent with others, or simply ⭐️ the repo. Thanks a million! 💛 Let's build something amazing together! 🎉 -In terms of licensing, please take a minute to read our short [License and Contributor Agreement](https://github.com/ModelEngine-Group/nexent/blob/main/LICENSE). The community also adheres to the [code of conduct](https://github.com/ModelEngine-Group/nexent/blob/main/CODE_OF_CONDUCT.md). +In terms of licensing, please take a minute to read our short [License and Contributor Agreement](https://modelengine-group.github.io/nexent/en/license). The community also adheres to the [code of conduct](https://modelengine-group.github.io/nexent/en/code-of-conduct). ## 🤔 How You Can Contribute @@ -86,7 +86,7 @@ git checkout -b your-branch-name ``` ### Step 4 Make Your Changes -🧙‍♂️ Code like a wizard! Follow our [Development Guide](./DEVELOPPER_NOTE.md) for setup instructions and coding standards. Ensure your changes are well-tested and documented. +🧙‍♂️ Code like a wizard! Follow our [Development Guide](https://modelengine-group.github.io/nexent/en/getting-started/development-guide) for setup instructions and coding standards. Ensure your changes are well-tested and documented. ### Step 5 Commit Your Changes 📝 Commit with a clear and concise message following our commit message guidelines： diff --git a/CONTRIBUTING_CN.md b/CONTRIBUTING_CN.md index 95abae7a7..b9d6ec6c9 100644 --- a/CONTRIBUTING_CN.md +++ b/CONTRIBUTING_CN.md @@ -5,7 +5,7 @@ 感谢您考虑为 Nexent 贡献力量！无论是代码、文档还是经验分享，您的每一份付出都能让 Nexent 变得更好。如果您愿意向他人推荐 Nexent 或在仓库点个 ⭐️，我们也非常感激。万分感谢！💛 让我们一起打造非凡之作！🎉 -关于许可证，请花一分钟阅读我们简短的[许可和贡献者协议](https://github.com/ModelEngine-Group/nexent/blob/main/LICENSE)。同时也请遵循[社区行为准则](https://github.com/ModelEngine-Group/nexent/blob/main/CODE_OF_CONDUCT.md)。 +关于许可证，请花一分钟阅读我们简短的[许可和贡献者协议](https://modelengine-group.github.io/nexent/zh/license)。同时也请遵循[社区行为准则](https://modelengine-group.github.io/nexent/zh/code-of-conduct)。 ## 🤔 如何贡献 @@ -89,7 +89,7 @@ git checkout -b 您的分支名 ``` ### 第四步：进行更改 -🧙‍♂️ 像魔法师一样编码！遵循我们的 [开发指南](./DEVELOPPER_NOTE_CN.md) 获取设置说明和编码标准。确保您的更改经过充分测试并有文档记录。 +🧙‍♂️ 像魔法师一样编码！遵循我们的 [开发指南](https://modelengine-group.github.io/nexent/zh/getting-started/development-guide) 获取设置说明和编码标准。确保您的更改经过充分测试并有文档记录。 ### 第五步：提交更改 📝 按照我们的提交消息规范，提交清晰简洁的消息（建议采用英文，让更多人理解你）： diff --git a/DEVELOPPER_NOTE.md b/DEVELOPPER_NOTE.md index 1ac77efbd..34313172d 100644 --- a/DEVELOPPER_NOTE.md +++ b/DEVELOPPER_NOTE.md @@ -338,6 +338,6 @@ docker builder prune -f && docker system prune -f 5. System prompts need thorough testing ### Getting Help 💬 -- Check the [FAQ](FAQ.md) +- Check Our [Documentation](https://modelengine-group.github.io/nexent/en/getting-started/overview) - Join our [Discord community](https://discord.gg/tb5H3S3wyv) - Submit [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) diff --git a/DEVELOPPER_NOTE_CN.md b/DEVELOPPER_NOTE_CN.md index b0a58d74b..d95e73049 100644 --- a/DEVELOPPER_NOTE_CN.md +++ b/DEVELOPPER_NOTE_CN.md @@ -344,6 +344,6 @@ docker builder prune -f && docker system prune -f 5. 系统提示词需充分测试 ### 获取帮助 💬 -- 查看 [常见问题](FAQ_CN.md) +- 查看 [我们的文档](https://modelengine-group.github.io/nexent/zh/getting-started/overview) - 加入 [Discord 社区](https://discord.gg/tb5H3S3wyv) - 提交 [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) diff --git a/README.md b/README.md index 2813aab6f..569ea25e3 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,7 @@ [![English](https://img.shields.io/badge/English-README-blue)](README.md) [![中文](https://img.shields.io/badge/中文-README-green)](README_CN.md) +[![Documentation](https://img.shields.io/badge/Documentation-yellow)](https://modelengine-group.github.io/nexent) [![codecov](https://codecov.io/gh/ModelEngine-Group/nexent/branch/develop/graph/badge.svg?token=00JM5YAD5V)](https://codecov.io/gh/ModelEngine-Group/nexent?branch=develop) Nexent is a zero-code platform for auto-generating agents — no orchestration, no complex drag-and-drop required, using pure language to develop any agent you want. Built on the MCP ecosystem with rich tool integration, Nexent also provides various built-in agents to meet your intelligent service needs in different scenarios such as work, travel, and daily life. Nexent offers powerful capabilities for agent running control, multi-agent collaboration, data processing and knowledge tracing, multimodal dialogue, and batch scaling. @@ -26,7 +27,7 @@ We have released **Nexent v1**, and the platform is now relatively stable. Howev > *Rome wasn't built in a day.* -If our vision speaks to you, jump in via the **[Contribution Guide](CONTRIBUTING.md)** and shape Nexent with us. +If our vision speaks to you, jump in via the **[Contribution Guide](https://modelengine-group.github.io/nexent/en/contributing)** and shape Nexent with us. Early contributors won't go unnoticed: from special badges and swag to other tangible rewards, we're committed to thanking the pioneers who help bring Nexent to life. @@ -53,105 +54,15 @@ bash deploy.sh When the containers are running, open **http://localhost:3000** in your browser and follow the setup wizard. -### 3. 🤖 Model Configuration & Provider Recommendations +# 🌱 MCP Tool Ecosystem -We recommend the following model providers: - -| Model Type | Provider | Notes | -|------------|----------|-------| -| LLM & VLLM | [Silicon Flow](https://siliconflow.cn/) | Free tier available | -| LLM & VLLM | [Alibaba Bailian](https://bailian.console.aliyun.com/) | Free tier available | -| Embedding | [Jina](https://jina.ai/) | Free tier available | -| TTS & STT | [Volcengine Voice](https://www.volcengine.com/product/voice-tech) | Free for personal use | -| Search | [EXA](https://exa.ai/) | Free tier available | - -You'll need to input the following information in the model configuration page: -- Base URL -- API Key -- Model Name - -The following configurations need to be added to your `.env` file (we'll make these configurable through the frontend soon): -- TTS and STT related configurations -- EXA search API Key - -> ℹ️ Due to core features development, currently, we only support Jina Embedding model. Support for other models will be added in future releases. For Jina API key setup, please refer to our [FAQ](FAQ.md). - -### 4. ❓ Need help? - -- Browse the [FAQ](FAQ.md) for common install issues. -- Drop questions in our [Discord community](https://discord.gg/tb5H3S3wyv). -- File bugs or feature ideas in [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues). - -### 5. 🔧 Hack on Nexent - -Want to build from source or add new features? Check the [Contribution Guide](CONTRIBUTING.md) for step-by-step instructions. - -### 6. 🛠️ Build from Source - -Prefer to run Nexent from source code? Follow our [Developer Guide](DEVELOPPER_NOTE.md) for detailed setup instructions and customization options. - -## 🌱 MCP Tool Ecosystem - -Nexent is built on the Model Context Protocol (MCP) tool ecosystem, providing a flexible and extensible framework for integrating various tools and services. MCP serves as the "USB-C of AI" - a universal interface standard that allows AI agents to seamlessly connect with external data sources, tools, and services. - -### 🌐 MCP Community Hub - -The global MCP ecosystem is thriving with multiple platforms supporting MCP development and deployment: - -| Platform | Description | Notes | -|----------|-------------|-------| -| **[GitHub MCP Server](https://github.com/github/github-mcp-server)** | Deep integration with Claude, GPT-4, Copilot etc., supports Go and Python | OAuth/GitHub account authorization | -| **[Qdrant MCP Vector Server](https://github.com/qdrant/mcp-server-qdrant)** | Semantic vector storage with Python/Go compatibility | Compatible with LangChain and other tools | -| **[Anthropic Reference MCP Servers](https://github.com/modelcontextprotocol/servers)** | Lightweight teaching and prototyping tools, Python | Includes fetch, git and other universal tools | -| **[AWS Labs MCP Server](https://github.com/awslabs/mcp)** | AWS+Go+CDK cloud reference services | Suitable for cloud environments | -| **[MCP Hub China](https://www.mcp-cn.com/)** | Chinese curated high-quality MCP service platform | Focuses on quality over quantity, community-driven | -| **[ModelScope MCP Marketplace](https://modelscope.cn/mcp)** | China's largest MCP community with 1,500+ services | From Amap to Alipay, comprehensive service coverage | -| **Community MCP Servers** | Various scenario-specific source code collection | Mostly experimental and innovative tools | - -### 🛠️ Recommended MCP Tools - -| Tool Name | Function | Description | -|-----------|----------|-------------| -| **[Amap Maps](https://modelscope.cn/mcp/servers/@amap/amap-maps)** | Geographic services and navigation | Comprehensive mapping, geocoding, routing, and location services | -| **[Bing Search (Chinese)](https://modelscope.cn/mcp/servers/@yan5236/bing-cn-mcp-server)** | Web search in Chinese | Optimized Chinese web search and information retrieval | -| **[12306 Train Ticket Query](https://modelscope.cn/mcp/servers/@Joooook/12306-mcp)** | China railway ticket booking | Real-time train schedules, ticket availability, and booking assistance | -| **[Alipay MCP](https://modelscope.cn/mcp/servers/@alipay/mcp-server-alipay)** | Payment and financial services | Digital payments, financial tools, and services integration | -| **[Variflight Aviation](https://modelscope.cn/mcp/servers/@variflight-ai/variflight-mcp)** | Flight information and aviation data | Real-time flight tracking, schedules, and aviation analytics | -| **[Sequential Thinking](https://modelscope.cn/mcp/servers/@modelcontextprotocol/sequentialthinking)** | Structured problem-solving framework | Break down complex problems into manageable, sequential steps | -| **[ArXiv AI Search](https://modelscope.cn/mcp/servers/@blazickjp/arxiv-mcp-server)** | Academic paper search and research | Advanced search and retrieval of scientific papers and research | -| **[Firecrawl MCP Server](https://modelscope.cn/mcp/servers/@mendableai/firecrawl-mcp-server)** | Web scraping and content extraction | Intelligent web scraping, data extraction, and content processing | +Check our [MCP Ecosystem page](https://modelengine-group.github.io/nexent/en/mcp-ecosystem/overview.html) for detailed information about the MCP tool ecosystem, including community hubs, recommended tools, and integration guides. ### 🚀 Suggested Agent Scenarios -With MCP's powerful ecosystem, you can create sophisticated AI agents for various scenarios: - -🌍 **Travel Planning Agent** -- Use Amap for route planning and navigation 📍 -- Integrate 12306 for train bookings 🚄 -- Connect Variflight for flight information ✈️ -- Enable Alipay for seamless payments 💳 - -🔬 **Research Assistant Agent** -- Leverage ArXiv search for academic papers 📚 -- Use Bing Search for comprehensive web research 🔍 -- Apply Sequential Thinking for structured analysis 🧠 -- Integrate Firecrawl for web data extraction 🕷️ +Check our [Agent Scenarios page](https://modelengine-group.github.io/nexent/en/mcp-ecosystem/use-cases.html) for detailed agent use cases and best practices, including travel planning, research assistant, business intelligence, smart lifestyle, and more scenarios. -💼 **Business Intelligence Agent** -- Connect multiple data sources through various MCP servers 📊 -- Use geographic tools for location-based insights 🗺️ -- Integrate payment systems for financial analysis 💰 -- Apply structured thinking frameworks for decision-making 🎯 - -🏠 **Smart Lifestyle Agent** -- Combine mapping services with payment integration 🛒 -- Use transportation tools for commute optimization 🚗 -- Integrate web search for local recommendations 🏪 -- Apply intelligent content extraction for information gathering 📱 - -The MCP ecosystem empowers you to build agents that can seamlessly interact with the real world, accessing live data, performing complex operations, and providing contextual assistance across virtually any domain. Each tool brings specialized capabilities that can be combined to create powerful, multi-functional AI experiences. - -## ✨ Key Features +# ✨ Key Features `1` **Smart agent prompt generation** Turn plain language into runnable prompts. Nexent automatically chooses the right tools and plans the best action path for every request. @@ -188,15 +99,29 @@ The MCP ecosystem empowers you to build agents that can seamlessly interact with ![Feature 7](./assets/Feature7.png) -# 🐛 Known Issues +# 🛠️ Developer Guide -1📝 **Code Output May Be Misinterpreted as Executable** - In Nexent conversations, if the model outputs code-like text, it may sometimes be misinterpreted as something that should be executed. We will fix this as soon as possible. +### 🤖 Model Configuration & Provider Recommendations +Check our [Model Providers page](https://modelengine-group.github.io/nexent/en/getting-started/model-providers.html) for detailed model configuration guides and recommended provider information. + +### 🔧 Hack on Nexent + +Want to build from source or add new features? Check the [Contribution Guide](https://modelengine-group.github.io/nexent/en/contributing) for step-by-step instructions. + +### 🛠️ Build from Source + +Prefer to run Nexent from source code? Follow our [Developer Guide](https://modelengine-group.github.io/nexent/en/getting-started/development-guide) for detailed setup instructions and customization options. + +# 🐛 Known Issues + +Check our [Known Issues page](https://modelengine-group.github.io/nexent/en/known-issues.html) for the latest issue status and solutions. # 💬 Community & contact -Join our [Discord community](https://discord.gg/tb5H3S3wyv) to chat with other developers and get help! +- Browse the [FAQ](https://modelengine-group.github.io/nexent/en/faq) for common install issues. +- Join our [Discord community](https://discord.gg/tb5H3S3wyv) to chat with other developers and get help! +- File bugs or feature ideas in [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues). # 📄 License diff --git a/README_CN.md b/README_CN.md index 08a9295c9..82d4c8587 100644 --- a/README_CN.md +++ b/README_CN.md @@ -2,6 +2,7 @@ [![English](https://img.shields.io/badge/English-README-blue)](README.md) [![中文](https://img.shields.io/badge/中文-README-green)](README_CN.md) +[![Documentation](https://img.shields.io/badge/Nexent-文档-yellow)](https://modelengine-group.github.io/nexent) [![codecov](https://codecov.io/gh/ModelEngine-Group/nexent/branch/develop/graph/badge.svg?token=00JM5YAD5V)](https://codecov.io/gh/ModelEngine-Group/nexent?branch=develop) Nexent 是一个零代码智能体自动生成平台 —— 无需编排，无需复杂的拖拉拽操作，使用纯语言开发你想要的任何智能体。基于MCP生态，具备丰富的工具集成，同时提供多种自带智能体，满足你的工作、旅行、生活等不同场景的智能服务需要。Nexent 还提供强大的智能体运行控制、多智能体协作、数据处理和知识溯源、多模态对话、批量扩展能力。 @@ -26,7 +27,7 @@ https://github.com/user-attachments/assets/b844e05d-5277-4509-9463-1c5b3516f11e > *Rome wasn't built in a day.* -如果我们的愿景与您产生共鸣，请通过 **[贡献指南](CONTRIBUTING_CN.md)** 加入我们，共同塑造 Nexent。 +如果我们的愿景与您产生共鸣，请通过 **[贡献指南](https://modelengine-group.github.io/nexent/zh/contributing)** 加入我们，共同塑造 Nexent。 早期贡献者不会被忽视：从特殊徽章和纪念品到其他实质性奖励，我们致力于感谢那些帮助 Nexent 诞生的先驱者。 @@ -53,105 +54,15 @@ bash deploy.sh 当容器运行后，在浏览器中打开 **http://localhost:3000** 并按照设置向导操作。 -### 3. 🤖 模型配置与模型提供商推荐 +# 🌱 MCP 工具生态 -我们建议使用以下模型提供商： - -| 模型类型 | 提供商 | 说明 | -|------------|----------|-------| -| LLM 与 VLLM | [硅基流动](https://siliconflow.cn/) | 提供免费额度 | -| LLM 与 VLLM | [阿里云百炼](https://bailian.console.aliyun.com/) | 提供免费额度 | -| Embedding | [Jina](https://jina.ai/) | 提供免费额度 | -| TTS 与 STT | [火山引擎语音](https://www.volcengine.com/product/voice-tech) | 个人用户免费 | -| 搜索 | [EXA](https://exa.ai/) | 提供免费额度 | - -您需要在模型配置页面输入以下信息： -- Base URL -- API Key -- Model Name - -以下配置需要添加到您的 `.env` 文件中（我们将尽快把这些配置前端化）： -- TTS 与 STT 相关配置 -- EXA 搜索 API Key - -> ℹ️ 由于开发紧张，目前我们仅支持 Jina Embedding 模型。其他模型的支持将在未来版本中添加。有关 Jina API 密钥获取，请参阅我们的[常见问题](FAQ_CN.md)。 - -### 4. ❓ 需要帮助？ - -- 浏览 [常见问题](FAQ_CN.md) 了解常见安装问题。 -- 在我们的 [Discord 社区](https://discord.gg/tb5H3S3wyv) 中提问。 -- 在 [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) 中提交错误报告或功能建议。 - -### 5. 🔧 开发 Nexent - -想要从源代码构建或添加新功能？查看 [贡献指南](CONTRIBUTING_CN.md) 获取分步说明。 - -### 6. 🛠️ 从源码构建 - -想要从源码运行 Nexent？查看我们的[开发者指南](DEVELOPPER_NOTE_CN.md)获取详细的设置说明和自定义选项。 - -## 🌱 MCP 工具生态 - -Nexent 基于模型上下文协议（MCP）工具生态系统构建，为集成各种工具和服务提供了灵活且可扩展的框架。MCP 被誉为"AI 的 USB-C" - 一个通用接口标准，让 AI 智能体能够无缝连接外部数据源、工具和服务。 - -### 🌐 MCP 社区中心 - -全球 MCP 生态系统蓬勃发展，多个平台支持 MCP 开发和部署： - -| 平台 | 描述 | 备注 | -|------|------|------| -| **[GitHub MCP Server](https://github.com/github/github-mcp-server)** | 深度集成 Claude、GPT-4、Copilot 等，支持 Go 和 Python | OAuth/GitHub 账户授权 | -| **[Qdrant MCP Vector Server](https://github.com/qdrant/mcp-server-qdrant)** | 语义向量存储，兼容 Python/Go | 与 LangChain 等工具兼容 | -| **[Anthropic Reference MCP Servers](https://github.com/modelcontextprotocol/servers)** | 轻量级教学与原型工具，Python | 包含 fetch、git 等通用工具 | -| **[AWS Labs MCP Server](https://github.com/awslabs/mcp)** | AWS+Go+CDK 云端参考服务 | 适合云环境 | -| **[MCP Hub 中国](https://www.mcp-cn.com/)** | 中国精选优质 MCP 服务平台 | 追求质量而非数量，社区驱动 | -| **[ModelScope MCP 广场](https://modelscope.cn/mcp)** | 中国最大的 MCP 社区，拥有 1,500+ 服务 | 从高德地图到支付宝，全面的服务覆盖 | -| **Community MCP Servers** | 各类场景源码聚集地 | 多为实验和创新工具 | - -### 🛠️ 推荐 MCP 工具 - -| 工具名称 | 功能 | 描述 | -|----------|------|------| -| **[高德地图](https://modelscope.cn/mcp/servers/@amap/amap-maps)** | 地理服务和导航 | 全面的地图、地理编码、路线规划和位置服务 | -| **[必应搜索中文](https://modelscope.cn/mcp/servers/@yan5236/bing-cn-mcp-server)** | 中文网络搜索 | 优化的中文网络搜索和信息检索 | -| **[12306车票查询工具](https://modelscope.cn/mcp/servers/@Joooook/12306-mcp)** | 中国铁路购票 | 实时列车时刻表、票务查询和订票助手 | -| **[支付宝MCP](https://modelscope.cn/mcp/servers/@alipay/mcp-server-alipay)** | 支付和金融服务 | 数字支付、金融工具和服务集成 | -| **[飞常准-Aviation](https://modelscope.cn/mcp/servers/@variflight-ai/variflight-mcp)** | 航班信息和航空数据 | 实时航班跟踪、时刻表和航空分析 | -| **[Sequential Thinking](https://modelscope.cn/mcp/servers/@modelcontextprotocol/sequentialthinking)** | 结构化问题解决框架 | 将复杂问题分解为可管理的连续步骤 | -| **[ArXiv AI搜索服务](https://modelscope.cn/mcp/servers/@blazickjp/arxiv-mcp-server)** | 学术论文搜索和研究 | 科学论文和研究的高级搜索和检索 | -| **[Firecrawl MCP 服务器](https://modelscope.cn/mcp/servers/@mendableai/firecrawl-mcp-server)** | 网页抓取和内容提取 | 智能网页抓取、数据提取和内容处理 | +查看我们的[MCP 生态系统页面](https://modelengine-group.github.io/nexent/zh/mcp-ecosystem/overview.html)了解 MCP 工具生态系统的详细信息，包括社区中心、推荐工具和集成指南。 ### 🚀 建议的智能体场景 -借助 MCP 强大的生态系统，您可以为各种场景创建复杂的 AI 智能体： - -🌍 **旅行规划智能体** -- 使用高德地图进行路线规划和导航 📍 -- 集成 12306 进行火车订票 🚄 -- 连接飞常准获取航班信息 ✈️ -- 启用支付宝实现无缝支付 💳 - -🔬 **研究助手智能体** -- 利用 ArXiv 搜索学术论文 📚 -- 使用必应搜索进行全面的网络研究 🔍 -- 应用 Sequential Thinking 进行结构化分析 🧠 -- 集成 Firecrawl 进行网络数据提取 🕷️ +查看我们的[智能体场景页面](https://modelengine-group.github.io/nexent/zh/mcp-ecosystem/use-cases.html)了解详细的智能体用例和最佳实践，包括旅行规划、研究助手、商业智能、智能生活等场景。 -💼 **商业智能智能体** -- 通过各种 MCP 服务器连接多个数据源 📊 -- 使用地理工具进行基于位置的洞察 🗺️ -- 集成支付系统进行财务分析 💰 -- 应用结构化思维框架进行决策制定 🎯 - -🏠 **智能生活智能体** -- 结合地图服务与支付集成 🛒 -- 使用交通工具优化通勤 🚗 -- 集成网络搜索获取本地推荐 🏪 -- 应用智能内容提取进行信息收集 📱 - -MCP 生态系统让您能够构建可以与现实世界无缝交互的智能体，访问实时数据、执行复杂操作，并在几乎任何领域提供上下文辅助。每个工具都带来了专门的能力，可以组合起来创建强大的、多功能的 AI 体验。 - -## ✨ 主要特性 +# ✨ 主要特性 `1` **智能体提示词自动生成** 将自然语言转化为可被Agent执行的提示词。Nexent可以根据你的需要自动选择正确的工具并为每个请求规划最佳执行路径。 @@ -188,16 +99,29 @@ MCP 生态系统让您能够构建可以与现实世界无缝交互的智能体 ![Feature 7](./assets/Feature7.png) -# 🐛 已知问题 +# 🛠️ 开发者指南 -1📝 **代码类输出可能被误认为可执行** - Nexent对话时如果模型输出代码类的文本，可能会被错误理解为需要被执行，我们会尽快修复。 +### 🤖 模型配置与模型提供商推荐 +查看我们的[模型提供商页面](https://modelengine-group.github.io/nexent/zh/getting-started/model-providers.html)了解详细的模型配置指南和推荐的提供商信息。 +### 🔧 开发 Nexent + +想要从源代码构建或添加新功能？查看 [贡献指南](https://modelengine-group.github.io/nexent/zh/contributing) 获取分步说明。 + +### 🛠️ 从源码构建 + +想要从源码运行 Nexent？查看我们的[开发者指南](https://modelengine-group.github.io/nexent/zh/getting-started/development-guide)获取详细的设置说明和自定义选项。 + +# 🐛 已知问题 + +查看我们的[已知问题页面](https://modelengine-group.github.io/nexent/zh/known-issues.html)了解最新的问题状态和解决方案。 # 💬 社区与联系方式 -加入我们的 [Discord 社区](https://discord.gg/tb5H3S3wyv) 与其他开发者交流并获取帮助！ +- 浏览 [常见问题](https://modelengine-group.github.io/nexent/zh/faq) 了解常见安装问题。 +- 加入我们的 [Discord 社区](https://discord.gg/tb5H3S3wyv) 与其他开发者交流并获取帮助！ +- 在 [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) 中提交错误报告或功能建议。 # 📄 许可证 diff --git a/backend/data_process_service.py b/backend/data_process_service.py index 0bb97d9ed..cd44ad398 100644 --- a/backend/data_process_service.py +++ b/backend/data_process_service.py @@ -140,6 +140,19 @@ def start_ray_cluster(self): except Exception as e: logger.debug(f"❌ Could not get cluster resources: {e}") + # Propagate Ray address to environment for child processes so that + # subsequently spawned worker processes can connect to the same Ray + # cluster without additional configuration. + try: + gcs_address = ray.get_runtime_context().gcs_address + if gcs_address: + os.environ["RAY_ADDRESS"] = gcs_address + # Store in config for potential later use + self.config['ray_address'] = gcs_address + logger.info(f"✅ RAY_ADDRESS environment variable set to {gcs_address}") + except Exception as e: + logger.debug(f"❌ Could not determine Ray address: {e}") + return True except Exception as e: diff --git a/backend/prompts/README.md b/backend/prompts/README.md deleted file mode 100644 index b866294c3..000000000 --- a/backend/prompts/README.md +++ /dev/null @@ -1,68 +0,0 @@ -# Prompt模板文件说明 - -本目录包含用于创建不同类型Agent的提示词模板文件。这些YAML文件定义了Agent的系统提示词、规划提示词和其他关键提示词组件。 - -## 文件命名规范 - -命名格式为`{agent_type}_agent.yaml`，其中： -- `agent_type`：描述Agent的主要功能或用途（如manager、search等） - -## 提示词模板结构 - -每个YAML文件包含以下主要部分： - -### 1. system_prompt - -系统提示词是Agent的核心部分，定义了Agent的角色、能力和行为规范。通常包含以下部分： - -- **核心职责**：Agent的主要职责和能力描述 -- **执行流程**：Agent执行任务的标准流程和方法 -- **可用资源**：Agent可以使用的工具和子Agent列表 -- **资源使用要求**：使用不同工具的优先级和策略 -- **Python代码规范**：编写代码的规范和约束 -- **示例模板**：展示Agent执行任务的示例 - -### 2. planning - -包含用于任务规划的各种提示词： - -- **initial_facts**：初始事实收集提示词 -- **initial_plan**：初始计划制定提示词 -- **update_facts_pre_messages**：更新事实前的提示词 -- **update_facts_post_messages**：更新事实后的提示词 -- **update_plan_pre_messages**：更新计划前的提示词 -- **update_plan_post_messages**：更新计划后的提示词 - -### 3. managed_agent - -定义与子Agent交互的提示词： - -- **task**：分配给子Agent的任务提示词 -- **report**：子Agent报告结果的提示词 - -### 4. final_answer - -定义最终答案生成的提示词： - -- **pre_messages**：生成最终答案前的提示词 -- **post_messages**：生成最终答案后的提示词 - -### 5. tools_requirement - -定义工具使用规范和优先级的提示词。 - -### 6. few_shots - -提供少样本学习示例的提示词，帮助Agent更好地理解任务执行方式。 - -## 模板变量 - -提示词模板中使用以下特殊变量进行动态替换： - -- `{{tools}}`：可用工具列表 -- `{{managed_agents}}`：可用子Agent列表 -- `{{task}}`：当前任务描述 -- `{{authorized_imports}}`：授权导入的Python模块 -- `{{facts_update}}`：更新后的事实列表 -- `{{answer_facts}}`：已知事实列表 -- `{{remaining_steps}}`：剩余执行步骤数 diff --git a/doc/docs/.vitepress/config.mts b/doc/docs/.vitepress/config.mts new file mode 100644 index 000000000..879be5ed5 --- /dev/null +++ b/doc/docs/.vitepress/config.mts @@ -0,0 +1,230 @@ +import { defineConfig } from 'vitepress' + +// https://vitepress.dev/reference/site-config +export default defineConfig({ + base: '/doc/', + title: "Nexent", + description: "A zero-code platform for auto-generating agents — no orchestration, no complex drag-and-drop required.", + + // Ignore localhost links as they are meant for local deployment access + ignoreDeadLinks: [ + // Ignore localhost links + /^http:\/\/localhost:3000/ + ], + + locales: { + root: { + label: 'English', + lang: 'en', + link: '/en/', + themeConfig: { + nav: [ + { text: 'Home', link: 'http://nexent.tech' }, + { text: 'Docs', link: '/en/getting-started/overview' } + ], + sidebar: [ + { + text: 'Getting Started', + items: [ + { text: 'Overview', link: '/en/getting-started/overview' }, + { text: 'Installation & Setup', link: '/en/getting-started/installation' }, + { text: 'Model Providers', link: '/en/getting-started/model-providers' }, + { text: 'Key Features', link: '/en/getting-started/features' }, + { text: 'Software Architecture', link: '/en/getting-started/software-architecture' }, + { text: 'Development Guide', link: '/en/getting-started/development-guide' }, + { text: 'FAQ', link: '/en/getting-started/faq' } + ] + }, + { + text: 'SDK Documentation', + items: [ + { text: 'SDK Overview', link: '/en/sdk/overview' }, + { text: 'Basic Usage', link: '/en/sdk/basic-usage' }, + { + text: 'Core Modules', + items: [ + { text: 'Tools', link: '/en/sdk/core/tools' }, + { text: 'Models', link: '/en/sdk/core/models' } + ] + }, + { text: 'Vector Database', link: '/en/sdk/vector-database' }, + { text: 'Data Processing', link: '/en/sdk/data-process' } + ] + }, + { + text: 'Frontend Development', + items: [ + { text: 'Frontend Overview', link: '/en/frontend/overview' } + ] + }, + { + text: 'Backend Development', + items: [ + { text: 'Backend Overview', link: '/en/backend/overview' }, + { text: 'API Reference', link: '/en/backend/api-reference' }, + { + text: 'Tools Integration', + items: [ + { text: 'LangChain Tools', link: '/en/backend/tools/langchain' }, + { text: 'MCP Tools', link: '/en/backend/tools/mcp' } + ] + }, + { text: 'Prompt Development', link: '/en/backend/prompt-development' } + ] + }, + { + text: 'AI Agents', + items: [ + { text: 'Agent Overview', link: '/en/agents/overview' } + ] + }, + { + text: 'Container Build & Containerized Development', + items: [ + { text: 'Docker Build', link: '/en/deployment/docker-build' }, + { text: 'Dev Container', link: '/en/deployment/devcontainer' } + ] + }, + { + text: 'MCP Ecosystem', + items: [ + { text: 'Overview', link: '/en/mcp-ecosystem/overview' }, + { text: 'Use Cases', link: '/en/mcp-ecosystem/use-cases' } + ] + }, + { + text: 'Testing', + items: [ + { text: 'Testing Overview', link: '/en/testing/overview' }, + { text: 'Backend Testing', link: '/en/testing/backend' } + ] + }, + { + text: 'Community', + items: [ + { text: 'Contributing', link: '/en/contributing' }, + { text: 'Code of Conduct', link: '/en/code-of-conduct' }, + { text: 'Security Policy', link: '/en/security' }, + { text: 'Core Contributors', link: '/en/contributors' }, + { text: 'Known Issues', link: '/en/known-issues' }, + { text: 'License', link: '/en/license' } + ] + } + ], + socialLinks: [ + { icon: 'github', link: 'https://github.com/ModelEngine-Group/nexent' } + ] + } + }, + zh: { + label: '简体中文', + lang: 'zh-CN', + link: '/zh/', + themeConfig: { + nav: [ + { text: '首页', link: 'http://nexent.tech' }, + { text: '文档', link: '/zh/getting-started/overview' } + ], + sidebar: [ + { + text: '快速开始', + items: [ + { text: '项目概览', link: '/zh/getting-started/overview' }, + { text: '安装与配置', link: '/zh/getting-started/installation' }, + { text: '模型提供商', link: '/zh/getting-started/model-providers' }, + { text: '核心特性', link: '/zh/getting-started/features' }, + { text: '软件架构', link: '/zh/getting-started/software-architecture' }, + { text: '开发指南', link: '/zh/getting-started/development-guide' }, + { text: '常见问题', link: '/zh/getting-started/faq' } + ] + }, + { + text: 'SDK 文档', + items: [ + { text: 'SDK 概览', link: '/zh/sdk/overview' }, + { text: '基本使用', link: '/zh/sdk/basic-usage' }, + { + text: '核心模块', + items: [ + { text: '工具模块', link: '/zh/sdk/core/tools' }, + { text: '模型模块', link: '/zh/sdk/core/models' } + ] + }, + { text: '向量数据库', link: '/zh/sdk/vector-database' }, + { text: '数据处理', link: '/zh/sdk/data-process' } + ] + }, + { + text: '前端开发', + items: [ + { text: '前端概览', link: '/zh/frontend/overview' } + ] + }, + { + text: '后端开发', + items: [ + { text: '后端概览', link: '/zh/backend/overview' }, + { text: 'API 文档', link: '/zh/backend/api-reference' }, + { + text: '工具集成', + items: [ + { text: 'LangChain 工具', link: '/zh/backend/tools/langchain' }, + { text: 'MCP 工具', link: '/zh/backend/tools/mcp' } + ] + }, + { text: '提示词开发', link: '/zh/backend/prompt-development' } + ] + }, + { + text: 'AI 智能体', + items: [ + { text: '智能体概览', link: '/zh/agents/overview' } + ] + }, + { + text: '容器构建与容器化开发', + items: [ + { text: 'Docker 构建', link: '/zh/deployment/docker-build' }, + { text: '开发容器', link: '/zh/deployment/devcontainer' } + ] + }, + { + text: 'MCP 生态系统', + items: [ + { text: '概览', link: '/zh/mcp-ecosystem/overview' }, + { text: '用例场景', link: '/zh/mcp-ecosystem/use-cases' } + ] + }, + { + text: '测试', + items: [ + { text: '测试概览', link: '/zh/testing/overview' }, + { text: '后端测试', link: '/zh/testing/backend' } + ] + }, + { + text: '社区', + items: [ + { text: '贡献指南', link: '/zh/contributing' }, + { text: '行为准则', link: '/zh/code-of-conduct' }, + { text: '安全政策', link: '/zh/security' }, + { text: '核心贡献者', link: '/zh/contributors' }, + { text: '已知问题', link: '/zh/known-issues' }, + { text: '许可证', link: '/zh/license' } + ] + } + ], + socialLinks: [ + { icon: 'github', link: 'https://github.com/ModelEngine-Group/nexent' } + ] + } + } + }, + + themeConfig: { + logo: '/Nexent Logo.jpg', + socialLinks: [ + { icon: 'github', link: 'https://github.com/ModelEngine-Group/nexent' } + ] + } +}) diff --git a/doc/docs/.vitepress/theme/index.ts b/doc/docs/.vitepress/theme/index.ts new file mode 100644 index 000000000..def4cfc87 --- /dev/null +++ b/doc/docs/.vitepress/theme/index.ts @@ -0,0 +1,17 @@ +// https://vitepress.dev/guide/custom-theme +import { h } from 'vue' +import type { Theme } from 'vitepress' +import DefaultTheme from 'vitepress/theme' +import './style.css' + +export default { + extends: DefaultTheme, + Layout: () => { + return h(DefaultTheme.Layout, null, { + // https://vitepress.dev/guide/extending-default-theme#layout-slots + }) + }, + enhanceApp({ app, router, siteData }) { + // ... + } +} satisfies Theme diff --git a/doc/docs/.vitepress/theme/style.css b/doc/docs/.vitepress/theme/style.css new file mode 100644 index 000000000..f5975ad7c --- /dev/null +++ b/doc/docs/.vitepress/theme/style.css @@ -0,0 +1,164 @@ +/** + * Customize default theme styling by overriding CSS variables: + * https://github.com/vuejs/vitepress/blob/main/src/client/theme-default/styles/vars.css + */ + +/** + * Colors + * + * Each colors have exact same color scale system with 3 levels of solid + * colors with different brightness, and 1 soft color. + * + * - `XXX-1`: The most solid color used mainly for colored text. It must + * satisfy the contrast ratio against when used on top of `XXX-soft`. + * + * - `XXX-2`: The color used mainly for hover state of the button. + * + * - `XXX-3`: The color for solid background, such as bg color of the button. + * It must satisfy the contrast ratio with pure white (#ffffff) text on + * top of it. + * + * - `XXX-soft`: The color used for subtle background such as custom container + * or badges. It must satisfy the contrast ratio when putting `XXX-1` colors + * on top of it. + * + * The soft color must be semi transparent alpha channel. This is crucial + * because it allows adding multiple "soft" colors on top of each other + * to create a accent, such as when having inline code block inside + * custom containers. + * + * - `default`: The color used purely for subtle indication without any + * special meanings attached to it such as bg color for menu hover state. + * + * - `brand`: Used for primary brand colors, such as link text, button with + * brand theme, etc. + * + * - `tip`: Used to indicate useful information. The default theme uses the + * brand color for this by default. + * + * - `warning`: Used to indicate warning to the users. Used in custom + * container, badges, etc. + * + * - `danger`: Used to show error, or dangerous message to the users. Used + * in custom container, badges, etc. + * -------------------------------------------------------------------------- */ + +:root { + --vp-c-default-1: var(--vp-c-gray-1); + --vp-c-default-2: var(--vp-c-gray-2); + --vp-c-default-3: var(--vp-c-gray-3); + --vp-c-default-soft: var(--vp-c-gray-soft); + + --vp-c-brand-1: var(--vp-c-indigo-1); + --vp-c-brand-2: var(--vp-c-indigo-2); + --vp-c-brand-3: var(--vp-c-indigo-3); + --vp-c-brand-soft: var(--vp-c-indigo-soft); + + --vp-c-tip-1: var(--vp-c-brand-1); + --vp-c-tip-2: var(--vp-c-brand-2); + --vp-c-tip-3: var(--vp-c-brand-3); + --vp-c-tip-soft: var(--vp-c-brand-soft); + + --vp-c-warning-1: var(--vp-c-yellow-1); + --vp-c-warning-2: var(--vp-c-yellow-2); + --vp-c-warning-3: var(--vp-c-yellow-3); + --vp-c-warning-soft: var(--vp-c-yellow-soft); + + --vp-c-danger-1: var(--vp-c-red-1); + --vp-c-danger-2: var(--vp-c-red-2); + --vp-c-danger-3: var(--vp-c-red-3); + --vp-c-danger-soft: var(--vp-c-red-soft); +} + +/** + * Component: Button + * -------------------------------------------------------------------------- */ + +:root { + --vp-button-brand-border: transparent; + --vp-button-brand-text: var(--vp-c-white); + --vp-button-brand-bg: var(--vp-c-brand-3); + --vp-button-brand-hover-border: transparent; + --vp-button-brand-hover-text: var(--vp-c-white); + --vp-button-brand-hover-bg: var(--vp-c-brand-2); + --vp-button-brand-active-border: transparent; + --vp-button-brand-active-text: var(--vp-c-white); + --vp-button-brand-active-bg: var(--vp-c-brand-1); +} + +/** + * Component: Home + * -------------------------------------------------------------------------- */ + +:root { + --vp-home-hero-name-color: transparent; + --vp-home-hero-name-background: -webkit-linear-gradient( + 120deg, + #bd34fe 30%, + #41d1ff + ); + + --vp-home-hero-image-background-image: linear-gradient( + -45deg, + #bd34fe 50%, + #47caff 50% + ); + --vp-home-hero-image-filter: blur(44px); +} + +@media (min-width: 640px) { + :root { + --vp-home-hero-image-filter: blur(56px); + } +} + +@media (min-width: 960px) { + :root { + --vp-home-hero-image-filter: blur(68px); + } +} + +/** + * Component: Custom Block + * -------------------------------------------------------------------------- */ + +:root { + --vp-custom-block-tip-border: transparent; + --vp-custom-block-tip-text: var(--vp-c-text-1); + --vp-custom-block-tip-bg: var(--vp-c-brand-soft); + --vp-custom-block-tip-code-bg: var(--vp-c-brand-soft); +} + +/** + * Component: Algolia + * -------------------------------------------------------------------------- */ + +.DocSearch { + --docsearch-primary-color: var(--vp-c-brand-1) !important; +} + +/** + * Component: Logo + * -------------------------------------------------------------------------- */ + +.VPNavBarTitle .logo { + border-radius: 8px; + padding: 0px; + background-color: rgba(255, 255, 255, 0.1); + transition: all 0.2s ease; +} + +.VPNavBarTitle .logo:hover { + background-color: rgba(255, 255, 255, 0.15); + transform: scale(1.4); +} + +/* Dark mode logo styling */ +.dark .VPNavBarTitle .logo { + background-color: rgba(255, 255, 255, 0.05); +} + +.dark .VPNavBarTitle .logo:hover { + background-color: rgba(255, 255, 255, 0.1); +} + diff --git a/doc/docs/api-examples.md b/doc/docs/api-examples.md new file mode 100644 index 000000000..6bd8bb5c1 --- /dev/null +++ b/doc/docs/api-examples.md @@ -0,0 +1,49 @@ +--- +outline: deep +--- + +# Runtime API Examples + +This page demonstrates usage of some of the runtime APIs provided by VitePress. + +The main `useData()` API can be used to access site, theme, and page data for the current page. It works in both `.md` and `.vue` files: + +```md + + +## Results + +### Theme Data +
{{ theme }}
+ +### Page Data +
{{ page }}
+ +### Page Frontmatter +
{{ frontmatter }}
+``` + + + +## Results + +### Theme Data +
{{ theme }}
+ +### Page Data +
{{ page }}
+ +### Page Frontmatter +
{{ frontmatter }}
+ +## More + +Check out the documentation for the [full list of runtime APIs](https://vitepress.dev/reference/runtime-api#usedata). diff --git a/doc/docs/assets/Feature1.png b/doc/docs/assets/Feature1.png new file mode 100644 index 000000000..a6d6ce400 Binary files /dev/null and b/doc/docs/assets/Feature1.png differ diff --git a/doc/docs/assets/Feature2.png b/doc/docs/assets/Feature2.png new file mode 100644 index 000000000..2b8d87b89 Binary files /dev/null and b/doc/docs/assets/Feature2.png differ diff --git a/doc/docs/assets/Feature3.png b/doc/docs/assets/Feature3.png new file mode 100644 index 000000000..6dd55cf36 Binary files /dev/null and b/doc/docs/assets/Feature3.png differ diff --git a/doc/docs/assets/Feature4.png b/doc/docs/assets/Feature4.png new file mode 100644 index 000000000..be1cc7779 Binary files /dev/null and b/doc/docs/assets/Feature4.png differ diff --git a/doc/docs/assets/Feature5.png b/doc/docs/assets/Feature5.png new file mode 100644 index 000000000..4d261b4c0 Binary files /dev/null and b/doc/docs/assets/Feature5.png differ diff --git a/doc/docs/assets/Feature6.png b/doc/docs/assets/Feature6.png new file mode 100644 index 000000000..6af25ee67 Binary files /dev/null and b/doc/docs/assets/Feature6.png differ diff --git a/doc/docs/assets/Feature7.png b/doc/docs/assets/Feature7.png new file mode 100644 index 000000000..daf5cc7ac Binary files /dev/null and b/doc/docs/assets/Feature7.png differ diff --git a/doc/docs/assets/Nexent Logo.jpg b/doc/docs/assets/Nexent Logo.jpg new file mode 100644 index 000000000..6de44bd4a Binary files /dev/null and b/doc/docs/assets/Nexent Logo.jpg differ diff --git a/doc/docs/assets/NexentBanner.png b/doc/docs/assets/NexentBanner.png new file mode 100644 index 000000000..05aea0e66 Binary files /dev/null and b/doc/docs/assets/NexentBanner.png differ diff --git a/doc/docs/assets/git-flow.png b/doc/docs/assets/git-flow.png new file mode 100644 index 000000000..43a826207 Binary files /dev/null and b/doc/docs/assets/git-flow.png differ diff --git a/doc/docs/assets/git-flow.svg b/doc/docs/assets/git-flow.svg new file mode 100644 index 000000000..d2de5fa1e --- /dev/null +++ b/doc/docs/assets/git-flow.svg @@ -0,0 +1,102 @@ +maindevelopfeature/feature1feature/feature2release/1.0.0hotfix/bug1InitializeFeature 1Feature 2Ready to release 1.0.0HotfixRelease 1.0.0 \ No newline at end of file diff --git a/doc/docs/en/agents/overview.md b/doc/docs/en/agents/overview.md new file mode 100644 index 000000000..cd33f6678 --- /dev/null +++ b/doc/docs/en/agents/overview.md @@ -0,0 +1,213 @@ +# AI Agent Development Overview + +Nexent provides a comprehensive framework for developing and deploying AI agents with advanced capabilities including tool integration, reasoning, and multi-modal interactions. + +## Agent Architecture + +### Core Components + +#### NexentAgent - Enterprise Agent Framework +The core of Nexent's agent system, providing complete intelligent agent solutions: + +- **Multi-model Support**: Supports OpenAI, vision language models, long-context models, etc. +- **MCP Integration**: Seamless integration with Model Context Protocol tool ecosystem +- **Dynamic Tool Loading**: Supports dynamic creation and management of local and MCP tools +- **Distributed Execution**: High-performance execution engine based on thread pools and async architecture +- **State Management**: Complete task state tracking and error recovery mechanisms + +#### CoreAgent - Code Execution Engine +Inherits and enhances SmolAgent's `CodeAgent`, providing the following key capabilities: + +- **Python Code Execution**: Supports parsing and executing Python code for dynamic task processing +- **Multi-language Support**: Built-in Chinese and English prompt templates, switchable as needed +- **Streaming Output**: Real-time streaming display of model output through MessageObserver +- **Step Tracking**: Records and displays each step of Agent execution for debugging and monitoring +- **Interrupt Control**: Supports task interruption and graceful stop mechanisms +- **Error Handling**: Complete error handling mechanisms to improve stability +- **State Management**: Maintains and passes execution state, supports continuous processing of complex tasks + +CoreAgent implements the ReAct framework's think-act-observe loop: +1. **Think**: Use large language models to generate solution code +2. **Act**: Execute the generated Python code +3. **Observe**: Collect execution results and logs +4. **Repeat**: Continue thinking and executing based on observation results until task completion + +### MessageObserver - Streaming Message Processing +Core implementation of the message observer pattern for handling Agent's streaming output: + +- **Streaming Output Capture**: Real-time capture of model-generated tokens +- **Process Type Distinction**: Format output based on different processing stages (model output, code parsing, execution logs, etc.) +- **Multi-language Support**: Supports Chinese and English output formats +- **Unified Interface**: Provides unified processing for messages from different sources + +ProcessType enumeration defines the following processing stages: +- `STEP_COUNT`: Current execution step +- `MODEL_OUTPUT_THINKING`: Model thinking process output +- `MODEL_OUTPUT_CODE`: Model code generation output +- `PARSE`: Code parsing results +- `EXECUTION_LOGS`: Code execution results +- `AGENT_NEW_RUN`: Agent basic information +- `FINAL_ANSWER`: Final summary results +- `SEARCH_CONTENT`: Search result content +- `PICTURE_WEB`: Web image processing results + +## Agent Development + +### Creating Basic Agents + +```python +from nexent.core import MessageObserver, ProcessType +from nexent.core.agents import CoreAgent, NexentAgent +from nexent.core.models import OpenAIModel +from nexent.core.tools import ExaSearchTool, KnowledgeBaseSearchTool + +# Create message observer +observer = MessageObserver() + +# Create model (model and Agent must use the same observer) +model = OpenAIModel( + observer=observer, + model_id="your-model-id", + api_key="your-api-key", + api_base="your-api-base" +) + +# Create tools +search_tool = ExaSearchTool(exa_api_key="your-exa-key", observer=observer, max_results=5) +kb_tool = KnowledgeBaseSearchTool(top_k=5, observer=observer) + +# Create Agent +agent = CoreAgent( + observer=observer, + tools=[search_tool, kb_tool], + model=model, + name="my_agent", + max_steps=5 +) + +# Run Agent +result = agent.run("Your question") +``` + +### Custom Agent Development + +#### System Prompt Templates +System prompt templates are located in `backend/prompts/`: + +- **knowledge_summary_agent.yaml**: Knowledge base summary agent +- **manager_system_prompt_template.yaml**: Manager system prompt template +- **utils/**: Prompt utilities + +#### Agent Implementation Steps + +1. **Create Agent Instance**: + ```python + from nexent.core.agents import CoreAgent + from nexent.core.models import OpenAIModel + + model = OpenAIModel( + model_id="your-model-id", + api_key="your-api-key", + api_base="your-api-base" + ) + agent = CoreAgent( + model=model, + tools=[your_tools], + system_prompt="Your system prompt" + ) + ``` + +2. **Configure Agent Behavior**: + - Add custom tools via `tools` parameter + - Set behavior via `system_prompt` + - Configure `max_steps`, `temperature`, and other parameters + +3. **Advanced Configuration**: + ```python + agent = CoreAgent( + model=model, + tools=custom_tools, + system_prompt=custom_prompt, + max_steps=10, + temperature=0.7, + verbose=True, + additional_authorized_imports=["requests", "pandas"] + ) + ``` + +## Tool Integration + +### Custom Tool Development + +Nexent is based on [Model Context Protocol (MCP)](https://github.com/modelcontextprotocol/python-sdk) for tool system implementation. + +#### Developing New Tools: +1. Implement logic in `backend/mcp_service/local_mcp_service.py` +2. Register with `@mcp.tool()` decorator +3. Restart MCP service + +#### Example: +```python +@mcp.tool(name="my_tool", description="My custom tool") +def my_tool(param1: str, param2: int) -> str: + # Implement tool logic + return f"Processing result: {param1} {param2}" +``` + +### Tool Development Guidelines + +For detailed tool development specifications and best practices, see: +- [Tool Development Guide](../sdk/core/tools) + +## Agent Execution Patterns + +### ReAct Pattern +The standard execution pattern for problem-solving agents: +1. **Reasoning**: Analyze the problem and plan approach +2. **Action**: Execute tools or generate code +3. **Observation**: Review results and outcomes +4. **Iteration**: Continue until task completion + +### Multi-Agent Collaboration +- **Hierarchical Agents**: Manager agents coordinating worker agents +- **Specialized Agents**: Domain-specific agents for particular tasks +- **Communication Protocols**: Standardized message passing between agents + +### Error Handling and Recovery +- **Graceful Degradation**: Fallback strategies when tools fail +- **State Persistence**: Saving agent state for recovery +- **Retry Mechanisms**: Automatic retry with backoff strategies + +## Performance Optimization + +### Execution Efficiency +- **Parallel Tool Execution**: Running independent tools concurrently +- **Caching Strategies**: Caching model responses and tool results +- **Resource Management**: Efficient memory and compute usage + +### Monitoring and Debugging +- **Execution Traces**: Detailed logging of agent decisions +- **Performance Metrics**: Timing and resource usage tracking +- **Debug Modes**: Verbose output for development + +## Best Practices + +### Agent Design +1. **Clear Objectives**: Define specific, measurable agent goals +2. **Appropriate Tools**: Select tools that match agent capabilities +3. **Robust Prompts**: Create comprehensive system prompts +4. **Error Handling**: Implement comprehensive error recovery + +### Development Workflow +1. **Iterative Development**: Build and test incrementally +2. **Prompt Engineering**: Refine prompts based on testing +3. **Tool Testing**: Validate individual tools before integration +4. **Performance Testing**: Monitor and optimize execution speed + +### Production Deployment +1. **Resource Allocation**: Ensure adequate compute resources +2. **Monitoring Setup**: Implement comprehensive logging and alerting +3. **Scaling Strategies**: Plan for increased load and usage +4. **Security Considerations**: Validate inputs and secure API access + +For detailed implementation examples and advanced patterns, see the [Development Guide](../getting-started/development-guide). \ No newline at end of file diff --git a/backend/apps/README.md b/doc/docs/en/backend/api-reference.md similarity index 99% rename from backend/apps/README.md rename to doc/docs/en/backend/api-reference.md index a4d97a6aa..2f3118433 100644 --- a/backend/apps/README.md +++ b/doc/docs/en/backend/api-reference.md @@ -1,8 +1,5 @@ # Nexent Community API Documentation -[![English](https://img.shields.io/badge/lang-English-blue.svg)](README.md) -[![简体中文](https://img.shields.io/badge/lang-简体中文-red.svg)](README_CN.md) - This document provides a comprehensive overview of all API endpoints available in the Nexent Community backend. ## Table of Contents diff --git a/doc/docs/en/backend/overview.md b/doc/docs/en/backend/overview.md new file mode 100644 index 000000000..9b6219182 --- /dev/null +++ b/doc/docs/en/backend/overview.md @@ -0,0 +1,203 @@ +# Backend Architecture Overview + +Nexent's backend is built with FastAPI and Python, providing a robust and scalable API platform for AI agent services. + +## Technology Stack + +- **Framework**: FastAPI +- **Language**: Python 3.10+ +- **Database**: PostgreSQL + Redis + Elasticsearch +- **File Storage**: MinIO +- **Task Queue**: Celery + Ray +- **AI Framework**: smolagents +- **Vector Database**: Elasticsearch + +## Directory Structure + +``` +backend/ +├── apps/ # API application layer +│ ├── base_app.py # FastAPI main application +│ ├── agent_app.py # Agent-related APIs +│ ├── conversation_management_app.py # Conversation management APIs +│ ├── file_management_app.py # File management APIs +│ ├── knowledge_app.py # Knowledge base APIs +│ ├── model_managment_app.py # Model management APIs +│ ├── config_sync_app.py # Configuration sync APIs +│ └── voice_app.py # Voice-related APIs +├── services/ # Business service layer +│ ├── agent_service.py # Agent business logic +│ ├── conversation_management_service.py # Conversation management +│ ├── elasticsearch_service.py # Search engine service +│ ├── model_health_service.py # Model health checks +│ ├── prompt_service.py # Prompt service +│ └── tenant_config_service.py # Tenant configuration service +├── database/ # Data access layer +│ ├── client.py # Database connections +│ ├── db_models.py # Database models +│ ├── agent_db.py # Agent data operations +│ ├── conversation_db.py # Conversation data operations +│ ├── knowledge_db.py # Knowledge base data operations +│ └── tenant_config_db.py # Tenant configuration data operations +├── agents/ # Agent core logic +│ ├── agent_run_manager.py # Agent execution manager +│ ├── create_agent_info.py # Agent information creation +│ └── default_agents/ # Default agent configurations +├── data_process/ # Data processing module +│ ├── app.py # Data processing application +│ ├── config.py # Data processing configuration +│ ├── tasks.py # Data processing tasks +│ ├── worker.py # Data processing worker +│ └── utils.py # Data processing utilities +├── utils/ # Utility classes +│ ├── auth_utils.py # Authentication utilities +│ ├── config_utils.py # Configuration utilities +│ ├── file_management_utils.py # File management utilities +│ ├── logging_utils.py # Logging utilities +│ └── thread_utils.py # Thread utilities +├── consts/ # Constants definition +│ ├── const.py # System constants +│ └── model.py # Data models +├── prompts/ # Prompt templates +│ ├── knowledge_summary_agent.yaml # Knowledge base summary agent +│ ├── manager_system_prompt_template.yaml # Manager system prompt +│ └── utils/ # Prompt utilities +├── sql/ # SQL scripts +├── assets/ # Backend resource files +├── main_service.py # Main service entry point +├── data_process_service.py # Data processing service entry point +└── requirements.txt # Python dependencies +``` + +## Architecture Responsibilities + +### **Application Layer (apps)** +- API route definitions +- Request parameter validation +- Response formatting +- Authentication and authorization + +### **Service Layer (services)** +- Core business logic implementation +- Data processing and transformation +- External service integration +- Business rule enforcement + +### **Data Layer (database)** +- Database operations and ORM models +- Data access interfaces +- Transaction management +- Data consistency and integrity + +### **Agent Layer (agents)** +- AI agent core logic and execution +- Tool calling and integration +- Reasoning and decision making +- Agent lifecycle management + +### **Utility Layer (utils)** +- Common utility functions +- Configuration management +- Logging and monitoring +- Thread and process management + +## Core Services + +### Agent Management +- Agent creation and configuration +- Execution lifecycle management +- Tool integration and calling +- Performance monitoring + +### Conversation Management +- Message handling and storage +- Context management +- History tracking +- Multi-tenant support + +### Knowledge Base +- Document processing and indexing +- Vector search and retrieval +- Content summarization +- Knowledge graph construction + +### File Management +- Multi-format file processing +- MinIO storage integration +- Batch processing capabilities +- Metadata extraction + +### Model Integration +- Multiple model provider support +- Health monitoring and failover +- Load balancing and caching +- Performance optimization + +## Data Flow Architecture + +### 1. User Request Flow +``` +User Input → Frontend Validation → API Call → Backend Routing → Business Service → Data Access → Database +``` + +### 2. AI Agent Execution Flow +``` +User Message → Agent Creation → Tool Calling → Model Inference → Streaming Response → Result Storage +``` + +### 3. Knowledge Base File Processing Flow +``` +File Upload → Temporary Storage → Data Processing → Vectorization → Knowledge Base Storage → Index Update +``` + +### 4. Real-time File Processing Flow +``` +File Upload → Temporary Storage → Data Processing → Agent → Response +``` + +## Deployment Architecture + +### Container Services +- **nexent**: Backend service (port 5010) +- **nexent-data-process**: Data processing service (port 5012) +- **nexent-postgresql**: Database (port 5434) +- **nexent-elasticsearch**: Search engine (port 9210) +- **nexent-minio**: Object storage (port 9010) +- **redis**: Cache service (port 6379) + +### Optional Services +- **nexent-openssh-server**: SSH server for Terminal tool (port 2222) + +## Development Setup + +### Environment Setup +```bash +cd backend +uv sync && uv pip install -e ../sdk +``` + +### Service Startup +```bash +python backend/data_process_service.py # Data processing service +python backend/main_service.py # Main service +python backend/nexent_mcp_service.py # MCP service +``` + +## Performance and Scalability + +### Async Architecture +- Based on asyncio for high-performance async processing +- Thread-safe concurrent processing mechanisms +- Optimized for distributed task queues + +### Caching Strategy +- Multi-layer caching for improved response speed +- Redis for session and temporary data +- Elasticsearch for search result caching + +### Load Balancing +- Intelligent concurrent limiting +- Resource pool management +- Auto-scaling capabilities + +For detailed backend development guidelines, see the [Development Guide](../getting-started/development-guide). \ No newline at end of file diff --git a/doc/docs/en/backend/prompt-development.md b/doc/docs/en/backend/prompt-development.md new file mode 100644 index 000000000..22fb27b85 --- /dev/null +++ b/doc/docs/en/backend/prompt-development.md @@ -0,0 +1,154 @@ +# Prompt Development Guide + +This guide provides comprehensive information about the prompt template system used in Nexent for creating different types of agents. The YAML files in the `backend/prompts/` directory define system prompts, planning prompts, and other key prompt components for various agent types. + +## File Naming Convention + +The naming format follows `{agent_type}_agent.yaml`, where: +- `agent_type`: Describes the main function or purpose of the agent (e.g., manager, search, etc.) + +## Prompt Template Structure + +Each YAML file contains the following main sections: + +### 1. system_prompt + +The system prompt is the core component of an agent, defining its role, capabilities, and behavioral guidelines. It typically includes: + +- **Core Responsibilities**: Main duties and capabilities description +- **Execution Flow**: Standard process and methods for agent task execution +- **Available Resources**: List of tools and sub-agents the agent can use +- **Resource Usage Requirements**: Priority and strategy for using different tools +- **Python Code Standards**: Code writing standards and constraints +- **Example Templates**: Examples demonstrating agent task execution + +### 2. planning + +Contains various prompts for task planning: + +- **initial_facts**: Initial fact collection prompts +- **initial_plan**: Initial plan formulation prompts +- **update_facts_pre_messages**: Prompts before updating facts +- **update_facts_post_messages**: Prompts after updating facts +- **update_plan_pre_messages**: Prompts before updating plans +- **update_plan_post_messages**: Prompts after updating plans + +### 3. managed_agent + +Defines prompts for sub-agent interactions: + +- **task**: Task assignment prompts for sub-agents +- **report**: Prompts for sub-agent result reporting + +### 4. final_answer + +Defines prompts for final answer generation: + +- **pre_messages**: Prompts before generating final answers +- **post_messages**: Prompts after generating final answers + +### 5. tools_requirement + +Defines prompts for tool usage standards and priorities. + +### 6. few_shots + +Provides few-shot learning examples to help agents better understand task execution methods. + +## Template Variables + +The prompt templates use the following special variables for dynamic replacement: + +- `{{tools}}`: Available tools list +- `{{managed_agents}}`: Available sub-agents list +- `{{task}}`: Current task description +- `{{authorized_imports}}`: Authorized Python module imports +- `{{facts_update}}`: Updated facts list +- `{{answer_facts}}`: Known facts list +- `{{remaining_steps}}`: Remaining execution steps + +## Available Prompt Templates + +### Core Templates + +1. **Manager Agent Templates** + - `manager_system_prompt_template.yaml` - Chinese version + - `manager_system_prompt_template_en.yaml` - English version + + These templates define the core manager agent that coordinates and dispatches various assistants and tools to efficiently solve complex tasks. + +2. **Managed Agent Templates** + - `managed_system_prompt_template.yaml` - Chinese version + - `managed_system_prompt_template_en.yaml` - English version + + These templates define specialized agents that perform specific tasks under the coordination of the manager agent. + +3. **Specialized Agent Templates** + - `knowledge_summary_agent.yaml` - Knowledge summary agent (Chinese) + - `knowledge_summary_agent_en.yaml` - Knowledge summary agent (English) + - `analyze_file.yaml` - File analysis agent (Chinese) + - `analyze_file_en.yaml` - File analysis agent (English) + +### Utility Templates + +Located in the `utils/` directory: + +1. **Prompt Generation Templates** + - `prompt_generate.yaml` - Chinese version + - `prompt_generate_en.yaml` - English version + + These templates help generate efficient and clear prompts for different agent types. + +2. **Prompt Fine-tuning Templates** + - `prompt_fine_tune.yaml` - Chinese version + - `prompt_fine_tune_en.yaml` - English version + + Templates for fine-tuning and optimizing existing prompts. + +3. **Title Generation Templates** + - `generate_title.yaml` - For generating titles and summaries + +## Execution Flow + +The standard agent execution flow follows this pattern: + +1. **Think**: Analyze current task status and progress +2. **Code**: Write simple Python code following standards +3. **Observe**: View code execution results +4. **Repeat**: Continue the cycle until the task is complete + +## Code Standards + +When writing Python code in prompts: + +1. Use the format `代码：\n```py\n` for executable code +2. Use the format `代码：\n```code:language_type\n` for display-only code +3. Use only defined variables that persist across calls +4. Use `print()` function to make variable information visible +5. Use keyword parameters for tool and agent calls +6. Avoid excessive tool calls in a single round +7. Only import from authorized modules: `{{authorized_imports}}` + +## Best Practices + +1. **Task Decomposition**: Break complex tasks into manageable sub-tasks +2. **Professional Matching**: Assign tasks based on agent expertise +3. **Information Integration**: Integrate outputs from different agents +4. **Efficiency Optimization**: Avoid redundant work +5. **Result Evaluation**: Assess agent return results and provide additional guidance when needed + +## Example Usage + +Here's an example of how a manager agent might coordinate with specialized agents: + +```yaml +# Example task assignment +managed_agent: + task: | + Please analyze the provided document and extract key insights. + + report: | + {{final_answer}} +``` + +This system allows for flexible and powerful agent coordination while maintaining clear standards and best practices for prompt development. \ No newline at end of file diff --git a/doc/docs/en/backend/tools/index.md b/doc/docs/en/backend/tools/index.md new file mode 100644 index 000000000..7076fce0a --- /dev/null +++ b/doc/docs/en/backend/tools/index.md @@ -0,0 +1,31 @@ +# Backend Tools Documentation + +This section covers the tool ecosystem available in Nexent's backend infrastructure. + +## Available Tool Categories + +### LangChain Tools +Integrate with the LangChain ecosystem for advanced AI workflows. +→ [LangChain Tools Guide](./langchain) + +### MCP Tools +Model Context Protocol tools for standardized AI agent communication. +→ [MCP Tools Development](./mcp) + +## Quick Start + +1. **Choose your tool type**: LangChain for general AI workflows, MCP for standardized agent communication +2. **Follow the integration guide**: Each tool type has specific setup requirements +3. **Test your integration**: Use the provided examples to validate your setup +4. **Build your agents**: Combine tools to create powerful AI agents + +## SDK Integration + +For SDK-level tool development, see: +→ [SDK Tools Documentation](../../sdk/core/tools) + +## Need Help? + +- Check our [FAQ](../../getting-started/faq) for common tool integration issues +- Join our [Discord community](https://discord.gg/tb5H3S3wyv) for real-time support +- Review [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) for known issues \ No newline at end of file diff --git a/backend/tool_collection/langchain/README.md b/doc/docs/en/backend/tools/langchain.md similarity index 100% rename from backend/tool_collection/langchain/README.md rename to doc/docs/en/backend/tools/langchain.md diff --git a/backend/tool_collection/mcp/README.md b/doc/docs/en/backend/tools/mcp.md similarity index 100% rename from backend/tool_collection/mcp/README.md rename to doc/docs/en/backend/tools/mcp.md diff --git a/doc/docs/en/code-of-conduct.md b/doc/docs/en/code-of-conduct.md new file mode 100644 index 000000000..906d04740 --- /dev/null +++ b/doc/docs/en/code-of-conduct.md @@ -0,0 +1,133 @@ + +# Contributor Covenant Code of Conduct + +## Our Pledge + +We as members, contributors, and leaders pledge to make participation in our +community a harassment-free experience for everyone, regardless of age, body +size, visible or invisible disability, ethnicity, sex characteristics, gender +identity and expression, level of experience, education, socio-economic status, +nationality, personal appearance, race, caste, color, religion, or sexual +identity and orientation. + +We pledge to act and interact in ways that contribute to an open, welcoming, +diverse, inclusive, and healthy community. + +## Our Standards + +Examples of behavior that contributes to a positive environment for our +community include: + +* Demonstrating empathy and kindness toward other people +* Being respectful of differing opinions, viewpoints, and experiences +* Giving and gracefully accepting constructive feedback +* Accepting responsibility and apologizing to those affected by our mistakes, + and learning from the experience +* Focusing on what is best not just for us as individuals, but for the overall + community + +Examples of unacceptable behavior include: + +* The use of sexualized language or imagery, and sexual attention or advances of + any kind +* Trolling, insulting or derogatory comments, and personal or political attacks +* Public or private harassment +* Publishing others' private information, such as a physical or email address, + without their explicit permission +* Other conduct which could reasonably be considered inappropriate in a + professional setting + +## Enforcement Responsibilities + +Community leaders are responsible for clarifying and enforcing our standards of +acceptable behavior and will take appropriate and fair corrective action in +response to any behavior that they deem inappropriate, threatening, offensive, +or harmful. + +Community leaders have the right and responsibility to remove, edit, or reject +comments, commits, code, wiki edits, issues, and other contributions that are +not aligned to this Code of Conduct, and will communicate reasons for moderation +decisions when appropriate. + +## Scope + +This Code of Conduct applies within all community spaces, and also applies when +an individual is officially representing the community in public spaces. +Examples of representing our community include using an official email address, +posting via an official social media account, or acting as an appointed +representative at an online or offline event. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be +reported to the community leaders responsible for enforcement at +[chenshuangrui@huawei.com]. +All complaints will be reviewed and investigated promptly and fairly. + +All community leaders are obligated to respect the privacy and security of the +reporter of any incident. + +## Enforcement Guidelines + +Community leaders will follow these Community Impact Guidelines in determining +the consequences for any action they deem in violation of this Code of Conduct: + +### 1. Correction + +**Community Impact**: Use of inappropriate language or other behavior deemed +unprofessional or unwelcome in the community. + +**Consequence**: A private, written warning from community leaders, providing +clarity around the nature of the violation and an explanation of why the +behavior was inappropriate. A public apology may be requested. + +### 2. Warning + +**Community Impact**: A violation through a single incident or series of +actions. + +**Consequence**: A warning with consequences for continued behavior. No +interaction with the people involved, including unsolicited interaction with +those enforcing the Code of Conduct, for a specified period of time. This +includes avoiding interactions in community spaces as well as external channels +like social media. Violating these terms may lead to a temporary or permanent +ban. + +### 3. Temporary Ban + +**Community Impact**: A serious violation of community standards, including +sustained inappropriate behavior. + +**Consequence**: A temporary ban from any sort of interaction or public +communication with the community for a specified period of time. No public or +private interaction with the people involved, including unsolicited interaction +with those enforcing the Code of Conduct, is allowed during this period. +Violating these terms may lead to a permanent ban. + +### 4. Permanent Ban + +**Community Impact**: Demonstrating a pattern of violation of community +standards, including sustained inappropriate behavior, harassment of an +individual, or aggression toward or disparagement of classes of individuals. + +**Consequence**: A permanent ban from any sort of public interaction within the +community. + +## Attribution + +This Code of Conduct is adapted from the [Contributor Covenant][homepage], +version 2.1, available at +[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1]. + +Community Impact Guidelines were inspired by +[Mozilla's code of conduct enforcement ladder][Mozilla CoC]. + +For answers to common questions about this code of conduct, see the FAQ at +[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at +[https://www.contributor-covenant.org/translations][translations]. + +[homepage]: https://www.contributor-covenant.org +[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html +[Mozilla CoC]: https://github.com/mozilla/diversity +[FAQ]: https://www.contributor-covenant.org/faq +[translations]: https://www.contributor-covenant.org/translations diff --git a/doc/docs/en/contributing.md b/doc/docs/en/contributing.md new file mode 100644 index 000000000..ab170010a --- /dev/null +++ b/doc/docs/en/contributing.md @@ -0,0 +1,169 @@ +# Nexent Contributing Guide + +Thank you for considering contributing to Nexent! From code to docs to sharing your experience, every bit helps make Nexent better for everyone. It also helps us if you share Nexent with others, or simply ⭐️ the repo. Thanks a million! 💛 Let's build something amazing together! 🎉 + +In terms of licensing, please take a minute to read our short [License and Contributor Agreement](https://github.com/ModelEngine-Group/nexent/blob/main/LICENSE). The community also adheres to the [code of conduct](https://github.com/ModelEngine-Group/nexent/blob/main/CODE_OF_CONDUCT.md). + +## 🤔 How You Can Contribute + +### 🐛 Found a Bug? + +If you've discovered a bug, please let us know! Your keen eye helps us improve Nexent for all users. + +### 💡 Have a Feature Idea? + +Got a brilliant idea to enhance Nexent? We'd love to hear it! Share your vision with us. + +### 💻 Want to Submit Code? + +Whether it's fixing a bug or adding a new feature, your code contributions are highly valued. + +### 📖 Want to Improve Documentation? + +Great documentation is key to a great project. Help us make Nexent easier to use and understand. + +## 🌳 Branching Strategy GitFlow + +![GitFlow Workflow](../assets/git-flow.svg) + +Gitflow is a branching model for Git that provides a structured approach to software development. It defines specific branches for different purposes, like features, releases, and hotfixes, and outlines how they should interact. This helps streamline the development process, manage releases effectively, and facilitate collaboration. + +### Main Branches +- **main**: Represents the official release history and should always be deployable. +- **develop**: The main branch for ongoing development. It integrates new features and bug fixes from feature branches. + +### Supporting Branches +- **feature branches**: Used for developing new features. They branch off from develop and are merged back into it once the feature is complete. +- **release branches**: Created when a new release is about to be prepared. They allow for final testing and minor adjustments before merging into main and develop. +- **hotfix branches**: Used for fixing critical bugs in production. They branch off from main and are merged back into both main and develop. + +### Benefits of Gitflow +- **Structured workflow**: Provides a clear and consistent process for managing different types of changes. +- **Improved collaboration**: Facilitates teamwork by defining clear roles for branches and how they should interact. +- **Efficient releases**: Streamlines the release process by isolating changes in dedicated branches and allowing for final testing. +- **Reduced conflicts**: By using feature branches and release branches, it helps minimize merge conflicts and makes it easier to resolve them. + +For a visual overview, see the diagram above. + +## 🐞 Submitting a Bug Report or Feature Request + +### Bug Reports +To help us quickly understand and fix the issue, please include: +- A **clear title** describing the bug. +- A **detailed description** of the issue, including steps to reproduce it. +- Any **error messages** or logs (if applicable). +- Expected behavior vs. actual behavior. +- Screenshots or screen recordings (if helpful). + +### Feature Requests +For feature ideas, please provide: +- A **clear title** summarizing the feature. +- A **detailed description** of the feature and its benefits. +- Any relevant **use cases** or examples. +- Screenshots or mockups (if applicable). + +**Where to submit?** +Open a new issue in our [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) section and select the appropriate template (Bug Report or Feature Request). + +## 💻 Submitting Code Changes + +### Step 1 Fork the Repository +🍴 Fork the [Nexent repository](https://github.com/ModelEngine-Group/nexent) to your GitHub account. + +### Step 2 Clone Your Fork +📦 Clone your forked repository locally: +```bash +git clone https://github.com/ModelEngine-Group/nexent.git +``` + +### Step 3 Create a Branch +🌿 Create a new branch for your changes: +```bash +git checkout -b your-branch-name +``` + +### Step 4 Make Your Changes +🧙‍♂️ Code like a wizard! Follow our [Development Guide](./getting-started/development-guide) for setup instructions and coding standards. Ensure your changes are well-tested and documented. + +### Step 5 Commit Your Changes +📝 Commit with a clear and concise message following our commit message guidelines： + +| Type | Icon | Description | +|-----------|------|-------------| +| Refactor | ♻️ | Code logic optimization without affecting functionality | +| Migration | 🚚 | Moving or migrating files or modules | +| Feature | ✨ | Adding new features or functionality | +| Bugfix | 🐛 | Fixing issues or bugs | +| Style | 🎨 | Improving code style, formatting without changing functionality | +| Chore | 🔨 | Updating tools, adjusting configurations | +| Docs | 📝 | Documentation changes only | +| Test | 🧪 | Add test cases or modify test cases | + +Example commit message： +```bash +git commit -m "✨ add user authentication" +git commit -m "🐛 resolve login timeout issue" +git commit -m "📝 update API documentation" +``` + +### Step 6 Sync with Upstream +⚙️ Keep your fork updated with the latest changes from the main repository: +```bash +git remote add upstream https://github.com/ModelEngine-Group/nexent.git +git fetch upstream +git merge upstream/main +``` + +### Step 7 Open a Pull Request (PR) +🚀 Push your changes to your fork and open a PR in the main repository. Include: +- A **clear title** and **description** of your changes. +- A reference to the related issue (e.g., `fixes #123`). +- Any additional context or screenshots. + +Our team will review your PR and provide feedback. Collaboration makes the magic happen! ✨ + +### Protected Branches and Code Owner Reviews + +When submitting changes to protected branches (like `main`), please note the following requirements： + +1. **Code Owner Review Required** + - The PR will automatically request reviews from relevant code owners + - You cannot approve your own PR + - Code owner approval is mandatory + +2. **Multiple Approvals Required** + - At least 2 approvals are required (including code owner approval) + - All CI checks must pass (lint, test, build, etc.) + +3. **Merge Process** + - After submitting the PR, the system will automatically request code owner reviews + - At least two approvals (including code owner) are needed + - The "Merge" button will only become available after all requirements are met + +4. **Restrictions** + - You cannot bypass reviews or force merge + - Direct pushes to protected branches are not allowed + - Self-approvals are not valid + +## 📖 Improving Documentation + +Great documentation is a team effort! You can help by： +- Fixing typos or clarifying unclear sections. +- Adding missing documentation for features or setup steps. +- Translating docs into other languages. + +To contribute： +1. Follow the same steps as for code changes (fork, clone, branch, etc.). +2. Edit the relevant documentation files (e.g., `README.md`, `docs/`). +3. Submit a PR with your improvements. + +## ❓ Need Help? + +Stuck or have questions? We're here to help! Reach out to us via： +- **GitHub Issues**: Open an issue for discussion. +- **Discord**: Join our [Nexent Community](https://discord.gg/YXH5C8SQ) for real-time chat. +- **Email**: Drop us a line at [chenshuangrui@huawei.com](mailto:chenshuangrui@huawei.com). + +## 🎉 Celebrate Your Contribution! + +Thank you for being part of the Nexent journey. Your contributions make a real difference, and we can't wait to see what you create! Happy coding! 🚀🌈 \ No newline at end of file diff --git a/doc/docs/en/contributors.md b/doc/docs/en/contributors.md new file mode 100644 index 000000000..0a4ae7734 --- /dev/null +++ b/doc/docs/en/contributors.md @@ -0,0 +1,45 @@ +# Core Contributors + +The Nexent project is made possible by the dedicated work of our core team members. We would like to acknowledge their contributions and expertise. + +## Nexent Team + +### Team Manager / Product Manager +- **Shuangrui Chen** @Phinease + +### Senior System Engineer +- **Simeng Bian** @Simeng Bian +- **Tao Liu** @liutao12138 + +### Development Group Leader +- **Jingyuan Li** @ljy65535 + +### Developer +- **Yichen Xia** @Jasonxia007 +- **Mingchen Wan** @WMC001 +- **Yu Lin** @linsensen222 +- **Wenqi Bai** @Bavichi +- **Feiyang Xiang** @feixiangkong + +### Operations Manager +- **Chenxue Jia** @Davina-jcx + +## Recognition + +We extend our sincere gratitude to all team members for their commitment to excellence and their contributions to making Nexent a powerful and reliable AI agent platform. + +Each team member brings unique expertise and perspective to the project, contributing to different aspects of the platform including: + +- System architecture and engineering +- Product management and strategy +- Development and implementation +- Operations and maintenance +- Quality assurance and testing + +## Join Our Team + +If you're interested in contributing to Nexent, please see our [Contributing Guide](/en/contributing) for more information on how to get involved. + +--- + +*This list represents our core team members. For a complete list of all contributors, please visit our GitHub repository.* \ No newline at end of file diff --git a/doc/docs/en/deployment/devcontainer.md b/doc/docs/en/deployment/devcontainer.md new file mode 100644 index 000000000..84a49f47e --- /dev/null +++ b/doc/docs/en/deployment/devcontainer.md @@ -0,0 +1,71 @@ +# Nexent Dev Container Usage Guide + +## 1. Environment Overview + +This development container configuration sets up a complete Nexent development environment, including the following components: + +- Main development container (`nexent-dev`): Based on nexent/nexent image with development tools +- Service containers: + - Elasticsearch (`nexent-elasticsearch`) + - PostgreSQL (`nexent-postgresql`) + - MinIO (`nexent-minio`) + - Nexent backend (`nexent`) + - Nexent frontend (`nexent-web`) + - Data processing service (`nexent-data-process`) + +## 2. Usage Steps + +### 2.1 Prerequisites + +1. Install Cursor/VS Code +2. Install Dev Containers extension (`anysphere.remote-containers` and `anysphere.remote-sshRemote`) +3. Ensure Docker and Docker Compose are installed and running + +### 2.2 Starting Project with Dev Container + +1. Clone the project locally +2. Open project folder in Cursor/VS Code +3. Run `docker/deploy.sh` script in `infrastructure` mode to start containers +4. Enter `nexent-minio` and `nexent-elasticsearch` containers, copy `MINIO_ACCESS_KEY`, `MINIO_SECRET_KEY`, `ELASTICSEARCH_API_KEY` environment variables to corresponding positions in `docker/docker-compose.dev.yml` +5. Press `F1` or `Ctrl+Shift+P`, type `Dev Containers: Reopen in Container ...` +6. Cursor will start the development container based on configuration in `.devcontainer` directory + +### 2.3 Development Workflow + +1. After container starts, Cursor automatically connects to development container +2. All file editing is done within the container +3. Develop, test, and build directly in container after modifications +4. Git change management can be done directly in container using `git commit` or `git push`; however, pulling remote code in container is not recommended as it may cause path issues + +## 3. Port Mapping + +The following ports are mapped in devcontainer.json: + +- 3000: Nexent Web interface +- 5010: Nexent backend service +- 5012: Data processing service +- 9010: MinIO API +- 9011: MinIO console +- 9210: Elasticsearch API +- 5434: PostgreSQL + +## 4. Customizing Development Environment + +You can customize the development environment by modifying: + +- `.devcontainer/devcontainer.json` - Plugin configuration +- `docker/docker-compose.dev.yml` - Development container build configuration, requires environment variable modification for proper startup + +## 5. Troubleshooting + +If you encounter permission issues, you may need to run in container: + +```bash +sudo chown -R $(id -u):$(id -g) /opt +``` + +If container startup fails, try: + +1. Rebuild container: Press `F1` or `Ctrl+Shift+P`, type `Dev Containers: Rebuild Container` +2. Check Docker logs: `docker logs nexent-dev` +3. Check if configuration in `.env` file is correct \ No newline at end of file diff --git a/make/README.md b/doc/docs/en/deployment/docker-build.md similarity index 55% rename from make/README.md rename to doc/docs/en/deployment/docker-build.md index 25f287b86..ea2f838e0 100644 --- a/make/README.md +++ b/doc/docs/en/deployment/docker-build.md @@ -15,6 +15,10 @@ docker buildx build --progress=plain --platform linux/amd64,linux/arm64 -t ccr.c # 🌐 build web frontend for multiple architectures docker buildx build --progress=plain --platform linux/amd64,linux/arm64 -t nexent/nexent-web -f make/web/Dockerfile . --push docker buildx build --progress=plain --platform linux/amd64,linux/arm64 -t ccr.ccs.tencentyun.com/nexent-hub/nexent-web -f make/web/Dockerfile . --push + +# 📚 build documentation for multiple architectures +docker buildx build --progress=plain --platform linux/amd64,linux/arm64 -t nexent/nexent-docs -f make/docs/Dockerfile . --push +docker buildx build --progress=plain --platform linux/amd64,linux/arm64 -t ccr.ccs.tencentyun.com/nexent-hub/nexent-docs -f make/docs/Dockerfile . --push ``` ### 💻 Local Development Build @@ -28,6 +32,9 @@ docker build --progress=plain -t nexent/nexent-data-process -f make/data_process # 🌐 Build web frontend image (current architecture only) docker build --progress=plain -t nexent/nexent-web -f make/web/Dockerfile . + +# 📚 Build documentation image (current architecture only) +docker build --progress=plain -t nexent/nexent-docs -f make/docs/Dockerfile . ``` ### 🧹 Clean up Docker resources @@ -37,6 +44,78 @@ docker build --progress=plain -t nexent/nexent-web -f make/web/Dockerfile . docker builder prune -f && docker system prune -f ``` +### 🔧 Image Descriptions + +#### Main Application Image (nexent/nexent) +- Contains backend API service +- Built from `make/main/Dockerfile` +- Provides core agent services + +#### Data Processing Image (nexent/nexent-data-process) +- Contains data processing service +- Built from `make/data_process/Dockerfile` +- Handles document parsing and vectorization + +#### Web Frontend Image (nexent/nexent-web) +- Contains Next.js frontend application +- Built from `make/web/Dockerfile` +- Provides user interface + +#### Documentation Image (nexent/nexent-docs) +- Contains Vitepress documentation site +- Built from `make/docs/Dockerfile` +- Provides project documentation and API reference + +### 🏷️ Tagging Strategy + +Each image is pushed to two repositories: +- `nexent/*` - Main public image repository +- `ccr.ccs.tencentyun.com/nexent-hub/*` - Tencent Cloud image repository (China region acceleration) + +All images include: +- `nexent/nexent` - Main application backend service +- `nexent/nexent-data-process` - Data processing service +- `nexent/nexent-web` - Next.js frontend application +- `nexent/nexent-docs` - Vitepress documentation site + +## 📚 Documentation Image Standalone Deployment + +The documentation image can be built and run independently to serve nexent.tech/doc: + +### Build Documentation Image + +```bash +docker build -t nexent/nexent-docs -f make/docs/Dockerfile . +``` + +### Run Documentation Container + +```bash +docker run -d --name nexent-docs -p 4173:4173 nexent/nexent-docs +``` + +### Check Container Status + +```bash +docker ps +``` + +### View Container Logs + +```bash +docker logs nexent-docs +``` + +### Stop and Remove Container + +```bash +docker stop nexent-docs +``` + +```bash +docker rm nexent-docs +``` + Notes: - 🔧 Use `--platform linux/amd64,linux/arm64` to specify target architectures - 📤 The `--push` flag automatically pushes the built images to Docker Hub diff --git a/doc/docs/en/frontend/overview.md b/doc/docs/en/frontend/overview.md new file mode 100644 index 000000000..bfccad3bc --- /dev/null +++ b/doc/docs/en/frontend/overview.md @@ -0,0 +1,131 @@ +# Frontend Architecture Overview + +Nexent's frontend is built with modern React technologies, providing a responsive and intuitive user interface for AI agent interactions. + +## Technology Stack + +- **Framework**: Next.js 14 (App Router) +- **Language**: TypeScript +- **UI Library**: React + Tailwind CSS +- **State Management**: React Hooks +- **Internationalization**: react-i18next +- **HTTP Client**: Fetch API + +## Directory Structure + +``` +frontend/ +├── app/ # Next.js App Router +│ └── [locale]/ # Internationalization routes (zh/en) +│ ├── chat/ # Chat interface +│ │ ├── internal/ # Chat core logic +│ │ ├── layout/ # Chat interface layout components +│ │ └── streaming/ # Streaming response handling +│ ├── setup/ # System settings pages +│ │ ├── agentSetup/ # Agent configuration +│ │ ├── knowledgeBaseSetup/ # Knowledge base configuration +│ │ └── modelSetup/ # Model configuration +│ └── layout.tsx # Global layout +├── components/ # Reusable UI components +│ ├── providers/ # Context providers +│ └── ui/ # Basic UI component library +├── services/ # API service layer +│ ├── api.ts # Basic API configuration +│ ├── conversationService.ts # Conversation service +│ ├── agentConfigService.ts # Agent configuration service +│ ├── knowledgeBaseService.ts # Knowledge base service +│ └── modelService.ts # Model service +├── hooks/ # Custom React Hooks +├── lib/ # Utility libraries +├── types/ # TypeScript type definitions +├── public/ # Static resources +│ └── locales/ # Internationalization files +└── middleware.ts # Next.js middleware +``` + +## Architecture Responsibilities + +### **Presentation Layer** +- User interface and interaction logic +- Component-based architecture for reusability +- Responsive design for multiple devices + +### **Service Layer** +- Encapsulates API calls and data transformation +- Handles communication with backend services +- Manages error handling and retry logic + +### **State Management** +- React Hooks for component state management +- Context providers for global state +- Real-time updates for streaming responses + +### **Internationalization** +- Support for English and Chinese languages +- Dynamic language switching +- Localized content and UI elements + +### **Routing Management** +- Based on Next.js App Router +- Locale-aware routing +- Dynamic route generation + +## Key Features + +### Real-time Chat Interface +- Streaming response handling +- Message history management +- Multi-modal input support (text, voice, images) + +### Configuration Management +- Model provider configuration +- Agent behavior customization +- Knowledge base management + +### Responsive Design +- Mobile-first approach +- Adaptive layouts +- Touch-friendly interactions + +### Performance Optimization +- Server-side rendering (SSR) +- Static site generation (SSG) +- Code splitting and lazy loading +- Image optimization + +## Development Workflow + +### Setup +```bash +cd frontend +npm install +npm run dev +``` + +### Building for Production +```bash +npm run build +npm start +``` + +### Code Quality +- ESLint for code linting +- Prettier for code formatting +- TypeScript for type safety +- Husky for pre-commit hooks + +## Integration Points + +### Backend Communication +- RESTful API calls +- WebSocket for real-time features +- Authentication and authorization +- Error handling and user feedback + +### External Services +- Model provider APIs +- File upload and management +- Voice processing integration +- Analytics and monitoring + +For detailed development guidelines and component documentation, see the [Development Guide](../getting-started/development-guide). \ No newline at end of file diff --git a/doc/docs/en/getting-started/development-guide.md b/doc/docs/en/getting-started/development-guide.md new file mode 100644 index 000000000..f8d67cc8b --- /dev/null +++ b/doc/docs/en/getting-started/development-guide.md @@ -0,0 +1,150 @@ +# Nexent Development Guide + +This guide provides comprehensive information for developers to understand and contribute to the Nexent project, covering architecture, technology stack, development environment setup, and best practices. + +## 🏗️ Overall Architecture + +``` +nexent/ +├── frontend/ # Frontend application (Next.js + TypeScript) +├── backend/ # Backend services (FastAPI + Python) +├── sdk/ # Python SDK +├── docker/ # Docker deployment configuration +├── make/ # Build scripts +├── test/ # Test code +└── assets/ # Static resources +``` + +## 🛠️ Technology Stack + +### Frontend Tech Stack +- **Framework**: Next.js 14 (App Router) +- **Language**: TypeScript +- **UI Library**: React + Tailwind CSS +- **State Management**: React Hooks +- **Internationalization**: react-i18next +- **HTTP Client**: Fetch API + +### Backend Tech Stack +- **Framework**: FastAPI +- **Language**: Python 3.10+ +- **Database**: PostgreSQL + Redis + Elasticsearch +- **File Storage**: MinIO +- **Task Queue**: Celery + Ray +- **AI Framework**: smolagents +- **Vector Database**: Elasticsearch + +### Deployment Tech Stack +- **Containerization**: Docker + Docker Compose +- **Reverse Proxy**: Nginx +- **Monitoring**: Built-in health checks +- **Logging**: Structured logging + +## 🚀 Development Environment Setup + +### Environment Requirements +- Python 3.10+ +- Node.js 18+ +- Docker & Docker Compose +- uv (Python package manager) + +### Backend Setup +```bash +cd backend +uv sync && uv pip install -e ../sdk +``` + +### Frontend Setup +```bash +cd frontend +npm install +npm run dev +``` + +### Service Startup +Nexent includes three core backend services that need to be started separately: +```bash +python backend/data_process_service.py # Data processing service +python backend/main_service.py # Main service +python backend/nexent_mcp_service.py # MCP service +``` + +## 🔧 Development Module Guide + +### 🎨 Frontend Development +- **Tech Stack**: Next.js 14 + TypeScript + React + Tailwind CSS +- **Core Features**: User interface, real-time chat, configuration management, internationalization +- **Details**: See [Frontend Overview](../frontend/overview.md) + +### 🔧 Backend Development +- **Tech Stack**: FastAPI + Python 3.10+ + PostgreSQL + Redis + Elasticsearch +- **Core Features**: API services, agent management, data processing, vector search +- **Details**: See [Backend Overview](../backend/overview.md) + +### 🤖 AI Agent Development +- **Framework**: Enterprise agent framework based on smolagents +- **Core Features**: Agent creation, tool integration, reasoning execution, multi-modal support +- **Custom Agents**: See [Agent Overview](../agents/overview.md) +- **System Prompts**: Located in `backend/prompts/` +- **Implementation Steps**: Create instance → Configure tools → Set prompts → Test run +- **Details**: See [Agent Overview](../agents/overview.md) + +### 🛠️ Tool Development +- **MCP Tool System**: Based on Model Context Protocol +- **Development Flow**: Implement logic → Register tool → Restart service +- **Protocol Compliance**: Tool development must follow MCP protocol +- **Detailed Specification**: See [Tool Development Guide](../sdk/core/tools.md) + +### 📦 SDK Development Kit +- **Features**: Complete interfaces for AI agents, model calling, tool integration +- **Modules**: Core agents, data processing, vector database +- **Details**: See [SDK Overview](../sdk/overview.md) + +### 📊 Data Processing +- **File Processing**: Supports 20+ formats +- **Chunking Strategies**: basic, by_title, none +- **Streaming Processing**: Memory optimization for large files +- **Details**: See [Data Processing Guide](../sdk/data-process.md) + +## 🏗️ Build & Deployment + +### Docker Build +For detailed build instructions, see [Docker Build Guide](../deployment/docker-build.md) + +## 📋 Development Best Practices & Important Notes + +### Code Quality +1. **Test-Driven**: Write unit tests and integration tests +2. **Code Review**: Follow team coding standards +3. **Documentation**: Update related documentation timely +4. **Error Handling**: Comprehensive exception handling and logging + +### Performance Optimization +1. **Async Processing**: Use async architecture for performance +2. **Caching Strategy**: Proper use of caching mechanisms +3. **Resource Management**: Pay attention to memory and connection pool management +4. **Monitoring & Debugging**: Use performance monitoring tools + +### Security Considerations +1. **Input Validation**: Strictly validate all input parameters +2. **Access Control**: Implement appropriate access controls +3. **Sensitive Information**: Properly handle sensitive data like API keys +4. **Security Updates**: Regular dependency and security updates + +### Important Development Notes +1. **Service Dependencies**: Ensure all services are started before testing +2. **Code Changes**: Restart related services after code modifications +3. **Development Mode**: Use debug mode in development environment +4. **Prompt Testing**: System prompts need thorough testing + +## 💡 Getting Help + +### Documentation Resources +- [Installation Guide](./installation.md) - Environment setup and deployment +- [Model Providers](./model-providers.md) - Model configuration and API acquisition +- [FAQ](./faq) - Frequently asked questions + +### Community Support +- [Discord Community](https://discord.gg/tb5H3S3wyv) - Real-time communication and support +- [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) - Issue reporting and feature requests +- [Contributing Guide](../contributing.md) - Participate in project development \ No newline at end of file diff --git a/FAQ.md b/doc/docs/en/getting-started/faq.md similarity index 67% rename from FAQ.md rename to doc/docs/en/getting-started/faq.md index d9ce03eaf..95b6ead27 100644 --- a/FAQ.md +++ b/doc/docs/en/getting-started/faq.md @@ -1,69 +1,67 @@ -# Nexent FAQ 🤔 - -[![English](https://img.shields.io/badge/English-FAQ-blue)](FAQ.md) -[![中文](https://img.shields.io/badge/中文-FAQ-green)](FAQ_CN.md) - -This FAQ addresses common questions and issues you might encounter while installing and deploying Nexent. For the basic installation steps, please refer to the [Quick Start Guide](../README.md#-have-a-try-first) in our README. - -## 🚀 Installation & Setup - -### 🔑 Jina API Key -- **Q: How do I get a Jina API key for embeddings?** - - A: To use Jina-based embedding model, you'll need to: - 1. Visit [Jina AI's website](https://jina.ai/), no need to sign up - 2. Navigate to the Embedding section to get your API key - 3. Add your API key in the app - -## 🚫 Common Errors & Operations - -### 🌐 Network Connection Issues -- **Q: How can a Docker container access models deployed on the host machine (e.g., Ollama)?** - - A: Since `localhost` inside the container refers to the container itself, use one of these methods to connect to host services: - - **Option 1: Use Docker's special DNS name `host.docker.internal`** - Supported environments: Mac/Windows and newer Docker Desktop versions (Linux version also supported) - ```bash - http://host.docker.internal:11434/v1 - ``` - - **Option 2: Use host machine's actual IP (ensure firewall allows access)** - ```bash - http://[HOST_IP]:11434/v1 - ``` - - **Option 3: Modify Docker Compose configuration** - Add to your docker-compose.yaml file: - ```yaml - extra_hosts: - - "host.docker.internal:host-gateway" - ``` - -### 🔌 Port Conflicts -- **Q: Port 3000 is already in use. How can I change it?** - - A: You can modify the port in the Docker Compose configuration file. - -### 📦 Container Issues -- **Q: How do I check container logs?** - - A: Use `docker logs ` to view logs for specific containers. - -## 🔍 Troubleshooting - -### 🔢 Embedding Model Issues - -- **Q: Why can't my Embedding model connect?** - - A: When creating a custom model and filling in the model URL, make sure to add the `/v1/embeddings` suffix. For example: `https://model.provider.com/v1/embeddings` - -### 📧 Email Tools Configuration -- **Q: How can I enable and configure email tools?** - - A: Our team has pre-implemented email tools based on IMAP and SMTP. To enable them: - 1. Configure email parameters in `.env` file - 2. Uncomment the email tool imports and registrations in `agent_utils.py` - 3. Switch to the email-enabled system prompt by using `code_agent_with_email.yaml` - 4. Restart the MCP service to apply changes - -## ❓ Still Need Help? - -If your question isn't answered here: -- Join our [Discord community](https://discord.gg/tb5H3S3wyv) for real-time support -- Check our [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) for similar problems -- Refer to our [Contribution Guide](CONTRIBUTING.md) for more detailed information +# Nexent FAQ + +This FAQ addresses common questions and issues you might encounter while installing and deploying Nexent. For the basic installation steps, please refer to the [Quick Start Guide](./overview#quick-start) in our documentation. + +## 🚀 Installation & Setup + +For model provider setup and API key acquisition, see our detailed **[Model Providers Guide](./model-providers)**. + +## 🚫 Common Errors & Operations + +### 🌐 Network Connection Issues +- **Q: How can a Docker container access models deployed on the host machine (e.g., Ollama)?** + - A: Since `localhost` inside the container refers to the container itself, use one of these methods to connect to host services: + + **Option 1: Use Docker's special DNS name `host.docker.internal`** + Supported environments: Mac/Windows and newer Docker Desktop versions (Linux version also supported) + ```bash + http://host.docker.internal:11434/v1 + ``` + + **Option 2: Use host machine's actual IP (ensure firewall allows access)** + ```bash + http://[HOST_IP]:11434/v1 + ``` + + **Option 3: Modify Docker Compose configuration** + Add to your docker-compose.yaml file: + ```yaml + extra_hosts: + - "host.docker.internal:host-gateway" + ``` + +### 🔌 Port Conflicts +- **Q: Port 3000 is already in use. How can I change it?** + - A: You can modify the port in the Docker Compose configuration file. + +### 📦 Container Issues +- **Q: How do I check container logs?** + - A: Use `docker logs ` to view logs for specific containers. + +## 🔍 Troubleshooting + +### 🔢 Model Connection Issues + +- **Q: Why can't my model connect?** + - A: Check the following: + 1. **Correct API endpoint**: Ensure you're using the right base URL + 2. **Valid API key**: Verify your API key has proper permissions + 3. **Model name**: Confirm the model identifier is correct + 4. **Network access**: Ensure your deployment can reach the provider's servers + + For specific provider setup, see our [Model Providers Guide](./model-providers). + +### 📧 Email Tools Configuration +- **Q: How can I enable and configure email tools?** + - A: Our team has pre-implemented email tools based on IMAP and SMTP. To enable them: + 1. Configure email parameters in `.env` file + 2. Uncomment the email tool imports and registrations in `agent_utils.py` + 3. Switch to the email-enabled system prompt by using `code_agent_with_email.yaml` + 4. Restart the MCP service to apply changes + +## 💡 Need Help + +If your question isn't answered here: +- Join our [Discord community](https://discord.gg/tb5H3S3wyv) for real-time support +- Check our [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) for similar problems +- Refer to our [Contribution Guide](../contributing) for more detailed information \ No newline at end of file diff --git a/doc/docs/en/getting-started/features.md b/doc/docs/en/getting-started/features.md new file mode 100644 index 000000000..e699c1f8c --- /dev/null +++ b/doc/docs/en/getting-started/features.md @@ -0,0 +1,78 @@ +# Key Features + +Nexent provides powerful capabilities for building and deploying AI agents with minimal effort. Here are the core features that make Nexent unique. + +## 🧠 Smart Agent Prompt Generation + +Turn plain language into runnable prompts. Nexent automatically chooses the right tools and plans the best action path for every request. + +![Feature 1](../../assets/Feature1.png) + +## ⚡ Scalable Data Process Engine + +Process 20+ data formats with fast OCR and table structure extraction, scaling smoothly from a single process to large-batch pipelines. + +![Feature 2](../../assets/Feature2.png) + +## 📚 Personal-Grade Knowledge Base + +Import files in real time, auto-summarise them, and let agents access both personal and global knowledge instantly, also knowing what it can get from each knowledge base. + +![Feature 3](../../assets/Feature3.png) + +## 🌐 Internet Knowledge Search + +Connect to 5+ web search providers so agents can mix fresh internet facts with your private data. + +![Feature 4](../../assets/Feature4.png) + +## 🔍 Knowledge-Level Traceability + +Serve answers with precise citations from web and knowledge-base sources, making every fact verifiable. + +![Feature 5](../../assets/Feature5.png) + +## 🎭 Multimodal Understanding & Dialogue + +Speak, type, files, or show images. Nexent understands voice, text, and pictures, and can even generate new images on demand. + +![Feature 6](../../assets/Feature6.png) + +## 🔧 MCP Tool Ecosystem + +Drop in or build Python plug-ins that follow the MCP spec; swap models, tools, and chains without touching core code. + +![Feature 7](../../assets/Feature7.png) + +## 🏗️ Architecture Benefits + +### ⚡ Distributed Processing Capabilities +- **Asynchronous Architecture**: High-performance asynchronous processing based on asyncio +- **Multi-threading Safety**: Thread-safe concurrent processing mechanisms +- **Celery Integration**: Optimized for distributed task queues +- **Batch Optimization**: Intelligent batch operations to reduce network overhead + +### 🏢 Enterprise-grade Scalability +- **Modular Design**: Loose-coupled module architecture for easy extension +- **Plugin-based Tools**: Standardized tool interfaces for rapid integration +- **Configuration Management**: Flexible configuration system supporting multi-environment deployment +- **Monitoring Friendly**: Comprehensive logging and status monitoring + +### 🚀 High-performance Optimization +- **Connection Pooling**: Intelligent reuse of database and HTTP connections +- **Memory Management**: Stream processing of large files and memory optimization +- **Concurrency Control**: Intelligent concurrency limiting and load balancing +- **Caching Strategy**: Multi-layer caching to improve response speed + +For detailed information about Nexent's software architecture and technical advantages, see our **[Software Architecture](./software-architecture)** guide. + +## 🎯 Use Cases + +Nexent is designed for various scenarios including: +- **Business Intelligence**: Automated data analysis and reporting +- **Customer Support**: Intelligent chat agents with knowledge base integration +- **Content Processing**: Document analysis, summarization, and extraction +- **Research Assistance**: Academic paper analysis and information synthesis +- **Personal Productivity**: Smart assistants for daily tasks and information management + +For detailed agent scenarios and real-world implementations, see our **[MCP Ecosystem Use Cases](../mcp-ecosystem/use-cases)**. \ No newline at end of file diff --git a/doc/docs/en/getting-started/installation.md b/doc/docs/en/getting-started/installation.md new file mode 100644 index 000000000..bd6fcffc8 --- /dev/null +++ b/doc/docs/en/getting-started/installation.md @@ -0,0 +1,124 @@ +# Installation & Setup + +## 🎯 Prerequisites + +| Resource | Minimum | +|----------|---------| +| **CPU** | 2 cores | +| **RAM** | 6 GiB | +| **Architecture** | x86_64 / ARM64 | +| **Software** | Docker & Docker Compose installed | + +## 🚀 Quick Start + +### 1. Download and Setup + +```bash +git clone https://github.com/ModelEngine-Group/nexent.git +cd nexent/docker +cp .env.example .env # Configure environment variables +``` + +### 2. Deployment Options + +The deployment script offers multiple modes: + +```bash +bash deploy.sh +``` + +**Available deployment modes:** +- **Development mode (default)**: Exposes all service ports for debugging +- **Infrastructure mode**: Only starts infrastructure services +- **Production mode**: Only exposes port 3000 for security +- **Beta mode**: Uses development branch images + +**Optional components:** +- **Terminal Tool**: Enables openssh-server for AI agent shell command execution +- **Regional optimization**: Mainland China users can use optimized image sources + +### 3. Access Your Installation + +When deployment completes successfully: +1. Open **http://localhost:3000** in your browser +2. Follow the setup wizard for initial configuration +3. Configure your model providers (see [Model Providers Guide](./model-providers)) + +## 🤖 Model Configuration + +Nexent supports all **OpenAI-compatible models**, including: +- **Large Language Models (LLM)**: Any OpenAI-compatible API provider +- **Multimodal Vision Models**: Text + image processing capabilities +- **Embedding Models**: All OpenAI-compatible embedding services +- **Text-to-Speech & Speech-to-Text**: Multiple provider support +- **Search Integration**: Web search and semantic retrieval + +### Quick Provider Setup + +For detailed setup instructions and API key acquisition, see our **[Model Providers Guide](./model-providers)**. + +**Recommended for Quick Start**: +- **LLM**: [Silicon Flow](https://siliconflow.cn/) (Free tier available) +- **Embedding**: [Jina AI](https://jina.ai/) (Free tier available) +- **Search**: [EXA](https://exa.ai/) (Free tier available) + +### Configuration Methods + +**Method 1: Web Interface** +1. Access model configuration at `http://localhost:3000` +2. Add provider details: Base URL, API Key, Model Name + +**Method 2: Environment Variables** +Add to your `.env` file: +```bash +LLM_BASE_URL=https://api.siliconflow.cn/v1 +LLM_API_KEY=your_api_key +EMBEDDING_API_KEY=your_jina_key +EXA_API_KEY=your_exa_key +``` + +## 🏗️ Service Architecture + +The deployment includes the following components: + +**Core Services:** +- `nexent`: Backend service (port 5010) +- `nexent-web`: Frontend interface (port 3000) +- `nexent-data-process`: Data processing service (port 5012) + +**Infrastructure Services:** +- `nexent-postgresql`: Database (port 5434) +- `nexent-elasticsearch`: Search engine (port 9210) +- `nexent-minio`: Object storage (port 9010, console 9011) +- `redis`: Cache service (port 6379) + +**Optional Services:** +- `nexent-openssh-server`: SSH server for Terminal tool (port 2222) + +## 🔌 Port Mapping + +| Service | Internal Port | External Port | Description | +|---------|---------------|---------------|-------------| +| Web Interface | 3000 | 3000 | Main application access | +| Backend API | 5010 | 5010 | Backend service | +| Data Processing | 5012 | 5012 | Data processing API | +| PostgreSQL | 5432 | 5434 | Database connection | +| Elasticsearch | 9200 | 9210 | Search engine API | +| MinIO API | 9000 | 9010 | Object storage API | +| MinIO Console | 9001 | 9011 | Storage management UI | +| Redis | 6379 | 6379 | Cache service | +| SSH Server | 2222 | 2222 | Terminal tool access | + +For complete port mapping details, see our [Dev Container Guide](../deployment/devcontainer.md#port-mapping). + +## 💡 Need Help + +- Browse the [FAQ](./faq) for common install issues +- Drop questions in our [Discord community](https://discord.gg/tb5H3S3wyv) +- File bugs or feature ideas in [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) + +## 🔧 Build from Source + +Want to build from source or add new features? Check the [Docker Build Guide](../deployment/docker-build) for step-by-step instructions. + +For detailed setup instructions and customization options, see our [Development Guide](./development-guide). \ No newline at end of file diff --git a/doc/docs/en/getting-started/model-providers.md b/doc/docs/en/getting-started/model-providers.md new file mode 100644 index 000000000..aa8217c1e --- /dev/null +++ b/doc/docs/en/getting-started/model-providers.md @@ -0,0 +1,151 @@ +# Third-Party Model Providers + +This guide helps you obtain API keys and configure various AI model providers supported by Nexent. Nexent supports all OpenAI-compatible models, multimodal embeddings, and multimodal language models. + +## 🤖 Large Language Models (LLM) + +### Silicon Flow +- **Website**: [siliconflow.cn](https://siliconflow.cn/) +- **Free Tier**: Available +- **Supported Models**: DeepSeek-R1, DeepSeek-V3, QwQ-32B, GLM-4-9B-Chat, and more + +**Getting Started**: +1. Visit [SiliconFlow](https://siliconflow.cn/) and create an account +2. Navigate to API Keys section in your dashboard +3. Create a new API key and copy it +4. Use base URL: `https://api.siliconflow.cn/v1` + +### Alibaba Cloud Bailian +- **Website**: [bailian.console.aliyun.com](https://bailian.console.aliyun.com/) +- **Free Tier**: Available +- **Supported Models**: Qwen series, ERNIE, and various open-source models + +**Getting Started**: +1. Sign up for Alibaba Cloud account +2. Access Bailian console +3. Create API credentials in the model service section +4. Configure your API endpoint and key + +### Other OpenAI-Compatible Providers +Nexent supports any provider that follows the OpenAI API specification: +- **OpenAI**: [platform.openai.com](https://platform.openai.com/) +- **Anthropic**: [console.anthropic.com](https://console.anthropic.com/) +- **Deepseek**: [platform.deepseek.com](https://platform.deepseek.com/) +- **Moonshot**: [platform.moonshot.cn](https://platform.moonshot.cn/) +- **Local Models**: Ollama, vLLM, or any OpenAI-compatible server + +## 🎭 Multimodal Vision Models + +Nexent supports multimodal models that can process both text and images: + +### Recommended Providers +- **GPT-4V** (OpenAI): Best overall performance +- **Claude-3** (Anthropic): Excellent reasoning capabilities +- **Qwen-VL** (Alibaba): Strong multilingual support +- **GLM-4V** (Zhipu): Good Chinese language support + +**Configuration**: Use the same API endpoints as LLM models, but specify multimodal model names. + +## 🔤 Embedding Models + +### Jina AI +- **Website**: [jina.ai](https://jina.ai/) +- **Free Tier**: Available +- **Specialties**: Multilingual embeddings, document understanding + +**Getting Started**: +1. Visit [Jina AI](https://jina.ai/) (no sign-up required for basic usage) +2. Navigate to the Embedding section +3. Get your API key from the dashboard +4. Use endpoint: `https://api.jina.ai/v1/embeddings` + +### Other Embedding Providers +- **OpenAI Embeddings**: text-embedding-3-small, text-embedding-3-large +- **Cohere**: High-quality multilingual embeddings +- **Local Embeddings**: BGE, E5, or any OpenAI-compatible embedding service + +## 🎤 Voice Models + +### Volcengine Voice +- **Website**: [volcengine.com/product/voice-tech](https://www.volcengine.com/product/voice-tech) +- **Free Tier**: Available for personal use +- **Specialties**: High-quality Chinese and English voice synthesis + +**Getting Started**: +1. Register for Volcengine account +2. Access Voice Technology service +3. Create application and get API credentials +4. Configure TTS/STT settings in your environment + +## 🔍 Search Providers + +### EXA +- **Website**: [exa.ai](https://exa.ai/) +- **Free Tier**: Available +- **Specialties**: Semantic search, web content retrieval + +**Getting Started**: +1. Sign up at [EXA](https://exa.ai/) +2. Generate API key in dashboard +3. Configure search parameters + +### Tavily +- **Website**: [tavily.com](https://tavily.com/) +- **Free Tier**: Available +- **Specialties**: AI-powered search, research assistant + +**Getting Started**: +1. Sign up at [Tavily](https://tavily.com/) +2. Get your API key from the dashboard +3. Configure search preferences + +### Linkup +- **Website**: [linkup.com](https://linkup.com/) +- **Free Tier**: Available +- **Specialties**: Real-time web search, content discovery + +**Getting Started**: +1. Register at [Linkup](https://linkup.com/) +2. Generate API credentials +3. Set up search parameters + +## 🛠️ Configuration in Nexent + +### Model Configuration Page +1. Access Nexent at `http://localhost:3000` +2. Navigate to Model Configuration page +3. Add your provider details: + - **Base URL**: Provider's API endpoint + - **API Key**: Your provider's API key + - **Model Name**: Specific model identifier + +## 💡 Tips for Choosing Providers + +### For Development & Testing +- **SiliconFlow**: Great free tier, multiple models +- **Jina AI**: Excellent embedding capabilities +- **Local Models**: Complete privacy and control + +### For Production +- **OpenAI**: Most reliable, best documentation +- **Anthropic**: Strong safety and reasoning +- **Enterprise Providers**: Dedicated support and SLAs + +### For Cost Optimization +- **Free Tiers**: Start with SiliconFlow, Jina, and EXA +- **Open Source**: Self-host with Ollama or vLLM +- **Regional Providers**: Often cheaper for local markets + +## 🔗 Quick Links + +- [Installation & Setup](./installation) - Complete deployment guide +- [Model Configuration FAQ](./faq) - Common configuration issues +- [Known Issues](../known-issues) - Current limitations and workarounds + +## 💡 Need Help + +If you encounter issues with model providers: +1. Check provider-specific documentation +2. Verify API key permissions and quotas +3. Test with provider's official examples +4. Join our [Discord community](https://discord.gg/tb5H3S3wyv) for support \ No newline at end of file diff --git a/doc/docs/en/getting-started/overview.md b/doc/docs/en/getting-started/overview.md new file mode 100644 index 000000000..843f1736f --- /dev/null +++ b/doc/docs/en/getting-started/overview.md @@ -0,0 +1,70 @@ +# Nexent + +Nexent is a zero-code platform for auto-generating agents — no orchestration, no complex drag-and-drop required, using pure language to develop any agent you want. Built on the MCP ecosystem with rich tool integration, Nexent also provides various built-in agents to meet your intelligent service needs in different scenarios such as work, travel, and daily life. Nexent offers powerful capabilities for agent running control, multi-agent collaboration, data processing and knowledge tracing, multimodal dialogue, and batch scaling. + +> One prompt. Endless reach. + +![Nexent Banner](../../assets/NexentBanner.png) + +## 🎬 Demo Video + + + +## 🤝 Join Our Community + +> *If you want to go fast, go alone; if you want to go far, go together.* + +We have released **Nexent v1**, and the platform is now relatively stable. However, there may still be some bugs, and we are continuously improving and adding new features. Stay tuned: we will announce **v2.0** soon! + +* **🗺️ Check our [Feature Map](https://github.com/orgs/ModelEngine-Group/projects/6)** to explore current and upcoming features. +* **🔍 Try the current build** and leave ideas or bugs in the [Issues](https://github.com/ModelEngine-Group/nexent/issues) tab. + +> *Rome wasn't built in a day.* + +If our vision speaks to you, jump in via the **[Contribution Guide](../contributing)** and shape Nexent with us. + +Early contributors won't go unnoticed: from special badges and swag to other tangible rewards, we're committed to thanking the pioneers who help bring Nexent to life. + +Most of all, we need visibility. Star ⭐ and watch the repo, share it with friends, and help more developers discover Nexent — your click brings new hands to the project and keeps the momentum growing. + +## ⚡ Quick Start + +Ready to get started? Here are your next steps: + +1. **📋 [Installation & Setup](./installation)** - System requirements and deployment guide +2. **🔧 [Development Guide](./development-guide)** - Build from source and customize +3. **❓ [FAQ](./faq)** - Common questions and troubleshooting + +## 🌱 MCP Tool Ecosystem + +Nexent is built on the Model Context Protocol (MCP) tool ecosystem, providing a flexible and extensible framework for integrating various tools and services. MCP serves as the "USB-C of AI" - a universal interface standard that allows AI agents to seamlessly connect with external data sources, tools, and services. + +Learn more about the MCP ecosystem: +- **[MCP Overview](../mcp-ecosystem/overview)** - Understanding the MCP ecosystem and tools +- **[Use Cases & Scenarios](../mcp-ecosystem/use-cases)** - Real-world agent scenarios and implementations + +## ✨ Key Features + +Nexent offers a comprehensive set of features for building powerful AI agents: + +- **🤖 Smart Agent Generation** - Zero-code agent creation using natural language +- **📊 Scalable Data Processing** - Handle 20+ file formats with intelligent extraction +- **🧠 Personal Knowledge Base** - Real-time file import with auto-summarization +- **🌐 Internet Integration** - Connect to multiple search providers and web sources +- **🔍 Knowledge Traceability** - Precise citation and source verification +- **🎭 Multimodal Support** - Voice, text, images, and file processing +- **🔧 MCP Ecosystem** - Extensible tool integration and custom development + +For detailed feature information and examples, see our **[Features Guide](./features)**. + +## 💬 Community & contact + +Join our [Discord community](https://discord.gg/tb5H3S3wyv) to chat with other developers and get help! + +## 📄 License + +Nexent is licensed under the [MIT](../license) with additional conditions. Please read the [LICENSE](../license) file for details. + diff --git a/doc/docs/en/getting-started/software-architecture.md b/doc/docs/en/getting-started/software-architecture.md new file mode 100644 index 000000000..2a1429307 --- /dev/null +++ b/doc/docs/en/getting-started/software-architecture.md @@ -0,0 +1,7 @@ +# Software Architecture + +We will update the comprehensive software architecture diagram and documentation soon. + +--- + +*This documentation is under active development. We will update it with comprehensive architecture details soon.* \ No newline at end of file diff --git a/doc/docs/en/index.md b/doc/docs/en/index.md new file mode 100644 index 000000000..2a7b01552 --- /dev/null +++ b/doc/docs/en/index.md @@ -0,0 +1,30 @@ +--- +# https://vitepress.dev/reference/default-theme-home-page +layout: home + +hero: + name: "Nexent" + text: "Zero-Code AI Agent Platform" + tagline: "A zero-code platform for auto-generating agents — no orchestration, no complex drag-and-drop required." + actions: + - theme: brand + text: Get Started + link: /en/getting-started/overview + - theme: alt + text: View on GitHub + link: https://github.com/ModelEngine-Group/nexent + +features: + - title: Smart Agent Generation + details: Turn plain language into runnable prompts. Nexent automatically chooses the right tools and plans the best action path for every request. + - title: Scalable Data Processing + details: Process 20+ data formats with fast OCR and table structure extraction, scaling smoothly from a single process to large-batch pipelines. + - title: Personal Knowledge Base + details: Import files in real time, auto-summarise them, and let agents access both personal and global knowledge instantly. + - title: Internet Knowledge Search + details: Connect to 5+ web search providers so agents can mix fresh internet facts with your private data. + - title: Knowledge Traceability + details: Serve answers with precise citations from web and knowledge-base sources, making every fact verifiable. + - title: Multimodal Understanding + details: Speak, type, files, or show images. Nexent understands voice, text, and pictures, and can even generate new images on demand. +--- \ No newline at end of file diff --git a/doc/docs/en/known-issues.md b/doc/docs/en/known-issues.md new file mode 100644 index 000000000..3f1b09ced --- /dev/null +++ b/doc/docs/en/known-issues.md @@ -0,0 +1,41 @@ +# Known Issues + +This page lists known issues and limitations in the current version of Nexent. We are actively working on fixing these issues and will update this page as solutions become available. + +## 🐛 Current Issues + +### 1. OpenSSH Container Software Installation Limitations + +**Issue**: Installing additional software packages in the OpenSSH container for Terminal tool usage is currently challenging due to container constraints. + +**Status**: Under development + +**Impact**: Users who need custom tools or packages in the Terminal environment may face limitations. + +**Planned Solution**: We are working on providing improved containers and documentation to make customization easier. This will include better package management and more flexible container configurations. + +**Expected Timeline**: Improved container support planned for upcoming releases. + +## 📝 Reporting Issues + +If you encounter any issues not listed here, please: + +1. **Check our [FAQ](./getting-started/faq)** for common solutions +2. **Search existing issues** on [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) +3. **Create a new issue** with detailed information including: + - Steps to reproduce + - Expected behavior + - Actual behavior + - System information + - Log files (if applicable) + +## 🔄 Issue Status Updates + +We regularly update this page with the status of known issues. Check back frequently for updates, or watch our [GitHub repository](https://github.com/ModelEngine-Group/nexent) for notifications. + +## 💬 Community Support + +For immediate help or discussion about issues: +- Join our [Discord community](https://discord.gg/tb5H3S3wyv) +- Ask questions in [GitHub Discussions](https://github.com/ModelEngine-Group/nexent/discussions) +- Check our [Contributing Guide](./contributing) if you'd like to help fix issues \ No newline at end of file diff --git a/doc/docs/en/license.md b/doc/docs/en/license.md new file mode 100644 index 000000000..e50e3c4d4 --- /dev/null +++ b/doc/docs/en/license.md @@ -0,0 +1,38 @@ +# License + +Nexent is licensed under the MIT License with additional conditions. This license allows for both open source and commercial use, but includes specific restrictions for certain use cases. + +For any legal questions or commercial licensing inquiries, please contact the official licensing team. + +## Full License Text + +``` +# Nexent Open Source License + +Nexent is licensed under the MIT License, with the following additional conditions: + +Nexent is permitted to be used commercially, including as a backend service for other applications or as an application development platform for enterprises. However, when the following conditions are met, you must contact the producer to obtain a commercial license: + +a. Multi-tenant SaaS service: Unless explicitly authorized by Nexent in writing, you may not use the Nexent source code to operate a multi-tenant SaaS service. +b. LOGO and copyright information: In the process of using Nexent's frontend, you may not remove or modify the LOGO or copyright information in the Nexent console or applications. This restriction is inapplicable to uses of Nexent that do not involve its frontend. + +Please contact chenshuangrui@huawei.com by email to inquire about licensing matters. + +As a contributor, you should agree that: + +a. The producer can adjust the open-source agreement to be more strict or relaxed as deemed necessary. +b. Your contributed code may be used for commercial purposes, such as Nexent's cloud business. + +Apart from the specific conditions mentioned above, all other rights and restrictions follow the MIT License. +Detailed information about the MIT License can be found at: https://opensource.org/licenses/MIT + +Copyright © 2025 Huawei Technologies Co., Ltd. +``` + +## Contact Information + +For licensing inquiries: +- **Email**: chenshuangrui@huawei.com +- **MIT License Details**: https://opensource.org/licenses/MIT + +Please ensure you understand and comply with all license terms before using Nexent in your projects. \ No newline at end of file diff --git a/doc/docs/en/mcp-ecosystem/overview.md b/doc/docs/en/mcp-ecosystem/overview.md new file mode 100644 index 000000000..a160507f5 --- /dev/null +++ b/doc/docs/en/mcp-ecosystem/overview.md @@ -0,0 +1,70 @@ +# MCP Tool Ecosystem + +Nexent is built on the Model Context Protocol (MCP) tool ecosystem, providing a flexible and extensible framework for integrating various tools and services. MCP serves as the "USB-C of AI" - a universal interface standard that allows AI agents to seamlessly connect with external data sources, tools, and services. + +## What is MCP? + +The Model Context Protocol (MCP) is an open protocol that enables AI applications to securely connect to external data sources and tools. It provides a standardized way for AI models to access and interact with external systems, making it easier to build powerful, context-aware AI applications. + +## MCP Community Hub + +The global MCP ecosystem is thriving with multiple platforms supporting MCP development and deployment: + +| Platform | Description | Notes | +|----------|-------------|-------| +| **[GitHub MCP Server](https://github.com/github/github-mcp-server)** | Deep integration with Claude, GPT-4, Copilot etc., supports Go and Python | OAuth/GitHub account authorization | +| **[Qdrant MCP Vector Server](https://github.com/qdrant/mcp-server-qdrant)** | Semantic vector storage with Python/Go compatibility | Compatible with LangChain and other tools | +| **[Anthropic Reference MCP Servers](https://github.com/modelcontextprotocol/servers)** | Lightweight teaching and prototyping tools, Python | Includes fetch, git and other universal tools | +| **[AWS Labs MCP Server](https://github.com/awslabs/mcp)** | AWS+Go+CDK cloud reference services | Suitable for cloud environments | +| **[MCP Hub China](https://www.mcp-cn.com/)** | Chinese curated high-quality MCP service platform | Focuses on quality over quantity, community-driven | +| **[ModelScope MCP Marketplace](https://modelscope.cn/mcp)** | China's largest MCP community with 1,500+ services | From Amap to Alipay, comprehensive service coverage | +| **Community MCP Servers** | Various scenario-specific source code collection | Mostly experimental and innovative tools | + +## Recommended MCP Tools + +| Tool Name | Function | Description | +|-----------|----------|-------------| +| **[Amap Maps](https://modelscope.cn/mcp/servers/@amap/amap-maps)** | Geographic services and navigation | Comprehensive mapping, geocoding, routing, and location services | +| **[Bing Search (Chinese)](https://modelscope.cn/mcp/servers/@yan5236/bing-cn-mcp-server)** | Web search in Chinese | Optimized Chinese web search and information retrieval | +| **[12306 Train Ticket Query](https://modelscope.cn/mcp/servers/@Joooook/12306-mcp)** | China railway ticket booking | Real-time train schedules, ticket availability, and booking assistance | +| **[Alipay MCP](https://modelscope.cn/mcp/servers/@alipay/mcp-server-alipay)** | Payment and financial services | Digital payments, financial tools, and services integration | +| **[Variflight Aviation](https://modelscope.cn/mcp/servers/@variflight-ai/variflight-mcp)** | Flight information and aviation data | Real-time flight tracking, schedules, and aviation analytics | +| **[Sequential Thinking](https://modelscope.cn/mcp/servers/@modelcontextprotocol/sequentialthinking)** | Structured problem-solving framework | Break down complex problems into manageable, sequential steps | +| **[ArXiv AI Search](https://modelscope.cn/mcp/servers/@blazickjp/arxiv-mcp-server)** | Academic paper search and research | Advanced search and retrieval of scientific papers and research | +| **[Firecrawl MCP Server](https://modelscope.cn/mcp/servers/@mendableai/firecrawl-mcp-server)** | Web scraping and content extraction | Intelligent web scraping, data extraction, and content processing | + +## Benefits of MCP + +### Standardization +- **Universal Interface**: MCP provides a consistent way to connect AI models with external tools +- **Interoperability**: Tools built for one MCP-compatible platform work with others +- **Reduced Development Time**: Standardized protocols mean less custom integration work + +### Security +- **Controlled Access**: MCP provides secure, permission-based access to external resources +- **Authentication**: Built-in support for various authentication methods +- **Audit Trail**: Track and monitor all external interactions + +### Scalability +- **Modular Design**: Add or remove tools without affecting the core application +- **Load Distribution**: Distribute tool execution across multiple servers +- **Version Management**: Handle different versions of tools gracefully + +## Getting Started with MCP + +1. **Explore Available Tools**: Browse the MCP marketplace to find tools that fit your needs +2. **Install Tools**: Add MCP tools to your Nexent instance +3. **Configure Access**: Set up authentication and permissions +4. **Create Agents**: Build agents that leverage multiple MCP tools +5. **Monitor Performance**: Track usage and optimize tool selection + +For detailed integration guides, see our [Backend Tools Documentation](../backend/tools/). + +## Building Custom MCP Tools + +Interested in building your own MCP tools? Check out: +- [LangChain Tools Guide](../backend/tools/langchain) +- [MCP Tools Development](../backend/tools/mcp) +- [SDK Tools Documentation](../sdk/core/tools) + +The MCP ecosystem empowers you to build agents that can seamlessly interact with the real world, accessing live data, performing complex operations, and providing contextual assistance across virtually any domain. \ No newline at end of file diff --git a/doc/docs/en/mcp-ecosystem/use-cases.md b/doc/docs/en/mcp-ecosystem/use-cases.md new file mode 100644 index 000000000..13cb608f3 --- /dev/null +++ b/doc/docs/en/mcp-ecosystem/use-cases.md @@ -0,0 +1,112 @@ +# Suggested Agent Scenarios + +With MCP's powerful ecosystem, you can create sophisticated AI agents for various scenarios. Here are some proven use cases that demonstrate the versatility and power of the Nexent platform. + +## 🌍 Travel Planning Agent + +Create a comprehensive travel assistant that handles every aspect of your journey: + +- **Route Planning**: Use Amap for detailed route planning and navigation 📍 +- **Transportation**: Integrate 12306 for train bookings and scheduling 🚄 +- **Flight Management**: Connect Variflight for real-time flight information ✈️ +- **Payment Processing**: Enable Alipay for seamless payment handling 💳 + +**Example Capabilities**: +- Plan multi-city itineraries with optimal routing +- Book train tickets and track delays +- Monitor flight status and gate changes +- Handle payments across different services +- Provide real-time updates and alternatives + +## 🔬 Research Assistant Agent + +Build a powerful research companion for academic and professional work: + +- **Academic Search**: Leverage ArXiv search for latest academic papers 📚 +- **Web Research**: Use Bing Search for comprehensive web information 🔍 +- **Structured Analysis**: Apply Sequential Thinking for methodical research 🧠 +- **Data Extraction**: Integrate Firecrawl for web data collection 🕷️ + +**Example Capabilities**: +- Conduct systematic literature reviews +- Extract and synthesize information from multiple sources +- Generate research summaries and bibliographies +- Track research trends and citations +- Organize findings into structured reports + +## 💼 Business Intelligence Agent + +Develop intelligent business analysis and decision support: + +- **Data Integration**: Connect multiple data sources through various MCP servers 📊 +- **Location Analytics**: Use geographic tools for location-based insights 🗺️ +- **Financial Analysis**: Integrate payment systems for financial data 💰 +- **Decision Support**: Apply structured thinking frameworks for analysis 🎯 + +**Example Capabilities**: +- Analyze market trends and customer behavior +- Generate automated business reports +- Provide location-based market insights +- Track financial performance metrics +- Support strategic decision making + +## 🏠 Smart Lifestyle Agent + +Create a personal assistant for daily life optimization: + +- **Shopping Assistant**: Combine mapping services with payment integration 🛒 +- **Commute Optimization**: Use transportation tools for route planning 🚗 +- **Local Discovery**: Integrate web search for local recommendations 🏪 +- **Information Management**: Apply intelligent content extraction 📱 + +**Example Capabilities**: +- Find and compare local services and products +- Optimize daily commute routes +- Discover new restaurants and activities +- Manage personal schedules and reminders +- Handle routine transactions and bookings + +## 🎓 Education Support Agent + +Build learning and teaching assistants: + +- **Content Research**: Access academic databases and papers +- **Interactive Learning**: Provide explanations and tutorials +- **Progress Tracking**: Monitor learning objectives and milestones +- **Resource Discovery**: Find relevant educational materials + +## 🏥 Healthcare Information Agent + +Create health-focused information assistants: + +- **Medical Research**: Search medical literature and guidelines +- **Appointment Management**: Handle scheduling and reminders +- **Health Tracking**: Monitor health metrics and trends +- **Information Synthesis**: Provide clear health information summaries + +## 💡 Creative Content Agent + +Develop agents for content creation and marketing: + +- **Market Research**: Analyze trends and competitor information +- **Content Planning**: Generate ideas and content calendars +- **SEO Optimization**: Research keywords and optimization strategies +- **Multi-platform Publishing**: Coordinate content across platforms + +## Getting Started with Agent Scenarios + +1. **Identify Your Use Case**: Choose a scenario that matches your needs +2. **Select MCP Tools**: Pick the appropriate tools from our ecosystem +3. **Configure Integration**: Set up authentication and permissions +4. **Create Your Agent**: Use Nexent's zero-code platform to build +5. **Test and Iterate**: Refine your agent based on real-world usage + +## Best Practices + +- **Start Simple**: Begin with basic functionality and expand gradually +- **Plan Integration**: Consider how different tools will work together +- **Monitor Performance**: Track usage and optimize tool selection +- **User Feedback**: Continuously improve based on user needs +- **Security First**: Ensure proper authentication and data protection + +Each scenario can be customized and extended based on your specific requirements. The MCP ecosystem provides the flexibility to combine tools in innovative ways, creating powerful AI experiences tailored to your exact needs. \ No newline at end of file diff --git a/doc/docs/en/sdk/basic-usage.md b/doc/docs/en/sdk/basic-usage.md new file mode 100644 index 000000000..005353b36 --- /dev/null +++ b/doc/docs/en/sdk/basic-usage.md @@ -0,0 +1,244 @@ +# 💡 Basic Usage + +This guide provides a comprehensive introduction to using the Nexent SDK for building intelligent agents. + +## 🚀 Quick Start + +### Basic Import + +```python +import nexent +from nexent.core import MessageObserver, ProcessType +from nexent.core.agents import CoreAgent, NexentAgent +from nexent.core.models import OpenAIModel +from nexent.core.tools import ExaSearchTool, KnowledgeBaseSearchTool +``` + +## 🤖 Creating Your First Agent + +### 🔧 Setting Up the Environment + +```python +# Create message observer for streaming output +observer = MessageObserver() + +# Create model (model and Agent must use the same observer) +model = OpenAIModel( + observer=observer, + model_id="your-model-id", + api_key="your-api-key", + api_base="your-api-base" +) +``` + +### 🛠️ Adding Tools + +```python +# Create search tools +search_tool = ExaSearchTool( + exa_api_key="your-exa-key", + observer=observer, + max_results=5 +) + +# Create knowledge base tool +kb_tool = KnowledgeBaseSearchTool( + top_k=5, + observer=observer +) +``` + +### 🤖 Building the Agent + +```python +# Create Agent with tools and model +agent = CoreAgent( + observer=observer, + tools=[search_tool, kb_tool], + model=model, + name="my_agent", + max_steps=5 +) +``` + +### 🚀 Running the Agent + +```python +# Run Agent with your question +result = agent.run("Your question here") + +# Access the final answer +print(result.final_answer) +``` + +## 🎯 Advanced Usage Patterns + +### 🔧 Custom Tool Integration + +```python +from nexent.core.tools import BaseTool + +class CustomTool(BaseTool): + def __init__(self, observer: MessageObserver): + super().__init__(observer=observer, name="custom_tool") + + def run(self, input_text: str) -> str: + # Your custom tool logic here + return f"Processed: {input_text}" + +# Add custom tool to agent +custom_tool = CustomTool(observer=observer) +agent.tools.append(custom_tool) +``` + +### 🎭 Multi-modal Agent Setup + +```python +from nexent.core.models import OpenAIVLMModel + +# Create vision-capable model +vision_model = OpenAIVLMModel( + observer=observer, + model_id="gpt-4-vision-preview", + api_key="your-api-key" +) + +# Create agent with vision capabilities +vision_agent = CoreAgent( + observer=observer, + tools=[search_tool], + model=vision_model, + name="vision_agent" +) +``` + +### 📡 Streaming Output Handling + +```python +# Monitor streaming output +def handle_stream(message: str, process_type: ProcessType): + if process_type == ProcessType.MODEL_OUTPUT_THINKING: + print(f"🤔 Thinking: {message}") + elif process_type == ProcessType.EXECUTION_LOGS: + print(f"⚙️ Executing: {message}") + elif process_type == ProcessType.FINAL_ANSWER: + print(f"✅ Answer: {message}") + +# Set up observer with custom handler +observer.set_message_handler(handle_stream) +``` + +## 🔧 Configuration Options + +### ⚙️ Agent Configuration + +```python +agent = CoreAgent( + observer=observer, + tools=[search_tool, kb_tool], + model=model, + name="my_agent", + max_steps=10, # Maximum execution steps + temperature=0.7, # Model creativity level + system_prompt="You are a helpful AI assistant." # Custom system prompt +) +``` + +### 🔧 Tool Configuration + +```python +# Configure search tool with specific parameters +search_tool = ExaSearchTool( + exa_api_key="your-exa-key", + observer=observer, + max_results=10, # Number of search results + search_type="neural", # Search type: neural, keyword, etc. + include_domains=["example.com"], # Limit search to specific domains + exclude_domains=["spam.com"] # Exclude specific domains +) +``` + +## 📊 Error Handling + +### 🛡️ Graceful Error Recovery + +```python +try: + result = agent.run("Your question") + print(f"Success: {result.final_answer}") +except Exception as e: + print(f"Error occurred: {e}") + # Handle error appropriately +``` + +### 🔧 Tool Error Handling + +```python +# Tools automatically handle errors and provide fallbacks +search_tool = ExaSearchTool( + exa_api_key="your-exa-key", + observer=observer, + max_results=5, + fallback_to_keyword=True # Fallback to keyword search if neural fails +) +``` + +## 🎭 Multi-modal Examples + +### 🖼️ Image Processing + +```python +# Process image with vision model +result = vision_agent.run( + "Describe what you see in this image", + image_path="path/to/image.jpg" +) +``` + +### 🎤 Voice Processing + +```python +from nexent.core.tools import SpeechToTextTool, TextToSpeechTool + +# Add voice capabilities +stt_tool = SpeechToTextTool(observer=observer) +tts_tool = TextToSpeechTool(observer=observer) + +voice_agent = CoreAgent( + observer=observer, + tools=[stt_tool, tts_tool, search_tool], + model=model, + name="voice_agent" +) +``` + +## 🔍 Best Practices + +### ⚡ Performance Optimization + +- **Connection Pooling**: Reuse connections for better performance +- **Batch Processing**: Process multiple requests together when possible +- **Caching**: Implement caching for frequently accessed data +- **Async Operations**: Use async/await for I/O intensive operations + +### 🔒 Security Considerations + +- **API Key Management**: Store API keys securely using environment variables +- **Input Validation**: Validate all inputs before processing +- **Rate Limiting**: Implement rate limiting to prevent abuse +- **Error Logging**: Log errors without exposing sensitive information + +### 🔍 Monitoring and Debugging + +```python +# Enable detailed logging +import logging +logging.basicConfig(level=logging.DEBUG) + +# Monitor agent execution +for step in agent.execution_steps: + print(f"Step {step.step_number}: {step.action}") + print(f"Result: {step.result}") +``` + +For more advanced usage patterns and detailed API documentation, see the **[Core Components](./overview)** and **[Tool Development](./core/tools)** guides. \ No newline at end of file diff --git a/doc/docs/en/sdk/core/models.md b/doc/docs/en/sdk/core/models.md new file mode 100644 index 000000000..044262144 --- /dev/null +++ b/doc/docs/en/sdk/core/models.md @@ -0,0 +1,58 @@ +# Nexent Model Architecture + +Nexent provides a comprehensive model architecture supporting multiple AI model types through OpenAI-compatible interfaces. The SDK supports large language models, multimodal models, embedding models, and speech processing capabilities. + +## Overview + +The models module provides standardized interfaces for various AI model providers and types: + +## Supported Model Categories + +### 🤖 Large Language Models (LLM) +- **OpenAI-compatible models**: Any provider following OpenAI API specification +- **Long context models**: Support for extended context windows +- **Multimodal language models**: Text + image processing capabilities +- **Local deployment**: Ollama, vLLM, and other self-hosted solutions + +### 🎭 Vision Language Models (VLM) +- **Multimodal understanding**: Process text, images, and documents simultaneously +- **OpenAI-compatible VLMs**: GPT-4V, Claude-3, and compatible models +- **Document analysis**: OCR, table extraction, and visual reasoning + +### 🔤 Embedding Models +- **Universal compatibility**: All OpenAI-compatible embedding services +- **Multilingual support**: International language processing +- **Specialized embeddings**: Document, code, and domain-specific embeddings +- **Vector database integration**: Seamless integration with vector stores + +### 🎤 Speech Processing Models +- **Text-to-Speech (TTS)**: Multiple provider support +- **Speech-to-Text (STT)**: Real-time and batch processing +- **Voice cloning**: Advanced voice synthesis capabilities +- **Multilingual speech**: Support for multiple languages and accents + +## Model Implementation Classes + +## Usage + +```python +from nexent.core.models import OpenAIModel + +# Initialize OpenAI model +model = OpenAIModel( + api_key="your-api-key", + model_name="gpt-4" +) + +# Use model for completion +response = model.complete("Hello, world!") +``` + +## Configuration + +Models can be configured through: +- Environment variables +- Configuration files +- Direct parameter passing + +For detailed usage examples and API reference, see the SDK documentation. \ No newline at end of file diff --git a/doc/docs/en/sdk/core/tools.md b/doc/docs/en/sdk/core/tools.md new file mode 100644 index 000000000..bd3184ba9 --- /dev/null +++ b/doc/docs/en/sdk/core/tools.md @@ -0,0 +1,355 @@ +# Nexent Tool Development Guidelines + +This document summarizes the complete guidelines and best practices for tool development in the Nexent SDK based on analysis of existing tools. + +## Tool Categories + +The current SDK includes the following tool types: + +### Search Tools +- **ExaSearchTool**: Web search tool based on EXA API +- **TavilySearchTool**: Web search tool based on Tavily API +- **LinkupSearchTool**: Search tool based on Linkup API +- **KnowledgeBaseSearchTool**: Local knowledge base search tool + +### File Management Tools +- **CreateFileTool**: Create new files with content +- **ReadFileTool**: Read file contents from the filesystem +- **DeleteFileTool**: Delete files from the filesystem +- **MoveItemTool**: Move or rename files and directories +- **CreateDirectoryTool**: Create new directories +- **DeleteDirectoryTool**: Delete directories and their contents +- **ListDirectoryTool**: List directory contents with detailed information + +### System Tools +- **TerminalTool**: Execute shell commands and system operations + +### Communication Tools +- **GetEmailTool**: Email retrieval tool via IMAP +- **SendEmailTool**: Email sending tool via SMTP + +## Common Characteristics + +### 1. Basic Architecture +- **Base Class Inheritance**: All tools must inherit from `smolagents.tools.Tool` +- **Parameter Management**: Use `pydantic.Field` for parameter definition and validation +- **Streaming Output**: Integrate `MessageObserver` for real-time message transmission +- **Multi-language Support**: Built-in Chinese and English bilingual prompts + +### 2. Core Attributes +Each tool class must include the following class attributes: + +```python +class ToolExample(Tool): + name = "tool_name" # Tool unique identifier + description = "Tool functionality description" # Detailed feature description + inputs = { # Input parameter definition + "param": {"type": "string", "description": "Parameter description"} + } + output_type = "string" # Output type + tool_sign = "x" # Tool identifier (optional) +``` + +### 3. Message Processing Mechanism +- **ProcessType Enumeration**: Use different types to distinguish messages (TOOL, CARD, SEARCH_CONTENT, PICTURE_WEB, etc.) +- **Observer Pattern**: Implement real-time message pushing through MessageObserver +- **JSON Format**: All message content uses JSON format to ensure consistency + +### 4. Exception Handling Strategy +- **Unified Exceptions**: Use Exception to throw error messages +- **Error Logging**: Use logging module to record detailed error information +- **Graceful Degradation**: Provide fallback solutions when possible + +## Naming Conventions + +### File Naming +- **Format**: `{function_name}_tool.py` +- **Style**: Lowercase letters, words connected by underscores +- **Examples**: `exa_search_tool.py`, `knowledge_base_search_tool.py` + +### Class Naming +- **Format**: `{FunctionName}Tool` +- **Style**: PascalCase +- **Examples**: `ExaSearchTool`, `KnowledgeBaseSearchTool` + +### Attribute and Method Naming +- **Format**: Lowercase letters, words connected by underscores +- **Private Methods**: Start with single underscore (e.g., `_filter_images`) +- **Examples**: `max_results`, `running_prompt_en`, `_decode_subject` + +### Tool Identifier Conventions +- **tool_sign**: Single letter identifier for distinguishing tool sources +- **Assignment Rules**: + - `a`: Knowledge base search (KnowledgeBaseSearchTool) + - `b`: Web search (ExaSearchTool, TavilySearchTool) + - `l`: Linkup search (LinkupSearchTool) + - Other letters assigned by functional type + +## Code Structure Templates + +### Basic Template + +```python +import json +import logging +from typing import Optional +from smolagents.tools import Tool +from pydantic import Field + +from ..utils.observer import MessageObserver, ProcessType + +logger = logging.getLogger("your_tool_name") + +class YourTool(Tool): + name = "your_tool" + description = "Detailed description of tool functionality, including use cases and methods" + inputs = { + "param1": { + "type": "string", + "description": "Detailed description of parameter 1" + }, + "param2": { + "type": "integer", + "description": "Detailed description of parameter 2", + "default": 10, + "nullable": True + } + } + output_type = "string" + tool_sign = "y" # Choose appropriate identifier + + def __init__( + self, + config_param: str = Field(description="Configuration parameter"), + observer: MessageObserver = Field(description="Message observer", default=None, exclude=True), + optional_param: int = Field(description="Optional parameter", default=5) + ): + super().__init__() + self.config_param = config_param + self.observer = observer + self.optional_param = optional_param + + # Multi-language prompt messages + self.running_prompt_zh = "正在执行..." + self.running_prompt_en = "Processing..." + + # Record operation sequence number (if needed) + self.record_ops = 0 + + def forward(self, param1: str, param2: int = 10) -> str: + """Main execution method of the tool + + Args: + param1: Description of parameter 1 + param2: Description of parameter 2 + + Returns: + JSON format string result + + Raises: + Exception: Detailed error information + """ + try: + # Send tool running message + if self.observer: + running_prompt = (self.running_prompt_zh + if self.observer.lang == "zh" + else self.running_prompt_en) + self.observer.add_message("", ProcessType.TOOL, running_prompt) + + # Send card information (optional) + card_content = [{"icon": "your_icon", "text": param1}] + self.observer.add_message("", ProcessType.CARD, + json.dumps(card_content, ensure_ascii=False)) + + # Main business logic + result = self._execute_main_logic(param1, param2) + + # Process results and return + return self._format_result(result) + + except Exception as e: + logger.error(f"Error in {self.name}: {str(e)}") + raise Exception(f"Error executing {self.name}: {str(e)}") + + def _execute_main_logic(self, param1: str, param2: int): + """Private method to execute main business logic""" + # Implement specific business logic + pass + + def _format_result(self, result) -> str: + """Format return result""" + formatted_result = { + "status": "success", + "data": result, + "tool": self.name + } + return json.dumps(formatted_result, ensure_ascii=False) +``` + +### Search Tool Template + +```python +import json +import logging +from typing import List +from smolagents.tools import Tool +from pydantic import Field + +from ..utils.observer import MessageObserver, ProcessType +from ..utils.tools_common_message import SearchResultTextMessage + +logger = logging.getLogger("search_tool_name") + +class SearchTool(Tool): + name = "search_tool" + description = "Detailed description of search tool, including search scope and use cases" + inputs = { + "query": {"type": "string", "description": "Search query"}, + "max_results": {"type": "integer", "description": "Maximum number of results", "default": 5, "nullable": True} + } + output_type = "string" + tool_sign = "s" + + def __init__( + self, + api_key: str = Field(description="API key"), + observer: MessageObserver = Field(description="Message observer", default=None, exclude=True), + max_results: int = Field(description="Maximum number of search results", default=5) + ): + super().__init__() + self.api_key = api_key + self.observer = observer + self.max_results = max_results + self.record_ops = 0 + + self.running_prompt_zh = "搜索中..." + self.running_prompt_en = "Searching..." + + def forward(self, query: str, max_results: int = None) -> str: + if max_results is None: + max_results = self.max_results + + # Send search status message + if self.observer: + running_prompt = (self.running_prompt_zh + if self.observer.lang == "zh" + else self.running_prompt_en) + self.observer.add_message("", ProcessType.TOOL, running_prompt) + card_content = [{"icon": "search", "text": query}] + self.observer.add_message("", ProcessType.CARD, + json.dumps(card_content, ensure_ascii=False)) + + try: + # Perform search + search_results = self._perform_search(query, max_results) + + if not search_results: + raise Exception("No search results found! Try a shorter or broader query.") + + # Format search results + formatted_results = self._format_search_results(search_results) + + # Record search content + if self.observer: + search_results_data = json.dumps(formatted_results["json"], ensure_ascii=False) + self.observer.add_message("", ProcessType.SEARCH_CONTENT, search_results_data) + + return json.dumps(formatted_results["return"], ensure_ascii=False) + + except Exception as e: + logger.error(f"Search error: {str(e)}") + raise Exception(f"Search failed: {str(e)}") + + def _perform_search(self, query: str, max_results: int): + """Execute actual search operation""" + # Implement specific search logic + pass + + def _format_search_results(self, results): + """Format search results into unified format""" + search_results_json = [] + search_results_return = [] + + for index, result in enumerate(results): + search_result_message = SearchResultTextMessage( + title=result.get("title", ""), + url=result.get("url", ""), + text=result.get("content", ""), + published_date=result.get("date", ""), + source_type="url", + filename="", + score=result.get("score", ""), + score_details=result.get("score_details", {}), + cite_index=self.record_ops + index, + search_type=self.name, + tool_sign=self.tool_sign + ) + search_results_json.append(search_result_message.to_dict()) + search_results_return.append(search_result_message.to_model_dict()) + + self.record_ops += len(search_results_return) + + return { + "json": search_results_json, + "return": search_results_return + } +``` + +## Development Process Guidelines + +### 1. Pre-development Preparation +- Determine tool functionality and use cases +- Select appropriate tool category and identifier +- Check for functionality duplication with existing tools + +### 2. Implementation Steps +1. **Create tool file**: Create `{name}_tool.py` according to naming conventions +2. **Define class structure**: Inherit from Tool base class, define necessary attributes +3. **Implement constructor**: Use pydantic Field to define parameters +4. **Implement forward method**: Core functionality logic +5. **Add private methods**: Split complex logic into private methods +6. **Integrate message observer**: Support streaming output and multi-language +7. **Exception handling**: Complete error handling and logging + +### 3. Testing and Integration +1. **Unit testing**: Test various input scenarios and edge cases +2. **Integration testing**: Integration testing with CoreAgent +3. **Update exports**: Add tool export in `__init__.py` +4. **Documentation updates**: Update related documentation and examples + +## Best Practices + +### 1. Performance Optimization +- **Asynchronous Processing**: Use asynchronous processing for time-consuming operations +- **Connection Pooling**: Reuse network connections to reduce latency +- **Caching Mechanism**: Use caching appropriately to improve response speed +- **Concurrency Control**: Use Semaphore to control concurrent request numbers + +### 2. Security +- **Input Validation**: Strictly validate input parameters +- **Sensitive Information**: API keys and other sensitive information should not appear in logs +- **Error Messages**: Avoid leaking sensitive information in error messages +- **Timeout Control**: Set reasonable timeout periods to prevent blocking + +### 3. Maintainability +- **Modular Design**: Split complex functionality into multiple methods +- **Clear Comments**: Add detailed comments for complex logic +- **Type Annotations**: Use complete type annotations +- **Documentation Strings**: Add documentation strings for all public methods + +### 4. User Experience +- **Multi-language Support**: Provide Chinese and English bilingual prompts +- **Progress Feedback**: Provide real-time feedback through MessageObserver +- **Error Prompts**: Provide clear error messages and solution suggestions +- **Parameter Validation**: Validate parameter validity before execution + +## Important Notes + +1. **Version Compatibility**: Ensure tools are compatible with different versions of dependency libraries +2. **Resource Cleanup**: Release network connections, file handles, and other resources promptly +3. **Log Levels**: Set appropriate log levels to avoid excessive debug information +4. **Configuration Management**: Support configuring key parameters through environment variables +5. **Error Recovery**: Provide error recovery mechanisms when possible + +By following these guidelines, you can ensure that newly developed tools maintain consistency with existing tools and have good maintainability and extensibility. \ No newline at end of file diff --git a/doc/docs/en/sdk/data-process.md b/doc/docs/en/sdk/data-process.md new file mode 100644 index 000000000..2745e4fdc --- /dev/null +++ b/doc/docs/en/sdk/data-process.md @@ -0,0 +1,83 @@ +# Data Processing Core + +## Overview + +`DataProcessCore` is a unified file processing core class that supports automatic detection and processing of multiple file formats, providing flexible chunking strategies and multiple input source support. + +## Key Features + +### 1. Core Processing Method: `file_process()` + +**Function Signature:** +```python +def file_process(self, + file_path_or_url: Optional[str] = None, + file_data: Optional[bytes] = None, + chunking_strategy: str = "basic", + destination: str = "local", + filename: Optional[str] = None, + **params) -> List[Dict] +``` + +**Parameters:** + +| Parameter | Type | Required | Description | Options | +|-----------|------|----------|-------------|---------| +| `file_path_or_url` | `str` | No* | Local file path or remote URL | Any valid file path or URL | +| `file_data` | `bytes` | No* | File byte data (for memory processing) | Any valid byte data | +| `chunking_strategy` | `str` | No | Chunking strategy | `"basic"`, `"by_title"`, `"none"` | +| `destination` | `str` | No | Destination type, indicating file source | `"local"`, `"minio"`, `"url"` | +| `filename` | `str` | No** | Filename | Any valid filename | +| `**params` | `dict` | No | Additional processing parameters | See parameter details below | + +*Note: Either `file_path_or_url` or `file_data` must be provided +**Note: When using `file_data`, `filename` is required + +**Chunking Strategy (`chunking_strategy`) Details:** + +| Strategy | Description | Use Case | Output Characteristics | +|----------|-------------|----------|----------------------| +| `"basic"` | Basic chunking strategy | Most document processing scenarios | Automatic chunking based on content length | +| `"by_title"` | Title-based chunking | Structured documents with clear headings | Chunks divided by document structure | +| `"none"` | No chunking | Small files or when full content is needed | Returns complete content without chunking | + +## Supported File Formats + +- **Text files**: .txt, .md, .csv +- **Documents**: .pdf, .docx, .pptx +- **Images**: .jpg, .png, .gif (with OCR) +- **Web content**: HTML, URLs +- **Archives**: .zip, .tar + +## Usage Examples + +```python +from nexent.data_process import DataProcessCore + +# Initialize processor +processor = DataProcessCore() + +# Process local file +results = processor.file_process( + file_path_or_url="/path/to/document.pdf", + chunking_strategy="by_title" +) + +# Process from URL +results = processor.file_process( + file_path_or_url="https://example.com/document.pdf", + destination="url" +) + +# Process from memory +with open("document.pdf", "rb") as f: + file_data = f.read() + +results = processor.file_process( + file_data=file_data, + filename="document.pdf", + chunking_strategy="basic" +) +``` + +For detailed configuration and advanced usage, see the complete SDK documentation. \ No newline at end of file diff --git a/doc/docs/en/sdk/overview.md b/doc/docs/en/sdk/overview.md new file mode 100644 index 000000000..73171b1a4 --- /dev/null +++ b/doc/docs/en/sdk/overview.md @@ -0,0 +1,151 @@ +# Nexent SDK + +Nexent is a powerful, enterprise-grade Agent SDK that revolutionizes intelligent agent development. Built on proven enterprise architecture, it provides a comprehensive solution for building production-ready AI agents with distributed processing, real-time streaming, multi-modal capabilities, and an extensive ecosystem of tools and models. + +## 🚀 Installation + +### User Installation +If you want to use nexent: + +```bash +# Install using uv (recommended) +uv add nexent + +# Or install from source +uv pip install . +``` + +### Development Environment Setup +If you are a third-party SDK developer: + +```bash +# Method 1: Install dependencies only (without nexent) +uv pip install -r requirements.txt + +# Method 2: Install complete development environment (including nexent) +uv pip install -e ".[dev]" # Includes all development tools (testing, code quality checks, etc.) +``` + +The development environment includes the following additional features: +- Code quality checking tools (ruff) +- Testing framework (pytest) +- Data processing dependencies (unstructured) +- Other development dependencies + +## 📖 Basic Usage + +For comprehensive usage examples and step-by-step guides, see our **[Basic Usage Guide](./basic-usage)**. + + + +## ⭐ Key Features + +### 🏢 Enterprise-grade Agent Framework +- **Extended from SmolAgent**: Supporting complex business scenarios +- **Production Ready**: Built for enterprise environments with proper scaling and monitoring +- **Comprehensive Testing**: Extensive test coverage ensuring reliability + +### ⚡ Distributed Processing Capabilities +- **Asynchronous Processing**: High-performance asynchronous architecture based on asyncio +- **Multi-threading Support**: Thread-safe concurrent processing mechanisms +- **Celery-friendly**: Design optimized for distributed task queues +- **Batch Operations**: Support for large-scale data batch processing and optimization + +### 🔧 Rich Agent Tool Ecosystem +- **Search Tools**: EXA, Tavily, Linkup web search and local knowledge base retrieval +- **Communication Tools**: IMAP/SMTP email functionality +- **MCP Integration**: Support for Model Context Protocol tool integration +- **Unified Standards**: All tools follow consistent development standards and interface design + +### 🎭 Multi-modal Support +- **Voice Services**: Integrated STT & TTS, supporting real-time voice interaction +- **Vision Models**: Support for image understanding and processing +- **Long Context Models**: Processing large-scale documents and conversation history + +### 📊 Powerful Data Processing Capabilities +- **Multi-format Support**: Processing 20+ formats including PDF, Word, Excel, HTML +- **Intelligent Chunking**: Support for basic chunking, title chunking, no chunking strategies +- **Memory Processing**: Support for streaming processing of large files in memory +- **Distributed Architecture**: Support for task queue management and parallel processing + +### 🔍 Vector Database Integration +- **Elasticsearch**: Enterprise-grade vector search and document management +- **Hybrid Search**: Combining exact matching and semantic search +- **Embedding Models**: Integration with mainstream embedding models like Jina +- **Large-scale Optimization**: Support for efficient retrieval of millions of documents + +## 🏗️ Core Components + +### 🤖 NexentAgent - Enterprise-grade Agent Framework + +The core of nexent is the `NexentAgent` and `CoreAgent` classes, providing complete intelligent agent solutions: + +- **Multi-model Support**: Support for OpenAI, vision language models, long context models, etc. +- **MCP Integration**: Seamless integration with Model Context Protocol tool ecosystem +- **Dynamic Tool Loading**: Support for dynamic creation and management of local and MCP tools +- **Distributed Execution**: High-performance execution engine based on thread pools and asynchronous architecture +- **State Management**: Comprehensive task state tracking and error recovery mechanisms + +### ⚙️ CoreAgent - Code Execution Engine + +Inherits and enhances SmolAgent's `CodeAgent`, providing the following key capabilities: + +- **Python Code Execution**: Support for parsing and executing Python code, capable of dynamic task processing +- **Multi-language Support**: Built-in Chinese and English prompt templates, switchable as needed +- **Streaming Output**: Real-time streaming display of model output through MessageObserver +- **Step Tracking**: Record and display each step of Agent execution for debugging and monitoring +- **Interrupt Control**: Support for task interruption and graceful stop mechanisms +- **Error Handling**: Comprehensive error handling mechanisms to improve stability +- **State Management**: Maintain and pass execution state, supporting continuous processing of complex tasks + +CoreAgent implements the ReAct framework's think-act-observe loop: +1. **Think**: Use large language models to generate solution code +2. **Act**: Execute the generated Python code +3. **Observe**: Collect execution results and logs +4. **Repeat**: Continue thinking and executing based on observation results until the task is completed + +### 📡 MessageObserver - Streaming Message Processing + +Core implementation of the message observer pattern, used to handle Agent's streaming output: + +- **Streaming Output Capture**: Real-time capture of tokens generated by the model +- **Process Type Distinction**: Format output according to different processing stages (model output, code parsing, execution logs, etc.) +- **Multi-language Support**: Support for Chinese and English output formats +- **Unified Interface**: Provide unified processing methods for messages from different sources + +ProcessType enumeration defines the following processing stages: +- `STEP_COUNT`: Current execution step +- `MODEL_OUTPUT_THINKING`: Model thinking process output +- `MODEL_OUTPUT_CODE`: Model code generation output +- `PARSE`: Code parsing results +- `EXECUTION_LOGS`: Code execution results +- `AGENT_NEW_RUN`: Agent basic information +- `FINAL_ANSWER`: Final summary results +- `SEARCH_CONTENT`: Search result content +- `PICTURE_WEB`: Web image processing results + +## 🛠️ Tool Suite + +Nexent provides a rich tool ecosystem supporting multiple types of task processing. All tools follow unified development standards to ensure consistency and extensibility. + +For detailed tool development standards and comprehensive tool documentation, please refer to: **[Tool Development Guide](./core/tools)** + +## 📊 Data Processing Service + +Enterprise-grade document processing service providing distributed processing capabilities with support for 20+ document formats, intelligent chunking strategies, and memory optimization. + +For detailed data processing capabilities and usage examples, please refer to: **[Data Processing Guide](./data-process)** + +## 🔍 Vector Database Service + +Provides enterprise-grade vector search and document management services with multiple search modes, embedding model integration, and large-scale optimization. + +For comprehensive vector database integration and configuration, please refer to: **[Vector Database Guide](./vector-database)** + +## 🤖 Model Services + +Provides unified multi-modal AI model services with support for various model types and providers. + +For comprehensive model integration and usage documentation, please refer to: **[Model Architecture Guide](./core/models)** + +For more detailed usage instructions, please refer to the dedicated documentation for each module. \ No newline at end of file diff --git a/doc/docs/en/sdk/vector-database.md b/doc/docs/en/sdk/vector-database.md new file mode 100644 index 000000000..a0f6dc626 --- /dev/null +++ b/doc/docs/en/sdk/vector-database.md @@ -0,0 +1,57 @@ +# Vector Database + +This document describes the vector database functionality in the Nexent SDK. + +## Overview + +The vector database module provides efficient storage and retrieval of high-dimensional vector embeddings with support for: + +- Vector similarity search +- Metadata filtering +- Batch operations +- Multiple backend support + +## Supported Backends + +### Elasticsearch +High-performance distributed search and analytics engine. + +### Qdrant +Vector similarity search engine optimized for performance. + +### FAISS +Library for efficient similarity search and clustering of dense vectors. + +## Usage + +```python +from nexent.vector_database import VectorDB + +# Initialize vector database +vector_db = VectorDB( + backend="elasticsearch", + host="localhost", + port=9200 +) + +# Insert vectors +vector_db.insert( + vectors=embeddings, + metadata={"source": "document.pdf"} +) + +# Search similar vectors +results = vector_db.search( + query_vector=query_embedding, + top_k=10 +) +``` + +## Configuration + +The vector database can be configured through: +- Connection parameters +- Index settings +- Search parameters + +For detailed configuration options, see the SDK documentation. \ No newline at end of file diff --git a/doc/docs/en/security.md b/doc/docs/en/security.md new file mode 100644 index 000000000..693d1898b --- /dev/null +++ b/doc/docs/en/security.md @@ -0,0 +1,29 @@ +# Security Policy + +**Please do not report security vulnerabilities through public GitHub issues, discussions, or other public channels.** + +Instead, please disclose them responsibly by contacting our security team at: +📧 [chenshuangrui@gmail.com](mailto:chenshuangrui@gmail.com) + +## What to Include: +- Detailed description of the vulnerability +- Steps to reproduce +- Potential impact assessment +- Suggested fixes or mitigations (if known) + +## Our Response Process: +- Acknowledgement within **48 hours** +- Initial assessment within **5 business days** +- Regular updates on remediation progress +- Public disclosure timeline coordinated with reporter + +## Security Updates +Critical security patches are released as soon as they're available. All security-related updates will be marked with **[SECURITY]** in release notes. + +## Recognition +While we don't currently have a formal bug bounty program, we gratefully acknowledge responsible disclosures by: +- Listing contributors in our Security Hall of Fame +- Providing written recommendations (upon request) +- Public thank-you in release notes (with permission) + +**Note:** This policy may be updated periodically. Last revised: January 2025 \ No newline at end of file diff --git a/doc/docs/en/testing/backend.md b/doc/docs/en/testing/backend.md new file mode 100644 index 000000000..879d0dfbc --- /dev/null +++ b/doc/docs/en/testing/backend.md @@ -0,0 +1,372 @@ +# Backend Testing + +This guide covers the comprehensive backend testing framework used in Nexent, including API testing, service layer testing, and utility function testing. + +## Test Structure + +The backend tests are organized in the following structure: + +``` +test/backend/ +├── app/ # API endpoint tests +│ ├── test_agent_app.py +│ ├── test_base_app.py +│ ├── test_config_sync_app.py +│ ├── test_conversation_management_app.py +│ ├── test_data_process_app.py +│ ├── test_elasticsearch_app.py +│ ├── test_file_management_app.py +│ ├── test_image_app.py +│ ├── test_knowledge_app.py +│ ├── test_knowledge_summary_app.py +│ ├── test_me_model_managment_app.py +│ ├── test_model_managment_app.py +│ ├── test_prompt_app.py +│ └── test_remote_mcp_app.py +├── services/ # Service layer tests +│ ├── test_agent_service.py +│ ├── test_conversation_management_service.py +│ ├── test_data_process_service.py +│ ├── test_elasticsearch_service.py +│ ├── test_file_management_service.py +│ ├── test_image_service.py +│ ├── test_knowledge_service.py +│ ├── test_knowledge_summary_service.py +│ ├── test_model_management_service.py +│ ├── test_prompt_service.py +│ └── test_remote_mcp_service.py +├── utils/ # Utility function tests +│ ├── test_langchain_utils.py +│ └── test_prompt_template_utils.py +└── run_all_test.py # Backend test runner +``` + +## Running Backend Tests + +### Complete Backend Test Suite + +```bash +# From project root +python test/backend/run_all_test.py + +# From test/backend directory +cd test/backend +python run_all_test.py +``` + +### Individual Test Categories + +```bash +# Run all API tests +python -m pytest test/backend/app/ -v + +# Run all service tests +python -m pytest test/backend/services/ -v + +# Run all utility tests +python -m pytest test/backend/utils/ -v +``` + +### Specific Test Files + +```bash +# Run specific API test +python -m pytest test/backend/app/test_agent_app.py -v + +# Run specific service test +python -m pytest test/backend/services/test_agent_service.py -v + +# Run specific utility test +python -m pytest test/backend/utils/test_langchain_utils.py -v +``` + +## API Testing + +API tests use FastAPI's TestClient to simulate HTTP requests without running an actual server. + +### Test Setup Pattern + +```python +import os +import sys +from unittest.mock import patch, MagicMock +from fastapi.testclient import TestClient +from fastapi import FastAPI + +# Dynamically determine the backend path +current_dir = os.path.dirname(os.path.abspath(__file__)) +backend_dir = os.path.abspath(os.path.join(current_dir, "../../../backend")) +sys.path.append(backend_dir) + +# Setup patches for dependencies before importing modules +patches = [ + patch('botocore.client.BaseClient._make_api_call', return_value={}), + patch('backend.database.client.MinioClient', MagicMock()), + patch('backend.database.client.db_client', MagicMock()), + patch('backend.utils.auth_utils.get_current_user_id', + MagicMock(return_value=('test_user', 'test_tenant'))), + patch('httpx.AsyncClient', MagicMock()) +] + +# Start all patches +for p in patches: + p.start() + +# Import modules after applying patches +from backend.apps.agent_app import router + +# Create test app +app = FastAPI() +app.include_router(router) +client = TestClient(app) +``` + +### API Test Example + +```python +class TestAgentApp(unittest.TestCase): + + def setUp(self): + # Setup test client and common mocks + pass + + def test_create_agent_success(self): + """Test successful agent creation""" + # Setup + agent_data = { + "name": "Test Agent", + "description": "A test agent", + "system_prompt": "You are a test agent" + } + + # Execute + response = client.post("/agents", json=agent_data) + + # Assert + self.assertEqual(response.status_code, 200) + self.assertIn("id", response.json()) + self.assertEqual(response.json()["name"], "Test Agent") + + def test_create_agent_invalid_data(self): + """Test agent creation with invalid data""" + # Setup + invalid_data = {"name": ""} # Missing required fields + + # Execute + response = client.post("/agents", json=invalid_data) + + # Assert + self.assertEqual(response.status_code, 422) # Validation error +``` + +## Service Layer Testing + +Service layer tests focus on business logic and data processing without HTTP overhead. + +### Service Test Pattern + +```python +class TestAgentService(unittest.TestCase): + + @patch("backend.database.agent_db.save_agent") + @patch("backend.utils.auth_utils.get_current_user_id") + async def test_create_agent_success(self, mock_get_user, mock_save_agent): + # Setup + mock_get_user.return_value = ("user123", "tenant456") + mock_save_agent.return_value = {"id": 1, "name": "Test Agent"} + + # Execute + result = await create_agent( + name="Test Agent", + description="A test agent", + system_prompt="You are a test agent" + ) + + # Assert + mock_save_agent.assert_called_once() + self.assertEqual(result["name"], "Test Agent") + self.assertIn("id", result) +``` + +### Mocking Database Operations + +```python +@patch("backend.database.agent_db.query_agent_by_id") +@patch("backend.database.agent_db.update_agent") +async def test_update_agent_success(self, mock_update, mock_query): + # Setup + mock_query.return_value = {"id": 1, "name": "Old Name"} + mock_update.return_value = {"id": 1, "name": "New Name"} + + # Execute + result = await update_agent(agent_id=1, name="New Name") + + # Assert + mock_update.assert_called_once_with(agent_id=1, name="New Name") + self.assertEqual(result["name"], "New Name") +``` + +## Utility Function Testing + +Utility functions are tested in isolation with mocked dependencies. + +### Utility Test Example + +```python +class TestLangchainUtils(unittest.TestCase): + + @patch("langchain.llms.openai.OpenAI") + def test_create_llm_instance(self, mock_openai): + # Setup + mock_openai.return_value = MagicMock() + + # Execute + llm = create_llm_instance(model_name="gpt-3.5-turbo") + + # Assert + mock_openai.assert_called_once() + self.assertIsNotNone(llm) +``` + +## Testing Asynchronous Code + +Backend tests handle both synchronous and asynchronous code: + +### Async Test Pattern + +```python +class TestAsyncService(unittest.TestCase): + + @patch("backend.database.agent_db.async_query") + async def test_async_operation(self, mock_async_query): + # Setup + mock_async_query.return_value = {"result": "success"} + + # Execute + result = await async_operation() + + # Assert + self.assertEqual(result["result"], "success") + mock_async_query.assert_called_once() +``` + +## Error Handling Tests + +Comprehensive error handling is tested: + +```python +def test_api_error_handling(self): + """Test API error responses""" + # Setup - mock service to raise exception + with patch('backend.services.agent_service.create_agent') as mock_service: + mock_service.side_effect = Exception("Database error") + + # Execute + response = client.post("/agents", json={"name": "Test"}) + + # Assert + self.assertEqual(response.status_code, 500) + self.assertIn("error", response.json()) +``` + +## Authentication and Authorization Tests + +Security-related functionality is thoroughly tested: + +```python +def test_authentication_required(self): + """Test that endpoints require authentication""" + # Execute without auth header + response = client.get("/agents") + + # Assert + self.assertEqual(response.status_code, 401) + +def test_tenant_isolation(self): + """Test that users can only access their tenant's data""" + # Setup - mock auth to return different tenant + with patch('backend.utils.auth_utils.get_current_user_id') as mock_auth: + mock_auth.return_value = ("user1", "tenant1") + + # Execute + response = client.get("/agents") + + # Assert - verify tenant filtering is applied + # This would check that the service layer filters by tenant +``` + +## Coverage Analysis + +Backend tests generate detailed coverage reports: + +### Coverage Commands + +```bash +# Generate coverage report +python -m pytest test/backend/ --cov=backend --cov-report=html --cov-report=xml + +# View coverage in terminal +python -m pytest test/backend/ --cov=backend --cov-report=term-missing +``` + +### Coverage Targets + +- **API Endpoints**: 90%+ coverage +- **Service Layer**: 85%+ coverage +- **Utility Functions**: 80%+ coverage +- **Error Handling**: 100% coverage for critical paths + +## Test Data Management + +### Fixtures and Test Data + +```python +class TestWithFixtures(unittest.TestCase): + + def setUp(self): + """Set up test data and mocks""" + self.test_agent = { + "id": 1, + "name": "Test Agent", + "description": "A test agent", + "system_prompt": "You are a test agent" + } + + self.test_user = ("user123", "tenant456") + + def tearDown(self): + """Clean up after tests""" + # Reset any global state if needed + pass +``` + +## Performance Testing + +Backend tests include performance considerations: + +```python +def test_api_response_time(self): + """Test that API responses are within acceptable time limits""" + import time + + start_time = time.time() + response = client.get("/agents") + end_time = time.time() + + # Assert response time is under 100ms + self.assertLess(end_time - start_time, 0.1) + self.assertEqual(response.status_code, 200) +``` + +## Best Practices for Backend Testing + +1. **Mock External Dependencies**: Always mock database, external APIs, and services +2. **Test Both Success and Failure**: Cover all possible code paths +3. **Use Descriptive Test Names**: Make it clear what each test validates +4. **Keep Tests Independent**: Each test should run in isolation +5. **Test Edge Cases**: Include boundary conditions and error scenarios +6. **Maintain Test Data**: Use consistent, realistic test data +7. **Document Complex Tests**: Add comments for complex test scenarios +8. **Regular Coverage Reviews**: Monitor and improve coverage over time + +This comprehensive backend testing framework ensures that all backend functionality is thoroughly validated before deployment, maintaining high code quality and reliability. \ No newline at end of file diff --git a/doc/docs/en/testing/overview.md b/doc/docs/en/testing/overview.md new file mode 100644 index 000000000..72f6aef7d --- /dev/null +++ b/doc/docs/en/testing/overview.md @@ -0,0 +1,240 @@ +# Testing Overview + +Nexent provides a comprehensive testing framework that ensures code quality and reliability across all components. This guide covers the testing strategy, tools, and best practices used throughout the project. + +## Testing Philosophy + +Our testing approach is built on four core principles: + +1. **Isolate the unit**: Mock all external dependencies +2. **Control the environment**: Set up precise test conditions +3. **Test the interface**: Focus on inputs and outputs +4. **Verify behavior**: Check both results and interactions + +This ensures that tests are reliable, fast, and don't affect real systems or data. + +## Testing Framework + +The project uses a combination of testing frameworks: + +- **unittest**: Python's standard unit testing framework for test organization and assertions +- **unittest.mock**: For mocking dependencies and isolating components +- **TestClient** from FastAPI: For testing API endpoints without running an actual server +- **pytest**: For advanced test discovery and execution +- **coverage**: For code coverage analysis and reporting + +## Test Structure + +``` +test/ +├── backend/ # Backend tests +│ ├── app/ # API endpoint tests +│ ├── services/ # Service layer tests +│ └── utils/ # Utility function tests +├── frontend/ # Frontend tests (future) +├── integration/ # Integration tests (future) +└── run_all_tests.py # Main test runner +``` + +## Key Features + +- 🔍 **Auto-discover test files** - Automatically finds all `test_*.py` files +- 📊 **Coverage reports** - Generates console, HTML, and XML format coverage reports +- 🔧 **Auto-install dependencies** - Automatically installs required packages if needed +- ✅ **Detailed output** - Shows the running status and results of each test +- 🚫 **Complete isolation** - No real external services are ever contacted +- ⚡ **Fast execution** - No network delays or external service processing time + +## Running Tests + +### Quick Start + +```bash +# Run all tests with coverage +cd test +python run_all_tests.py +``` + +### Backend Tests + +```bash +# Run backend tests only +python test/backend/run_all_test.py +``` + +### Individual Test Files + +```bash +# Run specific test file +python -m pytest test/backend/services/test_agent_service.py -v +``` + +## Output Files + +When tests complete, you'll find: + +- `coverage_html/` - Detailed HTML format coverage report +- `coverage.xml` - XML format coverage report (for CI/CD) +- `.coverage` - Coverage data file +- Console output with detailed test results and coverage statistics + +## Testing Strategy + +### 1. Dependency Isolation + +External modules are mocked before imports to avoid real connections: + +- Database connections are mocked +- ElasticSearch and other external services are mocked +- No actual database operations are performed during tests +- HTTP clients are mocked to prevent network calls + +### 2. Mock-based Testing + +- HTTP requests are simulated using FastAPI's TestClient +- External service calls are intercepted with mock objects +- No actual network connections or port bindings occur +- Authentication functions are mocked to return predictable test values + +### 3. Test Organization + +- Tests are organized as classes inheriting from `unittest.TestCase` +- Each API endpoint or function has multiple test cases (success, failure, exception scenarios) +- Comprehensive patches are applied to isolate the code under test +- Tests follow a clear setup-execute-assert pattern + +### 4. API Testing + +- API endpoints are tested for correct response codes, payload structure, and error handling +- Both synchronous and asynchronous endpoints are covered +- Streaming responses are tested through specialized test cases +- Authentication and authorization are thoroughly tested + +## Module Patching Technique + +A critical technique used in the test suite is patching modules before they're imported. This prevents any real connections to external services. + +### Example: Patching Before Import + +```python +# Dynamically determine the backend path +current_dir = os.path.dirname(os.path.abspath(__file__)) +backend_dir = os.path.abspath(os.path.join(current_dir, "../../../backend")) +sys.path.append(backend_dir) + +# Setup patches for dependencies before importing modules +patches = [ + patch('botocore.client.BaseClient._make_api_call', return_value={}), + patch('backend.database.client.MinioClient', MagicMock()), + patch('backend.database.client.db_client', MagicMock()), + patch('backend.utils.auth_utils.get_current_user_id', + MagicMock(return_value=('test_user', 'test_tenant'))), + patch('httpx.AsyncClient', MagicMock()) +] + +# Start all patches +for p in patches: + p.start() + +# Now import the modules after applying all patches +from backend.apps.file_management_app import router +``` + +### Benefits of This Approach + +1. **Complete Isolation**: No real external services are ever contacted +2. **No Side Effects**: Tests can't modify production databases or services +3. **Faster Tests**: No network delays or external service processing time +4. **Predictable Results**: Tests use controlled mock data for consistent results +5. **No Port Binding**: The FastAPI application never binds to a real network port + +## Test Example + +Here's a detailed example of how tests are structured: + +```python +@patch("utils.auth_utils.get_current_user_id") +@patch("database.agent_db.query_all_tools") +async def test_list_tools_api_success(self, mock_query_all_tools, mock_get_current_user_id): + # Setup + mock_get_current_user_id.return_value = ("user123", "tenant456") + expected_tools = [{"id": 1, "name": "Tool1"}, {"id": 2, "name": "Tool2"}] + mock_query_all_tools.return_value = expected_tools + + # Execute + result = await list_tools_api(authorization="Bearer fake_token") + + # Assert + mock_get_current_user_id.assert_called_once_with("Bearer fake_token") + mock_query_all_tools.assert_called_once_with(tenant_id="tenant456") + self.assertEqual(result, expected_tools) +``` + +## Coverage Reporting + +The test suite generates comprehensive coverage reports: + +- **Console Output**: Line-by-line coverage details +- **HTML Report**: Detailed coverage report in `coverage_html/` +- **XML Report**: Coverage data for CI/CD integration +- **Summary Statistics**: Overall coverage percentage and missing lines + +## Sample Output + +``` +Nexent Community - Unit Test Runner +============================================================ +Discovered Test Files: +---------------------------------------- + • backend/services/test_agent_service.py + • backend/services/test_conversation_management_service.py + • backend/services/test_knowledge_summary_service.py + +Total: 3 test files + +============================================================ +Running Unit Tests with Coverage +============================================================ + +test_get_enable_tool_id_by_agent_id ... ok +test_save_message_with_string_content ... ok +test_load_knowledge_prompts ... ok +... + +============================================================ +Coverage Report +============================================================ +Name Stmts Miss Cover Missing +-------------------------------------------------------------------------------- +backend/services/agent_service.py 120 15 88% 45-50, 78-82 +backend/services/conversation_management_service.py 180 25 86% 123-128, 156-162 +backend/services/knowledge_summary_service.py 45 8 82% 35-42 +-------------------------------------------------------------------------------- +TOTAL 345 48 86% + +HTML coverage report generated in: test/coverage_html +XML coverage report generated: test/coverage.xml + +============================================================ +✅ All tests passed! +``` + +## Dependencies + +The test runner automatically installs required packages if they're not already available: + +- `pytest-cov` - For pytest coverage integration +- `coverage` - For code coverage analysis +- `pytest` - For advanced test discovery and execution + +## Best Practices + +1. **Always mock external dependencies** before importing modules +2. **Use descriptive test names** that explain what is being tested +3. **Follow the setup-execute-assert pattern** for clear test structure +4. **Test both success and failure scenarios** for comprehensive coverage +5. **Keep tests independent** - each test should be able to run in isolation +6. **Use meaningful mock data** that represents real-world scenarios +7. **Document complex test scenarios** with clear comments + +This testing framework ensures that all code changes are thoroughly validated before deployment, maintaining high code quality and reliability across the entire Nexent platform. \ No newline at end of file diff --git a/doc/docs/index.md b/doc/docs/index.md new file mode 100644 index 000000000..7e91efbe0 --- /dev/null +++ b/doc/docs/index.md @@ -0,0 +1,24 @@ +--- +# https://vitepress.dev/reference/default-theme-home-page +layout: home + +hero: + name: "Nexent" + text: "Zero-Code AI Agent Platform\n零代码智能体平台" + tagline: "Choose your language / 选择您的语言" + actions: + - theme: brand + text: English + link: /en/ + - theme: alt + text: 简体中文 + link: /zh/ + +features: + - title: 🌍 Multi-Language Support + details: Documentation available in English and Chinese + - title: 📚 Comprehensive Docs + details: Complete guides, FAQ, and developer documentation + - title: 🔧 Developer Resources + details: Contributing guidelines, security policies, and code of conduct +--- diff --git a/doc/docs/markdown-examples.md b/doc/docs/markdown-examples.md new file mode 100644 index 000000000..f9258a550 --- /dev/null +++ b/doc/docs/markdown-examples.md @@ -0,0 +1,85 @@ +# Markdown Extension Examples + +This page demonstrates some of the built-in markdown extensions provided by VitePress. + +## Syntax Highlighting + +VitePress provides Syntax Highlighting powered by [Shiki](https://github.com/shikijs/shiki), with additional features like line-highlighting: + +**Input** + +````md +```js{4} +export default { + data () { + return { + msg: 'Highlighted!' + } + } +} +``` +```` + +**Output** + +```js{4} +export default { + data () { + return { + msg: 'Highlighted!' + } + } +} +``` + +## Custom Containers + +**Input** + +```md +::: info +This is an info box. +::: + +::: tip +This is a tip. +::: + +::: warning +This is a warning. +::: + +::: danger +This is a dangerous warning. +::: + +::: details +This is a details block. +::: +``` + +**Output** + +::: info +This is an info box. +::: + +::: tip +This is a tip. +::: + +::: warning +This is a warning. +::: + +::: danger +This is a dangerous warning. +::: + +::: details +This is a details block. +::: + +## More + +Check out the documentation for the [full list of markdown extensions](https://vitepress.dev/guide/markdown). diff --git a/doc/docs/public/Nexent Logo.jpg b/doc/docs/public/Nexent Logo.jpg new file mode 100644 index 000000000..6de44bd4a Binary files /dev/null and b/doc/docs/public/Nexent Logo.jpg differ diff --git a/doc/docs/zh/agents/overview.md b/doc/docs/zh/agents/overview.md new file mode 100644 index 000000000..75bf3ba90 --- /dev/null +++ b/doc/docs/zh/agents/overview.md @@ -0,0 +1,213 @@ +# AI 智能体开发概览 + +Nexent 提供全面的 AI 智能体开发和部署框架，具备高级功能，包括工具集成、推理和多模态交互。 + +## 智能体架构 + +### 核心组件 + +#### NexentAgent - 企业级智能体框架 +Nexent 智能体系统的核心，提供完整的智能体解决方案： + +- **多模型支持**: 支持 OpenAI、视觉语言模型、长上下文模型等 +- **MCP 集成**: 无缝集成 Model Context Protocol 工具生态 +- **动态工具加载**: 支持本地工具和 MCP 工具的动态创建和管理 +- **分布式执行**: 基于线程池和异步架构的高性能执行引擎 +- **状态管理**: 完善的任务状态追踪和错误恢复机制 + +#### CoreAgent - 代码执行引擎 +继承并增强了 SmolAgent 的 `CodeAgent`，提供以下关键能力： + +- **Python代码执行**: 支持解析和执行Python代码，能够动态处理任务 +- **多语言支持**: 内置中英文提示词模板，可根据需要切换语言 +- **流式输出**: 通过 MessageObserver 实现模型输出的实时流式显示 +- **步骤追踪**: 记录并展示Agent执行的每个步骤，便于调试和监控 +- **中断控制**: 支持任务中断和优雅停止机制 +- **错误处理**: 完善的错误处理机制，提高稳定性 +- **状态管理**: 维护和传递执行状态，支持复杂任务的连续处理 + +CoreAgent 实现了ReAct框架的思考-行动-观察循环： +1. **思考**: 使用大语言模型生成解决方案代码 +2. **行动**: 执行生成的Python代码 +3. **观察**: 收集执行结果和日志 +4. **重复**: 根据观察结果继续思考和执行，直到任务完成 + +### MessageObserver - 流式消息处理 +消息观察者模式的核心实现，用于处理 Agent 的流式输出： + +- **流式输出捕获**: 实时捕获模型生成的token +- **过程类型区分**: 根据不同的处理阶段（模型输出、代码解析、执行日志等）格式化输出 +- **多语言支持**: 支持中英文输出格式 +- **统一接口**: 为不同来源的消息提供统一处理方式 + +ProcessType枚举定义了以下处理阶段： +- `STEP_COUNT`: 当前执行步骤 +- `MODEL_OUTPUT_THINKING`: 模型思考过程输出 +- `MODEL_OUTPUT_CODE`: 模型代码生成输出 +- `PARSE`: 代码解析结果 +- `EXECUTION_LOGS`: 代码执行结果 +- `AGENT_NEW_RUN`: Agent基本信息 +- `FINAL_ANSWER`: 最终总结结果 +- `SEARCH_CONTENT`: 搜索结果内容 +- `PICTURE_WEB`: 网络图片处理结果 + +## 智能体开发 + +### 创建基本智能体 + +```python +from nexent.core import MessageObserver, ProcessType +from nexent.core.agents import CoreAgent, NexentAgent +from nexent.core.models import OpenAIModel +from nexent.core.tools import ExaSearchTool, KnowledgeBaseSearchTool + +# 创建消息观察者 +observer = MessageObserver() + +# 创建模型（model和Agent必须使用同一个observer） +model = OpenAIModel( + observer=observer, + model_id="your-model-id", + api_key="your-api-key", + api_base="your-api-base" +) + +# 创建工具 +search_tool = ExaSearchTool(exa_api_key="your-exa-key", observer=observer, max_results=5) +kb_tool = KnowledgeBaseSearchTool(top_k=5, observer=observer) + +# 创建Agent +agent = CoreAgent( + observer=observer, + tools=[search_tool, kb_tool], + model=model, + name="my_agent", + max_steps=5 +) + +# 运行Agent +result = agent.run("你的问题") +``` + +### 自定义智能体开发 + +#### 系统提示词模板 +系统提示词模板位于 `backend/prompts/`： + +- **knowledge_summary_agent.yaml**: 知识库摘要代理 +- **manager_system_prompt_template.yaml**: 管理器系统提示词模板 +- **utils/**: 提示词工具 + +#### 智能体实现步骤 + +1. **创建智能体实例**: + ```python + from nexent.core.agents import CoreAgent + from nexent.core.models import OpenAIModel + + model = OpenAIModel( + model_id="your-model-id", + api_key="your-api-key", + api_base="your-api-base" + ) + agent = CoreAgent( + model=model, + tools=[your_tools], + system_prompt="你的系统提示词" + ) + ``` + +2. **配置智能体行为**: + - 通过 `tools` 参数添加自定义工具 + - 通过 `system_prompt` 设置行为 + - 配置 `max_steps`、`temperature` 等参数 + +3. **高级配置**: + ```python + agent = CoreAgent( + model=model, + tools=custom_tools, + system_prompt=custom_prompt, + max_steps=10, + temperature=0.7, + verbose=True, + additional_authorized_imports=["requests", "pandas"] + ) + ``` + +## 工具集成 + +### 自定义工具开发 + +Nexent 基于 [Model Context Protocol (MCP)](https://github.com/modelcontextprotocol/python-sdk) 实现工具系统。 + +#### 开发新工具: +1. 在 `backend/mcp_service/local_mcp_service.py` 实现逻辑 +2. 用 `@mcp.tool()` 装饰器注册 +3. 重启 MCP 服务 + +#### 示例: +```python +@mcp.tool(name="my_tool", description="我的自定义工具") +def my_tool(param1: str, param2: int) -> str: + # 实现工具逻辑 + return f"处理结果: {param1} {param2}" +``` + +### 工具开发规范 + +详细的工具开发规范和最佳实践，请参阅： +- [工具开发指南](../sdk/core/tools.md) + +## 智能体执行模式 + +### ReAct 模式 +问题解决智能体的标准执行模式： +1. **推理**: 分析问题并制定方法 +2. **行动**: 执行工具或生成代码 +3. **观察**: 检查结果和输出 +4. **迭代**: 继续直到任务完成 + +### 多智能体协作 +- **分层智能体**: 管理智能体协调工作智能体 +- **专业智能体**: 特定任务的领域专用智能体 +- **通信协议**: 智能体间的标准化消息传递 + +### 错误处理和恢复 +- **优雅降级**: 工具失败时的备选策略 +- **状态持久化**: 保存智能体状态以便恢复 +- **重试机制**: 带退避策略的自动重试 + +## 性能优化 + +### 执行效率 +- **并行工具执行**: 独立工具的并发运行 +- **缓存策略**: 缓存模型响应和工具结果 +- **资源管理**: 高效的内存和计算使用 + +### 监控和调试 +- **执行跟踪**: 智能体决策的详细日志 +- **性能指标**: 时间和资源使用追踪 +- **调试模式**: 开发时的详细输出 + +## 最佳实践 + +### 智能体设计 +1. **明确目标**: 定义具体、可测量的智能体目标 +2. **适当工具**: 选择与智能体能力匹配的工具 +3. **强大提示词**: 创建全面的系统提示词 +4. **错误处理**: 实现全面的错误恢复 + +### 开发工作流 +1. **迭代开发**: 增量构建和测试 +2. **提示词工程**: 基于测试结果优化提示词 +3. **工具测试**: 集成前验证单个工具 +4. **性能测试**: 监控和优化执行速度 + +### 生产部署 +1. **资源分配**: 确保充足的计算资源 +2. **监控设置**: 实现全面的日志和告警 +3. **扩展策略**: 规划增加的负载和使用 +4. **安全考虑**: 验证输入并保护API访问 + +详细的实现示例和高级模式，请参阅 [开发指南](../getting-started/development-guide)。 \ No newline at end of file diff --git a/backend/apps/README_CN.md b/doc/docs/zh/backend/api-reference.md similarity index 99% rename from backend/apps/README_CN.md rename to doc/docs/zh/backend/api-reference.md index 944e24a68..1c0d0b0e3 100644 --- a/backend/apps/README_CN.md +++ b/doc/docs/zh/backend/api-reference.md @@ -1,8 +1,5 @@ # Nexent Community API 文档 -[![English](https://img.shields.io/badge/lang-English-blue.svg)](README.md) -[![简体中文](https://img.shields.io/badge/lang-简体中文-red.svg)](README_CN.md) - 本文档提供了 Nexent Community 后端所有 API 端点的全面概述。 ## 目录 diff --git a/doc/docs/zh/backend/overview.md b/doc/docs/zh/backend/overview.md new file mode 100644 index 000000000..9d1b89427 --- /dev/null +++ b/doc/docs/zh/backend/overview.md @@ -0,0 +1,203 @@ +# 后端架构概览 + +Nexent 的后端采用 FastAPI 和 Python 构建，为 AI 智能体服务提供强大且可扩展的 API 平台。 + +## 技术栈 + +- **框架**: FastAPI +- **语言**: Python 3.10+ +- **数据库**: PostgreSQL + Redis + Elasticsearch +- **文件存储**: MinIO +- **任务队列**: Celery + Ray +- **AI框架**: smolagents +- **向量数据库**: Elasticsearch + +## 目录结构 + +``` +backend/ +├── apps/ # API应用层 +│ ├── base_app.py # FastAPI主应用 +│ ├── agent_app.py # 代理相关API +│ ├── conversation_management_app.py # 对话管理API +│ ├── file_management_app.py # 文件管理API +│ ├── knowledge_app.py # 知识库API +│ ├── model_managment_app.py # 模型管理API +│ ├── config_sync_app.py # 配置同步API +│ └── voice_app.py # 语音相关API +├── services/ # 业务服务层 +│ ├── agent_service.py # 代理业务逻辑 +│ ├── conversation_management_service.py # 对话管理 +│ ├── elasticsearch_service.py # 搜索引擎服务 +│ ├── model_health_service.py # 模型健康检查 +│ ├── prompt_service.py # 提示词服务 +│ └── tenant_config_service.py # 租户配置服务 +├── database/ # 数据访问层 +│ ├── client.py # 数据库连接 +│ ├── db_models.py # 数据库模型 +│ ├── agent_db.py # 代理数据操作 +│ ├── conversation_db.py # 对话数据操作 +│ ├── knowledge_db.py # 知识库数据操作 +│ └── tenant_config_db.py # 租户配置数据操作 +├── agents/ # 代理核心逻辑 +│ ├── agent_run_manager.py # 代理运行管理器 +│ ├── create_agent_info.py # 代理信息创建 +│ └── default_agents/ # 默认代理配置 +├── data_process/ # 数据处理模块 +│ ├── app.py # 数据处理应用 +│ ├── config.py # 数据处理配置 +│ ├── tasks.py # 数据处理任务 +│ ├── worker.py # 数据处理工作器 +│ └── utils.py # 数据处理工具 +├── utils/ # 工具类 +│ ├── auth_utils.py # 认证工具 +│ ├── config_utils.py # 配置工具 +│ ├── file_management_utils.py # 文件管理工具 +│ ├── logging_utils.py # 日志工具 +│ └── thread_utils.py # 线程工具 +├── consts/ # 常量定义 +│ ├── const.py # 系统常量 +│ └── model.py # 数据模型 +├── prompts/ # 提示词模板 +│ ├── knowledge_summary_agent.yaml # 知识库摘要代理 +│ ├── manager_system_prompt_template.yaml # 管理器系统提示词 +│ └── utils/ # 提示词工具 +├── sql/ # SQL脚本 +├── assets/ # 后端资源文件 +├── main_service.py # 主服务入口 +├── data_process_service.py # 数据处理服务入口 +└── requirements.txt # Python依赖 +``` + +## 架构职责 + +### **应用层 (apps)** +- API路由定义 +- 请求参数验证 +- 响应格式化 +- 身份验证和授权 + +### **服务层 (services)** +- 核心业务逻辑实现 +- 数据处理和转换 +- 外部服务集成 +- 业务规则执行 + +### **数据层 (database)** +- 数据库操作和ORM模型 +- 数据访问接口 +- 事务管理 +- 数据一致性和完整性 + +### **代理层 (agents)** +- AI代理核心逻辑和执行 +- 工具调用和集成 +- 推理和决策制定 +- 代理生命周期管理 + +### **工具层 (utils)** +- 通用工具函数 +- 配置管理 +- 日志和监控 +- 线程和进程管理 + +## 核心服务 + +### 代理管理 +- 代理创建和配置 +- 执行生命周期管理 +- 工具集成和调用 +- 性能监控 + +### 对话管理 +- 消息处理和存储 +- 上下文管理 +- 历史记录跟踪 +- 多租户支持 + +### 知识库 +- 文档处理和索引 +- 向量搜索和检索 +- 内容摘要 +- 知识图谱构建 + +### 文件管理 +- 多格式文件处理 +- MinIO存储集成 +- 批处理能力 +- 元数据提取 + +### 模型集成 +- 多模型提供商支持 +- 健康监控和故障转移 +- 负载均衡和缓存 +- 性能优化 + +## 数据流架构 + +### 1. 用户请求流程 +``` +用户输入 → 前端验证 → API调用 → 后端路由 → 业务服务 → 数据访问 → 数据库 +``` + +### 2. AI Agent执行流程 +``` +用户消息 → Agent创建 → 工具调用 → 模型推理 → 流式响应 → 结果保存 +``` + +### 3. 知识库文件处理流程 +``` +文件上传 → 临时存储 → 数据处理 → 向量化 → 知识库存储 → 索引更新 +``` + +### 4. 实时文件处理流程 +``` +文件上传 → 临时存储 → 数据处理 → Agent → 回答 +``` + +## 部署架构 + +### 容器服务 +- **nexent**: 后端服务 (端口 5010) +- **nexent-data-process**: 数据处理服务 (端口 5012) +- **nexent-postgresql**: 数据库 (端口 5434) +- **nexent-elasticsearch**: 搜索引擎 (端口 9210) +- **nexent-minio**: 对象存储 (端口 9010) +- **redis**: 缓存服务 (端口 6379) + +### 可选服务 +- **nexent-openssh-server**: 终端工具的SSH服务器 (端口 2222) + +## 开发设置 + +### 环境搭建 +```bash +cd backend +uv sync && uv pip install -e ../sdk +``` + +### 服务启动 +```bash +python backend/data_process_service.py # 数据处理服务 +python backend/main_service.py # 主服务 +python backend/nexent_mcp_service.py # MCP服务 +``` + +## 性能和可扩展性 + +### 异步架构 +- 基于asyncio的高性能异步处理 +- 线程安全的并发处理机制 +- 针对分布式任务队列优化 + +### 缓存策略 +- 多层缓存提升响应速度 +- Redis用于会话和临时数据 +- Elasticsearch用于搜索结果缓存 + +### 负载均衡 +- 智能并发限制 +- 资源池管理 +- 自动扩展能力 + +详细的后端开发指南，请参阅 [开发指南](../getting-started/development-guide)。 \ No newline at end of file diff --git a/doc/docs/zh/backend/prompt-development.md b/doc/docs/zh/backend/prompt-development.md new file mode 100644 index 000000000..298589ad0 --- /dev/null +++ b/doc/docs/zh/backend/prompt-development.md @@ -0,0 +1,154 @@ +# 提示词开发指南 + +本指南提供了关于 Nexent 中用于创建不同类型智能体的提示词模板系统的全面信息。`backend/prompts/` 目录中的 YAML 文件定义了各种智能体类型的系统提示词、规划提示词和其他关键提示词组件。 + +## 文件命名规范 + +命名格式为 `{agent_type}_agent.yaml`，其中： +- `agent_type`：描述智能体的主要功能或用途（如 manager、search 等） + +## 提示词模板结构 + +每个 YAML 文件包含以下主要部分： + +### 1. system_prompt + +系统提示词是智能体的核心部分，定义了智能体的角色、能力和行为规范。通常包含以下部分： + +- **核心职责**：智能体的主要职责和能力描述 +- **执行流程**：智能体执行任务的标准流程和方法 +- **可用资源**：智能体可以使用的工具和子智能体列表 +- **资源使用要求**：使用不同工具的优先级和策略 +- **Python代码规范**：编写代码的规范和约束 +- **示例模板**：展示智能体执行任务的示例 + +### 2. planning + +包含用于任务规划的各种提示词： + +- **initial_facts**：初始事实收集提示词 +- **initial_plan**：初始计划制定提示词 +- **update_facts_pre_messages**：更新事实前的提示词 +- **update_facts_post_messages**：更新事实后的提示词 +- **update_plan_pre_messages**：更新计划前的提示词 +- **update_plan_post_messages**：更新计划后的提示词 + +### 3. managed_agent + +定义与子智能体交互的提示词： + +- **task**：分配给子智能体的任务提示词 +- **report**：子智能体报告结果的提示词 + +### 4. final_answer + +定义最终答案生成的提示词： + +- **pre_messages**：生成最终答案前的提示词 +- **post_messages**：生成最终答案后的提示词 + +### 5. tools_requirement + +定义工具使用规范和优先级的提示词。 + +### 6. few_shots + +提供少样本学习示例的提示词，帮助智能体更好地理解任务执行方式。 + +## 模板变量 + +提示词模板中使用以下特殊变量进行动态替换： + +- `{{tools}}`：可用工具列表 +- `{{managed_agents}}`：可用子智能体列表 +- `{{task}}`：当前任务描述 +- `{{authorized_imports}}`：授权导入的Python模块 +- `{{facts_update}}`：更新后的事实列表 +- `{{answer_facts}}`：已知事实列表 +- `{{remaining_steps}}`：剩余执行步骤数 + +## 可用的提示词模板 + +### 核心模板 + +1. **管理器智能体模板** + - `manager_system_prompt_template.yaml` - 中文版本 + - `manager_system_prompt_template_en.yaml` - 英文版本 + + 这些模板定义了核心管理器智能体，负责协调和调度各种助手和工具来高效解决复杂任务。 + +2. **被管理智能体模板** + - `managed_system_prompt_template.yaml` - 中文版本 + - `managed_system_prompt_template_en.yaml` - 英文版本 + + 这些模板定义了专门的智能体，在管理器智能体的协调下执行特定任务。 + +3. **专业智能体模板** + - `knowledge_summary_agent.yaml` - 知识总结智能体（中文） + - `knowledge_summary_agent_en.yaml` - 知识总结智能体（英文） + - `analyze_file.yaml` - 文件分析智能体（中文） + - `analyze_file_en.yaml` - 文件分析智能体（英文） + +### 工具模板 + +位于 `utils/` 目录中： + +1. **提示词生成模板** + - `prompt_generate.yaml` - 中文版本 + - `prompt_generate_en.yaml` - 英文版本 + + 这些模板帮助为不同智能体类型生成高效、清晰的提示词。 + +2. **提示词微调模板** + - `prompt_fine_tune.yaml` - 中文版本 + - `prompt_fine_tune_en.yaml` - 英文版本 + + 用于微调和优化现有提示词的模板。 + +3. **标题生成模板** + - `generate_title.yaml` - 用于生成标题和摘要 + +## 执行流程 + +标准智能体执行流程遵循以下模式： + +1. **思考**：分析当前任务状态和进展 +2. **代码**：编写简单的Python代码 +3. **观察**：查看代码执行结果 +4. **重复**：继续循环直到任务完成 + +## 代码规范 + +在提示词中编写Python代码时： + +1. 使用格式 `代码：\n```py\n` 表示可执行代码 +2. 使用格式 `代码：\n```code:语言类型\n` 表示仅用于展示的代码 +3. 只使用已定义的变量，变量将在多次调用之间持续保持 +4. 使用 `print()` 函数让变量信息可见 +5. 使用关键字参数进行工具和智能体调用 +6. 避免在一轮对话中进行过多的工具调用 +7. 只能从授权模块导入：`{{authorized_imports}}` + +## 最佳实践 + +1. **任务分解**：将复杂任务分解为可管理的子任务 +2. **专业匹配**：根据智能体专长分配任务 +3. **信息整合**：整合不同智能体的输出 +4. **效率优化**：避免重复工作 +5. **结果评估**：评估智能体返回结果，必要时提供额外指导 + +## 使用示例 + +以下是管理器智能体如何与专业智能体协调的示例： + +```yaml +# 任务分配示例 +managed_agent: + task: | + 请分析提供的文档并提取关键见解。 + + report: | + {{final_answer}} +``` + +这个系统允许灵活而强大的智能体协调，同时保持清晰的提示词开发标准和最佳实践。 \ No newline at end of file diff --git a/doc/docs/zh/backend/tools/index.md b/doc/docs/zh/backend/tools/index.md new file mode 100644 index 000000000..208425aed --- /dev/null +++ b/doc/docs/zh/backend/tools/index.md @@ -0,0 +1,31 @@ +# 后端工具文档 + +本节介绍 Nexent 后端基础设施中可用的工具生态系统。 + +## 可用工具类别 + +### LangChain 工具 +与 LangChain 生态系统集成，实现高级 AI 工作流。 +→ [LangChain 工具指南](./langchain) + +### MCP 工具 +模型上下文协议工具，用于标准化 AI 智能体通信。 +→ [MCP 工具开发](./mcp) + +## 快速开始 + +1. **选择工具类型**: LangChain 用于通用 AI 工作流，MCP 用于标准化智能体通信 +2. **遵循集成指南**: 每种工具类型都有特定的设置要求 +3. **测试集成**: 使用提供的示例验证设置 +4. **构建智能体**: 组合工具来创建强大的 AI 智能体 + +## SDK 集成 + +有关 SDK 级别的工具开发，请参阅： +→ [SDK 工具文档](../../sdk/core/tools) + +## 需要帮助？ + +- 查看我们的 [常见问题](../../getting-started/faq) 了解常见工具集成问题 +- 加入我们的 [Discord 社区](https://discord.gg/tb5H3S3wyv) 获取实时支持 +- 查看 [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) 了解已知问题 \ No newline at end of file diff --git a/backend/tool_collection/langchain/README_CN.md b/doc/docs/zh/backend/tools/langchain.md similarity index 100% rename from backend/tool_collection/langchain/README_CN.md rename to doc/docs/zh/backend/tools/langchain.md diff --git a/backend/tool_collection/mcp/README_CN.md b/doc/docs/zh/backend/tools/mcp.md similarity index 100% rename from backend/tool_collection/mcp/README_CN.md rename to doc/docs/zh/backend/tools/mcp.md diff --git a/doc/docs/zh/code-of-conduct.md b/doc/docs/zh/code-of-conduct.md new file mode 100644 index 000000000..8c97723b1 --- /dev/null +++ b/doc/docs/zh/code-of-conduct.md @@ -0,0 +1,84 @@ +# 贡献者行为准则 + +## 我们的承诺 + +作为成员、贡献者和领导者，我们承诺让每个人都能参与我们的社区，无论年龄、体型、明显或不明显的残疾、种族、性别特征、性别认同和表达、经验水平、教育程度、社会经济地位、国籍、外貌、种族、血统、肤色、宗教或性认同和性取向如何，都不会遭受骚扰。 + +我们承诺以有助于开放、友好、多元、包容和健康社区的方式行事和互动。 + +## 我们的标准 + +有助于为我们社区创造积极环境的行为示例包括： + +* 对他人表现出同理心和善意 +* 尊重不同的意见、观点和经历 +* 给出并优雅地接受建设性反馈 +* 承担责任并向受我们错误影响的人道歉，并从中学习 +* 专注于对个人和整个社区最有利的事情 + +不可接受的行为示例包括： + +* 使用性化的语言或图像，以及任何形式的性关注或性骚扰 +* 恶意评论、侮辱或贬损性评论，以及人身或政治攻击 +* 公开或私下骚扰 +* 未经明确许可发布他人的私人信息，如实际地址或电子邮件地址 +* 在专业环境中可能被合理地认为不适当的其他行为 + +## 执行责任 + +社区领导者有责任澄清和执行我们的可接受行为标准，并将对他们认为不适当、威胁、攻击性或有害的任何行为采取适当和公平的纠正措施。 + +社区领导者有权利和责任删除、编辑或拒绝与本行为准则不符的评论、提交、代码、wiki编辑、问题和其他贡献，并将在适当时说明审核决定的原因。 + +## 适用范围 + +本行为准则适用于所有社区空间，也适用于个人在公共空间正式代表社区时。代表我们社区的示例包括使用官方电子邮件地址、通过官方社交媒体账户发布或在在线或离线活动中担任指定代表。 + +## 执行 + +可以向负责执行的社区领导者举报滥用、骚扰或其他不可接受的行为实例，联系邮箱：[chenshuangrui@huawei.com]。 +所有投诉都将得到及时和公平的审查和调查。 + +所有社区领导者都有义务尊重任何事件举报者的隐私和安全。 + +## 执行指南 + +社区领导者将遵循这些社区影响指南来确定他们认为违反本行为准则的任何行为的后果： + +### 1. 纠正 + +**社区影响**：使用不当语言或其他在社区中被认为不专业或不受欢迎的行为。 + +**后果**：社区领导者发出私人书面警告，澄清违规性质并解释行为不当的原因。可能要求公开道歉。 + +### 2. 警告 + +**社区影响**：通过单一事件或一系列行为的违规。 + +**后果**：对持续行为后果的警告。在指定时间内不得与相关人员互动，包括主动与执行行为准则的人员互动。这包括避免在社区空间以及外部渠道（如社交媒体）中的互动。违反这些条款可能导致临时或永久禁止。 + +### 3. 临时禁止 + +**社区影响**：严重违反社区标准，包括持续的不当行为。 + +**后果**：在指定时间内临时禁止与社区进行任何形式的互动或公共交流。在此期间不允许与相关人员进行公开或私人互动，包括主动与执行行为准则的人员互动。违反这些条款可能导致永久禁止。 + +### 4. 永久禁止 + +**社区影响**：表现出违反社区标准的模式，包括持续的不当行为、对个人的骚扰或对个人群体的侵犯或贬低。 + +**后果**：永久禁止在社区内进行任何形式的公共互动。 + +## 归属 + +本行为准则改编自[贡献者公约][homepage]2.1版，可在[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1]获得。 + +社区影响指南的灵感来自[Mozilla的行为准则执行阶梯][Mozilla CoC]。 + +有关此行为准则的常见问题的答案，请参见[https://www.contributor-covenant.org/faq][FAQ]的FAQ。翻译版本可在[https://www.contributor-covenant.org/translations][translations]获得。 + +[homepage]: https://www.contributor-covenant.org +[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html +[Mozilla CoC]: https://github.com/mozilla/diversity +[FAQ]: https://www.contributor-covenant.org/faq +[translations]: https://www.contributor-covenant.org/translations \ No newline at end of file diff --git a/doc/docs/zh/contributing.md b/doc/docs/zh/contributing.md new file mode 100644 index 000000000..23ebcd9f9 --- /dev/null +++ b/doc/docs/zh/contributing.md @@ -0,0 +1,172 @@ +# Nexent 贡献指南 + +感谢您考虑为 Nexent 贡献力量！无论是代码、文档还是经验分享，您的每一份付出都能让 Nexent 变得更好。如果您愿意向他人推荐 Nexent 或在仓库点个 ⭐️，我们也非常感激。万分感谢！💛 让我们一起打造非凡之作！🎉 + +关于许可证，请花一分钟阅读我们简短的[许可和贡献者协议](https://github.com/ModelEngine-Group/nexent/blob/main/LICENSE)。同时也请遵循[社区行为准则](https://github.com/ModelEngine-Group/nexent/blob/main/CODE_OF_CONDUCT.md)。 + +## 🤔 如何贡献 + +### 🐛 发现了一个 Bug？ + +如果您发现了 Bug，请告诉我们！您的敏锐观察将帮助所有用户获得更好的 Nexent。 + +### 💡 有功能创意？ + +有提升 Nexent 的绝妙想法？我们非常乐意倾听！与我们分享您的愿景。 + +### 💻 想提交代码？ + +无论是修复 Bug 还是添加新功能，您的代码贡献都非常宝贵。 + +### 📖 想改进文档？ + +优秀的文档是优秀项目的关键。帮助我们让 Nexent 更易用、更易懂。 + +--- + +## 🌳 分支策略 GitFlow + +![GitFlow 工作流](../assets/git-flow.svg) + + +Gitflow 是一种结构化的 Git 分支管理模型，为软件开发提供了清晰的流程。它为不同目的（如功能、发布、热修复）定义了专用分支，并规定了它们的交互方式，有助于规范开发流程、高效管理发布并促进协作。 + +### 主要分支 +- **main**：代表正式发布历史，始终保持可部署状态。 +- **develop**：日常开发的主分支，集成 feature 分支的新功能和 bugfix。 + +### 辅助分支 +- **feature 分支**：用于开发新功能，从 develop 分支创建，开发完成后合并回 develop。 +- **release 分支**：用于准备新版本发布，允许最终测试和小调整，发布后合并到 main 和 develop。 +- **hotfix 分支**：用于生产环境紧急修复，从 main 分支创建，修复后合并回 main 和 develop。 + +### Gitflow 优势 +- **结构化流程**：为不同类型的更改提供清晰一致的管理方式。 +- **提升协作**：通过明确分支角色和交互方式，促进团队协作。 +- **高效发布**：通过专用分支隔离变更，便于最终测试和快速发布。 +- **减少冲突**：feature 和 release 分支的使用，有助于减少合并冲突并简化解决过程。 + +如上图所示，GitFlow 工作流一目了然。 + +## 🐞 提交 Bug 报告或功能请求 + +### Bug 报告 +为了帮助我们快速理解和修复问题，请包含以下内容： +- 描述 Bug 的**清晰标题**。 +- 问题的**详细描述**，包括重现步骤。 +- 任何**错误信息**或日志（如果有）。 +- 预期行为与实际行为的对比。 +- 截图或屏幕录像（如果有帮助）。 + +### 功能请求 +对于功能创意，请提供： +- 总结功能的**清晰标题**。 +- 功能的**详细描述**及其优势。 +- 任何相关的**用例**或示例。 +- 截图或设计稿（如果有）。 + +**提交到哪里？** +在我们的 [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) 中新建一个 Issue，并选择合适的模板（Bug 报告或功能请求）。 + +## 💻 提交代码更改 + +### 第一步：Fork 仓库 +🍴 Fork [Nexent 仓库](https://github.com/ModelEngine-Group/nexent) 到您的 GitHub 账户。 + +### 第二步：克隆您的 Fork +📦 将您的 Fork 克隆到本地： +```bash +git clone https://github.com/ModelEngine-Group/nexent.git +``` + +### 第三步：创建分支 +🌿 为您的更改创建一个新分支： +```bash +git checkout -b 您的分支名 +``` + +### 第四步：进行更改 +🧙‍♂️ 像魔法师一样编码！遵循我们的 [开发指南](./getting-started/development-guide) 获取设置说明和编码标准。确保您的更改经过充分测试并有文档记录。 + +### 第五步：提交更改 +📝 按照我们的提交消息规范，提交清晰简洁的消息（建议采用英文，让更多人理解你）： + +| 类型 | 图标 | 描述 | +|---------|------|-----------------| +| 重构 | ♻️ | 代码逻辑优化，不影响功能 | +| 代码迁移 | 🚚 | 移动、迁移文件或模块 | +| 新需求/新特性 | ✨ | 增加新功能、新特性 | +| Bug修复 | 🐛 | 修复问题或错误 | +| 风格优化 | 🎨 | 改进代码风格、格式化，不改功能 | +| 工程优化 | 🔨 | 工程工具更新、配置调整 | +| 文档更新 | 📝 | 只改动文档内容 | +| 添加测试用例 | 🧪 | 添加测试用例或修改测试用例 | + +示例提交消息： +```bash +git commit -m "✨ add user authentication" +git commit -m "🐛 resolve login timeout issue" +git commit -m "📝 update API documentation" +``` + +### 第六步：与上游同步 +⚙️ 让您的 Fork 与主仓库的最新更改保持同步： +```bash +git remote add upstream https://github.com/ModelEngine-Group/nexent.git +git fetch upstream +git merge upstream/main +``` + +### 第七步：发起拉取请求（PR） +🚀 将您的更改推送到您的 Fork，并在主仓库中发起 PR。包括： +- 更改的**清晰标题**和**描述**。 +- 相关 Issue 的引用（例如 `fixes #123`）。 +- 任何额外的上下文或截图。 + +我们的团队将审核您的 PR 并提供反馈。协作创造奇迹！✨ + +### 保护分支和代码所有者审查 + +当向保护分支（如 `main`）提交更改时，请注意以下要求： + +1. **需要代码所有者审查** + - PR 将自动请求相关代码所有者的审查 + - 您不能批准自己的 PR + - 代码所有者的批准是必需的 + +2. **需要多个批准** + - 至少需要 2 个批准（包括代码所有者的批准） + - 所有 CI 检查必须通过（lint、测试、构建等） + +3. **合并流程** + - 提交 PR 后，系统将自动请求代码所有者审查 + - 需要至少两个批准（包括代码所有者） + - 只有在满足所有要求后，"合并"按钮才会变为可用 + +4. **限制** + - 不能绕过审查或强制合并 + - 不允许直接推送到保护分支 + - 自我批准无效 + +## 📖 改进文档 + +优秀的文档是团队共同努力的结果！您可以通过以下方式帮助： +- 修复拼写错误或澄清不清楚的部分。 +- 为功能或设置步骤添加缺失的文档。 +- 将文档翻译成其他语言。 + +贡献步骤： +1. 遵循与代码更改相同的步骤（Fork、克隆、分支等）。 +2. 编辑相关文档文件（例如 `README.md`、`docs/`）。 +3. 提交包含您改进的 PR。 + +## ❓ 需要帮助？ + +遇到困难或有疑问？我们随时为您提供帮助！通过以下方式联系我们： +- **GitHub Issues**：新建一个 Issue 进行讨论。 +- **Discord**：加入我们的 [Nexent 社区](https://discord.gg/YXH5C8SQ) 进行实时聊天。 +- **电子邮件**：给我们发邮件至 [chenshuangrui@huawei.com](mailto:chenshuangrui@huawei.com)。 + +## 🎉 庆祝您的贡献！ + +感谢您参与 Nexent 的旅程。您的贡献意义重大，我们迫不及待想看看您创造的内容！编码愉快！🚀🌈 diff --git a/doc/docs/zh/contributors.md b/doc/docs/zh/contributors.md new file mode 100644 index 000000000..d525c2250 --- /dev/null +++ b/doc/docs/zh/contributors.md @@ -0,0 +1,45 @@ +# 核心贡献者 + +Nexent 项目得益于我们核心团队成员的辛勤工作。我们想感谢他们的贡献和专业知识。 + +## Nexent 团队 + +### 团队经理 / 产品经理 +- **Shuangrui Chen** @Phinease + +### 高级系统工程师 +- **Simeng Bian** @Simeng Bian +- **Tao Liu** @liutao12138 + +### 开发组组长 +- **Jingyuan Li** @ljy65535 + +### 开发者 +- **Yichen Xia** @Jasonxia007 +- **Mingchen Wan** @WMC001 +- **Yu Lin** @linsensen222 +- **Wenqi Bai** @Bavichi +- **Feiyang Xiang** @feixiangkong + +### 运营经理 +- **Chenxue Jia** @Davina-jcx + +## 致谢 + +我们向所有团队成员表示诚挚的感谢，感谢他们对卓越的承诺以及为将 Nexent 打造成强大可靠的 AI 智能体平台所做的贡献。 + +每个团队成员都为项目带来了独特的专业知识和视角，为平台的不同方面做出贡献，包括： + +- 系统架构和工程 +- 产品管理和策略 +- 开发和实施 +- 运营和维护 +- 质量保证和测试 + +## 加入我们的团队 + +如果您有兴趣为 Nexent 做出贡献，请查看我们的[贡献指南](/zh/contributing)了解更多参与方式。 + +--- + +*此列表代表我们的核心团队成员。如需查看所有贡献者的完整列表，请访问我们的 GitHub 仓库。* \ No newline at end of file diff --git a/doc/docs/zh/deployment/devcontainer.md b/doc/docs/zh/deployment/devcontainer.md new file mode 100644 index 000000000..2ce184901 --- /dev/null +++ b/doc/docs/zh/deployment/devcontainer.md @@ -0,0 +1,71 @@ +# Nexent Dev Container 使用指南 + +## 1. 环境说明 + +此开发容器配置了一个完整的 Nexent 开发环境，包含以下组件： + +- 主要开发容器 (`nexent-dev`)：基于 nexent/nexent 镜像，添加了开发工具 +- 服务容器： + - Elasticsearch (`nexent-elasticsearch`) + - PostgreSQL (`nexent-postgresql`) + - MinIO (`nexent-minio`) + - Nexent 后端 (`nexent`) + - Nexent 前端 (`nexent-web`) + - 数据处理服务 (`nexent-data-process`) + +## 2. 使用步骤 + +### 2.1 准备工作 + +1. 安装 Cursor +02. 安装 Dev Containers 插件 (`anysphere.remote-containers` 与 `anysphere.remote-sshRemote`) +3. 确保 Docker 和 Docker Compose 已安装并运行 + +### 2.2 使用 Dev Container 启动项目 + +1. 克隆项目到本地 +2. 在 Cursor 中打开项目文件夹 +3. 运行 `docker/deploy.sh` 脚本，在`infrastructure` 模式下启动容器 +4. 进入 `nexent-minio` 与 `nexent-elasticsearch` 容器, 将 `MINIO_ACCESS_KEY`, `MINIO_SECRET_KEY`, `ELASTICSEARCH_API_KEY` 环境变量复制到 `docker/docker-compose.dev.yml` 中的相应环境变量位置 +5. 按下 `F1` 或 `Ctrl+Shift+P`，输入 `Dev Containers: Reopen in Container ...` +6. Cursor 将根据 `.devcontainer` 目录中的配置启动开发容器 + +### 2.3 开发工作流 + +1. 容器启动后，Cursor 会自动连接到开发容器 +2. 所有文件编辑都在容器内完成 +3. 进行开发、测试，修改完成后可以直接在容器内构建和运行 +4. 可以直接在容器内进行 git 的变更管理，如使用 `git commit` 或 `git push`；但不建议在容器内拉取远程代码，容易导致路径问题 + +## 3. 端口映射 + +以下端口已在 devcontainer.json 中配置了映射： + +- 3000: Nexent Web 界面 +- 5010: Nexent 后端服务 +- 5012: 数据处理服务 +- 9010: MinIO API +- 9011: MinIO 控制台 +- 9210: Elasticsearch API +- 5434: PostgreSQL + +## 4. 自定义开发环境 + +您可以通过修改以下文件来自定义开发环境： + +- `.devcontainer/devcontainer.json` - 插件配置项 +- `docker/docker-compose.dev.yml` - 开发容器的具体构筑项，需要修改环境变量值才能正常启动 + +## 6. 常见问题解决 + +如果遇到权限问题，可能需要在容器内运行： + +```bash +sudo chown -R $(id -u):$(id -g) /opt +``` + +如果容器启动失败，可以尝试： + +1. 重建容器：按下 `F1` 或 `Ctrl+Shift+P`，输入 `Dev Containers: Rebuild Container` +2. 检查 Docker 日志：`docker logs nexent-dev` +3. 检查 `.env` 文件中的配置是否正确 \ No newline at end of file diff --git a/doc/docs/zh/deployment/docker-build.md b/doc/docs/zh/deployment/docker-build.md new file mode 100644 index 000000000..6df8eff99 --- /dev/null +++ b/doc/docs/zh/deployment/docker-build.md @@ -0,0 +1,118 @@ +# Docker 构建指南 + +这个文档介绍如何构建和推送 Nexent 的 Docker 镜像。 + +## 🏗️ 构建和推送镜像 + +```bash +# 🛠️ 创建并使用支持多架构构建的新构建器实例 +docker buildx create --name nexent_builder --use + +# 🚀 为多个架构构建应用程序 +docker buildx build --progress=plain --platform linux/amd64,linux/arm64 -t nexent/nexent -f make/main/Dockerfile . --push +docker buildx build --progress=plain --platform linux/amd64,linux/arm64 -t ccr.ccs.tencentyun.com/nexent-hub/nexent -f make/web/Dockerfile . --push + +# 📊 为多个架构构建数据处理服务 +docker buildx build --progress=plain --platform linux/amd64,linux/arm64 -t nexent/nexent-data-process -f make/data_process/Dockerfile . --push +docker buildx build --progress=plain --platform linux/amd64,linux/arm64 -t ccr.ccs.tencentyun.com/nexent-hub/nexent-data-process -f make/web/Dockerfile . --push + +# 🌐 为多个架构构建前端 +docker buildx build --progress=plain --platform linux/amd64,linux/arm64 -t nexent/nexent-web -f make/web/Dockerfile . --push +docker buildx build --progress=plain --platform linux/amd64,linux/arm64 -t ccr.ccs.tencentyun.com/nexent-hub/nexent-web -f make/web/Dockerfile . --push + +# 📚 为多个架构构建文档 +docker buildx build --progress=plain --platform linux/amd64,linux/arm64 -t nexent/nexent-docs -f make/docs/Dockerfile . --push +docker buildx build --progress=plain --platform linux/amd64,linux/arm64 -t ccr.ccs.tencentyun.com/nexent-hub/nexent-docs -f make/docs/Dockerfile . --push +``` + +## 💻 本地开发构建 + +```bash +# 🚀 构建应用程序镜像（仅当前架构） +docker build --progress=plain -t nexent/nexent -f make/main/Dockerfile . + +# 📊 构建数据处理镜像（仅当前架构） +docker build --progress=plain -t nexent/nexent-data-process -f make/data_process/Dockerfile . + +# 🌐 构建前端镜像（仅当前架构） +docker build --progress=plain -t nexent/nexent-web -f make/web/Dockerfile . + +# 📚 构建文档镜像（仅当前架构） +docker build --progress=plain -t nexent/nexent-docs -f make/docs/Dockerfile . +``` + +## 🔧 镜像说明 + +### 主应用镜像 (nexent/nexent) +- 包含后端 API 服务 +- 基于 `make/main/Dockerfile` 构建 +- 提供核心的智能体服务 + +### 数据处理镜像 (nexent/nexent-data-process) +- 包含数据处理服务 +- 基于 `make/data_process/Dockerfile` 构建 +- 处理文档解析和向量化 + +### 前端镜像 (nexent/nexent-web) +- 包含 Next.js 前端应用 +- 基于 `make/web/Dockerfile` 构建 +- 提供用户界面 + +### 文档镜像 (nexent/nexent-docs) +- 包含 Vitepress 文档站点 +- 基于 `make/docs/Dockerfile` 构建 +- 提供项目文档和 API 参考 + +## 🏷️ 标签策略 + +每个镜像都会推送到两个仓库： +- `nexent/*` - 主要的公共镜像仓库 +- `ccr.ccs.tencentyun.com/nexent-hub/*` - 腾讯云镜像仓库（中国地区加速） + +所有镜像包括： +- `nexent/nexent` - 主应用后端服务 +- `nexent/nexent-data-process` - 数据处理服务 +- `nexent/nexent-web` - Next.js 前端应用 +- `nexent/nexent-docs` - Vitepress 文档站点 + +## 📚 文档镜像独立部署 + +文档镜像可以独立构建和运行，用于为 nexent.tech/doc 提供服务： + +### 构建文档镜像 + +```bash +docker build -t nexent/nexent-docs -f make/docs/Dockerfile . +``` + +### 运行文档容器 + +```bash +docker run -d --name nexent-docs -p 4173:4173 nexent/nexent-docs +``` + +### 查看容器状态 + +```bash +docker ps +``` + +### 查看容器日志 + +```bash +docker logs nexent-docs +``` + +### 停止和删除容器 + +```bash +docker stop nexent-docs +``` + +```bash +docker rm nexent-docs +``` + +## 🚀 部署建议 + +构建完成后，可以使用 `docker/deploy.sh` 脚本进行部署，或者直接使用 `docker-compose` 启动服务。 \ No newline at end of file diff --git a/doc/docs/zh/frontend/overview.md b/doc/docs/zh/frontend/overview.md new file mode 100644 index 000000000..247845a49 --- /dev/null +++ b/doc/docs/zh/frontend/overview.md @@ -0,0 +1,131 @@ +# 前端架构概览 + +Nexent 的前端采用现代 React 技术构建，为 AI 智能体交互提供响应式和直观的用户界面。 + +## 技术栈 + +- **框架**: Next.js 14 (App Router) +- **语言**: TypeScript +- **UI库**: React + Tailwind CSS +- **状态管理**: React Hooks +- **国际化**: react-i18next +- **HTTP客户端**: Fetch API + +## 目录结构 + +``` +frontend/ +├── app/ # Next.js App Router +│ └── [locale]/ # 国际化路由 (zh/en) +│ ├── chat/ # 聊天界面 +│ │ ├── internal/ # 聊天核心逻辑 +│ │ ├── layout/ # 聊天界面布局组件 +│ │ └── streaming/ # 流式响应处理 +│ ├── setup/ # 系统设置页面 +│ │ ├── agentSetup/ # 代理配置 +│ │ ├── knowledgeBaseSetup/ # 知识库配置 +│ │ └── modelSetup/ # 模型配置 +│ └── layout.tsx # 全局布局 +├── components/ # 可复用UI组件 +│ ├── providers/ # 上下文提供者 +│ └── ui/ # 基础UI组件库 +├── services/ # API服务层 +│ ├── api.ts # API基础配置 +│ ├── conversationService.ts # 对话服务 +│ ├── agentConfigService.ts # 代理配置服务 +│ ├── knowledgeBaseService.ts # 知识库服务 +│ └── modelService.ts # 模型服务 +├── hooks/ # 自定义React Hooks +├── lib/ # 工具库 +├── types/ # TypeScript类型定义 +├── public/ # 静态资源 +│ └── locales/ # 国际化文件 +└── middleware.ts # Next.js中间件 +``` + +## 架构职责 + +### **展示层** +- 用户界面和交互逻辑 +- 基于组件的可复用架构 +- 多设备响应式设计 + +### **服务层** +- 封装API调用和数据转换 +- 处理与后端服务的通信 +- 管理错误处理和重试逻辑 + +### **状态管理** +- 使用React Hooks管理组件状态 +- Context提供者管理全局状态 +- 流式响应的实时更新 + +### **国际化** +- 支持中英文语言切换 +- 动态语言切换 +- 本地化内容和UI元素 + +### **路由管理** +- 基于Next.js App Router +- 语言感知路由 +- 动态路由生成 + +## 核心特性 + +### 实时聊天界面 +- 流式响应处理 +- 消息历史管理 +- 多模态输入支持（文本、语音、图像） + +### 配置管理 +- 模型提供商配置 +- 智能体行为自定义 +- 知识库管理 + +### 响应式设计 +- 移动优先方法 +- 自适应布局 +- 触摸友好交互 + +### 性能优化 +- 服务器端渲染 (SSR) +- 静态站点生成 (SSG) +- 代码分割和懒加载 +- 图像优化 + +## 开发工作流 + +### 设置 +```bash +cd frontend +npm install +npm run dev +``` + +### 生产构建 +```bash +npm run build +npm start +``` + +### 代码质量 +- ESLint 代码检查 +- Prettier 代码格式化 +- TypeScript 类型安全 +- Husky 预提交钩子 + +## 集成点 + +### 后端通信 +- RESTful API 调用 +- WebSocket 实时功能 +- 身份验证和授权 +- 错误处理和用户反馈 + +### 外部服务 +- 模型提供商 API +- 文件上传和管理 +- 语音处理集成 +- 分析和监控 + +详细的开发指南和组件文档，请参阅 [开发指南](../getting-started/development-guide)。 \ No newline at end of file diff --git a/doc/docs/zh/getting-started/development-guide.md b/doc/docs/zh/getting-started/development-guide.md new file mode 100644 index 000000000..a4fa85b0b --- /dev/null +++ b/doc/docs/zh/getting-started/development-guide.md @@ -0,0 +1,150 @@ +# Nexent 开发指南 + +本指南为开发者提供全面的信息，帮助理解并参与 Nexent 项目，涵盖架构、技术栈、开发环境搭建和最佳实践。 + +## 🏗️ 整体架构 + +``` +nexent/ +├── frontend/ # 前端应用 (Next.js + TypeScript) +├── backend/ # 后端服务 (FastAPI + Python) +├── sdk/ # Python SDK +├── docker/ # Docker 部署配置 +├── make/ # 构建脚本 +├── test/ # 测试代码 +└── assets/ # 静态资源 +``` + +## 🛠️ 技术栈 + +### 前端技术栈 +- **框架**: Next.js 14 (App Router) +- **语言**: TypeScript +- **UI库**: React + Tailwind CSS +- **状态管理**: React Hooks +- **国际化**: react-i18next +- **HTTP客户端**: Fetch API + +### 后端技术栈 +- **框架**: FastAPI +- **语言**: Python 3.10+ +- **数据库**: PostgreSQL + Redis + Elasticsearch +- **文件存储**: MinIO +- **任务队列**: Celery + Ray +- **AI框架**: smolagents +- **向量数据库**: Elasticsearch + +### 部署技术栈 +- **容器化**: Docker + Docker Compose +- **反向代理**: Nginx +- **监控**: 内置健康检查 +- **日志**: 结构化日志 + +## 🚀 开发环境搭建 + +### 环境要求 +- Python 3.10+ +- Node.js 18+ +- Docker & Docker Compose +- uv (Python 包管理器) + +### 后端设置 +```bash +cd backend +uv sync && uv pip install -e ../sdk +``` + +### 前端设置 +```bash +cd frontend +npm install +npm run dev +``` + +### 服务启动 +Nexent 包含三个核心后端服务，需要分别启动： +```bash +python backend/data_process_service.py # 数据处理服务 +python backend/main_service.py # 主服务 +python backend/nexent_mcp_service.py # MCP 服务 +``` + +## 🔧 开发模块指南 + +### 🎨 前端开发 +- **技术栈**: Next.js 14 + TypeScript + React + Tailwind CSS +- **核心功能**: 用户界面、实时聊天、配置管理、国际化 +- **详细信息**: 查看 [前端概览](../frontend/overview) + +### 🔧 后端开发 +- **技术栈**: FastAPI + Python 3.10+ + PostgreSQL + Redis + Elasticsearch +- **核心功能**: API服务、智能体管理、数据处理、向量搜索 +- **详细信息**: 查看 [后端概览](../backend/overview) + +### 🤖 AI 智能体开发 +- **框架**: 基于 smolagents 的企业级智能体框架 +- **核心功能**: 智能体创建、工具集成、推理执行、多模态支持 +- **自定义智能体**: 查看 [智能体概览](../agents/overview) +- **系统提示词**: 位于 `backend/prompts/` +- **实现步骤**: 创建实例 → 配置工具 → 设置提示词 → 测试运行 +- **详细信息**: 查看 [智能体概览](../agents/overview) + +### 🛠️ 工具开发 +- **MCP 工具系统**: 基于 Model Context Protocol +- **开发流程**: 实现逻辑 → 注册工具 → 重启服务 +- **协议遵循**: 工具开发需遵循 MCP 协议 +- **详细规范**: 查看 [工具开发指南](../sdk/core/tools) + +### 📦 SDK 开发工具包 +- **功能**: 提供完整的AI代理、模型调用、工具集成接口 +- **模块**: 核心代理、数据处理、向量数据库 +- **详细信息**: 查看 [SDK 概览](../sdk/overview) + +### 📊 数据处理 +- **文件处理**: 支持 20+ 种格式 +- **分块策略**: basic、by_title、none +- **流式处理**: 大文件内存优化 +- **详细信息**: 查看 [数据处理指南](../sdk/data-process) + +## 🏗️ 构建与部署 + +### Docker 构建 +详细的构建指南请参考 [Docker 构建指南](../deployment/docker-build) + +## 📋 开发最佳实践与注意事项 + +### 代码质量 +1. **测试驱动**: 编写单元测试和集成测试 +2. **代码审查**: 遵循团队代码规范 +3. **文档更新**: 及时更新相关文档 +4. **错误处理**: 完善的异常处理和日志记录 + +### 性能优化 +1. **异步处理**: 使用异步架构提升性能 +2. **缓存策略**: 合理使用缓存机制 +3. **资源管理**: 注意内存和连接池管理 +4. **监控调试**: 使用性能监控工具 + +### 安全考虑 +1. **输入验证**: 严格验证所有输入参数 +2. **权限控制**: 实现适当的访问控制 +3. **敏感信息**: 妥善处理API密钥等敏感数据 +4. **安全更新**: 定期更新依赖和安全补丁 + +### 重要开发注意事项 +1. **服务依赖**: 确保所有服务都已启动后再测试 +2. **代码修改**: 修改代码后需重启相关服务 +3. **开发模式**: 开发环境建议用调试模式 +4. **提示词测试**: 系统提示词需充分测试 + +## 💡 获取帮助 + +### 文档资源 +- [安装指南](./installation.md) - 环境搭建和部署 +- [模型提供商](./model-providers.md) - 模型配置和API获取 +- [常见问题](./faq) - 常见问题解答 + +### 社区支持 +- [Discord 社区](https://discord.gg/tb5H3S3wyv) - 实时交流和支持 +- [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) - 问题报告和功能请求 +- [贡献指南](../contributing.md) - 参与项目开发 diff --git a/FAQ_CN.md b/doc/docs/zh/getting-started/faq.md similarity index 66% rename from FAQ_CN.md rename to doc/docs/zh/getting-started/faq.md index 4e7998963..42688f056 100644 --- a/FAQ_CN.md +++ b/doc/docs/zh/getting-started/faq.md @@ -1,63 +1,65 @@ -# Nexent 常见问题 🤔 - -[![English](https://img.shields.io/badge/English-FAQ-blue)](FAQ.md) -[![中文](https://img.shields.io/badge/中文-FAQ-green)](FAQ_CN.md) - -本常见问题解答主要针对安装和部署 Nexent 过程中可能遇到的问题。如需了解基本安装步骤，请参考 README 中的[快速开始指南](../README_CN.md#-先来试试看)。 - -## 🚀 安装与设置 - -### 🔑 Jina API 密钥 -- **Q: 如何获取用于嵌入模型的 Jina API 密钥？** - - A: 要使用基于 Jina 的嵌入模型，您需要： - 1. 访问 [Jina AI 官网](https://jina.ai/)，无需注册 - 3. 进入 Embedding 部分页面获取您的 API 密钥 - 4. 将 API 密钥添加到页面中 - -## 🚫 常见错误与运维方式 - -### 🌐 网络连接问题 -- **Q: Docker 容器如何访问宿主机上部署的模型（如 Ollama）？** - - A: 由于容器内的 `localhost` 指向容器自身，需要通过以下方式连接宿主机服务： - - **方案一：使用Docker特殊DNS名称 host.docker.internal** - 适用场景：Mac/Windows和较新版本的Docker Desktop(Linux版本也支持) - ```bash - http://host.docker.internal:11434/v1 - ``` - **方案二：使用宿主机真实 IP（需确保防火墙放行）** - ```bash - http://[宿主机IP]:11434/v1 - ``` - **方案三：修改Docker Compose配置** - 在docker-compose.yaml中添加： - ```yaml - extra_hosts: - - "host.docker.internal:host-gateway" - -### 🔌 端口冲突 -- **Q: 端口 3000 已被占用，如何修改？** - - A: 可以在 Docker Compose 配置文件中修改端口。 - -### 📦 容器问题 -- **Q: 如何查看容器日志？** - - A: 使用 `docker logs <容器名称>` 命令查看特定容器的日志。 - -### 🔢 Embedding模型问题 -- **Q: 为什么我的Embedding模型无法连通？** - - A: 创建模型填写模型URL时，注意添加 `/v1/embeddings` 后缀。完整示例如 `https://model.provider.com/v1/embeddings` - -### 📧 邮件工具配置 -- **Q: 如何启用和配置邮件工具？** - - A: 我们团队已经预制实现了基于 IMAP 和 SMTP 的邮件工具。要启用它们： - 1. 在 `.env` 文件中配置邮件参数 - 2. 在 `agent_utils.py` 中取消邮件工具相关的注释 - 3. 切换到支持邮件的系统提示词 `code_agent_with_email.yaml` - 4. 重启 MCP 服务使更改生效 - -## ❓ 需要更多帮助？ - -如果这里没有找到您的问题答案： -- 加入我们的 [Discord 社区](https://discord.gg/tb5H3S3wyv) 获取实时支持 -- 查看我们的 [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) 寻找类似问题 -- 参考我们的[贡献指南](CONTRIBUTING_CN.md)获取更详细的信息 \ No newline at end of file +# Nexent 常见问题 + +本常见问题解答主要针对安装和部署 Nexent 过程中可能遇到的问题。如需了解基本安装步骤，请参考我们文档中的[快速开始指南](./overview#quick-start)。 + +## 🚀 安装与设置 + +有关模型提供商设置和 API 密钥获取，请参阅我们的详细 **[模型提供商指南](./model-providers)**。 + +## 🚫 常见错误与运维方式 + +### 🌐 网络连接问题 +- **Q: Docker 容器如何访问宿主机上部署的模型（如 Ollama）？** + - A: 由于容器内的 `localhost` 指向容器自身，需要通过以下方式连接宿主机服务： + + **方案一：使用Docker特殊DNS名称 host.docker.internal** + 适用场景：Mac/Windows和较新版本的Docker Desktop(Linux版本也支持) + ```bash + http://host.docker.internal:11434/v1 + ``` + **方案二：使用宿主机真实 IP（需确保防火墙放行）** + ```bash + http://[宿主机IP]:11434/v1 + ``` + **方案三：修改Docker Compose配置** + 在docker-compose.yaml中添加： + ```yaml + extra_hosts: + - "host.docker.internal:host-gateway" + ``` + +### 🔌 端口冲突 +- **Q: 端口 3000 已被占用，如何修改？** + - A: 可以在 Docker Compose 配置文件中修改端口。 + +### 📦 容器问题 +- **Q: 如何查看容器日志？** + - A: 使用 `docker logs <容器名称>` 命令查看特定容器的日志。 + +## 🔍 故障排除 + +### 🔢 模型连接问题 + +- **Q: 为什么我的模型无法连接？** + - A: 请检查以下项目： + 1. **正确的 API 端点**: 确保您使用正确的 base URL + 2. **有效的 API 密钥**: 验证您的 API 密钥具有适当权限 + 3. **模型名称**: 确认模型标识符正确 + 4. **网络访问**: 确保您的部署可以访问提供商的服务器 + + 有关特定提供商设置，请参阅我们的 [模型提供商指南](./model-providers)。 + +### 📧 邮件工具配置 +- **Q: 如何启用和配置邮件工具？** + - A: 我们团队已经预制实现了基于 IMAP 和 SMTP 的邮件工具。要启用它们： + 1. 在 `.env` 文件中配置邮件参数 + 2. 在 `agent_utils.py` 中取消邮件工具相关的注释 + 3. 切换到支持邮件的系统提示词 `code_agent_with_email.yaml` + 4. 重启 MCP 服务使更改生效 + +## 💡 需要帮助 + +如果这里没有找到您的问题答案： +- 加入我们的 [Discord 社区](https://discord.gg/tb5H3S3wyv) 获取实时支持 +- 查看我们的 [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) 寻找类似问题 +- 参考我们的[贡献指南](../contributing)获取更详细的信息 \ No newline at end of file diff --git a/doc/docs/zh/getting-started/features.md b/doc/docs/zh/getting-started/features.md new file mode 100644 index 000000000..894cc8db9 --- /dev/null +++ b/doc/docs/zh/getting-started/features.md @@ -0,0 +1,78 @@ +# 核心特性 + +Nexent 提供强大的功能来构建和部署 AI 智能体，只需最少的工作量。以下是让 Nexent 独特的核心特性。 + +## 🧠 智能体提示词生成 + +将自然语言转换为可执行的提示词。Nexent 自动选择正确的工具并为每个请求规划最佳的执行路径。 + +![特性 1](../../assets/Feature1.png) + +## ⚡ 可扩展的数据处理引擎 + +处理 20+ 种数据格式，具备快速 OCR 和表格结构提取能力，从单一流程平滑扩展到大批量管道处理。 + +![特性 2](../../assets/Feature2.png) + +## 📚 个人级知识库 + +实时导入文件，自动总结内容，让智能体能够即时访问个人和全局知识，并知道从每个知识库能获取什么。 + +![特性 3](../../assets/Feature3.png) + +## 🌐 互联网知识搜索 + +连接 5+ 个网络搜索提供商，让智能体能够将最新的互联网信息与你的私有数据相结合。 + +![特性 4](../../assets/Feature4.png) + +## 🔍 知识级溯源 + +提供来自网络和知识库来源的精确引用，让每个事实都可验证。 + +![特性 5](../../assets/Feature5.png) + +## 🎭 多模态理解与对话 + +支持语音、文字、文件或图像输入。Nexent 理解语音、文本和图片，甚至可以按需生成新图像。 + +![特性 6](../../assets/Feature6.png) + +## 🔧 MCP 工具生态系统 + +插入或构建遵循 MCP 规范的 Python 插件；在不触及核心代码的情况下交换模型、工具和链。 + +![特性 7](../../assets/Feature7.png) + +## 🏗️ 架构优势 + +### ⚡ 分布式处理能力 +- **异步架构**：基于 asyncio 的高性能异步处理 +- **多线程安全**：线程安全的并发处理机制 +- **Celery 集成**：专为分布式任务队列优化 +- **批量优化**：智能批量操作减少网络开销 + +### 🏢 企业级可扩展性 +- **模块化设计**：松耦合的模块架构便于扩展 +- **插件化工具**：标准化的工具接口支持快速集成 +- **配置管理**：灵活的配置系统支持多环境部署 +- **监控友好**：完善的日志和状态监控 + +### 🚀 高性能优化 +- **连接池**：数据库和HTTP连接的智能复用 +- **内存管理**：大文件的流式处理和内存优化 +- **并发控制**：智能的并发限制和负载均衡 +- **缓存策略**：多层缓存提升响应速度 + +有关 Nexent 软件架构和技术优势的详细信息，请参阅我们的 **[软件架构](./software-architecture)** 指南。 + +## 🎯 使用场景 + +Nexent 专为各种场景设计，包括： +- **商业智能**: 自动数据分析和报告 +- **客户支持**: 集成知识库的智能聊天代理 +- **内容处理**: 文档分析、总结和提取 +- **研究助手**: 学术论文分析和信息综合 +- **个人生产力**: 日常任务和信息管理的智能助手 + +有关详细的智能体场景和真实世界的实现，请参阅我们的 **[MCP 生态系统用例场景](../mcp-ecosystem/use-cases)**。 \ No newline at end of file diff --git a/doc/docs/zh/getting-started/installation.md b/doc/docs/zh/getting-started/installation.md new file mode 100644 index 000000000..20333ae2c --- /dev/null +++ b/doc/docs/zh/getting-started/installation.md @@ -0,0 +1,124 @@ +# 安装与配置 + +## 🎯 系统要求 + +| 资源 | 最低要求 | +|----------|---------| +| **CPU** | 2 核 | +| **内存** | 6 GiB | +| **架构** | x86_64 / ARM64 | +| **软件** | 已安装 Docker 和 Docker Compose | + +## 🚀 快速开始 + +### 1. 下载和设置 + +```bash +git clone https://github.com/ModelEngine-Group/nexent.git +cd nexent/docker +cp .env.example .env # 配置环境变量 +``` + +### 2. 部署选项 + +部署脚本提供多种模式： + +```bash +bash deploy.sh +``` + +**可用部署模式:** +- **开发模式 (默认)**: 暴露所有服务端口以便调试 +- **基础设施模式**: 仅启动基础设施服务 +- **生产模式**: 为安全起见仅暴露端口 3000 +- **测试模式**: 使用开发分支镜像 + +**可选组件:** +- **终端工具**: 启用 openssh-server 供 AI 智能体执行 shell 命令 +- **区域优化**: 中国大陆用户可使用优化的镜像源 + +### 3. 访问您的安装 + +部署成功完成后： +1. 在浏览器中打开 **http://localhost:3000** +2. 按照设置向导进行初始配置 +3. 配置您的模型提供商（参见 [模型提供商指南](./model-providers)） + +## 🤖 模型配置 + +Nexent 支持所有 **OpenAI 兼容的模型**，包括： +- **大语言模型 (LLM)**: 任何 OpenAI 兼容的 API 提供商 +- **多模态视觉模型**: 文本 + 图像处理能力 +- **嵌入模型**: 所有 OpenAI 兼容的嵌入服务 +- **文本转语音和语音转文本**: 多提供商支持 +- **搜索集成**: 网络搜索和语义检索 + +### 快速提供商设置 + +有关详细设置说明和 API 密钥获取，请参阅我们的 **[模型提供商指南](./model-providers)**。 + +**快速开始推荐**: +- **LLM**: [硅基流动](https://siliconflow.cn/) (有免费额度) +- **嵌入**: [Jina AI](https://jina.ai/) (有免费额度) +- **搜索**: [EXA](https://exa.ai/) (有免费额度) + +### 配置方法 + +**方法一：Web 界面** +1. 访问 `http://localhost:3000` 的模型配置 +2. 添加提供商详细信息：Base URL、API Key、Model Name + +**方法二：环境变量** +添加到您的 `.env` 文件： +```bash +LLM_BASE_URL=https://api.siliconflow.cn/v1 +LLM_API_KEY=your_api_key +EMBEDDING_API_KEY=your_jina_key +EXA_API_KEY=your_exa_key +``` + +## 🏗️ 服务架构 + +部署包含以下组件： + +**核心服务:** +- `nexent`: 后端服务 (端口 5010) +- `nexent-web`: 前端界面 (端口 3000) +- `nexent-data-process`: 数据处理服务 (端口 5012) + +**基础设施服务:** +- `nexent-postgresql`: 数据库 (端口 5434) +- `nexent-elasticsearch`: 搜索引擎 (端口 9210) +- `nexent-minio`: 对象存储 (端口 9010，控制台 9011) +- `redis`: 缓存服务 (端口 6379) + +**可选服务:** +- `nexent-openssh-server`: 终端工具的 SSH 服务器 (端口 2222) + +## 🔌 端口映射 + +| 服务 | 内部端口 | 外部端口 | 描述 | +|---------|---------------|---------------|-------------| +| Web 界面 | 3000 | 3000 | 主应用程序访问 | +| 后端 API | 5010 | 5010 | 后端服务 | +| 数据处理 | 5012 | 5012 | 数据处理 API | +| PostgreSQL | 5432 | 5434 | 数据库连接 | +| Elasticsearch | 9200 | 9210 | 搜索引擎 API | +| MinIO API | 9000 | 9010 | 对象存储 API | +| MinIO 控制台 | 9001 | 9011 | 存储管理 UI | +| Redis | 6379 | 6379 | 缓存服务 | +| SSH 服务器 | 2222 | 2222 | 终端工具访问 | + +有关完整的端口映射详细信息，请参阅我们的 [开发容器指南](../deployment/devcontainer.md#port-mapping)。 + +## 💡 需要帮助 + +- 浏览 [常见问题](./faq) 了解常见安装问题 +- 在我们的 [Discord 社区](https://discord.gg/tb5H3S3wyv) 提问 +- 在 [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) 提交错误报告或功能建议 + +## 🔧 从源码构建 + +想要从源码构建或添加新功能？查看 [Docker 构建指南](../deployment/docker-build) 获取详细说明。 + +有关详细的安装说明和自定义选项，请查看我们的 [开发指南](./development-guide)。 \ No newline at end of file diff --git a/doc/docs/zh/getting-started/model-providers.md b/doc/docs/zh/getting-started/model-providers.md new file mode 100644 index 000000000..051cfb2d9 --- /dev/null +++ b/doc/docs/zh/getting-started/model-providers.md @@ -0,0 +1,151 @@ +# 第三方模型提供商 + +本指南帮助您获取 API 密钥并配置 Nexent 支持的各种 AI 模型提供商。Nexent 支持所有 OpenAI 兼容的模型、多模态嵌入模型和多模态大语言模型。 + +## 🤖 大语言模型 (LLM) + +### 硅基流动 (Silicon Flow) +- **网站**: [siliconflow.cn](https://siliconflow.cn/) +- **免费额度**: 可用 +- **支持模型**: DeepSeek-R1、DeepSeek-V3、QwQ-32B、GLM-4-9B-Chat 等 + +**开始使用**: +1. 访问 [硅基流动](https://siliconflow.cn/) 并创建账户 +2. 在控制台导航到 API 密钥部分 +3. 创建新的 API 密钥并复制 +4. 使用 base URL: `https://api.siliconflow.cn/v1` + +### 阿里云百炼 +- **网站**: [bailian.console.aliyun.com](https://bailian.console.aliyun.com/) +- **免费额度**: 可用 +- **支持模型**: 通义千问系列、ERNIE 及各种开源模型 + +**开始使用**: +1. 注册阿里云账户 +2. 访问百炼控制台 +3. 在模型服务部分创建 API 凭证 +4. 配置您的 API 端点和密钥 + +### 其他 OpenAI 兼容提供商 +Nexent 支持任何遵循 OpenAI API 规范的提供商： +- **OpenAI**: [platform.openai.com](https://platform.openai.com/) +- **Anthropic**: [console.anthropic.com](https://console.anthropic.com/) +- **Deepseek**: [platform.deepseek.com](https://platform.deepseek.com/) +- **月之暗面**: [platform.moonshot.cn](https://platform.moonshot.cn/) +- **本地模型**: Ollama、vLLM 或任何 OpenAI 兼容服务器 + +## 🎭 多模态视觉模型 + +Nexent 支持可以处理文本和图像的多模态模型： + +### 推荐提供商 +- **GPT-4V** (OpenAI): 最佳整体性能 +- **Claude-3** (Anthropic): 出色的推理能力 +- **Qwen-VL** (阿里巴巴): 强大的多语言支持 +- **GLM-4V** (智谱): 良好的中文语言支持 + +**配置**: 使用与 LLM 模型相同的 API 端点，但指定多模态模型名称。 + +## 🔤 嵌入模型 + +### Jina AI +- **网站**: [jina.ai](https://jina.ai/) +- **免费额度**: 可用 +- **特色**: 多语言嵌入、文档理解 + +**开始使用**: +1. 访问 [Jina AI](https://jina.ai/)（基本使用无需注册） +2. 导航到嵌入部分 +3. 从控制台获取您的 API 密钥 +4. 使用端点: `https://api.jina.ai/v1/embeddings` + +### 其他嵌入提供商 +- **OpenAI Embeddings**: text-embedding-3-small、text-embedding-3-large +- **Cohere**: 高质量多语言嵌入 +- **本地嵌入**: BGE、E5 或任何 OpenAI 兼容嵌入服务 + +## 🎤 语音模型 + +### 火山引擎语音 +- **网站**: [volcengine.com/product/voice-tech](https://www.volcengine.com/product/voice-tech) +- **免费额度**: 个人使用可用 +- **特色**: 高质量中英文语音合成 + +**开始使用**: +1. 注册火山引擎账户 +2. 访问语音技术服务 +3. 创建应用并获取 API 凭证 +4. 在环境中配置 TTS/STT 设置 + +## 🔍 搜索提供商 + +### EXA +- **网站**: [exa.ai](https://exa.ai/) +- **免费额度**: 可用 +- **特色**: 语义搜索、网络内容检索 + +**开始使用**: +1. 在 [EXA](https://exa.ai/) 注册 +2. 在控制台生成 API 密钥 +3. 配置搜索参数 + +### Tavily +- **网站**: [tavily.com](https://tavily.com/) +- **免费额度**: 可用 +- **特色**: AI 驱动的搜索、研究助手 + +**开始使用**: +1. 在 [Tavily](https://tavily.com/) 注册 +2. 从控制台获取您的 API 密钥 +3. 配置搜索偏好 + +### Linkup +- **网站**: [linkup.com](https://linkup.com/) +- **免费额度**: 可用 +- **特色**: 实时网络搜索、内容发现 + +**开始使用**: +1. 在 [Linkup](https://linkup.com/) 注册 +2. 生成 API 凭证 +3. 设置搜索参数 + +## 🛠️ 在 Nexent 中配置 + +### 模型配置页面 +1. 访问 Nexent `http://localhost:3000` +2. 导航到模型配置页面 +3. 添加您的提供商详细信息： + - **Base URL**: 提供商的 API 端点 + - **API Key**: 您的提供商 API 密钥 + - **Model Name**: 特定模型标识符 + +## 💡 选择提供商的建议 + +### 开发和测试 +- **硅基流动**: 优秀的免费额度，多种模型 +- **Jina AI**: 出色的嵌入能力 +- **本地模型**: 完全隐私和控制 + +### 生产环境 +- **OpenAI**: 最可靠，最佳文档 +- **Anthropic**: 强大的安全性和推理 +- **企业提供商**: 专用支持和 SLA + +### 成本优化 +- **免费额度**: 从硅基流动、Jina 和 EXA 开始 +- **开源**: 使用 Ollama 或 vLLM 自托管 +- **区域提供商**: 本地市场通常更便宜 + +## 🔗 快速链接 + +- [安装与配置](./installation) - 完整部署指南 +- [模型配置常见问题](./faq) - 常见配置问题 +- [已知问题](../known-issues) - 当前限制和解决方法 + +## 💡 需要帮助 + +如果您在模型提供商方面遇到问题： +1. 查看提供商特定文档 +2. 验证 API 密钥权限和配额 +3. 使用提供商官方示例进行测试 +4. 加入我们的 [Discord 社区](https://discord.gg/tb5H3S3wyv) 获取支持 \ No newline at end of file diff --git a/doc/docs/zh/getting-started/overview.md b/doc/docs/zh/getting-started/overview.md new file mode 100644 index 000000000..4237efa41 --- /dev/null +++ b/doc/docs/zh/getting-started/overview.md @@ -0,0 +1,69 @@ +# Nexent + +Nexent 是一个零代码智能体自动生成平台 —— 无需编排，无需复杂的拖拉拽操作，使用纯语言开发你想要的任何智能体。基于MCP生态，具备丰富的工具集成，同时提供多种自带智能体，满足你的工作、旅行、生活等不同场景的智能服务需要。Nexent 还提供强大的智能体运行控制、多智能体协作、数据处理和知识溯源、多模态对话、批量扩展能力。 + +> 一个提示词，无限种可能。 + +![Nexent Banner](../../assets/NexentBanner.png) + +## 🎬 Demo 视频 + + + +## 🤝 加入我们的社区 + +> *If you want to go fast, go alone; if you want to go far, go together.* + +我们已经发布了 **Nexent v1**，目前功能已经相对稳定，但仍可能存在一些 bug，我们会持续改进并不断增加新功能。敬请期待，我们很快也会公布 **v2.0** 版本！ + +* **🗺️ 查看我们的 [功能地图](https://github.com/orgs/ModelEngine-Group/projects/6)** 探索当前和即将推出的功能。 +* **🔍 试用当前版本** 并在 [问题反馈](https://github.com/ModelEngine-Group/nexent/issues) 中留下想法或报告错误。 + +> *Rome wasn't built in a day.* + +如果我们的愿景与您产生共鸣，请通过 **[贡献指南](../contributing)** 加入我们，共同塑造 Nexent。 + +早期贡献者不会被忽视：从特殊徽章和纪念品到其他实质性奖励，我们致力于感谢那些帮助 Nexent 诞生的先驱者。 + +最重要的是，我们需要关注度。请为仓库点星 ⭐ 并关注，与朋友分享，帮助更多开发者发现 Nexent —— 您的每一次点击都能为项目带来新的参与者，保持发展势头。 + +## ⚡ 快速开始 + +准备好开始了吗？以下是您的下一步： + +1. **📋 [安装与配置](./installation)** - 系统要求和部署指南 +2. **🔧 [开发指南](./development-guide)** - 从源码构建和自定义 +3. **❓ [常见问题](./faq)** - 常见问题和故障排除 + +## 🌱 MCP 工具生态 + +Nexent 基于模型上下文协议（MCP）工具生态系统构建，为集成各种工具和服务提供了灵活且可扩展的框架。MCP 被誉为"AI 的 USB-C" - 一个通用接口标准，让 AI 智能体能够无缝连接外部数据源、工具和服务。 + +了解更多关于 MCP 生态系统的信息： +- **[MCP 概览](../mcp-ecosystem/overview)** - 了解 MCP 生态系统和工具 +- **[用例与场景](../mcp-ecosystem/use-cases)** - 真实世界的智能体场景和实现 + +## ✨ 主要特性 + +Nexent 为构建强大的 AI 智能体提供全面的功能集： + +- **🤖 智能体生成** - 使用自然语言进行零代码智能体创建 +- **📊 可扩展数据处理** - 处理 20+ 种文件格式和智能提取 +- **🧠 个人知识库** - 实时文件导入和自动摘要 +- **🌐 互联网集成** - 连接多个搜索提供商和网络资源 +- **🔍 知识溯源** - 精确引用和来源验证 +- **🎭 多模态支持** - 语音、文本、图像和文件处理 +- **🔧 MCP 生态系统** - 可扩展的工具集成和自定义开发 + +有关详细的功能信息和示例，请参阅我们的 **[功能指南](./features)**。 + +## 💬 社区与联系方式 + +加入我们的 [Discord 社区](https://discord.gg/tb5H3S3wyv) 与其他开发者交流并获取帮助！ + +## 📄 许可证 + +Nexent 采用 [MIT](../license) 许可证，并附有额外条件。请阅读 [LICENSE](../license) 文件了解详情。 \ No newline at end of file diff --git a/doc/docs/zh/getting-started/software-architecture.md b/doc/docs/zh/getting-started/software-architecture.md new file mode 100644 index 000000000..462290c21 --- /dev/null +++ b/doc/docs/zh/getting-started/software-architecture.md @@ -0,0 +1,7 @@ +# 软件架构 + +我们将很快更新全面的软件架构图和文档。 + +--- + +*本文档正在积极开发中。我们将很快更新全面的架构详细信息。* \ No newline at end of file diff --git a/doc/docs/zh/index.md b/doc/docs/zh/index.md new file mode 100644 index 000000000..e9e69c446 --- /dev/null +++ b/doc/docs/zh/index.md @@ -0,0 +1,30 @@ +--- +# https://vitepress.dev/reference/default-theme-home-page +layout: home + +hero: + name: "Nexent 文档" + text: "零代码智能体平台" + tagline: "零代码智能体自动生成平台 —— 无需编排，无需复杂的拖拉拽操作，使用纯语言开发你想要的任何智能体。" + actions: + - theme: brand + text: 开始使用 + link: /zh/getting-started/overview + - theme: alt + text: 查看GitHub + link: https://github.com/ModelEngine-Group/nexent + +features: + - title: 智能体自动生成 + details: 将自然语言转换为可执行的智能体。Nexent自动选择正确的工具并为每个请求规划最佳的执行路径。 + - title: 可扩展的数据处理引擎 + details: 处理20+种数据格式，具备快速OCR和表格结构提取能力，从单一流程平滑扩展到大批量管道处理。 + - title: 个人级知识库 + details: 实时导入文件，自动总结内容，让智能体能够即时访问个人和全局知识，并知道从每个知识库能获取什么。 + - title: 互联网知识搜索 + details: 连接5+个网络搜索提供商，让智能体能够将最新的互联网信息与你的私有数据相结合。 + - title: 知识级溯源 + details: 提供来自网络和知识库来源的精确引用，让每个事实都可验证。 + - title: 多模态理解与对话 + details: 支持语音、文字、文件或图像输入。Nexent理解语音、文本和图片，甚至可以按需生成新图像。 +--- \ No newline at end of file diff --git a/doc/docs/zh/known-issues.md b/doc/docs/zh/known-issues.md new file mode 100644 index 000000000..559392e96 --- /dev/null +++ b/doc/docs/zh/known-issues.md @@ -0,0 +1,41 @@ +# 已知问题 + +此页面列出了当前版本 Nexent 中的已知问题和限制。我们正在积极修复这些问题，并会随着解决方案的推出更新此页面。 + +## 🐛 当前问题 + +### 1. OpenSSH 容器软件安装限制 + +**问题描述**: 在 OpenSSH 容器中为终端工具使用安装其他软件包目前由于容器限制而比较困难。 + +**状态**: 开发中 + +**影响**: 需要在终端环境中使用自定义工具或软件包的用户可能面临限制。 + +**计划解决方案**: 我们正在努力提供改进的容器和文档，使自定义变得更容易。这将包括更好的包管理和更灵活的容器配置。 + +**预期时间线**: 改进的容器支持计划在即将发布的版本中提供。 + +## 📝 问题报告 + +如果您遇到此处未列出的任何问题，请： + +1. **查看我们的 [常见问题](./getting-started/faq)** 寻找常见解决方案 +2. **搜索现有问题** 在 [GitHub Issues](https://github.com/ModelEngine-Group/nexent/issues) +3. **创建新问题** 并提供详细信息，包括： + - 重现步骤 + - 预期行为 + - 实际行为 + - 系统信息 + - 日志文件（如适用） + +## 🔄 问题状态更新 + +我们定期更新此页面的已知问题状态。请经常查看更新，或关注我们的 [GitHub 仓库](https://github.com/ModelEngine-Group/nexent) 以获取通知。 + +## 💬 社区支持 + +如需即时帮助或讨论问题： +- 加入我们的 [Discord 社区](https://discord.gg/tb5H3S3wyv) +- 在 [GitHub Discussions](https://github.com/ModelEngine-Group/nexent/discussions) 提问 +- 如果您想帮助修复问题，请查看我们的 [贡献指南](./contributing) \ No newline at end of file diff --git a/doc/docs/zh/license.md b/doc/docs/zh/license.md new file mode 100644 index 000000000..af455ff43 --- /dev/null +++ b/doc/docs/zh/license.md @@ -0,0 +1,38 @@ +# 许可证 + +Nexent 基于 MIT 许可证，并附加额外条件。此许可证允许开源和商业使用，但对某些使用场景包含特定限制。 + +如有任何法律问题或商业许可咨询，请联系官方许可团队。 + +## 完整许可证文本 + +``` +# Nexent Open Source License + +Nexent is licensed under the MIT License, with the following additional conditions: + +Nexent is permitted to be used commercially, including as a backend service for other applications or as an application development platform for enterprises. However, when the following conditions are met, you must contact the producer to obtain a commercial license: + +a. Multi-tenant SaaS service: Unless explicitly authorized by Nexent in writing, you may not use the Nexent source code to operate a multi-tenant SaaS service. +b. LOGO and copyright information: In the process of using Nexent's frontend, you may not remove or modify the LOGO or copyright information in the Nexent console or applications. This restriction is inapplicable to uses of Nexent that do not involve its frontend. + +Please contact chenshuangrui@huawei.com by email to inquire about licensing matters. + +As a contributor, you should agree that: + +a. The producer can adjust the open-source agreement to be more strict or relaxed as deemed necessary. +b. Your contributed code may be used for commercial purposes, such as Nexent's cloud business. + +Apart from the specific conditions mentioned above, all other rights and restrictions follow the MIT License. +Detailed information about the MIT License can be found at: https://opensource.org/licenses/MIT + +Copyright © 2025 Huawei Technologies Co., Ltd. +``` + +## 联系信息 + +许可证咨询： +- **邮箱**: chenshuangrui@huawei.com +- **MIT 许可证详情**: https://opensource.org/licenses/MIT + +请确保在项目中使用 Nexent 之前理解并遵守所有许可证条款。 \ No newline at end of file diff --git a/doc/docs/zh/mcp-ecosystem/overview.md b/doc/docs/zh/mcp-ecosystem/overview.md new file mode 100644 index 000000000..6c97460b9 --- /dev/null +++ b/doc/docs/zh/mcp-ecosystem/overview.md @@ -0,0 +1,70 @@ +# MCP 工具生态系统 + +Nexent 基于模型上下文协议（MCP）工具生态系统构建，提供灵活且可扩展的框架来集成各种工具和服务。MCP 作为"AI 的 USB-C"——一个通用接口标准，允许 AI 智能体无缝连接外部数据源、工具和服务。 + +## 什么是 MCP？ + +模型上下文协议（MCP）是一个开放协议，使 AI 应用程序能够安全地连接到外部数据源和工具。它为 AI 模型访问和与外部系统交互提供了标准化方式，使构建强大的、上下文感知的 AI 应用程序变得更加容易。 + +## MCP 社区中心 + +全球 MCP 生态系统正在蓬勃发展，多个平台支持 MCP 开发和部署： + +| 平台 | 描述 | 备注 | +|----------|-------------|-------| +| **[GitHub MCP Server](https://github.com/github/github-mcp-server)** | 与 Claude、GPT-4、Copilot 等深度集成，支持 Go 和 Python | OAuth/GitHub 账户授权 | +| **[Qdrant MCP Vector Server](https://github.com/qdrant/mcp-server-qdrant)** | 语义向量存储，Python/Go 兼容 | 与 LangChain 和其他工具兼容 | +| **[Anthropic Reference MCP Servers](https://github.com/modelcontextprotocol/servers)** | 轻量级教学和原型工具，Python | 包括 fetch、git 和其他通用工具 | +| **[AWS Labs MCP Server](https://github.com/awslabs/mcp)** | AWS+Go+CDK 云参考服务 | 适用于云环境 | +| **[MCP Hub China](https://www.mcp-cn.com/)** | 中文精选高质量 MCP 服务平台 | 注重质量而非数量，社区驱动 | +| **[ModelScope MCP Marketplace](https://modelscope.cn/mcp)** | 中国最大的 MCP 社区，拥有 1,500+ 服务 | 从高德地图到支付宝，全面的服务覆盖 | +| **社区 MCP 服务器** | 各种特定场景的源代码集合 | 主要是实验性和创新工具 | + +## 推荐的 MCP 工具 + +| 工具名称 | 功能 | 描述 | +|-----------|----------|-------------| +| **[高德地图](https://modelscope.cn/mcp/servers/@amap/amap-maps)** | 地理服务和导航 | 综合地图、地理编码、路由和位置服务 | +| **[必应搜索（中文）](https://modelscope.cn/mcp/servers/@yan5236/bing-cn-mcp-server)** | 中文网络搜索 | 优化的中文网络搜索和信息检索 | +| **[12306 火车票查询](https://modelscope.cn/mcp/servers/@Joooook/12306-mcp)** | 中国铁路票务预订 | 实时列车时刻表、票务可用性和预订协助 | +| **[支付宝 MCP](https://modelscope.cn/mcp/servers/@alipay/mcp-server-alipay)** | 支付和金融服务 | 数字支付、金融工具和服务集成 | +| **[飞常准航空](https://modelscope.cn/mcp/servers/@variflight-ai/variflight-mcp)** | 航班信息和航空数据 | 实时航班跟踪、时刻表和航空分析 | +| **[顺序思考](https://modelscope.cn/mcp/servers/@modelcontextprotocol/sequentialthinking)** | 结构化问题解决框架 | 将复杂问题分解为可管理的顺序步骤 | +| **[ArXiv AI 搜索](https://modelscope.cn/mcp/servers/@blazickjp/arxiv-mcp-server)** | 学术论文搜索和研究 | 高级搜索和检索科学论文和研究 | +| **[Firecrawl MCP 服务器](https://modelscope.cn/mcp/servers/@mendableai/firecrawl-mcp-server)** | 网络爬虫和内容提取 | 智能网络爬虫、数据提取和内容处理 | + +## MCP 的优势 + +### 标准化 +- **通用接口**: MCP 提供连接 AI 模型与外部工具的一致方式 +- **互操作性**: 为一个 MCP 兼容平台构建的工具可在其他平台上工作 +- **减少开发时间**: 标准化协议意味着减少自定义集成工作 + +### 安全性 +- **受控访问**: MCP 提供安全的、基于权限的外部资源访问 +- **身份验证**: 内置支持各种身份验证方法 +- **审计跟踪**: 跟踪和监控所有外部交互 + +### 可扩展性 +- **模块化设计**: 在不影响核心应用程序的情况下添加或删除工具 +- **负载分配**: 在多个服务器间分布工具执行 +- **版本管理**: 优雅地处理工具的不同版本 + +## MCP 入门 + +1. **探索可用工具**: 浏览 MCP 市场以找到适合您需求的工具 +2. **安装工具**: 将 MCP 工具添加到您的 Nexent 实例 +3. **配置访问**: 设置身份验证和权限 +4. **创建智能体**: 构建利用多个 MCP 工具的智能体 +5. **监控性能**: 跟踪使用情况并优化工具选择 + +有关详细的集成指南，请参阅我们的 [后端工具文档](../backend/tools/)。 + +## 构建自定义 MCP 工具 + +有兴趣构建自己的 MCP 工具？请查看： +- [LangChain 工具指南](../backend/tools/langchain) +- [MCP 工具开发](../backend/tools/mcp) +- [SDK 工具文档](../sdk/core/tools) + +MCP 生态系统使您能够构建可以与现实世界无缝交互的智能体，访问实时数据，执行复杂操作，并在几乎任何领域提供上下文帮助。 \ No newline at end of file diff --git a/doc/docs/zh/mcp-ecosystem/use-cases.md b/doc/docs/zh/mcp-ecosystem/use-cases.md new file mode 100644 index 000000000..0409fe83e --- /dev/null +++ b/doc/docs/zh/mcp-ecosystem/use-cases.md @@ -0,0 +1,112 @@ +# 建议的智能体场景 + +借助 MCP 强大的生态系统，您可以为各种场景创建复杂的 AI 智能体。以下是一些经过验证的用例，展示了 Nexent 平台的多样性和强大功能。 + +## 🌍 旅行规划智能体 + +创建一个处理旅程各个方面的综合旅行助手： + +- **路线规划**: 使用高德地图进行详细的路线规划和导航 📍 +- **交通运输**: 集成 12306 进行火车票预订和时刻表查询 🚄 +- **航班管理**: 连接飞常准获取实时航班信息 ✈️ +- **支付处理**: 启用支付宝进行无缝支付处理 💳 + +**示例功能**: +- 规划具有最优路线的多城市行程 +- 预订火车票并跟踪延误信息 +- 监控航班状态和登机口变更 +- 处理不同服务的支付 +- 提供实时更新和替代方案 + +## 🔬 研究助手智能体 + +为学术和专业工作构建强大的研究伙伴： + +- **学术搜索**: 利用 ArXiv 搜索最新学术论文 📚 +- **网络研究**: 使用必应搜索获取全面的网络信息 🔍 +- **结构化分析**: 应用顺序思考进行系统化研究 🧠 +- **数据提取**: 集成 Firecrawl 进行网络数据收集 🕷️ + +**示例功能**: +- 进行系统性文献综述 +- 从多个来源提取和综合信息 +- 生成研究摘要和参考文献 +- 跟踪研究趋势和引用 +- 将发现整理成结构化报告 + +## 💼 商业智能智能体 + +开发智能商业分析和决策支持： + +- **数据集成**: 通过各种 MCP 服务器连接多个数据源 📊 +- **位置分析**: 使用地理工具进行基于位置的洞察 🗺️ +- **财务分析**: 集成支付系统获取财务数据 💰 +- **决策支持**: 应用结构化思维框架进行分析 🎯 + +**示例功能**: +- 分析市场趋势和客户行为 +- 生成自动化商业报告 +- 提供基于位置的市场洞察 +- 跟踪财务绩效指标 +- 支持战略决策制定 + +## 🏠 智能生活助手 + +为日常生活优化创建个人助手： + +- **购物助手**: 结合地图服务和支付集成 🛒 +- **通勤优化**: 使用交通工具进行路线规划 🚗 +- **本地发现**: 集成网络搜索获取本地推荐 🏪 +- **信息管理**: 应用智能内容提取 📱 + +**示例功能**: +- 查找和比较本地服务和产品 +- 优化日常通勤路线 +- 发现新的餐厅和活动 +- 管理个人日程和提醒 +- 处理例行交易和预订 + +## 🎓 教育支持智能体 + +构建学习和教学助手： + +- **内容研究**: 访问学术数据库和论文 +- **互动学习**: 提供解释和教程 +- **进度跟踪**: 监控学习目标和里程碑 +- **资源发现**: 找到相关的教育材料 + +## 🏥 健康信息智能体 + +创建专注于健康的信息助手： + +- **医学研究**: 搜索医学文献和指南 +- **预约管理**: 处理日程安排和提醒 +- **健康跟踪**: 监控健康指标和趋势 +- **信息综合**: 提供清晰的健康信息摘要 + +## 💡 创意内容智能体 + +为内容创作和营销开发智能体： + +- **市场研究**: 分析趋势和竞争对手信息 +- **内容规划**: 生成创意和内容日历 +- **SEO 优化**: 研究关键词和优化策略 +- **多平台发布**: 协调跨平台内容 + +## 智能体场景入门 + +1. **确定用例**: 选择符合您需求的场景 +2. **选择 MCP 工具**: 从我们的生态系统中选择合适的工具 +3. **配置集成**: 设置身份验证和权限 +4. **创建智能体**: 使用 Nexent 的零代码平台构建 +5. **测试和迭代**: 基于实际使用情况完善您的智能体 + +## 最佳实践 + +- **从简单开始**: 从基本功能开始，逐步扩展 +- **规划集成**: 考虑不同工具如何协同工作 +- **监控性能**: 跟踪使用情况并优化工具选择 +- **用户反馈**: 基于用户需求持续改进 +- **安全优先**: 确保适当的身份验证和数据保护 + +每个场景都可以根据您的具体要求进行定制和扩展。MCP 生态系统提供了以创新方式组合工具的灵活性，创造针对您确切需求量身定制的强大 AI 体验。 \ No newline at end of file diff --git a/doc/docs/zh/sdk/basic-usage.md b/doc/docs/zh/sdk/basic-usage.md new file mode 100644 index 000000000..480ff65a8 --- /dev/null +++ b/doc/docs/zh/sdk/basic-usage.md @@ -0,0 +1,244 @@ +# 💡 基本使用 + +本指南提供使用 Nexent SDK 构建智能体的全面介绍。 + +## 🚀 快速开始 + +### 💡 基本导入 + +```python +import nexent +from nexent.core import MessageObserver, ProcessType +from nexent.core.agents import CoreAgent, NexentAgent +from nexent.core.models import OpenAIModel +from nexent.core.tools import ExaSearchTool, KnowledgeBaseSearchTool +``` + +## 🤖 创建你的第一个智能体 + +### 🔧 设置环境 + +```python +# 创建消息观察者用于流式输出 +observer = MessageObserver() + +# 创建模型（模型和智能体必须使用同一个观察者） +model = OpenAIModel( + observer=observer, + model_id="your-model-id", + api_key="your-api-key", + api_base="your-api-base" +) +``` + +### 🛠️ 添加工具 + +```python +# 创建搜索工具 +search_tool = ExaSearchTool( + exa_api_key="your-exa-key", + observer=observer, + max_results=5 +) + +# 创建知识库工具 +kb_tool = KnowledgeBaseSearchTool( + top_k=5, + observer=observer +) +``` + +### 🤖 构建智能体 + +```python +# 使用工具和模型创建智能体 +agent = CoreAgent( + observer=observer, + tools=[search_tool, kb_tool], + model=model, + name="my_agent", + max_steps=5 +) +``` + +### 🚀 运行智能体 + +```python +# 用你的问题运行智能体 +result = agent.run("你的问题") + +# 访问最终答案 +print(result.final_answer) +``` + +## 🎯 高级使用模式 + +### 🔧 自定义工具集成 + +```python +from nexent.core.tools import BaseTool + +class CustomTool(BaseTool): + def __init__(self, observer: MessageObserver): + super().__init__(observer=observer, name="custom_tool") + + def run(self, input_text: str) -> str: + # 你的自定义工具逻辑 + return f"已处理: {input_text}" + +# 将自定义工具添加到智能体 +custom_tool = CustomTool(observer=observer) +agent.tools.append(custom_tool) +``` + +### 🎭 多模态智能体设置 + +```python +from nexent.core.models import OpenAIVLMModel + +# 创建支持视觉的模型 +vision_model = OpenAIVLMModel( + observer=observer, + model_id="gpt-4-vision-preview", + api_key="your-api-key" +) + +# 创建具有视觉能力的智能体 +vision_agent = CoreAgent( + observer=observer, + tools=[search_tool], + model=vision_model, + name="vision_agent" +) +``` + +### 📡 流式输出处理 + +```python +# 监控流式输出 +def handle_stream(message: str, process_type: ProcessType): + if process_type == ProcessType.MODEL_OUTPUT_THINKING: + print(f"🤔 思考中: {message}") + elif process_type == ProcessType.EXECUTION_LOGS: + print(f"⚙️ 执行中: {message}") + elif process_type == ProcessType.FINAL_ANSWER: + print(f"✅ 答案: {message}") + +# 设置带有自定义处理器的观察者 +observer.set_message_handler(handle_stream) +``` + +## 🔧 配置选项 + +### ⚙️ 智能体配置 + +```python +agent = CoreAgent( + observer=observer, + tools=[search_tool, kb_tool], + model=model, + name="my_agent", + max_steps=10, # 最大执行步骤 + temperature=0.7, # 模型创造力水平 + system_prompt="你是一个有用的AI助手。" # 自定义系统提示 +) +``` + +### 🔧 工具配置 + +```python +# 使用特定参数配置搜索工具 +search_tool = ExaSearchTool( + exa_api_key="your-exa-key", + observer=observer, + max_results=10, # 搜索结果数量 + search_type="neural", # 搜索类型: neural, keyword 等 + include_domains=["example.com"], # 限制搜索到特定域名 + exclude_domains=["spam.com"] # 排除特定域名 +) +``` + +## 📊 错误处理 + +### 🛡️ 优雅的错误恢复 + +```python +try: + result = agent.run("你的问题") + print(f"成功: {result.final_answer}") +except Exception as e: + print(f"发生错误: {e}") + # 适当处理错误 +``` + +### 🔧 工具错误处理 + +```python +# 工具自动处理错误并提供回退方案 +search_tool = ExaSearchTool( + exa_api_key="your-exa-key", + observer=observer, + max_results=5, + fallback_to_keyword=True # 如果神经搜索失败，回退到关键词搜索 +) +``` + +## 🎭 多模态示例 + +### 💡 图像处理 + +```python +# 使用视觉模型处理图像 +result = vision_agent.run( + "描述你在这张图片中看到的内容", + image_path="path/to/image.jpg" +) +``` + +### 💡 语音处理 + +```python +from nexent.core.tools import SpeechToTextTool, TextToSpeechTool + +# 添加语音能力 +stt_tool = SpeechToTextTool(observer=observer) +tts_tool = TextToSpeechTool(observer=observer) + +voice_agent = CoreAgent( + observer=observer, + tools=[stt_tool, tts_tool, search_tool], + model=model, + name="voice_agent" +) +``` + +## 🔍 最佳实践 + +### 💡 性能优化 + +- **连接池**: 重用连接以获得更好的性能 +- **批量处理**: 在可能的情况下一起处理多个请求 +- **缓存**: 为频繁访问的数据实现缓存 +- **异步操作**: 对I/O密集型操作使用 async/await + +### 💡 安全考虑 + +- **API密钥管理**: 使用环境变量安全存储API密钥 +- **输入验证**: 在处理前验证所有输入 +- **速率限制**: 实现速率限制以防止滥用 +- **错误日志**: 记录错误而不暴露敏感信息 + +### 💡 监控和调试 + +```python +# 启用详细日志 +import logging +logging.basicConfig(level=logging.DEBUG) + +# 监控智能体执行 +for step in agent.execution_steps: + print(f"步骤 {step.step_number}: {step.action}") + print(f"结果: {step.result}") +``` + +有关更高级的使用模式和详细的API文档，请参阅 **[核心组件](./overview)** 和 **[工具开发](./core/tools)** 指南。 \ No newline at end of file diff --git a/doc/docs/zh/sdk/core/models.md b/doc/docs/zh/sdk/core/models.md new file mode 100644 index 000000000..d762bcf35 --- /dev/null +++ b/doc/docs/zh/sdk/core/models.md @@ -0,0 +1,218 @@ +# Nexent 模型模块 + +本模块提供了多种AI模型服务，包括语音服务、嵌入模型、大语言模型和视觉语言模型。每个模型都遵循统一的接口设计，支持配置管理和错误处理。 + +## 目录 + +- [语音服务 (STT & TTS)](#语音服务-stt--tts) +- [嵌入模型](#嵌入模型) +- [大语言模型](#大语言模型) +- [视觉语言模型](#视觉语言模型) + +## 语音服务 (STT & TTS) + +本模块提供了一个统一的语音服务，在单个端口上同时运行语音识别(STT)和语音合成(TTS)服务，使用WebSocket进行实时通信。 + +### 功能特点 + +- **语音识别(STT)**: 通过WebSocket连接进行实时音频转写 +- **语音合成(TTS)**: 通过WebSocket流式传输将文本转换为音频 +- **单一端口**: 两种服务在同一端口上运行，简化部署和使用 +- **仅WebSocket**: 两种服务使用一致的WebSocket API模式 +- **流式处理**: 支持实时流式音频识别和合成，提供低延迟体验 +- **错误处理**: 完善的错误处理和状态反馈机制 + +### 设置 + +1. 创建一个包含API凭证的`.env`文件: + +``` +# STT配置 +APPID=your_stt_appid +TOKEN=your_token + +# TTS配置 +APPID=your_tts_appid +TOKEN=your_tts_token +CLUSTER=your_cluster +VOICE_TYPE=your_voice_type +``` + +### API端点 + +#### 语音识别(STT) + +- WebSocket: `/stt/ws` + - **请求格式**: 以二进制块流式传输PCM音频数据 + - **音频要求**: 16kHz采样率, 16位深度, 单声道, PCM原始格式 + - **响应格式**: 实时JSON转写结果 + - **响应字段**: + - `result` 或 `trans_result.text`: 识别的文本 + - `is_final`: 是否为最终结果 + - `status`: 服务状态信息 + - `error`: 如有错误，包含错误信息 + +#### 语音合成(TTS) + +- WebSocket: `/tts/ws` + - **请求格式**: 发送JSON格式的文本: `{"text": "要合成的文本"}` + - **响应格式**: 二进制音频块 (默认为MP3格式) + - **完成信号**: 最终消息: `{"status": "completed"}` + - **错误响应**: `{"error": "错误信息"}` + +## 嵌入模型 + +嵌入模型提供了将文本、图像等多种数据类型转换为向量表示的能力，支持多种后端服务。 + +### 功能特点 + +- **多后端支持**: 支持Jina和OpenAI等多种嵌入服务。 +- **统一文本接口**: 所有模型均提供统一的 `get_embeddings` 方法，接受字符串或字符串列表作为输入，方便处理纯文本数据。 +- **多模态能力**: 像 `JinaEmbedding` 这样的多模态模型，额外提供了 `get_multimodal_embeddings` 方法，可以处理包含文本和图像URL的复杂输入。 +- **配置灵活**: 支持通过参数或环境变量进行配置。 +- **连接测试**: 内置 `check_connectivity()` 方法，用于验证与API服务的连接状态。 + +### 使用示例 + +#### 获取文本嵌入 (所有模型通用) + +所有嵌入模型都使用 `get_embeddings` 方法来获取文本的嵌入向量。此方法接受单个字符串或字符串列表。 + +```python +from nexent.core.models.embedding_model import JinaEmbedding, OpenAICompatibleEmbedding + +# 初始化Jina模型 (同样适用于OpenAICompatibleEmbedding) +embedding = JinaEmbedding(api_key="your_jina_api_key") + +# 获取单个文本的嵌入 +text_input = "Hello, Nexent!" +embeddings = embedding.get_embeddings(text_input) +print(f"单文本嵌入向量数量: {len(embeddings)}") + +# 获取多个文本的嵌入 +text_list_input = ["这是第一段文本。", "这是第二段文本。"] +embeddings_list = embedding.get_embeddings(text_list_input) +print(f"多文本嵌入向量数量: {len(embeddings_list)}") +``` + +#### 获取多模态嵌入 (JinaEmbedding) + +对于支持多模态输入的模型（如 `JinaEmbedding`），可以使用 `get_multimodal_embeddings` 方法来处理包含文本和图像的混合输入。 + +```python +from nexent.core.models.embedding_model import JinaEmbedding + +# 初始化Jina模型 +embedding = JinaEmbedding(api_key="your_jina_api_key") + +# 定义包含文本和图像的多模态输入 +multimodal_input = [ + {"text": "A beautiful sunset over the beach"}, + {"image": "https://example.com/sunset.jpg"} +] + +# 获取多模态嵌入 +multimodal_embeddings = embedding.get_multimodal_embeddings(multimodal_input) +print(f"多模态嵌入向量数量: {len(multimodal_embeddings)}") +``` + + +## 大语言模型 + +大语言模型提供了文本生成和对话能力，基于OpenAI API实现。 + +### 功能特点 + +- **流式输出**: 支持实时流式文本生成 +- **温度控制**: 可调节生成文本的随机性 +- **上下文管理**: 支持多轮对话和上下文保持 +- **工具调用**: 支持函数调用和工具使用 + +### 使用示例 + +```python +from nexent.core.models.openai_llm import OpenAIModel +from nexent.core.utils.observer import MessageObserver + +# 初始化模型 +observer = MessageObserver() +model = OpenAIModel(observer=observer, temperature=0.2, top_p=0.95) + +# 发送消息 +messages = [{"role": "user", "content": "Hello"}] +response = model(messages=messages) +``` + +## 视觉语言模型 + +视觉语言模型结合了图像理解和文本生成能力，支持图像描述和视觉问答。 + +### 功能特点 + +- **图像处理**: 支持本地图像文件和URL +- **流式输出**: 支持实时流式文本生成 +- **提示词定制**: 可自定义系统提示词 +- **多模态理解**: 结合视觉和语言理解能力 + +### 使用示例 + +```python +from nexent.core.models.openai_vlm import OpenAIVLModel +from nexent.core.utils.observer import MessageObserver + +# 初始化模型 +observer = MessageObserver() +model = OpenAIVLModel(observer=observer) + +# 分析图像 +image_path = "path/to/image.jpg" +result = model.analyze_image(image_path, system_prompt="请描述这张图片") +``` + +## 通用特性 + +所有模型都支持以下通用特性： + +### 错误处理 + +- 连接错误捕获和处理 +- 服务状态监控和反馈 +- 客户端友好的错误消息 + +### 配置管理 + +- 环境变量配置 +- .env文件支持 +- 运行时配置覆盖 + +### 连接测试 + +所有模型都实现了`check_connectivity()`方法，用于测试与远程服务的连接状态： + +```python +# 测试连接 +if model.check_connectivity(): + print("服务连接正常") +else: + print("服务连接失败") +``` + +## 实现细节 + +### 模块化设计 + +- 每个模型都是独立的类，实现特定的功能 +- 通过抽象基类定义统一接口 +- 支持灵活的配置和扩展 + +### 性能优化 + +- 异步处理提高并发性能 +- 流式处理减少延迟 +- 连接池和资源管理 + +### 安全性 + +- API密钥管理 +- 请求验证和授权 +- 错误处理和日志记录 \ No newline at end of file diff --git a/doc/docs/zh/sdk/core/tools.md b/doc/docs/zh/sdk/core/tools.md new file mode 100644 index 000000000..ca0ff06bc --- /dev/null +++ b/doc/docs/zh/sdk/core/tools.md @@ -0,0 +1,355 @@ +# Nexent 工具开发规范 + +本文档基于对现有工具的分析，总结了 Nexent SDK 中工具开发的完整规范和最佳实践。 + +## 工具分类 + +当前 SDK 包含以下工具类型： + +### 搜索工具 +- **ExaSearchTool**: 基于 EXA API 的网络搜索工具 +- **TavilySearchTool**: 基于 Tavily API 的网络搜索工具 +- **LinkupSearchTool**: 基于 Linkup API 的搜索工具 +- **KnowledgeBaseSearchTool**: 本地知识库搜索工具 + +### 文件管理工具 +- **CreateFileTool**: 创建包含内容的新文件 +- **ReadFileTool**: 从文件系统读取文件内容 +- **DeleteFileTool**: 从文件系统删除文件 +- **MoveItemTool**: 移动或重命名文件和目录 +- **CreateDirectoryTool**: 创建新目录 +- **DeleteDirectoryTool**: 删除目录及其内容 +- **ListDirectoryTool**: 列出目录内容和详细信息 + +### 系统工具 +- **TerminalTool**: 执行 shell 命令和系统操作 + +### 通信工具 +- **GetEmailTool**: 通过 IMAP 的邮件获取工具 +- **SendEmailTool**: 通过 SMTP 的邮件发送工具 + +## 工具共性特征 + +### 1. 基础架构 +- **基类继承**: 所有工具必须继承自 `smolagents.tools.Tool` +- **参数管理**: 使用 `pydantic.Field` 进行参数定义和验证 +- **流式输出**: 集成 `MessageObserver` 支持实时消息传递 +- **多语言支持**: 内置中英文双语提示信息 + +### 2. 核心属性 +每个工具类必须包含以下类属性： + +```python +class ToolExample(Tool): + name = "tool_name" # 工具唯一标识符 + description = "工具功能描述" # 详细功能说明 + inputs = { # 输入参数定义 + "param": {"type": "string", "description": "参数描述"} + } + output_type = "string" # 输出类型 + tool_sign = "x" # 工具标识符（可选） +``` + +### 3. 消息处理机制 +- **ProcessType 枚举**: 使用不同类型区分消息（TOOL, CARD, SEARCH_CONTENT, PICTURE_WEB 等） +- **Observer 模式**: 通过 MessageObserver 实现实时消息推送 +- **JSON 格式**: 所有消息内容使用 JSON 格式确保一致性 + +### 4. 异常处理策略 +- **统一异常**: 使用 Exception 抛出错误信息 +- **错误日志**: 使用 logging 模块记录详细错误信息 +- **优雅降级**: 在可能的情况下提供备选方案 + +## 命名规范 + +### 文件命名 +- **格式**: `{功能名}_tool.py` +- **风格**: 小写字母，单词间用下划线连接 +- **示例**: `exa_search_tool.py`, `knowledge_base_search_tool.py` + +### 类命名 +- **格式**: `{功能名}Tool` +- **风格**: 大驼峰命名法（PascalCase） +- **示例**: `ExaSearchTool`, `KnowledgeBaseSearchTool` + +### 属性和方法命名 +- **格式**: 小写字母，单词间用下划线连接 +- **私有方法**: 以单下划线开头（如 `_filter_images`） +- **示例**: `max_results`, `running_prompt_en`, `_decode_subject` + +### 工具标识符规范 +- **tool_sign**: 单字母标识符，用于区分工具来源 +- **分配规则**: + - `a`: 知识库搜索 (KnowledgeBaseSearchTool) + - `b`: 网络搜索 (ExaSearchTool, TavilySearchTool) + - `l`: Linkup搜索 (LinkupSearchTool) + - 其他字母按功能类型分配 + +## 代码结构模板 + +### 基础模板 + +```python +import json +import logging +from typing import Optional +from smolagents.tools import Tool +from pydantic import Field + +from ..utils.observer import MessageObserver, ProcessType + +logger = logging.getLogger("your_tool_name") + +class YourTool(Tool): + name = "your_tool" + description = "工具功能的详细描述，说明适用场景和使用方法" + inputs = { + "param1": { + "type": "string", + "description": "参数1的详细描述" + }, + "param2": { + "type": "integer", + "description": "参数2的详细描述", + "default": 10, + "nullable": True + } + } + output_type = "string" + tool_sign = "y" # 选择合适的标识符 + + def __init__( + self, + config_param: str = Field(description="配置参数"), + observer: MessageObserver = Field(description="消息观察者", default=None, exclude=True), + optional_param: int = Field(description="可选参数", default=5) + ): + super().__init__() + self.config_param = config_param + self.observer = observer + self.optional_param = optional_param + + # 多语言提示信息 + self.running_prompt_zh = "正在执行..." + self.running_prompt_en = "Processing..." + + # 记录操作序号（如果需要） + self.record_ops = 0 + + def forward(self, param1: str, param2: int = 10) -> str: + """工具的主要执行方法 + + Args: + param1: 参数1说明 + param2: 参数2说明 + + Returns: + JSON格式的字符串结果 + + Raises: + Exception: 详细的错误信息 + """ + try: + # 发送工具运行消息 + if self.observer: + running_prompt = (self.running_prompt_zh + if self.observer.lang == "zh" + else self.running_prompt_en) + self.observer.add_message("", ProcessType.TOOL, running_prompt) + + # 发送卡片信息（可选） + card_content = [{"icon": "your_icon", "text": param1}] + self.observer.add_message("", ProcessType.CARD, + json.dumps(card_content, ensure_ascii=False)) + + # 主要业务逻辑 + result = self._execute_main_logic(param1, param2) + + # 处理结果并返回 + return self._format_result(result) + + except Exception as e: + logger.error(f"Error in {self.name}: {str(e)}") + raise Exception(f"执行{self.name}时发生错误: {str(e)}") + + def _execute_main_logic(self, param1: str, param2: int): + """执行主要业务逻辑的私有方法""" + # 实现具体的业务逻辑 + pass + + def _format_result(self, result) -> str: + """格式化返回结果""" + formatted_result = { + "status": "success", + "data": result, + "tool": self.name + } + return json.dumps(formatted_result, ensure_ascii=False) +``` + +### 搜索工具模板 + +```python +import json +import logging +from typing import List +from smolagents.tools import Tool +from pydantic import Field + +from ..utils.observer import MessageObserver, ProcessType +from ..utils.tools_common_message import SearchResultTextMessage + +logger = logging.getLogger("search_tool_name") + +class SearchTool(Tool): + name = "search_tool" + description = "搜索工具的详细描述，包括搜索范围和适用场景" + inputs = { + "query": {"type": "string", "description": "搜索查询"}, + "max_results": {"type": "integer", "description": "最大结果数", "default": 5, "nullable": True} + } + output_type = "string" + tool_sign = "s" + + def __init__( + self, + api_key: str = Field(description="API密钥"), + observer: MessageObserver = Field(description="消息观察者", default=None, exclude=True), + max_results: int = Field(description="最大搜索结果数", default=5) + ): + super().__init__() + self.api_key = api_key + self.observer = observer + self.max_results = max_results + self.record_ops = 0 + + self.running_prompt_zh = "搜索中..." + self.running_prompt_en = "Searching..." + + def forward(self, query: str, max_results: int = None) -> str: + if max_results is None: + max_results = self.max_results + + # 发送搜索状态消息 + if self.observer: + running_prompt = (self.running_prompt_zh + if self.observer.lang == "zh" + else self.running_prompt_en) + self.observer.add_message("", ProcessType.TOOL, running_prompt) + card_content = [{"icon": "search", "text": query}] + self.observer.add_message("", ProcessType.CARD, + json.dumps(card_content, ensure_ascii=False)) + + try: + # 执行搜索 + search_results = self._perform_search(query, max_results) + + if not search_results: + raise Exception("未找到搜索结果！请尝试更短或更宽泛的查询。") + + # 格式化搜索结果 + formatted_results = self._format_search_results(search_results) + + # 记录搜索内容 + if self.observer: + search_results_data = json.dumps(formatted_results["json"], ensure_ascii=False) + self.observer.add_message("", ProcessType.SEARCH_CONTENT, search_results_data) + + return json.dumps(formatted_results["return"], ensure_ascii=False) + + except Exception as e: + logger.error(f"搜索错误: {str(e)}") + raise Exception(f"搜索失败: {str(e)}") + + def _perform_search(self, query: str, max_results: int): + """执行实际的搜索操作""" + # 实现具体的搜索逻辑 + pass + + def _format_search_results(self, results): + """格式化搜索结果为统一格式""" + search_results_json = [] + search_results_return = [] + + for index, result in enumerate(results): + search_result_message = SearchResultTextMessage( + title=result.get("title", ""), + url=result.get("url", ""), + text=result.get("content", ""), + published_date=result.get("date", ""), + source_type="url", + filename="", + score=result.get("score", ""), + score_details=result.get("score_details", {}), + cite_index=self.record_ops + index, + search_type=self.name, + tool_sign=self.tool_sign + ) + search_results_json.append(search_result_message.to_dict()) + search_results_return.append(search_result_message.to_model_dict()) + + self.record_ops += len(search_results_return) + + return { + "json": search_results_json, + "return": search_results_return + } +``` + +## 开发流程规范 + +### 1. 开发前准备 +- 确定工具功能和适用场景 +- 选择合适的工具分类和标识符 +- 检查是否与现有工具功能重复 + +### 2. 实现步骤 +1. **创建工具文件**: 按命名规范创建 `{name}_tool.py` +2. **定义类结构**: 继承 Tool 基类，定义必要属性 +3. **实现构造函数**: 使用 pydantic Field 定义参数 +4. **实现 forward 方法**: 核心功能逻辑 +5. **添加私有方法**: 将复杂逻辑拆分为私有方法 +6. **集成消息观察者**: 支持流式输出和多语言 +7. **异常处理**: 完善的错误处理和日志记录 + +### 3. 测试和集成 +1. **单元测试**: 测试各种输入情况和边界条件 +2. **集成测试**: 与 CoreAgent 集成测试 +3. **更新导出**: 在 `__init__.py` 中添加工具导出 +4. **文档更新**: 更新相关文档和示例 + +## 最佳实践 + +### 1. 性能优化 +- **异步处理**: 对于耗时操作使用异步处理 +- **连接池**: 复用网络连接减少延迟 +- **缓存机制**: 适当使用缓存提升响应速度 +- **并发控制**: 使用 Semaphore 控制并发请求数 + +### 2. 安全性 +- **输入验证**: 严格验证输入参数 +- **敏感信息**: API密钥等敏感信息不应出现在日志中 +- **错误信息**: 避免在错误信息中泄露敏感信息 +- **超时控制**: 设置合理的超时时间防止阻塞 + +### 3. 可维护性 +- **模块化设计**: 将复杂功能拆分为多个方法 +- **清晰注释**: 为复杂逻辑添加详细注释 +- **类型注解**: 使用完整的类型注解 +- **文档字符串**: 为所有公共方法添加文档字符串 + +### 4. 用户体验 +- **多语言支持**: 提供中英文双语提示 +- **进度反馈**: 通过 MessageObserver 提供实时反馈 +- **错误提示**: 提供清晰的错误信息和解决建议 +- **参数验证**: 在执行前验证参数有效性 + +## 注意事项 + +1. **版本兼容**: 确保工具与不同版本的依赖库兼容 +2. **资源清理**: 及时释放网络连接、文件句柄等资源 +3. **日志级别**: 合理设置日志级别，避免过多调试信息 +4. **配置管理**: 支持通过环境变量配置关键参数 +5. **错误恢复**: 在可能的情况下提供错误恢复机制 + +通过遵循这些规范，可以确保新开发的工具与现有工具保持一致性，并具备良好的可维护性和可扩展性。 diff --git a/doc/docs/zh/sdk/data-process.md b/doc/docs/zh/sdk/data-process.md new file mode 100644 index 000000000..a3fd9aad4 --- /dev/null +++ b/doc/docs/zh/sdk/data-process.md @@ -0,0 +1,283 @@ +# DataProcessCore 使用指南 + +## 概述 + +`DataProcessCore` 是一个统一的文件处理核心类，支持多种文件格式的自动检测和处理，提供灵活的分块策略和多种输入源支持。 + +## 主要功能 + +### 1. 核心处理方法：`file_process()` + +**函数签名：** +```python +def file_process(self, + file_path_or_url: Optional[str] = None, + file_data: Optional[bytes] = None, + chunking_strategy: str = "basic", + destination: str = "local", + filename: Optional[str] = None, + **params) -> List[Dict] +``` + +**参数说明：** + +| 参数名 | 类型 | 必需 | 描述 | 可选值 | +|--------|------|------|------|--------| +| `file_path_or_url` | `str` | 否* | 本地文件路径或远程URL | 任何有效的文件路径或URL | +| `file_data` | `bytes` | 否* | 文件的字节数据（用于内存处理） | 任何有效的字节数据 | +| `chunking_strategy` | `str` | 否 | 分块策略 | `"basic"`, `"by_title"`, `"none"` | +| `destination` | `str` | 否 | 目标类型，指示文件来源 | `"local"`, `"minio"`, `"url"` | +| `filename` | `str` | 否** | 文件名 | 任何有效的文件名 | +| `**params` | `dict` | 否 | 额外的处理参数 | 见下方参数详情 | + +*注：`file_path_or_url` 和 `file_data` 必须提供其中一个 +**注：使用 `file_data` 时，`filename` 为必需参数 + +**分块策略 (`chunking_strategy`) 详解：** + +| 策略值 | 描述 | 适用场景 | 输出特点 | +|--------|------|----------|----------| +| `"basic"` | 基础分块策略 | 大多数文档处理场景 | 根据内容长度自动分块 | +| `"by_title"` | 按标题分块 | 结构化文档（如技术文档、报告） | 以标题为界限进行分块 | +| `"none"` | 不分块 | 短文档或需要完整内容的场景 | 返回单个包含全部内容的块 | + +**目标类型 (`destination`) 详解：** + +| 目标值 | 描述 | 使用场景 | 要求 | +|--------|------|----------|------| +| `"local"` | 本地文件 | 处理本地存储的文件 | 提供有效的本地文件路径 | +| `"minio"` | MinIO存储 | 处理云存储中的文件 | 需要数据库依赖 | +| `"url"` | 远程URL | 处理网络资源 | 需要数据库依赖 | + +**额外参数 (`**params`) 详解：** + +| 参数名 | 类型 | 默认值 | 描述 | 适用处理器 | +|--------|------|--------|------|-----------| +| `max_characters` | `int` | `1500` | 每个块的最大字符数 | Generic | +| `new_after_n_chars` | `int` | `1200` | 达到此字符数后开始新块 | Generic | +| `strategy` | `str` | `"fast"` | 处理策略 | Generic | +| `skip_infer_table_types` | `list` | `[]` | 跳过推断的表格类型 | Generic | +| `task_id` | `str` | `""` | 任务标识符 | Generic | + +**返回值格式：** + +返回 `List[Dict]`，每个字典包含以下字段： + +**通用字段：** +| 字段名 | 类型 | 描述 | 示例 | +|--------|------|------|------| +| `content` | `str` | 文本内容 | `"这是文档的第一段..."` | +| `path_or_url` | `str` | 文件路径或URL | `"/path/to/file.pdf"` | +| `filename` | `str` | 文件名 | `"document.pdf"` | + +**Excel文件额外字段：** +| 字段名 | 类型 | 描述 | 示例 | +|--------|------|------|------| +| `metadata` | `dict` | 元数据信息 | `{"chunk_index": 0, "file_type": "xlsx"}` | + +**Generic文件额外字段：** +| 字段名 | 类型 | 描述 | 示例 | +|--------|------|------|------| +| `language` | `str` | 语言标识（可选） | `"en"` | +| `metadata` | `dict` | 元数据信息（可选） | `{"chunk_index": 0}` | + +## 支持的文件类型 + +### Excel文件 +- `.xlsx` - Excel 2007及更高版本 +- `.xls` - Excel 97-2003版本 + +### 通用文件 +- `.txt` - 纯文本文件 +- `.pdf` - PDF文档 +- `.docx` - Word 2007及更高版本 +- `.doc` - Word 97-2003版本 +- `.html`, `.htm` - HTML文档 +- `.md` - Markdown文件 +- `.rtf` - 富文本格式 +- `.odt` - OpenDocument文本 +- `.pptx` - PowerPoint 2007及更高版本 +- `.ppt` - PowerPoint 97-2003版本 + +## 使用示例 + +### 示例1：处理本地文本文件 +```python +from nexent.data_process import DataProcessCore + +core = DataProcessCore() + +# 基础处理 +result = core.file_process( + file_path_or_url="/path/to/document.txt", + destination="local", + chunking_strategy="basic" +) + +print(f"处理得到 {len(result)} 个块") +for i, chunk in enumerate(result): + print(f"块 {i}: {chunk['content'][:100]}...") +``` + +### 示例2：处理Excel文件 +```python +# 处理Excel文件 +result = core.file_process( + file_path_or_url="/path/to/spreadsheet.xlsx", + destination="local", + chunking_strategy="none" # Excel通常不需要分块 +) + +for chunk in result: + print(f"文件: {chunk['filename']}") + print(f"内容: {chunk['content']}") + print(f"元数据: {chunk['metadata']}") +``` + +### 示例3：处理内存中的文件 +```python +# 读取文件到内存 +with open("/path/to/document.pdf", "rb") as f: + file_bytes = f.read() + +# 处理内存中的文件 +result = core.file_process( + file_data=file_bytes, + filename="document.pdf", + chunking_strategy="by_title", + max_characters=2000 # 自定义参数 +) +``` + +### 示例4：处理远程文件（需要数据库依赖） +```python +# 处理MinIO中的文件 +result = core.file_process( + file_path_or_url="minio://bucket/path/to/file.docx", + destination="minio", + filename="file.docx", + chunking_strategy="basic" +) +``` + +## 辅助方法 + +### 1. 获取支持的文件类型 +```python +core = DataProcessCore() +supported_types = core.get_supported_file_types() +print("Excel文件:", supported_types["excel"]) +print("通用文件:", supported_types["generic"]) +``` + +### 2. 验证文件类型 +```python +is_supported = core.validate_file_type("document.pdf") +print(f"PDF文件是否支持: {is_supported}") +``` + +### 3. 获取处理器信息 +```python +info = core.get_processor_info("spreadsheet.xlsx") +print(f"处理器类型: {info['processor_type']}") +print(f"文件扩展名: {info['file_extension']}") +print(f"是否支持: {info['is_supported']}") +``` + +### 4. 获取支持的策略和目标类型 +```python +strategies = core.get_supported_strategies() +destinations = core.get_supported_destinations() +print(f"支持的分块策略: {strategies}") +print(f"支持的目标类型: {destinations}") +``` + +## 错误处理 + +### 常见异常 + +| 异常类型 | 触发条件 | 解决方案 | +|----------|----------|----------| +| `ValueError` | 参数无效（如同时提供file_path_or_url和file_data） | 检查参数组合 | +| `FileNotFoundError` | 本地文件不存在或远程文件无法获取 | 验证文件路径 | +| `ImportError` | 处理远程文件时缺少数据库依赖 | 安装相关依赖 | + +### 错误处理示例 +```python +try: + result = core.file_process( + file_path_or_url="/nonexistent/file.txt", + destination="local" + ) +except FileNotFoundError as e: + print(f"文件未找到: {e}") +except ValueError as e: + print(f"参数错误: {e}") +except Exception as e: + print(f"处理失败: {e}") +``` + +## 性能优化建议 + +1. **选择合适的分块策略**： + - 小文件使用 `"none"` + - 大文件使用 `"basic"` + - 结构化文档使用 `"by_title"` + +2. **调整分块参数**： + - 根据下游处理需求调整 `max_characters` + - 平衡处理速度和内存使用 + +3. **文件类型优化**： + - Excel文件通常不需要分块 + - PDF文件建议使用较大的 `max_characters` + +4. **批量处理**： + - 复用 `DataProcessCore` 实例 + - 避免重复初始化 + +## 数据流架构 + +Nexent 系统中的数据处理遵循以下流程模式： + +### 1. 用户请求流程 +``` +用户输入 → 前端验证 → API调用 → 后端路由 → 业务服务 → 数据访问 → 数据库 +``` + +### 2. AI Agent执行流程 +``` +用户消息 → Agent创建 → 工具调用 → 模型推理 → 流式响应 → 结果保存 +``` + +### 3. 知识库文件处理流程 +``` +文件上传 → 临时存储 → 数据处理 → 向量化 → 知识库存储 → 索引更新 +``` + +**详细步骤**： +1. **文件上传**: 前端接收用户上传的文件 +2. **临时存储**: 文件存储到临时位置或MinIO +3. **数据处理**: 使用 `DataProcessCore` 进行格式转换和分块 +4. **向量化**: 通过嵌入模型生成向量表示 +5. **知识库存储**: 将处理后的内容存储到Elasticsearch +6. **索引更新**: 更新搜索索引以支持检索 + +### 4. 实时文件处理流程 +``` +文件上传 → 临时存储 → 数据处理 → Agent处理 → 实时回答 +``` + +**详细步骤**： +1. **文件上传**: 用户在对话中上传文件 +2. **临时存储**: 文件临时保存用于处理 +3. **数据处理**: 实时提取文件内容和结构 +4. **Agent处理**: AI智能体分析文件内容 +5. **实时回答**: 基于文件内容提供即时回复 + +### 数据处理优化策略 + +- **异步处理**: 大文件处理使用异步任务队列 +- **批量操作**: 多文件处理时使用批量优化 +- **缓存机制**: 重复文件的处理结果缓存 +- **流式处理**: 大文件的内存流式处理 \ No newline at end of file diff --git a/doc/docs/zh/sdk/overview.md b/doc/docs/zh/sdk/overview.md new file mode 100644 index 000000000..17f0d8c16 --- /dev/null +++ b/doc/docs/zh/sdk/overview.md @@ -0,0 +1,151 @@ +# Nexent SDK + +Nexent 是一个强大的企业级 Agent SDK，革命性地简化了智能体开发。基于经过验证的企业架构构建，为构建生产就绪的 AI 智能体提供全面解决方案，支持分布式处理、实时流式传输、多模态能力以及丰富的工具和模型生态系统。 + +## 🚀 安装方式 + +### 用户安装 +如果您想使用 nexent： + +```bash +# 使用 uv 安装 (推荐) +uv add nexent + +# 或从源码安装 +uv pip install . +``` + +### 开发环境设置 +如果您是第三方 SDK 开发者： + +```bash +# 方式 1：仅安装依赖（不安装 nexent） +uv pip install -r requirements.txt + +# 方式 2：安装完整开发环境（包括 nexent） +uv pip install -e ".[dev]" # 包含所有开发工具（测试、代码质量检查等） +``` + +开发环境包含以下额外功能： +- 代码质量检查工具 (ruff) +- 测试框架 (pytest) +- 数据处理依赖 (unstructured) +- 其他开发依赖 + +## 📖 基本使用 + +有关详细的使用示例和分步指南，请参阅我们的 **[基本使用指南](./basic-usage)**。 + + + +## ⭐ 主要特性 + +### 🏢 企业级Agent框架 +- **基于 SmolAgent 扩展**：支持复杂业务场景 +- **生产就绪**：专为企业环境构建，具备适当的扩展和监控 +- **全面测试**：广泛的测试覆盖确保可靠性 + +### ⚡ 分布式处理能力 +- **异步处理**：基于 asyncio 的高性能异步架构 +- **多线程支持**：线程安全的并发处理机制 +- **Celery 友好**：专为分布式任务队列优化的设计 +- **批量操作**：支持大规模数据的批量处理和优化 + +### 🔧 丰富的 Agent 工具生态 +- **搜索工具**：EXA、Tavily、Linkup 网络搜索和本地知识库检索 +- **通信工具**：IMAP/SMTP 邮件收发功能 +- **MCP 集成**：支持 Model Context Protocol 工具集成 +- **统一规范**：所有工具遵循一致的开发标准和接口设计 + +### 🎭 多模态支持 +- **语音服务**：集成 STT & TTS，支持实时语音交互 +- **视觉模型**：支持图像理解和处理 +- **长上下文模型**：处理大规模文档和对话历史 + +### 📊 强大的数据处理能力 +- **多格式支持**：处理 PDF、Word、Excel、HTML 等 20+ 种格式 +- **智能分块**：支持基础分块、标题分块、无分块等策略 +- **内存处理**：支持大文件的内存流式处理 +- **分布式架构**：支持任务队列管理和并行处理 + +### 🔍 向量数据库集成 +- **Elasticsearch**：企业级向量搜索和文档管理 +- **混合搜索**：结合精确匹配和语义搜索 +- **嵌入模型**：集成 Jina 等主流嵌入模型 +- **大规模优化**：支持数百万级文档的高效检索 + +## 🏗️ 核心组件 + +### 🤖 NexentAgent - 企业级Agent框架 + +nexent 的核心是 `NexentAgent` 和 `CoreAgent` 类，提供完整的智能体解决方案： + +- **多模型支持**：支持 OpenAI、视觉语言模型、长上下文模型等 +- **MCP 集成**：无缝集成 Model Context Protocol 工具生态 +- **动态工具加载**：支持本地工具和 MCP 工具的动态创建和管理 +- **分布式执行**：基于线程池和异步架构的高性能执行引擎 +- **状态管理**：完善的任务状态追踪和错误恢复机制 + +### ⚙️ CoreAgent - 代码执行引擎 + +继承并增强了 SmolAgent 的 `CodeAgent`，提供以下关键能力： + +- **Python代码执行**：支持解析和执行Python代码，能够动态处理任务 +- **多语言支持**：内置中英文提示词模板，可根据需要切换语言 +- **流式输出**：通过 MessageObserver 实现模型输出的实时流式显示 +- **步骤追踪**：记录并展示Agent执行的每个步骤，便于调试和监控 +- **中断控制**：支持任务中断和优雅停止机制 +- **错误处理**：完善的错误处理机制，提高稳定性 +- **状态管理**：维护和传递执行状态，支持复杂任务的连续处理 + +CoreAgent 实现了ReAct框架的思考-行动-观察循环： +1. **思考**：使用大语言模型生成解决方案代码 +2. **行动**：执行生成的Python代码 +3. **观察**：收集执行结果和日志 +4. **重复**：根据观察结果继续思考和执行，直到任务完成 + +### 📡 MessageObserver - 流式消息处理 + +消息观察者模式的核心实现，用于处理 Agent 的流式输出： + +- **流式输出捕获**：实时捕获模型生成的token +- **过程类型区分**：根据不同的处理阶段（模型输出、代码解析、执行日志等）格式化输出 +- **多语言支持**：支持中英文输出格式 +- **统一接口**：为不同来源的消息提供统一处理方式 + +ProcessType枚举定义了以下处理阶段： +- `STEP_COUNT`: 当前执行步骤 +- `MODEL_OUTPUT_THINKING`: 模型思考过程输出 +- `MODEL_OUTPUT_CODE`: 模型代码生成输出 +- `PARSE`: 代码解析结果 +- `EXECUTION_LOGS`: 代码执行结果 +- `AGENT_NEW_RUN`: Agent基本信息 +- `FINAL_ANSWER`: 最终总结结果 +- `SEARCH_CONTENT`: 搜索结果内容 +- `PICTURE_WEB`: 网络图片处理结果 + +## 🛠️ 工具集 + +nexent 提供了丰富的工具生态系统，支持多种类型的任务处理。所有工具都遵循统一的开发规范，确保一致性和可扩展性。 + +详细的工具开发规范和完整的工具文档，请参考：**[工具开发指南](./core/tools)** + +## 📊 数据处理服务 + +企业级文档处理服务，提供分布式处理能力，支持 20+ 种文档格式、智能分块策略和内存优化。 + +详细的数据处理能力和使用示例，请参考：**[数据处理指南](./data-process)** + +## 🔍 向量数据库服务 + +提供企业级向量搜索和文档管理服务，支持多种搜索模式、嵌入模型集成和大规模优化。 + +全面的向量数据库集成和配置文档，请参考：**[向量数据库指南](./vector-database)** + +## 🤖 模型服务 + +提供统一的多模态 AI 模型服务，支持各种模型类型和提供商。 + +全面的模型集成和使用文档，请参考：**[模型架构指南](./core/models)** + +更多详细使用说明，请参考各模块的专门文档。 diff --git a/doc/docs/zh/sdk/vector-database.md b/doc/docs/zh/sdk/vector-database.md new file mode 100644 index 000000000..68c72af45 --- /dev/null +++ b/doc/docs/zh/sdk/vector-database.md @@ -0,0 +1,871 @@ +# Elasticsearch 向量数据库 + +一个用于 Elasticsearch 的向量搜索和文档管理服务，支持 Jina 嵌入模型集成。 + +## 环境设置 + +1. 创建一个包含凭据的 `.env` 文件: + +``` +ELASTICSEARCH_HOST=https://localhost:9200 +ELASTICSEARCH_API_KEY=your_api_key_here +JINA_API_URL=https://api.jina.ai/v1/embeddings +JINA_MODEL=jian_model_name +JINA_API_KEY=your_jina_api_key_here +``` + +2. 安装依赖: + +```bash +pip install elasticsearch python-dotenv requests fastapi uvicorn +``` + +## Docker 部署指南 + +### 前置条件 + +1. 安装Docker + - 访问 [Get Docker](https://www.docker.com/products/docker-desktop) 安装Docker + - 如果使用Docker Desktop，请确保分配至少4GB内存 + - 可以在Docker Desktop的 **Settings > Resources** 中调整内存使用 + +2. 创建Docker网络 + ```bash + docker network create elastic + ``` + +### Elasticsearch部署 + +1. 拉取Elasticsearch镜像 + ```bash + docker pull docker.elastic.co/elasticsearch/elasticsearch:8.17.4 + ``` + +2. 启动Elasticsearch容器 (静默模式，等待3-5分钟) + ```bash + docker run -d --name es01 --net elastic -p 9200:9200 -m 6GB -e "xpack.ml.use_auto_machine_memory_percent=true" docker.elastic.co/elasticsearch/elasticsearch:8.17.4 + ``` + +3. 查看Elasticsearch日志 + ```bash + docker logs -f es01 + ``` + +4. 重置密码（确认Yes） + ```bash + docker exec -it es01 /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic + ``` + +5. 保存重要信息 + - 容器启动时会显示 `elastic` 用户密码和Kibana的注册令牌 + - 建议将密码保存为环境变量： + ```bash + export ELASTIC_PASSWORD="your_password" + ``` + +6. 复制SSL证书 + ```bash + docker cp es01:/usr/share/elasticsearch/config/certs/http_ca.crt . + ``` + +7. 验证部署 + ```bash + curl --cacert http_ca.crt -u elastic:$ELASTIC_PASSWORD https://localhost:9200 -k + ``` + +8. 获取api_key + ```bash + curl --cacert http_ca.crt \ + -u elastic:$ELASTIC_PASSWORD \ + --request POST \ + --url https://localhost:9200/_security/api_key \ + --header 'Content-Type: application/json' \ + --data '{ + "name": "取个名字" + }' + ``` + +9. 检验key有效 + ```bash + curl --request GET \ + --url https://XXX.XX.XXX.XX:9200/_cluster/health \ + --header 'Authorization: ApiKey API-KEY' + ``` + +### Kibana部署 (可选) + +1. 拉取Kibana镜像 + ```bash + docker pull docker.elastic.co/kibana/kibana:8.17.4 + ``` + +2. 启动Kibana容器 + ```bash + docker run -d --name kib01 --net elastic -p 5601:5601 docker.elastic.co/kibana/kibana:8.17.4 + ``` + +3. 查看Kibana日志 + ```bash + docker logs -f kib01 + ``` + +4. 配置Kibana + - 生成令牌，运行： + ```bash + docker exec -it es01 /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana + ``` + - 在浏览器中，访问http://localhost:5601输入生成的注册令牌 + - 可能需要`docker logs -f kib01`查看验证码 + +5. 使用elastic用户和之前生成的密码登录Kibana + +### 常用管理命令 + +```bash +# 停止容器 +docker stop es01 +docker stop kib01 + +# 删除容器 +docker rm es01 +docker rm kib01 + +# 删除网络 +docker network rm elastic +``` + +### 生产环境注意事项 + +1. 数据持久化 + - 必须绑定数据卷到 `/usr/share/elasticsearch/data` + - 启动命令示例: + ```bash + docker run -d --name es01 --net elastic -p 9200:9200 -m 6GB -v es_data:/usr/share/elasticsearch/data docker.elastic.co/elasticsearch/elasticsearch:8.17.4 + ``` + +2. 内存配置 + - 根据实际需求调整容器内存限制 + - 建议至少分配6GB内存 + +3. 故障排除 + - 内存不足: 检查Docker Desktop的内存设置 + - 端口冲突: 确保9200端口未被占用 + - 证书问题: 确保正确复制了SSL证书 + - 昇腾服务器vm.max_map_count问题: + ```bash + # 错误信息 + # node validation exception: bootstrap checks failed + # max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] + + # 解决方案（在宿主机执行）： + sudo sysctl -w vm.max_map_count=262144 + + # 永久生效，编辑 /etc/sysctl.conf 添加： + vm.max_map_count=262144 + + # 然后执行： + sudo sysctl -p + ``` + +### 远程部署调试指南 + +当Elasticsearch部署在远程服务器上时，可能会遇到一些网络访问的问题。以下是常见问题和解决方案： + +1. 远程访问被拒绝 + - 症状：curl请求返回 "Connection reset by peer" + - 解决方案： + ```bash + # 使用SSH隧道进行端口转发 + ssh -L 9200:localhost:9200 user@remote_server + + # 在新终端中通过本地端口访问 + curl -H "Authorization: ApiKey your_api_key" https://localhost:9200/_cluster/health\?pretty -k + ``` + +2. 网络配置检查清单 + - 确保远程服务器的防火墙允许9201端口访问 + ```bash + # 对于使用iptables的系统 + sudo iptables -A INPUT -p tcp --dport 9200 -j ACCEPT + sudo service iptables save + ``` + + - 检查Elasticsearch网络配置 + ```yaml + # elasticsearch.yml 配置示例 + network.host: 0.0.0.0 + http.cors.enabled: true + http.cors.allow-origin: "*" + ``` + +3. 安全配置建议 + - 在生产环境中，建议： + - 限制CORS的 `allow-origin` 为特定域名 + - 使用反向代理（如Nginx）管理SSL终端 + - 配置适当的网络安全组规则 + - 使用SSL证书而不是自签名证书 + +4. 使用环境变量 + - 在 `.env` 文件中配置远程连接： + ``` + ELASTICSEARCH_HOST=https://remote_server:9200 + ELASTICSEARCH_API_KEY=your_api_key + ``` + + - 如果使用SSH隧道，可以保持使用localhost： + ``` + ELASTICSEARCH_HOST=https://localhost:9200 + ``` + +5. 故障排除命令 + ```bash + # 检查端口监听状态 + netstat -tulpn | grep 9200 + + # 检查ES日志 + docker logs es01 + + # 测试SSL连接 + openssl s_client -connect remote_server:9200 + ``` + +## 核心组件 + +- `elasticsearch_core.py`: 主类，包含所有 Elasticsearch 操作 +- `embedding_model.py`: 处理使用 Jina AI 模型生成嵌入向量 +- `utils.py`: 数据格式化和显示的工具函数 +- `elasticsearch_service.py`: FastAPI 服务，提供 REST API 接口 + +## 使用示例 + +### 基本初始化 + +```python +from nexent.vector_database.elasticsearch_core import ElasticSearchCore + +# 使用 .env 文件中的凭据初始化 +es_core = ElasticSearchCore() + +# 或直接指定凭据 +es_core = ElasticSearchCore( + host="https://localhost:9200", + api_key="your_api_key", + verify_certs=False, + ssl_show_warn=False, +) +``` + +### 索引管理 + +```python +# 创建新的向量索引 +es_core.create_vector_index("my_documents") + +# 列出所有用户索引 +indices = es_core.get_user_indices() +print(indices) + +# 获取所有索引的统计信息 +all_indices_stats = es_core.get_all_indices_stats() +print(all_indices_stats) + +# 删除索引 +es_core.delete_index("my_documents") + +# 创建测试知识库 +index_name, doc_count = es_core.create_test_knowledge_base() +print(f"创建了测试知识库 {index_name}，包含 {doc_count} 个文档") +``` + +### 文档操作 + +```python +# 索引文档（自动生成嵌入向量） +documents = [ + { + "id": "doc1", + "title": "文档 1", + "file": "文件1.txt", + "path_or_url": "https://example.com/doc1", + "content": "这是文档 1 的内容", + "process_source": "Web", + "embedding_model_name": "jina-embeddings-v2-base-en", # 指定嵌入模型 + "file_size": 1024, # 文件大小（字节） + "create_time": "2023-06-01T10:30:00" # 文件创建时间 + }, + { + "id": "doc2", + "title": "文档 2", + "file": "文件2.txt", + "path_or_url": "https://example.com/doc2", + "content": "这是文档 2 的内容", + "process_source": "Web" + # 如果未提供其他字段，将使用默认值 + } +] +# 支持批量处理，默认批处理大小为3000 +total_indexed = es_core.index_documents("my_documents", documents, batch_size=3000) +print(f"成功索引了 {total_indexed} 个文档") + +# 通过 URL 或路径删除文档 +deleted_count = es_core.delete_documents_by_path_or_url("my_documents", "https://example.com/doc1") +print(f"删除了 {deleted_count} 个文档") +``` + +### 搜索功能 + +```python +# 文本精确搜索 +results = es_core.accurate_search("my_documents", "示例查询", top_k=5) +for result in results: + print(f"得分: {result['score']}, 文档: {result['document']['title']}") + +# 语义向量搜索 +results = es_core.semantic_search("my_documents", "示例查询", top_k=5) +for result in results: + print(f"得分: {result['score']}, 文档: {result['document']['title']}") + +# 混合搜索 +results = es_core.hybrid_search( + "my_documents", + "示例查询", + top_k=5, + weight_accurate=0.3 # 精确搜索权重为0.3，向量搜索权重为0.7 +) +for result in results: + print(f"得分: {result['score']}, 文档: {result['document']['title']}") +``` + +### 统计和监控 + +```python +# 获取索引统计信息 +stats = es_core.get_index_stats("my_documents") +print(stats) + +# 获取文件列表及详细信息 +file_details = es_core.get_file_list_with_details("my_documents") +print(file_details) + +# 获取嵌入模型信息 +embedding_model = es_core.get_embedding_model_info("my_documents") +print(f"使用的嵌入模型: {embedding_model}") + +# 打印所有索引信息 +es_core.print_all_indices_info() +``` + +## ElasticSearchCore 主要功能 + +ElasticSearchCore 类提供了以下主要功能: + +- **索引管理**: 创建和删除索引，获取用户索引列表和统计信息 +- **文档操作**: 批量索引带有嵌入向量的文档，删除指定文档 +- **搜索操作**: 提供精确文本搜索、语义向量搜索、以及混合搜索 +- **统计和监控**: 获取索引统计数据，查看数据源、创建时间和文件列表等信息 + +### 新增高级功能 + +```python +# 获取索引的文件列表及详细信息 +files = es_core.get_file_list_with_details("my_documents") +for file in files: + print(f"文件路径: {file['path_or_url']}") + print(f"文件名: {file['file']}") + print(f"文件大小: {file['file_size']} 字节") + print(f"创建时间: {file['create_time']}") + print("---") + +# 获取嵌入模型信息 +model_info = es_core.get_embedding_model_info("my_documents") +print(f"使用的嵌入模型: {model_info}") + +# 获取所有索引的综合统计信息 +all_stats = es_core.get_all_indices_stats() +for index_name, stats in all_stats.items(): + print(f"索引: {index_name}") + print(f"文档数: {stats['base_info']['doc_count']}") + print(f"唯一源数量: {stats['base_info']['unique_sources_count']}") + print(f"使用的嵌入模型: {stats['base_info']['embedding_model']}") + print("---") +``` + +## API 服务接口 + +通过 `elasticsearch_service.py` 提供的 FastAPI 服务，可使用 REST API 访问上述所有功能。 + +### 服务启动 + +```bash +python -m nexent.service.elasticsearch_service +``` + +服务默认在 `http://localhost:8000` 运行。 + +### API 接口文档 + +#### 健康检查 + +- **GET** `/health`: 检查 API 和 Elasticsearch 连接状态 + - 返回示例: `{"status": "healthy", "elasticsearch": "connected", "indices_count": 5}` + +#### 索引管理 +- **POST** `/indices/{index_name}`: 创建索引 + - 参数: + - `index_name`: 索引名称 (路径参数) + - `embedding_dim`: 向量化维度 (查询参数，可选) + - 返回示例: `{"status": "success", "message": "Index my_documents created successfully"}` + +- **DELETE** `/indices/{index_name}`: 删除索引 + - 参数: `index_name`: 索引名称 (路径参数) + - 返回示例: `{"status": "success", "message": "Index my_documents deleted successfully"}` + +- **GET** `/indices`: 列出所有索引，可选包含详细统计信息 + - 参数: + - `pattern`: 索引名称匹配模式 (查询参数，默认为 "*") + - `include_stats`: 是否包含索引统计信息 (查询参数，默认为 false) + - 基本返回示例: `{"indices": ["index1", "index2"], "count": 2}` + - 包含统计信息的返回示例: + ```json + { + "indices": ["index1", "index2"], + "count": 2, + "indices_info": [ + { + "name": "index1", + "stats": { + "base_info": { + "doc_count": 100, + "unique_sources_count": 10, + "store_size": "1.2 MB", + "process_source": "Web", + "embedding_model": "jina-embeddings-v2-base-en", + "creation_date": "2023-06-01 12:00:00", + "update_date": "2023-06-02 15:30:00" + }, + "search_performance": { + "total_search_count": 150, + "hit_count": 120 + } + } + }, + { + "name": "index2", + "stats": { "..." } + } + ] + } + ``` + +- **GET** `/indices/{index_name}/info`: 获取索引的综合信息 + - 参数: + - `index_name`: 索引名称 (路径参数) + - `include_files`: 是否包含文件列表信息 (查询参数，默认为 true) + - `include_chunks`: 是否包含文本块信息 (查询参数，默认为 false) + - 返回综合信息，包括基本信息、搜索性能、字段列表、文件列表和文本块列表 + - 返回示例: + ```json + { + "base_info": { + "doc_count": 100, + "unique_sources_count": 10, + "store_size": "1.2 MB", + "process_source": "Web", + "embedding_model": "jina-embeddings-v2-base-en", + "embedding_dim": 1024, + "creation_date": "2023-06-01 12:00:00", + "update_date": "2023-06-02 15:30:00" + }, + "search_performance": { + "total_search_count": 150, + "hit_count": 120 + }, + "fields": ["id", "title", "content", "embedding", "embedding_model_name", "file_size", "create_time", "..."], + "files": [ + { + "path_or_url": "https://example.com/doc1", + "file": "文件1.txt", + "file_size": 1024, + "create_time": "2023-06-01T10:30:00", + "chunks_count": 6, + "status": "PROCESSING", + "chunks": [] + }, + { + "path_or_url": "https://example.com/doc2", + "file": "文件2.txt", + "file_size": 2048, + "create_time": "2023-06-01T11:45:00", + "chunks_count": 10, + "status": "WAITING", + "chunks": [] + }, + { + "path_or_url": "https://example.com/doc3", + "file": "文件3.txt", + "file_size": 0, + "create_time": "2023-06-01T12:00:00", + "chunks_count": 0, + "status": "COMPLETED", + "chunks": [ + { + "id": "task-0", + "title": "title-0", + "content": "content-0", + "create_time": "2023-06-01T12:30:00" + }, + { + "id": "task-1", + "title": "title-1", + "content": "content-1", + "create_time": "2023-06-01T12:30:00" + } + ], + } + ] + } + ``` + - 文件状态说明： + - `WAITING`: 文件正在等待处理 + - `PROCESSING`: 文件正在被处理 + - `FORWARDING`: 文件正在被转发到向量知识库服务 + - `COMPLETED`: 文件已完成处理并成功入库 + - `FAILED`: 文件处理失败 + - 文件列表包含： + - 已存在于ES中的文件（状态为 COMPLETED 或活跃任务中的状态） + - 正在数据清洗服务中处理但尚未进入ES的文件（状态为 WAITING/PROCESSING/FORWARDING/FAILED） + +#### 文档操作 + +- **POST** `/indices/{index_name}/documents`: 索引文档 + - 参数: + - `index_name`: 索引名称 (路径参数) + - `data`: 包含任务ID和文档的请求体 (IndexingRequest) + - `embedding_model_name`: 指定要使用的嵌入模型名称 (查询参数，可选) + - IndexingRequest 格式示例: + ```json + { + "task_id": "task-123", + "index_name": "my_documents", + "results": [ + { + "metadata": { + "title": "文档标题", + "filename": "文件名.txt", + "languages": ["zh"], + "author": "作者", + "file_size": 1024, + "creation_date": "2023-06-01T10:30:00" + }, + "source": "https://example.com/doc1", + "source_type": "url", + "text": "文档内容" + } + ], + "embedding_dim": 1024 + } + ``` + - 返回示例: + ```json + { + "success": true, + "message": "Successfully indexed 1 documents", + "total_indexed": 1, + "total_submitted": 1 + } + ``` + +- **DELETE** `/indices/{index_name}/documents`: 删除文档 + - 参数: + - `index_name`: 索引名称 (路径参数) + - `path_or_url`: 文档路径或URL (查询参数) + - 返回示例: `{"status": "success", "deleted_count": 1}` + +#### 搜索操作 + +- **POST** `/indices/search/accurate`: 精确文本搜索 + - 请求体 (SearchRequest): + ```json + { + "index_names": ["index1", "index2"], + "query": "搜索关键词", + "top_k": 5 + } + ``` + - 返回格式: + ```json + { + "results": [ + { + "id": "doc1", + "title": "文档标题", + "file": "文件名.txt", + "path_or_url": "https://example.com/doc1", + "content": "文档内容", + "process_source": "Web", + "embedding_model_name": "jina-embeddings-v2-base-en", + "file_size": 1024, + "create_time": "2023-06-01T10:30:00", + "score": 0.95, + "index": "index1" + }, + { + "id": "doc2", + "title": "文档标题", + "file": "文件名.txt", + "path_or_url": "https://example.com/doc2", + "content": "文档内容", + "process_source": "Web", + "embedding_model_name": "jina-embeddings-v2-base-en", + "file_size": 1024, + "create_time": "2023-06-01T10:30:00", + "score": 0.85, + "index": "index2" + } + ], + "total": 2, + "query_time_ms": 25.4 + } + ``` + +- **POST** `/indices/search/semantic`: 语义向量搜索 + - 请求体格式与精确搜索相同 (SearchRequest) + - 返回格式与精确搜索相同，但基于语义相似度评分 + +- **POST** `/indices/search/hybrid`: 混合搜索 + - 请求体 (HybridSearchRequest): + ```json + { + "index_names": ["index1", "index2"], + "query": "搜索关键词", + "top_k": 5, + "weight_accurate": 0.3 + } + ``` + - 返回格式与精确搜索相同，但包含详细的得分信息： + ```json + { + "results": [ + { + "id": "doc1", + "title": "文档标题", + "file": "文件名.txt", + "path_or_url": "https://example.com/doc1", + "content": "文档内容", + "process_source": "Web", + "embedding_model_name": "jina-embeddings-v2-base-en", + "file_size": 1024, + "create_time": "2023-06-01T10:30:00", + "score": 0.798, + "index": "index1", + "score_details": { + "accurate": 0.80, + "semantic": 0.90 + } + }, + { + "id": "doc2", + "title": "文档标题", + "file": "文件名.txt", + "path_or_url": "https://example.com/doc2", + "content": "文档内容", + "process_source": "Web", + "embedding_model_name": "jina-embeddings-v2-base-en", + "file_size": 1024, + "create_time": "2023-06-01T10:30:00", + "score": 0.756, + "index": "index1", + "score_details": { + "accurate": 0.60, + "semantic": 0.90 + } + } + ], + "total": 2, + "query_time_ms": 35.2 + } + ``` + +### API 使用示例 + +#### 使用 curl 请求示例 + +```bash +# 健康检查 +curl -X GET "http://localhost:8000/health" + +# 列出所有索引（包含统计信息） +curl -X GET "http://localhost:8000/indices?include_stats=true" + +# 获取索引详细信息（包含文本块列表） +curl -X GET "http://localhost:8000/indices/my_documents/info?include_chunks=true" + +# 精确搜索（支持多索引搜索） +curl -X POST "http://localhost:8000/indices/search/accurate" \ + -H "Content-Type: application/json" \ + -d '{ + "index_names": ["my_documents", "other_index"], + "query": "示例查询", + "top_k": 3 + }' + +# 语义搜索（支持多索引搜索） +curl -X POST "http://localhost:8000/indices/search/semantic" \ + -H "Content-Type: application/json" \ + -d '{ + "index_names": ["my_documents", "other_index"], + "query": "相似含义查询", + "top_k": 3 + }' + +# 混合搜索（支持多索引搜索） +curl -X POST "http://localhost:8000/indices/search/hybrid" \ + -H "Content-Type: application/json" \ + -d '{ + "index_names": ["my_documents", "other_index"], + "query": "示例查询", + "top_k": 3, + "weight_accurate": 0.3 + }' + +# 删除文档 +curl -X DELETE "http://localhost:8000/indices/my_documents/documents?path_or_url=https://example.com/doc1" + +# 创建索引 +curl -X POST "http://localhost:8000/indices/my_documents" + +# 删除索引 +curl -X DELETE "http://localhost:8000/indices/my_documents" +``` + +#### 使用 Python requests 示例 + +```python +import requests +import json +import time + +BASE_URL = "http://localhost:8000" + +# 当前时间，ISO格式 +current_time = time.strftime("%Y-%m-%dT%H:%M:%S", time.gmtime()) + +# 准备 IndexingRequest +indexing_request = { + "task_id": f"task-{int(time.time())}", + "index_name": "my_documents", + "results": [ + { + "metadata": { + "title": "示例文档", + "filename": "example.txt", + "language": "zh", + "author": "作者", + "file_size": 1024, + "creation_date": current_time + }, + "source": "https://example.com/doc1", + "text": "这是一个示例文档" + } + ], + "embedding_dim": 1024 +} + +# 索引文档 +response = requests.post( + f"{BASE_URL}/indices/my_documents/documents", + json=indexing_request, + params={ + "embedding_model_name": "jina-embeddings-v2-base-en" # 可选参数：指定嵌入模型 + } +) +print(response.json()) + +# 获取索引信息，包含文件列表 +response = requests.get( + f"{BASE_URL}/indices/my_documents/info", + params={"include_files": True} +) +print(json.dumps(response.json(), indent=2, ensure_ascii=False)) + +# 获取所有索引信息，包含统计 +response = requests.get( + f"{BASE_URL}/indices", + params={"include_stats": True} +) +print(json.dumps(response.json(), indent=2, ensure_ascii=False)) + +# 精确搜索 +response = requests.post( + f"{BASE_URL}/indices/search/accurate", + json={ + "index_names": ["my_documents", "other_index"], + "query": "示例内容", + "top_k": 3 + } +) +print(json.dumps(response.json(), indent=2, ensure_ascii=False)) + +# 语义搜索 +response = requests.post( + f"{BASE_URL}/indices/search/semantic", + json={ + "index_names": ["my_documents", "other_index"], + "query": "示例内容", + "top_k": 3 + } +) +print(json.dumps(response.json(), indent=2, ensure_ascii=False)) + +# 混合搜索 +response = requests.post( + f"{BASE_URL}/indices/search/hybrid", + json={ + "index_names": ["my_documents", "other_index"], + "query": "示例内容", + "top_k": 3, + "weight_accurate": 0.3 + } +) +print(json.dumps(response.json(), indent=2, ensure_ascii=False)) +``` + +## 完整示例 + +查看 ElasticSearchCore 类的 main 函数，了解完整功能演示: + +```python +# 初始化 ElasticSearchCore +es_core = ElasticSearchCore() + +# 获取或创建测试知识库 +index_name = "sample_articles" + +# 列出所有用户索引 +user_indices = es_core.get_user_indices() +for idx in user_indices: + print(f" - {idx}") + +# 执行搜索 +if index_name in user_indices: + # 精确搜索 + query = "Doctor" + accurate_results = es_core.accurate_search(index_name, query, top_k=2) + + # 语义搜索 + query = "medical professionals in London" + semantic_results = es_core.semantic_search(index_name, query, top_k=2) + + # 混合搜索 + query = "medical professionals in London" + semantic_results = es_core.hybrid_search(index_name, query, top_k=2, weight_accurate=0.5) + + # 获取索引统计信息 + stats = es_core.get_index_stats(index_name) + fields = es_core.get_index_mapping(index_name) + unique_sources = es_core.get_unique_sources_count(index_name) +``` + +## 许可证 + +该项目根据 MIT 许可证授权 - 详情请参阅 LICENSE 文件。 diff --git a/doc/docs/zh/security.md b/doc/docs/zh/security.md new file mode 100644 index 000000000..fe1830886 --- /dev/null +++ b/doc/docs/zh/security.md @@ -0,0 +1,29 @@ +# 安全政策 + +**请勿通过公开的 GitHub issues、讨论或其他公开渠道报告安全漏洞。** + +相反，请通过联系我们的安全团队负责任地披露： +📧 [chenshuangrui@gmail.com](mailto:chenshuangrui@gmail.com) + +## 需要包含的内容： +- 漏洞的详细描述 +- 重现步骤 +- 潜在影响评估 +- 建议的修复或缓解措施（如果已知） + +## 我们的响应流程： +- **48小时内**确认收到 +- **5个工作日内**进行初步评估 +- 定期更新修复进度 +- 与报告者协调公开披露时间表 + +## 安全更新 +关键安全补丁会在可用时立即发布。所有安全相关更新将在发布说明中标记为 **[SECURITY]**。 + +## 致谢 +虽然我们目前没有正式的漏洞赏金计划，但我们感谢负责任的披露： +- 在我们的安全名人堂中列出贡献者 +- 提供书面推荐（根据要求） +- 在发布说明中公开致谢（经许可） + +**注意：** 此政策可能会定期更新。最后修订：2025年1月 \ No newline at end of file diff --git a/doc/docs/zh/testing/backend.md b/doc/docs/zh/testing/backend.md new file mode 100644 index 000000000..f7831ef76 --- /dev/null +++ b/doc/docs/zh/testing/backend.md @@ -0,0 +1,372 @@ +# 后端测试 + +本指南涵盖了 Nexent 中使用的全面后端测试框架，包括 API 测试、服务层测试和工具函数测试。 + +## 测试结构 + +后端测试按以下结构组织： + +``` +test/backend/ +├── app/ # API 端点测试 +│ ├── test_agent_app.py +│ ├── test_base_app.py +│ ├── test_config_sync_app.py +│ ├── test_conversation_management_app.py +│ ├── test_data_process_app.py +│ ├── test_elasticsearch_app.py +│ ├── test_file_management_app.py +│ ├── test_image_app.py +│ ├── test_knowledge_app.py +│ ├── test_knowledge_summary_app.py +│ ├── test_me_model_managment_app.py +│ ├── test_model_managment_app.py +│ ├── test_prompt_app.py +│ └── test_remote_mcp_app.py +├── services/ # 服务层测试 +│ ├── test_agent_service.py +│ ├── test_conversation_management_service.py +│ ├── test_data_process_service.py +│ ├── test_elasticsearch_service.py +│ ├── test_file_management_service.py +│ ├── test_image_service.py +│ ├── test_knowledge_service.py +│ ├── test_knowledge_summary_service.py +│ ├── test_model_management_service.py +│ ├── test_prompt_service.py +│ └── test_remote_mcp_service.py +├── utils/ # 工具函数测试 +│ ├── test_langchain_utils.py +│ └── test_prompt_template_utils.py +└── run_all_test.py # 后端测试运行器 +``` + +## 运行后端测试 + +### 完整的后端测试套件 + +```bash +# 从项目根目录 +python test/backend/run_all_test.py + +# 从 test/backend 目录 +cd test/backend +python run_all_test.py +``` + +### 单个测试类别 + +```bash +# 运行所有 API 测试 +python -m pytest test/backend/app/ -v + +# 运行所有服务测试 +python -m pytest test/backend/services/ -v + +# 运行所有工具测试 +python -m pytest test/backend/utils/ -v +``` + +### 特定测试文件 + +```bash +# 运行特定 API 测试 +python -m pytest test/backend/app/test_agent_app.py -v + +# 运行特定服务测试 +python -m pytest test/backend/services/test_agent_service.py -v + +# 运行特定工具测试 +python -m pytest test/backend/utils/test_langchain_utils.py -v +``` + +## API 测试 + +API 测试使用 FastAPI 的 TestClient 来模拟 HTTP 请求，而无需运行实际服务器。 + +### 测试设置模式 + +```python +import os +import sys +from unittest.mock import patch, MagicMock +from fastapi.testclient import TestClient +from fastapi import FastAPI + +# 动态确定后端路径 +current_dir = os.path.dirname(os.path.abspath(__file__)) +backend_dir = os.path.abspath(os.path.join(current_dir, "../../../backend")) +sys.path.append(backend_dir) + +# 在导入模块之前设置依赖项补丁 +patches = [ + patch('botocore.client.BaseClient._make_api_call', return_value={}), + patch('backend.database.client.MinioClient', MagicMock()), + patch('backend.database.client.db_client', MagicMock()), + patch('backend.utils.auth_utils.get_current_user_id', + MagicMock(return_value=('test_user', 'test_tenant'))), + patch('httpx.AsyncClient', MagicMock()) +] + +# 启动所有补丁 +for p in patches: + p.start() + +# 应用补丁后导入模块 +from backend.apps.agent_app import router + +# 创建测试应用 +app = FastAPI() +app.include_router(router) +client = TestClient(app) +``` + +### API 测试示例 + +```python +class TestAgentApp(unittest.TestCase): + + def setUp(self): + # 设置测试客户端和通用模拟 + pass + + def test_create_agent_success(self): + """测试成功的智能体创建""" + # 设置 + agent_data = { + "name": "Test Agent", + "description": "A test agent", + "system_prompt": "You are a test agent" + } + + # 执行 + response = client.post("/agents", json=agent_data) + + # 断言 + self.assertEqual(response.status_code, 200) + self.assertIn("id", response.json()) + self.assertEqual(response.json()["name"], "Test Agent") + + def test_create_agent_invalid_data(self): + """测试使用无效数据的智能体创建""" + # 设置 + invalid_data = {"name": ""} # 缺少必需字段 + + # 执行 + response = client.post("/agents", json=invalid_data) + + # 断言 + self.assertEqual(response.status_code, 422) # 验证错误 +``` + +## 服务层测试 + +服务层测试专注于业务逻辑和数据处理，无需 HTTP 开销。 + +### 服务测试模式 + +```python +class TestAgentService(unittest.TestCase): + + @patch("backend.database.agent_db.save_agent") + @patch("backend.utils.auth_utils.get_current_user_id") + async def test_create_agent_success(self, mock_get_user, mock_save_agent): + # 设置 + mock_get_user.return_value = ("user123", "tenant456") + mock_save_agent.return_value = {"id": 1, "name": "Test Agent"} + + # 执行 + result = await create_agent( + name="Test Agent", + description="A test agent", + system_prompt="You are a test agent" + ) + + # 断言 + mock_save_agent.assert_called_once() + self.assertEqual(result["name"], "Test Agent") + self.assertIn("id", result) +``` + +### 模拟数据库操作 + +```python +@patch("backend.database.agent_db.query_agent_by_id") +@patch("backend.database.agent_db.update_agent") +async def test_update_agent_success(self, mock_update, mock_query): + # 设置 + mock_query.return_value = {"id": 1, "name": "Old Name"} + mock_update.return_value = {"id": 1, "name": "New Name"} + + # 执行 + result = await update_agent(agent_id=1, name="New Name") + + # 断言 + mock_update.assert_called_once_with(agent_id=1, name="New Name") + self.assertEqual(result["name"], "New Name") +``` + +## 工具函数测试 + +工具函数在隔离环境中测试，使用模拟的依赖项。 + +### 工具测试示例 + +```python +class TestLangchainUtils(unittest.TestCase): + + @patch("langchain.llms.openai.OpenAI") + def test_create_llm_instance(self, mock_openai): + # 设置 + mock_openai.return_value = MagicMock() + + # 执行 + llm = create_llm_instance(model_name="gpt-3.5-turbo") + + # 断言 + mock_openai.assert_called_once() + self.assertIsNotNone(llm) +``` + +## 测试异步代码 + +后端测试处理同步和异步代码： + +### 异步测试模式 + +```python +class TestAsyncService(unittest.TestCase): + + @patch("backend.database.agent_db.async_query") + async def test_async_operation(self, mock_async_query): + # 设置 + mock_async_query.return_value = {"result": "success"} + + # 执行 + result = await async_operation() + + # 断言 + self.assertEqual(result["result"], "success") + mock_async_query.assert_called_once() +``` + +## 错误处理测试 + +全面测试错误处理： + +```python +def test_api_error_handling(self): + """测试 API 错误响应""" + # 设置 - 模拟服务抛出异常 + with patch('backend.services.agent_service.create_agent') as mock_service: + mock_service.side_effect = Exception("Database error") + + # 执行 + response = client.post("/agents", json={"name": "Test"}) + + # 断言 + self.assertEqual(response.status_code, 500) + self.assertIn("error", response.json()) +``` + +## 身份验证和授权测试 + +彻底测试安全相关功能： + +```python +def test_authentication_required(self): + """测试端点需要身份验证""" + # 执行 - 没有身份验证头 + response = client.get("/agents") + + # 断言 + self.assertEqual(response.status_code, 401) + +def test_tenant_isolation(self): + """测试用户只能访问其租户的数据""" + # 设置 - 模拟身份验证返回不同租户 + with patch('backend.utils.auth_utils.get_current_user_id') as mock_auth: + mock_auth.return_value = ("user1", "tenant1") + + # 执行 + response = client.get("/agents") + + # 断言 - 验证应用了租户过滤 + # 这将检查服务层是否按租户过滤 +``` + +## 覆盖率分析 + +后端测试生成详细的覆盖率报告： + +### 覆盖率命令 + +```bash +# 生成覆盖率报告 +python -m pytest test/backend/ --cov=backend --cov-report=html --cov-report=xml + +# 在终端中查看覆盖率 +python -m pytest test/backend/ --cov=backend --cov-report=term-missing +``` + +### 覆盖率目标 + +- **API 端点**：90%+ 覆盖率 +- **服务层**：85%+ 覆盖率 +- **工具函数**：80%+ 覆盖率 +- **错误处理**：关键路径 100% 覆盖率 + +## 测试数据管理 + +### 固定装置和测试数据 + +```python +class TestWithFixtures(unittest.TestCase): + + def setUp(self): + """设置测试数据和模拟""" + self.test_agent = { + "id": 1, + "name": "Test Agent", + "description": "A test agent", + "system_prompt": "You are a test agent" + } + + self.test_user = ("user123", "tenant456") + + def tearDown(self): + """测试后清理""" + # 如果需要，重置任何全局状态 + pass +``` + +## 性能测试 + +后端测试包括性能考虑： + +```python +def test_api_response_time(self): + """测试 API 响应时间在可接受的时间限制内""" + import time + + start_time = time.time() + response = client.get("/agents") + end_time = time.time() + + # 断言响应时间小于 100ms + self.assertLess(end_time - start_time, 0.1) + self.assertEqual(response.status_code, 200) +``` + +## 后端测试最佳实践 + +1. **模拟外部依赖**：始终模拟数据库、外部 API 和服务 +2. **测试成功和失败**：覆盖所有可能的代码路径 +3. **使用描述性测试名称**：清楚说明每个测试验证的内容 +4. **保持测试独立**：每个测试都应该能够独立运行 +5. **测试边缘情况**：包括边界条件和错误场景 +6. **维护测试数据**：使用一致、真实的测试数据 +7. **记录复杂测试**：为复杂的测试场景添加注释 +8. **定期覆盖率审查**：监控并随时间改进覆盖率 + +这个全面的后端测试框架确保所有后端功能在部署前都经过彻底验证，保持高代码质量和可靠性。 \ No newline at end of file diff --git a/doc/docs/zh/testing/overview.md b/doc/docs/zh/testing/overview.md new file mode 100644 index 000000000..1321b7f30 --- /dev/null +++ b/doc/docs/zh/testing/overview.md @@ -0,0 +1,240 @@ +# 测试概览 + +Nexent 提供了一个全面的测试框架，确保所有组件的代码质量和可靠性。本指南涵盖了项目中使用的测试策略、工具和最佳实践。 + +## 测试理念 + +我们的测试方法建立在四个核心原则之上： + +1. **隔离单元**：模拟所有外部依赖 +2. **控制环境**：设置精确的测试条件 +3. **测试接口**：专注于输入和输出 +4. **验证行为**：检查结果和交互 + +这确保了测试的可靠性、快速性，并且不会影响真实的系统或数据。 + +## 测试框架 + +项目使用多种测试框架的组合： + +- **unittest**：Python 标准单元测试框架，用于测试组织和断言 +- **unittest.mock**：用于模拟依赖项和隔离组件 +- **TestClient** from FastAPI：用于测试 API 端点而无需运行实际服务器 +- **pytest**：用于高级测试发现和执行 +- **coverage**：用于代码覆盖率分析和报告 + +## 测试结构 + +``` +test/ +├── backend/ # 后端测试 +│ ├── app/ # API 端点测试 +│ ├── services/ # 服务层测试 +│ └── utils/ # 工具函数测试 +├── frontend/ # 前端测试（未来） +├── integration/ # 集成测试（未来） +└── run_all_tests.py # 主测试运行器 +``` + +## 主要特性 + +- 🔍 **自动发现测试文件** - 自动查找所有 `test_*.py` 文件 +- 📊 **覆盖率报告** - 生成控制台、HTML 和 XML 格式的覆盖率报告 +- 🔧 **自动安装依赖** - 自动安装所需的包（如果需要） +- ✅ **详细输出** - 显示每个测试的运行状态和结果 +- 🚫 **完全隔离** - 从不接触真实的外部服务 +- ⚡ **快速执行** - 无网络延迟或外部服务处理时间 + +## 运行测试 + +### 快速开始 + +```bash +# 运行所有测试并生成覆盖率报告 +cd test +python run_all_tests.py +``` + +### 后端测试 + +```bash +# 仅运行后端测试 +python test/backend/run_all_test.py +``` + +### 单个测试文件 + +```bash +# 运行特定测试文件 +python -m pytest test/backend/services/test_agent_service.py -v +``` + +## 输出文件 + +测试完成后，您将找到： + +- `coverage_html/` - 详细的 HTML 格式覆盖率报告 +- `coverage.xml` - XML 格式覆盖率报告（用于 CI/CD） +- `.coverage` - 覆盖率数据文件 +- 包含详细测试结果和覆盖率统计的控制台输出 + +## 测试策略 + +### 1. 依赖隔离 + +在导入之前模拟外部模块以避免真实连接： + +- 模拟数据库连接 +- 模拟 ElasticSearch 和其他外部服务 +- 测试期间不执行实际的数据库操作 +- 模拟 HTTP 客户端以防止网络调用 + +### 2. 基于模拟的测试 + +- 使用 FastAPI 的 TestClient 模拟 HTTP 请求 +- 使用模拟对象拦截外部服务调用 +- 不发生实际的网络连接或端口绑定 +- 模拟身份验证函数以返回可预测的测试值 + +### 3. 测试组织 + +- 测试组织为继承自 `unittest.TestCase` 的类 +- 每个 API 端点或函数都有多个测试用例（成功、失败、异常场景） +- 应用全面的补丁来隔离被测试的代码 +- 测试遵循清晰的设置-执行-断言模式 + +### 4. API 测试 + +- 测试 API 端点的正确响应代码、负载结构和错误处理 +- 涵盖同步和异步端点 +- 通过专门的测试用例测试流式响应 +- 彻底测试身份验证和授权 + +## 模块补丁技术 + +测试套件中使用的关键技术是在导入之前修补模块。这可以防止任何真实的外部服务连接。 + +### 示例：导入前补丁 + +```python +# 动态确定后端路径 +current_dir = os.path.dirname(os.path.abspath(__file__)) +backend_dir = os.path.abspath(os.path.join(current_dir, "../../../backend")) +sys.path.append(backend_dir) + +# 在导入模块之前设置依赖项补丁 +patches = [ + patch('botocore.client.BaseClient._make_api_call', return_value={}), + patch('backend.database.client.MinioClient', MagicMock()), + patch('backend.database.client.db_client', MagicMock()), + patch('backend.utils.auth_utils.get_current_user_id', + MagicMock(return_value=('test_user', 'test_tenant'))), + patch('httpx.AsyncClient', MagicMock()) +] + +# 启动所有补丁 +for p in patches: + p.start() + +# 现在在应用所有补丁后导入模块 +from backend.apps.file_management_app import router +``` + +### 这种方法的优势 + +1. **完全隔离**：从不接触真实的外部服务 +2. **无副作用**：测试无法修改生产数据库或服务 +3. **更快的测试**：无网络延迟或外部服务处理时间 +4. **可预测的结果**：测试使用受控的模拟数据以获得一致的结果 +5. **无端口绑定**：FastAPI 应用程序从不绑定到真实的网络端口 + +## 测试示例 + +以下是测试结构化的详细示例： + +```python +@patch("utils.auth_utils.get_current_user_id") +@patch("database.agent_db.query_all_tools") +async def test_list_tools_api_success(self, mock_query_all_tools, mock_get_current_user_id): + # 设置 + mock_get_current_user_id.return_value = ("user123", "tenant456") + expected_tools = [{"id": 1, "name": "Tool1"}, {"id": 2, "name": "Tool2"}] + mock_query_all_tools.return_value = expected_tools + + # 执行 + result = await list_tools_api(authorization="Bearer fake_token") + + # 断言 + mock_get_current_user_id.assert_called_once_with("Bearer fake_token") + mock_query_all_tools.assert_called_once_with(tenant_id="tenant456") + self.assertEqual(result, expected_tools) +``` + +## 覆盖率报告 + +测试套件生成全面的覆盖率报告： + +- **控制台输出**：逐行覆盖率详细信息 +- **HTML 报告**：`coverage_html/` 中的详细覆盖率报告 +- **XML 报告**：用于 CI/CD 集成的覆盖率数据 +- **摘要统计**：总体覆盖率百分比和缺失行 + +## 示例输出 + +``` +Nexent Community - Unit Test Runner +============================================================ +发现的测试文件： +---------------------------------------- + • backend/services/test_agent_service.py + • backend/services/test_conversation_management_service.py + • backend/services/test_knowledge_summary_service.py + +总计：3 个测试文件 + +============================================================ +运行单元测试和覆盖率 +============================================================ + +test_get_enable_tool_id_by_agent_id ... ok +test_save_message_with_string_content ... ok +test_load_knowledge_prompts ... ok +... + +============================================================ +覆盖率报告 +============================================================ +名称 Stmts Miss Cover Missing +-------------------------------------------------------------------------------- +backend/services/agent_service.py 120 15 88% 45-50, 78-82 +backend/services/conversation_management_service.py 180 25 86% 123-128, 156-162 +backend/services/knowledge_summary_service.py 45 8 82% 35-42 +-------------------------------------------------------------------------------- +总计 345 48 86% + +HTML 覆盖率报告生成在：test/coverage_html +XML 覆盖率报告生成：test/coverage.xml + +============================================================ +✅ 所有测试通过！ +``` + +## 依赖项 + +测试运行器会自动安装所需的包（如果尚未可用）： + +- `pytest-cov` - 用于 pytest 覆盖率集成 +- `coverage` - 用于代码覆盖率分析 +- `pytest` - 用于高级测试发现和执行 + +## 最佳实践 + +1. **始终在导入模块之前模拟外部依赖** +2. **使用描述性的测试名称**来解释正在测试的内容 +3. **遵循设置-执行-断言模式**以获得清晰的测试结构 +4. **测试成功和失败场景**以获得全面的覆盖率 +5. **保持测试独立** - 每个测试都应该能够独立运行 +6. **使用有意义的模拟数据**来代表真实世界的场景 +7. **用清晰的注释记录复杂的测试场景** + +这个测试框架确保所有代码更改在部署前都经过彻底验证，在整个 Nexent 平台中保持高代码质量和可靠性。 \ No newline at end of file diff --git a/doc/package-lock.json b/doc/package-lock.json new file mode 100644 index 000000000..0e5f229df --- /dev/null +++ b/doc/package-lock.json @@ -0,0 +1,2435 @@ +{ + "name": "doc", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "devDependencies": { + "vitepress": "^1.6.3" + } + }, + "node_modules/@algolia/abtesting": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/@algolia/abtesting/-/abtesting-1.1.0.tgz", + "integrity": "sha512-sEyWjw28a/9iluA37KLGu8vjxEIlb60uxznfTUmXImy7H5NvbpSO6yYgmgH5KiD7j+zTUUihiST0jEP12IoXow==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/client-common": "5.35.0", + "@algolia/requester-browser-xhr": "5.35.0", + "@algolia/requester-fetch": "5.35.0", + "@algolia/requester-node-http": "5.35.0" + }, + "engines": { + "node": ">= 14.0.0" + } + }, + "node_modules/@algolia/autocomplete-core": { + "version": "1.17.7", + "resolved": "https://registry.npmjs.org/@algolia/autocomplete-core/-/autocomplete-core-1.17.7.tgz", + "integrity": "sha512-BjiPOW6ks90UKl7TwMv7oNQMnzU+t/wk9mgIDi6b1tXpUek7MW0lbNOUHpvam9pe3lVCf4xPFT+lK7s+e+fs7Q==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/autocomplete-plugin-algolia-insights": "1.17.7", + "@algolia/autocomplete-shared": "1.17.7" + } + }, + "node_modules/@algolia/autocomplete-plugin-algolia-insights": { + "version": "1.17.7", + "resolved": "https://registry.npmjs.org/@algolia/autocomplete-plugin-algolia-insights/-/autocomplete-plugin-algolia-insights-1.17.7.tgz", + "integrity": "sha512-Jca5Ude6yUOuyzjnz57og7Et3aXjbwCSDf/8onLHSQgw1qW3ALl9mrMWaXb5FmPVkV3EtkD2F/+NkT6VHyPu9A==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/autocomplete-shared": "1.17.7" + }, + "peerDependencies": { + "search-insights": ">= 1 < 3" + } + }, + "node_modules/@algolia/autocomplete-preset-algolia": { + "version": "1.17.7", + "resolved": "https://registry.npmjs.org/@algolia/autocomplete-preset-algolia/-/autocomplete-preset-algolia-1.17.7.tgz", + "integrity": "sha512-ggOQ950+nwbWROq2MOCIL71RE0DdQZsceqrg32UqnhDz8FlO9rL8ONHNsI2R1MH0tkgVIDKI/D0sMiUchsFdWA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/autocomplete-shared": "1.17.7" + }, + "peerDependencies": { + "@algolia/client-search": ">= 4.9.1 < 6", + "algoliasearch": ">= 4.9.1 < 6" + } + }, + "node_modules/@algolia/autocomplete-shared": { + "version": "1.17.7", + "resolved": "https://registry.npmjs.org/@algolia/autocomplete-shared/-/autocomplete-shared-1.17.7.tgz", + "integrity": "sha512-o/1Vurr42U/qskRSuhBH+VKxMvkkUVTLU6WZQr+L5lGZZLYWyhdzWjW0iGXY7EkwRTjBqvN2EsR81yCTGV/kmg==", + "dev": true, + "license": "MIT", + "peerDependencies": { + "@algolia/client-search": ">= 4.9.1 < 6", + "algoliasearch": ">= 4.9.1 < 6" + } + }, + "node_modules/@algolia/client-abtesting": { + "version": "5.35.0", + "resolved": "https://registry.npmjs.org/@algolia/client-abtesting/-/client-abtesting-5.35.0.tgz", + "integrity": "sha512-uUdHxbfHdoppDVflCHMxRlj49/IllPwwQ2cQ8DLC4LXr3kY96AHBpW0dMyi6ygkn2MtFCc6BxXCzr668ZRhLBQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/client-common": "5.35.0", + "@algolia/requester-browser-xhr": "5.35.0", + "@algolia/requester-fetch": "5.35.0", + "@algolia/requester-node-http": "5.35.0" + }, + "engines": { + "node": ">= 14.0.0" + } + }, + "node_modules/@algolia/client-analytics": { + "version": "5.35.0", + "resolved": "https://registry.npmjs.org/@algolia/client-analytics/-/client-analytics-5.35.0.tgz", + "integrity": "sha512-SunAgwa9CamLcRCPnPHx1V2uxdQwJGqb1crYrRWktWUdld0+B2KyakNEeVn5lln4VyeNtW17Ia7V7qBWyM/Skw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/client-common": "5.35.0", + "@algolia/requester-browser-xhr": "5.35.0", + "@algolia/requester-fetch": "5.35.0", + "@algolia/requester-node-http": "5.35.0" + }, + "engines": { + "node": ">= 14.0.0" + } + }, + "node_modules/@algolia/client-common": { + "version": "5.35.0", + "resolved": "https://registry.npmjs.org/@algolia/client-common/-/client-common-5.35.0.tgz", + "integrity": "sha512-ipE0IuvHu/bg7TjT2s+187kz/E3h5ssfTtjpg1LbWMgxlgiaZIgTTbyynM7NfpSJSKsgQvCQxWjGUO51WSCu7w==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 14.0.0" + } + }, + "node_modules/@algolia/client-insights": { + "version": "5.35.0", + "resolved": "https://registry.npmjs.org/@algolia/client-insights/-/client-insights-5.35.0.tgz", + "integrity": "sha512-UNbCXcBpqtzUucxExwTSfAe8gknAJ485NfPN6o1ziHm6nnxx97piIbcBQ3edw823Tej2Wxu1C0xBY06KgeZ7gA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/client-common": "5.35.0", + "@algolia/requester-browser-xhr": "5.35.0", + "@algolia/requester-fetch": "5.35.0", + "@algolia/requester-node-http": "5.35.0" + }, + "engines": { + "node": ">= 14.0.0" + } + }, + "node_modules/@algolia/client-personalization": { + "version": "5.35.0", + "resolved": "https://registry.npmjs.org/@algolia/client-personalization/-/client-personalization-5.35.0.tgz", + "integrity": "sha512-/KWjttZ6UCStt4QnWoDAJ12cKlQ+fkpMtyPmBgSS2WThJQdSV/4UWcqCUqGH7YLbwlj3JjNirCu3Y7uRTClxvA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/client-common": "5.35.0", + "@algolia/requester-browser-xhr": "5.35.0", + "@algolia/requester-fetch": "5.35.0", + "@algolia/requester-node-http": "5.35.0" + }, + "engines": { + "node": ">= 14.0.0" + } + }, + "node_modules/@algolia/client-query-suggestions": { + "version": "5.35.0", + "resolved": "https://registry.npmjs.org/@algolia/client-query-suggestions/-/client-query-suggestions-5.35.0.tgz", + "integrity": "sha512-8oCuJCFf/71IYyvQQC+iu4kgViTODbXDk3m7yMctEncRSRV+u2RtDVlpGGfPlJQOrAY7OONwJlSHkmbbm2Kp/w==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/client-common": "5.35.0", + "@algolia/requester-browser-xhr": "5.35.0", + "@algolia/requester-fetch": "5.35.0", + "@algolia/requester-node-http": "5.35.0" + }, + "engines": { + "node": ">= 14.0.0" + } + }, + "node_modules/@algolia/client-search": { + "version": "5.35.0", + "resolved": "https://registry.npmjs.org/@algolia/client-search/-/client-search-5.35.0.tgz", + "integrity": "sha512-FfmdHTrXhIduWyyuko1YTcGLuicVbhUyRjO3HbXE4aP655yKZgdTIfMhZ/V5VY9bHuxv/fGEh3Od1Lvv2ODNTg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/client-common": "5.35.0", + "@algolia/requester-browser-xhr": "5.35.0", + "@algolia/requester-fetch": "5.35.0", + "@algolia/requester-node-http": "5.35.0" + }, + "engines": { + "node": ">= 14.0.0" + } + }, + "node_modules/@algolia/ingestion": { + "version": "1.35.0", + "resolved": "https://registry.npmjs.org/@algolia/ingestion/-/ingestion-1.35.0.tgz", + "integrity": "sha512-gPzACem9IL1Co8mM1LKMhzn1aSJmp+Vp434An4C0OBY4uEJRcqsLN3uLBlY+bYvFg8C8ImwM9YRiKczJXRk0XA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/client-common": "5.35.0", + "@algolia/requester-browser-xhr": "5.35.0", + "@algolia/requester-fetch": "5.35.0", + "@algolia/requester-node-http": "5.35.0" + }, + "engines": { + "node": ">= 14.0.0" + } + }, + "node_modules/@algolia/monitoring": { + "version": "1.35.0", + "resolved": "https://registry.npmjs.org/@algolia/monitoring/-/monitoring-1.35.0.tgz", + "integrity": "sha512-w9MGFLB6ashI8BGcQoVt7iLgDIJNCn4OIu0Q0giE3M2ItNrssvb8C0xuwJQyTy1OFZnemG0EB1OvXhIHOvQwWw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/client-common": "5.35.0", + "@algolia/requester-browser-xhr": "5.35.0", + "@algolia/requester-fetch": "5.35.0", + "@algolia/requester-node-http": "5.35.0" + }, + "engines": { + "node": ">= 14.0.0" + } + }, + "node_modules/@algolia/recommend": { + "version": "5.35.0", + "resolved": "https://registry.npmjs.org/@algolia/recommend/-/recommend-5.35.0.tgz", + "integrity": "sha512-AhrVgaaXAb8Ue0u2nuRWwugt0dL5UmRgS9LXe0Hhz493a8KFeZVUE56RGIV3hAa6tHzmAV7eIoqcWTQvxzlJeQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/client-common": "5.35.0", + "@algolia/requester-browser-xhr": "5.35.0", + "@algolia/requester-fetch": "5.35.0", + "@algolia/requester-node-http": "5.35.0" + }, + "engines": { + "node": ">= 14.0.0" + } + }, + "node_modules/@algolia/requester-browser-xhr": { + "version": "5.35.0", + "resolved": "https://registry.npmjs.org/@algolia/requester-browser-xhr/-/requester-browser-xhr-5.35.0.tgz", + "integrity": "sha512-diY415KLJZ6x1Kbwl9u96Jsz0OstE3asjXtJ9pmk1d+5gPuQ5jQyEsgC+WmEXzlec3iuVszm8AzNYYaqw6B+Zw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/client-common": "5.35.0" + }, + "engines": { + "node": ">= 14.0.0" + } + }, + "node_modules/@algolia/requester-fetch": { + "version": "5.35.0", + "resolved": "https://registry.npmjs.org/@algolia/requester-fetch/-/requester-fetch-5.35.0.tgz", + "integrity": "sha512-uydqnSmpAjrgo8bqhE9N1wgcB98psTRRQXcjc4izwMB7yRl9C8uuAQ/5YqRj04U0mMQ+fdu2fcNF6m9+Z1BzDQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/client-common": "5.35.0" + }, + "engines": { + "node": ">= 14.0.0" + } + }, + "node_modules/@algolia/requester-node-http": { + "version": "5.35.0", + "resolved": "https://registry.npmjs.org/@algolia/requester-node-http/-/requester-node-http-5.35.0.tgz", + "integrity": "sha512-RgLX78ojYOrThJHrIiPzT4HW3yfQa0D7K+MQ81rhxqaNyNBu4F1r+72LNHYH/Z+y9I1Mrjrd/c/Ue5zfDgAEjQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/client-common": "5.35.0" + }, + "engines": { + "node": ">= 14.0.0" + } + }, + "node_modules/@babel/helper-string-parser": { + "version": "7.27.1", + "resolved": "https://registry.npmjs.org/@babel/helper-string-parser/-/helper-string-parser-7.27.1.tgz", + "integrity": "sha512-qMlSxKbpRlAridDExk92nSobyDdpPijUq2DW6oDnUqd0iOGxmQjyqhMIihI9+zv4LPyZdRje2cavWPbCbWm3eA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/helper-validator-identifier": { + "version": "7.27.1", + "resolved": "https://registry.npmjs.org/@babel/helper-validator-identifier/-/helper-validator-identifier-7.27.1.tgz", + "integrity": "sha512-D2hP9eA+Sqx1kBZgzxZh0y1trbuU+JoDkiEwqhQ36nodYqJwyEIhPSdMNd7lOm/4io72luTPWH20Yda0xOuUow==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@babel/parser": { + "version": "7.28.0", + "resolved": "https://registry.npmjs.org/@babel/parser/-/parser-7.28.0.tgz", + "integrity": "sha512-jVZGvOxOuNSsuQuLRTh13nU0AogFlw32w/MT+LV6D3sP5WdbW61E77RnkbaO2dUvmPAYrBDJXGn5gGS6tH4j8g==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/types": "^7.28.0" + }, + "bin": { + "parser": "bin/babel-parser.js" + }, + "engines": { + "node": ">=6.0.0" + } + }, + "node_modules/@babel/types": { + "version": "7.28.2", + "resolved": "https://registry.npmjs.org/@babel/types/-/types-7.28.2.tgz", + "integrity": "sha512-ruv7Ae4J5dUYULmeXw1gmb7rYRz57OWCPM57pHojnLq/3Z1CK2lNSLTCVjxVk1F/TZHwOZZrOWi0ur95BbLxNQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/helper-string-parser": "^7.27.1", + "@babel/helper-validator-identifier": "^7.27.1" + }, + "engines": { + "node": ">=6.9.0" + } + }, + "node_modules/@docsearch/css": { + "version": "3.8.2", + "resolved": "https://registry.npmjs.org/@docsearch/css/-/css-3.8.2.tgz", + "integrity": "sha512-y05ayQFyUmCXze79+56v/4HpycYF3uFqB78pLPrSV5ZKAlDuIAAJNhaRi8tTdRNXh05yxX/TyNnzD6LwSM89vQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/@docsearch/js": { + "version": "3.8.2", + "resolved": "https://registry.npmjs.org/@docsearch/js/-/js-3.8.2.tgz", + "integrity": "sha512-Q5wY66qHn0SwA7Taa0aDbHiJvaFJLOJyHmooQ7y8hlwwQLQ/5WwCcoX0g7ii04Qi2DJlHsd0XXzJ8Ypw9+9YmQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@docsearch/react": "3.8.2", + "preact": "^10.0.0" + } + }, + "node_modules/@docsearch/react": { + "version": "3.8.2", + "resolved": "https://registry.npmjs.org/@docsearch/react/-/react-3.8.2.tgz", + "integrity": "sha512-xCRrJQlTt8N9GU0DG4ptwHRkfnSnD/YpdeaXe02iKfqs97TkZJv60yE+1eq/tjPcVnTW8dP5qLP7itifFVV5eg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/autocomplete-core": "1.17.7", + "@algolia/autocomplete-preset-algolia": "1.17.7", + "@docsearch/css": "3.8.2", + "algoliasearch": "^5.14.2" + }, + "peerDependencies": { + "@types/react": ">= 16.8.0 < 19.0.0", + "react": ">= 16.8.0 < 19.0.0", + "react-dom": ">= 16.8.0 < 19.0.0", + "search-insights": ">= 1 < 3" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "react": { + "optional": true + }, + "react-dom": { + "optional": true + }, + "search-insights": { + "optional": true + } + } + }, + "node_modules/@esbuild/aix-ppc64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.21.5.tgz", + "integrity": "sha512-1SDgH6ZSPTlggy1yI6+Dbkiz8xzpHJEVAlF/AM1tHPLsf5STom9rwtjE4hKAF20FfXXNTFqEYXyJNWh1GiZedQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "aix" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/android-arm": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.21.5.tgz", + "integrity": "sha512-vCPvzSjpPHEi1siZdlvAlsPxXl7WbOVUBBAowWug4rJHb68Ox8KualB+1ocNvT5fjv6wpkX6o/iEpbDrf68zcg==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/android-arm64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.21.5.tgz", + "integrity": "sha512-c0uX9VAUBQ7dTDCjq+wdyGLowMdtR/GoC2U5IYk/7D1H1JYC0qseD7+11iMP2mRLN9RcCMRcjC4YMclCzGwS/A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/android-x64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.21.5.tgz", + "integrity": "sha512-D7aPRUUNHRBwHxzxRvp856rjUHRFW1SdQATKXH2hqA0kAZb1hKmi02OpYRacl0TxIGz/ZmXWlbZgjwWYaCakTA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/darwin-arm64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.21.5.tgz", + "integrity": "sha512-DwqXqZyuk5AiWWf3UfLiRDJ5EDd49zg6O9wclZ7kUMv2WRFr4HKjXp/5t8JZ11QbQfUS6/cRCKGwYhtNAY88kQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/darwin-x64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.21.5.tgz", + "integrity": "sha512-se/JjF8NlmKVG4kNIuyWMV/22ZaerB+qaSi5MdrXtd6R08kvs2qCN4C09miupktDitvh8jRFflwGFBQcxZRjbw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/freebsd-arm64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.21.5.tgz", + "integrity": "sha512-5JcRxxRDUJLX8JXp/wcBCy3pENnCgBR9bN6JsY4OmhfUtIHe3ZW0mawA7+RDAcMLrMIZaf03NlQiX9DGyB8h4g==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/freebsd-x64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.21.5.tgz", + "integrity": "sha512-J95kNBj1zkbMXtHVH29bBriQygMXqoVQOQYA+ISs0/2l3T9/kj42ow2mpqerRBxDJnmkUDCaQT/dfNXWX/ZZCQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/linux-arm": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.21.5.tgz", + "integrity": "sha512-bPb5AHZtbeNGjCKVZ9UGqGwo8EUu4cLq68E95A53KlxAPRmUyYv2D6F0uUI65XisGOL1hBP5mTronbgo+0bFcA==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/linux-arm64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.21.5.tgz", + "integrity": "sha512-ibKvmyYzKsBeX8d8I7MH/TMfWDXBF3db4qM6sy+7re0YXya+K1cem3on9XgdT2EQGMu4hQyZhan7TeQ8XkGp4Q==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/linux-ia32": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.21.5.tgz", + "integrity": "sha512-YvjXDqLRqPDl2dvRODYmmhz4rPeVKYvppfGYKSNGdyZkA01046pLWyRKKI3ax8fbJoK5QbxblURkwK/MWY18Tg==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/linux-loong64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.21.5.tgz", + "integrity": "sha512-uHf1BmMG8qEvzdrzAqg2SIG/02+4/DHB6a9Kbya0XDvwDEKCoC8ZRWI5JJvNdUjtciBGFQ5PuBlpEOXQj+JQSg==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/linux-mips64el": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.21.5.tgz", + "integrity": "sha512-IajOmO+KJK23bj52dFSNCMsz1QP1DqM6cwLUv3W1QwyxkyIWecfafnI555fvSGqEKwjMXVLokcV5ygHW5b3Jbg==", + "cpu": [ + "mips64el" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/linux-ppc64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.21.5.tgz", + "integrity": "sha512-1hHV/Z4OEfMwpLO8rp7CvlhBDnjsC3CttJXIhBi+5Aj5r+MBvy4egg7wCbe//hSsT+RvDAG7s81tAvpL2XAE4w==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/linux-riscv64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.21.5.tgz", + "integrity": "sha512-2HdXDMd9GMgTGrPWnJzP2ALSokE/0O5HhTUvWIbD3YdjME8JwvSCnNGBnTThKGEB91OZhzrJ4qIIxk/SBmyDDA==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/linux-s390x": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.21.5.tgz", + "integrity": "sha512-zus5sxzqBJD3eXxwvjN1yQkRepANgxE9lgOW2qLnmr8ikMTphkjgXu1HR01K4FJg8h1kEEDAqDcZQtbrRnB41A==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/linux-x64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.21.5.tgz", + "integrity": "sha512-1rYdTpyv03iycF1+BhzrzQJCdOuAOtaqHTWJZCWvijKD2N5Xu0TtVC8/+1faWqcP9iBCWOmjmhoH94dH82BxPQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/netbsd-x64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.21.5.tgz", + "integrity": "sha512-Woi2MXzXjMULccIwMnLciyZH4nCIMpWQAs049KEeMvOcNADVxo0UBIQPfSmxB3CWKedngg7sWZdLvLczpe0tLg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/openbsd-x64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.21.5.tgz", + "integrity": "sha512-HLNNw99xsvx12lFBUwoT8EVCsSvRNDVxNpjZ7bPn947b8gJPzeHWyNVhFsaerc0n3TsbOINvRP2byTZ5LKezow==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/sunos-x64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.21.5.tgz", + "integrity": "sha512-6+gjmFpfy0BHU5Tpptkuh8+uw3mnrvgs+dSPQXQOv3ekbordwnzTVEb4qnIvQcYXq6gzkyTnoZ9dZG+D4garKg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "sunos" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/win32-arm64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.21.5.tgz", + "integrity": "sha512-Z0gOTd75VvXqyq7nsl93zwahcTROgqvuAcYDUr+vOv8uHhNSKROyU961kgtCD1e95IqPKSQKH7tBTslnS3tA8A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/win32-ia32": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.21.5.tgz", + "integrity": "sha512-SWXFF1CL2RVNMaVs+BBClwtfZSvDgtL//G/smwAc5oVK/UPu2Gu9tIaRgFmYFFKrmg3SyAjSrElf0TiJ1v8fYA==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@esbuild/win32-x64": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.21.5.tgz", + "integrity": "sha512-tQd/1efJuzPC6rCFwEvLtci/xNFcTZknmXs98FYDfGE4wP9ClFV98nyKrzJKVPMhdDnjzLhdUyMX4PsQAPjwIw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=12" + } + }, + "node_modules/@iconify-json/simple-icons": { + "version": "1.2.45", + "resolved": "https://registry.npmjs.org/@iconify-json/simple-icons/-/simple-icons-1.2.45.tgz", + "integrity": "sha512-POOz+NjYQDy2fy1u+sIZi05N6r6oSooIGBaBcZLh7w8QOmLgJAZ6mBt+7Messp7ku9ucRua61if33BPoOZCwRQ==", + "dev": true, + "license": "CC0-1.0", + "dependencies": { + "@iconify/types": "*" + } + }, + "node_modules/@iconify/types": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/@iconify/types/-/types-2.0.0.tgz", + "integrity": "sha512-+wluvCrRhXrhyOmRDJ3q8mux9JkKy5SJ/v8ol2tu4FVjyYvtEzkc/3pK15ET6RKg4b4w4BmTk1+gsCUhf21Ykg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@jridgewell/sourcemap-codec": { + "version": "1.5.4", + "resolved": "https://registry.npmjs.org/@jridgewell/sourcemap-codec/-/sourcemap-codec-1.5.4.tgz", + "integrity": "sha512-VT2+G1VQs/9oz078bLrYbecdZKs912zQlkelYpuf+SXF+QvZDYJlbx/LSx+meSAwdDFnF8FVXW92AVjjkVmgFw==", + "dev": true, + "license": "MIT" + }, + "node_modules/@rollup/rollup-android-arm-eabi": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.46.2.tgz", + "integrity": "sha512-Zj3Hl6sN34xJtMv7Anwb5Gu01yujyE/cLBDB2gnHTAHaWS1Z38L7kuSG+oAh0giZMqG060f/YBStXtMH6FvPMA==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ] + }, + "node_modules/@rollup/rollup-android-arm64": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm64/-/rollup-android-arm64-4.46.2.tgz", + "integrity": "sha512-nTeCWY83kN64oQ5MGz3CgtPx8NSOhC5lWtsjTs+8JAJNLcP3QbLCtDDgUKQc/Ro/frpMq4SHUaHN6AMltcEoLQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ] + }, + "node_modules/@rollup/rollup-darwin-arm64": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-arm64/-/rollup-darwin-arm64-4.46.2.tgz", + "integrity": "sha512-HV7bW2Fb/F5KPdM/9bApunQh68YVDU8sO8BvcW9OngQVN3HHHkw99wFupuUJfGR9pYLLAjcAOA6iO+evsbBaPQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@rollup/rollup-darwin-x64": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-x64/-/rollup-darwin-x64-4.46.2.tgz", + "integrity": "sha512-SSj8TlYV5nJixSsm/y3QXfhspSiLYP11zpfwp6G/YDXctf3Xkdnk4woJIF5VQe0of2OjzTt8EsxnJDCdHd2xMA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@rollup/rollup-freebsd-arm64": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-arm64/-/rollup-freebsd-arm64-4.46.2.tgz", + "integrity": "sha512-ZyrsG4TIT9xnOlLsSSi9w/X29tCbK1yegE49RYm3tu3wF1L/B6LVMqnEWyDB26d9Ecx9zrmXCiPmIabVuLmNSg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ] + }, + "node_modules/@rollup/rollup-freebsd-x64": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-freebsd-x64/-/rollup-freebsd-x64-4.46.2.tgz", + "integrity": "sha512-pCgHFoOECwVCJ5GFq8+gR8SBKnMO+xe5UEqbemxBpCKYQddRQMgomv1104RnLSg7nNvgKy05sLsY51+OVRyiVw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ] + }, + "node_modules/@rollup/rollup-linux-arm-gnueabihf": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-gnueabihf/-/rollup-linux-arm-gnueabihf-4.46.2.tgz", + "integrity": "sha512-EtP8aquZ0xQg0ETFcxUbU71MZlHaw9MChwrQzatiE8U/bvi5uv/oChExXC4mWhjiqK7azGJBqU0tt5H123SzVA==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-arm-musleabihf": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm-musleabihf/-/rollup-linux-arm-musleabihf-4.46.2.tgz", + "integrity": "sha512-qO7F7U3u1nfxYRPM8HqFtLd+raev2K137dsV08q/LRKRLEc7RsiDWihUnrINdsWQxPR9jqZ8DIIZ1zJJAm5PjQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-arm64-gnu": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-gnu/-/rollup-linux-arm64-gnu-4.46.2.tgz", + "integrity": "sha512-3dRaqLfcOXYsfvw5xMrxAk9Lb1f395gkoBYzSFcc/scgRFptRXL9DOaDpMiehf9CO8ZDRJW2z45b6fpU5nwjng==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-arm64-musl": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-arm64-musl/-/rollup-linux-arm64-musl-4.46.2.tgz", + "integrity": "sha512-fhHFTutA7SM+IrR6lIfiHskxmpmPTJUXpWIsBXpeEwNgZzZZSg/q4i6FU4J8qOGyJ0TR+wXBwx/L7Ho9z0+uDg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-loongarch64-gnu": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-loongarch64-gnu/-/rollup-linux-loongarch64-gnu-4.46.2.tgz", + "integrity": "sha512-i7wfGFXu8x4+FRqPymzjD+Hyav8l95UIZ773j7J7zRYc3Xsxy2wIn4x+llpunexXe6laaO72iEjeeGyUFmjKeA==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-ppc64-gnu": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-ppc64-gnu/-/rollup-linux-ppc64-gnu-4.46.2.tgz", + "integrity": "sha512-B/l0dFcHVUnqcGZWKcWBSV2PF01YUt0Rvlurci5P+neqY/yMKchGU8ullZvIv5e8Y1C6wOn+U03mrDylP5q9Yw==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-riscv64-gnu": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-riscv64-gnu/-/rollup-linux-riscv64-gnu-4.46.2.tgz", + "integrity": "sha512-32k4ENb5ygtkMwPMucAb8MtV8olkPT03oiTxJbgkJa7lJ7dZMr0GCFJlyvy+K8iq7F/iuOr41ZdUHaOiqyR3iQ==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-riscv64-musl": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-riscv64-musl/-/rollup-linux-riscv64-musl-4.46.2.tgz", + "integrity": "sha512-t5B2loThlFEauloaQkZg9gxV05BYeITLvLkWOkRXogP4qHXLkWSbSHKM9S6H1schf/0YGP/qNKtiISlxvfmmZw==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-s390x-gnu": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-s390x-gnu/-/rollup-linux-s390x-gnu-4.46.2.tgz", + "integrity": "sha512-YKjekwTEKgbB7n17gmODSmJVUIvj8CX7q5442/CK80L8nqOUbMtf8b01QkG3jOqyr1rotrAnW6B/qiHwfcuWQA==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-x64-gnu": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-gnu/-/rollup-linux-x64-gnu-4.46.2.tgz", + "integrity": "sha512-Jj5a9RUoe5ra+MEyERkDKLwTXVu6s3aACP51nkfnK9wJTraCC8IMe3snOfALkrjTYd2G1ViE1hICj0fZ7ALBPA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-linux-x64-musl": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-linux-x64-musl/-/rollup-linux-x64-musl-4.46.2.tgz", + "integrity": "sha512-7kX69DIrBeD7yNp4A5b81izs8BqoZkCIaxQaOpumcJ1S/kmqNFjPhDu1LHeVXv0SexfHQv5cqHsxLOjETuqDuA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@rollup/rollup-win32-arm64-msvc": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-arm64-msvc/-/rollup-win32-arm64-msvc-4.46.2.tgz", + "integrity": "sha512-wiJWMIpeaak/jsbaq2HMh/rzZxHVW1rU6coyeNNpMwk5isiPjSTx0a4YLSlYDwBH/WBvLz+EtsNqQScZTLJy3g==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@rollup/rollup-win32-ia32-msvc": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-ia32-msvc/-/rollup-win32-ia32-msvc-4.46.2.tgz", + "integrity": "sha512-gBgaUDESVzMgWZhcyjfs9QFK16D8K6QZpwAaVNJxYDLHWayOta4ZMjGm/vsAEy3hvlS2GosVFlBlP9/Wb85DqQ==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@rollup/rollup-win32-x64-msvc": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/@rollup/rollup-win32-x64-msvc/-/rollup-win32-x64-msvc-4.46.2.tgz", + "integrity": "sha512-CvUo2ixeIQGtF6WvuB87XWqPQkoFAFqW+HUo/WzHwuHDvIwZCtjdWXoYCcr06iKGydiqTclC4jU/TNObC/xKZg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@shikijs/core": { + "version": "2.5.0", + "resolved": "https://registry.npmjs.org/@shikijs/core/-/core-2.5.0.tgz", + "integrity": "sha512-uu/8RExTKtavlpH7XqnVYBrfBkUc20ngXiX9NSrBhOVZYv/7XQRKUyhtkeflY5QsxC0GbJThCerruZfsUaSldg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@shikijs/engine-javascript": "2.5.0", + "@shikijs/engine-oniguruma": "2.5.0", + "@shikijs/types": "2.5.0", + "@shikijs/vscode-textmate": "^10.0.2", + "@types/hast": "^3.0.4", + "hast-util-to-html": "^9.0.4" + } + }, + "node_modules/@shikijs/engine-javascript": { + "version": "2.5.0", + "resolved": "https://registry.npmjs.org/@shikijs/engine-javascript/-/engine-javascript-2.5.0.tgz", + "integrity": "sha512-VjnOpnQf8WuCEZtNUdjjwGUbtAVKuZkVQ/5cHy/tojVVRIRtlWMYVjyWhxOmIq05AlSOv72z7hRNRGVBgQOl0w==", + "dev": true, + "license": "MIT", + "dependencies": { + "@shikijs/types": "2.5.0", + "@shikijs/vscode-textmate": "^10.0.2", + "oniguruma-to-es": "^3.1.0" + } + }, + "node_modules/@shikijs/engine-oniguruma": { + "version": "2.5.0", + "resolved": "https://registry.npmjs.org/@shikijs/engine-oniguruma/-/engine-oniguruma-2.5.0.tgz", + "integrity": "sha512-pGd1wRATzbo/uatrCIILlAdFVKdxImWJGQ5rFiB5VZi2ve5xj3Ax9jny8QvkaV93btQEwR/rSz5ERFpC5mKNIw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@shikijs/types": "2.5.0", + "@shikijs/vscode-textmate": "^10.0.2" + } + }, + "node_modules/@shikijs/langs": { + "version": "2.5.0", + "resolved": "https://registry.npmjs.org/@shikijs/langs/-/langs-2.5.0.tgz", + "integrity": "sha512-Qfrrt5OsNH5R+5tJ/3uYBBZv3SuGmnRPejV9IlIbFH3HTGLDlkqgHymAlzklVmKBjAaVmkPkyikAV/sQ1wSL+w==", + "dev": true, + "license": "MIT", + "dependencies": { + "@shikijs/types": "2.5.0" + } + }, + "node_modules/@shikijs/themes": { + "version": "2.5.0", + "resolved": "https://registry.npmjs.org/@shikijs/themes/-/themes-2.5.0.tgz", + "integrity": "sha512-wGrk+R8tJnO0VMzmUExHR+QdSaPUl/NKs+a4cQQRWyoc3YFbUzuLEi/KWK1hj+8BfHRKm2jNhhJck1dfstJpiw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@shikijs/types": "2.5.0" + } + }, + "node_modules/@shikijs/transformers": { + "version": "2.5.0", + "resolved": "https://registry.npmjs.org/@shikijs/transformers/-/transformers-2.5.0.tgz", + "integrity": "sha512-SI494W5X60CaUwgi8u4q4m4s3YAFSxln3tzNjOSYqq54wlVgz0/NbbXEb3mdLbqMBztcmS7bVTaEd2w0qMmfeg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@shikijs/core": "2.5.0", + "@shikijs/types": "2.5.0" + } + }, + "node_modules/@shikijs/types": { + "version": "2.5.0", + "resolved": "https://registry.npmjs.org/@shikijs/types/-/types-2.5.0.tgz", + "integrity": "sha512-ygl5yhxki9ZLNuNpPitBWvcy9fsSKKaRuO4BAlMyagszQidxcpLAr0qiW/q43DtSIDxO6hEbtYLiFZNXO/hdGw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@shikijs/vscode-textmate": "^10.0.2", + "@types/hast": "^3.0.4" + } + }, + "node_modules/@shikijs/vscode-textmate": { + "version": "10.0.2", + "resolved": "https://registry.npmjs.org/@shikijs/vscode-textmate/-/vscode-textmate-10.0.2.tgz", + "integrity": "sha512-83yeghZ2xxin3Nj8z1NMd/NCuca+gsYXswywDy5bHvwlWL8tpTQmzGeUuHd9FC3E/SBEMvzJRwWEOz5gGes9Qg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/estree": { + "version": "1.0.8", + "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.8.tgz", + "integrity": "sha512-dWHzHa2WqEXI/O1E9OjrocMTKJl2mSrEolh1Iomrv6U+JuNwaHXsXx9bLu5gG7BUWFIN0skIQJQ/L1rIex4X6w==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/hast": { + "version": "3.0.4", + "resolved": "https://registry.npmjs.org/@types/hast/-/hast-3.0.4.tgz", + "integrity": "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/unist": "*" + } + }, + "node_modules/@types/linkify-it": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/@types/linkify-it/-/linkify-it-5.0.0.tgz", + "integrity": "sha512-sVDA58zAw4eWAffKOaQH5/5j3XeayukzDk+ewSsnv3p4yJEZHCCzMDiZM8e0OUrRvmpGZ85jf4yDHkHsgBNr9Q==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/markdown-it": { + "version": "14.1.2", + "resolved": "https://registry.npmjs.org/@types/markdown-it/-/markdown-it-14.1.2.tgz", + "integrity": "sha512-promo4eFwuiW+TfGxhi+0x3czqTYJkG8qB17ZUJiVF10Xm7NLVRSLUsfRTU/6h1e24VvRnXCx+hG7li58lkzog==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/linkify-it": "^5", + "@types/mdurl": "^2" + } + }, + "node_modules/@types/mdast": { + "version": "4.0.4", + "resolved": "https://registry.npmjs.org/@types/mdast/-/mdast-4.0.4.tgz", + "integrity": "sha512-kGaNbPh1k7AFzgpud/gMdvIm5xuECykRR+JnWKQno9TAXVa6WIVCGTPvYGekIDL4uwCZQSYbUxNBSb1aUo79oA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/unist": "*" + } + }, + "node_modules/@types/mdurl": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/@types/mdurl/-/mdurl-2.0.0.tgz", + "integrity": "sha512-RGdgjQUZba5p6QEFAVx2OGb8rQDL/cPRG7GiedRzMcJ1tYnUANBncjbSB1NRGwbvjcPeikRABz2nshyPk1bhWg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/unist": { + "version": "3.0.3", + "resolved": "https://registry.npmjs.org/@types/unist/-/unist-3.0.3.tgz", + "integrity": "sha512-ko/gIFJRv177XgZsZcBwnqJN5x/Gien8qNOn0D5bQU/zAzVf9Zt3BlcUiLqhV9y4ARk0GbT3tnUiPNgnTXzc/Q==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/web-bluetooth": { + "version": "0.0.21", + "resolved": "https://registry.npmjs.org/@types/web-bluetooth/-/web-bluetooth-0.0.21.tgz", + "integrity": "sha512-oIQLCGWtcFZy2JW77j9k8nHzAOpqMHLQejDA48XXMWH6tjCQHz5RCFz1bzsmROyL6PUm+LLnUiI4BCn221inxA==", + "dev": true, + "license": "MIT" + }, + "node_modules/@ungap/structured-clone": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@ungap/structured-clone/-/structured-clone-1.3.0.tgz", + "integrity": "sha512-WmoN8qaIAo7WTYWbAZuG8PYEhn5fkz7dZrqTBZ7dtt//lL2Gwms1IcnQ5yHqjDfX8Ft5j4YzDM23f87zBfDe9g==", + "dev": true, + "license": "ISC" + }, + "node_modules/@vitejs/plugin-vue": { + "version": "5.2.4", + "resolved": "https://registry.npmjs.org/@vitejs/plugin-vue/-/plugin-vue-5.2.4.tgz", + "integrity": "sha512-7Yx/SXSOcQq5HiiV3orevHUFn+pmMB4cgbEkDYgnkUWb0WfeQ/wa2yFv6D5ICiCQOVpjA7vYDXrC7AGO8yjDHA==", + "dev": true, + "license": "MIT", + "engines": { + "node": "^18.0.0 || >=20.0.0" + }, + "peerDependencies": { + "vite": "^5.0.0 || ^6.0.0", + "vue": "^3.2.25" + } + }, + "node_modules/@vue/compiler-core": { + "version": "3.5.18", + "resolved": "https://registry.npmjs.org/@vue/compiler-core/-/compiler-core-3.5.18.tgz", + "integrity": "sha512-3slwjQrrV1TO8MoXgy3aynDQ7lslj5UqDxuHnrzHtpON5CBinhWjJETciPngpin/T3OuW3tXUf86tEurusnztw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/parser": "^7.28.0", + "@vue/shared": "3.5.18", + "entities": "^4.5.0", + "estree-walker": "^2.0.2", + "source-map-js": "^1.2.1" + } + }, + "node_modules/@vue/compiler-dom": { + "version": "3.5.18", + "resolved": "https://registry.npmjs.org/@vue/compiler-dom/-/compiler-dom-3.5.18.tgz", + "integrity": "sha512-RMbU6NTU70++B1JyVJbNbeFkK+A+Q7y9XKE2EM4NLGm2WFR8x9MbAtWxPPLdm0wUkuZv9trpwfSlL6tjdIa1+A==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vue/compiler-core": "3.5.18", + "@vue/shared": "3.5.18" + } + }, + "node_modules/@vue/compiler-sfc": { + "version": "3.5.18", + "resolved": "https://registry.npmjs.org/@vue/compiler-sfc/-/compiler-sfc-3.5.18.tgz", + "integrity": "sha512-5aBjvGqsWs+MoxswZPoTB9nSDb3dhd1x30xrrltKujlCxo48j8HGDNj3QPhF4VIS0VQDUrA1xUfp2hEa+FNyXA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@babel/parser": "^7.28.0", + "@vue/compiler-core": "3.5.18", + "@vue/compiler-dom": "3.5.18", + "@vue/compiler-ssr": "3.5.18", + "@vue/shared": "3.5.18", + "estree-walker": "^2.0.2", + "magic-string": "^0.30.17", + "postcss": "^8.5.6", + "source-map-js": "^1.2.1" + } + }, + "node_modules/@vue/compiler-ssr": { + "version": "3.5.18", + "resolved": "https://registry.npmjs.org/@vue/compiler-ssr/-/compiler-ssr-3.5.18.tgz", + "integrity": "sha512-xM16Ak7rSWHkM3m22NlmcdIM+K4BMyFARAfV9hYFl+SFuRzrZ3uGMNW05kA5pmeMa0X9X963Kgou7ufdbpOP9g==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vue/compiler-dom": "3.5.18", + "@vue/shared": "3.5.18" + } + }, + "node_modules/@vue/devtools-api": { + "version": "7.7.7", + "resolved": "https://registry.npmjs.org/@vue/devtools-api/-/devtools-api-7.7.7.tgz", + "integrity": "sha512-lwOnNBH2e7x1fIIbVT7yF5D+YWhqELm55/4ZKf45R9T8r9dE2AIOy8HKjfqzGsoTHFbWbr337O4E0A0QADnjBg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vue/devtools-kit": "^7.7.7" + } + }, + "node_modules/@vue/devtools-kit": { + "version": "7.7.7", + "resolved": "https://registry.npmjs.org/@vue/devtools-kit/-/devtools-kit-7.7.7.tgz", + "integrity": "sha512-wgoZtxcTta65cnZ1Q6MbAfePVFxfM+gq0saaeytoph7nEa7yMXoi6sCPy4ufO111B9msnw0VOWjPEFCXuAKRHA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vue/devtools-shared": "^7.7.7", + "birpc": "^2.3.0", + "hookable": "^5.5.3", + "mitt": "^3.0.1", + "perfect-debounce": "^1.0.0", + "speakingurl": "^14.0.1", + "superjson": "^2.2.2" + } + }, + "node_modules/@vue/devtools-shared": { + "version": "7.7.7", + "resolved": "https://registry.npmjs.org/@vue/devtools-shared/-/devtools-shared-7.7.7.tgz", + "integrity": "sha512-+udSj47aRl5aKb0memBvcUG9koarqnxNM5yjuREvqwK6T3ap4mn3Zqqc17QrBFTqSMjr3HK1cvStEZpMDpfdyw==", + "dev": true, + "license": "MIT", + "dependencies": { + "rfdc": "^1.4.1" + } + }, + "node_modules/@vue/reactivity": { + "version": "3.5.18", + "resolved": "https://registry.npmjs.org/@vue/reactivity/-/reactivity-3.5.18.tgz", + "integrity": "sha512-x0vPO5Imw+3sChLM5Y+B6G1zPjwdOri9e8V21NnTnlEvkxatHEH5B5KEAJcjuzQ7BsjGrKtfzuQ5eQwXh8HXBg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vue/shared": "3.5.18" + } + }, + "node_modules/@vue/runtime-core": { + "version": "3.5.18", + "resolved": "https://registry.npmjs.org/@vue/runtime-core/-/runtime-core-3.5.18.tgz", + "integrity": "sha512-DUpHa1HpeOQEt6+3nheUfqVXRog2kivkXHUhoqJiKR33SO4x+a5uNOMkV487WPerQkL0vUuRvq/7JhRgLW3S+w==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vue/reactivity": "3.5.18", + "@vue/shared": "3.5.18" + } + }, + "node_modules/@vue/runtime-dom": { + "version": "3.5.18", + "resolved": "https://registry.npmjs.org/@vue/runtime-dom/-/runtime-dom-3.5.18.tgz", + "integrity": "sha512-YwDj71iV05j4RnzZnZtGaXwPoUWeRsqinblgVJwR8XTXYZ9D5PbahHQgsbmzUvCWNF6x7siQ89HgnX5eWkr3mw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vue/reactivity": "3.5.18", + "@vue/runtime-core": "3.5.18", + "@vue/shared": "3.5.18", + "csstype": "^3.1.3" + } + }, + "node_modules/@vue/server-renderer": { + "version": "3.5.18", + "resolved": "https://registry.npmjs.org/@vue/server-renderer/-/server-renderer-3.5.18.tgz", + "integrity": "sha512-PvIHLUoWgSbDG7zLHqSqaCoZvHi6NNmfVFOqO+OnwvqMz/tqQr3FuGWS8ufluNddk7ZLBJYMrjcw1c6XzR12mA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vue/compiler-ssr": "3.5.18", + "@vue/shared": "3.5.18" + }, + "peerDependencies": { + "vue": "3.5.18" + } + }, + "node_modules/@vue/shared": { + "version": "3.5.18", + "resolved": "https://registry.npmjs.org/@vue/shared/-/shared-3.5.18.tgz", + "integrity": "sha512-cZy8Dq+uuIXbxCZpuLd2GJdeSO/lIzIspC2WtkqIpje5QyFbvLaI5wZtdUjLHjGZrlVX6GilejatWwVYYRc8tA==", + "dev": true, + "license": "MIT" + }, + "node_modules/@vueuse/core": { + "version": "12.8.2", + "resolved": "https://registry.npmjs.org/@vueuse/core/-/core-12.8.2.tgz", + "integrity": "sha512-HbvCmZdzAu3VGi/pWYm5Ut+Kd9mn1ZHnn4L5G8kOQTPs/IwIAmJoBrmYk2ckLArgMXZj0AW3n5CAejLUO+PhdQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/web-bluetooth": "^0.0.21", + "@vueuse/metadata": "12.8.2", + "@vueuse/shared": "12.8.2", + "vue": "^3.5.13" + }, + "funding": { + "url": "https://github.com/sponsors/antfu" + } + }, + "node_modules/@vueuse/integrations": { + "version": "12.8.2", + "resolved": "https://registry.npmjs.org/@vueuse/integrations/-/integrations-12.8.2.tgz", + "integrity": "sha512-fbGYivgK5uBTRt7p5F3zy6VrETlV9RtZjBqd1/HxGdjdckBgBM4ugP8LHpjolqTj14TXTxSK1ZfgPbHYyGuH7g==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vueuse/core": "12.8.2", + "@vueuse/shared": "12.8.2", + "vue": "^3.5.13" + }, + "funding": { + "url": "https://github.com/sponsors/antfu" + }, + "peerDependencies": { + "async-validator": "^4", + "axios": "^1", + "change-case": "^5", + "drauu": "^0.4", + "focus-trap": "^7", + "fuse.js": "^7", + "idb-keyval": "^6", + "jwt-decode": "^4", + "nprogress": "^0.2", + "qrcode": "^1.5", + "sortablejs": "^1", + "universal-cookie": "^7" + }, + "peerDependenciesMeta": { + "async-validator": { + "optional": true + }, + "axios": { + "optional": true + }, + "change-case": { + "optional": true + }, + "drauu": { + "optional": true + }, + "focus-trap": { + "optional": true + }, + "fuse.js": { + "optional": true + }, + "idb-keyval": { + "optional": true + }, + "jwt-decode": { + "optional": true + }, + "nprogress": { + "optional": true + }, + "qrcode": { + "optional": true + }, + "sortablejs": { + "optional": true + }, + "universal-cookie": { + "optional": true + } + } + }, + "node_modules/@vueuse/metadata": { + "version": "12.8.2", + "resolved": "https://registry.npmjs.org/@vueuse/metadata/-/metadata-12.8.2.tgz", + "integrity": "sha512-rAyLGEuoBJ/Il5AmFHiziCPdQzRt88VxR+Y/A/QhJ1EWtWqPBBAxTAFaSkviwEuOEZNtW8pvkPgoCZQ+HxqW1A==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/antfu" + } + }, + "node_modules/@vueuse/shared": { + "version": "12.8.2", + "resolved": "https://registry.npmjs.org/@vueuse/shared/-/shared-12.8.2.tgz", + "integrity": "sha512-dznP38YzxZoNloI0qpEfpkms8knDtaoQ6Y/sfS0L7Yki4zh40LFHEhur0odJC6xTHG5dxWVPiUWBXn+wCG2s5w==", + "dev": true, + "license": "MIT", + "dependencies": { + "vue": "^3.5.13" + }, + "funding": { + "url": "https://github.com/sponsors/antfu" + } + }, + "node_modules/algoliasearch": { + "version": "5.35.0", + "resolved": "https://registry.npmjs.org/algoliasearch/-/algoliasearch-5.35.0.tgz", + "integrity": "sha512-Y+moNhsqgLmvJdgTsO4GZNgsaDWv8AOGAaPeIeHKlDn/XunoAqYbA+XNpBd1dW8GOXAUDyxC9Rxc7AV4kpFcIg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@algolia/abtesting": "1.1.0", + "@algolia/client-abtesting": "5.35.0", + "@algolia/client-analytics": "5.35.0", + "@algolia/client-common": "5.35.0", + "@algolia/client-insights": "5.35.0", + "@algolia/client-personalization": "5.35.0", + "@algolia/client-query-suggestions": "5.35.0", + "@algolia/client-search": "5.35.0", + "@algolia/ingestion": "1.35.0", + "@algolia/monitoring": "1.35.0", + "@algolia/recommend": "5.35.0", + "@algolia/requester-browser-xhr": "5.35.0", + "@algolia/requester-fetch": "5.35.0", + "@algolia/requester-node-http": "5.35.0" + }, + "engines": { + "node": ">= 14.0.0" + } + }, + "node_modules/birpc": { + "version": "2.5.0", + "resolved": "https://registry.npmjs.org/birpc/-/birpc-2.5.0.tgz", + "integrity": "sha512-VSWO/W6nNQdyP520F1mhf+Lc2f8pjGQOtoHHm7Ze8Go1kX7akpVIrtTa0fn+HB0QJEDVacl6aO08YE0PgXfdnQ==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/antfu" + } + }, + "node_modules/ccount": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/ccount/-/ccount-2.0.1.tgz", + "integrity": "sha512-eyrF0jiFpY+3drT6383f1qhkbGsLSifNAjA61IUjZjmLCWjItY6LB9ft9YhoDgwfmclB2zhu51Lc7+95b8NRAg==", + "dev": true, + "license": "MIT", + "funding": { + "type": "github", + "url": "https://github.com/sponsors/wooorm" + } + }, + "node_modules/character-entities-html4": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/character-entities-html4/-/character-entities-html4-2.1.0.tgz", + "integrity": "sha512-1v7fgQRj6hnSwFpq1Eu0ynr/CDEw0rXo2B61qXrLNdHZmPKgb7fqS1a2JwF0rISo9q77jDI8VMEHoApn8qDoZA==", + "dev": true, + "license": "MIT", + "funding": { + "type": "github", + "url": "https://github.com/sponsors/wooorm" + } + }, + "node_modules/character-entities-legacy": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/character-entities-legacy/-/character-entities-legacy-3.0.0.tgz", + "integrity": "sha512-RpPp0asT/6ufRm//AJVwpViZbGM/MkjQFxJccQRHmISF/22NBtsHqAWmL+/pmkPWoIUJdWyeVleTl1wydHATVQ==", + "dev": true, + "license": "MIT", + "funding": { + "type": "github", + "url": "https://github.com/sponsors/wooorm" + } + }, + "node_modules/comma-separated-tokens": { + "version": "2.0.3", + "resolved": "https://registry.npmjs.org/comma-separated-tokens/-/comma-separated-tokens-2.0.3.tgz", + "integrity": "sha512-Fu4hJdvzeylCfQPp9SGWidpzrMs7tTrlu6Vb8XGaRGck8QSNZJJp538Wrb60Lax4fPwR64ViY468OIUTbRlGZg==", + "dev": true, + "license": "MIT", + "funding": { + "type": "github", + "url": "https://github.com/sponsors/wooorm" + } + }, + "node_modules/copy-anything": { + "version": "3.0.5", + "resolved": "https://registry.npmjs.org/copy-anything/-/copy-anything-3.0.5.tgz", + "integrity": "sha512-yCEafptTtb4bk7GLEQoM8KVJpxAfdBJYaXyzQEgQQQgYrZiDp8SJmGKlYza6CYjEDNstAdNdKA3UuoULlEbS6w==", + "dev": true, + "license": "MIT", + "dependencies": { + "is-what": "^4.1.8" + }, + "engines": { + "node": ">=12.13" + }, + "funding": { + "url": "https://github.com/sponsors/mesqueeb" + } + }, + "node_modules/csstype": { + "version": "3.1.3", + "resolved": "https://registry.npmjs.org/csstype/-/csstype-3.1.3.tgz", + "integrity": "sha512-M1uQkMl8rQK/szD0LNhtqxIPLpimGm8sOBwU7lLnCpSbTyY3yeU1Vc7l4KT5zT4s/yOxHH5O7tIuuLOCnLADRw==", + "dev": true, + "license": "MIT" + }, + "node_modules/dequal": { + "version": "2.0.3", + "resolved": "https://registry.npmjs.org/dequal/-/dequal-2.0.3.tgz", + "integrity": "sha512-0je+qPKHEMohvfRTCEo3CrPG6cAzAYgmzKyxRiYSSDkS6eGJdyVJm7WaYA5ECaAD9wLB2T4EEeymA5aFVcYXCA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6" + } + }, + "node_modules/devlop": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/devlop/-/devlop-1.1.0.tgz", + "integrity": "sha512-RWmIqhcFf1lRYBvNmr7qTNuyCt/7/ns2jbpp1+PalgE/rDQcBT0fioSMUpJ93irlUhC5hrg4cYqe6U+0ImW0rA==", + "dev": true, + "license": "MIT", + "dependencies": { + "dequal": "^2.0.0" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/wooorm" + } + }, + "node_modules/emoji-regex-xs": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/emoji-regex-xs/-/emoji-regex-xs-1.0.0.tgz", + "integrity": "sha512-LRlerrMYoIDrT6jgpeZ2YYl/L8EulRTt5hQcYjy5AInh7HWXKimpqx68aknBFpGL2+/IcogTcaydJEgaTmOpDg==", + "dev": true, + "license": "MIT" + }, + "node_modules/entities": { + "version": "4.5.0", + "resolved": "https://registry.npmjs.org/entities/-/entities-4.5.0.tgz", + "integrity": "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw==", + "dev": true, + "license": "BSD-2-Clause", + "engines": { + "node": ">=0.12" + }, + "funding": { + "url": "https://github.com/fb55/entities?sponsor=1" + } + }, + "node_modules/esbuild": { + "version": "0.21.5", + "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.21.5.tgz", + "integrity": "sha512-mg3OPMV4hXywwpoDxu3Qda5xCKQi+vCTZq8S9J/EpkhB2HzKXq4SNFZE3+NK93JYxc8VMSep+lOUSC/RVKaBqw==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "bin": { + "esbuild": "bin/esbuild" + }, + "engines": { + "node": ">=12" + }, + "optionalDependencies": { + "@esbuild/aix-ppc64": "0.21.5", + "@esbuild/android-arm": "0.21.5", + "@esbuild/android-arm64": "0.21.5", + "@esbuild/android-x64": "0.21.5", + "@esbuild/darwin-arm64": "0.21.5", + "@esbuild/darwin-x64": "0.21.5", + "@esbuild/freebsd-arm64": "0.21.5", + "@esbuild/freebsd-x64": "0.21.5", + "@esbuild/linux-arm": "0.21.5", + "@esbuild/linux-arm64": "0.21.5", + "@esbuild/linux-ia32": "0.21.5", + "@esbuild/linux-loong64": "0.21.5", + "@esbuild/linux-mips64el": "0.21.5", + "@esbuild/linux-ppc64": "0.21.5", + "@esbuild/linux-riscv64": "0.21.5", + "@esbuild/linux-s390x": "0.21.5", + "@esbuild/linux-x64": "0.21.5", + "@esbuild/netbsd-x64": "0.21.5", + "@esbuild/openbsd-x64": "0.21.5", + "@esbuild/sunos-x64": "0.21.5", + "@esbuild/win32-arm64": "0.21.5", + "@esbuild/win32-ia32": "0.21.5", + "@esbuild/win32-x64": "0.21.5" + } + }, + "node_modules/estree-walker": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/estree-walker/-/estree-walker-2.0.2.tgz", + "integrity": "sha512-Rfkk/Mp/DL7JVje3u18FxFujQlTNR2q6QfMSMB7AvCBx91NGj/ba3kCfza0f6dVDbw7YlRf/nDrn7pQrCCyQ/w==", + "dev": true, + "license": "MIT" + }, + "node_modules/focus-trap": { + "version": "7.6.5", + "resolved": "https://registry.npmjs.org/focus-trap/-/focus-trap-7.6.5.tgz", + "integrity": "sha512-7Ke1jyybbbPZyZXFxEftUtxFGLMpE2n6A+z//m4CRDlj0hW+o3iYSmh8nFlYMurOiJVDmJRilUQtJr08KfIxlg==", + "dev": true, + "license": "MIT", + "dependencies": { + "tabbable": "^6.2.0" + } + }, + "node_modules/fsevents": { + "version": "2.3.3", + "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz", + "integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^8.16.0 || ^10.6.0 || >=11.0.0" + } + }, + "node_modules/hast-util-to-html": { + "version": "9.0.5", + "resolved": "https://registry.npmjs.org/hast-util-to-html/-/hast-util-to-html-9.0.5.tgz", + "integrity": "sha512-OguPdidb+fbHQSU4Q4ZiLKnzWo8Wwsf5bZfbvu7//a9oTYoqD/fWpe96NuHkoS9h0ccGOTe0C4NGXdtS0iObOw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/hast": "^3.0.0", + "@types/unist": "^3.0.0", + "ccount": "^2.0.0", + "comma-separated-tokens": "^2.0.0", + "hast-util-whitespace": "^3.0.0", + "html-void-elements": "^3.0.0", + "mdast-util-to-hast": "^13.0.0", + "property-information": "^7.0.0", + "space-separated-tokens": "^2.0.0", + "stringify-entities": "^4.0.0", + "zwitch": "^2.0.4" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/unified" + } + }, + "node_modules/hast-util-whitespace": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/hast-util-whitespace/-/hast-util-whitespace-3.0.0.tgz", + "integrity": "sha512-88JUN06ipLwsnv+dVn+OIYOvAuvBMy/Qoi6O7mQHxdPXpjy+Cd6xRkWwux7DKO+4sYILtLBRIKgsdpS2gQc7qw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/hast": "^3.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/unified" + } + }, + "node_modules/hookable": { + "version": "5.5.3", + "resolved": "https://registry.npmjs.org/hookable/-/hookable-5.5.3.tgz", + "integrity": "sha512-Yc+BQe8SvoXH1643Qez1zqLRmbA5rCL+sSmk6TVos0LWVfNIB7PGncdlId77WzLGSIB5KaWgTaNTs2lNVEI6VQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/html-void-elements": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/html-void-elements/-/html-void-elements-3.0.0.tgz", + "integrity": "sha512-bEqo66MRXsUGxWHV5IP0PUiAWwoEjba4VCzg0LjFJBpchPaTfyfCKTG6bc5F8ucKec3q5y6qOdGyYTSBEvhCrg==", + "dev": true, + "license": "MIT", + "funding": { + "type": "github", + "url": "https://github.com/sponsors/wooorm" + } + }, + "node_modules/is-what": { + "version": "4.1.16", + "resolved": "https://registry.npmjs.org/is-what/-/is-what-4.1.16.tgz", + "integrity": "sha512-ZhMwEosbFJkA0YhFnNDgTM4ZxDRsS6HqTo7qsZM08fehyRYIYa0yHu5R6mgo1n/8MgaPBXiPimPD77baVFYg+A==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=12.13" + }, + "funding": { + "url": "https://github.com/sponsors/mesqueeb" + } + }, + "node_modules/magic-string": { + "version": "0.30.17", + "resolved": "https://registry.npmjs.org/magic-string/-/magic-string-0.30.17.tgz", + "integrity": "sha512-sNPKHvyjVf7gyjwS4xGTaW/mCnF8wnjtifKBEhxfZ7E/S8tQ0rssrwGNn6q8JH/ohItJfSQp9mBtQYuTlH5QnA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jridgewell/sourcemap-codec": "^1.5.0" + } + }, + "node_modules/mark.js": { + "version": "8.11.1", + "resolved": "https://registry.npmjs.org/mark.js/-/mark.js-8.11.1.tgz", + "integrity": "sha512-1I+1qpDt4idfgLQG+BNWmrqku+7/2bi5nLf4YwF8y8zXvmfiTBY3PV3ZibfrjBueCByROpuBjLLFCajqkgYoLQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/mdast-util-to-hast": { + "version": "13.2.0", + "resolved": "https://registry.npmjs.org/mdast-util-to-hast/-/mdast-util-to-hast-13.2.0.tgz", + "integrity": "sha512-QGYKEuUsYT9ykKBCMOEDLsU5JRObWQusAolFMeko/tYPufNkRffBAQjIE+99jbA87xv6FgmjLtwjh9wBWajwAA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/hast": "^3.0.0", + "@types/mdast": "^4.0.0", + "@ungap/structured-clone": "^1.0.0", + "devlop": "^1.0.0", + "micromark-util-sanitize-uri": "^2.0.0", + "trim-lines": "^3.0.0", + "unist-util-position": "^5.0.0", + "unist-util-visit": "^5.0.0", + "vfile": "^6.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/unified" + } + }, + "node_modules/micromark-util-character": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/micromark-util-character/-/micromark-util-character-2.1.1.tgz", + "integrity": "sha512-wv8tdUTJ3thSFFFJKtpYKOYiGP2+v96Hvk4Tu8KpCAsTMs6yi+nVmGh1syvSCsaxz45J6Jbw+9DD6g97+NV67Q==", + "dev": true, + "funding": [ + { + "type": "GitHub Sponsors", + "url": "https://github.com/sponsors/unifiedjs" + }, + { + "type": "OpenCollective", + "url": "https://opencollective.com/unified" + } + ], + "license": "MIT", + "dependencies": { + "micromark-util-symbol": "^2.0.0", + "micromark-util-types": "^2.0.0" + } + }, + "node_modules/micromark-util-encode": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/micromark-util-encode/-/micromark-util-encode-2.0.1.tgz", + "integrity": "sha512-c3cVx2y4KqUnwopcO9b/SCdo2O67LwJJ/UyqGfbigahfegL9myoEFoDYZgkT7f36T0bLrM9hZTAaAyH+PCAXjw==", + "dev": true, + "funding": [ + { + "type": "GitHub Sponsors", + "url": "https://github.com/sponsors/unifiedjs" + }, + { + "type": "OpenCollective", + "url": "https://opencollective.com/unified" + } + ], + "license": "MIT" + }, + "node_modules/micromark-util-sanitize-uri": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/micromark-util-sanitize-uri/-/micromark-util-sanitize-uri-2.0.1.tgz", + "integrity": "sha512-9N9IomZ/YuGGZZmQec1MbgxtlgougxTodVwDzzEouPKo3qFWvymFHWcnDi2vzV1ff6kas9ucW+o3yzJK9YB1AQ==", + "dev": true, + "funding": [ + { + "type": "GitHub Sponsors", + "url": "https://github.com/sponsors/unifiedjs" + }, + { + "type": "OpenCollective", + "url": "https://opencollective.com/unified" + } + ], + "license": "MIT", + "dependencies": { + "micromark-util-character": "^2.0.0", + "micromark-util-encode": "^2.0.0", + "micromark-util-symbol": "^2.0.0" + } + }, + "node_modules/micromark-util-symbol": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/micromark-util-symbol/-/micromark-util-symbol-2.0.1.tgz", + "integrity": "sha512-vs5t8Apaud9N28kgCrRUdEed4UJ+wWNvicHLPxCa9ENlYuAY31M0ETy5y1vA33YoNPDFTghEbnh6efaE8h4x0Q==", + "dev": true, + "funding": [ + { + "type": "GitHub Sponsors", + "url": "https://github.com/sponsors/unifiedjs" + }, + { + "type": "OpenCollective", + "url": "https://opencollective.com/unified" + } + ], + "license": "MIT" + }, + "node_modules/micromark-util-types": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/micromark-util-types/-/micromark-util-types-2.0.2.tgz", + "integrity": "sha512-Yw0ECSpJoViF1qTU4DC6NwtC4aWGt1EkzaQB8KPPyCRR8z9TWeV0HbEFGTO+ZY1wB22zmxnJqhPyTpOVCpeHTA==", + "dev": true, + "funding": [ + { + "type": "GitHub Sponsors", + "url": "https://github.com/sponsors/unifiedjs" + }, + { + "type": "OpenCollective", + "url": "https://opencollective.com/unified" + } + ], + "license": "MIT" + }, + "node_modules/minisearch": { + "version": "7.1.2", + "resolved": "https://registry.npmjs.org/minisearch/-/minisearch-7.1.2.tgz", + "integrity": "sha512-R1Pd9eF+MD5JYDDSPAp/q1ougKglm14uEkPMvQ/05RGmx6G9wvmLTrTI/Q5iPNJLYqNdsDQ7qTGIcNWR+FrHmA==", + "dev": true, + "license": "MIT" + }, + "node_modules/mitt": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/mitt/-/mitt-3.0.1.tgz", + "integrity": "sha512-vKivATfr97l2/QBCYAkXYDbrIWPM2IIKEl7YPhjCvKlG3kE2gm+uBo6nEXK3M5/Ffh/FLpKExzOQ3JJoJGFKBw==", + "dev": true, + "license": "MIT" + }, + "node_modules/nanoid": { + "version": "3.3.11", + "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.11.tgz", + "integrity": "sha512-N8SpfPUnUp1bK+PMYW8qSWdl9U+wwNWI4QKxOYDy9JAro3WMX7p2OeVRF9v+347pnakNevPmiHhNmZ2HbFA76w==", + "dev": true, + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/ai" + } + ], + "license": "MIT", + "bin": { + "nanoid": "bin/nanoid.cjs" + }, + "engines": { + "node": "^10 || ^12 || ^13.7 || ^14 || >=15.0.1" + } + }, + "node_modules/oniguruma-to-es": { + "version": "3.1.1", + "resolved": "https://registry.npmjs.org/oniguruma-to-es/-/oniguruma-to-es-3.1.1.tgz", + "integrity": "sha512-bUH8SDvPkH3ho3dvwJwfonjlQ4R80vjyvrU8YpxuROddv55vAEJrTuCuCVUhhsHbtlD9tGGbaNApGQckXhS8iQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "emoji-regex-xs": "^1.0.0", + "regex": "^6.0.1", + "regex-recursion": "^6.0.2" + } + }, + "node_modules/perfect-debounce": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/perfect-debounce/-/perfect-debounce-1.0.0.tgz", + "integrity": "sha512-xCy9V055GLEqoFaHoC1SoLIaLmWctgCUaBaWxDZ7/Zx4CTyX7cJQLJOok/orfjZAh9kEYpjJa4d0KcJmCbctZA==", + "dev": true, + "license": "MIT" + }, + "node_modules/picocolors": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz", + "integrity": "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==", + "dev": true, + "license": "ISC" + }, + "node_modules/postcss": { + "version": "8.5.6", + "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.6.tgz", + "integrity": "sha512-3Ybi1tAuwAP9s0r1UQ2J4n5Y0G05bJkpUIO0/bI9MhwmD70S5aTWbXGBwxHrelT+XM1k6dM0pk+SwNkpTRN7Pg==", + "dev": true, + "funding": [ + { + "type": "opencollective", + "url": "https://opencollective.com/postcss/" + }, + { + "type": "tidelift", + "url": "https://tidelift.com/funding/github/npm/postcss" + }, + { + "type": "github", + "url": "https://github.com/sponsors/ai" + } + ], + "license": "MIT", + "dependencies": { + "nanoid": "^3.3.11", + "picocolors": "^1.1.1", + "source-map-js": "^1.2.1" + }, + "engines": { + "node": "^10 || ^12 || >=14" + } + }, + "node_modules/preact": { + "version": "10.27.0", + "resolved": "https://registry.npmjs.org/preact/-/preact-10.27.0.tgz", + "integrity": "sha512-/DTYoB6mwwgPytiqQTh/7SFRL98ZdiD8Sk8zIUVOxtwq4oWcwrcd1uno9fE/zZmUaUrFNYzbH14CPebOz9tZQw==", + "dev": true, + "license": "MIT", + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/preact" + } + }, + "node_modules/property-information": { + "version": "7.1.0", + "resolved": "https://registry.npmjs.org/property-information/-/property-information-7.1.0.tgz", + "integrity": "sha512-TwEZ+X+yCJmYfL7TPUOcvBZ4QfoT5YenQiJuX//0th53DE6w0xxLEtfK3iyryQFddXuvkIk51EEgrJQ0WJkOmQ==", + "dev": true, + "license": "MIT", + "funding": { + "type": "github", + "url": "https://github.com/sponsors/wooorm" + } + }, + "node_modules/regex": { + "version": "6.0.1", + "resolved": "https://registry.npmjs.org/regex/-/regex-6.0.1.tgz", + "integrity": "sha512-uorlqlzAKjKQZ5P+kTJr3eeJGSVroLKoHmquUj4zHWuR+hEyNqlXsSKlYYF5F4NI6nl7tWCs0apKJ0lmfsXAPA==", + "dev": true, + "license": "MIT", + "dependencies": { + "regex-utilities": "^2.3.0" + } + }, + "node_modules/regex-recursion": { + "version": "6.0.2", + "resolved": "https://registry.npmjs.org/regex-recursion/-/regex-recursion-6.0.2.tgz", + "integrity": "sha512-0YCaSCq2VRIebiaUviZNs0cBz1kg5kVS2UKUfNIx8YVs1cN3AV7NTctO5FOKBA+UT2BPJIWZauYHPqJODG50cg==", + "dev": true, + "license": "MIT", + "dependencies": { + "regex-utilities": "^2.3.0" + } + }, + "node_modules/regex-utilities": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/regex-utilities/-/regex-utilities-2.3.0.tgz", + "integrity": "sha512-8VhliFJAWRaUiVvREIiW2NXXTmHs4vMNnSzuJVhscgmGav3g9VDxLrQndI3dZZVVdp0ZO/5v0xmX516/7M9cng==", + "dev": true, + "license": "MIT" + }, + "node_modules/rfdc": { + "version": "1.4.1", + "resolved": "https://registry.npmjs.org/rfdc/-/rfdc-1.4.1.tgz", + "integrity": "sha512-q1b3N5QkRUWUl7iyylaaj3kOpIT0N2i9MqIEQXP73GVsN9cw3fdx8X63cEmWhJGi2PPCF23Ijp7ktmd39rawIA==", + "dev": true, + "license": "MIT" + }, + "node_modules/rollup": { + "version": "4.46.2", + "resolved": "https://registry.npmjs.org/rollup/-/rollup-4.46.2.tgz", + "integrity": "sha512-WMmLFI+Boh6xbop+OAGo9cQ3OgX9MIg7xOQjn+pTCwOkk+FNDAeAemXkJ3HzDJrVXleLOFVa1ipuc1AmEx1Dwg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/estree": "1.0.8" + }, + "bin": { + "rollup": "dist/bin/rollup" + }, + "engines": { + "node": ">=18.0.0", + "npm": ">=8.0.0" + }, + "optionalDependencies": { + "@rollup/rollup-android-arm-eabi": "4.46.2", + "@rollup/rollup-android-arm64": "4.46.2", + "@rollup/rollup-darwin-arm64": "4.46.2", + "@rollup/rollup-darwin-x64": "4.46.2", + "@rollup/rollup-freebsd-arm64": "4.46.2", + "@rollup/rollup-freebsd-x64": "4.46.2", + "@rollup/rollup-linux-arm-gnueabihf": "4.46.2", + "@rollup/rollup-linux-arm-musleabihf": "4.46.2", + "@rollup/rollup-linux-arm64-gnu": "4.46.2", + "@rollup/rollup-linux-arm64-musl": "4.46.2", + "@rollup/rollup-linux-loongarch64-gnu": "4.46.2", + "@rollup/rollup-linux-ppc64-gnu": "4.46.2", + "@rollup/rollup-linux-riscv64-gnu": "4.46.2", + "@rollup/rollup-linux-riscv64-musl": "4.46.2", + "@rollup/rollup-linux-s390x-gnu": "4.46.2", + "@rollup/rollup-linux-x64-gnu": "4.46.2", + "@rollup/rollup-linux-x64-musl": "4.46.2", + "@rollup/rollup-win32-arm64-msvc": "4.46.2", + "@rollup/rollup-win32-ia32-msvc": "4.46.2", + "@rollup/rollup-win32-x64-msvc": "4.46.2", + "fsevents": "~2.3.2" + } + }, + "node_modules/search-insights": { + "version": "2.17.3", + "resolved": "https://registry.npmjs.org/search-insights/-/search-insights-2.17.3.tgz", + "integrity": "sha512-RQPdCYTa8A68uM2jwxoY842xDhvx3E5LFL1LxvxCNMev4o5mLuokczhzjAgGwUZBAmOKZknArSxLKmXtIi2AxQ==", + "dev": true, + "license": "MIT", + "peer": true + }, + "node_modules/shiki": { + "version": "2.5.0", + "resolved": "https://registry.npmjs.org/shiki/-/shiki-2.5.0.tgz", + "integrity": "sha512-mI//trrsaiCIPsja5CNfsyNOqgAZUb6VpJA+340toL42UpzQlXpwRV9nch69X6gaUxrr9kaOOa6e3y3uAkGFxQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@shikijs/core": "2.5.0", + "@shikijs/engine-javascript": "2.5.0", + "@shikijs/engine-oniguruma": "2.5.0", + "@shikijs/langs": "2.5.0", + "@shikijs/themes": "2.5.0", + "@shikijs/types": "2.5.0", + "@shikijs/vscode-textmate": "^10.0.2", + "@types/hast": "^3.0.4" + } + }, + "node_modules/source-map-js": { + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz", + "integrity": "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==", + "dev": true, + "license": "BSD-3-Clause", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/space-separated-tokens": { + "version": "2.0.2", + "resolved": "https://registry.npmjs.org/space-separated-tokens/-/space-separated-tokens-2.0.2.tgz", + "integrity": "sha512-PEGlAwrG8yXGXRjW32fGbg66JAlOAwbObuqVoJpv/mRgoWDQfgH1wDPvtzWyUSNAXBGSk8h755YDbbcEy3SH2Q==", + "dev": true, + "license": "MIT", + "funding": { + "type": "github", + "url": "https://github.com/sponsors/wooorm" + } + }, + "node_modules/speakingurl": { + "version": "14.0.1", + "resolved": "https://registry.npmjs.org/speakingurl/-/speakingurl-14.0.1.tgz", + "integrity": "sha512-1POYv7uv2gXoyGFpBCmpDVSNV74IfsWlDW216UPjbWufNf+bSU6GdbDsxdcxtfwb4xlI3yxzOTKClUosxARYrQ==", + "dev": true, + "license": "BSD-3-Clause", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/stringify-entities": { + "version": "4.0.4", + "resolved": "https://registry.npmjs.org/stringify-entities/-/stringify-entities-4.0.4.tgz", + "integrity": "sha512-IwfBptatlO+QCJUo19AqvrPNqlVMpW9YEL2LIVY+Rpv2qsjCGxaDLNRgeGsQWJhfItebuJhsGSLjaBbNSQ+ieg==", + "dev": true, + "license": "MIT", + "dependencies": { + "character-entities-html4": "^2.0.0", + "character-entities-legacy": "^3.0.0" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/wooorm" + } + }, + "node_modules/superjson": { + "version": "2.2.2", + "resolved": "https://registry.npmjs.org/superjson/-/superjson-2.2.2.tgz", + "integrity": "sha512-5JRxVqC8I8NuOUjzBbvVJAKNM8qoVuH0O77h4WInc/qC2q5IreqKxYwgkga3PfA22OayK2ikceb/B26dztPl+Q==", + "dev": true, + "license": "MIT", + "dependencies": { + "copy-anything": "^3.0.2" + }, + "engines": { + "node": ">=16" + } + }, + "node_modules/tabbable": { + "version": "6.2.0", + "resolved": "https://registry.npmjs.org/tabbable/-/tabbable-6.2.0.tgz", + "integrity": "sha512-Cat63mxsVJlzYvN51JmVXIgNoUokrIaT2zLclCXjRd8boZ0004U4KCs/sToJ75C6sdlByWxpYnb5Boif1VSFew==", + "dev": true, + "license": "MIT" + }, + "node_modules/trim-lines": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/trim-lines/-/trim-lines-3.0.1.tgz", + "integrity": "sha512-kRj8B+YHZCc9kQYdWfJB2/oUl9rA99qbowYYBtr4ui4mZyAQ2JpvVBd/6U2YloATfqBhBTSMhTpgBHtU0Mf3Rg==", + "dev": true, + "license": "MIT", + "funding": { + "type": "github", + "url": "https://github.com/sponsors/wooorm" + } + }, + "node_modules/unist-util-is": { + "version": "6.0.0", + "resolved": "https://registry.npmjs.org/unist-util-is/-/unist-util-is-6.0.0.tgz", + "integrity": "sha512-2qCTHimwdxLfz+YzdGfkqNlH0tLi9xjTnHddPmJwtIG9MGsdbutfTc4P+haPD7l7Cjxf/WZj+we5qfVPvvxfYw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/unist": "^3.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/unified" + } + }, + "node_modules/unist-util-position": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/unist-util-position/-/unist-util-position-5.0.0.tgz", + "integrity": "sha512-fucsC7HjXvkB5R3kTCO7kUjRdrS0BJt3M/FPxmHMBOm8JQi2BsHAHFsy27E0EolP8rp0NzXsJ+jNPyDWvOJZPA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/unist": "^3.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/unified" + } + }, + "node_modules/unist-util-stringify-position": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/unist-util-stringify-position/-/unist-util-stringify-position-4.0.0.tgz", + "integrity": "sha512-0ASV06AAoKCDkS2+xw5RXJywruurpbC4JZSm7nr7MOt1ojAzvyyaO+UxZf18j8FCF6kmzCZKcAgN/yu2gm2XgQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/unist": "^3.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/unified" + } + }, + "node_modules/unist-util-visit": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/unist-util-visit/-/unist-util-visit-5.0.0.tgz", + "integrity": "sha512-MR04uvD+07cwl/yhVuVWAtw+3GOR/knlL55Nd/wAdblk27GCVt3lqpTivy/tkJcZoNPzTwS1Y+KMojlLDhoTzg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/unist": "^3.0.0", + "unist-util-is": "^6.0.0", + "unist-util-visit-parents": "^6.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/unified" + } + }, + "node_modules/unist-util-visit-parents": { + "version": "6.0.1", + "resolved": "https://registry.npmjs.org/unist-util-visit-parents/-/unist-util-visit-parents-6.0.1.tgz", + "integrity": "sha512-L/PqWzfTP9lzzEa6CKs0k2nARxTdZduw3zyh8d2NVBnsyvHjSX4TWse388YrrQKbvI8w20fGjGlhgT96WwKykw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/unist": "^3.0.0", + "unist-util-is": "^6.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/unified" + } + }, + "node_modules/vfile": { + "version": "6.0.3", + "resolved": "https://registry.npmjs.org/vfile/-/vfile-6.0.3.tgz", + "integrity": "sha512-KzIbH/9tXat2u30jf+smMwFCsno4wHVdNmzFyL+T/L3UGqqk6JKfVqOFOZEpZSHADH1k40ab6NUIXZq422ov3Q==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/unist": "^3.0.0", + "vfile-message": "^4.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/unified" + } + }, + "node_modules/vfile-message": { + "version": "4.0.3", + "resolved": "https://registry.npmjs.org/vfile-message/-/vfile-message-4.0.3.tgz", + "integrity": "sha512-QTHzsGd1EhbZs4AsQ20JX1rC3cOlt/IWJruk893DfLRr57lcnOeMaWG4K0JrRta4mIJZKth2Au3mM3u03/JWKw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/unist": "^3.0.0", + "unist-util-stringify-position": "^4.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/unified" + } + }, + "node_modules/vite": { + "version": "5.4.19", + "resolved": "https://registry.npmjs.org/vite/-/vite-5.4.19.tgz", + "integrity": "sha512-qO3aKv3HoQC8QKiNSTuUM1l9o/XX3+c+VTgLHbJWHZGeTPVAg2XwazI9UWzoxjIJCGCV2zU60uqMzjeLZuULqA==", + "dev": true, + "license": "MIT", + "dependencies": { + "esbuild": "^0.21.3", + "postcss": "^8.4.43", + "rollup": "^4.20.0" + }, + "bin": { + "vite": "bin/vite.js" + }, + "engines": { + "node": "^18.0.0 || >=20.0.0" + }, + "funding": { + "url": "https://github.com/vitejs/vite?sponsor=1" + }, + "optionalDependencies": { + "fsevents": "~2.3.3" + }, + "peerDependencies": { + "@types/node": "^18.0.0 || >=20.0.0", + "less": "*", + "lightningcss": "^1.21.0", + "sass": "*", + "sass-embedded": "*", + "stylus": "*", + "sugarss": "*", + "terser": "^5.4.0" + }, + "peerDependenciesMeta": { + "@types/node": { + "optional": true + }, + "less": { + "optional": true + }, + "lightningcss": { + "optional": true + }, + "sass": { + "optional": true + }, + "sass-embedded": { + "optional": true + }, + "stylus": { + "optional": true + }, + "sugarss": { + "optional": true + }, + "terser": { + "optional": true + } + } + }, + "node_modules/vitepress": { + "version": "1.6.3", + "resolved": "https://registry.npmjs.org/vitepress/-/vitepress-1.6.3.tgz", + "integrity": "sha512-fCkfdOk8yRZT8GD9BFqusW3+GggWYZ/rYncOfmgcDtP3ualNHCAg+Robxp2/6xfH1WwPHtGpPwv7mbA3qomtBw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@docsearch/css": "3.8.2", + "@docsearch/js": "3.8.2", + "@iconify-json/simple-icons": "^1.2.21", + "@shikijs/core": "^2.1.0", + "@shikijs/transformers": "^2.1.0", + "@shikijs/types": "^2.1.0", + "@types/markdown-it": "^14.1.2", + "@vitejs/plugin-vue": "^5.2.1", + "@vue/devtools-api": "^7.7.0", + "@vue/shared": "^3.5.13", + "@vueuse/core": "^12.4.0", + "@vueuse/integrations": "^12.4.0", + "focus-trap": "^7.6.4", + "mark.js": "8.11.1", + "minisearch": "^7.1.1", + "shiki": "^2.1.0", + "vite": "^5.4.14", + "vue": "^3.5.13" + }, + "bin": { + "vitepress": "bin/vitepress.js" + }, + "peerDependencies": { + "markdown-it-mathjax3": "^4", + "postcss": "^8" + }, + "peerDependenciesMeta": { + "markdown-it-mathjax3": { + "optional": true + }, + "postcss": { + "optional": true + } + } + }, + "node_modules/vue": { + "version": "3.5.18", + "resolved": "https://registry.npmjs.org/vue/-/vue-3.5.18.tgz", + "integrity": "sha512-7W4Y4ZbMiQ3SEo+m9lnoNpV9xG7QVMLa+/0RFwwiAVkeYoyGXqWE85jabU4pllJNUzqfLShJ5YLptewhCWUgNA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@vue/compiler-dom": "3.5.18", + "@vue/compiler-sfc": "3.5.18", + "@vue/runtime-dom": "3.5.18", + "@vue/server-renderer": "3.5.18", + "@vue/shared": "3.5.18" + }, + "peerDependencies": { + "typescript": "*" + }, + "peerDependenciesMeta": { + "typescript": { + "optional": true + } + } + }, + "node_modules/zwitch": { + "version": "2.0.4", + "resolved": "https://registry.npmjs.org/zwitch/-/zwitch-2.0.4.tgz", + "integrity": "sha512-bXE4cR/kVZhKZX/RjPEflHaKVhUVl85noU3v6b8apfQEc1x4A+zBxjZ4lN8LqGd6WZ3dl98pY4o717VFmoPp+A==", + "dev": true, + "license": "MIT", + "funding": { + "type": "github", + "url": "https://github.com/sponsors/wooorm" + } + } + } +} diff --git a/doc/package.json b/doc/package.json new file mode 100644 index 000000000..6c16a1043 --- /dev/null +++ b/doc/package.json @@ -0,0 +1,10 @@ +{ + "devDependencies": { + "vitepress": "^1.6.3" + }, + "scripts": { + "docs:dev": "vitepress dev docs", + "docs:build": "vitepress build docs", + "docs:preview": "vitepress preview docs" + } +} \ No newline at end of file diff --git a/frontend/app/[locale]/chat/internal/chatInterface.tsx b/frontend/app/[locale]/chat/internal/chatInterface.tsx index 4c3a5064d..43cc4b88e 100644 --- a/frontend/app/[locale]/chat/internal/chatInterface.tsx +++ b/frontend/app/[locale]/chat/internal/chatInterface.tsx @@ -903,6 +903,11 @@ export function ChatInterface() { // Trigger scroll to bottom setShouldScrollToBottom(true); + + // Reset shouldScrollToBottom after a delay to ensure scrolling completes. + setTimeout(() => { + setShouldScrollToBottom(false); + }, 1000); // Refresh history list fetchConversationList().catch(err => { @@ -934,6 +939,12 @@ export function ChatInterface() { // 缓存有内容，正常显示 setIsLoadingHistoricalConversation(false); setIsLoading(false); // 确保 isLoading 状态也被重置 + + // For cases where there are cached messages, also trigger scrolling to the bottom. + setShouldScrollToBottom(true); + setTimeout(() => { + setShouldScrollToBottom(false); + }, 1000); } } @@ -1013,6 +1024,11 @@ export function ChatInterface() { // Trigger scroll to bottom setShouldScrollToBottom(true); + + // Reset shouldScrollToBottom after a delay to ensure scrolling completes. + setTimeout(() => { + setShouldScrollToBottom(false); + }, 1000); // Refresh history list fetchConversationList().catch(err => { diff --git a/frontend/app/[locale]/chat/streaming/chatStreamMain.tsx b/frontend/app/[locale]/chat/streaming/chatStreamMain.tsx index 4e7647fe9..17454f375 100644 --- a/frontend/app/[locale]/chat/streaming/chatStreamMain.tsx +++ b/frontend/app/[locale]/chat/streaming/chatStreamMain.tsx @@ -279,11 +279,13 @@ export function ChatStreamMain({ setShowTopFade(false); } - // Control autoScroll based on user's scroll position - if (distanceToBottom < 50) { - setAutoScroll(true); - } else if (distanceToBottom > 80) { - setAutoScroll(false); + // Only if shouldScrollToBottom is false does autoScroll adjust based on user scroll position. + if (!shouldScrollToBottom) { + if (distanceToBottom < 50) { + setAutoScroll(true); + } else if (distanceToBottom > 80) { + setAutoScroll(false); + } } }; @@ -296,7 +298,7 @@ export function ChatStreamMain({ return () => { scrollAreaElement.removeEventListener('scroll', handleScroll); }; - }, []); + }, [shouldScrollToBottom]); // Scroll to bottom function const scrollToBottom = (smooth = false) => { @@ -322,17 +324,11 @@ export function ChatStreamMain({ useEffect(() => { if (shouldScrollToBottom && processedMessages.finalMessages.length > 0) { setAutoScroll(true); + scrollToBottom(false); + setTimeout(() => { scrollToBottom(false); - - setTimeout(() => { - scrollToBottom(false); - }, 300); - - setTimeout(() => { - scrollToBottom(false); - }, 800); - }, 100); + }, 300); } }, [shouldScrollToBottom, processedMessages.finalMessages.length]); @@ -345,12 +341,12 @@ export function ChatStreamMain({ const { scrollTop, scrollHeight, clientHeight } = scrollAreaElement as HTMLElement; const distanceToBottom = scrollHeight - scrollTop - clientHeight; - // Only auto-scroll if user is near the bottom (within 50px) - if (distanceToBottom < 50) { + // When shouldScrollToBottom is true, force scroll to the bottom, regardless of distance. + if (shouldScrollToBottom || distanceToBottom < 50) { scrollToBottom(); } } - }, [processedMessages.finalMessages.length, processedMessages.conversationGroups.size, autoScroll]); + }, [processedMessages.finalMessages.length, processedMessages.conversationGroups.size, autoScroll, shouldScrollToBottom]); // Scroll to bottom when task messages are updated useEffect(() => { @@ -361,12 +357,12 @@ export function ChatStreamMain({ const { scrollTop, scrollHeight, clientHeight } = scrollAreaElement as HTMLElement; const distanceToBottom = scrollHeight - scrollTop - clientHeight; - // Only auto-scroll if user is near the bottom (within 150px) - if (distanceToBottom < 150) { + // When shouldScrollToBottom is true, force scroll to the bottom, regardless of distance. + if (shouldScrollToBottom || distanceToBottom < 150) { scrollToBottom(); } } - }, [processedMessages.taskMessages.length, isStreaming, autoScroll]); + }, [processedMessages.taskMessages.length, isStreaming, autoScroll, shouldScrollToBottom]); return (
diff --git a/make/docs/Dockerfile b/make/docs/Dockerfile new file mode 100644 index 000000000..3b0301dae --- /dev/null +++ b/make/docs/Dockerfile @@ -0,0 +1,25 @@ +# 使用 Node.js 18 作为基础镜像 +FROM node:18-alpine +ARG MIRROR + +WORKDIR /app + +# 复制文档项目 +COPY doc . + +# 安装系统依赖 +RUN apk add --no-cache wget + +# 安装依赖并构建 +RUN npm add -D vitepress && \ + npm run docs:build + +# 暴露端口 +EXPOSE 4173 + +# 设置健康检查 +HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 \ + CMD wget --no-verbose --tries=1 --spider http://localhost:4173/ || exit 1 + +# 启动 VitePress 预览服务（服务构建后的静态文件） +CMD ["npm", "run", "docs:preview", "--", "--host", "0.0.0.0", "--port", "4173"] \ No newline at end of file diff --git a/sdk/README.md b/sdk/README.md deleted file mode 100644 index 8b73135a2..000000000 --- a/sdk/README.md +++ /dev/null @@ -1,413 +0,0 @@ -# Nexent - -[![English](https://img.shields.io/badge/Language-English-blue.svg)](README_EN.md) - -nexent 是一个企业级、高性能的 Agent SDK，提供了完整的智能体开发解决方案，支持分布式处理、流式输出、多模态交互和丰富的工具生态系统。 - -## 安装方式 - -### 用户安装 -如果您想使用 nexent： - -```bash -# 使用 uv 安装 (推荐) -uv add nexent - -# 或从源码安装 -uv pip install . -``` - -### 开发环境设置 -如果您是第三方 SDK 开发者： - -```bash -# 方式 1：仅安装依赖（不安装 nexent） -uv pip install -r requirements.txt - -# 方式 2：安装完整开发环境（包括 nexent） -uv pip install -e ".[dev]" # 包含所有开发工具（测试、代码质量检查等） -``` - -开发环境包含以下额外功能： -- 代码质量检查工具 (ruff) -- 测试框架 (pytest) -- 数据处理依赖 (unstructured) -- 其他开发依赖 - -## 使用方式 - -```python -import nexent -from nexent.core import MessageObserver, ProcessType -from nexent.core.agents import CoreAgent, NexentAgent -from nexent.core.models import OpenAIModel -from nexent.core.tools import ExaSearchTool, KnowledgeBaseSearchTool -``` - -### 创建基本 Agent - -```python -# 创建消息观察者 -observer = MessageObserver() - -# 创建模型（model和Agent必须使用同一个observer） -model = OpenAIModel( - observer=observer, - model_id="your-model-id", - api_key="your-api-key", - api_base="your-api-base" -) - -# 创建工具 -search_tool = ExaSearchTool(exa_api_key="your-exa-key", observer=observer, max_results=5) -kb_tool = KnowledgeBaseSearchTool(top_k=5, observer=observer) - -# 创建Agent -agent = CoreAgent( - observer=observer, - tools=[search_tool, kb_tool], - model=model, - name="my_agent", - max_steps=5 -) - -# 运行Agent -result = agent.run("你的问题") -``` - -### 使用分布式数据处理服务 - -```python -from nexent.data_process import DataProcessCore - -# 创建数据处理核心实例 -core = DataProcessCore() - -# 处理单个文档（支持内存处理） -with open("document.pdf", "rb") as f: - file_data = f.read() - -result = core.file_process( - file_data=file_data, - filename="document.pdf", - chunking_strategy="by_title" -) - -# 批量处理多个文档 -documents = ["doc1.pdf", "doc2.docx", "sheet.xlsx"] -for doc in documents: - result = core.file_process( - file_path_or_url=doc, - destination="local", - chunking_strategy="basic" - ) -``` - -### 使用向量数据库服务 - -```python -from nexent.vector_database import ElasticSearchCore -from nexent.core.models import BaseEmbedding - -# 初始化Elasticsearch核心服务 -es_core = ElasticSearchCore( - host="https://localhost:9200", - api_key="your-elasticsearch-api-key" -) - -# 创建索引并插入文档 -index_name = "company_docs" -documents = [ - {"content": "文档内容", "title": "文档标题", "metadata": {...}} -] - -# 支持混合搜索、语义搜索、精确搜索 -results = es_core.hybrid_search( - index_names=[index_name], - query_text="搜索查询", - embedding_model=embedding_model, - top_k=5 -) -``` - -## 主要特性 - -- **企业级Agent框架**：基于 SmolAgent 扩展，支持复杂业务场景 -- **分布式处理能力**： - - **异步处理**：基于 asyncio 的高性能异步架构 - - **多线程支持**：线程安全的并发处理机制 - - **Celery 友好**：专为分布式任务队列优化的设计 - - **批量操作**：支持大规模数据的批量处理和优化 -- **丰富的 Agent 工具生态**： - - **搜索工具**：EXA、Tavily、Linkup 网络搜索和本地知识库检索 - - **通信工具**：IMAP/SMTP 邮件收发功能 - - **MCP 集成**：支持 Model Context Protocol 工具集成 - - **统一规范**：所有工具遵循一致的开发标准和接口设计 -- **多模态支持**： - - **语音服务**：集成 STT & TTS，支持实时语音交互 - - **视觉模型**：支持图像理解和处理 - - **长上下文模型**：处理大规模文档和对话历史 -- **强大的数据处理能力**： - - **多格式支持**：处理 PDF、Word、Excel、HTML 等 20+ 种格式 - - **智能分块**：支持基础分块、标题分块、无分块等策略 - - **内存处理**：支持大文件的内存流式处理 - - **分布式架构**：支持任务队列管理和并行处理 -- **向量数据库集成**： - - **Elasticsearch**：企业级向量搜索和文档管理 - - **混合搜索**：结合精确匹配和语义搜索 - - **嵌入模型**：集成 Jina 等主流嵌入模型 - - **大规模优化**：支持数百万级文档的高效检索 - -## 核心组件 - -### NexentAgent - 企业级Agent框架 - -nexent 的核心是 `NexentAgent` 和 `CoreAgent` 类，提供完整的智能体解决方案： - -- **多模型支持**：支持 OpenAI、视觉语言模型、长上下文模型等 -- **MCP 集成**：无缝集成 Model Context Protocol 工具生态 -- **动态工具加载**：支持本地工具和 MCP 工具的动态创建和管理 -- **分布式执行**：基于线程池和异步架构的高性能执行引擎 -- **状态管理**：完善的任务状态追踪和错误恢复机制 - -### CoreAgent - 代码执行引擎 - -继承并增强了 SmolAgent 的 `CodeAgent`，提供以下关键能力： - -- **Python代码执行**：支持解析和执行Python代码，能够动态处理任务 -- **多语言支持**：内置中英文提示词模板，可根据需要切换语言 -- **流式输出**：通过 MessageObserver 实现模型输出的实时流式显示 -- **步骤追踪**：记录并展示Agent执行的每个步骤，便于调试和监控 -- **中断控制**：支持任务中断和优雅停止机制 -- **错误处理**：完善的错误处理机制，提高稳定性 -- **状态管理**：维护和传递执行状态，支持复杂任务的连续处理 - -CoreAgent 实现了ReAct框架的思考-行动-观察循环： -1. **思考**：使用大语言模型生成解决方案代码 -2. **行动**：执行生成的Python代码 -3. **观察**：收集执行结果和日志 -4. **重复**：根据观察结果继续思考和执行，直到任务完成 - -### MessageObserver - 流式消息处理 - -消息观察者模式的核心实现，用于处理 Agent 的流式输出： - -- **流式输出捕获**：实时捕获模型生成的token -- **过程类型区分**：根据不同的处理阶段（模型输出、代码解析、执行日志等）格式化输出 -- **多语言支持**：支持中英文输出格式 -- **统一接口**：为不同来源的消息提供统一处理方式 - -ProcessType枚举定义了以下处理阶段： -- `STEP_COUNT`: 当前执行步骤 -- `MODEL_OUTPUT_THINKING`: 模型思考过程输出 -- `MODEL_OUTPUT_CODE`: 模型代码生成输出 -- `PARSE`: 代码解析结果 -- `EXECUTION_LOGS`: 代码执行结果 -- `AGENT_NEW_RUN`: Agent基本信息 -- `FINAL_ANSWER`: 最终总结结果 -- `SEARCH_CONTENT`: 搜索结果内容 -- `PICTURE_WEB`: 网络图片处理结果 - -### 工具集 - -nexent 提供了丰富的工具生态系统，支持多种类型的任务处理。所有工具都遵循统一的开发规范，确保一致性和可扩展性。 - -#### 搜索工具 -- **ExaSearchTool**: 基于 EXA API 的高质量网络搜索工具，支持实时爬取和图像过滤 -- **TavilySearchTool**: 基于 Tavily API 的网络搜索工具，提供准确的搜索结果 -- **LinkupSearchTool**: 基于 Linkup API 的搜索工具，支持文本和图像搜索结果 -- **KnowledgeBaseSearchTool**: 本地知识库检索工具，支持混合搜索、精确匹配和语义搜索三种模式 - -#### 通信工具 -- **GetEmailTool**: 邮件获取工具，支持 IMAP 协议，可按时间范围和发件人过滤邮件 -- **SendEmailTool**: 邮件发送工具，支持 SMTP 协议，支持 HTML 格式和多收件人 - -#### 工具开发规范 - -所有工具都遵循以下设计原则： - -1. **统一架构**： - - 继承自 `smolagents.tools.Tool` 基类 - - 使用 `pydantic.Field` 进行参数管理 - - 集成 `MessageObserver` 支持流式输出 - - 内置中英文双语支持 - -2. **标准接口**： - - 必须实现 `forward()` 方法作为主要执行入口 - - 统一的 JSON 格式输入输出 - - 完善的异常处理和日志记录 - - 支持参数验证和类型检查 - -3. **命名规范**： - - 文件名：`{功能名}_tool.py` - - 类名：`{功能名}Tool` - - 属性和方法：小写字母 + 下划线 - -4. **扩展性**： - - 模块化设计，便于功能扩展 - - 支持自定义配置和环境变量 - - 提供丰富的回调和钩子机制 - -详细的工具开发规范请参考工具开发文档：[中文版](nexent/core/tools/README.md) | [English](nexent/core/tools/README_EN.md)。 - -#### 工具使用示例 - -```python -from nexent.core.tools import ExaSearchTool, KnowledgeBaseSearchTool - -# 网络搜索工具 -search_tool = ExaSearchTool( - exa_api_key="your-api-key", - observer=observer, - max_results=5 -) - -# 知识库搜索工具 -kb_tool = KnowledgeBaseSearchTool( - top_k=5, - index_names=["company_docs"], - observer=observer, - embedding_model=embedding_model, - es_core=es_client -) - -# 在Agent中使用 -agent = CoreAgent( - observer=observer, - tools=[search_tool, kb_tool], - model=model, - name="search_agent" -) -``` - -### 数据处理服务 - -基于 Unstructured IO 和 OpenPyxl 构建的企业级文档处理服务，提供分布式处理能力： - -- **多格式支持**：处理 20+ 种文档格式，包括 PDF、Word、Excel、PPT、HTML、Email 等 -- **多源处理**：支持本地文件、远程 URL、MinIO 对象存储和内存数据处理 -- **智能分块**：提供基础分块、标题分块、无分块等多种策略 -- **分布式架构**：支持大规模文档的并行处理和批量操作 -- **内存优化**：支持大文件的流式处理，减少内存占用 -- **状态追踪**：完善的任务状态管理和错误恢复机制 -- **Celery 友好**：专为分布式任务队列设计的架构 - -支持的文档格式： - -| 文档类型 | 表格支持 | 策略选项 | 处理器 | -|---------|---------|---------|---------| -| CSV/TSV文件 | 是 | 无 | OpenPyxl | -| 电子邮件 (.eml/.msg) | 否 | 无 | Unstructured | -| 文档类 (.docx/.doc/.odt/.rtf) | 是 | 无 | Unstructured | -| 表格类 (.xlsx/.xls) | 是 | 无 | OpenPyxl | -| 演示类 (.pptx/.ppt) | 是 | 无 | Unstructured | -| PDF文档 | 是 | auto/fast/hi_res/ocr_only | Unstructured | -| 图像文件 | 是 | auto/hi_res/ocr_only | Unstructured | -| HTML/XML | 否 | 无 | Unstructured | -| 纯文本/代码文件 | 否 | 无 | Unstructured | -| Markdown/ReStructured Text | 是 | 无 | Unstructured | - -使用方式： -```bash -# 启动分布式数据处理服务 -python -m nexent.data_process --host 0.0.0.0 --port 8000 --workers 3 --data-dir ./data -``` - -### 向量数据库服务 - -提供企业级的向量搜索和文档管理服务，基于 Elasticsearch 构建： - -- **多种搜索模式**： - - **精确搜索**：基于关键词的传统文本匹配 - - **语义搜索**：基于向量相似度的语义理解搜索 - - **混合搜索**：结合精确匹配和语义搜索的最佳实践 -- **嵌入模型集成**：支持 Jina、OpenAI 等主流嵌入模型 -- **大规模优化**： - - **批量操作**：支持数百万级文档的高效批量插入 - - **分布式友好**：Celery 兼容的上下文管理 - - **连接池管理**：优化的连接复用和资源管理 -- **索引管理**： - - **动态索引创建**：自动创建和配置索引结构 - - **索引统计**：实时监控索引状态和性能指标 - - **索引优化**：自动优化索引设置以提升性能 -- **安全性**：支持 API Key 认证和 SSL/TLS 加密传输 - -### 模型服务 - -提供统一的多模态 AI 模型服务： - -#### 语音服务 (STT & TTS) -- **语音识别(STT)**: 通过WebSocket连接进行实时音频转写 -- **语音合成(TTS)**: 通过WebSocket流式传输将文本转换为音频 -- **单一端口**: 两种服务在同一端口上运行，简化部署和使用 -- **流式处理**: 支持实时流式音频识别和合成，提供低延迟体验 - -#### 大语言模型 -- **OpenAI 集成**: 支持 GPT 系列模型 -- **长上下文支持**: 专门优化的长上下文处理模型 -- **流式输出**: 实时 token 流式生成 -- **多配置管理**: 支持多个模型配置的动态切换 - -#### 视觉语言模型 -- **多模态理解**: 支持图像和文本的联合理解 -- **视觉问答**: 基于图像内容的智能问答 -- **图像描述**: 自动生成图像描述和分析 - -#### 嵌入模型 -- **多提供商支持**: Jina、OpenAI 等主流嵌入服务 -- **批量处理**: 支持大规模文本的批量向量化 -- **缓存优化**: 智能缓存机制提升处理效率 - -## 使用语音服务 - -```bash -uv run python -m nexent.service.voice_service --env .env --port 8000 -``` - -更多详细使用说明，请参考各模块的专门文档。 - -## 架构优势 - -### 1. 分布式处理能力 -- **异步架构**：基于 asyncio 的高性能异步处理 -- **多线程安全**：线程安全的并发处理机制 -- **Celery 集成**：专为分布式任务队列优化 -- **批量优化**：智能批量操作减少网络开销 - -### 2. 企业级可扩展性 -- **模块化设计**：松耦合的模块架构便于扩展 -- **插件化工具**：标准化的工具接口支持快速集成 -- **配置管理**：灵活的配置系统支持多环境部署 -- **监控友好**：完善的日志和状态监控 - -### 3. 高性能优化 -- **连接池**：数据库和HTTP连接的智能复用 -- **内存管理**：大文件的流式处理和内存优化 -- **并发控制**：智能的并发限制和负载均衡 -- **缓存策略**：多层缓存提升响应速度 - -## 开发团队 - -- Shuangrui Chen -- Simeng Bian -- Tao Liu -- Jingyuan Li -- Mingchen Wan -- Yichen Xia -- Peiling Jiang -- Yu Lin - -## Future Features - -- Observer全量重构到Callback架构 -- 提供NL2SQL能力 -- Embedding模型抽象化 -- 多用户使用能力 -- 提供DeepDoc数据处理能力 -- 提供向量知识库自动化Summary与自动化识别入库 -- 提供可视化工具能力 -- 提供邮件发送、提醒发送能力 -- 提供文搜图、图搜图能力 -- 多模态对话能力 -- Ray 分布式计算集成 -- Kubernetes 原生支持 diff --git a/sdk/README_EN.md b/sdk/README_EN.md deleted file mode 100644 index 529b3a9bc..000000000 --- a/sdk/README_EN.md +++ /dev/null @@ -1,413 +0,0 @@ -# Nexent - -[![中文](https://img.shields.io/badge/Language-中文-blue.svg)](README.md) - -Nexent is an enterprise-grade, high-performance Agent SDK that provides a complete intelligent agent development solution, supporting distributed processing, streaming output, multi-modal interaction, and a rich tool ecosystem. - -## Installation - -### User Installation -If you want to use nexent: - -```bash -# Install using uv (recommended) -uv add nexent - -# Or install from source -uv pip install . -``` - -### Development Environment Setup -If you are a third-party SDK developer: - -```bash -# Method 1: Install dependencies only (without nexent) -uv pip install -r requirements.txt - -# Method 2: Install complete development environment (including nexent) -uv pip install -e ".[dev]" # Includes all development tools (testing, code quality checks, etc.) -``` - -The development environment includes the following additional features: -- Code quality checking tools (ruff) -- Testing framework (pytest) -- Data processing dependencies (unstructured) -- Other development dependencies - -## Usage - -```python -import nexent -from nexent.core import MessageObserver, ProcessType -from nexent.core.agents import CoreAgent, NexentAgent -from nexent.core.models import OpenAIModel -from nexent.core.tools import ExaSearchTool, KnowledgeBaseSearchTool -``` - -### Creating a Basic Agent - -```python -# Create message observer -observer = MessageObserver() - -# Create model (model and Agent must use the same observer) -model = OpenAIModel( - observer=observer, - model_id="your-model-id", - api_key="your-api-key", - api_base="your-api-base" -) - -# Create tools -search_tool = ExaSearchTool(exa_api_key="your-exa-key", observer=observer, max_results=5) -kb_tool = KnowledgeBaseSearchTool(top_k=5, observer=observer) - -# Create Agent -agent = CoreAgent( - observer=observer, - tools=[search_tool, kb_tool], - model=model, - name="my_agent", - max_steps=5 -) - -# Run Agent -result = agent.run("Your question") -``` - -### Using Distributed Data Processing Service - -```python -from nexent.data_process import DataProcessCore - -# Create data processing core instance -core = DataProcessCore() - -# Process single document (supports in-memory processing) -with open("document.pdf", "rb") as f: - file_data = f.read() - -result = core.file_process( - file_data=file_data, - filename="document.pdf", - chunking_strategy="by_title" -) - -# Batch process multiple documents -documents = ["doc1.pdf", "doc2.docx", "sheet.xlsx"] -for doc in documents: - result = core.file_process( - file_path_or_url=doc, - destination="local", - chunking_strategy="basic" - ) -``` - -### Using Vector Database Service - -```python -from nexent.vector_database import ElasticSearchCore -from nexent.core.models import BaseEmbedding - -# Initialize Elasticsearch core service -es_core = ElasticSearchCore( - host="https://localhost:9200", - api_key="your-elasticsearch-api-key" -) - -# Create index and insert documents -index_name = "company_docs" -documents = [ - {"content": "Document content", "title": "Document title", "metadata": {...}} -] - -# Support hybrid search, semantic search, exact search -results = es_core.hybrid_search( - index_names=[index_name], - query_text="Search query", - embedding_model=embedding_model, - top_k=5 -) -``` - -## Key Features - -- **Enterprise-grade Agent Framework**: Extended from SmolAgent, supporting complex business scenarios -- **Distributed Processing Capabilities**: - - **Asynchronous Processing**: High-performance asynchronous architecture based on asyncio - - **Multi-threading Support**: Thread-safe concurrent processing mechanisms - - **Celery-friendly**: Design optimized for distributed task queues - - **Batch Operations**: Support for large-scale data batch processing and optimization -- **Rich Agent Tool Ecosystem**: - - **Search Tools**: EXA, Tavily, Linkup web search and local knowledge base retrieval - - **Communication Tools**: IMAP/SMTP email functionality - - **MCP Integration**: Support for Model Context Protocol tool integration - - **Unified Standards**: All tools follow consistent development standards and interface design -- **Multi-modal Support**: - - **Voice Services**: Integrated STT & TTS, supporting real-time voice interaction - - **Vision Models**: Support for image understanding and processing - - **Long Context Models**: Processing large-scale documents and conversation history -- **Powerful Data Processing Capabilities**: - - **Multi-format Support**: Processing 20+ formats including PDF, Word, Excel, HTML - - **Intelligent Chunking**: Support for basic chunking, title chunking, no chunking strategies - - **Memory Processing**: Support for streaming processing of large files in memory - - **Distributed Architecture**: Support for task queue management and parallel processing -- **Vector Database Integration**: - - **Elasticsearch**: Enterprise-grade vector search and document management - - **Hybrid Search**: Combining exact matching and semantic search - - **Embedding Models**: Integration with mainstream embedding models like Jina - - **Large-scale Optimization**: Support for efficient retrieval of millions of documents - -## Core Components - -### NexentAgent - Enterprise-grade Agent Framework - -The core of nexent is the `NexentAgent` and `CoreAgent` classes, providing complete intelligent agent solutions: - -- **Multi-model Support**: Support for OpenAI, vision language models, long context models, etc. -- **MCP Integration**: Seamless integration with Model Context Protocol tool ecosystem -- **Dynamic Tool Loading**: Support for dynamic creation and management of local and MCP tools -- **Distributed Execution**: High-performance execution engine based on thread pools and asynchronous architecture -- **State Management**: Comprehensive task state tracking and error recovery mechanisms - -### CoreAgent - Code Execution Engine - -Inherits and enhances SmolAgent's `CodeAgent`, providing the following key capabilities: - -- **Python Code Execution**: Support for parsing and executing Python code, capable of dynamic task processing -- **Multi-language Support**: Built-in Chinese and English prompt templates, switchable as needed -- **Streaming Output**: Real-time streaming display of model output through MessageObserver -- **Step Tracking**: Record and display each step of Agent execution for debugging and monitoring -- **Interrupt Control**: Support for task interruption and graceful stop mechanisms -- **Error Handling**: Comprehensive error handling mechanisms to improve stability -- **State Management**: Maintain and pass execution state, supporting continuous processing of complex tasks - -CoreAgent implements the ReAct framework's think-act-observe loop: -1. **Think**: Use large language models to generate solution code -2. **Act**: Execute the generated Python code -3. **Observe**: Collect execution results and logs -4. **Repeat**: Continue thinking and executing based on observation results until the task is completed - -### MessageObserver - Streaming Message Processing - -Core implementation of the message observer pattern, used to handle Agent's streaming output: - -- **Streaming Output Capture**: Real-time capture of tokens generated by the model -- **Process Type Distinction**: Format output according to different processing stages (model output, code parsing, execution logs, etc.) -- **Multi-language Support**: Support for Chinese and English output formats -- **Unified Interface**: Provide unified processing methods for messages from different sources - -ProcessType enumeration defines the following processing stages: -- `STEP_COUNT`: Current execution step -- `MODEL_OUTPUT_THINKING`: Model thinking process output -- `MODEL_OUTPUT_CODE`: Model code generation output -- `PARSE`: Code parsing results -- `EXECUTION_LOGS`: Code execution results -- `AGENT_NEW_RUN`: Agent basic information -- `FINAL_ANSWER`: Final summary results -- `SEARCH_CONTENT`: Search result content -- `PICTURE_WEB`: Web image processing results - -### Tool Suite - -Nexent provides a rich tool ecosystem supporting multiple types of task processing. All tools follow unified development standards to ensure consistency and extensibility. - -#### Search Tools -- **ExaSearchTool**: High-quality web search tool based on EXA API, supporting real-time crawling and image filtering -- **TavilySearchTool**: Web search tool based on Tavily API, providing accurate search results -- **LinkupSearchTool**: Search tool based on Linkup API, supporting text and image search results -- **KnowledgeBaseSearchTool**: Local knowledge base retrieval tool, supporting hybrid search, exact matching, and semantic search modes - -#### Communication Tools -- **GetEmailTool**: Email retrieval tool supporting IMAP protocol, can filter emails by time range and sender -- **SendEmailTool**: Email sending tool supporting SMTP protocol, HTML format, and multiple recipients - -#### Tool Development Standards - -All tools follow these design principles: - -1. **Unified Architecture**: - - Inherit from `smolagents.tools.Tool` base class - - Use `pydantic.Field` for parameter management - - Integrate `MessageObserver` for streaming output support - - Built-in Chinese and English bilingual support - -2. **Standard Interface**: - - Must implement `forward()` method as the main execution entry - - Unified JSON format input and output - - Comprehensive exception handling and logging - - Support parameter validation and type checking - -3. **Naming Conventions**: - - File name: `{function_name}_tool.py` - - Class name: `{FunctionName}Tool` - - Attributes and methods: lowercase letters + underscores - -4. **Extensibility**: - - Modular design for easy functional extension - - Support custom configuration and environment variables - - Provide rich callback and hook mechanisms - -For detailed tool development standards, please refer to the tool development documentation: [中文版](nexent/core/tools/README.md) | [English](nexent/core/tools/README_EN.md). - -#### Tool Usage Examples - -```python -from nexent.core.tools import ExaSearchTool, KnowledgeBaseSearchTool - -# Web search tool -search_tool = ExaSearchTool( - exa_api_key="your-api-key", - observer=observer, - max_results=5 -) - -# Knowledge base search tool -kb_tool = KnowledgeBaseSearchTool( - top_k=5, - index_names=["company_docs"], - observer=observer, - embedding_model=embedding_model, - es_core=es_client -) - -# Use in Agent -agent = CoreAgent( - observer=observer, - tools=[search_tool, kb_tool], - model=model, - name="search_agent" -) -``` - -### Data Processing Service - -Enterprise-grade document processing service based on Unstructured IO and OpenPyxl, providing distributed processing capabilities: - -- **Multi-format Support**: Process 20+ document formats including PDF, Word, Excel, PPT, HTML, Email, etc. -- **Multi-source Processing**: Support local files, remote URLs, MinIO object storage, and in-memory data processing -- **Intelligent Chunking**: Provide basic chunking, title chunking, no chunking, and other strategies -- **Distributed Architecture**: Support parallel processing and batch operations for large-scale documents -- **Memory Optimization**: Support streaming processing of large files to reduce memory usage -- **State Tracking**: Comprehensive task state management and error recovery mechanisms -- **Celery-friendly**: Architecture designed for distributed task queues - -Supported document formats: - -| Document Type | Table Support | Strategy Options | Processor | -|---------|---------|---------|---------| -| CSV/TSV files | Yes | None | OpenPyxl | -| Email (.eml/.msg) | No | None | Unstructured | -| Document types (.docx/.doc/.odt/.rtf) | Yes | None | Unstructured | -| Spreadsheet types (.xlsx/.xls) | Yes | None | OpenPyxl | -| Presentation types (.pptx/.ppt) | Yes | None | Unstructured | -| PDF documents | Yes | auto/fast/hi_res/ocr_only | Unstructured | -| Image files | Yes | auto/hi_res/ocr_only | Unstructured | -| HTML/XML | No | None | Unstructured | -| Plain text/code files | No | None | Unstructured | -| Markdown/ReStructured Text | Yes | None | Unstructured | - -Usage: -```bash -# Start distributed data processing service -python -m nexent.data_process --host 0.0.0.0 --port 8000 --workers 3 --data-dir ./data -``` - -### Vector Database Service - -Provides enterprise-grade vector search and document management services based on Elasticsearch: - -- **Multiple Search Modes**: - - **Exact Search**: Traditional text matching based on keywords - - **Semantic Search**: Semantic understanding search based on vector similarity - - **Hybrid Search**: Best practices combining exact matching and semantic search -- **Embedding Model Integration**: Support for mainstream embedding models like Jina, OpenAI -- **Large-scale Optimization**: - - **Batch Operations**: Support efficient batch insertion of millions of documents - - **Distributed-friendly**: Celery-compatible context management - - **Connection Pool Management**: Optimized connection reuse and resource management -- **Index Management**: - - **Dynamic Index Creation**: Automatic creation and configuration of index structures - - **Index Statistics**: Real-time monitoring of index status and performance metrics - - **Index Optimization**: Automatic optimization of index settings to improve performance -- **Security**: Support API Key authentication and SSL/TLS encrypted transmission - -### Model Services - -Provides unified multi-modal AI model services: - -#### Voice Services (STT & TTS) -- **Speech Recognition (STT)**: Real-time audio transcription through WebSocket connections -- **Text-to-Speech (TTS)**: Streaming text-to-audio conversion through WebSocket -- **Single Port**: Both services run on the same port, simplifying deployment and usage -- **Streaming Processing**: Support real-time streaming audio recognition and synthesis with low latency - -#### Large Language Models -- **OpenAI Integration**: Support for GPT series models -- **Long Context Support**: Specially optimized long context processing models -- **Streaming Output**: Real-time token streaming generation -- **Multi-configuration Management**: Support dynamic switching of multiple model configurations - -#### Vision Language Models -- **Multi-modal Understanding**: Support joint understanding of images and text -- **Visual Q&A**: Intelligent Q&A based on image content -- **Image Description**: Automatic generation of image descriptions and analysis - -#### Embedding Models -- **Multi-provider Support**: Mainstream embedding services like Jina, OpenAI -- **Batch Processing**: Support large-scale text batch vectorization -- **Cache Optimization**: Intelligent caching mechanisms to improve processing efficiency - -## Using Voice Services - -```bash -uv run python -m nexent.service.voice_service --env .env --port 8000 -``` - -For more detailed usage instructions, please refer to the dedicated documentation for each module. - -## Architecture Advantages - -### 1. Distributed Processing Capabilities -- **Asynchronous Architecture**: High-performance asynchronous processing based on asyncio -- **Multi-thread Safety**: Thread-safe concurrent processing mechanisms -- **Celery Integration**: Optimized for distributed task queues -- **Batch Optimization**: Intelligent batch operations to reduce network overhead - -### 2. Enterprise-grade Scalability -- **Modular Design**: Loosely coupled modular architecture for easy extension -- **Plugin Tools**: Standardized tool interfaces supporting rapid integration -- **Configuration Management**: Flexible configuration system supporting multi-environment deployment -- **Monitoring-friendly**: Comprehensive logging and status monitoring - -### 3. High-performance Optimization -- **Connection Pooling**: Intelligent reuse of database and HTTP connections -- **Memory Management**: Streaming processing and memory optimization for large files -- **Concurrency Control**: Intelligent concurrency limits and load balancing -- **Caching Strategy**: Multi-layer caching to improve response speed - -## Development Team - -- Shuangrui Chen -- Simeng Bian -- Tao Liu -- Jingyuan Li -- Mingchen Wan -- Yichen Xia -- Peiling Jiang -- Yu Lin - -## Future Features - -- Complete refactoring of Observer to Callback architecture -- Provide NL2SQL capabilities -- Embedding model abstraction -- Multi-user support capabilities -- Provide DeepDoc data processing capabilities -- Provide automated Summary and automated recognition for vector knowledge bases -- Provide visualization tool capabilities -- Provide email sending and reminder capabilities -- Provide text-to-image and image-to-image capabilities -- Multi-modal conversation capabilities -- Ray distributed computing integration -- Kubernetes native support \ No newline at end of file diff --git a/test/README.md b/test/README.md deleted file mode 100644 index e03cdd8d8..000000000 --- a/test/README.md +++ /dev/null @@ -1,309 +0,0 @@ -# Unit Test Runner - -A script to run all unit tests and generate coverage reports. - -## Usage - -```bash -cd test -python run_all_tests.py -``` - -## Features - -- 🔍 **Auto-discover test files** - Automatically finds all `test_*.py` files -- 📊 **Coverage reports** - Generates console, HTML, and XML format coverage reports -- 🔧 **Auto-install dependencies** - Automatically installs `coverage` package if needed -- ✅ **Detailed output** - Shows the running status and results of each test - -## Output Files - -- `coverage_html/` - Detailed HTML format coverage report -- `coverage.xml` - XML format coverage report (for CI/CD) -- `.coverage` - Coverage data file - -## Backend Test Suite - -The `run_all_test.py` script allows you to run all backend tests with coverage reporting. - -### How to Run - -```bash -python test/backend/run_all_test.py -``` - -Or from the test/backend directory: - -```bash -python run_all_test.py -``` - -### What's Included - -The test suite automatically: - -1. Discovers and runs all test files that follow the pattern `test_*.py` in: - - `test/backend/app/` directory - - `test/backend/services/` directory - -2. Generates code coverage reports for the backend code: - - Console output with line-by-line coverage details - - HTML coverage report in `test/backend/coverage_html/` - - XML coverage report at `test/backend/coverage.xml` - -3. Provides a summary of test results: - - Total tests run - - Tests passed/failed - - Overall pass rate - - Code coverage percentage - -### Dependencies - -The script will automatically install required packages if they're not already available: -- `pytest-cov` -- `coverage` - -## Testing Framework and Approach - -### Testing Framework - -The backend tests primarily use: -- **unittest**: Python's standard unit testing framework for test organization and assertions -- **unittest.mock**: For mocking dependencies and isolating components -- **TestClient** from FastAPI: For testing API endpoints without running an actual server - -### Testing Strategy - -The test files follow these key principles: - -1. **Dependency Isolation**: - - External modules are mocked before imports to avoid real connections - - Database connections, ElasticSearch, and other external services are mocked - - No actual database operations are performed during tests - -2. **Mock-based Testing**: - - HTTP requests are simulated using FastAPI's TestClient - - External service calls are intercepted with mock objects - - No actual network connections or port bindings occur - -3. **Test Organization**: - - Tests are organized as classes inheriting from `unittest.TestCase` - - Each API endpoint or function has multiple test cases (success, failure, exception scenarios) - - Comprehensive patches are applied to isolate the code under test - -4. **API Testing**: - - API endpoints are tested for correct response codes, payload structure, and error handling - - Both synchronous and asynchronous endpoints are covered - - Streaming responses are tested through specialized test cases - -This approach ensures that tests can run in any environment without impacting real infrastructure, databases, or services. - -## Module Patching Example - -A critical technique used in the test suite is patching modules before they're imported. This prevents any real connections to external services. Here's an example from `test_file_management_app.py`: - -```python -# Dynamically determine the backend path -current_dir = os.path.dirname(os.path.abspath(__file__)) -backend_dir = os.path.abspath(os.path.join(current_dir, "../../../backend")) -sys.path.append(backend_dir) - -# Setup patches for dependencies before importing modules -patches = [ - patch('botocore.client.BaseClient._make_api_call', return_value={}), - patch('backend.database.client.MinioClient', MagicMock()), - patch('backend.database.client.db_client', MagicMock()), - patch('backend.utils.auth_utils.get_current_user_id', MagicMock(return_value=('test_user', 'test_tenant'))), - patch('backend.utils.attachment_utils.convert_image_to_text', - MagicMock(side_effect=lambda query, image_input, tenant_id, language='zh': 'mocked image text')), - patch('backend.utils.attachment_utils.convert_long_text_to_text', - MagicMock(side_effect=lambda query, file_context, tenant_id, language='zh': 'mocked text content')), - patch('httpx.AsyncClient', MagicMock()) -] - -# Start all patches -for p in patches: - p.start() - -# Now import the modules after applying all patches -from fastapi.testclient import TestClient -from fastapi import UploadFile, HTTPException -from fastapi import FastAPI -from backend.apps.file_management_app import router - -# Create a FastAPI app and include the router for testing -app = FastAPI() -app.include_router(router) -client = TestClient(app) -``` - -### Code Explanation - -1. **Path Configuration**: - - The code dynamically determines the path to the backend directory - - Adds it to Python's module search path using `sys.path.append()` - - This allows tests to import backend modules using relative paths - -2. **Patches Creation**: - - Creates a list of patches that target specific modules and functions - - `patch('botocore.client.BaseClient._make_api_call', return_value={})` prevents any AWS API calls - - `patch('backend.database.client.MinioClient', MagicMock())` replaces the file storage client - - `patch('backend.database.client.db_client', MagicMock())` mocks the database client - - Authentication functions are mocked to return predictable test values - - Utility functions for file processing are mocked with simple return values - - HTTP clients are mocked to prevent network calls - -3. **Applying Patches**: - - All patches are activated with `p.start()` before any imports happen - - This is crucial: modules are patched BEFORE they are imported - - When a module is imported, it will use the mocked dependencies instead of real ones - -4. **Module Imports**: - - Only after all patches are active, the code imports the necessary modules - - These imported modules will use mocked dependencies instead of real ones - - The router being tested is imported from the backend apps - -5. **Test Client Setup**: - - Creates a FastAPI test application and includes the router - - Sets up a TestClient to simulate HTTP requests without starting a real server - - The client will use all the mocked dependencies for testing - -### Benefits of This Approach - -This "patch-before-import" strategy ensures: - -1. **Complete Isolation**: No real external services are ever contacted -2. **No Side Effects**: Tests can't modify production databases or services -3. **Faster Tests**: No network delays or external service processing time -4. **Predictable Results**: Tests use controlled mock data for consistent results -5. **No Port Binding**: The FastAPI application never binds to a real network port - -This technique is used throughout the test suite to ensure that tests can be run safely in any environment without affecting real infrastructure. - -## Detailed Test Example - -Below is a detailed walkthrough of a sample test function (`test_list_tools_api_success`) to demonstrate the testing approach: - -```python -@patch("utils.auth_utils.get_current_user_id") -@patch("database.agent_db.query_all_tools") -async def test_list_tools_api_success(self, mock_query_all_tools, mock_get_current_user_id): - # Setup - mock_get_current_user_id.return_value = ("user123", "tenant456") - expected_tools = [{"id": 1, "name": "Tool1"}, {"id": 2, "name": "Tool2"}] - mock_query_all_tools.return_value = expected_tools - - # Execute - result = await list_tools_api(authorization="Bearer fake_token") - - # Assert - mock_get_current_user_id.assert_called_once_with("Bearer fake_token") - mock_query_all_tools.assert_called_once_with(tenant_id="tenant456") - self.assertEqual(result, expected_tools) -``` - -### Line-by-Line Explanation - -1. **Decorators**: - ```python - @patch("utils.auth_utils.get_current_user_id") - @patch("database.agent_db.query_all_tools") - ``` - - These `@patch` decorators temporarily replace the real functions with mock objects - - The patched functions are replaced in reverse order (bottom to top) - - This prevents the code from accessing real external services like authentication and databases - -2. **Function Declaration**: - ```python - async def test_list_tools_api_success(self, mock_query_all_tools, mock_get_current_user_id): - ``` - - This is an asynchronous test method using Python's `async/await` syntax - - The mock objects are automatically passed as parameters in the reverse order of the decorators - - `self` refers to the test class instance (TestCase) - -3. **Test Setup**: - ```python - mock_get_current_user_id.return_value = ("user123", "tenant456") - expected_tools = [{"id": 1, "name": "Tool1"}, {"id": 2, "name": "Tool2"}] - mock_query_all_tools.return_value = expected_tools - ``` - - Sets up the behavior of mock objects - - Defines what these mocked functions should return when called - - Creates test data that simulates the expected result from the database - -4. **Test Execution**: - ```python - result = await list_tools_api(authorization="Bearer fake_token") - ``` - - Calls the actual function being tested (`list_tools_api`) - - Passes a fake authorization token - - Uses `await` because `list_tools_api` is an asynchronous function - -5. **Test Assertions**: - ```python - mock_get_current_user_id.assert_called_once_with("Bearer fake_token") - ``` - - Verifies that the authentication function was called exactly once - - Ensures it was called with the expected parameter (the token) - - ```python - mock_query_all_tools.assert_called_once_with(tenant_id="tenant456") - ``` - - Verifies the database query function was called once - - Confirms it was called with the correct tenant ID (extracted from authentication) - - ```python - self.assertEqual(result, expected_tools) - ``` - - Verifies the function's return value matches the expected result - - Ensures the API correctly returns the tools from the database - -### Testing Philosophy - -This example demonstrates the core testing philosophy: - -1. **Isolate the unit**: Mock all external dependencies -2. **Control the environment**: Set up precise test conditions -3. **Test the interface**: Focus on inputs and outputs -4. **Verify behavior**: Check both results and interactions - -By following this pattern, tests are reliable, fast, and don't affect real systems or data. - -## Sample Output - -``` -Nexent Community - Unit Test Runner -============================================================ -Discovered Test Files: ----------------------------------------- - • backend/services/test_agent_service.py - • backend/services/test_conversation_management_service.py - • backend/services/test_knowledge_summary_service.py - -Total: 3 test files - -============================================================ -Running Unit Tests with Coverage -============================================================ - -test_get_enable_tool_id_by_agent_id ... ok -test_save_message_with_string_content ... ok -test_load_knowledge_prompts ... ok -... - -============================================================ -Coverage Report -============================================================ -Name Stmts Miss Cover Missing --------------------------------------------------------------------------------- -backend/services/agent_service.py 120 15 88% 45-50, 78-82 -backend/services/conversation_management_service.py 180 25 86% 123-128, 156-162 -backend/services/knowledge_summary_service.py 45 8 82% 35-42 --------------------------------------------------------------------------------- -TOTAL 345 48 86% - -HTML coverage report generated in: test/coverage_html -XML coverage report generated: test/coverage.xml - -============================================================ -✅ All tests passed! \ No newline at end of file