Skip to content

Commit cb8dc86

Browse files
yaozheng-fangyaozheng-fang
authored
feat: multimedia agent a2a (#54)
2 parents a743541 + aed7733 commit cb8dc86

File tree

97 files changed

+7741
-16
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

97 files changed

+7741
-16
lines changed

‎02-use-cases/multimedia/.gitignore‎

Lines changed: 191 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,191 @@
1+
# Byte-compiled / optimized / DLL files
2+
__pycache__/
3+
*.py[cod]
4+
*$py.class
5+
6+
# C extensions
7+
*.so
8+
9+
# Distribution / packaging
10+
.Python
11+
build/
12+
develop-eggs/
13+
dist/
14+
downloads/
15+
eggs/
16+
.eggs/
17+
lib/
18+
lib64/
19+
parts/
20+
sdist/
21+
var/
22+
wheels/
23+
share/python-wheels/
24+
*.egg-info/
25+
.installed.cfg
26+
*.egg
27+
MANIFEST
28+
29+
# PyInstaller
30+
# Usually these files are written by a python script from a template
31+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
32+
*.manifest
33+
*.spec
34+
35+
# Installer logs
36+
pip-log.txt
37+
pip-delete-this-directory.txt
38+
39+
# Unit test / coverage reports
40+
htmlcov/
41+
.tox/
42+
.nox/
43+
.coverage
44+
.coverage.*
45+
.cache
46+
nosetests.xml
47+
coverage.xml
48+
*.cover
49+
*.py,cover
50+
.hypothesis/
51+
.pytest_cache/
52+
cover/
53+
54+
# Translations
55+
*.mo
56+
*.pot
57+
58+
# Django stuff:
59+
*.log
60+
local_settings.py
61+
db.sqlite3
62+
db.sqlite3-journal
63+
64+
# Flask stuff:
65+
instance/
66+
.webassets-cache
67+
68+
# Scrapy stuff:
69+
.scrapy
70+
71+
# Sphinx documentation
72+
docs/_build/
73+
74+
# PyBuilder
75+
.pybuilder/
76+
target/
77+
78+
# Jupyter Notebook
79+
.ipynb_checkpoints
80+
81+
# IPython
82+
profile_default/
83+
ipython_config.py
84+
85+
# pyenv
86+
# For a library or package, you might want to ignore these files since the code is
87+
# intended to run in multiple environments; otherwise, check them in:
88+
# .python-version
89+
90+
# pipenv
91+
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
92+
# However, in case of collaboration, if having platform-specific dependencies or dependencies
93+
# having no cross-platform support, pipenv may install dependencies that don't work, or not
94+
# install all needed dependencies.
95+
#Pipfile.lock
96+
97+
# UV
98+
# Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
99+
# This is especially recommended for binary packages to ensure reproducibility, and is more
100+
# commonly ignored for libraries.
101+
uv.lock
102+
103+
# poetry
104+
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
105+
# This is especially recommended for binary packages to ensure reproducibility, and is more
106+
# commonly ignored for libraries.
107+
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
108+
#poetry.lock
109+
110+
# pdm
111+
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
112+
#pdm.lock
113+
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
114+
# in version control.
115+
# https://pdm.fming.dev/latest/usage/project/#working-with-version-control
116+
.pdm.toml
117+
.pdm-python
118+
.pdm-build/
119+
120+
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
121+
__pypackages__/
122+
123+
# Celery stuff
124+
celerybeat-schedule
125+
celerybeat.pid
126+
127+
# SageMath parsed files
128+
*.sage.py
129+
130+
# Environments
131+
.env
132+
.venv
133+
env/
134+
venv/
135+
ENV/
136+
env.bak/
137+
venv.bak/
138+
139+
# Spyder project settings
140+
.spyderproject
141+
.spyproject
142+
143+
# Rope project settings
144+
.ropeproject
145+
146+
# mkdocs documentation
147+
/site
148+
149+
# mypy
150+
.mypy_cache/
151+
.dmypy.json
152+
dmypy.json
153+
154+
# Pyre type checker
155+
.pyre/
156+
157+
# pytype static type analyzer
158+
.pytype/
159+
160+
# Cython debug symbols
161+
cython_debug/
162+
163+
# PyCharm
164+
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
165+
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
166+
# and can be added to the global gitignore or merged into this file. For a more nuclear
167+
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
168+
.idea/
169+
170+
# Ruff stuff:
171+
.ruff_cache/
172+
173+
# PyPI configuration file
174+
.pypirc
175+
/src/logs/
176+
177+
# Deepeval
178+
.deepeval/
179+
180+
# MacOS dev
181+
.DS_Store
182+
**/config.yaml
183+
184+
# docs
185+
**/node_modules/
186+
187+
# 忽略所有 .temp 目录
188+
**/.temp/
189+
190+
**/tmp-json/
191+
/app/merged_videos/

‎02-use-cases/multimedia/README.md‎

Lines changed: 202 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,202 @@
1+
# 电商营销视频生成 E-commerce Marketing Video Generation
2+
3+
## 概述
4+
5+
> 本项目通过支持 A2A 的 Multi-Agent 实现电商营销视频生成，该系统由营销策划、视频导演、评估、合成与发布 4 个 Agent 组成，提供从视频创意构思、高质量视频生成、到视频上线发布的端到端解决方案。面向需要快速、批量化生产营销短视频的电商客户或营销团队，旨在降低视频制作门槛，提高营销内容生产效率。
6+
7+
## 核心功能
8+
9+
本项目提供以下核心功能：
10+
11+
- **智能营销策划**：自动分析商品信息，生成营销策略和创意方案
12+
- **多模态内容生成**：支持文本、图片、视频等多种媒体格式的智能生成
13+
- **质量评估优化**：通过AI评估机制确保生成内容的质量
14+
- **一键发布服务**：提供完整的视频合成和发布解决方案
15+
16+
## Agent 能力
17+
18+
系统包含4个核心Agent，各司其职：
19+
20+
- **营销策划 Agent (market-agent)**：负责解析用户输入（如商品链接），进行市场分析并形成初步的营销策略和视频创意
21+
- **视频导演 Agent (director-agent)**：根据营销策略，生成具体的视频脚本、文案，并调用多模态能力（文生图、图生视频）产出视频素材
22+
- **评估 Agent (evaluate-agent)**：对生成的视频素材进行质量评估和筛选，通过自主评测机制进行抽卡优化，确保视频质量
23+
- **合成与发布 Agent (release-agent)**：将筛选后的素材合成为最终视频，并提供发布能力
24+
25+
### 费用说明
26+
27+
| 相关服务 | 描述 | 计费说明 |
28+
| --- | --- | --- |
29+
| [Doubao-Seed-1.6](https://console.volcengine.com/ark/region:ark+cn-beijing/model/detail?Id=doubao-seed-1-6) | 负责理解用户信息并转化为工具调用。 | [多种计费方式](https://www.volcengine.com/docs/82379/1099320) |
30+
| [Doubao-Seedance 1.0 pro](https://console.volcengine.com/ark/region:ark+cn-beijing/model/detail?Id=doubao-seedance-1-0-pro) | 负责将图片和文字描述转为视频。 | [多种计费方式](https://www.volcengine.com/docs/82379/1099320) |
31+
| [Doubao-Seedream 4.5 pro](https://console.volcengine.com/ark/region:ark+cn-beijing/model/detail?Id=doubao-seedream-4-5) | 负责根据文字或参考图生成图片 | [多种计费方式](https://www.volcengine.com/docs/82379/1099320) |
32+
33+
## 本地运行
34+
35+
### 环境准备
36+
37+
开始前，请确保您的开发环境满足以下要求：
38+
39+
- Python 3.10 或更高版本
40+
- VeADK 0.2.28 或更高版本
41+
- Playwright 1.55.0 或更高版本
42+
- 推荐使用 `uv` 进行依赖管理
43+
- <a target="_blank" href="https://console.volcengine.com/ark/region:ark+cn-beijing/apiKey">获取火山方舟 API KEY</a>
44+
- <a target="_blank" href="https://console.volcengine.com/iam/keymanage/">获取火山引擎 AK/SK</a>
45+
46+
### 快速入门
47+
48+
请按照以下步骤在本地部署和运行本项目。
49+
50+
#### 1. 下载代码并安装依赖
51+
52+
```bash
53+
# 克隆代码仓库
54+
git clone https://github.com/volcengine/veadk-ecommerce-video.git
55+
cd veadk-ecommerce-video
56+
57+
# 安装项目依赖
58+
uv sync
59+
```
60+
61+
#### 2. 配置环境变量
62+
63+
本项目包含多个 Agent，每个 Agent 都需要独立的配置。请参考 `config.yaml.example` 文件为每个 Agent 创建 `config.yaml` 并填入必要的密钥信息。
64+
65+
以 `director-agent` 为例：
66+
67+
```bash
68+
# 进入 director-agent 目录
69+
cd app/director-agent
70+
71+
# 复制配置文件
72+
cp config.yaml.example config.yaml
73+
```
74+
75+
然后，编辑 `config.yaml` 文件，填入您的火山方舟 API Key、火山引擎 AK/SK 等信息。请为 `market-agent`、`evaluate-agent`、`release-agent`、`multimedia-agent` 重复此操作。
76+
77+
具体配置项可参考 <a target="_blank" href="https://github.com/volcengine/veadk-python/blob/main/config.yaml.full">veadk-python config.yaml 配置文档</a>。
78+
79+
#### 3. 安装 Playwright 浏览器组件
80+
81+
`market-agent` 需要 Playwright 来解析网页内容。
82+
83+
```bash
84+
# market-agent
85+
# 安装 Playwright 浏览器依赖
86+
87+
playwright install
88+
```
89+
90+
#### 4. 启动服务
91+
92+
请按顺序启动各个 Agent 服务。
93+
94+
```bash
95+
# 激活虚拟环境
96+
# Windows (Powershell)
97+
# .\.venv\Scripts\activate
98+
# macOS/Linux
99+
# source .venv/bin/activate
100+
101+
# 启动 market-agent
102+
cd app/market-agent/src
103+
python -m uvicorn app:app --host 127.0.0.1 --port 8000 --loop asyncio
104+
105+
# 启动 director-agent
106+
cd app/director-agent/src
107+
python -m uvicorn app:app --host 127.0.0.1 --port 8001 --loop asyncio
108+
109+
# 启动 evaluate-agent
110+
cd app/evaluate-agent/src
111+
python -m uvicorn app:app --host 127.0.0.1 --port 8002 --loop asyncio
112+
113+
# 启动 release-agent
114+
cd app/release-agent/src
115+
python -m uvicorn app:app --host 127.0.0.1 --port 8003 --loop asyncio
116+
117+
# 最后启动 multimedia-agent
118+
cd app/multimedia-agent/src
119+
python -m uvicorn server:app --host 127.0.0.1 --port 8004 --loop asyncio
120+
121+
# 启动 short_link 服务
122+
cd app/short_link
123+
python -m uvicorn app:app --host 127.0.0.1 --port 8005 --loop asyncio
124+
```
125+
126+
#### 5. 测试服务
127+
128+
所有服务启动后，可运行测试脚本验证。
129+
130+
```bash
131+
python app/main.py
132+
```
133+
134+
## AgentKit 部署
135+
136+
> todo
137+
138+
### 技术实现
139+
140+
本项目核心为一套基于 VeADK 构建的多 Agent 协作框架。各 Agent 职责明确，通过 A2A (Agent-to-Agent) 通信协同工作，完成从需求理解到视频发布的完整流程。
141+
142+
- **营销策划 Agent (`market-agent`)**: 负责解析用户输入（如商品链接），进行市场分析并形成初步的营销策略和视频创意。
143+
- **视频导演 Agent (`director-agent`)**: 根据营销策略，生成具体的视频脚本、文案，并调用多模态能力（文生图、图生视频）产出视频素材。
144+
- **评估 Agent (`evaluate-agent`)**: 对生成的视频素材进行质量评估和筛选，通过自主评测机制进行抽卡优化，确保视频质量。
145+
- **合成与发布 Agent (`release-agent`)**: 将筛选后的素材合成为最终视频，并提供发布能力。
146+
147+
## 目录结构说明
148+
149+
```plaintext
150+
/
151+
├── README.md # 本文档
152+
├── backend/app/
153+
│ ├── __init__.py
154+
│ ├── director-agent/ # 视频导演Agent
155+
│ │ ├── config.yaml.example # 配置文件示例
156+
│ │ └── src/ # Agent源码
157+
│ ├── evaluate-agent/ # 评估Agent
158+
│ │ ├── config.yaml.example
159+
│ │ └── src/
160+
│ ├── main.py # 测试用主程序
161+
│ ├── market-agent/ # 营销策划Agent
162+
│ │ ├── config.yaml.example
163+
│ │ └── src/
164+
│ ├── multimedia-agent/ # 主Agent，负责协调其他Agent
165+
│ │ ├── config.yaml.example
166+
│ │ └── src/
167+
│ ├── release-agent/ # 发布Agent
168+
│ │ ├── config.yaml.example
169+
│ │ └── src/
170+
│ └── short_link/ # 视频短链接生成工具
171+
│ ├── app.py
172+
│ └── requirements.txt
173+
└── ... (其他项目文件)
174+
```
175+
176+
## 示例提示词
177+
178+
以下是一些常用的提示词示例：
179+
180+
- `根据https://...这个网站中的商品信息，给我生成一段视频`
181+
- `为这款连衣裙制作一个30秒的营销视频`
182+
- `生成一个介绍手机壳功能的短视频`
183+
- `基于这个产品链接制作促销视频`
184+
185+
## 效果展示
186+
187+
系统能够：
188+
189+
- ✅ 自动解析商品信息并生成营销策略
190+
- ✅ 创建高质量的视频脚本和分镜
191+
- ✅ 生成吸引人的营销文案
192+
- ✅ 制作专业的电商营销视频
193+
- ✅ 提供视频质量评估和优化
194+
- ✅ 支持一键发布到多个平台
195+
196+
## 常见问题
197+
198+
常见问题列表待补充。
199+
200+
## 代码许可
201+
202+
本项目采用开源许可证，详情请参考项目根目录下的 LICENSE 文件。

0 commit comments

Comments
 (0)