Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
22 changes: 11 additions & 11 deletions .asf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,9 @@ github:
- serverless-workflow
- message-bus
enabled_merge_buttons:
squash: true
merge: false
rebase: false
squash: true
merge: false
rebase: false
protected_branches:
main:
required_status_checks:
Expand All @@ -47,11 +47,11 @@ github:
dismiss_stale_reviews: true
required_approving_review_count: 0 # Temporary 0 to allow committers to merge themselves PR
notifications:
commits: [email protected]
# Send all issue emails (new, closed, comments) to issues@
issues: [email protected]
# Send new/closed PR notifications to dev@
pullrequests_status: [email protected]
# Send individual PR comments/reviews to issues@
pullrequests_comment: [email protected]
jira_options: link label worklog
commits: [email protected]
# Send all issue emails (new, closed, comments) to issues@
issues: [email protected]
# Send new/closed PR notifications to dev@
pullrequests_status: [email protected]
# Send individual PR comments/reviews to issues@
pullrequests_comment: [email protected]
jira_options: link label worklog
17 changes: 1 addition & 16 deletions .github/workflows/config/eventmesh.properties
Original file line number Diff line number Diff line change
Expand Up @@ -43,12 +43,10 @@ eventMesh.server.session.expiredInMills=60000
eventMesh.server.tcp.msgReqnumPerSecond=15000
eventMesh.server.http.msgReqnumPerSecond=15000
eventMesh.server.session.upstreamBufferSize=20

# for single event publish, maximum size allowed per event
eventMesh.server.maxEventSize=1000
# for batch event publish, maximum number of events allowed in one batch
eventMesh.server.maxEventBatchSize=10

# thread number about global scheduler
eventMesh.server.global.scheduler=5
eventMesh.server.tcp.taskHandleExecutorPoolSize=8
Expand All @@ -59,32 +57,26 @@ eventMesh.server.retry.async.pushRetryDelayInMills=500
eventMesh.server.retry.sync.pushRetryDelayInMills=500
eventMesh.server.retry.pushRetryQueueSize=10000
eventMesh.server.retry.plugin.type=default

# sleep interval between closing client of different group in server graceful shutdown
eventMesh.server.gracefulShutdown.sleepIntervalInMills=1000
eventMesh.server.rebalanceRedirect.sleepIntervalInMills=200

# ip address blacklist
eventMesh.server.blacklist.ipv4=0.0.0.0/8,127.0.0.0/8,169.254.0.0/16,255.255.255.255/32
eventMesh.server.blacklist.ipv6=::/128,::1/128,ff00::/8

########################## EventMesh Plugin Configuration ##########################
# storage plugin
eventMesh.storage.plugin.type=rocketmq

# security plugin
eventMesh.server.security.enabled=false
eventMesh.security.plugin.type=security
eventMesh.security.validation.type.token=false
eventMesh.security.publickey=

# metaStorage plugin
eventMesh.metaStorage.plugin.enabled=true
eventMesh.metaStorage.plugin.type=nacos
eventMesh.metaStorage.plugin.server-addr=127.0.0.1:8848
eventMesh.metaStorage.plugin.username=nacos
eventMesh.metaStorage.plugin.password=nacos

# metaStorage plugin: nacos
#eventMesh.metaStorage.nacos.endpoint=
#eventMesh.metaStorage.nacos.accessKey=
Expand All @@ -93,46 +85,39 @@ eventMesh.metaStorage.plugin.password=nacos
#eventMesh.metaStorage.nacos.namespace=
# The default value is half of CPU's num
#eventMesh.metaStorage.nacos.namingPollingThreadCount=5

# metaStorage plugin: zookeeper
#eventMesh.metaStorage.zookeeper.scheme=
#eventMesh.metaStorage.zookeeper.auth=
#eventMesh.metaStorage.zookeeper.connectionTimeoutMs=
#eventMesh.metaStorage.zookeeper.sessionTimeoutMs=

# Fully qualified name of org.apache.curator.RetryPolicy implementation
#eventMesh.metaStorage.zookeeper.retryPolicy.class=

# Constructor arguments for different org.apache.curator.RetryPolicy implementations
#eventMesh.metaStorage.zookeeper.retryPolicy.baseSleepTimeMs=
#eventMesh.metaStorage.zookeeper.retryPolicy.maxRetries=
#eventMesh.metaStorage.zookeeper.retryPolicy.maxSleepTimeMs=
#eventMesh.metaStorage.zookeeper.retryPolicy.retryIntervalMs=
#eventMesh.metaStorage.zookeeper.retryPolicy.nTimes=
#eventMesh.metaStorage.zookeeper.retryPolicy.sleepMsBetweenRetries=

# The TLS configuration of metaStorage plugin: consul
# keyStoreInstanceType's value can refer to com.ecwid.consul.transport.TLSConfig.KeyStoreInstanceType
#eventMesh.metaStorage.consul.tls.keyStoreInstanceType=
#eventMesh.metaStorage.consul.tls.certificatePath=
#eventMesh.metaStorage.consul.tls.certificatePassword=
#eventMesh.metaStorage.consul.tls.keyStorePath=
#eventMesh.metaStorage.consul.tls.keyStorePassword=

# metrics plugin, if you have multiple plugin, you can use ',' to split
eventMesh.metrics.plugin=prometheus

# trace plugin
eventMesh.server.trace.enabled=false
eventMesh.trace.plugin=zipkin

# webhook
# Start webhook admin service
eventMesh.webHook.admin.start=true
# Webhook event configuration storage mode. Currently, only file and nacos are supported
eventMesh.webHook.operationMode=file
# The file storage path of the file storage mode. If #{eventMeshHome} is written, it is in the EventMesh root directory
eventMesh.webHook.fileMode.filePath= #{eventMeshHome}/webhook
eventMesh.webHook.fileMode.filePath=#{eventMeshHome}/webhook
# Nacos storage mode, and the configuration naming rule is EventMesh webHook. nacosMode. {nacos native configuration key} please see the specific configuration [nacos github api](https://github.com/alibaba/nacos/blob/develop/api/src/main/java/com/alibaba/nacos/api/SystemPropertyKeyConst.java)
## Address of Nacos
eventMesh.webHook.nacosMode.serverAddr=127.0.0.1:8848
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/license.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
#

name: 'License Check'
on: [pull_request]
on: [ pull_request ]

permissions:
contents: read
Expand Down
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@

## Introduction

The EventMesh Dashboard is under development and will support functionalities such as Connection management, cluster health checks, etc. Feel free to reach out to the [EventMesh Assistant](https://github.com/apache/eventmesh?tab=readme-ov-file#community) to contribute.
The EventMesh Dashboard is under development and will support functionalities such as Connection management, cluster health checks, etc. Feel free to
reach out to the [EventMesh Assistant](https://github.com/apache/eventmesh?tab=readme-ov-file#community) to contribute.

The Dashboard for EventMesh, maintained during v1.8.0 ~ v1.10.0, is a pure frontend project located at the [Next.js Dashboard branch](https://github.com/apache/eventmesh-dashboard/tree/nextjs-dashboard).
The Dashboard for EventMesh, maintained during v1.8.0 ~ v1.10.0, is a pure frontend project located at
the [Next.js Dashboard branch](https://github.com/apache/eventmesh-dashboard/tree/nextjs-dashboard).

Weekly development meeting documents for EventMesh Dashboard: https://docs.qq.com/doc/DQmhVbklUdGNNWGZi

Expand Down Expand Up @@ -67,15 +69,15 @@ cd eventmesh-dashboard
./mvnw clean package
```

>TODO download mysql-connector-j manually
> TODO download mysql-connector-j manually

```
java -DDB_ADDRESS=$DB_ADDRESS -DDB_USERNAME=$DB_USERNAME -DDB_PASSWORD=$DB_PASSWORD -jar eventmesh-dashboard-console/target/eventmesh-dashboard-console-0.0.1-SNAPSHOT.jar
```

### Build and Run with Docker

>To be updated
> To be updated

```
cd eventmesh-dashboard
Expand Down
40 changes: 22 additions & 18 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@

## 介绍

EventMesh Dashboard 处于开发中,将支持 Connection 管理、集群健康检查等功能。欢迎联系 [EventMesh 小助手](https://github.com/apache/eventmesh?tab=readme-ov-file#community)参与贡献。
EventMesh Dashboard 处于开发中,将支持 Connection
管理、集群健康检查等功能。欢迎联系 [EventMesh 小助手](https://github.com/apache/eventmesh?tab=readme-ov-file#community)参与贡献。

EventMesh 于 v1.8.0 ~ v1.10.0 期间维护的 Dashboard 纯前端项目位于 [Next.js Dashboard 分支](https://github.com/apache/eventmesh-dashboard/tree/nextjs-dashboard)。
EventMesh 于 v1.8.0 ~ v1.10.0 期间维护的 Dashboard
纯前端项目位于 [Next.js Dashboard 分支](https://github.com/apache/eventmesh-dashboard/tree/nextjs-dashboard)。

EventMesh Dashboard 每周开发例会文档:https://docs.qq.com/doc/DQmhVbklUdGNNWGZi

Expand All @@ -16,18 +18,18 @@ EventMesh Dashboard 每周开发例会文档:https://docs.qq.com/doc/DQmhVbklU

### 环境

- JDK 8/11
- JDK 17
- Maven 3.9.x
- Spring Boot 2.7.x

### 模块介绍

1. eventmesh-dashboard-console 业务模块的代码,调用service接口
2. eventmesh-dashboard-observe 监控模块的代码
3. eventmesh-dashboard-core 对EventMesh Runtime, Meta以及相关组件的代码,提供service实现
4. eventmesh-dashboard-service 公用API接口,对core的抽象
5. eventmesh-dashboard-common 公共模块的代码
6. eventmesh-dashboard-view 前端代码
1. eventmesh-dashboard-console 业务模块的代码,调用service接口
2. eventmesh-dashboard-observe 监控模块的代码
3. eventmesh-dashboard-core 对EventMesh Runtime, Meta以及相关组件的代码,提供service实现
4. eventmesh-dashboard-service 公用API接口,对core的抽象
5. eventmesh-dashboard-common 公共模块的代码
6. eventmesh-dashboard-view 前端代码

## 自动部署最新版 EventMesh Dashboard

Expand Down Expand Up @@ -67,15 +69,15 @@ cd eventmesh-dashboard
./mvnw clean package
```

>TODO download mysql-connector-j manually
> TODO download mysql-connector-j manually

```
java -DDB_ADDRESS=$DB_ADDRESS -DDB_USERNAME=$DB_USERNAME -DDB_PASSWORD=$DB_PASSWORD -jar eventmesh-dashboard-console/target/eventmesh-dashboard-console-0.0.1-SNAPSHOT.jar
```

### 构建并运行 Docker 镜像

>To be updated
> To be updated

```
cd eventmesh-dashboard
Expand All @@ -88,17 +90,19 @@ docker run -d --name eventmesh-dashboard -p 8080:8080 yourname/eventmesh-dashboa
```

### 开发准备
1. 配置格式化
1. 后端格式文档地址:https://eventmesh.apache.org/zh/community/contribute/contribute/
2. 前端使用 eslint
2. 配置协议头工具 license-eye。
1. 检查命令:license-eye header check
2. 使用命令:license-eye header fix
3. 下载地址: https://skywalking.apache.org/downloads/

1. 配置格式化
1. 后端格式文档地址:https://eventmesh.apache.org/zh/community/contribute/contribute/
2. 前端使用 eslint
2. 配置协议头工具 license-eye。
1. 检查命令:license-eye header check
2. 使用命令:license-eye header fix
3. 下载地址: https://skywalking.apache.org/downloads/

### 提交代码前

在项目根目录执行下面命令

```
license-eye -v info -c .licenserc.yaml header check
./mvnw -B package -DskipTests --file pom.xml
Expand Down
107 changes: 107 additions & 0 deletions doc/TODO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
## 文档任务

- 整体设计文档
- metadata 数据 设计文档
- 部署设计文档
- Cluster 设计文档
- 同步设计文档
- 心跳设计文档
- SDK 设计文档
- 远程服务设计文档
- 日志说明与使用设计文档
- 采集设计文档
- 采集数据库设计文档
- 模拟数据 同步测试文档
- RocketMQ 同步测试文档
- 部署测试文档
- 采集测试文档
- 增加一个 存储 整个开发流程
- 增加一个 meta 整个开发流程

## 问题

- 自动化部署之后,如何确定部署是否成功,部署测试如何做?
- group 与 group memer 是 cluster 级别还是 runtime 级别
- 如果是 cluster 级别,rocketmq 需要支持 cluster 级别操作,如果不支持着需要全量删除
- group 与 group memer 只读,不允许创建
- config 问题
- 不同的 存储 与 runtime 对不同的 metadata 有的支持 config ,有的不支持 config
- 需要一个 config 管理模块
- 对 runtime(包含 存储)的 metadata 进行标记
- 支持 runtime(包含 存储) metadata 双维度
- 不同的 存储 与 runtime 对 config 的修改不同
- kakfa 支持一个一个 config 的修改
- rocketmq 只能对 config 的 主体 进行修改。比如 修改 queue ,那么需要 对 Topic 全量修改
- 通过 config 模块 对 config 的行为标记做出不同的处理
- 修改 rocketmq 的 metadata 的 config , 需要对 metadata 主体进行标准,然后触发 metadata 对象的 同步操作

## 任务 or issue

- SyncStatus 状态补充
- LogManage 记录模块
- 操作记录模块
- 优点是:可以进行记录
- 部署与操作流模块
- 是否设计运维操作模块
- rometing service 入参 与 返回 讨论
- 手动添加 集群 与 节点 信息时的 url 设计
- kubernetes 部署 RocketMQ 的 IP 问题。client 调用 broker 是 kubernetes node ip,broker 调用 broker 是 kubernetes pod ip
- TODO 内容整理之后去除
- 文档与代码整理
- 讨论:手动添加集群与节点
- 分析:db 的负载。 维度:业务操作,统计操作,sync and 部署等操作
- 用户进来,是进入默认组织 首页

1. 通过左上角切换
2. 组织视图与 cluster 共存,这样是否会出现三级路由
3. 人员添加,可以从组织添加,也可以从 cluster 里面添加。如果从 cluster 里面添加,是否就属于合格组织一员?

6. 删除 cluster 或则 runtime 之后对应的信息是否全部作废。
7. 暂停状态数据如何标记,激活 对应的 metadata 之前相关的数据是否启动
8. 删除 cluster 之前之后校验 关联。如何关联存在是否允许删除
9. kafka 的查询维度全是 cluster
10. RocketMQ 大集群查询,是否支持 metadata 数据查询,如何对数据进行展示? 是否现实差异化,是否提示差异字段
1. 那些 topic 有差异?
2. 是否配置 不允许有差异?
3. 如果不允许有差异, console 进行处理的 metadata 是否要进行具体标识
11. RocketMQ 支持大集群,小集群,runtime 维度的展示
12. 指标数据表结构,目前不支持表修改。是否需要支持变修改
13. 是否需要支持 不同的 cluster 使用不同的 指标数据修改

## SyncStatus 状态补充

- 同步表,需要添加 SyncStatus 字段
- Entity 需要一个 SyncStatus 类
- 状态转换确认 。 RemotingService(是否独立出来) 里面 成功与失败状态
- 所有查询 需要过滤 成功状态
- 操作数据时, 需要修改 SyncStatus
- 需要一个 SyncStatus 管理类

无法进行确认,支持同类型修改

项目发起人,项目架构师,项目设计师,项目经理,
交互设计师,半个ui设计
前端,后台,运维,产品,测试,

## 用户域

### 用户首页

1. 查看 组织

### 组织首页

1. 左边是一级菜单:
1. 工作台
2. eventmesh集群
3.
2. 集群信息
1. 集群列表

### 日志分类

1. 心跳加载日志
2. 数据同步日志
3. 运维操作日志
4. 数据采集日志
5. metadata 数据日志
Loading
Loading