Skip to content

Commit eece29b

Browse files
YamingPeiYamingPei
authored
feat: support plugin-based formatting and writing components (#43)
* chore: delete unused header files * feat: influx format enhancement, field values support special characters * chore: remove the unreasonable interfaces from the writer * chore: code optimization * docs: add CODE_OF_CONDUCT.md * feat: writer registration mechanism * feat: plugin-based transformation of the writer component * feat: plugin-based transformation of the formatter component * docs: add the specification document * fix: resolve compilation errors * fix: fix the failed test cases related to connection issues * fix: fix the failed test cases related to connection issues * fix: resolve the race condition issue related to MQTT connection closure * feat: optimize the reliability of MQTT publishing * docs: add performance test data * docs: add the CPU model to the test environment * docs: display server information in tabular form
1 parent 98fd4f5 commit eece29b

File tree

87 files changed

+1910
-945
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

87 files changed

+1910
-945
lines changed

‎CMakeLists.txt‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -136,8 +136,6 @@ find_package(Iconv REQUIRED)
136136
find_package(yaml-cpp REQUIRED)
137137
find_package(luajit REQUIRED)
138138
find_package(nlohmann_json REQUIRED)
139-
find_package(PahoMqttCpp REQUIRED)
140-
find_package(RdKafka REQUIRED)
141139
find_package(Threads REQUIRED)
142140
find_package(spdlog REQUIRED)
143141

‎CODE_OF_CONDUCT.md‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
# Contributor Covenant 3.0 Code of Conduct
2+
3+
## Our Pledge
4+
5+
We pledge to make our community welcoming, safe, and equitable for all.
6+
7+
We are committed to fostering an environment that respects and promotes the dignity, rights, and contributions of all individuals, regardless of characteristics including race, ethnicity, caste, color, age, physical characteristics, neurodiversity, disability, sex or gender, gender identity or expression, sexual orientation, language, philosophy or religion, national or social origin, socio-economic position, level of education, or other status. The same privileges of participation are extended to everyone who participates in good faith and in accordance with this Covenant.
8+
9+
## Encouraged Behaviors
10+
11+
While acknowledging differences in social norms, we all strive to meet our community's expectations for positive behavior. We also understand that our words and actions may be interpreted differently than we intend based on culture, background, or native language.
12+
13+
With these considerations in mind, we agree to behave mindfully toward each other and act in ways that center our shared values, including:
14+
15+
1. Respecting the **purpose** **of our community**, our activities, and our ways of gathering.
16+
2. Engaging **kindly and honestly** with others.
17+
3. Respecting **different viewpoints** and experiences.
18+
4. **Taking responsibility** for our actions and contributions.
19+
5. Gracefully giving and accepting **constructive feedback**.
20+
6. Committing to **repairing harm** when it occurs.
21+
7. Behaving in other ways that promote and sustain the **well-being of our community**.
22+
23+
24+
## Restricted Behaviors
25+
26+
We agree to restrict the following behaviors in our community. Instances, threats, and promotion of these behaviors are violations of this Code of Conduct.
27+
28+
1. **Harassment.** Violating explicitly expressed boundaries or engaging in unnecessary personal attention after any clear request to stop.
29+
2. **Character attacks.** Making insulting, demeaning, or pejorative comments directed at a community member or group of people.
30+
3. **Stereotyping or discrimination.** Characterizing anyone’s personality or behavior on the basis of immutable identities or traits.
31+
4. **Sexualization.** Behaving in a way that would generally be considered inappropriately intimate in the context or purpose of the community.
32+
5. **Violating confidentiality**. Sharing or acting on someone's personal or private information without their permission.
33+
6. **Endangerment.** Causing, encouraging, or threatening violence or other harm toward any person or group.
34+
7. Behaving in other ways that **threaten the well-being** of our community.
35+
36+
### Other Restrictions
37+
38+
1. **Misleading identity.** Impersonating someone else for any reason, or pretending to be someone else to evade enforcement actions.
39+
2. **Failing to credit sources.** Not properly crediting the sources of content you contribute.
40+
3. **Promotional materials**. Sharing marketing or other commercial content in a way that is outside the norms of the community.
41+
4. **Irresponsible communication.** Failing to responsibly present content which includes, links or describes any other restricted behaviors.
42+
43+
44+
## Reporting an Issue
45+
46+
Tensions can occur between community members even when they are trying their best to collaborate. Not every conflict represents a code of conduct violation, and this Code of Conduct reinforces encouraged behaviors and norms that can help avoid conflicts and minimize harm.
47+
48+
When an incident does occur, it is important to report it promptly. **To report a possible violation, send an email to conduct@ethicalsource.dev**
49+
50+
Community Moderators take reports of violations seriously and will make every effort to respond in a timely manner. They will investigate all reports of code of conduct violations, reviewing messages, logs, and recordings, or interviewing witnesses and other participants. Community Moderators will keep investigation and enforcement actions as transparent as possible while prioritizing safety and confidentiality. In order to honor these values, enforcement actions are carried out in private with the involved parties, but communicating to the whole community may be part of a mutually agreed upon resolution.
51+
52+
## Addressing and Repairing Harm
53+
54+
If an investigation by the Community Moderators finds that this Code of Conduct has been violated, the following enforcement ladder may be used to determine how best to repair harm, based on the incident's impact on the individuals involved and the community as a whole. Depending on the severity of a violation, lower rungs on the ladder may be skipped.
55+
56+
1) Warning
57+
1) Event: A violation involving a single incident or series of incidents.
58+
2) Consequence: A private, written warning from the Community Moderators.
59+
3) Repair: Examples of repair include a private written apology, acknowledgement of responsibility, and seeking clarification on expectations.
60+
2) Temporarily Limited Activities
61+
1) Event: A repeated incidence of a violation that previously resulted in a warning, or the first incidence of a more serious violation.
62+
2) Consequence: A private, written warning with a time-limited cooldown period designed to underscore the seriousness of the situation and give the community members involved time to process the incident. The cooldown period may be limited to particular communication channels or interactions with particular community members.
63+
3) Repair: Examples of repair may include making an apology, using the cooldown period to reflect on actions and impact, and being thoughtful about re-entering community spaces after the period is over.
64+
3) Temporary Suspension
65+
1) Event: A pattern of repeated violation which the Community Moderators have tried to address with warnings, or a single serious violation.
66+
2) Consequence: A private written warning with conditions for return from suspension. In general, temporary suspensions give the person being suspended time to reflect upon their behavior and possible corrective actions.
67+
3) Repair: Examples of repair include respecting the spirit of the suspension, meeting the specified conditions for return, and being thoughtful about how to reintegrate with the community when the suspension is lifted.
68+
4) Permanent Ban
69+
1) Event: A pattern of repeated code of conduct violations that other steps on the ladder have failed to resolve, or a violation so serious that the Community Moderators determine there is no way to keep the community safe with this person as a member.
70+
2) Consequence: Access to all community spaces, tools, and communication channels is removed. In general, permanent bans should be rarely used, should have strong reasoning behind them, and should only be resorted to if working through other remedies has failed to change the behavior.
71+
3) Repair: There is no possible repair in cases of this severity.
72+
73+
74+
75+
This enforcement ladder is intended as a guideline. It does not limit the ability of Community Managers to use their discretion and judgment, in keeping with the best interests of our community.
76+
77+
## Scope
78+
79+
This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public or other spaces. Examples of representing our community include using an official email address, posting via an official social media account, or acting as an appointed representative at an online or offline event.
80+
81+
## Attribution
82+
83+
This Code of Conduct is adapted from the Contributor Covenant, version 3.0, permanently available at [https://www.contributor-covenant.org/version/3/0/](https://www.contributor-covenant.org/version/3/0/).
84+
85+
Contributor Covenant is stewarded by the Organization for Ethical Source and licensed under CC BY-SA 4.0. To view a copy of this license, visit [https://creativecommons.org/licenses/by-sa/4.0/](https://creativecommons.org/licenses/by-sa/4.0/)
86+
87+
For answers to common questions about Contributor Covenant, see the FAQ at [https://www.contributor-covenant.org/faq](https://www.contributor-covenant.org/faq). Translations are provided at [https://www.contributor-covenant.org/translations](https://www.contributor-covenant.org/translations). Additional enforcement and community guideline resources can be found at [https://www.contributor-covenant.org/resources](https://www.contributor-covenant.org/resources). The enforcement ladder was inspired by the work of [Mozilla’s code of conduct team](https://github.com/mozilla/inclusion).

‎CONTRIBUTING-CN.md‎

Lines changed: 189 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,189 @@
1+
# CONTRIBUTING.md
2+
3+
感谢你对 taosgen 的关注与贡献！本文档用于规范贡献流程与代码标准，确保代码库的一致性、可维护性与高质量。请在提交变更前认真阅读本指南。
4+
5+
## 行为准则
6+
本项目遵循 [Contributor Covenant v3.0 Code of Conduct](./CODE_OF_CONDUCT.md)。所有贡献者都需遵守该准则；如遇违反行为，请按文件说明进行举报。
7+
8+
---
9+
10+
## 开发环境与支持平台
11+
- 语言标准：C++17
12+
- 支持平台：Linux、macOS
13+
- 编译器要求：GCC 9+ / Clang 11+
14+
- 构建系统与依赖管理：
15+
- CMake 3.19+
16+
- Conan 2.19+
17+
18+
快速构建（在项目根目录）：
19+
```shell
20+
mkdir build && cd build
21+
conan install .. --build=missing --output-folder=./conan --settings=build_type=Release
22+
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=./conan/conan_toolchain.cmake
23+
cmake --build .
24+
```
25+
26+
macOS 如需指定 SDK：
27+
```shell
28+
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_OSX_SYSROOT=$(xcrun --show-sdk-path) -DCMAKE_TOOLCHAIN_FILE=./conan/conan_toolchain.cmake
29+
```
30+
31+
---
32+
33+
## 提交流程（Issues 与 PR）
34+
### 提交 Issue
35+
- 先搜索以避免重复
36+
- 必填信息：taosgen 版本/Commit、配置参数、TDengine 版本、操作系统、复现步骤、日志/堆栈等
37+
38+
### 分支与命名
39+
- 从 `main` 分支创建功能/修复分支
40+
- 分支命名：
41+
- 修复：`fix/[brief]`（如：`fix/deadlock-in-scheduler`）
42+
- 功能：`feat/[feature-name]`（如：`feat/batch-metrics-exporter`）
43+
44+
### 提交信息（Commit）
45+
- 推荐使用 Conventional Commits：
46+
- `feat: ...`、`fix: ...`、`docs: ...`、`test: ...`、`refactor: ...`、`perf: ...`、`build: ...`、`ci: ...`、`chore: ...`
47+
- 要求使用 DCO（Developer Certificate of Origin）
48+
- 请在提交时添加签名：`git commit -s -m "feat: add xxx"`
49+
- 提交中会自动添加 `Signed-off-by: Your Name <you@example.com>`
50+
51+
### 提交 PR
52+
- PR 内容聚焦单一主题；链接相关 Issue，描述动机、设计、实现与验证
53+
- 必须附带/更新单元测试（修改代码的覆盖率目标：80%+）
54+
- 本地确保构建与测试通过（`ctest`）
55+
- PR 检查清单（在描述中逐项自检）：
56+
- [ ] 已遵循代码风格与命名规范
57+
- [ ] 添加/更新了必要的单元测试
58+
- [ ] 通过本地构建与 `ctest`
59+
- [ ] 无未使用依赖/头文件
60+
- [ ] 未引入 ABI 破坏（公共头/接口变更已注明）
61+
- [ ] 变更附带必要的文档/注释
62+
- [ ] 提交信息遵循 Conventional Commits 且包含 DCO 签名
63+
64+
---
65+
66+
## 代码规范
67+
68+
### 1) 命名约定
69+
与现有代码（如 `JobScheduler.hpp/cpp`）保持一致：
70+
| 元素 | 风格 | 示例 |
71+
|-----------------------|---------------------|--------------------------------------------|
72+
| 类/结构体 | PascalCase | `JobScheduler`, `StepExecutionStrategy` |
73+
| 函数/方法 | snake_case | `worker_loop()`, `has_failure()` |
74+
| 变量（局部/成员） | snake_case | `remaining_jobs_`, `step_strategy_` |
75+
| 私有成员 | snake_case + `_` | `config_`, `dag_` |
76+
| 常量/宏 | UPPER_SNAKE_CASE | `MAX_CONCURRENCY`, `DEFAULT_TIMEOUT_MS` |
77+
78+
### 2) 格式化与风格
79+
- 缩进 4 空格，行宽 ≤ 120 字符
80+
- 大括号 K&R 风格：
81+
```cpp
82+
if (stop_execution_.load()) {
83+
return;
84+
}
85+
// 单行允许
86+
if (node == nullptr) return;
87+
```
88+
- 注释：类/函数使用 Doxygen 风格；行内说明用 `//`
89+
```cpp
90+
/**
91+
* @brief 运行任务调度器并执行全部 DAG 任务
92+
* @return 全部成功返回 true，任一步失败返回 false
93+
*/
94+
bool run();
95+
```
96+
- 强制使用 clang-format（提交前本地执行）；建议配置 pre-commit 钩子统一格式
97+
98+
### 3) 内存与资源管理（关键）
99+
- 全面遵循 RAII
100+
- 禁止使用原始 `new`/`delete` 管理所有权；禁止以裸指针表达所有权
101+
- 首选栈分配；堆对象默认使用 `std::unique_ptr`
102+
```cpp
103+
auto dag = std::make_unique<JobDAG>(config.jobs); // 推荐
104+
// JobDAG* dag = new JobDAG(config.jobs); // 禁止
105+
```
106+
- 仅在确有共享所有权时使用 `std::shared_ptr`
107+
- 使用 `std::make_unique`/`std::make_shared` 创建智能指针
108+
- 析构函数不得抛出异常；避免在异常传播路径中执行不必要的分配/IO
109+
110+
### 4) 并发与线程安全
111+
- 明确所有共享状态的所有权与生命周期；避免数据竞争
112+
- 使用 `std::mutex` + `std::lock_guard`/`std::scoped_lock`；避免自旋忙等
113+
- 跨线程通信优先使用线程安全队列/条件变量；避免无界队列
114+
- 使用 `std::atomic` 管理原子标志；避免非原子共享布尔/计数器
115+
- 线程创建集中管理；提供可中断/可取消的退出路径（如 `stop_token`）
116+
- 定时等待使用超时（避免无限阻塞）；确保退出时无悬挂线程
117+
118+
### 5) 错误处理与日志
119+
- 对不可恢复错误使用异常（如 DAG 检测到环）：
120+
```cpp
121+
if (dag_->has_cycle()) {
122+
throw std::runtime_error("DAG contains cycles");
123+
}
124+
```
125+
- 可恢复错误返回 `bool`/错误码，严禁静默失败
126+
- 使用 `LogUtils` 输出具备上下文的日志（模块/任务/步骤）
127+
- 不在析构函数中抛异常；异常边界处做好清理与回滚
128+
129+
### 6) 依赖与模块边界
130+
- 第三方依赖统一通过 Conan 管理；新增依赖需：
131+
1) 在相关 `conanfile`/`CMakeLists.txt` 中声明并最小化作用域
132+
2) 通过 `option()` 控制可选特性，避免强绑定
133+
3) 说明许可证与兼容性（MIT 兼容）
134+
- 模块间依赖自上而下，禁止循环依赖；保持目录结构清晰（见 README“Appendix”）
135+
136+
### 7) 许可与版权
137+
- 避免引入受限版权内容；遵循第三方依赖许可证条款并保留必要声明
138+
139+
---
140+
141+
## 测试与质量保障
142+
143+
### 单元测试
144+
- 使用 `ctest` 统一运行
145+
- 新增/修改功能必须配套测试；修改代码的测试覆盖率目标：80%+
146+
- 用例放置在对应模块的 `test` 子目录
147+
- 测试函数命名以 `test_` 开头（如 `test_job_scheduler_with_order`）
148+
- 新增测试文件需更新对应目录下的 `CMakeLists.txt`
149+
150+
运行方式：
151+
```shell
152+
# 在 build 目录
153+
ctest --output-on-failure
154+
# 按名称筛选
155+
ctest -R TestJobScheduler
156+
# 并发执行
157+
ctest -j 4
158+
```
159+
160+
### 性能与基准（可选）
161+
- 性能关键路径需附带简单基准或指标对比
162+
- 避免不必要的拷贝/临时对象；优先使用 `string_view`、`span` 等非拥有视图（在确保生命周期安全前提下）
163+
164+
---
165+
166+
## 文档与示例
167+
- 公共接口与复杂实现需包含 Doxygen 风格注释与必要示例
168+
- 修改行为影响用户的功能需同步更新 README/参考手册链接的文档
169+
- 配置项/命令行参数变更需在参考文档中明确标注
170+
171+
---
172+
173+
## CI/CD 与合并条件
174+
- PR 必须在 CI 通过（构建 + 测试 + 可用的静态检查）
175+
- 维护者在 3 个工作日内完成初审；作者需及时响应并修正
176+
- 通过评审、满足标准后方可合并
177+
178+
---
179+
180+
## 安全报告
181+
如发现安全漏洞，请不要在公开 Issue 披露。请邮件联系：support@taosdata.com，我们会尽快响应与修复。
182+
183+
---
184+
185+
## 常见问题与支持
186+
- 有疑问可提 `question` 标签的 Issue，或加入 TDengine 社区交流
187+
- 贡献即默认同意你的代码以 [MIT License](./LICENSE) 授权
188+
189+
感谢你的贡献，欢迎一起让 taosgen 更加卓越！

0 commit comments

Comments
 (0)