Skip to content

2025-08-25T15:34:51+0800#876

Merged
yushiqie merged 5 commits intosecretflow:mainfrom
gaoyonglong:main
Aug 26, 2025
Merged

2025-08-25T15:34:51+0800#876
yushiqie merged 5 commits intosecretflow:mainfrom
gaoyonglong:main

Conversation

@gaoyonglong
Copy link
Collaborator

@gaoyonglong gaoyonglong commented Aug 25, 2025

新增
● [功能] 新增 Delete DomainData 物理文件接口 (外部贡献 @peter5232
● [功能] Hive 数据源支持(alpha版本 外部贡献 @peter5232
● [功能] Kuscia Task 资源及连通性前置检查(外部贡献 @MiKKiYang
● [功能] Kuscia images import 支持校验镜像架构是否匹配当前 Kuscia架构(alpha版本 外部贡献 @exyb
● [功能] KusciaDeployment 默认加上反亲和性(PodAntiAffinity)配置,多副本时,尽量部署到不同节点
● [功能] Kuscia DomainData 支持 PostgreSQL 代理数据源连接参数配置
● [功能] 支持通过 envoy 日志对 Kuscia task 进行异常分析
修改
● [优化] Kuscia PostgreSQL 数据源数据代理写入优化
● [优化] runp 运行时 kill pod, agent 发送 sigterm 信号量
● [文档] 完善 Kuscia 文档
重大变更
● [无]
修复
● [问题修复] Kuscia 使用 PostgreSQL 作为元数据存储时连接异常问题

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary of Changes

Hello @gaoyonglong, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

此PR对Kuscia进行了重大增强,重点关注诊断能力、数据管理和系统稳定性。主要更新包括引入Envoy日志分析工具、物理数据删除API,以及对资源管理和PostgreSQL数据处理的优化。此外,还改进了容器的优雅停机机制,并对文档和CRD校验进行了全面更新,以提升用户体验和系统可靠性。

Highlights

  • Envoy日志分析工具: 新增了 kuscia diagnose log 命令,用于任务级别的详细网络诊断,帮助用户快速定位任务执行中的网络异常。
  • 物理数据删除API: 新增了 DeleteDomainDataAndRaw 接口,支持对本地文件系统、MySQL和PostgreSQL等各种数据源进行全面的数据对象及其物理数据清理。
  • 资源管理改进: 增强了Agent的资源报告能力,优先使用cgroup信息进行CPU和内存的统计,并为容器注入 CONTAINER_CPU_LIMITCONTAINER_MEM_LIMIT 环境变量,提升了资源管理的准确性和灵活性。
  • PostgreSQL集成优化: 优化了PostgreSQL数据源的数据写入性能,采用了批量插入机制,并改进了连接参数的解析和类型处理,同时修复了作为元数据存储时的连接异常问题。
  • 优雅停机: 为 runp 运行时容器实现了基于 SIGTERM 的优雅停机机制,在强制终止前尝试平滑关闭进程,提高了服务的稳定性。
  • 文档与CRD校验增强: 更新了大量文档,涵盖了新功能的使用说明和现有功能的优化细节。同时,为 ClusterDomainRouteDomainRoute CRD的 host 字段增加了正则表达式校验,提升了配置的健壮性。
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in issue comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

本次合并请求引入了大量新功能和改进。主要新增内容包括:用于分析 Envoy 日志的 diagnose log 命令、对数据源中 PostgreSQL 连接参数的支持,以及使用 SIGTERM 实现的容器优雅终止。代码库也进行了一些重要的修复,例如防止路径遍历漏洞、确保正确的 SQL 引号使用,以及修复 Arrow 记录处理中的内存泄漏问题。文档也进行了大量更新,以反映这些变化并提高清晰度。

有几个地方可以改进。我指出了新诊断命令中的一些代码重复、任务插件管理器中移除重试逻辑可能导致的回归问题,以及一些可以提高测试覆盖率的地方。

总的来说,这是一次内容丰富且有价值的贡献。

@yushiqie yushiqie merged commit 349febb into secretflow:main Aug 26, 2025
13 of 15 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Aug 26, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants