diff --git a/README.md b/README.md index e593b5f..8611d58 100644 --- a/README.md +++ b/README.md @@ -149,9 +149,10 @@ -### 4. 自定义更多的通知方式和处理手段 -1. 可通过实现自定义``Reply``类添加如邮箱,私有机器人等多种通知方式,具体教程参见[reply.md](doc/reply.md) +### 4.自定义更多的通知方式和处理手段 + +1. 可通过实现自定义``Response``类添加如邮箱,私有机器人等多种通知方式,具体教程参见[response.md](doc/response.md) 2. 可通过自定义更多的``Review Handle``引入自定义的代码审查逻辑,具体教程参见[review.md](doc/review.md) diff --git a/config/config.py b/config/config.py index 733ce1c..cefa30b 100644 --- a/config/config.py +++ b/config/config.py @@ -31,7 +31,9 @@ # Prompt gpt_message = """ - 你是一位资深编程专家,gitlab的分支代码变更将以git diff 字符串的形式提供,请你帮忙review本段代码。然后你review内容的返回内容必须严格遵守下面的格式,包括标题内容。模板中的变量内容解释:变量5是代码中的优点儿 变量1是给review打分,分数区间为0~100分。 变量2 是code review发现的问题点。 变量3是具体的修改建议。变量4是你给出的修改后的代码。 必须要求:1. 以精炼的语言、严厉的语气指出存在的问题。2. 你的反馈内容必须使用严谨的markdown格式 3. 不要携带变量内容解释信息。4. 有清晰的标题结构。有清晰的标题结构。有清晰的标题结构。 + 你是一位资深编程专家,gitlab的分支代码变更将以git diff 字符串的形式提供,请你帮忙review本段代码。然后你review内容的返回内容必须严格遵守下面的格式,包括标题内容。模板中的变量内容解释: + 变量5为: 代码中的优点。变量1:给review打分,分数区间为0~100分。变量2:code review发现的问题点。变量3:具体的修改建议。变量4:是你给出的修改后的代码。 + 必须要求:1. 以精炼的语言、严厉的语气指出存在的问题。2. 你的反馈内容必须使用严谨的markdown格式 3. 不要携带变量内容解释信息。4. 有清晰的标题结构。有清晰的标题结构。有清晰的标题结构。 返回格式严格如下: diff --git a/doc/reply.md b/doc/reply.md deleted file mode 100644 index d7c7dc8..0000000 --- a/doc/reply.md +++ /dev/null @@ -1,222 +0,0 @@ -# Reply 模块中文说明文档 - - - -## 1. 代码架构 - -### 树形图 - -``` -reply_module/ -├── reply.py -├── reply_factory.py -├── reply_target/ -│ ├── dingtalk_reply.py -│ ├── gitlab_reply.py -│ └── 更多自定义reply -└── abstract_reply.py -``` - -### 文件功能简要说明 - -- **reply.py**: 主要负责回复消息的管理和发送逻辑。包括添加回复消息、发送所有消息以及实时发送单条消息。 -- **reply_factory.py**: 实现了回复目标的工厂模式,用于创建不同类型的回复实例。 -- **abstract_reply.py**: 定义了一个抽象基类 `AbstractReply`,所有具体的回复类型都需要继承这个基类并实现其抽象方法,即**开发者需要通过继承此类来实现添加新Reply**。 -- **reply_target/**: 存放具体的回复实现类,例如 `dingtalk_reply.py` 和 `gitlab_reply.py`,**自定义的回复类可以放于此处**。 - -## 2. 如何添加自定义的通知方式 - ->> 🚀 **增强功能**: 添加新的通知方式可以扩展系统的功能,使项目能够支持更多的消息发送平台。例如,除了现有的 Gitlab 和 Dingtalk 外,还可以添加对 Slack、Email 或其他平台的支持。 - -### 步骤详细说明 - -1. **创建新的 Reply 类** - - * 在 `reply_target` 目录下创建一个新的 Python 文件,例如 `slack_reply.py`。 - * 文件中新建一个Reply类,例如`SlackReply`,并实现`AbstractReply`类,示例如下: - - ```python - from reply_module.abstract_reply import AbstractReply - - class SlackReply(AbstractReply): - def __init__(self, config): - self.config = config - - def send(self, message): - # 这里实现发送消息到 Slack 的逻辑 - print(f"Sending message to Slack: {message}") - return True - ``` - - * config 主要包含了需要处理的请求的类型(`type`),如 `merge_request`,`push`等,参见[Config参数说明](#31-config)。 - * message 为`String`,内容为要发送的信息。 - -2. **将新的 Reply 类添加到工厂中** - - 在 `reply_factory.py` 文件中注册新的 Reply 类: - - ```python - from reply_module.reply_target.slack_reply import SlackReply - - ReplyFactory.register_target('slack', SlackReply) - ``` - - 这样,工厂类 `ReplyFactory` 就可以自动创建新的 `SlackReply` 实例了。 - -3. **使用自定义类** - - 可以在自定义的Handle中使用新定义的类,使用方法参考使用示例。 - -## 3. 参数说明 - -### 3.1 Config - -#### 3.1.1 功能 - -`config` 是一个字典,包含了初始化 Reply 实例时需要的配置信息。其功能如下: - -1. **说明当前 Hook 的类型**: 如 `merge_request`,`push` 等。 -2. **包含项目的参数**: 如 `project_id`,`merge_request_iid` 等。 - -#### 3.1.2 格式 - -##### 基本格式 - -- `type`: 每个 `config` 一定包含该参数,根据 `type` 的不同,其他参数会有所不同。 -- **目前项目只会有 `merge_request` 一种 `type`,其他事件加急开发中**。 - -```python -config = { - "type": "merge_request" - # 其他参数 -} -``` - -##### merge_request 事件 - -- `project_id`: - - 类型: `int` - - 说明: 项目的唯一标识符,用于标识具体的项目。 - -- `merge_request_iid`: - - 类型: `int` - - 说明: 合并请求的唯一标识符,用于标识具体的合并请求。 - - -```python -config = { - "type": "merge_request", - "project_id": 95536, # 项目ID - "merge_request_iid": 10 # 合并请求IID -} -``` - -### 3.2 Reply Message (reply_msg) - -#### 3.2.1 功能 - -`reply_msg` 是一个字典,包含了发送消息时所需的信息。其功能如下: - -1. **包含消息的实际内容**: 如消息的文本内容、标题等。 -2. **定义消息的类型**: 如 `MAIN`,`TITLE_IGNORE`,`SINGLE`,`NORM` 等。 -3. **分组消息**: 通过 `group_id` 将相同组的消息一起发送。 - -#### 3.2.2 格式 - -##### 基本格式 - -- `content`: 每个 `reply_msg` 一定包含该参数,表示消息的实际内容。 -- `title`: 可选参数,表示消息的标题。 -- `msg_type`: 表示消息的类型,默认值为 `NORM`。 -- ``target``:标识发送给哪些平台,默认为``all`` -- `group_id`: 表示消息的分组ID,默认值为 `0`。 - -```python -reply_msg = { - "content": "This is a message content", - "title": "Optional Title", - "msg_type": "NORM", - "target": "all", - "group_id": 0 -} -``` - -##### 字段说明 - -- `content`: - - 类型: `str` - - 说明: 必须包含的字段,表示消息的实际内容。 - -- `title`: - - 类型: `str` - - 说明: 可选字段,表示消息的标题,如果无此字段或内容为空,则等同于``msg_type``为``TITLE_IGNORE``。 - -- `msg_type`: - - 类型: `str` - - 说明: 表示消息的类型, 可以为多个类型,通过逗号``,``分割。默认值为 `NORM`,可选值包括: - - `MAIN`: 标识主消息,要求唯一,项目自带handle默认使用。 - - `TITLE_IGNORE`: 忽略标题,即只发送内容。 - - `SINGLE`: 直接发送单条消息。 - - `NORMAL`: 正常消息类型,等待所有handle处理完成后拼接成一条消息发送。 - -- ``target``: - - 类型:``str`` - - 说明:标识调用哪些Reply通知类进行·发送,可以同时选择多个Reply,通过逗号``,``分割。默认值为 `all`,可选值包括: - - ``all``:发送给所有在``reply_factory.py``中注册过的Reply通知类。 - - ``gitlab``:发送给gitlab平台,即在merge界面发送comment - - ``dingtalk``:配置好钉钉机器人后,可以通过机器人发送到钉钉 - - ``自定义``:可以参考上文自定义Reply并在``reply_factory.py``中注册,然后可以使用自定义的通知类。 -- `group_id`: - - 类型: `int` - - 说明: 表示消息的分组ID。相同 `group_id` 的消息会一起发送。默认值为 `0`。 -- - -##### 示例 - -```python -reply_msg = { - "content": "This is the main content of the message.", - "title": "Important Update", - "msg_type": "MAIN, SINGLE", - "target": "dingtalk, gitlab", - "group_id": 1 -} -``` - -在上述示例中,`reply_msg` 包含了一个主要类型的消息,带有标题,并且属于组 `1`。 - -## 4. 其他说明 - -### 示例代码 - -以下是一个简单的使用示例: - -```python -from reply_module.reply import Reply - -# 配置字典 -config = { - 'type': 'merge_request', - 'project_id': 9885, - 'merge_request_iid': 18 -} - -# 创建 Reply 实例 -reply = Reply(config) - -# 添加回复消息 -reply.add_reply({ - "target": "slack", - "content": "This is a test message", - "title": "Test Title", - "msg_type": "NORM", - "group_id": 0 -}) - -# 发送所有消息 -success = reply.send() -print(f"Messages sent successfully: {success}") -``` - -通过以上步骤和示例代码,您可以轻松地在项目中添加和使用新的回复类型。 - diff --git a/doc/response.md b/doc/response.md new file mode 100644 index 0000000..eade303 --- /dev/null +++ b/doc/response.md @@ -0,0 +1,283 @@ +# Response 模块中文说明文档 + + + +## 1. 代码架构 + +### 树形图 + +``` +response_module/ +├── response_controller.py +├── response_factory.py +├── response_target/ +│ ├── msg_response +│ │ ├──dingtalk_response.py +│ │ ├──gitlab_response.py +│ │ └──更多自定义文字类型回复... +│ └── other_type_response +│ └──更多自定义非文字类型回复 +└── abstract_response.py +``` + +### 文件功能简要说明 + +- **response_controller.py**: 主要负责回复消息的管理和发送逻辑。包括添加回复消息、发送所有消息以及实时发送单条消息。 +- **response_factory.py**: 实现了回复目标的工厂模式,用于创建不同类型的回复实例。 +- **abstract_response.py**: 定义了一个抽象基类 `AbstractResponse`,所有具体的回复类型都需要继承这个基类并实现其抽象方法,即**开发者需要通过继承此类来实现添加新Response**。 +- **response_target/**: 存放具体的回复实现类,例如 `dingtalk_response.py` 和 `gitlab_response.py`,**自定义的回复类可以放于此处**。 + +## 2. 如何添加自定义的通知方式 + +>> 🚀 **增强功能**: 添加新的通知方式可以扩展系统的功能,使项目能够支持更多的消息发送平台。例如,除了现有的 Gitlab 和 Dingtalk 外,还可以添加对 Slack、Email 或其他平台的支持。 +>> +>> 💡 **Response类型**: 自定义回复类型分为两种,最常用的是文本类型,为提高自定义程度,也支持不太常用的其他类型。 + +### 步骤详细说明 + +#### 文本类型Response类(最常用): + +> 文本类型即发送文字内容的回复,比如:邮箱提醒,钉钉提醒,gitlab评论等。 + +1. **创建新的 Response 类** + + * 在 `response_module/response_target/msg_response` 目录下创建一个新的 Python 文件,例如 `slack_response.py`。 + * 文件中新建一个Response类,例如`SlackResponse`,并实现`AbstractResponseMessage`类,示例如下: + + ```python + from response_module.abstract_response import AbstractResponseMessage + + class SlackResponse(AbstractResponseMessage): + def __init__(self, config): + super().__init__(config) + + def send(self, message): + # 这里实现发送消息到 Slack 的逻辑 + print(f"Sending message to Slack: {message}") + return True + ``` + + * config 主要包含了需要处理的请求的类型(`type`),如 `merge_request`,`push`等,参见[Config参数说明](#31-config)。 + * message 为`String`,内容为要发送的信息。 + +2. **将新的 Response 类添加到工厂中** + + 在 `response_factory.py` 文件中注册新的 Response 类: + + ```python + from response_module.response_target.slack_response import SlackResponse + + ResponseFactory.register_target('slack', SlackResponse) + ``` + + 这样,工厂类 `ResponseFactory` 就可以自动创建新的 `SlackResponse` 实例了。 + +3. **使用自定义类** + + 可以在自定义的Handle中使用新定义的类,使用方法参考使用示例。 + +#### 其他类型Response类(一般用户可忽略): + +> 其他类型不局限发送回复的形式,比如用户需要在自定义handler检测出某严重问题后直接发送给服务器某些指令可以通过该类完成 + +1. **创建新的 Response 类** + + * 在 `response_module/response_target/other_type_response` 目录下创建一个新的 Python 文件,例如 `server_response.py`。 + * 文件中新建一个Response类,例如`ServerResponse`,并实现`AbstractResponseOther`类,示例如下: + + ```python + from response_module.abstract_response import AbstractResponseOther + + class ServerResponse(AbstractResponseOther): + def __init__(self, config): + super().__init__(config) + + @abstractmethod + def set_state(self, *args, **kwargs): + # 如果需要,请在调用send()方法前先调用该方法,可以用于配置一些内容 + pass + + @abstractmethod + def send(self, *args, **kwargs): + # set_state()后调用该方法,请实现发送逻辑 + pass + ``` + + * config 主要包含了需要处理的请求的类型(`type`),如 `merge_request`,`push`等,参见[Config参数说明](#31-config)。 + * set_state 方法可以传入各种参数,用于配置参数等 + * send 中实现发送逻辑 + +2. **将新的 Response 类添加到工厂中** + + 在 `response_factory.py` 文件中注册新的 Response 类: + + ```python + from response_module.response_target.slack_response import ServerResponse + + ResponseFactory.register_target('server', ServerResponse) + ``` + + 这样,工厂类 `ResponseFactory` 就可以自动创建新的 `ServerResponse` 实例了。 + +3. **使用自定义类** + + 可以在自定义的Handle中使用新定义的类,使用方法参考使用示例。 + +## 3. 参数说明 + +### 3.1 Config + +#### 3.1.1 功能 + +`config` 是一个字典,包含了初始化 Response 实例时需要的配置信息。其功能如下: + +1. **说明当前 Hook 的类型**: 如 `merge_request`,`push` 等。 +2. **包含项目的参数**: 如 `project_id`,`merge_request_iid` 等。 + +#### 3.1.2 格式 + +##### 基本格式 + +- `type`: 每个 `config` 一定包含该参数,根据 `type` 的不同,其他参数会有所不同。 +- **目前项目只会有 `merge_request` 一种 `type`,其他事件加急开发中**。 + +```python +config = { + "type": "merge_request" + # 其他参数 +} +``` + +##### merge_request 事件 + +- `project_id`: + - 类型: `int` + - 说明: 项目的唯一标识符,用于标识具体的项目。 + +- `merge_request_iid`: + - 类型: `int` + - 说明: 合并请求的唯一标识符,用于标识具体的合并请求。 + + +```python +config = { + "type": "merge_request", + "project_id": 95536, # 项目ID + "merge_request_iid": 10 # 合并请求IID +} +``` + +### 3.2 Response Message (response_msg) + +#### 3.2.1 功能 + +`response_msg` 是一个字典,包含了发送消息时所需的信息。其功能如下: + +1. **包含消息的实际内容**: 如消息的文本内容、标题等。 +2. **定义消息的类型**: 如 `MAIN`,`TITLE_IGNORE`,`SINGLE`,`NORM` 等。 +3. **分组消息**: 通过 `group_id` 将相同组的消息一起发送。 + +#### 3.2.2 格式 + +##### 基本格式 + +- `content`: 每个 `response_msg` 一定包含该参数,表示消息的实际内容。 +- `title`: 可选参数,表示消息的标题。 +- `msg_type`: 表示消息的类型,默认值为 `NORM`。 +- ``target``:标识发送给哪些平台,默认为``all`` +- `group_id`: 表示消息的分组ID,默认值为 `0`。 + +```python +response_msg = { + "content": "This is a message content", + "title": "Optional Title", + "msg_type": "NORM", + "target": "all", + "group_id": 0 +} +``` + +##### 字段说明 + +- `content`: + - 类型: `str` + - 说明: 必须包含的字段,表示消息的实际内容。 + +- `title`: + - 类型: `str` + - 说明: 可选字段,表示消息的标题,如果无此字段或内容为空,则等同于``msg_type``为``TITLE_IGNORE``。 + +- `msg_type`: + - 类型: `str` + - 说明: 表示消息的类型, 可以为多个类型,通过逗号``,``分割。默认值为 `NORM`,可选值包括: + - `MAIN`: 标识主消息,要求唯一,项目自带handle默认使用。 + - `TITLE_IGNORE`: 忽略标题,即只发送内容。 + - `SINGLE`: 直接发送单条消息。 + - `NORMAL`: 正常消息类型,等待所有handle处理完成后拼接成一条消息发送。 + +- ``target``: + - 类型:``str`` + - 说明:标识调用哪些Response通知类进行·发送,可以同时选择多个Response,通过逗号``,``分割。默认值为 `all`,可选值包括: + - ``all``:发送给所有在``response_factory.py``中注册过的Response通知类。 + - ``gitlab``:发送给gitlab平台,即在merge界面发送comment + - ``dingtalk``:配置好钉钉机器人后,可以通过机器人发送到钉钉 + - ``自定义``:可以参考上文自定义Response并在``response_factory.py``中注册,然后可以使用自定义的通知类。 +- `group_id`: + - 类型: `int` + - 说明: 表示消息的分组ID。相同 `group_id` 的消息会一起发送。默认值为 `0`。 +- + +##### 示例 + +```python +response_msg = { + "content": "This is the main content of the message.", + "title": "Important Update", + "msg_type": "MAIN, SINGLE", + "target": "dingtalk, gitlab", + "group_id": 1 +} +``` + +在上述示例中,`response_msg` 包含了一个主要类型的消息,带有标题,并且属于组 `1`。 + +## 4. 其他说明 + +### 示例代码 + +以下是一个简单的使用示例: + +```python +from response_module.response_controller import ReviewResponse + +# 配置字典 +config = { + 'type': 'merge_request', + 'project_id': 9885, + 'merge_request_iid': 18 +} + +# 创建 Response 实例 +response = ReviewResponse(config) + +# 添加文本类型回复 +response.add_response({ + "target": "slack", + "content": "This is a test message", + "title": "Test Title", + "msg_type": "NORM", + "group_id": 0 +}) + +# 发送所有消息 +success = response.send() +print(f"Messages sent successfully: {success}") + +# 发送其他类型回复 +response.set_state("server", "param1", "param2", {"key1": "xxx"}) # 设置Server类状态 +response.send_by_other("server", "param1", "param2", {"key1": "xxx"}) # 发送回复 + +``` + +通过以上步骤和示例代码,您可以轻松地在项目中添加和使用新的回复类型。 + diff --git a/doc/review.md b/doc/review.md index 78578f1..cd18711 100644 --- a/doc/review.md +++ b/doc/review.md @@ -44,8 +44,11 @@ review_engine/ 在新类中实现 `merge_handle` 方法,编写具体的代码审查逻辑,相关参数的详细说明见**参数说明**部分: - - [changes](#41-changes) :merge变更文件的内容 - - [merge_info](#42-merge_info) :merge的相关信息 + - [gitlabMergeRequestFetcher](#41-GitlabMergeRequestFetcher):gitlab merge信息管理类,可以通过调用相关方法获取以下信息: + - [changes](#411-changes) :merge变更文件的内容 + - [merge_info](#412-merge_info) :merge的相关信息 + + - [gitlabRepoManager](#42-GitlabRepoManager):gitlab项目仓库等管理类,可以通过该类查找仓库中指定内容 - [hook_info](#43-hook_info) :hook请求接收到的信息 - [reply](#44-reply) :发送生成review的模块 - [model](#45-model) :统一的大模型接口模块 @@ -58,8 +61,12 @@ review_engine/ from review_engine.abstract_handler import ReviewHandle class CustomReviewHandle(ReviewHandle): - def merge_handle(self, changes, merge_info, hook_info, reply, model): + def merge_handle(self, gitlabMergeRequestFetcher, gitlabRepoManager, hook_info, reply, model): # 自定义的代码审查逻辑 + changes = gitlabMergeRequestFetcher.get_changes() + merge_info = gitlabMergeRequestFetcher.get_info() + source_branch_name = merge_info['source_branch'] + # 其他逻辑 pass ``` @@ -67,8 +74,26 @@ class CustomReviewHandle(ReviewHandle): ## 4. 参数说明 📊 -### 4.1 Changes - +### 4.1 GitlabMergeRequestFetcher + +* **位置**:`gitlab_integration.gitlab_fetcher.GitlabMergeRequestFetcher` +* **主要功能**:获取gitlab中关于MergeRequest的相关信息 +* **主要方法**: + * `def get_changes(force=False)`:获取merge request的change信息。 + * `force` (bool, 可选): 是否强制刷新缓存,默认为 `False`。如果设置为 `True`,即使缓存中已有文件内容,也会重新从 GitLab 获取changes内容。 + * 返回的changes信息具体内容参加[changes](#411-changes)。 + * `get_info(force=False)`:获取merge request的merge_info信息。 + * `force` (bool, 可选): 是否强制刷新缓存,默认为 `False`。如果设置为 `True`,即使缓存中已有文件内容,也会重新从 GitLab 获取merge_info内容。 + * 返回的merge_info信息具体内容参加[merge_info](#412-merge_info)。 + * `get_file_content(file_path, branch_name='main', force=False)`:用于从 GitLab 仓库中获取指定文件的内容。该方法会尝试从缓存中读取文件内容,如果缓存中没有该文件或强制刷新缓存,则会通过 GitLab API 获取文件内容。 + * `file_path` (str): 文件的路径,请直接提供用`/`分割的文件路径。该路径会在内部转换,将路径中的斜杠 `/` 替换为 `%2F`,以符合 URL 编码的要求。 + * `branch_name` (str, 可选): 分支的名称,默认为 `'main'`。该参数用于指定从哪个分支获取文件内容。 + * `force` (bool, 可选): 是否强制刷新缓存,默认为 `False`。如果设置为 `True`,即使缓存中已有文件内容,也会重新从 GitLab 获取文件内容。 + * 返回值:如果请求成功,返回文件的内容(字符串)。如果请求失败,返回 `None`。 + +#### 4.1.1 Changes + +- **获取方式**:`gitlabMergeRequestFetcher.get_changes()` - **来源**:gitlab api中`projects/{project_id}/merge_requests/{iid}/changes` 中的 `changes` 字段。 - **类型**:字典列表。 - **示例**: @@ -90,7 +115,9 @@ class CustomReviewHandle(ReviewHandle): - `old_path` 和 `new_path`:文件路径。 - `diff`:文件变更的详细内容。 -### 4.2 Merge_info +#### 4.1.2 Merge_info + +* **获取方式**:gitlabMergeRequestFetcher.get_info() - **来源**:gitlab api中`projects/{project_id}/merge_requests/{iid}`的所有信息。 @@ -152,6 +179,42 @@ class CustomReviewHandle(ReviewHandle): - **web_url**: 合并请求的网页 URL。 - **head_pipeline**: 合并请求的最新流水线信息。 +### 4.2 GitlabRepoManager + +* **位置**:`gitlab_integration.gitlab_fetcher.GitlabRepoManager` + +* **主要功能**:可以通过浅clone的方式获取项目中指定分支的内容,并提供支持正则语法的全文查找功能 + +* **主要方法**: + + * `get_info()`:用于获取项目的信息。该方法通过 GitLab API 获取项目的详细信息。 + - 返回值:如果请求成功,返回项目的信息(JSON 格式)。如果请求失败,返回 `None`。 + + * `shallow_clone(branch_name='main')`:执行仓库的浅克隆操作。浅克隆只会克隆指定分支的最新提交记录。 + + - `branch_name` (str, 可选): 要克隆的分支名称,默认为 `'main'`。该参数用于指定要克隆的分支。 + + - 该方法会删除目标目录中已有的仓库,并使用构建的认证 URL 执行 `git clone` 命令。如果克隆失败,会记录错误日志。 + + * `checkout_branch(branch_name, force=False)`:切换到指定的分支。如果仓库尚未克隆,则会执行浅克隆操作。 + + - `branch_name` (str): 要切换到的分支名称。 + + - `force` (bool, 可选): 是否强制切换分支,默认为 `False`。如果设置为 `True`,即使当前分支已经是目标分支,也会重新克隆。 + + - 该方法会检查是否已经在目标分支上,如果不是或 `force` 为 `True`,则会执行浅克隆。 + + * `delete_repo()`:删除现有的仓库目录。 + - 该方法会检查目标目录是否存在,如果存在则删除整个目录及其内容。 + + * `find_files_by_keyword(keyword, branch_name='main')`:查找仓库中包含指定关键词的文件列表。 + + - `keyword` (str): 要查找的关键词。该关键词会被编译成正则表达式,用于在文件内容中搜索。 + + - `branch_name` (str, 可选): 要搜索的分支名称,默认为 `'main'`。该参数用于指定要搜索的分支。 + + - 返回值:返回一个包含匹配文件路径的列表。如果文件无法读取(例如编码错误、文件不存在或权限问题),则会跳过该文件。 + ### 4.3 Hook_info - **来源**:Webhook 接收到的内容。 diff --git a/gitlab_integration/gitlab_fetcher.py b/gitlab_integration/gitlab_fetcher.py index 57bc34b..f09e651 100644 --- a/gitlab_integration/gitlab_fetcher.py +++ b/gitlab_integration/gitlab_fetcher.py @@ -1,19 +1,32 @@ +import os +import re +import shutil +import subprocess +import time + import requests from retrying import retry from config.config import * from utils.logger import log +from utils.tools import run_command + class GitlabMergeRequestFetcher: def __init__(self, project_id, merge_request_iid): self.project_id = project_id self.iid = merge_request_iid + self._changes_cache = None + self._file_content_cache = {} + self._info_cache = None @retry(stop_max_attempt_number=3, wait_fixed=2000) - def get_changes(self): + def get_changes(self, force=False): """ Get the changes of the merge request :return: changes """ + if self._changes_cache and not force: + return self._changes_cache # URL for the GitLab API endpoint url = f"{gitlab_server_url}/api/v4/projects/{self.project_id}/merge_requests/{self.iid}/changes" @@ -27,12 +40,49 @@ def get_changes(self): # Check if the request was successful if response.status_code == 200: + self._changes_cache = response.json()["changes"] return response.json()["changes"] else: return None @retry(stop_max_attempt_number=3, wait_fixed=2000) - def get_info(self): + # 获取文件内容 + def get_file_content(self, file_path, branch_name='main', force=False): + """ + Get the content of the file + :param file_path: The path of the file + :return: The content of the file + """ + # 对file_path中的'/'转换为'%2F' + file_path = file_path.replace('/', '%2F') + if file_path in self._file_content_cache and not force: + return self._file_content_cache[file_path] + # URL for the GitLab API endpoint + url = f"{gitlab_server_url}/api/v4/projects/{self.project_id}/repository/files/{file_path}/raw?ref={branch_name}" + + # Headers for the request + headers = { + "PRIVATE-TOKEN": gitlab_private_token + } + + # Make the GET request + response = requests.get(url, headers=headers) + + # Check if the request was successful + if response.status_code == 200: + self._file_content_cache[file_path] = response.text + return response.text + else: + return None + + @retry(stop_max_attempt_number=3, wait_fixed=2000) + def get_info(self, force=False): + """ + Get the merge request information + :return: Merge request information + """ + if self._info_cache and not force: + return self._info_cache # URL for the GitLab API endpoint url = f"{gitlab_server_url}/api/v4/projects/{self.project_id}/merge_requests/{self.iid}" @@ -44,8 +94,112 @@ def get_info(self): # Make the GET request response = requests.get(url, headers=headers) + # Check if the request was successful + if response.status_code == 200: + self._info_cache = response.json() + return response.json() + else: + return None + +# gitlab仓库clone和管理 +class GitlabRepoManager: + def __init__(self, project_id, branch_name = ""): + self.project_id = project_id + self.timestamp = int(time.time() * 1000) + self.repo_path = f"./repo/{self.project_id}_{self.timestamp}" + self.has_cloned = False + + def get_info(self): + """ + Get the project information + :return: Project information + """ + # URL for the GitLab API endpoint + url = f"{gitlab_server_url}/api/v4/projects/{self.project_id}" + + # Headers for the request + headers = { + "PRIVATE-TOKEN": gitlab_private_token + } + + # Make the GET request + response = requests.get(url, headers=headers) + # Check if the request was successful if response.status_code == 200: return response.json() else: - return None \ No newline at end of file + return None + + @retry(stop_max_attempt_number=3, wait_fixed=2000) + def shallow_clone(self, branch_name = "main"): + """ + Perform a shallow clone of the repository + param branch_name: The name of the branch to clone + """ + # If the target directory exists, remove it + self.delete_repo() + + # Build the authenticated URL + authenticated_url = self._build_authenticated_url(self.get_info()["http_url_to_repo"]) + + # Build the Git command + command = ["git", "clone", authenticated_url, "--depth", "1"] + if branch_name: + command.extend(["--branch", branch_name]) + command.extend([self.repo_path + "/" + str(branch_name)]) + else: + command.extend([self.repo_path + "/default"]) + # command 添加clone到的位置: + if run_command(command) != 0: + log.error("Failed to clone the repository") + self.has_cloned = True + + # 切换分支 + def checkout_branch(self, branch_name, force=False): + # Build the Git command + if not self.has_cloned: + self.shallow_clone(branch_name) + else: + # 检查是否已经在目标分支上 + if not force and os.path.exists(self.repo_path + "/" + str(branch_name) + "/.git"): + return + else: + self.shallow_clone(branch_name) + + # 删除库 + def delete_repo(self): + if os.path.exists(self.repo_path): + shutil.rmtree(self.repo_path) + + # 查找相关文件列表 + def find_files_by_keyword(self, keyword, branch_name="main"): + matching_files = [] + regex = re.compile(keyword) + self.checkout_branch(branch_name) + for root, _, files in os.walk(self.repo_path + "/" + str(branch_name)): + for file in files: + file_path = os.path.join(root, file) + try: + with open(file_path, 'r', encoding='utf-8') as f: + content = f.read() + if regex.search(content): + matching_files.append(file_path) + except (UnicodeDecodeError, FileNotFoundError, PermissionError): + # 跳过无法读取的文件 + continue + + return matching_files + + + # 构建带有身份验证信息的 URL + def _build_authenticated_url(self, repo_url): + # 如果 URL 使用 https + token = gitlab_private_token + if repo_url.startswith("https://"): + return f"https://oauth2:{token}@{repo_url[8:]}" + # 如果 URL 使用 http + elif repo_url.startswith("http://"): + return f"http://oauth2:{token}@{repo_url[7:]}" + else: + raise ValueError("Unsupported URL scheme") \ No newline at end of file diff --git a/gitlab_integration/webhook_listener.py b/gitlab_integration/webhook_listener.py index e3cbb93..823fd9c 100644 --- a/gitlab_integration/webhook_listener.py +++ b/gitlab_integration/webhook_listener.py @@ -3,8 +3,8 @@ from flask import request, jsonify -from gitlab_integration.gitlab_fetcher import GitlabMergeRequestFetcher -from reply_module.reply import Reply +from gitlab_integration.gitlab_fetcher import GitlabMergeRequestFetcher, GitlabRepoManager +from response_module.response_controller import ReviewResponse from review_engine.review_engine import ReviewEngine from utils.logger import log @@ -31,14 +31,14 @@ def call_handle(self, gitlab_payload, event_type): 'project_id': gitlab_payload.get('project')['id'], 'merge_request_iid': gitlab_payload.get('object_attributes')['iid'] } - reply = Reply(config) + reply = ReviewResponse(config) return self.handle_merge_request(gitlab_payload, reply) elif event_type == 'push': config = { 'type': 'push', 'project_id': gitlab_payload.get('project')['id'] } - reply = Reply(config) + reply = ReviewResponse(config) return self.handle_push(gitlab_payload, reply) else: @@ -46,7 +46,7 @@ def call_handle(self, gitlab_payload, event_type): 'type': 'other', 'project_id': gitlab_payload.get('project')['id'] } - reply = Reply(config) + reply = ReviewResponse(config) return self.handle_other(gitlab_payload, reply) def handle_merge_request(self, gitlab_payload, reply): @@ -58,11 +58,9 @@ def handle_merge_request(self, gitlab_payload, reply): project_id = gitlab_payload.get('project')['id'] merge_request_iid = gitlab_payload.get("object_attributes")["iid"] review_engine = ReviewEngine(reply) - fetcher = GitlabMergeRequestFetcher(project_id, merge_request_iid) - - changes = fetcher.get_changes() - info = fetcher.get_info() - thread = threading.Thread(target=review_engine.handle_merge, args=(changes, info, gitlab_payload)) + gitlabMergeRequestFetcher = GitlabMergeRequestFetcher(project_id, merge_request_iid) + gitlabRepoManager = GitlabRepoManager(project_id) + thread = threading.Thread(target=review_engine.handle_merge, args=(gitlabMergeRequestFetcher, gitlabRepoManager, gitlab_payload)) thread.start() return jsonify({'status': 'success'}), 200 diff --git a/reply_module/abstract_reply.py b/reply_module/abstract_reply.py deleted file mode 100644 index 16179e2..0000000 --- a/reply_module/abstract_reply.py +++ /dev/null @@ -1,20 +0,0 @@ -from abc import ABC, abstractmethod - -class AbstractReply(ABC): - @abstractmethod - def __init__(self, config): - self.config = config - - @abstractmethod - def send(self, message): - pass - - # # 发送失败调用 - # @abstractmethod - # def send_failed(self, message): - # pass - # - # # 发送成功调用 - # @abstractmethod - # def send_success(self, message): - # pass diff --git a/reply_module/reply_factory.py b/reply_module/reply_factory.py deleted file mode 100644 index ca618f5..0000000 --- a/reply_module/reply_factory.py +++ /dev/null @@ -1,28 +0,0 @@ -from reply_module.reply_target.dingtalk_reply import DingtalkReply -from reply_module.reply_target.gitlab_reply import GitlabReply - -class ReplyFactory: - _registry = {} - - @classmethod - def register_target(cls, target, target_class): - cls._registry[target] = target_class - - @classmethod - def get_reply_instance(cls, target, config): - if target not in cls._registry: - raise ValueError(f"Unknown target: {target}") - return cls._registry[target](config) - - @classmethod - def get_all_reply_instance(cls, config): - return [target_class(config) for target_class in cls._registry.values()] - - @classmethod - def get_all_targets(cls): - return list(cls._registry.keys()) - - - -ReplyFactory.register_target('gitlab', GitlabReply) -ReplyFactory.register_target('dingtalk', DingtalkReply) \ No newline at end of file diff --git a/reply_module/__init__.py b/response_module/__init__.py similarity index 100% rename from reply_module/__init__.py rename to response_module/__init__.py diff --git a/response_module/abstract_response.py b/response_module/abstract_response.py new file mode 100644 index 0000000..a40e915 --- /dev/null +++ b/response_module/abstract_response.py @@ -0,0 +1,30 @@ +from abc import ABC, abstractmethod + +class AbstractResponse(ABC): + @abstractmethod + def __init__(self, config): + self.config = config + + +class AbstractResponseMessage(AbstractResponse): + @abstractmethod + def __init__(self, config): + super().__init__(config) + + @abstractmethod + def send(self, message): + pass + + +class AbstractResponseOther(AbstractResponse): + @abstractmethod + def __init__(self, config): + super().__init__(config) + + @abstractmethod + def set_state(self, *args, **kwargs): + pass + + @abstractmethod + def send(self, *args, **kwargs): + pass \ No newline at end of file diff --git a/reply_module/reply.py b/response_module/response_controller.py similarity index 82% rename from reply_module/reply.py rename to response_module/response_controller.py index 428d2e3..a1eaa24 100644 --- a/reply_module/reply.py +++ b/response_module/response_controller.py @@ -1,9 +1,9 @@ import threading -from reply_module.reply_factory import ReplyFactory +from response_module.response_factory import ResponseFactory -class Reply: +class ReviewResponse: def __init__(self, config): """ 初始化 Reply 实例 @@ -17,6 +17,7 @@ def __init__(self, config): self.config = config self.replies = [] self.lock = threading.Lock() + self.oter_res_state = {} def add_reply(self, reply_msg): """ @@ -68,7 +69,7 @@ def send(self): self.__parse_msg(main_msg, msg_groups) ret = True for target, msg_group in msg_groups.items(): - reply_target = ReplyFactory.get_reply_instance(target, self.config) + reply_target = ResponseFactory.get_message_instance(target, self.config) for msg in msg_group: ret &= reply_target.send(msg) return ret @@ -84,10 +85,10 @@ def send_single_message(self, reply): """ targets = [t.strip() for t in reply['target'].split(',')] if 'all' in targets: - targets = ReplyFactory.get_all_targets() + targets = ResponseFactory.get_all_message_targets() ret = True for target in targets: - reply_target = ReplyFactory.get_reply_instance(target, self.config) + reply_target = ResponseFactory.get_message_instance(target, self.config) if ('TITLE_IGNORE' in reply['msg_type'] or 'MAIN' in reply['msg_type'] or 'title' not in reply or not reply['title']): ret &= reply_target.send(reply['content']) @@ -99,7 +100,7 @@ def send_single_message(self, reply): def __parse_msg(self, msg, msg_groups): targets = [t.strip() for t in msg['target'].split(',')] if 'target' not in msg or 'all' in targets: - targets = ReplyFactory.get_all_targets() + targets = ResponseFactory.get_all_message_targets() for target in targets: if target not in msg_groups: msg_groups[target] = {} @@ -112,8 +113,19 @@ def __parse_msg(self, msg, msg_groups): title = f"## {msg['title']}\n\n" if 'title' in msg else '' msg_groups[target][msg['group_id']].append(f"{title}{msg['content']}\n\n") + def set_state(self, res_type, *args, **kwargs): + self.oter_res_state[res_type] = (args, kwargs) + + def send_by_other(self, response_type, *args, **kwargs): + sender = ResponseFactory.get_other_instance(response_type, self.config) + if sender is None: + raise Exception(f'No such type {response_type} in other response.') + if self.oter_res_state.get(response_type): + sender.set_state(*self.oter_res_state[response_type]) + return sender.send(*args, **kwargs) + if __name__ == '__main__': - reply = Reply({'type': 'merge_request', + reply = ReviewResponse({'type': 'merge_request', 'project_id': 9885, 'merge_request_iid': 18}) threads = [] diff --git a/response_module/response_factory.py b/response_module/response_factory.py new file mode 100644 index 0000000..521fb92 --- /dev/null +++ b/response_module/response_factory.py @@ -0,0 +1,52 @@ +from response_module.abstract_response import AbstractResponse, AbstractResponseMessage +from response_module.response_target.msg_response.dingtalk_response import DingtalkResponse +from response_module.response_target.msg_response.gitlab_response import GitlabResponse + + +class ResponseFactory: + _registry_msg = {} + _registry_other = {} + + @classmethod + def register_target(cls, target, target_class): + # 检测是否实现了AbstractResponseMessage接口 + if not issubclass(target_class, AbstractResponse): + raise TypeError(f'{target_class} does not implement AbstractResponse') + if not issubclass(target_class, AbstractResponseMessage): + cls._registry_other[target] = target_class + cls._registry_msg[target] = target_class + + @classmethod + def get_message_instance(cls, target, config): + if target not in cls._registry_msg: + return None + return cls._registry_msg[target](config) + + @classmethod + def get_other_instance(cls, target, config): + if target not in cls._registry_other: + return None + return cls._registry_other[target](config) + + @classmethod + def get_all_message_instance(cls, config): + return [target_class(config) for target_class in cls._registry_msg.values()] + + @classmethod + def get_all_other_instance(cls, *args, **kwargs): + return [target_class(*args, **kwargs) for target_class in cls._registry_other.values()] + + @classmethod + def get_all_message_targets(cls): + return list(cls._registry_msg.keys()) + + @classmethod + def get_all_other_targets(cls): + return list(cls._registry_other.keys()) + + +ResponseFactory.register_target('gitlab', GitlabResponse) +ResponseFactory.register_target('dingtalk', DingtalkResponse) +# ResponseFactory.register_target('temp', TemplateResponse) +if __name__ == '__main__': + print(ResponseFactory.get_all_message_targets()) diff --git a/reply_module/reply_target/__init__.py b/response_module/response_target/__init__.py similarity index 100% rename from reply_module/reply_target/__init__.py rename to response_module/response_target/__init__.py diff --git a/response_module/response_target/msg_response/__init__.py b/response_module/response_target/msg_response/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/reply_module/reply_target/dingtalk_reply.py b/response_module/response_target/msg_response/dingtalk_response.py similarity index 95% rename from reply_module/reply_target/dingtalk_reply.py rename to response_module/response_target/msg_response/dingtalk_response.py index 85cad1b..daa61d6 100644 --- a/reply_module/reply_target/dingtalk_reply.py +++ b/response_module/response_target/msg_response/dingtalk_response.py @@ -7,10 +7,10 @@ import json from config.config import * from utils.logger import * -from reply_module.abstract_reply import AbstractReply +from response_module.abstract_response import AbstractResponseMessage -class DingtalkReply(AbstractReply): +class DingtalkResponse(AbstractResponseMessage): def __init__(self, config): super().__init__(config) self.type = config['type'] @@ -106,5 +106,5 @@ def __get_sign(self, timestamp): return sign if __name__ == '__main__': - dingtalk = DingtalkReply(1, 1) + dingtalk = DingtalkResponse(1, 1) dingtalk.send("test message") \ No newline at end of file diff --git a/reply_module/reply_target/gitlab_reply.py b/response_module/response_target/msg_response/gitlab_response.py similarity index 92% rename from reply_module/reply_target/gitlab_reply.py rename to response_module/response_target/msg_response/gitlab_response.py index 7532237..4bda38e 100644 --- a/reply_module/reply_target/gitlab_reply.py +++ b/response_module/response_target/msg_response/gitlab_response.py @@ -1,11 +1,11 @@ import requests from retrying import retry from config.config import * -from reply_module.abstract_reply import AbstractReply +from response_module.abstract_response import AbstractResponseMessage from utils.logger import log # 继承AbstractReply类,实现send方法 -class GitlabReply(AbstractReply): +class GitlabResponse(AbstractResponseMessage): def __init__(self, config): super().__init__(config) self.type = config['type'] diff --git a/response_module/response_target/other_type_response/__init__.py b/response_module/response_target/other_type_response/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/response_module/response_target/other_type_response/template_response.py b/response_module/response_target/other_type_response/template_response.py new file mode 100644 index 0000000..97eba49 --- /dev/null +++ b/response_module/response_target/other_type_response/template_response.py @@ -0,0 +1,10 @@ +from response_module.abstract_response import AbstractResponseOther + + +class TemplateResponse(AbstractResponseOther): + def __init__(self, config): + super().__init__(config) + + def send(self, *args, **kwargs): + print(f'{self.__class__.__name__} send: {args} {kwargs}') + return True \ No newline at end of file diff --git a/review_engine/abstract_handler.py b/review_engine/abstract_handler.py index 0207987..1900630 100644 --- a/review_engine/abstract_handler.py +++ b/review_engine/abstract_handler.py @@ -7,5 +7,5 @@ class ReviewHandle(object): def __init__(self): pass - def merge_handle(self, changes, merge_info, hook_info, reply, model): + def merge_handle(self, gitlabMergeRequestFetcher, gitlabRepoManager, hook_info, reply, model): pass \ No newline at end of file diff --git a/review_engine/handler/default_handler.py b/review_engine/handler/default_handler.py index 3298778..aa582fc 100644 --- a/review_engine/handler/default_handler.py +++ b/review_engine/handler/default_handler.py @@ -9,14 +9,14 @@ from utils.logger import log -def chat_review(changes, model): +def chat_review(changes, generate_review, *args, **kwargs): log.info('开始code review') with concurrent.futures.ThreadPoolExecutor() as executor: review_results = [] result_lock = threading.Lock() def process_change(change): - result = generate_review_note(change, model) + result = generate_review(change, *args, **kwargs) with result_lock: review_results.append(result) @@ -52,15 +52,7 @@ def generate_review_note(change, model): total_tokens = model.get_respond_tokens() review_note = f'# 📚`{new_path}`' + '\n\n' review_note += f'({total_tokens} tokens) {"AI review 意见如下:"}' + '\n\n' - review_note += response_content + """ - ---- - ---- - ---- - ---- - ---- - ---- - ---- - """ + review_note += response_content + "\n\n---\n\n---\n\n" log.info(f'对 {new_path} review结束') return review_note except Exception as e: @@ -68,14 +60,16 @@ def generate_review_note(change, model): class MainReviewHandle(ReviewHandle): - def merge_handle(self, changes, merge_info, hook_info, reply, model): + def merge_handle(self, gitlabMergeRequestFetcher, gitlabRepoManager, hook_info, reply, model): + changes = gitlabMergeRequestFetcher.get_changes() + merge_info = gitlabMergeRequestFetcher.get_info() self.default_handle(changes, merge_info, hook_info, reply, model) def default_handle(self, changes, merge_info, hook_info, reply, model): maximum_files = 50 if changes and len(changes) <= maximum_files: # Code Review 信息 - review_info = chat_review(changes, model) + review_info = chat_review(changes, generate_review_note, model) if review_info: reply.add_reply({ 'content': review_info, @@ -136,64 +130,4 @@ def default_handle(self, changes, merge_info, hook_info, reply, model): log.error(f"获取merge_request信息失败,project_id: {hook_info['project']['id']} |" f" merge_iid: {hook_info['object_attributes']['iid']}") -if __name__ == '__main__': - main_handle = MainReviewHandle() - from gitlab_integration.gitlab_fetcher import GitlabMergeRequestFetcher - from reply_module.reply import Reply - fetcher = GitlabMergeRequestFetcher(9885, 18) - changes = fetcher.get_changes() - info = fetcher.get_info() - reply = Reply({'type': 'merge_request', - 'project_id': 9885, - 'merge_request_iid': 18}) - from large_model.llm_generator import LLMGenerator - model = LLMGenerator.new_model() - hook_info = { - "object_kind": "merge_request", - "event_type": "merge_request", - "user": { - "id": 1, - "name": "John Doe", - "username": "johndoe", - "avatar_url": "https://example.com/uploads/user/avatar/1/index.jpg" - }, - "project": { - "id": 15, - "name": "Example Project", - "description": "An example project", - "web_url": "https://example.com/example/project", - "avatar_url": None, - "git_ssh_url": "git@example.com:example/project.git", - "git_http_url": "https://example.com/example/project.git", - "namespace": "Example", - "visibility_level": 20, - "path_with_namespace": "example/project", - "default_branch": "main", - "homepage": "https://example.com/example/project", - "url": "https://example.com/example/project.git", - "ssh_url": "git@example.com:example/project.git", - "http_url": "https://example.com/example/project.git" - }, - "object_attributes": { - "id": 99, - "iid": 1, - "target_branch": "main", - "source_branch": "feature-branch", - "source_project_id": 15, - "target_project_id": 15, - "title": "Merge feature-branch into main", - "state": "opened", - "merge_status": "can_be_merged", - "url": "https://example.com/example/project/-/merge_requests/1", - "created_at": "2025-02-10T12:34:56Z", - "updated_at": "2025-02-10T12:34:56Z" - }, - "changes": { - "total_changes": 51, - "files": [ - {"old_path": "file1.txt", "new_path": "file1.txt", "a_mode": "100644", "b_mode": "100644", "diff": "diff content"}, - # ... 50 more file changes ... - ] - } - } - main_handle.merge_handle(changes, info, hook_info, reply, model) + diff --git a/review_engine/review_engine.py b/review_engine/review_engine.py index 1837799..c323107 100644 --- a/review_engine/review_engine.py +++ b/review_engine/review_engine.py @@ -14,13 +14,14 @@ def __init__(self, reply): for handle in ReviewHandle.__subclasses__(): self.handles.append(handle()) - def handle_merge(self, changes, merge_info, webhook_info): + def handle_merge(self, gitlabMergeRequestFetcher, gitlabRepoManager, webhook_info): # 多线程处理 - threads = [threading.Thread(target=handle.merge_handle, args=(changes, merge_info, webhook_info, - self.reply, LLMGenerator.new_model())) - for handle in self.handles] + threads = [threading.Thread(target=handle.merge_handle, + args=(gitlabMergeRequestFetcher, gitlabRepoManager, webhook_info, self.reply, + LLMGenerator.new_model())) for handle in self.handles] for thread in threads: thread.start() for thread in threads: thread.join() + gitlabRepoManager.delete_repo() self.reply.send() \ No newline at end of file diff --git a/utils/args_check.py b/utils/args_check.py index 81ee1a6..51f68a9 100644 --- a/utils/args_check.py +++ b/utils/args_check.py @@ -54,8 +54,8 @@ def check_dingding_config(config): """ result = {'passed': True, 'errors': []} try: - from reply_module.reply_target.dingtalk_reply import DingtalkReply - dingtalk_reply = DingtalkReply({'type': 'merge_request', 'project_id': 1, 'merge_request_iid': 1}) + from response_module.response_target.msg_response.dingtalk_response import DingtalkResponse + dingtalk_reply = DingtalkResponse({'type': 'merge_request', 'project_id': 1, 'merge_request_iid': 1}) response = dingtalk_reply.send("连通性测试:测试消息,请勿回复。") if not response: error_msg = "Dingding configuration is invalid" diff --git a/utils/gitlab_parser.py b/utils/gitlab_parser.py index b551408..f329c6d 100644 --- a/utils/gitlab_parser.py +++ b/utils/gitlab_parser.py @@ -6,4 +6,27 @@ def filter_diff_content(diff_content): filtered_content = re.sub(r'(^-.*\n)|(^@@.*\n)', '', diff_content, flags=re.MULTILINE) # 处理代码,去掉以 + 开头的行的第一个字符 processed_code = '\n'.join([line[1:] if line.startswith('+') else line for line in filtered_content.split('\n')]) - return processed_code \ No newline at end of file + return processed_code + +def filter_diff_new_line(diff_content): + # 获取diff中的行号 + line_numbers = [] + current_line_num = None + + for line in diff_content.split('\n'): + if line.startswith('@@'): + # 提取新的行号 + match = re.match(r'@@ -\d+(,\d+)? \+(\d+)(,\d+)? @@', line) + if match: + current_line_num = int(match.group(2)) + line_numbers.append(current_line_num) + if match.group(3): + # 去除match.group(3)的,然后转成int + current_line_num += int(match.group(3)[1:]) - 1 + line_numbers.append(current_line_num) + + return line_numbers + +if __name__ == "__main__": + diff_content = "@@ -3 +1,5 @@\n-hello\n+hello world\n" + print(filter_diff_new_line(diff_content)) \ No newline at end of file diff --git a/utils/tools.py b/utils/tools.py index d382acf..0611d01 100644 --- a/utils/tools.py +++ b/utils/tools.py @@ -1,8 +1,11 @@ import importlib import os import pkgutil +import subprocess import sys +from utils.logger import log + def import_submodules(package_name): # 确保正确的工作目录 @@ -14,9 +17,60 @@ def import_submodules(package_name): for _, module_name, _ in pkgutil.iter_modules(package.__path__): importlib.import_module(f"{package_name}.{module_name}") +def run_command(command): + process = subprocess.Popen( + command, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True + ) + + while True: + output = process.stdout.readline() + if output == '' and process.poll() is not None: + break + if output: + log.info(output.strip()) + + process.wait() + out = process.communicate() + out_len = len(out) + # 获取是否有[1] + if out_len > 1: + stdout_output = out[1] + if stdout_output: + log.info(stdout_output.strip()) + if out_len > 2: + stderr_output = out[2] + if stderr_output: + log.error(stderr_output.strip()) + return process.returncode if __name__ == "__main__": - from review_engine.review_engine import ReviewEngine - from reply_module.reply import Reply - re = ReviewEngine(Reply(9885, 18)) - re.handle_merge("changes", "info") \ No newline at end of file + from config.config import * + def _build_authenticated_url(repo_url): + # 如果 URL 使用 https + token = gitlab_private_token + if repo_url.startswith("https://"): + return f"https://oauth2:{token}@{repo_url[8:]}" + # 如果 URL 使用 http + elif repo_url.startswith("http://"): + return f"http://oauth2:{token}@{repo_url[7:]}" + else: + raise ValueError("Unsupported URL scheme") + authenticated_url = _build_authenticated_url(gitlab_server_url) + + # Build the Git command + branch_name = "test3" + repo_path = "./repo" + command = ["git", "clone", "--depth", "1"] + if branch_name: + command.extend(["--branch", branch_name]) + command.extend([authenticated_url, repo_path + "/" + str(branch_name)]) + else: + command.extend([authenticated_url, repo_path + "/default"]) + # command 打印为字符串 + print(" ".join(command)) + # command 添加clone到的位置: + if run_command(command) != 0: + log.error("Failed to clone the repository") \ No newline at end of file