Skip to content

Commit b4f9a60

Browse files
njnu-seafishnjnu-seafish
authored
Merge branch 'dev' into Improvement-17874
2 parents 33f6b50 + b12b682 commit b4f9a60

File tree

27 files changed

+765
-93
lines changed

27 files changed

+765
-93
lines changed

‎.github/actions/maximize-build-space‎

Lines changed: 0 additions & 1 deletion
This file was deleted.

‎.github/workflows/api-test.yml‎

Lines changed: 9 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -51,19 +51,13 @@ jobs:
5151
runs-on: ubuntu-latest
5252
timeout-minutes: 20
5353
steps:
54-
- uses: actions/checkout@v4
55-
with:
56-
submodules: true
57-
- name: Maximize runner space
58-
uses: ./.github/actions/maximize-build-space
59-
with:
60-
root-reserve-mb: 28672
61-
temp-reserve-mb: 10240
62-
remove-dotnet: 'true'
63-
remove-android: 'true'
64-
remove-haskell: 'true'
65-
remove-codeql: 'true'
66-
remove-docker-images: 'true'
54+
- run: df -h
55+
- name: "node-cleanup"
56+
run: |
57+
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
58+
sudo docker image prune --all --force
59+
sudo docker builder prune -a
60+
- run: df -h
6761
- uses: actions/checkout@v4
6862
with:
6963
submodules: true
@@ -152,6 +146,8 @@ jobs:
152146
- name: Load Docker Images
153147
run: |
154148
docker load -i /tmp/standalone-image.tar
149+
- name: df -h
150+
run: df -h
155151
- name: Run Test
156152
run: |
157153
./mvnw -B -f dolphinscheduler-api-test/pom.xml -am \

‎.github/workflows/backend.yml‎

Lines changed: 8 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -56,19 +56,6 @@ jobs:
5656
java: [ '8', '11' ]
5757
timeout-minutes: 30
5858
steps:
59-
- uses: actions/checkout@v4
60-
with:
61-
submodules: true
62-
- name: Maximize runner space
63-
uses: ./.github/actions/maximize-build-space
64-
with:
65-
root-reserve-mb: 28672
66-
temp-reserve-mb: 10240
67-
remove-dotnet: 'true'
68-
remove-android: 'true'
69-
remove-haskell: 'true'
70-
remove-codeql: 'true'
71-
remove-docker-images: 'true'
7259
- uses: actions/checkout@v4
7360
with:
7461
submodules: true
@@ -86,6 +73,7 @@ jobs:
8673
path: ~/.m2/repository
8774
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}-backend
8875
restore-keys: ${{ runner.os }}-maven-
76+
- run: df -h
8977
- name: Build and Package on ${{ matrix.java }}
9078
run: |
9179
./mvnw -B clean install \
@@ -119,6 +107,13 @@ jobs:
119107
- name: cluster-test-postgresql-with-postgresql-registry
120108
script: .github/workflows/cluster-test/postgresql_with_postgresql_registry/start-job.sh
121109
steps:
110+
- run: df -h
111+
- name: "node-cleanup"
112+
run: |
113+
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
114+
sudo docker image prune --all --force
115+
sudo docker builder prune -a
116+
- run: df -h
122117
- uses: actions/checkout@v4
123118
with:
124119
submodules: true

‎.github/workflows/e2e-k8s.yml‎

Lines changed: 8 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -51,22 +51,17 @@ jobs:
5151
runs-on: ubuntu-latest
5252
timeout-minutes: 40
5353
steps:
54+
- run: df -h
55+
- name: "node-cleanup"
56+
run: |
57+
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
58+
sudo docker image prune --all --force
59+
sudo docker builder prune -a
60+
- run: df -h
5461
- uses: actions/checkout@v4
5562
with:
5663
submodules: true
57-
- name: Maximize runner space
58-
uses: ./.github/actions/maximize-build-space
59-
with:
60-
root-reserve-mb: 28672
61-
temp-reserve-mb: 10240
62-
remove-dotnet: 'true'
63-
remove-android: 'true'
64-
remove-haskell: 'true'
65-
remove-codeql: 'true'
66-
remove-docker-images: 'true'
67-
- uses: actions/checkout@v4
68-
with:
69-
submodules: true
64+
- run: df -h
7065
- name: Build Image
7166
run: |
7267
./mvnw -B clean package \

‎.github/workflows/e2e.yml‎

Lines changed: 8 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -53,19 +53,6 @@ jobs:
5353
runs-on: ubuntu-latest
5454
timeout-minutes: 20
5555
steps:
56-
- uses: actions/checkout@v4
57-
with:
58-
submodules: true
59-
- name: Maximize runner space
60-
uses: ./.github/actions/maximize-build-space
61-
with:
62-
root-reserve-mb: 28672
63-
temp-reserve-mb: 10240
64-
remove-dotnet: 'true'
65-
remove-android: 'true'
66-
remove-haskell: 'true'
67-
remove-codeql: 'true'
68-
remove-docker-images: 'true'
6956
- uses: actions/checkout@v4
7057
with:
7158
submodules: true
@@ -150,19 +137,13 @@ jobs:
150137
env:
151138
RECORDING_PATH: /tmp/recording-${{ matrix.case.name }}
152139
steps:
153-
- uses: actions/checkout@v4
154-
with:
155-
submodules: true
156-
- name: Maximize runner space
157-
uses: ./.github/actions/maximize-build-space
158-
with:
159-
root-reserve-mb: 28672
160-
temp-reserve-mb: 10240
161-
remove-dotnet: 'true'
162-
remove-android: 'true'
163-
remove-haskell: 'true'
164-
remove-codeql: 'true'
165-
remove-docker-images: 'true'
140+
- run: df -h
141+
- name: "node-cleanup"
142+
run: |
143+
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /opt/hostedtoolcache/CodeQL
144+
sudo docker image prune --all --force
145+
sudo docker builder prune -a
146+
- run: df -h
166147
- uses: actions/checkout@v4
167148
with:
168149
submodules: true
@@ -189,6 +170,7 @@ jobs:
189170
- name: Load Docker Images
190171
run: |
191172
docker load -i /tmp/standalone-image.tar
173+
- run: df -h
192174
- name: Run Test
193175
run: |
194176
./mvnw -B -f dolphinscheduler-e2e/pom.xml -am \

‎.gitmodules‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,3 @@
2828
[submodule ".github/actions/workflow-telemetry-action"]
2929
path = .github/actions/workflow-telemetry-action
3030
url = https://github.com/catchpoint/workflow-telemetry-action
31-
[submodule ".github/actions/maximize-build-space"]
32-
path = .github/actions/maximize-build-space
33-
url = https://github.com/easimon/maximize-build-space

‎README.md‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,6 @@
44
![codecov](https://codecov.io/gh/apache/dolphinscheduler/branch/dev/graph/badge.svg)
55
[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=apache-dolphinscheduler&metric=alert_status)](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler)
66
[![Twitter Follow](https://img.shields.io/twitter/follow/dolphinschedule.svg?style=social&label=Follow)](https://twitter.com/dolphinschedule) <!-- markdown-link-check-disable-line -->
7-
[![Slack Status](https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack&style=social)](https://s.apache.org/dolphinscheduler-slack)
87
[![CN doc](https://img.shields.io/badge/文档-中文版-blue.svg)](README_zh_CN.md)
98

109
## About
@@ -64,7 +63,7 @@ Check out good first issue in [here](https://github.com/apache/dolphinscheduler/
6463

6564
Welcome to join the Apache DolphinScheduler community by:
6665

67-
- Join the [DolphinScheduler Slack](https://s.apache.org/dolphinscheduler-slack) to keep in touch with the community
66+
- Use [GitHub Issues](https://github.com/apache/dolphinscheduler/issues) for questions, discussions, and bug reports
6867
- Follow the [DolphinScheduler Twitter](https://twitter.com/dolphinschedule) and get the latest news <!-- markdown-link-check-disable-line -->
6968
- Subscribe DolphinScheduler mail list, [users@dolphinscheduler.apache.org](mailto:users-subscribe@dolphinscheduler.apache.org) for users and [dev@dolphinscheduler.apache.org](mailto:dev-subscribe@dolphinscheduler.apache.org) for developers
7069

‎README_zh_CN.md‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,6 @@
44
[![codecov](https://codecov.io/gh/apache/dolphinscheduler/branch/dev/graph/badge.svg)]()
55
[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=apache-dolphinscheduler&metric=alert_status)](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler)
66
[![Twitter Follow](https://img.shields.io/twitter/follow/dolphinschedule.svg?style=social&label=Follow)](https://twitter.com/dolphinschedule) <!-- markdown-link-check-disable-line -->
7-
[![Slack Status](https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack&style=social)](https://s.apache.org/dolphinscheduler-slack)
87
[![EN doc](https://img.shields.io/badge/document-English-blue.svg)](README.md)
98

109
## 关于
@@ -60,7 +59,7 @@ DolphinScheduler 的主要特性如下：
6059

6160
欢迎通过以方式加入社区：
6261

63-
- 加入 [DolphinScheduler Slack](https://s.apache.org/dolphinscheduler-slack)
62+
- 讨论和提问请通过 [GitHub Issues](https://github.com/apache/dolphinscheduler/issues)
6463
- 关注 [DolphinScheduler Twitter](https://twitter.com/dolphinschedule) 来获取最新消息 <!-- markdown-link-check-disable-line -->
6564
- 订阅 DolphinScheduler 邮件列表, 用户订阅 [users@dolphinscheduler.apache.org](mailto:users-subscribe@dolphinscheduler.apache.org) 开发者请订阅 [dev@dolphinscheduler.apache.org](mailto:dev-subscribe@dolphinscheduler.apache.org)
6665

‎docs/docs/en/architecture/configuration.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -291,6 +291,8 @@ Location: `master-server/conf/application.yaml`
291291
| master.command-fetch-strategy.type | ID_SLOT_BASED | The command fetch strategy, only support `ID_SLOT_BASED` |
292292
| master.command-fetch-strategy.config.id-step | 1 | The id auto incremental step of t_ds_command in db |
293293
| master.command-fetch-strategy.config.fetch-size | 10 | The number of commands fetched by master |
294+
| master.task-dispatch-policy.dispatch-timeout-enabled | false | Indicates whether the dispatch timeout checking mechanism is enabled |
295+
| master.task-dispatch-policy.max-task-dispatch-duration | 1h | The maximum allowed duration a task may wait in the dispatch queue before being assigned to a worker |
294296

295297
### Worker Server related configuration
296298

‎docs/docs/zh/architecture/configuration.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -298,6 +298,8 @@ common.properties配置文件目前主要是配置hadoop/s3/yarn/applicationId
298298
| master.command-fetch-strategy.type | ID_SLOT_BASED | Command拉取策略, 目前仅支持 `ID_SLOT_BASED` |
299299
| master.command-fetch-strategy.config.id-step | 1 | 数据库中t_ds_command的id自增步长 |
300300
| master.command-fetch-strategy.config.fetch-size | 10 | master拉取command数量 |
301+
| master.task-dispatch-policy.dispatch-timeout-enabled | false | 是否开启master分派超时检测功能 |
302+
| master.task-dispatch-policy.max-task-dispatch-duration | 1h | master分派检测的超时时长，默认为一小时 |
301303

302304
## Worker Server相关配置
303305

0 commit comments

Comments
 (0)