Skip to content

Commit f061a36

Browse files
qiong-zhouqiong-zhou
committed
Merge remote-tracking branch 'origin/Improvement-17788' into Improvement-17788
2 parents 57099df + 6e059cc commit f061a36

File tree

40 files changed

+306
-481
lines changed

40 files changed

+306
-481
lines changed

‎.github/workflows/cluster-test/mysql_with_mysql_registry/running_test.sh‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -46,22 +46,22 @@ do
4646

4747
if [[ $i -eq $TIMEOUT ]];then
4848
if [[ $MASTER_HTTP_STATUS -ne 200 ]];then
49-
cat ~/ds_test/master-server/logs/dolphinscheduler-master.log
49+
cat ~/ds_test/master-server/logs/*.log
5050
cat ~/ds_test/master-server/logs/*.out
5151
echo "master start health check failed"
5252
fi
5353
if [[ $WORKER_HTTP_STATUS -ne 200 ]]; then
54-
cat ~/ds_test/worker-server/logs/dolphinscheduler-master.log
54+
cat ~/ds_test/worker-server/logs/*.log
5555
cat ~/ds_test/worker-server/logs/*.out
5656
echo "worker start health check failed"
5757
fi
5858
if [[ $API_HTTP_STATUS -ne 200 ]]; then
59-
cat ~/ds_test/api-server/logs/dolphinscheduler-master.log
59+
cat ~/ds_test/api-server/logs/*.log
6060
cat ~/ds_test/api-server/logs/*.out
6161
echo "api start health check failed"
6262
fi
6363
if [[ $ALERT_HTTP_STATUS -ne 200 ]]; then
64-
cat ~/ds_test/alert-server/logs/dolphinscheduler-master.log
64+
cat ~/ds_test/alert-server/logs/*.log
6565
cat ~/ds_test/alert-server/logs/*.out
6666
echo "alert start health check failed"
6767
fi

‎.github/workflows/cluster-test/mysql_with_zookeeper_registry/running_test.sh‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -46,22 +46,22 @@ do
4646

4747
if [[ $i -eq $TIMEOUT ]];then
4848
if [[ $MASTER_HTTP_STATUS -ne 200 ]];then
49-
cat ~/ds_test/master-server/logs/dolphinscheduler-master.log
49+
cat ~/ds_test/master-server/logs/*.log
5050
cat ~/ds_test/master-server/logs/*.out
5151
echo "master start health check failed"
5252
fi
5353
if [[ $WORKER_HTTP_STATUS -ne 200 ]]; then
54-
cat ~/ds_test/worker-server/logs/dolphinscheduler-master.log
54+
cat ~/ds_test/worker-server/logs/*.log
5555
cat ~/ds_test/worker-server/logs/*.out
5656
echo "worker start health check failed"
5757
fi
5858
if [[ $API_HTTP_STATUS -ne 200 ]]; then
59-
cat ~/ds_test/api-server/logs/dolphinscheduler-master.log
59+
cat ~/ds_test/api-server/logs/*.log
6060
cat ~/ds_test/api-server/logs/*.out
6161
echo "api start health check failed"
6262
fi
6363
if [[ $ALERT_HTTP_STATUS -ne 200 ]]; then
64-
cat ~/ds_test/alert-server/logs/dolphinscheduler-master.log
64+
cat ~/ds_test/alert-server/logs/*.log
6565
cat ~/ds_test/alert-server/logs/*.out
6666
echo "alert start health check failed"
6767
fi

‎.github/workflows/cluster-test/postgresql_with_postgresql_registry/running_test.sh‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -46,22 +46,22 @@ do
4646

4747
if [[ $i -eq $TIMEOUT ]];then
4848
if [[ $MASTER_HTTP_STATUS -ne 200 ]];then
49-
cat ~/ds_test/master-server/logs/dolphinscheduler-master.log
49+
cat ~/ds_test/master-server/logs/*.log
5050
cat ~/ds_test/master-server/logs/*.out
5151
echo "master start health check failed"
5252
fi
5353
if [[ $WORKER_HTTP_STATUS -ne 200 ]]; then
54-
cat ~/ds_test/worker-server/logs/dolphinscheduler-master.log
54+
cat ~/ds_test/worker-server/logs/*.log
5555
cat ~/ds_test/worker-server/logs/*.out
5656
echo "worker start health check failed"
5757
fi
5858
if [[ $API_HTTP_STATUS -ne 200 ]]; then
59-
cat ~/ds_test/api-server/logs/dolphinscheduler-master.log
59+
cat ~/ds_test/api-server/logs/*.log
6060
cat ~/ds_test/api-server/logs/*.out
6161
echo "api start health check failed"
6262
fi
6363
if [[ $ALERT_HTTP_STATUS -ne 200 ]]; then
64-
cat ~/ds_test/alert-server/logs/dolphinscheduler-master.log
64+
cat ~/ds_test/alert-server/logs/*.log
6565
cat ~/ds_test/alert-server/logs/*.out
6666
echo "alert start health check failed"
6767
fi

‎.github/workflows/cluster-test/postgresql_with_zookeeper_registry/running_test.sh‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -46,22 +46,22 @@ do
4646

4747
if [[ $i -eq $TIMEOUT ]];then
4848
if [[ $MASTER_HTTP_STATUS -ne 200 ]];then
49-
cat ~/ds_test/master-server/logs/dolphinscheduler-master.log
49+
cat ~/ds_test/master-server/logs/*.log
5050
cat ~/ds_test/master-server/logs/*.out
5151
echo "master start health check failed"
5252
fi
5353
if [[ $WORKER_HTTP_STATUS -ne 200 ]]; then
54-
cat ~/ds_test/worker-server/logs/dolphinscheduler-master.log
54+
cat ~/ds_test/worker-server/logs/*.log
5555
cat ~/ds_test/worker-server/logs/*.out
5656
echo "worker start health check failed"
5757
fi
5858
if [[ $API_HTTP_STATUS -ne 200 ]]; then
59-
cat ~/ds_test/api-server/logs/dolphinscheduler-master.log
59+
cat ~/ds_test/api-server/logs/*.log
6060
cat ~/ds_test/api-server/logs/*.out
6161
echo "api start health check failed"
6262
fi
6363
if [[ $ALERT_HTTP_STATUS -ne 200 ]]; then
64-
cat ~/ds_test/alert-server/logs/dolphinscheduler-master.log
64+
cat ~/ds_test/alert-server/logs/*.log
6565
cat ~/ds_test/alert-server/logs/*.out
6666
echo "alert start health check failed"
6767
fi

‎.github/workflows/docs.yml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,6 @@ jobs:
108108
needs:
109109
- style
110110
- img-check
111-
- dead-link
112111
if: always()
113112
steps:
114113
- name: Status

‎deploy/kubernetes/dolphinscheduler/README.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -306,8 +306,9 @@ Please refer to the [Quick Start in Kubernetes](../../../docs/docs/en/guide/inst
306306
| security.authentication.ldap.ssl.jksbase64content | string | `""` | LDAP jks file base64 content. If you use macOS, please run `base64 -b 0 -i /path/to/your.jks`. If you use Linux, please run `base64 -w 0 /path/to/your.jks`. If you use Windows, please run `certutil -f -encode /path/to/your.jks`. Then copy the base64 content to below field in one line |
307307
| security.authentication.ldap.ssl.truststore | string | `"/opt/ldapkeystore.jks"` | LDAP jks file absolute path, do not change this value |
308308
| security.authentication.ldap.ssl.truststorepassword | string | `""` | LDAP jks password |
309-
| security.authentication.ldap.urls | string | `"ldap://ldap.forumsys.com:389/"` | LDAP urls |
310-
| security.authentication.ldap.user.admin | string | `"read-only-admin"` | Admin user account when you log-in with LDAP |
309+
| security.authentication.ldap.url | string | `"ldap://ldap.forumsys.com:389/"` | LDAP url |
310+
| security.authentication.ldap.user.adminuserfilter | string | `"(&(uid={0})(memberOf=cn=admin-group,ou=apps,dc=example,dc=com))"` | Any user matching this filter becomes an Admin. If set to `""`, ignore this option |
311+
| security.authentication.ldap.user.adminusername | string | `"read-only-admin"` | Admin user account when you log-in with LDAP. If the filter doesn't match anything this also acts as fallback. Set to `""` to ignore this option if you want to use filter only |
311312
| security.authentication.ldap.user.emailattribute | string | `"mail"` | LDAP user email attribute |
312313
| security.authentication.ldap.user.identityattribute | string | `"uid"` | LDAP user identity attribute |
313314
| security.authentication.ldap.user.notexistaction | string | `"CREATE"` | action when ldap user is not exist,default value: CREATE. Optional values include(CREATE,DENY) |

‎deploy/kubernetes/dolphinscheduler/templates/_helpers.tpl‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -195,16 +195,18 @@ Create a security environment variables.
195195
- name: SECURITY_AUTHENTICATION_TYPE
196196
value: {{ .Values.security.authentication.type | quote }}
197197
{{- if eq .Values.security.authentication.type "LDAP" }}
198-
- name: SECURITY_AUTHENTICATION_LDAP_URLS
199-
value: {{ .Values.security.authentication.ldap.urls | quote }}
198+
- name: SECURITY_AUTHENTICATION_LDAP_URL
199+
value: {{ .Values.security.authentication.ldap.url | quote }}
200200
- name: SECURITY_AUTHENTICATION_LDAP_BASE_DN
201201
value: {{ .Values.security.authentication.ldap.basedn | quote }}
202202
- name: SECURITY_AUTHENTICATION_LDAP_USERNAME
203203
value: {{ .Values.security.authentication.ldap.username | quote }}
204204
- name: SECURITY_AUTHENTICATION_LDAP_PASSWORD
205205
value: {{ .Values.security.authentication.ldap.password | quote }}
206-
- name: SECURITY_AUTHENTICATION_LDAP_USER_ADMIN
207-
value: {{ .Values.security.authentication.ldap.user.admin | quote }}
206+
- name: SECURITY_AUTHENTICATION_LDAP_USER_ADMIN_USERNAME
207+
value: {{ .Values.security.authentication.ldap.user.adminusername | quote }}
208+
- name: SECURITY_AUTHENTICATION_LDAP_USER_ADMIN_USER_FILTER
209+
value: {{ .Values.security.authentication.ldap.user.adminuserfilter | quote }}
208210
- name: SECURITY_AUTHENTICATION_LDAP_USER_IDENTITY_ATTRIBUTE
209211
value: {{ .Values.security.authentication.ldap.user.identityattribute | quote }}
210212
- name: SECURITY_AUTHENTICATION_LDAP_USER_EMAIL_ATTRIBUTE

‎deploy/kubernetes/dolphinscheduler/values.yaml‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -221,17 +221,19 @@ security:
221221
type: PASSWORD
222222
# IF you set type `LDAP`, below config will be effective
223223
ldap:
224-
# -- LDAP urls
225-
urls: ldap://ldap.forumsys.com:389/
224+
# -- LDAP url
225+
url: ldap://ldap.forumsys.com:389/
226226
# -- LDAP base dn
227227
basedn: dc=example,dc=com
228228
# -- LDAP username
229229
username: cn=read-only-admin,dc=example,dc=com
230230
# -- LDAP password
231231
password: password
232232
user:
233-
# -- Admin user account when you log-in with LDAP
234-
admin: read-only-admin
233+
# -- Any user matching this filter becomes an Admin. If set to `""`, ignore this option
234+
adminuserfilter: (&(uid={0})(memberOf=cn=admin-group,ou=apps,dc=example,dc=com))
235+
# -- Admin user account when you log-in with LDAP. If the filter doesn't match anything this also acts as fallback. Set to `""` to ignore this option if you want to use filter only
236+
adminusername: read-only-admin
235237
# -- LDAP user identity attribute
236238
identityattribute: uid
237239
# -- LDAP user email attribute

‎docs/docs/en/guide/installation/registry-plugins/jdbc.md‎

Lines changed: 4 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -38,19 +38,13 @@ registry:
3838
heartbeat-refresh-interval: 3s
3939
# Once the client's heartbeat is not refresh in this time, the server will consider the client is offline.
4040
session-timeout: 60s
41-
# The hikari configuration, default will use the same datasource pool as DolphinScheduler.
42-
hikari-config:
43-
jdbc-url: jdbc:mysql://127.0.0.1:3306/dolphinscheduler
44-
username: root
45-
password: root
46-
maximum-pool-size: 5
47-
connection-timeout: 9000
48-
idle-timeout: 600000
4941
```
5042

51-
## Use different database configuration for jdbc registry center
43+
## Set DataSource for worker
5244

53-
You need to set the registry properties in master/worker/api's application.yml
45+
Since worker server doesn't contain datasource, so you need to config datasource for worker.
46+
47+
You need to set the registry hikari-config properties at worker's application.yml
5448

5549
### Use Mysql as registry center
5650

‎docs/docs/en/guide/upgrade/incompatible.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,4 +36,6 @@ This document records the incompatible updates between each version. You need to
3636

3737
* Renamed the publicKey field to privateKey in the SSH connection parameters under the datasource configuration. ([#17666])(https://github.com/apache/dolphinscheduler/pull/17666)
3838
* Add table t_ds_serial_command. ([#17531])(https://github.com/apache/dolphinscheduler/pull/17531)
39+
* Remove the default value of `python-gateway.auth-token` at `api-server/application.yaml`. ([#17801])(https://github.com/apache/dolphinscheduler/pull/17801)
40+
* Refactor the task plugins which use ShellCommandExecutor ([#17790])(https://github.com/apache/dolphinscheduler/pull/17790)
3941

0 commit comments

Comments
 (0)