Skip to content

Commit ea869b1

Browse files
committed
Merge branch 'dev-new-web' into dev-1.2.0_k8s
# Conflicts: # web/src/apps/scriptis/module/workbench/script/script.vue # web/src/dss/module/footer/guide.vue
2 parents 6ce7b23 + fc1fed0 commit ea869b1

File tree

1,201 files changed

+74105
-10163
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

1,201 files changed

+74105
-10163
lines changed

.vscode/settings.json

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"editor.formatOnPaste": true,
3+
"editor.formatOnType": true,
4+
"editor.formatOnSave": true
5+
}

README-ZH.md

Lines changed: 73 additions & 111 deletions
Large diffs are not rendered by default.

README.md

Lines changed: 19 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -33,75 +33,77 @@ Please be patient, it will take some time to load gif.
3333

3434
       c. [Qualitis](https://github.com/WeBankFinTech/Qualitis) - Data Quality Management Tool
3535

36-
       d. [Azkaban](https://azkaban.github.io/) - Batch workflow job scheduler
36+
       d. [Schedulis](https://github.com/WeBankFinTech/Schedulis) - Batch workflow job scheduler
37+
38+
       f. [Exchangis](https://github.com/WeBankFinTech/Exchangis) - Data Exchange Tool
3739

3840
![DSS one-stop video](images/en_US/readme/onestop.gif)
3941

40-
### 2. AppJoint, based on Linkis,defines a unique design concept
42+
### 2. AppConn, based on Linkis,defines a unique design concept
4143

42-
       AppJoint——application joint, defining unified front-end and back-end
44+
       AppConn——application connector, defining unified front-end and back-end
4345
integration specifications, can quickly and easily integrate with external data application systems,
4446
making them as part of DSS data application development.
4547

46-
       DSS arranges multiple AppJoints in series to form a workflow that supports real-time execution and scheduled execution. Users can complete the entire process development of data applications with simple drag and drop operations.
48+
       DSS arranges multiple AppConns in series to form a workflow that supports real-time execution and scheduled execution. Users can complete the entire process development of data applications with simple drag and drop operations.
4749

48-
       Since AppJoint is integrated with Linkis, the external data application system shares the capabilities of resource management, concurrent limiting, and high performance. AppJoint also allows sharable context across system level and completely gets away from application silos.
50+
       Since AppConn is integrated with Linkis, the external data application system shares the capabilities of resource management, concurrent limiting, and high performance. AppConn also allows sharable context across system level and completely gets away from application silos.
4951

5052
### 3. Project, as the management unit
5153

5254
       With Project as the management unit, DSS organizes and manages the business applications of each data application system, and defines a set of common standards for collaborative development of projects across data application systems.
5355

5456
### 4. Integrated data application components
5557

56-
      a. Azkaban AppJoint —— Batch workflow job scheduler
58+
      a. Schedulis AppConn —— Batch workflow job scheduler
5759

5860
         Many data applications developed by users usually require periodic scheduling capability.
5961

6062
         At present, the open source scheduling system in the community is pretty unfriendly to integrate with other data application systems.
6163

62-
         DSS implements Azkaban AppJoint, which allows users to publish DSS workflows to Azkaban for regular scheduling.
64+
         DSS implements Schedulis AppConn, which allows users to publish DSS workflows to Azkaban for regular scheduling.
6365

6466
         DSS also defines standard and generic workflow parsing and publishing specifications for scheduling systems, allowing other scheduling systems to easily achieve low-cost integration with DSS.
6567

6668
![Azkaban](images/en_US/readme/Azkaban_AppJoint.gif)
6769

68-
      b. Scriptis AppJoint —— Data Development IDE Tool
70+
      b. Scriptis AppConn —— Data Development IDE Tool
6971

7072
         What is [Scriptis](https://github.com/WeBankFinTech/Scriptis)?
7173

7274
         Scriptis is for interactive data analysis with script development(SQL, Pyspark, HiveQL), task submission(Spark, Hive), UDF, function, resource management and intelligent diagnosis.
7375

74-
         Scriptis AppJoint integrates the data development capabilities of Scriptis to DSS, and allows various script types of Scriptis to serve as nodes in the DSS workflow to participate in the application development process.
76+
         Scriptis AppConn integrates the data development capabilities of Scriptis to DSS, and allows various script types of Scriptis to serve as nodes in the DSS workflow to participate in the application development process.
7577

7678
         Currently supports HiveSQL, SparkSQL, Pyspark, Scala and other script node types.
7779

7880
![Scriptis](images/en_US/readme/Scriptis_AppJoint.gif)
7981

80-
      c. Visualis AppJoint —— Data Visualization Tool
82+
      c. Visualis AppConn —— Data Visualization Tool
8183

8284
         What is [Visualis](https://github.com/WeBankFinTech/Visualis)?
8385

8486
         Visualis is a BI tool for data visualization. It provides financial-grade data visualization capabilities on the basis of data security and permissions, based on the open source project Davinci contributed by CreditEase.
8587

86-
         Visualis AppJoint integrates data visualization capabilities to DSS, and allows displays and dashboards, as nodes of DSS workflows, to be associated with upstream data market.
88+
         Visualis AppConn integrates data visualization capabilities to DSS, and allows displays and dashboards, as nodes of DSS workflows, to be associated with upstream data market.
8789

8890
![Visualis](images/en_US/readme/Visualis_AppJoint.gif)
8991

90-
      d. Qualitis AppJoint —— Data quality management Tool
92+
      d. Qualitis AppConn —— Data quality management Tool
9193

92-
         Qualitis AppJoint integrates data quality verification capabilities for DSS, allows Qualitis as a node in DSS workflow
94+
         Qualitis AppConn integrates data quality verification capabilities for DSS, allows Qualitis as a node in DSS workflow
9395

9496
![Qualitis](images/en_US/readme/Qualitis_AppJoint.gif)
9597

96-
      e. Data Sender——Sender AppJoint
98+
      e. Data Sender——Sender AppConn
9799

98-
         Sender AppJoint provides data delivery capability for DSS. Currently it supports the SendEmail node type, and the result sets of all other nodes can be sent via email.
100+
         Sender AppConn provides data delivery capability for DSS. Currently it supports the SendEmail node type, and the result sets of all other nodes can be sent via email.
99101

100102
         For example, the SendEmail node can directly send the screen shot of a display as an email.
101103

102-
      f. Signal AppJoint —— Signal Nodes
104+
      f. Signal AppConn —— Signal Nodes
103105

104-
         Signal AppJoint is used to strengthen the correlation between business and process while keeping them decoupled.
106+
         Signal AppConn is used to strengthen the correlation between business and process while keeping them decoupled.
105107

106108
         DataChecker Node:Checks whether a table or partition exists.
107109

assembly/dss-package/pom.xml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -21,13 +21,13 @@
2121
<parent>
2222
<artifactId>dss</artifactId>
2323
<groupId>com.webank.wedatasphere.dss</groupId>
24-
<version>1.0.0</version>
24+
<version>1.0.1</version>
2525
</parent>
2626
<modelVersion>4.0.0</modelVersion>
2727
<artifactId>dss-package</artifactId>
2828
<dependencies>
2929
<dependency>
30-
<groupId>com.webank.wedatasphere.linkis</groupId>
30+
<groupId>org.apache.linkis</groupId>
3131
<artifactId>linkis-rpc</artifactId>
3232
<version>${linkis.version}</version>
3333
<exclusions>
@@ -88,7 +88,7 @@
8888
</dependency>
8989

9090
<dependency>
91-
<groupId>com.webank.wedatasphere.linkis</groupId>
91+
<groupId>org.apache.linkis</groupId>
9292
<artifactId>linkis-hadoop-common</artifactId>
9393
<version>${linkis.version}</version>
9494
</dependency>
@@ -111,12 +111,12 @@
111111
</dependency>
112112

113113
<dependency>
114-
<groupId>com.webank.wedatasphere.linkis</groupId>
114+
<groupId>org.apache.linkis</groupId>
115115
<artifactId>linkis-mybatis</artifactId>
116116
<version>${linkis.version}</version>
117117
</dependency>
118118
<dependency>
119-
<groupId>com.webank.wedatasphere.linkis</groupId>
119+
<groupId>org.apache.linkis</groupId>
120120
<artifactId>linkis-storage</artifactId>
121121
<version>${linkis.version}</version>
122122
</dependency>

assembly/pom.xml

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
<?xml version="1.0" encoding="UTF-8"?>
22
<!--
33
~ Copyright 2019 WeBank
4+
~
45
~ Licensed under the Apache License, Version 2.0 (the "License");
5-
~ you may not use this file except in compliance with the License.
6+
~ you may not use this file except in compliance with the License.
67
~ You may obtain a copy of the License at
78
~
89
~ http://www.apache.org/licenses/LICENSE-2.0
@@ -21,7 +22,7 @@
2122
<parent>
2223
<artifactId>dss</artifactId>
2324
<groupId>com.webank.wedatasphere.dss</groupId>
24-
<version>1.0.0</version>
25+
<version>1.0.1</version>
2526
</parent>
2627
<packaging>pom</packaging>
2728
<modelVersion>4.0.0</modelVersion>

conf/dss-apiservice-server.properties

Lines changed: 13 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,17 @@
11
#
2-
# /*
3-
# * Copyright 2019 WeBank
4-
# *
5-
# * Licensed under the Apache License, Version 2.0 (the "License");
6-
# * you may not use this file except in compliance with the License.
7-
# * You may obtain a copy of the License at
8-
# *
9-
# * http://www.apache.org/licenses/LICENSE-2.0
10-
# *
11-
# * Unless required by applicable law or agreed to in writing, software
12-
# * distributed under the License is distributed on an "AS IS" BASIS,
13-
# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14-
# * See the License for the specific language governing permissions and
15-
# * limitations under the License.
16-
# */
2+
# Copyright 2019 WeBank
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
#
1715
#
1816

1917
# Spring configurations
@@ -38,11 +36,6 @@ wds.linkis.test.user=
3836
#dsm
3937
wds.linkis.server.dsm.admin.users=
4038

41-
#bml
42-
#wds.linkis.gateway.ip=127.0.0.1
43-
#wds.linkis.gateway.port=9001
44-
#wds.linkis.gateway.url=http://127.0.0.1:9001/
45-
4639

4740
#用于执行的datasource配置
4841
wds.linkis.datasource.hikari.maximumPoolSize=100
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
#
2+
# Copyright 2019 WeBank
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
#
15+
#
16+
17+
# Spring configurations
18+
spring.server.port=9009
19+
spring.spring.application.name=dss-datapipe-server
20+
21+
wds.linkis.server.mybatis.mapperLocations=classpath*:com/webank/wedatasphere/dss/datapipe/dao/mapper/*.xml
22+
wds.linkis.server.mybatis.typeAliasesPackage=com.webank.wedatasphere.dss.datapipe.vo
23+
wds.linkis.server.mybatis.BasePackage=com.webank.wedatasphere.dss.datapipe.dao
24+
25+
wds.linkis.server.restful.scan.packages=com.webank.wedatasphere.dss.datapipe.restful
26+
27+
#sit
28+
wds.linkis.server.version=v1
29+
wds.linkis.server.url=
30+
31+
#test
32+
wds.linkis.test.mode=false
33+
wds.linkis.test.user=

conf/dss-flow-execution-server.properties

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -20,16 +20,16 @@ spring.server.port=9006
2020
spring.spring.application.name=dss-flow-entrance
2121

2222
##mybatis
23-
wds.linkis.server.mybatis.mapperLocations=classpath*:com/webank/wedatasphere/dss/flow/execution/entrance/dao/impl/*.xml,classpath*:com/webank/wedatasphere/linkis/jobhistory/dao/impl/*.xml
23+
wds.linkis.server.mybatis.mapperLocations=classpath*:com/webank/wedatasphere/dss/flow/execution/entrance/dao/impl/*.xml,classpath*:org/apache/linkis/jobhistory/dao/impl/*.xml
2424

2525
wds.linkis.server.mybatis.typeAliasesPackage=
2626

27-
wds.linkis.server.mybatis.BasePackage=com.webank.wedatasphere.dss.flow.execution.entrance.dao,com.webank.wedatasphere.linkis.jobhistory.dao
27+
wds.linkis.server.mybatis.BasePackage=com.webank.wedatasphere.dss.flow.execution.entrance.dao,org.apache.linkis.jobhistory.dao
2828

2929

30-
wds.linkis.server.restful.scan.packages=com.webank.wedatasphere.linkis.entrance.restful,com.webank.wedatasphere.dss.flow.execution.entrance.restful
30+
wds.linkis.server.restful.scan.packages=org.apache.linkis.entrance.restful,com.webank.wedatasphere.dss.flow.execution.entrance.restful
3131

32-
#wds.linkis.server.component.exclude.classes=com.webank.wedatasphere.linkis.DataWorkCloudApplication
32+
#wds.linkis.server.component.exclude.classes=org.apache.linkis.DataWorkCloudApplication
3333

3434
wds.linkis.engine.application.name=flowExecutionEngine
3535
wds.linkis.enginemanager.application.name=flowExecution
@@ -45,9 +45,9 @@ wds.linkis.server.socket.mode=true
4545
wds.linkis.client.flow.adminuser=ws
4646
wds.linkis.client.flow.author.user.token=172.0.0.1
4747

48-
wds.linkis.server.component.exclude.classes=com.webank.wedatasphere.linkis.entranceclient.conf.ClientForEntranceSpringConfiguration,com.webank.wedatasphere.linkis.entranceclient.conf.ClientSpringConfiguration
48+
wds.linkis.server.component.exclude.classes=org.apache.linkis.entranceclient.conf.ClientForEntranceSpringConfiguration,org.apache.linkis.entranceclient.conf.ClientSpringConfiguration
4949

50-
wds.linkis.server.component.exclude.packages=com.webank.wedatasphere.linkis.entrance.restful.
50+
wds.linkis.server.component.exclude.packages=org.apache.linkis.entrance.restful.
5151
spring.spring.main.allow-bean-definition-overriding=true
5252

5353
wds.linkis.entrance.config.log.path=file:///appcom/tmp/dss/

conf/dss.properties

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,4 +28,10 @@ wds.dss.esb.appid=
2828
wds.dss.esb.token=
2929

3030
wds.dss.appconn.scheduler.job.label=dev
31-
wds.dss.ds.token=172.0.0.1
31+
wds.dss.ds.token=172.0.0.1
32+
33+
34+
wds.linkis.reflect.scan.package=org.apache.linkis,com.webank.wedatasphere.dss
35+
spring.spring.mvc.servlet.path=/api/rest_j/v1
36+
spring.spring.servlet.multipart.max-file-size=200MB
37+
spring.spring.servlet.multipart.max-request-size=200MB

conf/log4j.properties

Lines changed: 13 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,17 @@
11
#
2-
# /*
3-
# * Copyright 2019 WeBank
4-
# *
5-
# * Licensed under the Apache License, Version 2.0 (the "License");
6-
# * you may not use this file except in compliance with the License.
7-
# * You may obtain a copy of the License at
8-
# *
9-
# * http://www.apache.org/licenses/LICENSE-2.0
10-
# *
11-
# * Unless required by applicable law or agreed to in writing, software
12-
# * distributed under the License is distributed on an "AS IS" BASIS,
13-
# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14-
# * See the License for the specific language governing permissions and
15-
# * limitations under the License.
16-
# */
2+
# Copyright 2019 WeBank
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
#
1715
#
1816

1917
### set log levels ###

0 commit comments

Comments
 (0)