Skip to content

Commit 9567f35

Browse files
zqburdezqburde
authored
Merge pull request #82 from zqburde/main
Update DolphinScheduler install link and add dss one click install doc.
2 parents 78202f6 + ce0248f commit 9567f35

File tree

5 files changed

+260
-4
lines changed

5 files changed

+260
-4
lines changed

‎en_US/Installation_and_Deployment/DSS&Linkis_one-click_deployment_document_stand-alone_version.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -395,4 +395,4 @@ HIVE_PASSWORD=xxx
395395
| Qualitis |Qualitis0.9.2 | [Qualitis deploy](https://github.com/WeBankFinTech/Qualitis/blob/master/docs/en_US/ch1/QuickDeploy.md) | [Qualitis AppConn install](https://github.com/WeBankFinTech/Qualitis/blob/master/docs/zh_CN/ch1/%E6%8E%A5%E5%85%A5%E5%B7%A5%E4%BD%9C%E6%B5%81%E6%8C%87%E5%8D%97.md) |
396396
| Prophecis | Prophecis0.3.2 | [Prophecis deploy](https://github.com/WeBankFinTech/Prophecis/blob/master/docs/zh_CN/QuickStartGuide.md) | [Prophecis AppConn install](https://github.com/WeBankFinTech/Prophecis/blob/master/docs/zh_CN/Deployment_Documents/Prophecis%20Appconn%E5%AE%89%E8%A3%85%E6%96%87%E6%A1%A3.md) |
397397
| Streamis | Streamis0.2.0 | [Streamis deploy](https://github.com/WeBankFinTech/Streamis/blob/main/docs/zh_CN/0.2.0/Streamis%E5%AE%89%E8%A3%85%E6%96%87%E6%A1%A3.md) | [Streamis AppConn install](https://github.com/WeBankFinTech/Streamis/blob/main/docs/en_US/0.2.0/development/StreamisAppConnInstallationDocument.md) |
398-
| DolphinScheduler | DolphinScheduler1.3.x | [DolphinScheduler deploy](https://dolphinscheduler.apache.org/zh-cn/docs/1.3.8/user_doc/standalone-deployment.html) | [DolphinScheduler AppConn install](DolphinScheduler_Plugin_Installation_Documentation.md) |
398+
| DolphinScheduler | DolphinScheduler1.3.x | [DolphinScheduler deploy](https://dolphinscheduler.apache.org/en-us/docs/1.3.9/standalone-deployment) | [DolphinScheduler AppConn install](DolphinScheduler_Plugin_Installation_Documentation.md) |

‎en_US/Installation_and_Deployment/DolphinScheduler_Plugin_Installation_Documentation.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ Before you deploy `DolphinSchedulerAppConn`, you must first start the `DolphinSc
66

77
**Please note: Currently `DolphinSchedulerAppConn` only supports DolphinScheduler 1.3.X. **
88

9-
For the installation and deployment of `DolphinScheduler`, please refer to：[`DolphinScheduler` Install and deploy documentation](https://dolphinscheduler.apache.org/en-us/docs/1.3.8/user_doc/standalone-deployment.html)
9+
For the installation and deployment of `DolphinScheduler`, please refer to：[`DolphinScheduler` Install and deploy documentation](https://dolphinscheduler.apache.org/en-us/docs/1.3.9/standalone-deployment)
1010

1111
## 2. Download and compile
1212

‎zh_CN/安装部署/DSS&Linkis一键部署文档单机版.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -410,4 +410,4 @@ HIVE_PASSWORD=xxx
410410
| Qualitis |Qualitis0.9.2 | [Qualitis部署](https://github.com/WeBankFinTech/Qualitis/blob/master/docs/zh_CN/ch1/%E5%BF%AB%E9%80%9F%E6%90%AD%E5%BB%BA%E6%89%8B%E5%86%8C%E2%80%94%E2%80%94%E5%8D%95%E6%9C%BA%E7%89%88.md) |[Qualitis AppConn安装](https://github.com/WeBankFinTech/Qualitis/blob/master/docs/zh_CN/ch1/%E6%8E%A5%E5%85%A5%E5%B7%A5%E4%BD%9C%E6%B5%81%E6%8C%87%E5%8D%97.md) |
411411
| Prophecis | Prophecis0.3.2 | [Prophecis部署](https://github.com/WeBankFinTech/Prophecis/blob/master/docs/zh_CN/QuickStartGuide.md) | [Prophecis AppConn安装](https://github.com/WeBankFinTech/Prophecis/blob/master/docs/zh_CN/Deployment_Documents/Prophecis%20Appconn%E5%AE%89%E8%A3%85%E6%96%87%E6%A1%A3.md) |
412412
| Streamis | Streamis0.2.0 | [Streamis部署](https://github.com/WeBankFinTech/Streamis/blob/main/docs/zh_CN/0.2.0/Streamis%E5%AE%89%E8%A3%85%E6%96%87%E6%A1%A3.md) | [Streamis AppConn安装](https://github.com/WeBankFinTech/Streamis/blob/main/docs/zh_CN/0.2.0/development/StreamisAppConn%E5%AE%89%E8%A3%85%E6%96%87%E6%A1%A3.md) |
413-
| DolphinScheduler | DolphinScheduler1.3.x | [DolphinScheduler部署](https://dolphinscheduler.apache.org/zh-cn/docs/1.3.8/user_doc/standalone-deployment.html) | [DolphinScheduler AppConn安装](DolphinScheduler插件安装文档.md) |
413+
| DolphinScheduler | DolphinScheduler1.3.x | [DolphinScheduler部署](https://dolphinscheduler.apache.org/zh-cn/docs/1.3.9/standalone-deployment) | [DolphinScheduler AppConn安装](DolphinScheduler插件安装文档.md) |

‎zh_CN/安装部署/DSS部署文档单机版.md‎

Lines changed: 256 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,256 @@
1+
# DataSphere Studio 单机一键部署文档
2+
3+
### 零、部署前注意事项（重要！！！）
4+
- 确保安装的系统为CentOS为6或者7
5+
6+
- 服务器存在多网卡问题。首先通过命令`ifconfig`命令查看服务器激活状态的网卡，若激活状态的网卡数大于1，那么用户就需要通过命令`ifconfig [NIC_NAME] down`(`[NIC_NAME]`为网卡名称)来关闭多余的网卡，以确保激活的网卡数只有1个
7+
8+
- 网卡多IP问题。在确保服务器只存在一个网卡是激活状态的情况下，通过命令`echo $(hostname -I)`查看网卡对应的IP数，若大于1，那么就需要去掉网卡中指定的IP，采用动态获取IP的方式，具体命令如下：
9+
```shell
10+
ip addr flush dev [NIC_NAME]
11+
ifdown [NIC_NAME]
12+
ifup [NIC_NAME]
13+
```
14+
15+
- hostname配置。在安装前用户需要配置hostname到ip的映射
16+
17+
- 若未进行上述设置，安装脚本会默认获取第一个网卡ip
18+
19+
### 一、基础软件安装
20+
21+
- 需要的命令工具（在正式安装前，脚本会自动检测这些命令是否可用，如果不存在会尝试自动安装，安装失败则需用户手动安装以下基础shell命令工具）：
22+
23+
*telnet; tar; sed; dos2unix; mysql; yum; java; unzip; zip; expect*
24+
25+
- 需要安装的软件：
26+
27+
MySQL (5.5+); JDK (1.8.0_141以上); Nginx
28+
29+
- Tips:
30+
*请确保已安装Links*
31+
32+
### 二、创建用户
33+
34+
1. 假设**部署用户是hadoop账号**（可以不是hadoop用户，但是推荐使用Hadoop的超级用户进行部署，这里只是一个示例）
35+
36+
37+
2. 在所有需要部署的机器上创建部署用户，用于安装 ，如下命令创建部署用户hadoop
38+
39+
```shell
40+
sudo useradd hadoop
41+
```
42+
43+
3. 改部署用户权限
44+
45+
编辑/etc/sudoers文件：
46+
47+
```shell
48+
vi /etc/sudoers
49+
```
50+
51+
在/etc/sudoers文件中添加下面内容：
52+
53+
```
54+
hadoop ALL=(ALL) NOPASSWD: NOPASSWD: ALL
55+
```
56+
57+
### 三、准备安装包
58+
59+
- 用户可以自行编译或者去 release 页面下载安装包：[DSS Release-1.1.1](https://github.com/WeBankFinTech/DataSphereStudio/releases/tag/1.1.1)
60+
61+
- DSS 一键安装部署包的层级目录结构如下：
62+
63+
```text
64+
├── dss_install # 一键部署主目录
65+
├── bin # 用于一键安装，以及一键启动 DSS
66+
├── conf # 一键部署的参数配置目录
67+
├── wedatasphere-dss-x.x.x-dist.tar.gz # DSS后端安装包
68+
├── wedatasphere-dss-web-x.x.x-dist.zip # DSS前端安装包
69+
```
70+
71+
- 如果用户选择采用下载安装包直接部署的形式，可直接跳转到 [修改配置](#1)
72+
73+
74+
- 如果用户选择自行编译DSS，请确保编译的是DSS `master` 分支的最新代码，编译方式可以参考:
75+
[DSS后端编译文档](../开发文档/DSS编译文档.md)
76+
[DSS前端编译文档](../开发文档/前端编译文档.md)
77+
78+
79+
1. 针对后端安装包可直接将上面的 DSS 后端安装包替换成编译后安装包即可。
80+
81+
2. 针对前端安装包，则需要特别注意，整个前端安装包目录结构如下：
82+
```
83+
├── wedatasphere-dss-web-x.x.x-dist # DSS前端安装包
84+
├── config.sh # 参数配置脚本
85+
├── install.sh # 前端部署脚本
86+
├── dist # DSS前端包
87+
```
88+
89+
3. DSS前端包可直接替换成用户编译后的相关安装包。
90+
91+
4. 用户在打包wedatasphere-dss-web-x.x.x-dist.zip的时候需要特别注意，不要在父级目录对其直接压缩，应全选目录下面的所有文件然后压缩。
92+
93+
94+
### <a id = "1">四、修改配置</a>
95+
96+
- 用户需要对 `xx/dss_install/conf` 目录下的 `config.sh` 和 `db.sh` 进行修改。
97+
98+
99+
- 打开 `config.sh`，按需修改相关配置参数，参数说明如下：
100+
101+
```properties
102+
### deploy user
103+
deployUser=hadoop
104+
105+
### DSS Web
106+
DSS_NGINX_IP=127.0.0.1
107+
DSS_WEB_PORT=8085
108+
109+
### DSS VERSION
110+
DSS_VERSION=1.1.1
111+
112+
###
113+
LINKIS_HOME=
114+
### EUREKA install information
115+
### You can access it in your browser at the address below:http://${EUREKA_INSTALL_IP}:${EUREKA_PORT}
116+
### Microservices Service Registration Discovery Center
117+
LINKIS_EUREKA_INSTALL_IP=127.0.0.1
118+
LINKIS_EUREKA_PORT=20303
119+
#LINKIS_EUREKA_PREFER_IP=true
120+
121+
### Gateway install information
122+
LINKIS_GATEWAY_INSTALL_IP=127.0.0.1
123+
LINKIS_GATEWAY_PORT=9001
124+
125+
126+
127+
################### The install Configuration of all DataSphereStudio's Micro-Services #####################
128+
#
129+
# NOTICE:
130+
# 1. If you just wanna try, the following micro-service configuration can be set without any settings.
131+
# These services will be installed by default on this machine.
132+
# 2. In order to get the most complete enterprise-level features, we strongly recommend that you install
133+
# the following microservice parameters
134+
#
135+
136+
### DSS_SERVER
137+
### This service is used to provide dss-server capability.
138+
139+
### project-server
140+
#DSS_FRAMEWORK_PROJECT_SERVER_INSTALL_IP=127.0.0.1
141+
#DSS_FRAMEWORK_PROJECT_SERVER_PORT=9002
142+
### orchestrator-server
143+
#DSS_FRAMEWORK_ORCHESTRATOR_SERVER_INSTALL_IP=127.0.0.1
144+
#DSS_FRAMEWORK_ORCHESTRATOR_SERVER_PORT=9003
145+
### apiservice-server
146+
#DSS_APISERVICE_SERVER_INSTALL_IP=127.0.0.1
147+
#DSS_APISERVICE_SERVER_PORT=9004
148+
### dss-workflow-server
149+
#DSS_WORKFLOW_SERVER_INSTALL_IP=127.0.0.1
150+
#DSS_WORKFLOW_SERVER_PORT=9005
151+
### dss-flow-execution-server
152+
#DSS_FLOW_EXECUTION_SERVER_INSTALL_IP=127.0.0.1
153+
#DSS_FLOW_EXECUTION_SERVER_PORT=9006
154+
###dss-scriptis-server
155+
#DSS_SCRIPTIS_SERVER_INSTALL_IP=127.0.0.1
156+
#DSS_SCRIPTIS_SERVER_PORT=9008
157+
158+
###dss-data-api-server
159+
#DSS_DATA_API_SERVER_INSTALL_IP=127.0.0.1
160+
#DSS_DATA_API_SERVER_PORT=9208
161+
###dss-data-governance-server
162+
#DSS_DATA_GOVERNANCE_SERVER_INSTALL_IP=127.0.0.1
163+
#DSS_DATA_GOVERNANCE_SERVER_PORT=9209
164+
###dss-guide-server
165+
#DSS_GUIDE_SERVER_INSTALL_IP=127.0.0.1
166+
#DSS_GUIDE_SERVER_PORT=9210
167+
########## DSS微服务配置完毕#####
168+
169+
############## ############## other default configuration 其他默认配置信息 ############## ##############
170+
171+
## java application default jvm memory
172+
export SERVER_HEAP_SIZE="512M"
173+
174+
175+
##sendemail配置，只影响DSS工作流中发邮件功能
176+
EMAIL_HOST=smtp.163.com
177+
EMAIL_PORT=25
178+
[email protected]
179+
EMAIL_PASSWORD=xxxxx
180+
EMAIL_PROTOCOL=smtp
181+
182+
### Save the file path exported by the orchestrator service
183+
ORCHESTRATOR_FILE_PATH=/appcom/tmp/dss
184+
### Save DSS flow execution service log path
185+
EXECUTION_LOG_PATH=/appcom/tmp/dss
186+
```
187+
188+
### 五、安装和使用
189+
190+
1. #### 停止机器上所有DSS服务
191+
192+
- 若从未安装过DSS服务，忽略此步骤
193+
194+
2. #### 将当前目录切换到bin目录
195+
```shell
196+
cd xx/dss_install/bin
197+
```
198+
3. #### 执行安装脚本
199+
```shell
200+
sh install.sh
201+
```
202+
- 该安装脚本会检查各项集成环境命令，如果没有请按照提示进行安装，以下命令为必须项：
203+
204+
*yum; java; mysql; unzip; expect; telnet; tar; sed; dos2unix; nginx*
205+
206+
- 安装时，脚本会询问您是否需要初始化数据库并导入元数据，**第一次安装必须选是**
207+
208+
- 通过查看控制台打印的日志信息查看是否安装成功，如果有错误信息，可以查看具体报错原因
209+
- *除非用户想重新安装整个应用，否则该命令执行一次即可*
210+
211+
4. #### 启动服务
212+
- 在xx/dss_install/bin目录下执行启动服务脚本
213+
214+
```shell
215+
sh start-all.sh
216+
```
217+
218+
- 如果启动产生了错误信息，可以查看具体报错原因。启动后，各项微服务都会进行**通信检测**，如果有异常则可以帮助用户定位异常日志和原因
219+
220+
5. #### 安装默认Appconn
221+
222+
```shell
223+
# 切换目录到dss，正常情况下dss目录就在xx/dss_install目录下，
224+
cd xx/dss_install/dss/bin
225+
226+
# 执行启动默认Appconn脚本
227+
sh install-default-appconn.sh
228+
```
229+
230+
- *该命令执行一次即可，除非用户想重新安装整个应用*
231+
232+
6. #### 查看验证是否成功
233+
234+
- 用户可以在Linkis的Eureka界面查看 DSS 后台各微服务的启动情况，默认情况下DSS有7个微服务
235+
236+
- 用户可以使用**谷歌浏览器**访问以下前端地址：`http://DSS_NGINX_IP:DSS_WEB_PORT` **启动日志会打印此访问地址（在xx/dss_install/conf/config.sh中也配置了此地址）**。登陆时默认管理员的用户名和密码均为部署用户为hadoop（用户若做了修改，可以查看xx/linkis/conf/linkis-mg-gateway.properties 文件中的 wds.linkis.admin.password 参数)
237+
238+
7. #### 停止服务
239+
```shell
240+
sh stop-all.sh
241+
```
242+
- 若用户需要停止所有服务可执行该命令`sh stop-all.sh`，重新启动所有服务就执行`sh start-all.sh`，这两条命令均在xx/dss_install/bin目录下执行
243+
244+
### 六、补充说明
245+
- DSS默认未安装调度系统，用户可以选择安装 Schedulis 或者 DolphinScheduler，具体安装方式见下面表格
246+
- DSS默认仅安装DateChecker, EventSender, EventReceiver AppConn，用户可参考文档安装其他AppConn，如Visualis, Exchangis, Qualitis, Prophecis, Streamis。调度系统可使用Schedulis或DolphinScheduler
247+
248+
| 组件名 | 组件版本要求 | 组件部署链接 | AppConn部署链接 |
249+
|-----------------|----------------|----------------------------------------|-------------------|
250+
| Schedulis | Schedulis0.7.0 | [Schedulis部署](https://github.com/WeBankFinTech/Schedulis/blob/master/docs/schedulis_deploy_cn.md) | [Schedulis AppConn安装](SchedulisAppConn插件安装文档.md)|
251+
| Visualis | Visualis1.0.0 | [Visualis部署](https://github.com/WeBankFinTech/Visualis/blob/master/visualis_docs/zh_CN/Visualis_deploy_doc_cn.md) |[Visualis AppConn安装](https://github.com/WeBankFinTech/Visualis/blob/master/visualis_docs/zh_CN/Visualis_appconn_install_cn.md)|
252+
| Exchangis | Exchangis1.0.0 | [Exchangis部署](https://github.com/WeBankFinTech/Exchangis/blob/master/docs/zh_CN/ch1/exchangis_deploy_cn.md) | [Exchangis AppConn安装](https://github.com/WeBankFinTech/Exchangis/blob/master/docs/zh_CN/ch1/exchangis_appconn_deploy_cn.md) |
253+
| Qualitis |Qualitis0.9.2 | [Qualitis部署](https://github.com/WeBankFinTech/Qualitis/blob/master/docs/zh_CN/ch1/%E5%BF%AB%E9%80%9F%E6%90%AD%E5%BB%BA%E6%89%8B%E5%86%8C%E2%80%94%E2%80%94%E5%8D%95%E6%9C%BA%E7%89%88.md) |[Qualitis AppConn安装](https://github.com/WeBankFinTech/Qualitis/blob/master/docs/zh_CN/ch1/%E6%8E%A5%E5%85%A5%E5%B7%A5%E4%BD%9C%E6%B5%81%E6%8C%87%E5%8D%97.md) |
254+
| Prophecis | Prophecis0.3.2 | [Prophecis部署](https://github.com/WeBankFinTech/Prophecis/blob/master/docs/zh_CN/QuickStartGuide.md) | [Prophecis AppConn安装](https://github.com/WeBankFinTech/Prophecis/blob/master/docs/zh_CN/Deployment_Documents/Prophecis%20Appconn%E5%AE%89%E8%A3%85%E6%96%87%E6%A1%A3.md) |
255+
| Streamis | Streamis0.2.0 | [Streamis部署](https://github.com/WeBankFinTech/Streamis/blob/main/docs/zh_CN/0.2.0/Streamis%E5%AE%89%E8%A3%85%E6%96%87%E6%A1%A3.md) | [Streamis AppConn安装](https://github.com/WeBankFinTech/Streamis/blob/main/docs/zh_CN/0.2.0/development/StreamisAppConn%E5%AE%89%E8%A3%85%E6%96%87%E6%A1%A3.md) |
256+
| DolphinScheduler | DolphinScheduler1.3.x | [DolphinScheduler部署](https://dolphinscheduler.apache.org/zh-cn/docs/1.3.8/user_doc/standalone-deployment.html) | [DolphinScheduler AppConn安装](DolphinScheduler插件安装文档.md) |

‎zh_CN/安装部署/DolphinScheduler插件安装文档.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66

77
**请注意：目前 `DolphinSchedulerAppConn` 只支持适配 DolphinScheduler1.3.X。在代码中默认配置的是1.3.9，用户若安装的是其他1.3.x版本就需要修改 dolphinscheduler module下的pom.xml中的dolphinscheuduler.version到相应版本**
88

9-
关于 `DolphinScheduler` 的安装部署，请参考：`DolphinScheduler` [安装部署文档](https://dolphinscheduler.apache.org/zh-cn/docs/1.3.8/user_doc/standalone-deployment.html)
9+
关于 `DolphinScheduler` 的安装部署，请参考：`DolphinScheduler` [安装部署文档](https://dolphinscheduler.apache.org/zh-cn/docs/1.3.9/standalone-deployment)
1010

1111
## 2. 下载和编译
1212

0 commit comments

Comments
 (0)