Skip to content

Commit 7f75261

Browse files
author
v_xiangbiaowu
committed
StreamisAppConnDesignDocument
1 parent 9f600c7 commit 7f75261

File tree

4 files changed

+418
-0
lines changed

4 files changed

+418
-0
lines changed

‎docs/en_US/0.1.0/StreamisAppConnDesignDocument.md‎

Lines changed: 209 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,209 @@
1+
# Streamis access AppConn
2+
3+
## Overall flow chart
4+
![Streamis access DSS](../../images/zh_CN/streamis_appconn.png)
5+
6+
## DSS project APPCONN plug-in streamis-appconn
7+
8+
### The configuration table
9+
Configure the following three tables:dss_appconn、dss_workspace_menu_appconn、dss_appconn_instance，appconn_name for realTimeJobCenter is appconn accessed by the graphical interface，appconn_name for streamis is appconn accessed by the API，The StreamisAppConn object is instantiated based on the configuration information in the table when DSS is started。
10+
```roomsql
11+
select @old_dss_appconn_id:=id from `dss_appconn` where `appconn_name` = 'streamis';
12+
13+
delete from `dss_workspace_menu_appconn` WHERE `appconn_id` = @old_dss_appconn_id;
14+
delete from `dss_appconn_instance` where `appconn_id` = @old_dss_appconn_id;
15+
delete from `dss_appconn` where `appconn_name`='streamis';
16+
17+
select @old_jobcenter_dss_appconn_id:=id from `dss_appconn` where `appconn_name` = 'realTimeJobCenter';
18+
19+
delete from `dss_workspace_menu_appconn` WHERE `appconn_id` = @old_jobcenter_dss_appconn_id;
20+
delete from `dss_appconn_instance` where `appconn_id` = @old_jobcenter_dss_appconn_id;
21+
delete from `dss_appconn` where `appconn_name`='realTimeJobCenter';
22+
23+
INSERT INTO dss_appconn
24+
(appconn_name, is_user_need_init, `level`, if_iframe, is_external, reference, class_name, appconn_class_path, resource)
25+
VALUES('streamis', 0, 1, 1, 1, NULL, 'com.webank.wedatasphere.streamis.dss.appconn.StreamisAppConn', NULL, NULL);
26+
INSERT INTO dss_appconn
27+
(appconn_name, is_user_need_init, `level`, if_iframe, is_external, reference, class_name, appconn_class_path, resource)
28+
VALUES('realTimeJobCenter', 0, 1, 1, 1, 'sso', '', NULL, NULL);
29+
30+
select @dss_appconn_id:=id from `dss_appconn` where `appconn_name` = 'streamis';
31+
select @jobcenter_dss_appconn_id:=id from `dss_appconn` where `appconn_name` = 'realTimeJobCenter';
32+
33+
INSERT INTO dss_workspace_menu_appconn
34+
(appconn_id, menu_id, title_en, title_cn, desc_en, desc_cn, labels_en, labels_cn, is_active, access_button_en, access_button_cn, manual_button_en, manual_button_cn, manual_button_url, icon, `order`, create_by, create_time, last_update_time, last_update_user, image)
35+
VALUES(@jobcenter_dss_appconn_id, 1, 'StreamSQL development', 'StreamSQL开发', 'Real-time application development is a streaming solution jointly built by WeDataSphere, Boss big data team and China Telecom ctcloud Big data team.', '实时应用开发是微众银行微数域(WeDataSphere)、Boss直聘大数据团队 和 中国电信天翼云大数据团队 社区联合共建的流式解决方案，以 Linkis 做为内核，基于 Flink Engine 构建的批流统一的 Flink SQL，助力实时化转型。',
36+
'streaming, realtime', '流式,实时', 0, 'under union construction', '联合共建中', 'related information', '相关资讯', 'http://127.0.0.1:8088/wiki/scriptis/manual/workspace_cn.html', 'shujukaifa-logo', NULL, NULL, NULL, NULL, NULL, 'shujukaifa-icon');
37+
38+
INSERT INTO dss_appconn_instance
39+
(appconn_id, label, url, enhance_json, homepage_uri)
40+
VALUES(@dss_appconn_id, 'DEV', 'http://Streamis_INSTALL_IP:Streamis_INSTALL_PORT/', '', 'http://Streamis_INSTALL_IP:Streamis_INSTALL_PORT/#/realTimeJobCenter');
41+
42+
INSERT INTO dss_appconn_instance
43+
(appconn_id, label, url, enhance_json, homepage_uri)
44+
VALUES(@jobcenter_dss_appconn_id, 'DEV', 'http://Streamis_INSTALL_IP:Streamis_INSTALL_PORT/#/realTimeJobCenter', NULL, NULL);
45+
```
46+
47+
### Concrete implementation description
48+
StreamisAppConn extends AbstractOnlySSOAppConn implements SecondlyAppConn，Override method to create StreamisStructureIntegrationStandard.StreamisAppConn currently only implements section-free login and organizational structure specification capabilities。
49+
50+
To create a StreamisProjectService by rewriting the methods in StreamisStructureIntegrationStandard, four methods need to be rewritten internally, and four operation classes will be created: StreamisProjectSearchOperation, StreamisProjectCreationOperation, StreamisProjectUpdateOperation, and StreamisPrejectDeleteOperation. The operation class calls the streamis application through HTTP to query, create, modify and delete project respectively, and synchronize the DSS project information to Streamis.
51+
52+
- When DSS creates a project, it will first call the query operation to query whether the same project name already exists in streamis. If it does, a prompt will pop up. If it does not exist, it will continue to call the create operation to create a new project in streamis;
53+
- When DSS modifies a project, it will call the modify operation to update the project information in streamis;
54+
- When DSS deletes an item, it will call the delete operation to delete the item information in streamis and change the deletion mark.
55+
56+
## API
57+
1 API name: query project
58+
- API path：GET/streamis/project/searchProject
59+
- Request parameters
60+
61+
|Parameter name |Whether it is necessary |Example |remarks |
62+
|-------------|---------|-------|--------|
63+
|projectName |yes | | |
64+
65+
- Return data
66+
67+
|name |type |Whether it is necessary |Default |remarks |
68+
|-------------|--------|---------|---------|--------|
69+
|method |string |no |
70+
|status |number |yes |
71+
|message |string |no |
72+
|data |object |yes |
73+
|- projectId |number |yes |
74+
75+
2 API name：create project
76+
- API path：GET/streamis/project/createProject
77+
- Request parameters
78+
79+
|Parameter name |Whether it is necessary |Example |remarks |
80+
|-------------|---------|-------|--------|
81+
|projectName |yes | | |
82+
|workspaceId |no | | |
83+
|releaseUsers |no | | |
84+
|editUsers |no | | |
85+
|accessUsers |no | | |
86+
87+
- Return data
88+
89+
|name |type |Whether it is necessary |Default |remarks |
90+
|----------|--------|---------|---------|--------|
91+
|method |string |no |
92+
|status |number |yes |
93+
|message |string |no |
94+
|data |object |yes |
95+
|- projectId |number |yes |
96+
|- projectName |string |no |
97+
98+
3 API name：update project
99+
- API path：GET/streamis/project/updateProject
100+
- Request parameters
101+
102+
|Parameter name |Whether it is necessary |Example |remarks |
103+
|-------------|---------|-------|--------|
104+
|projectId |yes | | |
105+
|projectName |yes | | |
106+
|workspaceId |no | | |
107+
|releaseUsers |no | | |
108+
|editUsers |no | | |
109+
|accessUsers |no | | |
110+
111+
- Return data
112+
113+
|name |type |Whether it is necessary |Default |remarks |
114+
|----------|--------|---------|---------|--------|
115+
|method |string |no |
116+
|status |number |yes |
117+
|message |string |no |
118+
|data |object |no |
119+
120+
4 API name：delete project
121+
- API path：GET/streamis/project/deleteProject
122+
- Request parameters
123+
124+
|Parameter name |Whether it is necessary |Example |remarks |
125+
|-------------|---------|-------|--------|
126+
|projectId |yes | | |
127+
|projectName |no | | |
128+
129+
- Return data
130+
131+
|name |type |Whether it is necessary |Default |remarks |
132+
|----------|--------|---------|---------|--------|
133+
|method |string |no |
134+
|status |number |yes |
135+
|message |string |no |
136+
|data |object |no |
137+
138+
## Streamis project streamis-project-server
139+
140+
### Related table operation
141+
There are 2 tables involved in streamis, including linkis_ stream_ Project and linkis_ stream_ project_ privilege.
142+
```roomsql
143+
--table already exists
144+
CREATE TABLE `linkis_stream_project` (
145+
`id` bigint(20) NOT NULL AUTO_INCREMENT,
146+
`workspace_id` bigint(20) DEFAULT NULL,
147+
`name` varchar(100) DEFAULT NULL,
148+
`create_by` varchar(50) DEFAULT NULL,
149+
PRIMARY KEY (`id`) USING BTREE
150+
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 ROW_FORMAT=COMPACT COMMENT='project table';
151+
152+
--newly added table
153+
CREATE TABLE `linkis_stream_project_privilege` (
154+
`id` bigint(20) NOT NULL AUTO_INCREMENT,
155+
`project_id` bigint(20) NOT NULL,
156+
`user_name` varchar(100) NOT NULL,
157+
`privilege` tinyint(1) DEFAULT '0' NOT NULL COMMENT '1:RELEASE ，2:EDIT ，3:ACCESS',
158+
PRIMARY KEY (`id`) USING BTREE
159+
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COMMENT='project privilege table';
160+
161+
--newly added field
162+
ALTER TABLE `linkis_stream_project` ADD create_time datetime DEFAULT NULL;
163+
ALTER TABLE `linkis_stream_project` ADD last_update_by varchar(50) DEFAULT NULL;
164+
ALTER TABLE `linkis_stream_project` ADD last_update_time datetime DEFAULT NULL;
165+
ALTER TABLE `linkis_stream_project` ADD is_deleted tinyint unsigned DEFAULT 0;
166+
```
167+
168+
### Concrete implementation description
169+
- The query operation will obtain the request parameter projectName in the table links_ stream_project Query the corresponding ID. If the query operation is successful, the returned status is 0, and the projectid is the queried ID. if the query result is empty, the projectid is null;
170+
- The creation operation will insert the project information (projectName、workspaceId) in the request parameters into the project table linkis_stream_project and auto increment the generated ID, associate the user in the permission information data (releaseUsers、editUsers、accessUsers) in the request parameters with the generated ID of the project table, and insert the table linkis_stream_project_privilege, the status value returned from the successful creation operation is 0, and the project table generation ID will be returned as the value of projectId;
171+
- The modification operation will update the request information data to the table linkis_stream_project and linkis_stream_project_privilege, the status value returned successfully is 0;
172+
- In the delete operation, the is_deleted field of the table linkis_stream_project will be marked as 1 according to the projectId. The relevant data in the table linkis_stream_project_privilege will be deleted, the status value returned successfully is 0.
173+
174+
# Authentication
175+
176+
## Authentication flow chart
177+
![Streamis project authentication operation](../../images/zh_CN/streamis_project_privilege.png)
178+
179+
### edit privilege API：
180+
181+
|RequestMethod |API path |name |
182+
|------|----------------------------------------------------------|-----------------|
183+
|POST |/streamis/streamProjectManager/project/files/upload |Project resource file - Import |
184+
|GET |/streamis/streamProjectManager/project/files/delete |Delete all versions of the file under the project |
185+
|GET |/streamis/streamProjectManager/project/files/version/delete |Delete version file |
186+
|GET |/streamis/streamProjectManager/project/files/download |Task details - Download |
187+
|POST |/streamis/streamJobManager/job/upload |Upload file |
188+
|POST |/streamis/streamJobManager/job/execute |start-up |
189+
|GET |/streamis/streamJobManager/job/stop |stop |
190+
|PUT |/streamis/streamJobManager/job//snapshot/{jobId:\w+} |Snapshot generation |
191+
|GET |/streamis/streamJobManager/config/json/{jobId:\w+} |Configuration - save |
192+
|POST |/streamis/streamJobManager/job/bulk/execution |Batch start |
193+
|POST |/streamis/streamJobManager/job/bulk/pause |Batch stop |
194+
195+
196+
### access privilege API：
197+
198+
|RequestMethod |API path |name |
199+
|------|----------------------------------------------------------|-------------|
200+
|GET |streamis/streamJobManager/job/list |Query the jobs that the current user can view |
201+
|GET |/streamis/streamProjectManager/project/files/list |prokect resource document |
202+
|GET |/streamis/streamProjectManager/project/files/version/list |Obtain all versions of the file under the project |
203+
|GET |/streamis/streamJobManager/job/version |Query job version |
204+
|GET |/streamis/streamJobManager/job/execute/history |Job execution history |
205+
|GET |/streamis/streamJobManager/job/progress |Get the latest task status of the current version of the job |
206+
|GET |/streamis/streamJobManager/job/jobContent |Task details |
207+
|GET |/streamis/streamJobManager/job/logs |Get log |
208+
|POST |/streamis/streamJobManager/config/json/{jobId:\w+} |Get task configuration |
209+
|GET |/streamis/streamJobManager/config/view |Query the current job configuration information |

‎docs/images/zh_CN/streamis_appconn.png‎

76.7 KB
Loading

‎docs/images/zh_CN/streamis_project_privilege.png‎

108 KB
Loading

0 commit comments

Comments
 (0)