Skip to content

Commit a990b19

Browse files
committed
Merge branch 'v2' into sp-159-upgrading-sailor
# Conflicts: # .gitignore # README.md
2 parents 897191a + 71c7dc1 commit a990b19

31 files changed

+2553
-829
lines changed

.gitignore.save

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
out/
2+
.idea

README.md

Lines changed: 81 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -14,25 +14,48 @@ Following actions are inside:
1414

1515
``LOOKUP BY PRIMARY KEY`` - this action will execute select query from specified table, as criteria can be used only [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY"). The action returns only one result (a primary key is unique).
1616

17+
``UPSERT BY PRIMARY KEY`` - this action will execute select command from specified table, as search criteria can be used only [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY"), and execute insert command by PRIMARY KEY with specified field, if result does not found, else - action will execute update command by PRIMARY KEY with specified field. The action returns only one result row (a primary key is unique).
18+
1719
``DELETE BY PRIMARY KEY`` - this action will execute delete query from specified table, as criteria can be used only [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY"). The action returns an integer value that indicates the number of rows affected, the returned value can be 0 or 1 (a primary key is unique).
1820
### How works
1921

2022
### Requirements
2123
Before you can deploy any code into elastic.io **you must be a registered elastic.io platform user**. Please see our home page at [http://www.elastic.io](http://www.elastic.io) to learn how.
2224
#### Environment variables
23-
For unit-testing
25+
For unit-testing is needed to specify following environment variables:
26+
1. Connection to MSSQL:
27+
- ``CONN_USER_MSSQL`` - user login
28+
- ``CONN_PASSWORD_MSSQL`` - user password
29+
- ``CONN_DBNAME_MSSQL`` - DataBase name
30+
- ``CONN_HOST_MSSQL`` - DataBase host
31+
- ``CONN_PORT_MSSQL`` - DataBase port
32+
2. Connection to MySQL:
33+
- ``CONN_USER_MYSQL`` - user login
34+
- ``CONN_PASSWORD_MYSQL`` - user password
35+
- ``CONN_DBNAME_MYSQL`` - DataBase name
36+
- ``CONN_HOST_MYSQL`` - DataBase host
37+
- ``CONN_PORT_MYSQL`` - DataBase port
38+
3. Connection to Oracle:
39+
- ``CONN_USER_ORACLE`` - user login
40+
- ``CONN_PASSWORD_ORACLE`` - user password
41+
- ``CONN_DBNAME_ORACLE`` - DataBase name
42+
- ``CONN_HOST_ORACLE`` - DataBase host
43+
- ``CONN_PORT_ORACLE`` - DataBase port
44+
4. Connection to PostgreSQL:
45+
- ``CONN_USER_POSTGRESQL`` - user login
46+
- ``CONN_PASSWORD_POSTGRESQL`` - user password
47+
- ``CONN_DBNAME_POSTGRESQL`` - DataBase name
48+
- ``CONN_HOST_POSTGRESQL`` - DataBase host
49+
- ``CONN_PORT_POSTGRESQL`` - DataBase port
2450
#### Others
2551
## Credentials
2652
You may use following properties to configure a connection:
2753
![image](https://user-images.githubusercontent.com/40201204/43577550-ce99efe6-9654-11e8-87ed-f3e0839d618a.png)
2854
You can add the authorisation methods during the integration flow design or by going to your Settings > Security credentials > REST client and adding there.
2955
### DB Engine
56+
You are able to choose one of existing database types:
3057
![image](https://user-images.githubusercontent.com/40201204/43577772-6f85bdea-9655-11e8-96e1-368493a36c9d.png)
31-
You are able to choose one of existing database types
32-
- ``MySQL`` - compatible with MySQL Server 5.5, 5.6, 5.7 and 8.0.
33-
- ``PostgreSQL`` - compatible with PostgreSQL 8.2 and higher
34-
- ``Oracle`` - compatible with Oracle Database 8.1.7 - 12.1.0.2
35-
- ``MSSQL`` - compatible with Microsoft SQL Server 2008 R2 and higher
58+
3659
### Connection URI
3760
In the Connection URI field please provide hostname of the server, e.g. ``acme.com``
3861
### Connection port
@@ -51,7 +74,7 @@ In the Password field please provide a password of the user that has permissions
5174
Validation will start right after click on a Save button. You will be able to continue working with component after validation if all provided credentials will be valid.
5275
## Triggers
5376
### SELECT
54-
You are able to provide SELECT queryOld with last execution timestamp as WHERE clause criteria.
77+
You are able to provide SELECT query with last execution timestamp as WHERE clause criteria.
5578
![image](https://user-images.githubusercontent.com/40201204/43591075-2a032dcc-967b-11e8-968d-851355c2646e.png)
5679
Before executing the the statement %%EIO_LAST_POLL%% will be replaced with ISO Date of the last execution or max value of the last pooled datetime, for example ``2018-08-01T00:00:00.000``.
5780
During the first execution, date will be equal to ["start" of Unix Time](https://en.wikipedia.org/wiki/Unix_time) - ``1970-01-01 00:00:00.000``.
@@ -81,8 +104,8 @@ The format of ``Start Polling From (optional)`` field should be like ``yyyy-mm-d
81104
## Actions
82105
### SELECT
83106
![image](https://user-images.githubusercontent.com/40201204/43592439-39ec5738-967e-11e8-8632-3655b08982d3.png)
84-
The action will execute an [SQL](https://en.wikipedia.org/wiki/SQL "SQL") queryOld that can return multiple results, it has limitations on the queryOld and suited only for SELECT type of queries.
85-
In SQL queryOld you can use clause variables with specific data types.
107+
The action will execute an [SQL](https://en.wikipedia.org/wiki/SQL "SQL") query that can return multiple results, it has limitations on the query and suited only for SELECT type of queries.
108+
In SQL query you can use clause variables with specific data types.
86109
Internally we use prepared statements, so all incoming data is
87110
validated against SQL injection, however we had to build a connection from JavaScript types to the SQL data types
88111
therefore when doing a prepared statements, you would need to add ``:type`` to **each prepared statement variable**.
@@ -95,7 +118,7 @@ FROM users
95118
WHERE userid = @id AND language = @lang
96119
```
97120

98-
you should add ``:type`` to each ``@parameter`` so your SQL queryOld will looks like this:
121+
you should add ``:type`` to each ``@parameter`` so your SQL query will looks like this:
99122

100123
```sql
101124
SELECT
@@ -116,24 +139,69 @@ Following types are supported:
116139
Checkbox ``Don't throw Error on an Empty Result`` allows to emit an empty response, otherwise you will get an error on empty response.
117140

118141
#### Input fields description
119-
Component supports dynamic incoming metadata - as soon as your queryOld is in place it will be parsed and incoming metadata will be generated accordingly.
142+
Component supports dynamic incoming metadata - as soon as your query is in place it will be parsed and incoming metadata will be generated accordingly.
120143

121144
### LOOKUP BY PRIMARY KEY
122145
![image](https://user-images.githubusercontent.com/40201204/43592505-5b6bbfe8-967e-11e8-845e-2ce8ac707357.png)
123-
The action will execute select queryOld from a ``Table`` dropdown field, as criteria can be used only [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY"). The action returns only one result (a primary key is unique).
146+
147+
The action will execute select query from a ``Table`` dropdown field, as criteria can be used only [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY"). The action returns only one result (a primary key is unique).
124148
Checkbox ``Don't throw Error on an Empty Result`` allows to emit an empty response, otherwise you will get an error on empty response.
125149
#### Input fields description
126150
![image](https://user-images.githubusercontent.com/40201204/43644579-f593d1c8-9737-11e8-9b97-ee9e575a19f7.png)
127151
As an input metadata you will get a Primary Key field to provide the data inside as a clause value.
128152

153+
### UPSERT BY PRIMARY KEY
154+
The action will execute ``SELECT`` command from a ``Tables`` dropdown field, as search criteria can be used only [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY"), and execute ``INSERT`` command by PRIMARY KEY with specified field, if result does not found, else - action will execute ``UPDATE`` command by PRIMARY KEY with specified field. The action returns only one result row (a primary key is unique).
155+
1. Find and select jdbc-component in the component repository
156+
![image](https://user-images.githubusercontent.com/16806832/44981615-c70a9d80-af7b-11e8-8055-3b553abe8212.png)
157+
158+
2. Create new or select existing credentials
159+
![image](https://user-images.githubusercontent.com/16806832/44981652-e86b8980-af7b-11e8-897e-04d1fc9a93cf.png)
160+
161+
3. Select action "Upsert Row By Primary Key" from list
162+
![image](https://user-images.githubusercontent.com/16806832/44981700-0d5ffc80-af7c-11e8-9ac3-aedb16e1d788.png)
163+
164+
4. Select table from ``Table`` dropdown list
165+
![image](https://user-images.githubusercontent.com/16806832/44981754-38e2e700-af7c-11e8-87d3-f029a7fec8fa.png)
166+
167+
5. Specify input data (field with red asterisk is Primary key), and click "Continue"
168+
![image](https://user-images.githubusercontent.com/16806832/44981854-83fcfa00-af7c-11e8-9ef2-8c06e77fed1e.png)
169+
170+
6. Retrieving sample
171+
![image](https://user-images.githubusercontent.com/16806832/44983059-86f9e980-af80-11e8-8178-77e463488c7a.png)
172+
173+
7. Retrieve sample result
174+
![image](https://user-images.githubusercontent.com/16806832/44982952-2ec2e780-af80-11e8-98b1-58c3adbc15b9.png)
175+
176+
8. Click "Continue"
177+
![image](https://user-images.githubusercontent.com/16806832/44983101-b0b31080-af80-11e8-82d8-0e70e4b4ff97.png)
178+
179+
9. Finish component configuration
180+
![image](https://user-images.githubusercontent.com/16806832/44983365-90378600-af81-11e8-9be4-4dbb39af0fdc.png)
181+
182+
#### Input fields description
183+
As an input metadata you will get all fields of selected table. [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY") is required field (will mark as asterisk) and other input fields are optional.
184+
![image](https://user-images.githubusercontent.com/16806832/44397461-1a76f780-a549-11e8-8247-9a6f9aa3f3b4.png)
185+
129186
### DELETE BY PRIMARY KEY
130187
![image](https://user-images.githubusercontent.com/40201204/43592505-5b6bbfe8-967e-11e8-845e-2ce8ac707357.png)
131-
The action will execute delete queryOld from a ``Table`` dropdown field, as criteria can be used only [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY"). The action returns count of affected rows.
188+
The action will execute delete query from a ``Table`` dropdown field, as criteria can be used only [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY"). The action returns count of affected rows.
132189
Checkbox ``Don't throw Error on an Empty Result`` allows to emit an empty response, otherwise you will get an error on empty response.
133190
#### Input fields description
134191
![image](https://user-images.githubusercontent.com/40201204/43644579-f593d1c8-9737-11e8-9b97-ee9e575a19f7.png)
135192
As an input metadata you will get a Primary Key field to provide the data inside as a clause value.
136193

194+
## Current limitations
195+
1. Only tables with one [PRIMARY KEY](https://en.wikipedia.org/wiki/Primary_key "PRIMARY KEY") is supported. You will see the message ``Table has not Primary Key. Should be one Primary Key
196+
``, if the selected table doesn't have a primary key. Also, you will see the message ``Composite Primary Key is not supported
197+
``, if the selected table has composite primary key.
198+
2. Only following versions of database types are supported:
199+
- ``MySQL`` - compatible with MySQL Server 5.5, 5.6, 5.7 and 8.0.
200+
- ``PostgreSQL`` - compatible with PostgreSQL 8.2 and higher
201+
- ``Oracle`` - compatible with Oracle Database 8.1.7 - 12.1.0.2
202+
- ``MSSQL`` - compatible with Microsoft SQL Server 2008 R2 and higher
203+
3. The current implementation of the action ``Upsert By Primary Key`` doesn't mark non-nullable fields as required fields at a dynamic metadata. In case of updating such fields with an empty value you will get SQL Exception ``Cannot insert the value NULL into...``. You should manually fill in all non-nullable fields with previous data, if you want to update part of columns in a row, even if data in that fields doesn't change.
204+
137205
## Known issues
138206
No known issues are there yet.
139207

changelog.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# jdbc Component Change Log
2+
3+
4+
## [Unrelased]
5+
### Added
6+
### Change
7+
### Deprecated
8+
### Removed
9+
### Fixed
10+
### Security
11+
12+
## [V2.0] 2018-09-19 elastic.io
13+
14+
### Added
15+
16+
Triggers
17+
- SELECT
18+
- GET ROWS POLLING
19+
20+
Actions
21+
- SELECT
22+
- LOOKUP BY PRIMARY KEY
23+
- UPSERT BY PRIMARY KEY (for migration)
24+
- DELETE BY PRIMARY KEY
25+
26+
### Removed
27+
Actions
28+
- CreateOrUpdateRecord
29+
30+
### Fixed
31+
- fix issue in postgresql - getDate(null)
32+
- fix null values as input for select
33+

component.json

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -173,6 +173,21 @@
173173
},
174174
"dynamicMetadata": "io.elastic.jdbc.PrimaryColumnNamesProvider"
175175
},
176+
"upsertRowByPrimaryKey": {
177+
"main": "io.elastic.jdbc.actions.UpsertRowByPrimaryKey",
178+
"title": "Upsert Row By Primary Key",
179+
"description": "Executes upsert by primary key",
180+
"fields": {
181+
"tableName": {
182+
"viewClass": "SelectView",
183+
"prompt": "Select a Table",
184+
"label": "Table",
185+
"required": true,
186+
"model": "io.elastic.jdbc.TableNameProvider"
187+
}
188+
},
189+
"dynamicMetadata": "io.elastic.jdbc.ColumnNamesWithPrimaryKeyProvider"
190+
},
176191
"deleteRowByPrimaryKey": {
177192
"main": "io.elastic.jdbc.actions.DeleteRowByPrimaryKey",
178193
"title": "Delete Row By Primary Key",

src/main/java/io/elastic/jdbc/ColumnNamesProvider.java

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717

1818
public class ColumnNamesProvider implements DynamicMetadataProvider, SelectModelProvider {
1919

20-
private static final Logger logger = LoggerFactory.getLogger(ColumnNamesProvider.class);
20+
private static final Logger LOGGER = LoggerFactory.getLogger(ColumnNamesProvider.class);
2121

2222
@Override
2323
public JsonObject getSelectModel(JsonObject configuration) {
@@ -57,8 +57,8 @@ public JsonObject getColumns(JsonObject configuration) {
5757
ResultSet rs = null;
5858
String schemaName = null;
5959
boolean isEmpty = true;
60-
Boolean isOracle = (configuration.getString("dbEngine").equals("oracle")) ? true : false;
61-
Boolean isMssql = (configuration.getString("dbEngine").equals("mssql")) ? true : false;
60+
Boolean isOracle = configuration.getString("dbEngine").equals("oracle");
61+
Boolean isMssql = configuration.getString("dbEngine").equals("mssql");
6262
try {
6363
connection = Utils.getConnection(configuration);
6464
DatabaseMetaData dbMetaData = connection.getMetaData();
@@ -71,14 +71,14 @@ public JsonObject getColumns(JsonObject configuration) {
7171
while (rs.next()) {
7272
JsonObjectBuilder field = Json.createObjectBuilder();
7373
String name = rs.getString("COLUMN_NAME");
74-
Boolean isRequired = false;
74+
Boolean isRequired;
75+
Integer isNullable = (rs.getObject("NULLABLE") != null) ? rs.getInt("NULLABLE") : 1;
7576
if (isMssql) {
7677
String isAutoincrement =
7778
(rs.getString("IS_AUTOINCREMENT") != null) ? rs.getString("IS_AUTOINCREMENT") : "";
78-
Integer isNullable = (rs.getObject("NULLABLE") != null) ? rs.getInt("NULLABLE") : 1;
7979
isRequired = isNullable == 0 && !isAutoincrement.equals("YES");
8080
} else {
81-
isRequired = false;
81+
isRequired = isNullable == 0;
8282
}
8383
field.add("required", isRequired)
8484
.add("title", name)
@@ -97,14 +97,14 @@ public JsonObject getColumns(JsonObject configuration) {
9797
try {
9898
rs.close();
9999
} catch (SQLException e) {
100-
logger.error("Failed to close result set {}", e.toString());
100+
LOGGER.error("Failed to close result set {}", e);
101101
}
102102
}
103103
if (connection != null) {
104104
try {
105105
connection.close();
106106
} catch (SQLException e) {
107-
logger.error("Failed to close connection {}", e.toString());
107+
LOGGER.error("Failed to close connection {}", e);
108108
}
109109
}
110110
}

0 commit comments

Comments
 (0)