Skip to content

Commit df8cdc2

Browse files
itsmekumariitsmekumari
committed
e2e mysql sink additional tests
1 parent 80e17e5 commit df8cdc2

File tree

4 files changed

+196
-0
lines changed

4 files changed

+196
-0
lines changed

‎mysql-plugin/src/e2e-test/features/mysqlsink/DesignTimeValidation.feature‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,3 +167,91 @@ Feature: MySQL Sink - Design time validation scenarios
167167
Then Enter input plugin property: "referenceName" with value: "targetRef"
168168
Then Click on the Validate button
169169
Then Verify that the Plugin is displaying an error message: "invalid.host.message" on the header
170+
171+
@Mysql_Required
172+
Scenario: Verify required fields missing validation messages
173+
Given Open Datafusion Project to configure pipeline
174+
When Expand Plugin group in the LHS plugins list: "Sink"
175+
When Select plugin: "MySQL" from the plugins list as: "Sink"
176+
Then Navigate to the properties page of plugin: "MySQL"
177+
Then Click on the Validate button
178+
Then Verify mandatory property error for below listed properties:
179+
| jdbcPluginName |
180+
| referenceName |
181+
| database |
182+
| tableName |
183+
184+
@Mysql_Required
185+
Scenario: Verify the validation error message with missing jdbc plugin name
186+
Given Open Datafusion Project to configure pipeline
187+
When Expand Plugin group in the LHS plugins list: "Sink"
188+
When Select plugin: "MySQL" from the plugins list as: "Sink"
189+
Then Navigate to the properties page of plugin: "MySQL"
190+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
191+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
192+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
193+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
194+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
195+
Then Click plugin property: "switch-useConnection"
196+
Then Click on the Validate button
197+
Then Verify that the Plugin is displaying an error message: "blank.jdbcPluginName.message" on the header
198+
199+
@MYSQL_SOURCE_TEST @MYSQL_TARGET_TEST @Mysql_Required
200+
Scenario: Verify the validation error message with blank password value
201+
Given Open Datafusion Project to configure pipeline
202+
When Expand Plugin group in the LHS plugins list: "Source"
203+
When Select plugin: "MySQL" from the plugins list as: "Source"
204+
When Expand Plugin group in the LHS plugins list: "Sink"
205+
When Select plugin: "MySQL" from the plugins list as: "Sink"
206+
Then Connect plugins: "MySQL" and "MySQL2" to establish connection
207+
Then Navigate to the properties page of plugin: "MySQL"
208+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
209+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
210+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
211+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
212+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
213+
Then Enter input plugin property: "referenceName" with value: "sourceRef"
214+
Then Replace input plugin property: "database" with value: "databaseName"
215+
Then Enter textarea plugin property: "importQuery" with value: "selectQuery"
216+
Then Validate "MySQL" plugin properties
217+
Then Close the Plugin Properties page
218+
Then Navigate to the properties page of plugin: "MySQL2"
219+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
220+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
221+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
222+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
223+
Then Replace input plugin property: "database" with value: "databaseName"
224+
Then Replace input plugin property: "tableName" with value: "targetTable"
225+
Then Enter input plugin property: "referenceName" with value: "targetRef"
226+
Then Click on the Validate button
227+
Then Verify that the Plugin is displaying an error message: "blank.connection.message" on the header
228+
229+
@MYSQL_SOURCE_TEST @MYSQL_TARGET_TEST @Mysql_Required
230+
Scenario: Verify the validation error message with blank port value
231+
Given Open Datafusion Project to configure pipeline
232+
When Expand Plugin group in the LHS plugins list: "Source"
233+
When Select plugin: "MySQL" from the plugins list as: "Source"
234+
When Expand Plugin group in the LHS plugins list: "Sink"
235+
When Select plugin: "MySQL" from the plugins list as: "Sink"
236+
Then Connect plugins: "MySQL" and "MySQL2" to establish connection
237+
Then Navigate to the properties page of plugin: "MySQL"
238+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
239+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
240+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
241+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
242+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
243+
Then Enter input plugin property: "referenceName" with value: "sourceRef"
244+
Then Replace input plugin property: "database" with value: "databaseName"
245+
Then Enter textarea plugin property: "importQuery" with value: "selectQuery"
246+
Then Validate "MySQL" plugin properties
247+
Then Close the Plugin Properties page
248+
Then Navigate to the properties page of plugin: "MySQL2"
249+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
250+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
251+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
252+
Then Replace input plugin property: "database" with value: "databaseName"
253+
Then Replace input plugin property: "tableName" with value: "targetTable"
254+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
255+
Then Enter input plugin property: "referenceName" with value: "targetRef"
256+
Then Click on the Validate button
257+
Then Verify that the Plugin is displaying an error message: "blank.connection.message" on the header

‎mysql-plugin/src/e2e-test/features/mysqlsink/RunTime.feature‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,3 +147,57 @@ Feature: MySQL Sink - Run time scenarios
147147
Then Open and capture logs
148148
Then Verify the pipeline status is "Succeeded"
149149
Then Validate the values of records transferred to target table is equal to the values from source table
150+
151+
@BQ_SOURCE_TEST @MYSQL_TARGET_TABLE @CONNECTION @Mysql_Required
152+
Scenario: To verify data is getting transferred from BigQuery to Mysql successfully with use connection
153+
Given Open Datafusion Project to configure pipeline
154+
When Expand Plugin group in the LHS plugins list: "Source"
155+
When Select plugin: "BigQuery" from the plugins list as: "Source"
156+
And Navigate to the properties page of plugin: "BigQuery"
157+
And Enter input plugin property: "referenceName" with value: "Reference"
158+
And Replace input plugin property: "project" with value: "projectId"
159+
And Enter input plugin property: "datasetProject" with value: "datasetprojectId"
160+
And Enter input plugin property: "dataset" with value: "dataset"
161+
And Enter input plugin property: "table" with value: "bqSourceTable"
162+
Then Click on the Get Schema button
163+
Then Verify the Output Schema matches the Expected Schema: "bqOutputDatatypesSchema"
164+
Then Validate "BigQuery" plugin properties
165+
And Close the Plugin Properties page
166+
When Expand Plugin group in the LHS plugins list: "Sink"
167+
When Select plugin: "MySQL" from the plugins list as: "Sink"
168+
Then Connect plugins: "BigQuery" and "MySQL" to establish connection
169+
Then Navigate to the properties page of plugin: "MySQL"
170+
And Click plugin property: "switch-useConnection"
171+
And Click on the Browse Connections button
172+
And Click on the Add Connection button
173+
And Select Mysql Connection
174+
And Enter input plugin property: "name" with value: "connection.name"
175+
Then Enter input plugin property: "referenceName" with value: "sinkRef"
176+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
177+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
178+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
179+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
180+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
181+
Then Click on the Test Connection button
182+
And Verify the test connection is successful
183+
Then Click on the Create button
184+
And Use new connection
185+
Then Enter input plugin property: "referenceName" with value: "targetRef"
186+
Then Replace input plugin property: "database" with value: "databaseName"
187+
Then Replace input plugin property: "tableName" with value: "targetTable"
188+
Then Click plugin property: "useCompression"
189+
Then Click plugin property: "autoReconnect"
190+
Then Validate "MySQL" plugin properties
191+
Then Close the Plugin Properties page
192+
Then Save the pipeline
193+
Then Preview and run the pipeline
194+
Then Verify the preview of pipeline is "success"
195+
Then Click on preview data for MySQL sink
196+
Then Verify preview output schema matches the outputSchema captured in properties
197+
Then Close the preview data
198+
Then Deploy the pipeline
199+
Then Run the Pipeline in Runtime
200+
Then Wait till pipeline is in running state
201+
Then Open and capture logs
202+
Then Verify the pipeline status is "Succeeded"
203+
Then Validate the values of records transferred to target MySQL table is equal to the values from source BigQuery table

‎mysql-plugin/src/e2e-test/features/mysqlsink/RunTimeWithMacros.feature‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,3 +139,55 @@ Feature: MySQL Sink - Run time scenarios (macro)
139139
And Open and capture logs
140140
And Verify the pipeline status is "Failed"
141141
And Close the pipeline logs
142+
143+
@MYSQL_SOURCE_TEST @MYSQL_TARGET_TEST @Mysql_Required
144+
Scenario: To verify data is getting transferred from Mysql to Mysql successfully with connection arguments, databasename and tablename macro enabled
145+
Given Open Datafusion Project to configure pipeline
146+
When Expand Plugin group in the LHS plugins list: "Source"
147+
When Select plugin: "MySQL" from the plugins list as: "Source"
148+
When Expand Plugin group in the LHS plugins list: "Sink"
149+
When Select plugin: "MySQL" from the plugins list as: "Sink"
150+
Then Connect plugins: "MySQL" and "MySQL2" to establish connection
151+
Then Navigate to the properties page of plugin: "MySQL"
152+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
153+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
154+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
155+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
156+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
157+
Then Enter input plugin property: "referenceName" with value: "sourceRef"
158+
Then Replace input plugin property: "database" with value: "databaseName"
159+
Then Enter textarea plugin property: "importQuery" with value: "selectQuery"
160+
Then Click on the Get Schema button
161+
Then Verify the Output Schema matches the Expected Schema: "outputSchema"
162+
Then Validate "MySQL" plugin properties
163+
Then Close the Plugin Properties page
164+
Then Navigate to the properties page of plugin: "MySQL2"
165+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
166+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
167+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
168+
And Click on the Macro button of Property: "database" and set the value to: "databaseName"
169+
And Click on the Macro button of Property: "tableName" and set the value to: "tableName"
170+
Then Click on the Macro button of Property: "connectionArguments" and set the value to: "connArgumentsSink"
171+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
172+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
173+
Then Enter input plugin property: "referenceName" with value: "targetRef"
174+
Then Validate "MySQL2" plugin properties
175+
Then Close the Plugin Properties page
176+
Then Save the pipeline
177+
Then Preview and run the pipeline
178+
Then Enter runtime argument value "databaseName" for key "databaseName"
179+
Then Enter runtime argument value "connectionArguments" for key "connArgumentsSink"
180+
Then Enter runtime argument value "targetTable" for key "tableName"
181+
And Run the preview of pipeline with runtime arguments
182+
Then Verify the preview of pipeline is "success"
183+
And Close the preview
184+
And Deploy the pipeline
185+
And Run the Pipeline in Runtime
186+
Then Enter runtime argument value "databaseName" for key "databaseName"
187+
Then Enter runtime argument value "connectionArguments" for key "connArgumentsSink"
188+
Then Enter runtime argument value "targetTable" for key "tableName"
189+
And Run the Pipeline in Runtime with runtime arguments
190+
Then Wait till pipeline is in running state
191+
Then Open and capture logs
192+
Then Verify the pipeline status is "Succeeded"
193+
Then Validate the values of records transferred to target table is equal to the values from source table

‎mysql-plugin/src/e2e-test/resources/errorMessage.properties‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,5 @@ blank.username.message=Username is required when password is given.
1414
errorMessageInvalidFetchSize=Invalid fetch size. Fetch size must be a positive integer.
1515
errorMessageInvalidNumberOfSplits=Invalid value for Number of Splits '0'. Must be at least 1.
1616
errorMessageInvalidImportQuery=Import Query select must contain the string '$CONDITIONS'.
17+
blank.connection.message=Exception while trying to validate schema of database table
18+
blank.jdbcPluginName.message=Required property 'jdbcPluginName' has no value.

0 commit comments

Comments
 (0)