Skip to content

Commit 934821c

Browse files
AnkitCLIAnkitCLI
authored
Merge pull request #538 from cloudsufi/e2e-tests_oraclesink
e2e tests Oracle sink
2 parents 93694c4 + e4474a9 commit 934821c

File tree

4 files changed

+205
-0
lines changed

4 files changed

+205
-0
lines changed

‎oracle-plugin/src/e2e-test/features/sink/OracleDesignTimeValidation.feature‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -173,3 +173,92 @@ Feature: Oracle sink- Verify Oracle sink plugin design time validation scenarios
173173
Then Enter input plugin property: "referenceName" with value: "sourceRef"
174174
Then Click on the Validate button
175175
Then Verify that the Plugin is displaying an error message: "errorMessageInvalidHost" on the header
176+
177+
@Oracle_Required
178+
Scenario: Verify the validation error message on header with blank database value
179+
Given Open Datafusion Project to configure pipeline
180+
When Expand Plugin group in the LHS plugins list: "Sink"
181+
When Select plugin: "Oracle" from the plugins list as: "Sink"
182+
Then Navigate to the properties page of plugin: "Oracle"
183+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
184+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
185+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
186+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
187+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
188+
Then Click plugin property: "switch-useConnection"
189+
Then Click on the Validate button
190+
Then Verify that the Plugin is displaying an error message: "blank.database.message" on the header
191+
192+
@ORACLE_SOURCE_DATATYPES_TEST @ORACLE_TARGET_DATATYPES_TEST @Oracle_Required
193+
Scenario: Verify the validation error message with blank password value
194+
Given Open Datafusion Project to configure pipeline
195+
When Expand Plugin group in the LHS plugins list: "Source"
196+
When Select plugin: "Oracle" from the plugins list as: "Source"
197+
When Expand Plugin group in the LHS plugins list: "Sink"
198+
When Select plugin: "Oracle" from the plugins list as: "Sink"
199+
Then Connect plugins: "Oracle" and "Oracle2" to establish connection
200+
Then Navigate to the properties page of plugin: "Oracle"
201+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
202+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
203+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
204+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
205+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
206+
Then Select radio button plugin property: "connectionType" with value: "service"
207+
Then Select radio button plugin property: "role" with value: "normal"
208+
Then Enter input plugin property: "referenceName" with value: "sourceRef"
209+
Then Replace input plugin property: "database" with value: "databaseName"
210+
Then Enter textarea plugin property: "importQuery" with value: "selectQuery"
211+
Then Click on the Get Schema button
212+
Then Verify the Output Schema matches the Expected Schema: "outputDatatypesSchema"
213+
Then Validate "Oracle" plugin properties
214+
Then Close the Plugin Properties page
215+
Then Navigate to the properties page of plugin: "Oracle2"
216+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
217+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
218+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
219+
Then Replace input plugin property: "database" with value: "databaseName"
220+
Then Replace input plugin property: "tableName" with value: "targetTable"
221+
Then Replace input plugin property: "dbSchemaName" with value: "schema"
222+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
223+
Then Enter input plugin property: "referenceName" with value: "targetRef"
224+
Then Select radio button plugin property: "connectionType" with value: "service"
225+
Then Select radio button plugin property: "role" with value: "normal"
226+
Then Click on the Validate button
227+
Then Verify that the Plugin is displaying an error message: "blank.connection.message" on the header
228+
229+
@ORACLE_SOURCE_DATATYPES_TEST @ORACLE_TARGET_DATATYPES_TEST @Oracle_Required
230+
Scenario: Verify the validation error message with blank host value
231+
Given Open Datafusion Project to configure pipeline
232+
When Expand Plugin group in the LHS plugins list: "Source"
233+
When Select plugin: "Oracle" from the plugins list as: "Source"
234+
When Expand Plugin group in the LHS plugins list: "Sink"
235+
When Select plugin: "Oracle" from the plugins list as: "Sink"
236+
Then Connect plugins: "Oracle" and "Oracle2" to establish connection
237+
Then Navigate to the properties page of plugin: "Oracle"
238+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
239+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
240+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
241+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
242+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
243+
Then Select radio button plugin property: "connectionType" with value: "service"
244+
Then Select radio button plugin property: "role" with value: "normal"
245+
Then Enter input plugin property: "referenceName" with value: "sourceRef"
246+
Then Replace input plugin property: "database" with value: "databaseName"
247+
Then Enter textarea plugin property: "importQuery" with value: "selectQuery"
248+
Then Click on the Get Schema button
249+
Then Verify the Output Schema matches the Expected Schema: "outputDatatypesSchema"
250+
Then Validate "Oracle" plugin properties
251+
Then Close the Plugin Properties page
252+
Then Navigate to the properties page of plugin: "Oracle2"
253+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
254+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
255+
Then Replace input plugin property: "database" with value: "databaseName"
256+
Then Replace input plugin property: "tableName" with value: "targetTable"
257+
Then Replace input plugin property: "dbSchemaName" with value: "schema"
258+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
259+
Then Enter input plugin property: "referenceName" with value: "targetRef"
260+
Then Select radio button plugin property: "connectionType" with value: "service"
261+
Then Select radio button plugin property: "role" with value: "normal"
262+
Then Click on the Validate button
263+
Then Verify that the Plugin is displaying an error message: "blank.connection.message" on the header
264+

‎oracle-plugin/src/e2e-test/features/sink/OracleRunTime.feature‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -218,3 +218,57 @@ Feature: Oracle - Verify data transfer from BigQuery source to Oracle sink
218218
Then Verify the pipeline status is "Succeeded"
219219
Then Validate records transferred to target table with record counts of BigQuery table
220220
Then Validate the values of records transferred to target Oracle table is equal to the values from source BigQuery table
221+
222+
@BQ_SOURCE_TEST @ORACLE_TEST_TABLE @CONNECTION
223+
Scenario: To verify data is getting transferred from BigQuery source to Oracle sink successfully with use connection
224+
Given Open Datafusion Project to configure pipeline
225+
When Expand Plugin group in the LHS plugins list: "Source"
226+
When Select plugin: "BigQuery" from the plugins list as: "Source"
227+
When Expand Plugin group in the LHS plugins list: "Sink"
228+
When Select plugin: "Oracle" from the plugins list as: "Sink"
229+
Then Connect plugins: "BigQuery" and "Oracle" to establish connection
230+
Then Navigate to the properties page of plugin: "BigQuery"
231+
Then Replace input plugin property: "project" with value: "projectId"
232+
Then Enter input plugin property: "datasetProject" with value: "projectId"
233+
Then Enter input plugin property: "referenceName" with value: "BQReferenceName"
234+
Then Enter input plugin property: "dataset" with value: "dataset"
235+
Then Enter input plugin property: "table" with value: "bqSourceTable"
236+
Then Click on the Get Schema button
237+
Then Verify the Output Schema matches the Expected Schema: "bqOutputDatatypesSchema"
238+
Then Validate "BigQuery" plugin properties
239+
Then Close the Plugin Properties page
240+
Then Navigate to the properties page of plugin: "Oracle"
241+
And Click plugin property: "switch-useConnection"
242+
And Click on the Browse Connections button
243+
And Click on the Add Connection button
244+
Then Click plugin property: "connector-Oracle"
245+
And Enter input plugin property: "name" with value: "connection.name"
246+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
247+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
248+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
249+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
250+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
251+
Then Select radio button plugin property: "connectionType" with value: "service"
252+
Then Select radio button plugin property: "role" with value: "normal"
253+
Then Replace input plugin property: "database" with value: "databaseName"
254+
Then Click on the Test Connection button
255+
And Verify the test connection is successful
256+
Then Click on the Create button
257+
Then Select connection: "connection.name"
258+
Then Enter input plugin property: "referenceName" with value: "targetRef"
259+
Then Replace input plugin property: "tableName" with value: "targetTable"
260+
Then Replace input plugin property: "dbSchemaName" with value: "schema"
261+
Then Validate "Oracle" plugin properties
262+
Then Close the Plugin Properties page
263+
Then Save the pipeline
264+
Then Preview and run the pipeline
265+
Then Verify the preview of pipeline is "success"
266+
Then Click on preview data for Oracle sink
267+
Then Close the preview data
268+
Then Deploy the pipeline
269+
Then Run the Pipeline in Runtime
270+
Then Wait till pipeline is in running state
271+
Then Open and capture logs
272+
Then Verify the pipeline status is "Succeeded"
273+
Then Validate records transferred to target table with record counts of BigQuery table
274+
Then Validate the values of records transferred to target Oracle table is equal to the values from source BigQuery table

‎oracle-plugin/src/e2e-test/features/sink/OracleRunTimeMacro.feature‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,3 +88,64 @@ Feature: Oracle - Verify data transfer to Oracle sink with macro arguments
8888
Then Close the pipeline logs
8989
Then Validate records transferred to target table with record counts of BigQuery table
9090
Then Validate the values of records transferred to target Oracle table is equal to the values from source BigQuery table
91+
92+
@ORACLE_SOURCE_DATATYPES_TEST @ORACLE_TARGET_DATATYPES_TEST @Oracle_Required
93+
Scenario: To verify data is getting transferred from Oracle to Oracle successfully with connection argument, transaction isolation macro enabled
94+
Given Open Datafusion Project to configure pipeline
95+
When Expand Plugin group in the LHS plugins list: "Source"
96+
When Select plugin: "Oracle" from the plugins list as: "Source"
97+
When Expand Plugin group in the LHS plugins list: "Sink"
98+
When Select plugin: "Oracle" from the plugins list as: "Sink"
99+
Then Connect plugins: "Oracle" and "Oracle2" to establish connection
100+
Then Navigate to the properties page of plugin: "Oracle"
101+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
102+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
103+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
104+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
105+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
106+
Then Select radio button plugin property: "connectionType" with value: "service"
107+
Then Select radio button plugin property: "role" with value: "normal"
108+
Then Enter input plugin property: "referenceName" with value: "sourceRef"
109+
Then Replace input plugin property: "database" with value: "databaseName"
110+
Then Enter textarea plugin property: "importQuery" with value: "selectQuery"
111+
Then Click on the Get Schema button
112+
Then Verify the Output Schema matches the Expected Schema: "outputDatatypesSchema"
113+
Then Validate "Oracle" plugin properties
114+
Then Close the Plugin Properties page
115+
Then Navigate to the properties page of plugin: "Oracle2"
116+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
117+
Then Replace input plugin property: "host" with value: "host" for Credentials and Authorization related fields
118+
Then Replace input plugin property: "port" with value: "port" for Credentials and Authorization related fields
119+
Then Replace input plugin property: "database" with value: "databaseName"
120+
Then Replace input plugin property: "tableName" with value: "targetTable"
121+
Then Replace input plugin property: "dbSchemaName" with value: "schema"
122+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
123+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
124+
Then Enter input plugin property: "referenceName" with value: "targetRef"
125+
Then Select radio button plugin property: "connectionType" with value: "service"
126+
Then Select radio button plugin property: "role" with value: "normal"
127+
Then Click on the Macro button of Property: "connectionArguments" and set the value to: "connArgumentsSink"
128+
Then Click on the Macro button of Property: "transactionIsolationLevel" and set the value to: "transactionIsolationLevel"
129+
Then Validate "Oracle2" plugin properties
130+
Then Close the Plugin Properties page
131+
Then Save the pipeline
132+
Then Preview and run the pipeline
133+
Then Enter runtime argument value "connectionArguments" for key "connArgumentsSink"
134+
Then Enter runtime argument value "transactionIsolationLevel" for key "transactionIsolationLevel"
135+
Then Run the preview of pipeline with runtime arguments
136+
Then Wait till pipeline preview is in running state
137+
Then Open and capture pipeline preview logs
138+
Then Verify the preview run status of pipeline in the logs is "succeeded"
139+
Then Close the pipeline logs
140+
Then Close the preview
141+
Then Deploy the pipeline
142+
Then Run the Pipeline in Runtime
143+
Then Enter runtime argument value "connectionArguments" for key "connArgumentsSink"
144+
Then Enter runtime argument value "transactionIsolationLevel" for key "transactionIsolationLevel"
145+
Then Run the Pipeline in Runtime with runtime arguments
146+
Then Wait till pipeline is in running state
147+
Then Open and capture logs
148+
Then Verify the pipeline status is "Succeeded"
149+
Then Close the pipeline logs
150+
Then Validate the values of records transferred to target table is equal to the values from source table
151+

‎oracle-plugin/src/e2e-test/resources/errorMessage.properties‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,3 +18,4 @@ errorMessageInvalidHost=Exception while trying to validate schema of database ta
1818
errorLogsMessageInvalidBoundingQuery=Spark program 'phase-1' failed with error: ORA-00936: missing expression . \
1919
Please check the system logs for more details.
2020
blank.database.message=Required property 'database' has no value.
21+
blank.connection.message=Exception while trying to validate schema of database table

0 commit comments

Comments
 (0)