Skip to content

Commit b9c1f70

Browse files
itsmekumariitsmekumari
committed
e2e additional cloudsqlpostgresql tests
1 parent 6a0146d commit b9c1f70

File tree

6 files changed

+295
-2
lines changed

6 files changed

+295
-2
lines changed

‎cloudsql-postgresql-plugin/src/e2e-test/features/sink/DesignTimeWithValidation.feature‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ Feature: CloudSQL-PostgreSQL Sink - Verify CloudSQL-postgreSQL Sink Plugin Error
2525
Then Click on the Validate button
2626
Then Verify mandatory property error for below listed properties:
2727
| jdbcPluginName |
28+
| connectionName |
2829
| referenceName |
2930
| database |
3031
| tableName |
@@ -141,3 +142,133 @@ Feature: CloudSQL-PostgreSQL Sink - Verify CloudSQL-postgreSQL Sink Plugin Error
141142
Then Replace input plugin property: "tableName" with value: "targetTable"
142143
Then Click on the Validate button
143144
Then Verify that the Plugin Property: "user" is displaying an in-line error message: "errorMessageBlankUsername"
145+
146+
@Sink_Required
147+
Scenario: To verify CloudSQLPostgreSQL sink plugin validation error message with invalid private connection name
148+
Given Open Datafusion Project to configure pipeline
149+
When Expand Plugin group in the LHS plugins list: "Sink"
150+
When Select plugin: "CloudSQL PostgreSQL" from the plugins list as: "Sink"
151+
Then Navigate to the properties page of plugin: "CloudSQL PostgreSQL"
152+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
153+
Then Select radio button plugin property: "instanceType" with value: "private"
154+
Then Replace input plugin property: "connectionName" with value: "invalidConnectionName"
155+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
156+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
157+
Then Enter input plugin property: "referenceName" with value: "targetRef"
158+
Then Replace input plugin property: "database" with value: "databaseName"
159+
Then Replace input plugin property: "tableName" with value: "targetTable"
160+
Then Click on the Validate button
161+
Then Verify that the Plugin Property: "connectionName" is displaying an in-line error message: "errorMessagePrivateConnectionName"
162+
163+
@CLOUDSQLPOSTGRESQL_SOURCE_TEST @CLOUDSQLPOSTGRESQL_TARGET_TEST @Sink_Required
164+
Scenario: To verify CloudSQLPostgreSQL sink plugin validation error message with blank password
165+
Given Open Datafusion Project to configure pipeline
166+
When Expand Plugin group in the LHS plugins list: "Source"
167+
When Select plugin: "CloudSQL PostgreSQL" from the plugins list as: "Source"
168+
When Expand Plugin group in the LHS plugins list: "Sink"
169+
When Select plugin: "CloudSQL PostgreSQL" from the plugins list as: "Sink"
170+
Then Connect plugins: "CloudSQL PostgreSQL" and "CloudSQL PostgreSQL2" to establish connection
171+
Then Navigate to the properties page of plugin: "CloudSQL PostgreSQL"
172+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
173+
Then Select radio button plugin property: "instanceType" with value: "public"
174+
Then Replace input plugin property: "connectionName" with value: "connectionName" for Credentials and Authorization related fields
175+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
176+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
177+
Then Enter input plugin property: "referenceName" with value: "sourceRef"
178+
Then Replace input plugin property: "database" with value: "databaseName"
179+
Then Enter textarea plugin property: "importQuery" with value: "selectQuery"
180+
Then Click on the Get Schema button
181+
Then Verify the Output Schema matches the Expected Schema: "datatypesSchema"
182+
Then Validate "CloudSQL PostgreSQL" plugin properties
183+
Then Close the Plugin Properties page
184+
Then Navigate to the properties page of plugin: "CloudSQL PostgreSQL2"
185+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
186+
Then Select radio button plugin property: "instanceType" with value: "public"
187+
Then Replace input plugin property: "connectionName" with value: "connectionName" for Credentials and Authorization related fields
188+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
189+
Then Enter input plugin property: "referenceName" with value: "targetRef"
190+
Then Replace input plugin property: "database" with value: "databaseName"
191+
Then Replace input plugin property: "tableName" with value: "targetTable"
192+
Then Click on the Validate button
193+
Then Verify that the Plugin is displaying an error message: "errorMessageWithBlankPassword" on the header
194+
195+
@CLOUDSQLPOSTGRESQL_SOURCE_TEST @CLOUDSQLPOSTGRESQL_TARGET_TEST @Sink_Required
196+
Scenario Outline: To verify CloudSQLPostgreSQL sink plugin validation error message for update, upsert operation name and table key
197+
Given Open Datafusion Project to configure pipeline
198+
When Expand Plugin group in the LHS plugins list: "Source"
199+
When Select plugin: "CloudSQL PostgreSQL" from the plugins list as: "Source"
200+
When Expand Plugin group in the LHS plugins list: "Sink"
201+
When Select plugin: "CloudSQL PostgreSQL" from the plugins list as: "Sink"
202+
Then Connect plugins: "CloudSQL PostgreSQL" and "CloudSQL PostgreSQL2" to establish connection
203+
Then Navigate to the properties page of plugin: "CloudSQL PostgreSQL"
204+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
205+
Then Select radio button plugin property: "instanceType" with value: "public"
206+
Then Replace input plugin property: "connectionName" with value: "connectionName" for Credentials and Authorization related fields
207+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
208+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
209+
Then Enter input plugin property: "referenceName" with value: "sourceRef"
210+
Then Replace input plugin property: "database" with value: "databaseName"
211+
Then Enter textarea plugin property: "importQuery" with value: "selectQuery"
212+
Then Click on the Get Schema button
213+
Then Verify the Output Schema matches the Expected Schema: "datatypesSchema"
214+
Then Validate "CloudSQL PostgreSQL" plugin properties
215+
Then Close the Plugin Properties page
216+
Then Navigate to the properties page of plugin: "CloudSQL PostgreSQL2"
217+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
218+
Then Select radio button plugin property: "instanceType" with value: "public"
219+
Then Replace input plugin property: "connectionName" with value: "connectionName" for Credentials and Authorization related fields
220+
Then Replace input plugin property: "database" with value: "databaseName"
221+
Then Replace input plugin property: "tableName" with value: "targetTable"
222+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
223+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
224+
Then Enter input plugin property: "referenceName" with value: "targetRef"
225+
Then Replace input plugin property: "dbSchemaName" with value: "schema"
226+
Then Select radio button plugin property: "operationName" with value: "<options>"
227+
Then Click on the Validate button
228+
Then Verify that the Plugin Property: "operationName" is displaying an in-line error message: "errorMessageUpdateUpsertOperationName"
229+
Then Verify that the Plugin Property: "relationTableKey" is displaying an in-line error message: "errorMessageUpdateUpsertOperationName"
230+
Examples:
231+
| options |
232+
| upsert |
233+
| update |
234+
235+
@CLOUDSQLPOSTGRESQL_SOURCE_TEST @CLOUDSQLPOSTGRESQL_TARGET_TEST @Sink_Required
236+
Scenario Outline: To verify CloudSQLPostgreSQL sink plugin validation error message with invalid table key
237+
Given Open Datafusion Project to configure pipeline
238+
When Expand Plugin group in the LHS plugins list: "Source"
239+
When Select plugin: "CloudSQL PostgreSQL" from the plugins list as: "Source"
240+
When Expand Plugin group in the LHS plugins list: "Sink"
241+
When Select plugin: "CloudSQL PostgreSQL" from the plugins list as: "Sink"
242+
Then Connect plugins: "CloudSQL PostgreSQL" and "CloudSQL PostgreSQL2" to establish connection
243+
Then Navigate to the properties page of plugin: "CloudSQL PostgreSQL"
244+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
245+
Then Select radio button plugin property: "instanceType" with value: "public"
246+
Then Replace input plugin property: "connectionName" with value: "connectionName" for Credentials and Authorization related fields
247+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
248+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
249+
Then Enter input plugin property: "referenceName" with value: "sourceRef"
250+
Then Replace input plugin property: "database" with value: "databaseName"
251+
Then Enter textarea plugin property: "importQuery" with value: "selectQuery"
252+
Then Click on the Get Schema button
253+
Then Verify the Output Schema matches the Expected Schema: "datatypesSchema"
254+
Then Validate "CloudSQL PostgreSQL" plugin properties
255+
Then Close the Plugin Properties page
256+
Then Navigate to the properties page of plugin: "CloudSQL PostgreSQL2"
257+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
258+
Then Select radio button plugin property: "instanceType" with value: "public"
259+
Then Replace input plugin property: "connectionName" with value: "connectionName" for Credentials and Authorization related fields
260+
Then Replace input plugin property: "database" with value: "databaseName"
261+
Then Replace input plugin property: "tableName" with value: "targetTable"
262+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
263+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
264+
Then Enter input plugin property: "referenceName" with value: "targetRef"
265+
Then Replace input plugin property: "dbSchemaName" with value: "schema"
266+
Then Select radio button plugin property: "operationName" with value: "<options>"
267+
Then Click on the Add Button of the property: "relationTableKey" with value:
268+
| invalidCloudSQLPostgreSQLTableKey |
269+
Then Click on the Validate button
270+
Then Verify that the Plugin Property: "relationTableKey" is displaying an in-line error message: "errorMessageInvalidTableKey"
271+
Examples:
272+
| options |
273+
| upsert |
274+
| update |

‎cloudsql-postgresql-plugin/src/e2e-test/features/sink/RunTime.feature‎

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,3 +144,101 @@ Feature: CloudSQL-PostgreSQL sink - Verify data transfer from BigQuery source to
144144
Then Open and capture logs
145145
Then Verify the pipeline status is "Succeeded"
146146
Then Validate the values of records transferred to target CloudPostgreSQL table is equal to the values from BigQuery table
147+
148+
@BQ_SOURCE_TEST @CLOUDSQLPOSTGRESQL_TEST_TABLE @Sink_Required
149+
Scenario Outline: To verify data is getting transferred from BigQuery to CloudSQLPostgreSQL successfully with Advanced operations Upsert and Update for table key
150+
Given Open Datafusion Project to configure pipeline
151+
When Expand Plugin group in the LHS plugins list: "Source"
152+
When Select plugin: "BigQuery" from the plugins list as: "Source"
153+
When Expand Plugin group in the LHS plugins list: "Sink"
154+
When Select plugin: "CloudSQL PostgreSQL" from the plugins list as: "Sink"
155+
Then Connect plugins: "BigQuery" and "CloudSQL PostgreSQL" to establish connection
156+
Then Navigate to the properties page of plugin: "BigQuery"
157+
Then Replace input plugin property: "project" with value: "projectId"
158+
Then Enter input plugin property: "datasetProject" with value: "projectId"
159+
Then Enter input plugin property: "referenceName" with value: "BQReferenceName"
160+
Then Enter input plugin property: "dataset" with value: "dataset"
161+
Then Enter input plugin property: "table" with value: "bqSourceTable"
162+
Then Click on the Get Schema button
163+
Then Verify the Output Schema matches the Expected Schema: "bqOutputMultipleDatatypesSchema"
164+
Then Validate "BigQuery" plugin properties
165+
Then Close the Plugin Properties page
166+
Then Navigate to the properties page of plugin: "CloudSQL PostgreSQL"
167+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
168+
Then Select radio button plugin property: "instanceType" with value: "public"
169+
Then Replace input plugin property: "connectionName" with value: "connectionName" for Credentials and Authorization related fields
170+
Then Replace input plugin property: "database" with value: "databaseName"
171+
Then Replace input plugin property: "tableName" with value: "targetTable"
172+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
173+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
174+
Then Enter input plugin property: "referenceName" with value: "targetRef"
175+
Then Replace input plugin property: "dbSchemaName" with value: "schema"
176+
Then Select radio button plugin property: "operationName" with value: "<options>"
177+
Then Click on the Add Button of the property: "relationTableKey" with value:
178+
| CloudSQLPostgreSQLTableKey |
179+
Then Validate "CloudSQL PostgreSQL2" plugin properties
180+
Then Close the Plugin Properties page
181+
Then Save the pipeline
182+
Then Preview and run the pipeline
183+
Then Verify the preview of pipeline is "success"
184+
Then Click on the Preview Data link on the Sink plugin node: "CloudSQLPostgreSQL"
185+
Then Close the preview data
186+
Then Deploy the pipeline
187+
Then Run the Pipeline in Runtime
188+
Then Wait till pipeline is in running state
189+
Then Open and capture logs
190+
Then Verify the pipeline status is "Succeeded"
191+
Then Validate the values of records transferred to target CloudPostgreSQL table is equal to the values from BigQuery table
192+
Examples:
193+
| options |
194+
| update |
195+
| upsert |
196+
197+
@BQ_SOURCE_TEST @CLOUDSQLPOSTGRESQL_TEST_TABLE @Sink_Required
198+
Scenario Outline: To verify data is getting transferred from BigQuery source to CloudSQLPostgreSQL sink with with different isolation levels
199+
Given Open Datafusion Project to configure pipeline
200+
When Expand Plugin group in the LHS plugins list: "Source"
201+
When Select plugin: "BigQuery" from the plugins list as: "Source"
202+
When Expand Plugin group in the LHS plugins list: "Sink"
203+
When Select plugin: "CloudSQL PostgreSQL" from the plugins list as: "Sink"
204+
Then Connect plugins: "BigQuery" and "CloudSQL PostgreSQL" to establish connection
205+
Then Navigate to the properties page of plugin: "BigQuery"
206+
Then Replace input plugin property: "project" with value: "projectId"
207+
Then Enter input plugin property: "datasetProject" with value: "projectId"
208+
Then Enter input plugin property: "referenceName" with value: "BQReferenceName"
209+
Then Enter input plugin property: "dataset" with value: "dataset"
210+
Then Enter input plugin property: "table" with value: "bqSourceTable"
211+
Then Click on the Get Schema button
212+
Then Verify the Output Schema matches the Expected Schema: "bqOutputMultipleDatatypesSchema"
213+
Then Validate "BigQuery" plugin properties
214+
Then Close the Plugin Properties page
215+
Then Navigate to the properties page of plugin: "CloudSQL PostgreSQL"
216+
Then Select dropdown plugin property: "select-jdbcPluginName" with option value: "driverName"
217+
Then Select radio button plugin property: "instanceType" with value: "public"
218+
Then Replace input plugin property: "connectionName" with value: "connectionName" for Credentials and Authorization related fields
219+
Then Replace input plugin property: "database" with value: "databaseName"
220+
Then Replace input plugin property: "tableName" with value: "targetTable"
221+
Then Replace input plugin property: "user" with value: "username" for Credentials and Authorization related fields
222+
Then Replace input plugin property: "password" with value: "password" for Credentials and Authorization related fields
223+
Then Enter input plugin property: "referenceName" with value: "targetRef"
224+
Then Replace input plugin property: "dbSchemaName" with value: "schema"
225+
Then Select dropdown plugin property: "transactionIsolationLevel" with option value: "<TransactionIsolationLevels>"
226+
Then Validate "CloudSQL PostgreSQL" plugin properties
227+
Then Close the Plugin Properties page
228+
Then Save the pipeline
229+
Then Preview and run the pipeline
230+
Then Verify the preview of pipeline is "success"
231+
Then Click on the Preview Data link on the Sink plugin node: "CloudSQLPostgreSQL"
232+
Then Close the preview data
233+
Then Deploy the pipeline
234+
Then Run the Pipeline in Runtime
235+
Then Wait till pipeline is in running state
236+
Then Open and capture logs
237+
Then Verify the pipeline status is "Succeeded"
238+
Then Validate the values of records transferred to target CloudPostgreSQL table is equal to the values from BigQuery table
239+
Examples:
240+
| TransactionIsolationLevels |
241+
| TRANSACTION_REPEATABLE_READ |
242+
| TRANSACTION_READ_UNCOMMITTED |
243+
| TRANSACTION_READ_COMMITTED |
244+
| TRANSACTION_SERIALIZABLE |

0 commit comments

Comments
 (0)