Wrap error thrown in Storage Client #1492

psainics · 2025-01-15T22:22:07Z

Test

Test Case

Raw Logs


2025-01-17 12:56:57,592 - ERROR [WorkflowDriver:i.c.c.i.a.r.w.WorkflowProgramController@90] - Workflow service 'workflow.default.GCS_COPY_TEST_RUN.DataPipelineWorkflow.08d9c862-d4a4-11ef-9d3e-000000f7989a' failed.
io.cdap.cdap.api.exception.WrappedStageException: Stage 'GCS Copy' encountered : io.cdap.cdap.api.exception.ProgramFailureException: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
POST https://storage.googleapis.com/storage/v1/b?project=cdf-entcon&projection=full
{
"code" : 403,
"errors" : [ {
  "domain" : "global",
  "message" : "[email protected] does not have storage.buckets.create access to the Google Cloud project. Permission 'storage.buckets.create' denied on resource (or it may not exist).",
  "reason" : "forbidden"
} ],
"message" : "[email protected] does not have storage.buckets.create access to the Google Cloud project. Permission 'storage.buckets.create' denied on resource (or it may not exist)."
}
  at io.cdap.cdap.etl.common.plugin.ExceptionWrappingCaller.call(ExceptionWrappingCaller.java:64)
  at io.cdap.cdap.etl.common.plugin.WrappedAction.run(WrappedAction.java:48)
  at io.cdap.cdap.etl.batch.customaction.PipelineAction.run(PipelineAction.java:91)
  at io.cdap.cdap.internal.app.runtime.AbstractContext.execute(AbstractContext.java:608)
  at io.cdap.cdap.internal.app.runtime.workflow.CustomActionExecutor.execute(CustomActionExecutor.java:90)
  at io.cdap.cdap.internal.app.runtime.workflow.WorkflowDriver.executeCustomAction(WorkflowDriver.java:449)
  at io.cdap.cdap.internal.app.runtime.workflow.WorkflowDriver.executeNode(WorkflowDriver.java:489)
  at io.cdap.cdap.internal.app.runtime.workflow.WorkflowDriver.executeAll(WorkflowDriver.java:669)
  at io.cdap.cdap.internal.app.runtime.workflow.WorkflowDriver.run(WorkflowDriver.java:653)
  at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:52)
  at java.lang.Thread.run(Thread.java:750)
Caused by: io.cdap.cdap.api.exception.ProgramFailureException: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
POST https://storage.googleapis.com/storage/v1/b?project=cdf-entcon&projection=full
{
"code" : 403,
"errors" : [ {
  "domain" : "global",
  "message" : "[email protected] does not have storage.buckets.create access to the Google Cloud project. Permission 'storage.buckets.create' denied on resource (or it may not exist).",
  "reason" : "forbidden"
} ],
"message" : "[email protected] does not have storage.buckets.create access to the Google Cloud project. Permission 'storage.buckets.create' denied on resource (or it may not exist)."
}
  at io.cdap.cdap.api.exception.ProgramFailureException$Builder.build(ProgramFailureException.java:236)
  at io.cdap.cdap.api.exception.ErrorUtils.getProgramFailureException(ErrorUtils.java:198)
  at io.cdap.plugin.gcp.common.GCPErrorDetailsProviderUtil.getProgramFailureException(GCPErrorDetailsProviderUtil.java:65)
  at io.cdap.plugin.gcp.common.GCPErrorDetailsProviderUtil.getHttpResponseExceptionDetailsFromChain(GCPErrorDetailsProviderUtil.java:83)
  at io.cdap.plugin.gcp.gcs.StorageClient.createBucketIfNotExists(StorageClient.java:142)
  at io.cdap.plugin.gcp.gcs.actions.GCSCopy.run(GCSCopy.java:79)
  at io.cdap.cdap.etl.common.plugin.WrappedAction.lambda$run$1(WrappedAction.java:49)
  at io.cdap.cdap.etl.common.plugin.Caller$1.call(Caller.java:30)
  at io.cdap.cdap.etl.common.plugin.ExceptionWrappingCaller.call(ExceptionWrappingCaller.java:62)
  ... 10 common frames omitted
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
POST https://storage.googleapis.com/storage/v1/b?project=cdf-entcon&projection=full
{
"code" : 403,
"errors" : [ {
  "domain" : "global",
  "message" : "[email protected] does not have storage.buckets.create access to the Google Cloud project. Permission 'storage.buckets.create' denied on resource (or it may not exist).",
  "reason" : "forbidden"
} ],
"message" : "[email protected] does not have storage.buckets.create access to the Google Cloud project. Permission 'storage.buckets.create' denied on resource (or it may not exist)."
}
  at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
  at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
  at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
  at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:428)
  at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
  at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:514)
  at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:455)
  at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:565)
  at com.google.cloud.storage.spi.v1.HttpStorageRpc.create(HttpStorageRpc.java:273)
  at com.google.cloud.storage.StorageImpl.lambda$create$0(StorageImpl.java:125)
  at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
  at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
  at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
  at com.google.cloud.storage.Retrying.run(Retrying.java:51)
  at com.google.cloud.storage.StorageImpl.run(StorageImpl.java:1374)
  at com.google.cloud.storage.StorageImpl.create(StorageImpl.java:124)
  at io.cdap.plugin.gcp.common.GCPUtils.createBucket(GCPUtils.java:297)
  at io.cdap.plugin.gcp.gcs.StorageClient.createBucketIfNotExists(StorageClient.java:130)
  ... 14 common frames omitted

POST v3/namespaces/{namespace-id}/apps/{app-id}/workflows/DataPipelineWorkflow/runs/{run-id}/classify

[
{
  "stageName": "GCS Copy",
  "errorCategory": "Plugin-'GCS Copy'",
  "errorReason": "403 Forbidden. Please check you have permission to access this resource. For more details, see https://cloud.google.com/storage/docs/json_api/v1/status-codes",
  "errorMessage": "com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden\nPOST https://storage.googleapis.com/storage/v1/b?project=cdf-entcon&projection=full\n{\n  \"code\" : 403,\n  \"errors\" : [ {\n    \"domain\" : \"global\",\n    \"message\" : \"[email protected] does not have storage.buckets.create access to the Google Cloud project. Permission 'storage.buckets.create' denied on resource (or it may not exist).\",\n    \"reason\" : \"forbidden\"\n  } ],\n  \"message\" : \"[email protected] does not have storage.buckets.create access to the Google Cloud project. Permission 'storage.buckets.create' denied on resource (or it may not exist).\"\n}",
  "errorType": "USER",
  "dependency": "true",
  "errorCodeType": "HTTP",
  "errorCode": "403",
  "supportedDocumentationUrl": "https://cloud.google.com/storage/docs/json_api/v1/status-codes"
}
]

itsankit-google

please add screenshots of testing in CDAP sandbox.

itsankit-google

There are some more methods in this class where storage client is used, we need to do the same thing there.

For example:

google-cloud/src/main/java/io/cdap/plugin/gcp/gcs/StorageClient.java

Line 92 in a9967d5

    
           storage.update(BlobInfo.newBuilder(blob.getBlobId()).setMetadata(metaData).build());

itsankit-google · 2025-01-17T12:11:55Z

src/main/java/io/cdap/plugin/gcp/gcs/StorageClient.java

-      throw new RuntimeException(
-        String.format("Unable to access source bucket %s. ", sourcePath.getBucket())
-          + "Ensure you entered the correct bucket path.", e);
+      String errorReason = String.format("Unable to access GCS bucket '%s'", destPath.getBucket());


sourcePath.getBucket()

psainics · 2025-01-17T15:49:02Z

There are some more methods in this class where storage client is used, we need to do the same thing there.

For example:

google-cloud/src/main/java/io/cdap/plugin/gcp/gcs/StorageClient.java

Line 92 in a9967d5

storage.update(BlobInfo.newBuilder(blob.getBlobId()).setMetadata(metaData).build());

Updated !

psainics · 2025-01-20T20:44:43Z

Did not wrap mapMetaDataForAllBlobs as when it's called a catch block is already present and only a warning is logged.

https://github.com/data-integrations/google-cloud/blob/a9967d5a075e9371a4da0fe4824e9404b51a372a/src/main/java/io/cdap/plugin/gcp/gcs/sink/GCSBatchSink.java#L212C1-L218

itsankit-google · 2025-01-21T05:46:00Z

Please squash commits before merge.

itsankit-google · 2025-01-21T05:47:23Z

Did not wrap mapMetaDataForAllBlobs as when it's called a catch block is already present and only a warning is logged.

https://github.com/data-integrations/google-cloud/blob/a9967d5a075e9371a4da0fe4824e9404b51a372a/src/main/java/io/cdap/plugin/gcp/gcs/sink/GCSBatchSink.java#L212C1-L218

It is okay, we can still wrap if that method gets used anywhere later.

psainics added the build Trigger unit test build label Jan 15, 2025

psainics self-assigned this Jan 15, 2025

psainics marked this pull request as ready for review January 16, 2025 06:01

psainics mentioned this pull request Jan 16, 2025

[PLUGIN-1842] Error Management catch known errors [GCSBucketCreate] #1488

Merged

psainics force-pushed the fem/action/StorageClient branch 2 times, most recently from 7cf93bd to 60162f4 Compare January 17, 2025 05:14

psainics requested a review from itsankit-google January 17, 2025 05:18

itsankit-google reviewed Jan 17, 2025

View reviewed changes

psainics requested a review from itsankit-google January 17, 2025 07:39

itsankit-google reviewed Jan 17, 2025

View reviewed changes

psainics requested a review from itsankit-google January 17, 2025 15:50

psainics force-pushed the fem/action/StorageClient branch 2 times, most recently from e65fed2 to 2d00e1b Compare January 20, 2025 02:46

itsankit-google approved these changes Jan 21, 2025

View reviewed changes

psainics force-pushed the fem/action/StorageClient branch from bb755ad to 941af45 Compare January 21, 2025 05:52

Catch known exceptions in ProgramFailureException

083b430

psainics force-pushed the fem/action/StorageClient branch from 941af45 to 083b430 Compare January 21, 2025 07:02

psainics merged commit d5a6e64 into data-integrations:develop Jan 21, 2025
15 of 16 checks passed

psainics deleted the fem/action/StorageClient branch January 21, 2025 07:59

psainics mentioned this pull request Jan 21, 2025

[🍒]Wrap error thrown in Storage Client #1503

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Wrap error thrown in Storage Client #1492

Wrap error thrown in Storage Client #1492

Uh oh!

psainics commented Jan 15, 2025 •

edited

Loading

Uh oh!

itsankit-google left a comment

Uh oh!

itsankit-google left a comment

Uh oh!

itsankit-google Jan 17, 2025

Uh oh!

psainics Jan 17, 2025

Uh oh!

psainics commented Jan 17, 2025

Uh oh!

psainics commented Jan 20, 2025

Uh oh!

itsankit-google commented Jan 21, 2025

Uh oh!

itsankit-google commented Jan 21, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Wrap error thrown in Storage Client #1492

Wrap error thrown in Storage Client #1492

Uh oh!

Conversation

psainics commented Jan 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test

Uh oh!

itsankit-google left a comment

Choose a reason for hiding this comment

Uh oh!

itsankit-google left a comment

Choose a reason for hiding this comment

Uh oh!

itsankit-google Jan 17, 2025

Choose a reason for hiding this comment

Uh oh!

psainics Jan 17, 2025

Choose a reason for hiding this comment

Uh oh!

psainics commented Jan 17, 2025

Uh oh!

psainics commented Jan 20, 2025

Uh oh!

itsankit-google commented Jan 21, 2025

Uh oh!

itsankit-google commented Jan 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

psainics commented Jan 15, 2025 •

edited

Loading

itsankit-google commented Jan 21, 2025 •

edited

Loading