Source google sheet but have AWS S3 in log???? #46389
Unanswered
ngohuuduc
asked this question in
Deployment
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I have install airbyte 1.1.0 using abctl on an Ubuntu Instance.
However i was not able to connect to google Sheet, i saw an error of 502 and then i tried my best to look into the log file located in the container (/var/log/containers/*.log).
I'm curious that I only try to access Google Sheet, why there is error of AWS S3 as below ? so weird that i'm unable to trouble shoot the issue. Please helP!
root@airbyte-abctl-control-plane:/var/log/containers# cat oogle-sheets-check-3f302c99-1dd9-4c20-a135-4413bf7cb333-0-ezlwb_airbyte-abctl_connector-sidecar-74e6bf9528998632b7cb954ab7f8f47ac9f9d2d4bc4aca2e397859dd58cbd46b.log 2024-10-05T07:01:05.810487822Z stdout F Unsetting empty environment variable 'DATA_PLANE_SERVICE_ACCOUNT_CREDENTIALS_PATH' 2024-10-05T07:01:05.810539926Z stdout F Unsetting empty environment variable 'KEYCLOAK_INTERNAL_REALM_ISSUER' 2024-10-05T07:01:05.810631833Z stdout F Unsetting empty environment variable 'DATA_PLANE_SERVICE_ACCOUNT_EMAIL' 2024-10-05T07:01:05.810698438Z stdout F Unsetting empty environment variable 'KEYCLOAK_CLIENT_ID' 2024-10-05T07:01:05.810775044Z stdout F Unsetting empty environment variable 'CONTROL_PLANE_AUTH_ENDPOINT' 2024-10-05T07:01:08.12047182Z stdout F 2024-10-05T07:01:08.118630479Z main INFO Loading mask data from '/seed/specs_secrets_mask.yaml 2024-10-05T07:01:13.088783204Z stdout F 2024-10-05 07:01:13 INFO i.m.c.e.DefaultEnvironment(<init>):168 - Established active environments: [k8s, cloud, worker-v2, control-plane, oss, local-secrets] 2024-10-05T07:01:21.388132638Z stdout F 2024-10-05 07:01:21 INFO i.a.c.ApplicationKt(main):20 - Context started 2024-10-05T07:01:21.402247876Z stdout F 2024-10-05 07:01:21 INFO i.a.c.ApplicationKt(main):21 - 12578057197.00 ns/exec (total: 12.58s, 1 executions) 2024-10-05T07:01:25.90642705Z stdout F 2024-10-05 07:01:25 INFO i.a.c.ApplicationKt(main):28 - Sidecar created 2024-10-05T07:01:25.906457152Z stdout F 2024-10-05 07:01:25 INFO i.a.c.ApplicationKt(main):29 - 8536856942.00 ns/exec (total: 17.07s, 2 executions) 2024-10-05T07:01:29.00903198Z stdout F 2024-10-05 07:01:29 INFO i.a.c.i.LineGobbler(voidCall):166 - 2024-10-05T07:01:29.090075001Z stdout F 2024-10-05T07:01:29.089122624Z pool-5-thread-1 ERROR Recursive call to appender SecretMaskRewrite 2024-10-05T07:01:31.439877562Z stdout F 2024-10-05 07:01:31 INFO i.a.c.i.LineGobbler(voidCall):166 - ----- START CHECK ----- 2024-10-05T07:01:31.442926208Z stdout F 2024-10-05 07:01:31 INFO i.a.c.i.LineGobbler(voidCall):166 - 2024-10-05T07:01:32.077989104Z stdout F 2024-10-05 07:01:32 WARN c.a.l.CommonsLog(warn):113 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant.If you are using Java 9+, you will need to include javax.xml.bind:jaxb-api as a dependency. 2024-10-05T07:01:32.902954679Z stdout F 2024-10-05 07:01:32 ERROR c.v.l.BufferPublisher(endPublish):69 - Cannot end publish with com.van.logging.aws.S3PublishHelper@686cc567 due to error 2024-10-05T07:01:32.902984481Z stdout F java.lang.RuntimeException: Cannot end publishing: Cannot publish to S3: Storage backend has reached its minimum free drive threshold. Please delete a few objects to proceed. (Service: Amazon S3; Status Code: 507; Error Code: XMinioStorageFull; Request ID: 17FB7C6D93A48F64; S3 Extended Request ID: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8; Proxy: null) 2024-10-05T07:01:32.902988982Z stdout F at com.van.logging.AbstractFilePublishHelper.end(AbstractFilePublishHelper.java:66) ~[appender-core-5.3.2.jar:?] 2024-10-05T07:01:32.902992682Z stdout F at com.van.logging.BufferPublisher.endPublish(BufferPublisher.java:67) ~[appender-core-5.3.2.jar:?] 2024-10-05T07:01:32.902995782Z stdout F at com.van.logging.LoggingEventCache.publishEventsFromFile(LoggingEventCache.java:198) ~[appender-core-5.3.2.jar:?] 2024-10-05T07:01:32.902998783Z stdout F at com.van.logging.LoggingEventCache.lambda$publishCache$0(LoggingEventCache.java:243) ~[appender-core-5.3.2.jar:?] 2024-10-05T07:01:32.903006383Z stdout F at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?] 2024-10-05T07:01:32.903010184Z stdout F at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?] 2024-10-05T07:01:32.903013084Z stdout F at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?] 2024-10-05T07:01:32.903016184Z stdout F at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?] 2024-10-05T07:01:32.903019784Z stdout F at java.base/java.lang.Thread.run(Thread.java:1583) [?:?] 2024-10-05T07:01:32.903023485Z stdout F Caused by: java.lang.RuntimeException: Cannot publish to S3: Storage backend has reached its minimum free drive threshold. Please delete a few objects to proceed. (Service: Amazon S3; Status Code: 507; Error Code: XMinioStorageFull; Request ID: 17FB7C6D93A48F64; S3 Extended Request ID: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8; Proxy: null) 2024-10-05T07:01:32.903039186Z stdout F at com.van.logging.aws.S3PublishHelper.publishFile(S3PublishHelper.java:131) ~[appender-core-5.3.2.jar:?] 2024-10-05T07:01:32.903042386Z stdout F at com.van.logging.AbstractFilePublishHelper.end(AbstractFilePublishHelper.java:61) ~[appender-core-5.3.2.jar:?] 2024-10-05T07:01:32.903045086Z stdout F ... 8 more 2024-10-05T07:01:32.903047987Z stdout F Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Storage backend has reached its minimum free drive threshold. Please delete a few objects to proceed. (Service: Amazon S3; Status Code: 507; Error Code: XMinioStorageFull; Request ID: 17FB7C6D93A48F64; S3 Extended Request ID: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8; Proxy: null) 2024-10-05T07:01:32.903051487Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1880) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903054387Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903057187Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903060088Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903065288Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.90309629Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903099991Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903102891Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903105591Z stdout F at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903108491Z stdout F at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903111192Z stdout F at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541) ~[aws-java-sdk-core-1.12.770.jar:?] 2024-10-05T07:01:32.903113892Z stdout F at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5575) ~[aws-java-sdk-s3-1.12.770.jar:?] 2024-10-05T07:01:32.903116692Z stdout F at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5522) ~[aws-java-sdk-s3-1.12.770.jar:?] 2024-10-05T07:01:32.903122993Z stdout F at com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:425) ~[aws-java-sdk-s3-1.12.770.jar:?] 2024-10-05T07:01:32.903125793Z stdout F at com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:6656) ~[aws-java-sdk-s3-1.12.770.jar:?] 2024-10-05T07:01:32.903128593Z stdout F at com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1908) ~[aws-java-sdk-s3-1.12.770.jar:?] 2024-10-05T07:01:32.903131293Z stdout F at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1868) ~[aws-java-sdk-s3-1.12.770.jar:?] 2024-10-05T07:01:32.903133994Z stdout F at com.van.logging.aws.S3PublishHelper.publishFile(S3PublishHelper.java:126) ~[appender-core-5.3.2.jar:?] 2024-10-05T07:01:32.903141294Z stdout F at com.van.logging.AbstractFilePublishHelper.end(AbstractFilePublishHelper.java:61) ~[appender-core-5.3.2.jar:?] 2024-10-05T07:01:32.903144194Z stdout F ... 8 more
Beta Was this translation helpful? Give feedback.
All reactions