Vector fails to write data to S3 without any explicit error

[ignacyklimont]

I am deploying a Vector instance (version 0.42.0-1.x86_64) at a client (K8S setup, writing to S3 bucket). The source is Kafka and the data undergoes simple transformation. The data is pulled as expected and transformation steps works correctly - I added sinking to console for debugging and I can see that the output is correctly delivered to console.

However, when writing to S3, I encounter an issue - no error but also no output written to S3. My Vector instance does not indicate any error nor logs show it, but when running "vector top" I can clearly see that no output is written to S3 (Bytes Out equal to zero). Simultaneously, Events In and Events Out indicate expected number of events. I double-checked my S3 configuration, but all looks good. Furthermore, had there been any connection issue, I expected explicit error in logs - and there is none.

The only reason why I suspect an implicit error occurring is that I see in logs information that:

> 2025-02-12T12:14:03.998036Z DEBUG sink{component_kind="sink" component_id=aws_s3 component_type=aws_s3}:request{request_id=1}:invoke{service=s3 operation=PutObject sdk_invocation_id=1191016}:try_op:try_attempt: aws_smithy_runtime_api::client::interceptors::context: entering 'after deserialization' phase
> 
> 5882025-02-12T12:14:03.998050Z DEBUG sink{component_kind="sink" component_id=aws_s3 component_type=aws_s3}:request{request_id=1}:invoke{service=s3 operation=PutObject sdk_invocation_id=1191016}:try_op: aws_smithy_runtime::client::retries::strategy::standard: not retrying because we are out of attempts attempts=1 max_attempts=1
> 
> 5892025-02-12T12:14:03.998054Z DEBUG sink{component_kind="sink" component_id=aws_s3 component_type=aws_s3}:request{request_id=1}:invoke{service=s3 operation=PutObject sdk_invocation_id=1191016}:try_op: aws_smithy_runtime::client::orchestrator: a retry is either unnecessary or not possible, exiting attempt loop

Thus, this indeed indicates that the attempt was unsuccessful. However, aside from that information, there is no indication of any issues. Logs indicate only "DEBUG" level information, and the sinking to console returns events in expected format.

What could be the problem here? For the context, I use batch set to 300 seconds and buffer set to memory and 200.000 events. Thanks so much for your help!