Error 1815 with S3 data storage

[jerrac]

I recently move my greptimedb instance to a dedicated vm, and configured the storage to be an S3 compatible destination. Specifically the S3 endpoint that NetApp provides.

I'm still on 0.17.2 while I finish migrating data. It is still inside a docker container.

I'm querying via Grafana and the MySQL endpoint.

Unfortunately, I've been running into errors like this: (redacted bits because I'm paranoid)

convert frame from rows error: Error 1815 (HY000): (EngineExecuteQuery): Unexpected (persistent) at => Unexpected (temporary) at read, context: { url: https://s3.example.org/gt-bucket/project-db-prod/data/greptime/project_veclogs/1278/1278_0000000000/uuid.parquet, called: http_util::Client::send, service: s3, path: data/greptime/project_veclogs/1278/1278_0000000000/uuid.parquet, range: 4-1567048 } => send http request, source: error sending request for url (https://s3.example.org/gt-bucket/project-db-prod/data/greptime/project_veclogs/1278/1278_0000000000/uuid.parquet)

That specific example comes from:

select * from project_veclogs.logs_docker where message like '%media-app%' order by greptime_timestamp desc limit 200;

When querying my data. So far I haven't figured out a pattern for which queries end up with an error, or which work just fine. And plenty do work just fine.

My searches haven't really uncovered anything. So I'm not sure where the issue might be. Anyone have any ideas?

Edit: Oh, I should mention that is the error Grafana is displaying when I run that query via the Explore interface.

Edit 2: Er. I'm pretty sure this is because my S3 bucket hit the quota limit. Does that seem right?

[jerrac]

I think this was a symptom of a larger issue. Lots more detail here: https://github.com/orgs/GreptimeTeam/discussions/7580