Skip to content

Commit 5997fda

Browse files
authored
Merge pull request #3830 from ClickHouse/kp/add-clickpipes-azure-blob-storage-limitation
2 parents 7646b4c + 178600d commit 5997fda

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

docs/integrations/data-ingestion/clickpipes/object-storage.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -124,10 +124,11 @@ Object Storage ClickPipes are scaled based on the minimum ClickHouse service siz
124124
To increase the throughput on large ingest jobs, we recommend scaling the ClickHouse service before creating the ClickPipe.
125125

126126
## Limitations {#limitations}
127-
- Any changes to the destination table, its materialized views (including cascading materialized views), or the materialized view's target tables won't be picked up automatically by the pipe and can result in errors. You must stop the pipe, make the necessary modifications, and then restart the pipe for the changes to be picked up and avoid errors and duplicate data due to retries.
127+
- Any changes to the destination table, its materialized views (including cascading materialized views), or the materialized view's target tables can result in temporary errors that will be retried. For best results we recommend to stop the pipe, make the necessary modifications, and then restart the pipe for the changes to be picked up and avoid errors.
128128
- There are limitations on the types of views that are supported. Please read the section on [exactly-once semantics](#exactly-once-semantics) and [view support](#view-support) for more information.
129129
- Role authentication is not available for S3 ClickPipes for ClickHouse Cloud instances deployed into GCP or Azure. It is only supported for AWS ClickHouse Cloud instances.
130130
- ClickPipes will only attempt to ingest objects at 10GB or smaller in size. If a file is greater than 10GB an error will be appended to the ClickPipes dedicated error table.
131+
- Azure Blob Storage pipes with continuous ingest on containers with over 100k files will have a latency of around 10–15 seconds in detecting new files. Latency increases with file count.
131132
- S3 / GCS ClickPipes **does not** share a listing syntax with the [S3 Table Function](/sql-reference/table-functions/s3), nor Azure with the [AzureBlobStorage Table function](/sql-reference/table-functions/azureBlobStorage).
132133
- `?` — Substitutes any single character
133134
- `*` — Substitutes any number of any characters except / including empty string

0 commit comments

Comments
 (0)