Skip to content

Commit dda58c9

Browse files
committed
Address comments
1 parent ab0b2b8 commit dda58c9

File tree

1 file changed

+8
-3
lines changed
  • docs/integrations/data-ingestion/clickpipes

1 file changed

+8
-3
lines changed

docs/integrations/data-ingestion/clickpipes/index.md

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -70,8 +70,13 @@ Steps:
7070
<Image img={cp_custom_role} alt="Assign a custom role" size="lg" border/>
7171

7272
## Error reporting {#error-reporting}
73-
ClickPipes will create a table next to your destination table with the postfix `<destination_table_name>_clickpipes_error`. This table will contain any errors from malformed data or mismatched schema and will include the entirety of the invalid message. In addition to this, error not related to data will be stored in the `system.clickpipes_log` table, this will store all other errors related to the operation of your ClickPipe (network, connectivity, etc.). Both error tables have a [TTL](/engines/table-engines/mergetree-family/mergetree#table_engine-mergetree-ttl) of 7 days.
74-
If ClickPipes cannot connect to a data source or destination after 15min., ClickPipes instance stops and stores an appropriate message in the system error table (providing the ClickHouse instance is available).
73+
ClickPipes will store errors in two separate tables depending on the type of error encountered during the ingestion process.
74+
### Record Errors
75+
ClickPipes will create a table next to your destination table with the postfix `<destination_table_name>_clickpipes_error`. This table will contain any errors from malformed data or mismatched schema and will include the entirety of the invalid message. This table has a [TTL](/engines/table-engines/mergetree-family/mergetree#table_engine-mergetree-ttl) of 7 days.
76+
### System Errors
77+
Errors related to the operation of the ClickPipe will be stored in the `system.clickpipes_log` table. This will store all other errors related to the operation of your ClickPipe (network, connectivity, etc.). This table has a [TTL](/engines/table-engines/mergetree-family/mergetree#table_engine-mergetree-ttl) of 7 days.
78+
79+
If ClickPipes cannot connect to a data source or destination after 15 min., the ClickPipes instance stops and stores an appropriate message in the system error table (providing the ClickHouse instance is available).
7580

7681
## F.A.Q {#faq}
7782
- **What is ClickPipes?**
@@ -88,4 +93,4 @@ If ClickPipes cannot connect to a data source or destination after 15min., Click
8893

8994
- **Is there a way to handle errors or failures when using ClickPipes for Kafka?**
9095

91-
Yes, ClickPipes for Kafka will automatically retry case of failures when consuming data from Kafka. ClickPipes also creates a dedicated error table that will hold errors and malformed data for 7 days.
96+
Yes, ClickPipes for Kafka will automatically retry in the event of failures when consuming data from Kafka for any operational issue including network issues, connectivity issues, etc. In the event of malformed data or invalid schema, ClickPipes will store the record in the record_error table and continue processing.

0 commit comments

Comments
 (0)