Skip to content

Commit c67a275

Browse files
committed
Add note about message compression for kafka
1 parent f205bf2 commit c67a275

File tree

1 file changed

+6
-0
lines changed
  • docs/integrations/data-ingestion/clickpipes

1 file changed

+6
-0
lines changed

docs/integrations/data-ingestion/clickpipes/kafka.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -208,6 +208,12 @@ The following virtual columns are supported for Kafka compatible streaming data
208208
Note that the _raw_message column is only recommended for JSON data. For use cases where only the JSON string is required (such as using ClickHouse [`JsonExtract*`](/sql-reference/functions/json-functions#jsonextract-functions) functions to populate a downstream materialized
209209
view), it may improve ClickPipes performance to delete all the "non-virtual" columns.
210210

211+
## Best Practices {#best-practices}
212+
213+
### Message Compression {#compression}
214+
We strongly recommend using compression for your Kafka topics. Compression can result in a significant saving in data transfer costs with virtually no performance hit.
215+
To learn more about message compression in Kafka, we recommend starting with this [guide](https://www.confluent.io/blog/apache-kafka-message-compression/).
216+
211217
## Limitations {#limitations}
212218

213219
- [DEFAULT](/sql-reference/statements/create/table#default) is not supported.

0 commit comments

Comments
 (0)