Skip to content

Commit 188c357

Browse files
BlargianBlargian
authored
Merge pull request #4026 from Blargian/kafka_restructure
Kafka docs restructure
2 parents d7ce7c0 + a5c6e3d commit 188c357

File tree

14 files changed

+568
-433
lines changed

14 files changed

+568
-433
lines changed

‎.gitignore

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,3 +71,11 @@ docs/use-cases/AI_ML/index.md
7171
**.translated
7272
**.translate
7373
/ClickHouse/
74+
75+
# Ignore table of contents files
76+
docs/cloud/reference/release-notes-index.md
77+
docs/whats-new/changelog/index.md
78+
docs/cloud/manage/api/api-reference-index.md
79+
docs/getting-started/index.md
80+
docs/data-modeling/projections/index.md
81+
docs/integrations/data-ingestion/clickpipes/kafka/index.md

‎docs/cloud/reference/changelog.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -344,7 +344,7 @@ You can now grant uni-directional access to a specific data source like AWS MSK.
344344

345345
### ClickPipes now supports IAM for AWS MSK {#clickpipes-now-supports-iam-for-aws-msk}
346346

347-
You can now use IAM authentication to connect to an MSK broker with AWS MSK ClickPipes. To get started, review our [documentation](/integrations/clickpipes/kafka#iam).
347+
You can now use IAM authentication to connect to an MSK broker with AWS MSK ClickPipes. To get started, review our [documentation](/integrations/clickpipes/kafka/best-practices/#iam).
348348

349349
### Maximum replica size for new services on AWS {#maximum-replica-size-for-new-services-on-aws}
350350

@@ -424,7 +424,7 @@ High throughput can demand extra resources to meet your data volume and latency
424424

425425
*Raw Message Ingestion for Kafka and Kinesis*
426426

427-
It is now possible to ingest an entire Kafka or Kinesis message without parsing it. ClickPipes now offers support for a `_raw_message` [virtual column](/integrations/clickpipes/kafka#kafka-virtual-columns), allowing users to map the full message into a single String column. This gives you the flexibility to work with raw data as needed.
427+
It is now possible to ingest an entire Kafka or Kinesis message without parsing it. ClickPipes now offers support for a `_raw_message` [virtual column](/integrations/clickpipes/kafka/reference/#kafka-virtual-columns), allowing users to map the full message into a single String column. This gives you the flexibility to work with raw data as needed.
428428

429429
## August 29, 2024 {#august-29-2024}
430430

‎docs/integrations/data-ingestion/clickpipes/index.md

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -32,21 +32,21 @@ import Image from '@theme/IdealImage';
3232

3333
## Supported Data Sources {#supported-data-sources}
3434

35-
| Name | Logo |Type| Status | Description |
36-
|----------------------|--------------------------------------------------------------------------------------------------|----|------------------|------------------------------------------------------------------------------------------------------|
37-
| Apache Kafka | <Kafkasvg class="image" alt="Apache Kafka logo" style={{width: '3rem', 'height': '3rem'}}/> |Streaming| Stable | Configure ClickPipes and start ingesting streaming data from Apache Kafka into ClickHouse Cloud. |
38-
| Confluent Cloud | <Confluentsvg class="image" alt="Confluent Cloud logo" style={{width: '3rem'}}/> |Streaming| Stable | Unlock the combined power of Confluent and ClickHouse Cloud through our direct integration. |
39-
| Redpanda | <Image img={redpanda_logo} size="logo" alt="Redpanda logo"/> |Streaming| Stable | Configure ClickPipes and start ingesting streaming data from Redpanda into ClickHouse Cloud. |
40-
| AWS MSK | <Msksvg class="image" alt="AWS MSK logo" style={{width: '3rem', 'height': '3rem'}}/> |Streaming| Stable | Configure ClickPipes and start ingesting streaming data from AWS MSK into ClickHouse Cloud. |
41-
| Azure Event Hubs | <Azureeventhubssvg class="image" alt="Azure Event Hubs logo" style={{width: '3rem'}}/> |Streaming| Stable | Configure ClickPipes and start ingesting streaming data from Azure Event Hubs into ClickHouse Cloud. |
42-
| WarpStream | <Warpstreamsvg class="image" alt="WarpStream logo" style={{width: '3rem'}}/> |Streaming| Stable | Configure ClickPipes and start ingesting streaming data from WarpStream into ClickHouse Cloud. |
43-
| Amazon S3 | <S3svg class="image" alt="Amazon S3 logo" style={{width: '3rem', height: 'auto'}}/> |Object Storage| Stable | Configure ClickPipes to ingest large volumes of data from object storage. |
44-
| Google Cloud Storage | <Gcssvg class="image" alt="Google Cloud Storage logo" style={{width: '3rem', height: 'auto'}}/> |Object Storage| Stable | Configure ClickPipes to ingest large volumes of data from object storage. |
45-
| DigitalOcean Spaces | <DOsvg class="image" alt="Digital Ocean logo" style={{width: '3rem', height: 'auto'}}/> | Object Storage | Stable | Configure ClickPipes to ingest large volumes of data from object storage.
46-
| Azure Blob Storage | <ABSsvg class="image" alt="Azure Blob Storage logo" style={{width: '3rem', height: 'auto'}}/> | Object Storage | Private Beta | Configure ClickPipes to ingest large volumes of data from object storage.
47-
| Amazon Kinesis | <Amazonkinesis class="image" alt="Amazon Kenesis logo" style={{width: '3rem', height: 'auto'}}/> |Streaming| Stable | Configure ClickPipes and start ingesting streaming data from Amazon Kinesis into ClickHouse cloud. |
48-
| Postgres | <Postgressvg class="image" alt="Postgres logo" style={{width: '3rem', height: 'auto'}}/> |DBMS| Stable | Configure ClickPipes and start ingesting data from Postgres into ClickHouse Cloud. |
49-
| MySQL | <Mysqlsvg class="image" alt="MySQL logo" style={{width: '3rem', height: 'auto'}}/> |DBMS| Private Beta | Configure ClickPipes and start ingesting data from MySQL into ClickHouse Cloud. |
35+
| Name | Logo |Type| Status | Description |
36+
|----------------------------------------------------|--------------------------------------------------------------------------------------------------|----|------------------|------------------------------------------------------------------------------------------------------|
37+
| [Apache Kafka](/integrations/clickpipes/kafka) | <Kafkasvg class="image" alt="Apache Kafka logo" style={{width: '3rem', 'height': '3rem'}}/> |Streaming| Stable | Configure ClickPipes and start ingesting streaming data from Apache Kafka into ClickHouse Cloud. |
38+
| Confluent Cloud | <Confluentsvg class="image" alt="Confluent Cloud logo" style={{width: '3rem'}}/> |Streaming| Stable | Unlock the combined power of Confluent and ClickHouse Cloud through our direct integration. |
39+
| Redpanda | <Image img={redpanda_logo} size="logo" alt="Redpanda logo"/> |Streaming| Stable | Configure ClickPipes and start ingesting streaming data from Redpanda into ClickHouse Cloud. |
40+
| AWS MSK | <Msksvg class="image" alt="AWS MSK logo" style={{width: '3rem', 'height': '3rem'}}/> |Streaming| Stable | Configure ClickPipes and start ingesting streaming data from AWS MSK into ClickHouse Cloud. |
41+
| Azure Event Hubs | <Azureeventhubssvg class="image" alt="Azure Event Hubs logo" style={{width: '3rem'}}/> |Streaming| Stable | Configure ClickPipes and start ingesting streaming data from Azure Event Hubs into ClickHouse Cloud. |
42+
| WarpStream | <Warpstreamsvg class="image" alt="WarpStream logo" style={{width: '3rem'}}/> |Streaming| Stable | Configure ClickPipes and start ingesting streaming data from WarpStream into ClickHouse Cloud. |
43+
| Amazon S3 | <S3svg class="image" alt="Amazon S3 logo" style={{width: '3rem', height: 'auto'}}/> |Object Storage| Stable | Configure ClickPipes to ingest large volumes of data from object storage. |
44+
| Google Cloud Storage | <Gcssvg class="image" alt="Google Cloud Storage logo" style={{width: '3rem', height: 'auto'}}/> |Object Storage| Stable | Configure ClickPipes to ingest large volumes of data from object storage. |
45+
| DigitalOcean Spaces | <DOsvg class="image" alt="Digital Ocean logo" style={{width: '3rem', height: 'auto'}}/> | Object Storage | Stable | Configure ClickPipes to ingest large volumes of data from object storage.
46+
| Azure Blob Storage | <ABSsvg class="image" alt="Azure Blob Storage logo" style={{width: '3rem', height: 'auto'}}/> | Object Storage | Private Beta | Configure ClickPipes to ingest large volumes of data from object storage.
47+
| [Amazon Kinesis](/integrations/clickpipes/kinesis) | <Amazonkinesis class="image" alt="Amazon Kenesis logo" style={{width: '3rem', height: 'auto'}}/> |Streaming| Stable | Configure ClickPipes and start ingesting streaming data from Amazon Kinesis into ClickHouse cloud. |
48+
| [Postgres](/integrations/clickpipes/postgres) | <Postgressvg class="image" alt="Postgres logo" style={{width: '3rem', height: 'auto'}}/> |DBMS| Stable | Configure ClickPipes and start ingesting data from Postgres into ClickHouse Cloud. |
49+
| [MySQL](/integrations/clickpipes/mysql) | <Mysqlsvg class="image" alt="MySQL logo" style={{width: '3rem', height: 'auto'}}/> |DBMS| Private Beta | Configure ClickPipes and start ingesting data from MySQL into ClickHouse Cloud. |
5050

5151

5252
More connectors will get added to ClickPipes, you can find out more by [contacting us](https://clickhouse.com/company/contact?loc=clickpipes).

0 commit comments

Comments
 (0)