Merge branch 'main' of https://github.com/ClickHouse/clickhouse-docs into related_blog_component

Author: Blargian

README.md

@@ -171,28 +171,28 @@ Have you noticed a typo or found some wonky formatting? For small contributions
 
 1. Each page in Clickhouse.com/docs has an **Edit this page** link at the top:
 
-    ![The ClickHouse Docs website with the edit button highlighted.](./images/readme-edit-this-page.png)
+    ![The ClickHouse Docs website with the edit button highlighted.](./static/images/contribute/readme-edit-this-page.png)
 
     Click this button to edit this page in GitHub.
 
 1. Once you're in GitHub, click the pencil icon to edit this page:
 
-    ![README Pencil Icon](./images/readme-pencil-icon.png)
+    ![README Pencil Icon](./static/images/contribute/readme-pencil-icon.png)
 
 1. GitHub will _fork_ the repository for you. This creates a copy of the `clickhouse-docs` repository on your personal GitHub account.
 1. Make your changes in the textbox. Once you're done, click **Commit changes**:
 
-    ![README Commit Changes](./images/readme-commit-changes.png)
+    ![README Commit Changes](./static/images/contribute/readme-commit-changes.png)
 
 1. In the **Propose changes** popup, enter a descriptive title to explain the changes you just made. Keep this title to 10 words or less. If your changes are fairly complex and need further explanation, enter your comments into the **Extended description** field.
 1. Make sure **Create a new branch** is selected, and click **Propose changes**:
 
-    ![README Propose Changes](./images/readme-propose-changes.png)
+    ![README Propose Changes](./static/images/contribute/readme-propose-changes.png)
 
 1. A new page should open with a new pull request. Double-check that the title and description are accurate.
 1. If you've spoken to someone on the docs team about your changes, tag them into the **Reviewers** section:
 
-    ![README Create Pull Request](./images/readme-create-pull-request.png)
+    ![README Create Pull Request](./static/images/contribute/readme-create-pull-request.png)
 
     If you haven't mentioned your changes to anyone yet, leave the **Reviewers** section blank.
 

docs/best-practices/partionning_keys.md renamed to docs/best-practices/partitioning_keys.mdx

@@ -11,18 +11,20 @@ While ClickHouse is known for its speed and reliability, optimal performance is
 If you've run up against one of these guardrails, it's possible that you are implementing your use case in an unoptimized way. Contact our support team and we will gladly help you refine your use case to avoid exceeding the guardrails or look together at how we can increase them in a controlled manner. 
 :::
 
-- **Databases**: 1000
-- **Tables**: 5000
-- **Columns**: ∼1000 (wide format is preferred to compact)
-- **Partitions**: 50k
-- **Parts**: 100k across the entire instance
-- **Part size**: 150gb
-- **Services per organization**: 20 (soft)
-- **Services per warehouse**: 5 (soft)
-- **Low cardinality**: 10k or less
-- **Primary keys in a table**: 4-5 that sufficiently filter down the data
-- **Query concurrency**: 1000
-- **Batch ingest**: anything > 1M will be split by the system in 1M row blocks
+| Dimension | Limit |
+|-----------|-------|
+|**Databases**| 1000|
+|**Tables**| 5000|
+|**Columns**| ∼1000 (wide format is preferred to compact)|
+|**Partitions**| 50k|
+|**Parts**| 100k across the entire instance|
+|**Part size**| 150gb|
+|**Services per organization**| 20 (soft)|
+|**Services per warehouse**| 5 (soft)|
+|**Low cardinality**| 10k or less|
+|**Primary keys in a table**| 4-5 that sufficiently filter down the data|
+|**Query concurrency**| 1000|
+|**Batch ingest**| anything > 1M will be split by the system in 1M row blocks|
 
 :::note
 For Single Replica Services, the maximum number of databases is restricted to 100, and the maximum number of tables is restricted to 500. In addition, storage for Basic Tier Services is limited to 1 TB.

docs/cloud/bestpractices/usagelimits.md

@@ -17,6 +17,8 @@ import byoc_vpcpeering4 from '@site/static/images/cloud/reference/byoc-vpcpeerin
 import byoc_plb from '@site/static/images/cloud/reference/byoc-plb.png';
 import byoc_security from '@site/static/images/cloud/reference/byoc-securitygroup.png';
 import byoc_inbound from '@site/static/images/cloud/reference/byoc-inbound-rule.png';
+import byoc_subnet_1 from '@site/static/images/cloud/reference/byoc-subnet-1.png';
+import byoc_subnet_2 from '@site/static/images/cloud/reference/byoc-subnet-2.png';
 
 ## Overview {#overview}
 
@@ -50,9 +52,11 @@ Metrics and logs are stored within the customer's BYOC VPC. Logs are currently s
 
 Customers can initiate the onboarding process by reaching out to [us](https://clickhouse.com/cloud/bring-your-own-cloud). Customers need to have a dedicated AWS account and know the region they will use. At this time, we are allowing users to launch BYOC services only in the regions that we support for ClickHouse Cloud.
 
-### Prepare a Dedicated AWS Account {#prepare-a-dedicated-aws-account}
+### Prepare an AWS Account {#prepare-an-aws-account}
 
-Customers must prepare a dedicated AWS account for hosting the ClickHouse BYOC deployment to ensure better isolation. With this and the initial organization admin email, you can contact ClickHouse support.
+Customers are recommended to prepare a dedicated AWS account for hosting the ClickHouse BYOC deployment to ensure better isolation. However, using a shared account and an existing VPC is also possible. See the details in *Setup BYOC Infrastructure* below.
+
+With this account and the initial organization admin email, you can contact ClickHouse support.
 
 ### Apply CloudFormation Template {#apply-cloudformation-template}
 
@@ -68,6 +72,36 @@ After creating the CloudFormation stack, you will be prompted to set up the infr
 - **The VPC CIDR range for BYOC**: By default, we use `10.0.0.0/16` for the BYOC VPC CIDR range. If you plan to use VPC peering with another account, ensure the CIDR ranges do not overlap. Allocate a proper CIDR range for BYOC, with a minimum size of `/22` to accommodate necessary workloads.
 - **Availability Zones for BYOC VPC**: If you plan to use VPC peering, aligning availability zones between the source and BYOC accounts can help reduce cross-AZ traffic costs. In AWS, availability zone suffixes (`a, b, c`) may represent different physical zone IDs across accounts. See the [AWS guide](https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/use-consistent-availability-zones-in-vpcs-across-different-aws-accounts.html) for details.
 
+#### Customer-managed VPC {#customer-managed-vpc}
+By default, ClickHouse Cloud will provision a dedicated VPC for better isolation in your BYOC deployment. However, you can also use an existing VPC in your account. This requires specific configuration and must be coordinated through ClickHouse Support.
+
+**Configure Your Existing VPC**
+1. Allocate at least 3 private subnets across 3 different availability zones for ClickHouse Cloud to use.
+2. Ensure each subnet has a minimum CIDR range of `/23` (e.g., 10.0.0.0/23) to provide sufficient IP addresses for the ClickHouse deployment.
+3. Add the tag `kubernetes.io/role/internal-elb=1` to each subnet to enable proper load balancer configuration.
+
+<br />
+
+<Image img={byoc_subnet_1} size="lg" alt="BYOC VPC Subnet" background='black'/>
+
+<br />
+
+<br />
+
+<Image img={byoc_subnet_2} size="lg" alt="BYOC VPC Subnet Tags" background='black'/>
+
+<br />
+
+**Contact ClickHouse Support**  
+Create a support ticket with the following information:
+
+* Your AWS account ID
+* The AWS region where you want to deploy the service
+* Your VPC ID
+* The Private Subnet IDs you've allocated for ClickHouse
+* The availability zones these subnets are in
+
+
 ### Optional: Setup VPC Peering {#optional-setup-vpc-peering}
 
 To create or delete VPC peering for ClickHouse BYOC, follow the steps:
@@ -129,30 +163,7 @@ In the peering AWS account,
 <br />
 
 #### Step 6 Edit Security Group to allow Peered VPC access {#step-6-edit-security-group-to-allow-peered-vpc-access}
-In ClickHouse BYOC account,
-1. In the ClickHouse BYOC account, navigate to EC2 and locate the Private Load Balancer named like infra-xx-xxx-ingress-private.
-
-<br />
-
-<Image img={byoc_plb} size="lg" alt="BYOC Private Load Balancer" border />
-
-<br />
-
-2. Under the Security tab on the Details page, find the associated Security Group, which follows a naming pattern like `k8s-istioing-istioing-xxxxxxxxx`.
-
-<br />
-
-<Image img={byoc_security} size="lg" alt="BYOC Private Load Balancer Security Group" border />
-
-<br />
-
-3. Edit the Inbound Rules of this Security Group and add the Peered VPC CIDR range (or specify the required CIDR range as needed).
-
-<br />
-
-<Image img={byoc_inbound} size="lg" alt="BYOC Security Group Inbound Rule" border />
-
-<br />
+In the ClickHouse BYOC account, you need to update the Security Group settings to allow traffic from your peered VPC. Please contact ClickHouse Support to request the addition of inbound rules that include the CIDR ranges of your peered VPC.
 
 ---
 The ClickHouse service should now be accessible from the peered VPC.

docs/cloud/reference/byoc.md

@@ -5,6 +5,6 @@ title: 'Choose a Low Cardinality Partitioning Key'
 description: 'Use a low cardinality partitioning key or avoid using any partitioning key for your table.'
 ---
 
-import Content from '@site/docs/best-practices/partionning_keys.md';
+import Content from '@site/docs/best-practices/partitioning_keys.mdx';
 
 <Content />

docs/guides/best-practices/partitioningkey.md

@@ -1494,7 +1494,7 @@ A compromise between fastest retrieval and optimal data compression is to use a
 
 ### A concrete example {#a-concrete-example}
 
-One concrete example is a the plaintext paste service https://pastila.nl that Alexey Milovidov developed and [blogged about](https://clickhouse.com/blog/building-a-paste-service-with-clickhouse/).
+One concrete example is a the plaintext paste service [https://pastila.nl](https://pastila.nl) that Alexey Milovidov developed and [blogged about](https://clickhouse.com/blog/building-a-paste-service-with-clickhouse/).
 
 On every change to the text-area, the data is saved automatically into a ClickHouse table row (one row per change).
 

docs/guides/best-practices/sparse-primary-indexes.md

@@ -14,6 +14,7 @@ import S3svg from '@site/static/images/integrations/logos/amazon_s3_logo.svg';
 import Amazonkinesis from '@site/static/images/integrations/logos/amazon_kinesis_logo.svg';
 import Gcssvg from '@site/static/images/integrations/logos/gcs.svg';
 import DOsvg from '@site/static/images/integrations/logos/digitalocean.svg';
+import ABSsvg from '@site/static/images/integrations/logos/azureblobstorage.svg';
 import Postgressvg from '@site/static/images/integrations/logos/postgresql.svg';
 import Mysqlsvg from '@site/static/images/integrations/logos/mysql.svg';
 import redpanda_logo from '@site/static/images/integrations/logos/logo_redpanda.png';
@@ -42,7 +43,7 @@ import Image from '@theme/IdealImage';
 | Amazon S3            | <S3svg class="image" alt="Amazon S3 logo" style={{width: '3rem', height: 'auto'}}/>              |Object Storage| Stable           | Configure ClickPipes to ingest large volumes of data from object storage.                            |
 | Google Cloud Storage | <Gcssvg class="image" alt="Google Cloud Storage logo" style={{width: '3rem', height: 'auto'}}/>  |Object Storage| Stable           | Configure ClickPipes to ingest large volumes of data from object storage.                            |
 | DigitalOcean Spaces | <DOsvg class="image" alt="Digital Ocean logo" style={{width: '3rem', height: 'auto'}}/> | Object Storage | Stable | Configure ClickPipes to ingest large volumes of data from object storage.
-
+| Azure Blob Storage | <ABSsvg class="image" alt="Azure Blob Storage logo" style={{width: '3rem', height: 'auto'}}/> | Object Storage | Private Beta | Configure ClickPipes to ingest large volumes of data from object storage.
 | Amazon Kinesis       | <Amazonkinesis class="image" alt="Amazon Kenesis logo" style={{width: '3rem', height: 'auto'}}/> |Streaming| Stable           | Configure ClickPipes and start ingesting streaming data from Amazon Kinesis into ClickHouse cloud.   |
 | Postgres             | <Postgressvg class="image" alt="Postgres logo" style={{width: '3rem', height: 'auto'}}/>         |DBMS| Public Beta      | Configure ClickPipes and start ingesting data from Postgres into ClickHouse Cloud.                   |
 | MySQL                | <Mysqlsvg class="image" alt="MySQL logo" style={{width: '3rem', height: 'auto'}}/>               |DBMS| Private Beta | Configure ClickPipes and start ingesting data from MySQL into ClickHouse Cloud.                      |

docs/integrations/data-ingestion/clickpipes/index.md

@@ -123,7 +123,8 @@ The supported formats are:
 
 ### Supported Data Types {#supported-data-types}
 
-The following ClickHouse data types are currently supported in ClickPipes:
+#### Standard types support {#standard-types-support}
+The following standard ClickHouse data types are currently supported in ClickPipes:
 
 - Base numeric types - \[U\]Int8/16/32/64 and Float32/64
 - Large integer types - \[U\]Int128/256
@@ -141,6 +142,28 @@ The following ClickHouse data types are currently supported in ClickPipes:
 - Map with keys and values using any of the above types (including Nullables)
 - Tuple and Array with elements using any of the above types (including Nullables, one level depth only)
 
+#### Variant type support (experimental) {#variant-type-support}
+Variant type support is automatic if your Cloud service is running ClickHouse 25.3 or later.  Otherwise, you will
+have to submit a support ticket to enable it on your service.
+
+ClickPipes supports the Variant type in the following circumstances:
+- Avro Unions.  If your Avro schema contains a union with multiple non-null types, ClickPipes will infer the
+appropriate variant type.  Variant types are not otherwise supported for Avro data.
+- JSON fields.  You can manually specify a Variant type (such as `Variant(String, Int64, DateTime)`) for any JSON field
+in the source data stream.  Because of the way ClickPipes determines the correct variant subtype to use, only one integer or datetime
+type can be used in the Variant definition - for example, `Variant(Int64, UInt32)` is not supported.
+
+#### JSON type support (experimental) {#json-type-support}
+JSON type support is automatic if your Cloud service is running ClickHouse 25.3 or later.  Otherwise, you will
+have to submit a support ticket to enable it on your service.
+
+ClickPipes support the JSON type in the following circumstances:
+- Avro Record types can always be assigned to a JSON column.
+- Avro String and Bytes types can be assigned to a JSON column if the column actually holds JSON String objects.
+- JSON fields that are always a JSON object can be assigned to a JSON destination column.
+
+Note that you will have to manually change the destination column to the desired JSON type, including any fixed or skipped paths.
+
 ### Avro {#avro}
 #### Supported Avro Data Types {#supported-avro-data-types}
 

docs/integrations/data-ingestion/clickpipes/kafka.md

@@ -91,6 +91,7 @@ The supported formats are:
 
 ## Supported Data Types {#supported-data-types}
 
+### Standard types support {#standard-types-support}
 The following ClickHouse data types are currently supported in ClickPipes:
 
 - Base numeric types - \[U\]Int8/16/32/64 and Float32/64
@@ -108,6 +109,21 @@ The following ClickHouse data types are currently supported in ClickPipes:
 - all ClickHouse LowCardinality types
 - Map with keys and values using any of the above types (including Nullables)
 - Tuple and Array with elements using any of the above types (including Nullables, one level depth only)
+- 
+### Variant type support (experimental) {#variant-type-support}
+Variant type support is automatic if your Cloud service is running ClickHouse 25.3 or later.  Otherwise, you will
+have to submit a support ticket to enable it on your service.
+
+You can manually specify a Variant type (such as `Variant(String, Int64, DateTime)`) for any JSON field
+in the source data stream.  Because of the way ClickPipes determines the correct variant subtype to use, only one integer or datetime
+type can be used in the Variant definition - for example, `Variant(Int64, UInt32)` is not supported.
+
+### JSON type support (experimental) {#json-type-support}
+JSON type support is automatic if your Cloud service is running ClickHouse 25.3 or later.  Otherwise, you will
+have to submit a support ticket to enable it on your service.
+
+JSON fields that are always a JSON object can be assigned to a JSON destination column.  You will have to manually change the destination
+column to the desired JSON type, including any fixed or skipped paths. 
 
 ## Kinesis Virtual Columns {#kinesis-virtual-columns}