Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog/release-notes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ Support for certain earlier versions will end following the release of v2.8. Ple
- Supports TCP keepalive configuration for PostgreSQL Rust sink and JDBC sink connections to prevent silent connection drops. [#24388](https://github.com/risingwavelabs/risingwave/pull/24388)
- Iceberg: Enforces merge-on-read for append-only tables to ensure data consistency. [#24487](https://github.com/risingwavelabs/risingwave/pull/24487)
- Redshift and Snowflake v2 sinks: Adds new parameters `writer_target_interval_seconds` (alias: `schedule_seconds`) to configure the interval for writing data to the target, and `write_intermediate_interval_seconds` to configure the interval for writing data to intermediate storage. [#23086](https://github.com/risingwavelabs/risingwave/pull/23086)
- Adds a new optional `schema.registry.ca_pem_path` parameter for Confluent Schema Registry connections. Set it to a file path to load a custom root CA certificate, or to `'ignore'` to skip TLS certificate verification (not recommended for production). Supports secret management via `SECRET ... AS FILE`. [#17826](https://github.com/risingwavelabs/risingwave/pull/17826)

## Cluster configuration changes

Expand Down
17 changes: 14 additions & 3 deletions ingestion/formats-and-encoding-options.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -273,19 +273,31 @@ Parquet sources require case-sensitive column names. However, PostgreSQL convert
| FORMAT | ENCODE | Parameter | Description | Required |
|:-------|:-------|:----------|:------------|:---------|
| PLAIN | AVRO | `schema.registry` | URL(s) of the schema registry. | Yes |
| PLAIN | AVRO | `schema.registry.username` | The username for schema registry. Optional, but if set it must be used together with `schema.registry.password`. | No |
| PLAIN | AVRO | `schema.registry.password` | The password for schema registry. Optional, but if set it must be used together with `schema.registry.username`. | No |
| PLAIN | AVRO | `schema.registry.ca_pem_path` | Path to a custom root CA certificate file (PEM) for the schema registry TLS connection, or `'ignore'` to skip certificate verification (not recommended for production). Supports `SECRET ... AS FILE`. | No |
| UPSERT | AVRO | `schema.registry` | URL(s) of the schema registry. | Yes |
| UPSERT | AVRO | `schema.registry.username` | The username for schema registry. Required if `schema.registry.password` is set. | No |
| UPSERT | AVRO | `schema.registry.password` | The password for schema registry. Required if `schema.registry.username` is set. | No |
| UPSERT | AVRO | `schema.registry.ca_pem_path` | Path to a custom root CA certificate file (PEM) for the schema registry TLS connection, or `'ignore'` to skip certificate verification (not recommended for production). Supports `SECRET ... AS FILE`. | No |
| DEBEZIUM | AVRO | `schema.registry` | URL(s) of the schema registry. | Yes |
| DEBEZIUM | AVRO | `schema.registry.username` | The username for schema registry. Required if `schema.registry.password` is set; configure together with `schema.registry.password`. | No |
| DEBEZIUM | AVRO | `schema.registry.password` | The password for schema registry. Required if `schema.registry.username` is set; configure together with `schema.registry.username`. | No |
| DEBEZIUM | AVRO | `schema.registry.ca_pem_path` | Path to a custom root CA certificate file (PEM) for the schema registry TLS connection, or `'ignore'` to skip certificate verification (not recommended for production). Supports `SECRET ... AS FILE`. | No |
| DEBEZIUM | AVRO | `message` | The main message name. | Yes |
| DEBEZIUM | AVRO | `ignore_key` | Whether to ignore the key (default: false). | No |
| PLAIN | JSON | `schema.registry` | URL(s) of the schema registry (optional). | No |
| PLAIN | JSON | `schema.registry.username` | The username for schema registry (optional). | No |
| PLAIN | JSON | `schema.registry.password` | The password for schema registry (optional). | No |
| PLAIN | JSON | `schema.registry.ca_pem_path` | Path to a custom root CA certificate file (PEM) for the schema registry TLS connection, or `'ignore'` to skip certificate verification (not recommended for production). Supports `SECRET ... AS FILE`. | No |
| UPSERT | JSON | `schema.registry` | URL(s) of the schema registry (optional). | No |
| UPSERT | JSON | `schema.registry.username` | The username for schema registry (optional). | No |
| UPSERT | JSON | `schema.registry.password` | The password for schema registry (optional). | No |
| UPSERT | JSON | `schema.registry.ca_pem_path` | Path to a custom root CA certificate file (PEM) for the schema registry TLS connection, or `'ignore'` to skip certificate verification (not recommended for production). Supports `SECRET ... AS FILE`. | No |
| PLAIN | PROTOBUF | `message` | Fully qualified name of the Protobuf message. | Yes |
| PLAIN | PROTOBUF | `schema.location` | URL of the schema file (or schema.registry). | Yes |
| PLAIN | PROTOBUF | `schema.registry` | URL(s) of the schema registry (alternative to schema.location). | Yes |
| PLAIN | PROTOBUF | `schema.location` | URL of the schema file (alternative to `schema.registry`). | Yes (one of) |
| PLAIN | PROTOBUF | `schema.registry` | URL(s) of the schema registry (alternative to `schema.location`). | Yes (one of) |
| PLAIN | PROTOBUF | `schema.registry.ca_pem_path` | Path to a custom root CA certificate file (PEM) for the schema registry TLS connection, or `'ignore'` to skip certificate verification (not recommended for production). Supports `SECRET ... AS FILE`. | No |
| DEBEZIUM | JSON | `ignore_key` | Whether to ignore the key (default: false). | No |
| PLAIN | CSV | `delimiter` | Delimiter character. | Yes |
| PLAIN | CSV | `without_header` | Whether the CSV data has a header row (default: false). | No |
Expand Down Expand Up @@ -318,4 +330,3 @@ You cannot set this parameter for these three combinations:
- `FORMAT DEBEZIUM_MONGO ENCODE JSON`
- `FORMAT MAXWELL ENCODE JSON`
- `FORMAT CANAL ENCODE JSON`

1 change: 1 addition & 0 deletions ingestion/sources/kafka-config.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ These parameters are used with `FORMAT` and `ENCODE` to specify how RisingWave s
| *schema.registry* | Confluent Schema Registry URL. Example: `http://127.0.0.1:8081`. <ul><li>For Avro data, you must specify a Confluent Schema Registry or an AWS Glue Schema Registry.</li><li>For Protobuf data, you must specify either a schema location or a Confluent Schema Registry but not both.</li></ul> |
| *schema.registry.username* | **Conditional**. User name for the schema registry. It must be specified with `schema.registry.password`. |
| *schema.registry.password* | **Conditional**. Password for the schema registry. It must be specified with `schema.registry.username`. |
| *schema.registry.ca_pem_path* | **Optional**. Path to a custom root CA certificate file (PEM format) for the schema registry HTTPS connection. Set to `'ignore'` to disable TLS certificate verification (not recommended for production). Supports secret management via `SECRET ... AS FILE`. |
| *access_key* | Required if loading descriptors from S3. The access key ID of AWS. |
| *secret_key* | Required if loading descriptors from S3. The secret access key of AWS. |
| *region* | Required if loading descriptors from S3. The AWS service region. |
Expand Down
Loading