You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A schema registry can be configured by when setting up a ClickPipe. This can be configured in one of three ways:
96
-
96
+
97
97
1. Providing the root schema registry URL (e.g. `https://registry.example.com`). **This is the preferred method.**
98
98
2. Providing a complete path to the schema id (e.g. `https://registry.example.com/schemas/ids/1000`)
99
99
3. Providing a complete path to the schema subject (e.g. `https://registry.example.com/subjects/events`)
100
100
- Optionally, a specific version can be referenced by appending `/versions/[version]` to the url (otherwise ClickPipes will retrieve the latest version).
101
101
102
-
### How it works
102
+
### How it works {#how-schema-registries-work}
103
103
ClickPipes dynamically retrieves and applies the Avro schema from the configured Schema Registry.
104
104
- If there's a schema id embedded in the message, it will use that to retrieve the schema.
105
105
- If there's no schema id embedded in the message, it will use the schema id or subject name specified in the ClickPipe configuration to retrieve the schema.
106
106
- If the message is written without an embedded schema id, and no schema id or subject name is specified in the ClickPipe configuration, then the schema will not be retrieved and the message will be skipped with a `SOURCE_SCHEMA_ERROR` logged in the ClickPipes errors table.
107
107
- If the message does not conform to the schema, then the message will be skipped with a `DATA_PARSING_ERROR` logged in the ClickPipes errors table.
108
108
109
-
### Schema mapping
109
+
### Schema mapping {#schema-mapping}
110
110
The following rules are applied to the mapping between the retrieved Avro schema and the ClickHouse destination table:
111
111
112
112
- If the Avro schema contains a field that is not included in the ClickHouse destination mapping, that field is ignored.
0 commit comments