22name : Schema Validation and Enforcement
33---
44
5- ### Schema Validation and Enforcement
5+ ## Schema Validation and Enforcement
66
7- #### Why
7+ ### Why do we need Schema Validation and Enforcement?
88
99In Apache Kafka, producing applications transfer messages to consuming applications. In order for messages to be transferred successfully
1010producers and consumers need to agree on the format of the message being transferred. If there is a mismatch between the format sent
@@ -20,7 +20,7 @@ If there is only a small number of applications, perhaps all managed by the same
2020informal agreements about what message format will be used on which topic. However, as the use of Kafka grows within the organization,
2121the amount of effort required to keep all the formats in agreement grows. Mistakes, and system downtime, become inevitable.
2222
23- ##### Schemas to the rescue
23+ #### Schemas to the rescue
2424
2525To overcome the problem, the Kafka ** client** ecosystem supports _ schemas_ . Schemas provide a programmatic description
2626of the message format.
@@ -33,7 +33,7 @@ inflexible as it means all applications need to be redeployed every time a schem
3333as the number of applications in the system grows. To counter this challenge organizations often choose to deploy a
3434* Schema Registry* .
3535
36- ##### Schema Registry
36+ #### Schema Registry
3737
3838The role of the ** Schema Registry** is to maintain a library of schemas that are in-use within an organisation.
3939
@@ -46,7 +46,7 @@ the registry. This allows the consumer to decode the message.
4646| :-----------------------------------------------------------------:|
4747| * Kafka applications making use of a Schema Registry* |
4848
49- #### So, what's the problem?
49+ ### So, what's the problem?
5050
5151If schemas are used correctly across the entire producing and consuming application estate, the potential for a
5252poison message to be introduced into the system is greatly diminished. However, the issue is that there's significant
@@ -57,7 +57,7 @@ In Kafka, **brokers** have no knowledge of format of the messages (to the broker
5757have no knowledge of the schema. The brokers cannot help us enforce that producing applications use schemas and use them
5858correctly.
5959
60- #### Kroxylicious Record Validation
60+ ### Kroxylicious Record Validation
6161
6262The Kroxylicious Record Validation Filter provides a solution to the problem.
6363
@@ -68,7 +68,7 @@ even if producing applications are misconfigured.
6868
6969The filter currently supports two modes of operation:
7070
71- 1 . Schema validation[ ^ 1 ] validates the content of the record against a schema. Use this for topics which have an entry in
71+ 1 . Schema validation[ ^ 3 ] validates the content of the record against a schema. Use this for topics which have an entry in
7272 the Schema Registry.
73732 . SyntacticallyCorrectJson validation ensures the producer is producing messages that contain syntactically valid JSON.
7474 Use for topics which do not have registered schemas.
0 commit comments