Skip to content

Commit accd70a

Browse files
committed
two-schema-registry-articles
1 parent 5d65bd2 commit accd70a

File tree

2 files changed

+37
-39
lines changed

2 files changed

+37
-39
lines changed

articles/event-hubs/schema-registry-client-side-enforcement.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Client-side Schema Enforcement - Schema Registry
2+
title: Client-Side Schema Enforcement - Schema Registry
33
description: This article provides information on using schemas in a schema registry when publishing or consuming events from Azure Event Hubs.
44
ms.topic: conceptual
55
ms.date: 04/26/2023
@@ -11,9 +11,9 @@ ms.author: spelluru
1111

1212
Client-side schema enforcement ensures that data is validated on the client side. The producer application sends the data and the consumer application receives it. That data is validated against schemas defined in the schema registry on the client side rather than the broker/server side.
1313

14-
This flow is illustrated as shown:
14+
This diagram illustrates the flow:
1515

16-
:::image type="content" source="./media/schema-registry-overview/information-flow.svg" alt-text="A diagram outlines the schema registry information flow." border="false":::
16+
:::image type="content" source="./media/schema-registry-overview/information-flow.svg" alt-text="Diagram that shows the schema registry information flow." border="false":::
1717

1818
> [!NOTE]
1919
> The diagram showcases the information flow when event producers and consumers use a schema registry with the Kafka protocol and Avro schema. Other protocols and schema formats work in a similar way.
@@ -25,7 +25,7 @@ This flow is illustrated as shown:
2525

2626
1. The serializer looks for the schema in the schema registry to serialize event data. If it finds the schema, then the corresponding schema ID is returned. You can configure the producer application to automatically register the schema with the schema registry if it doesn't exist.
2727

28-
1. The serializer prepends the schema ID to the serialized data that is published to the Event Hubs.
28+
1. The serializer prepends the schema ID to the serialized data that's published to the event hub.
2929

3030
### Consumer
3131

articles/event-hubs/schema-registry-concepts.md

Lines changed: 33 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -7,37 +7,37 @@ author: spelluru
77
ms.author: spelluru
88
---
99

10-
# Schema Registry in Azure Event Hubs
10+
# Schema registry concepts in Azure Event Hubs
1111

1212
Schema registries in Azure Event Hubs have many benefits. They help maintain data consistency, simplify schema evolution, enhance interoperability, and reduce development effort in loosely coupled and event-streaming workflows. Large distributed organizations that employ a centralized repository for schemas can use schema registries to achieve highly reliable data processing and governance with little operational overhead.
1313

14-
Azure Event Hubs schema registries fulfill many roles in schema-driven, event-streaming scenarios:
14+
Azure Event Hubs schema registries fulfill many roles in schema-driven event-streaming scenarios:
1515

16-
* They provide a repository where multiple schemas can be registered, managed, and evolved
17-
* They manage schema evolution with multiple compatibility rules
18-
* They perform data validation for all schematized data
19-
* They provide client-side libraries (serializers and deserializers) for producers and consumers
20-
* They improve network throughput efficiency by passing the schema ID instead of the schema definition for every payload
16+
* They provide a repository where multiple schemas can be registered, managed, and evolved.
17+
* They manage schema evolution with multiple compatibility rules.
18+
* They perform data validation for all schematized data.
19+
* They provide client-side libraries (serializers and deserializers) for producers and consumers.
20+
* They improve network throughput efficiency by passing the schema ID instead of the schema definition for every payload.
2121

2222
Azure Event Hubs schema registries are supported on Standard, Premium, and Dedicated tiers.
2323

2424
## Schema registry components
2525

26-
A schema registry lives in the context of the Event Hubs namespace, but can also be used with other message or events brokers, including Azure messaging services. It comprises multiple schema groups, which act as a logical grouping of schemas and can be managed independent of other schema groups.
26+
A schema registry lives in the context of the Event Hubs namespace but can also be used with other message or event brokers, including Azure messaging services. It comprises multiple schema groups, which act as a logical grouping of schemas and can be managed independently of other schema groups.
2727

28-
:::image type="content" source="./media/schema-registry-overview/elements.png" alt-text="A diagram shows the components of a schema registry in Azure Event Hubs." border="false":::
28+
:::image type="content" source="./media/schema-registry-overview/elements.png" alt-text="Diagram that shows the components of a schema registry in Azure Event Hubs." border="false":::
2929

3030
### Schemas
3131

32-
In any loosely coupled system, multiple applications communicate, primarily through data. Schemas define the structure of the data in a declarative way. As a result, the contract between producer and consumer applications is well-defined, ensuring reliable processing at scale.
32+
In any loosely coupled system, multiple applications communicate, primarily through data. Schemas define the structure of the data in a declarative way. As a result, the contract between producer and consumer applications is well defined, ensuring reliable processing at scale.
3333

3434
A schema definition includes:
3535

36-
* Fields: Individual data elements like name, book title, or address
37-
* Data types: The type of data that can be stored, like string, date-time, or array
38-
* Structure: How the fields are organized, like nested structures or arrays
36+
* Fields: Individual data elements like name, book title, or address.
37+
* Data types: The type of data that can be stored, like string, date-time, or array.
38+
* Structure: How the fields are organized, like nested structures or arrays.
3939

40-
Schemas define the contract between producers and consumers. A schema defined in an Event Hubs schema registry helps manage the contract outside of event data, which removes the payload overhead.
40+
Schemas define the contract between producers and consumers. A schema defined in an Event Hubs schema registry helps manage the contract outside event data, which removes the payload overhead.
4141

4242
#### Schema formats
4343

@@ -54,31 +54,29 @@ To learn more about using Avro schema format with an Event Hubs schema registry,
5454

5555
##### JSON schema
5656

57-
[JSON (JavaScript Object Notation) schema](https://json-schema.org/) is a standardized way of defining the structure and data types of the events. JSON schema enables the confident and reliable use of the JSON data format in event streaming.
57+
A [JSON (JavaScript Object Notation) schema](https://json-schema.org/) is a standardized way of defining the structure and data types of the events. A JSON schema enables the confident and reliable use of the JSON data format in event streaming.
5858

59-
To learn more about using the JSON schema format with an Event Hubs schema registry, see:
60-
61-
* [How to use a schema registry with Kafka and JSON schema](schema-registry-json-schema-kafka.md)
59+
To learn more about using the JSON schema format with an Event Hubs schema registry, see [How to use a schema registry with Kafka and JSON schema](schema-registry-json-schema-kafka.md).
6260

6361
##### Protocol Buffers
6462

65-
[Protocol Buffers (Protobuf)](https://protobuf.dev/) is a language-neutral, platform-neutral, extensible mechanism for serializing structured data. It's used for efficiently defining data structures and serializing them into a compact, binary format.
63+
[Protocol Buffers (Protobuf)](https://protobuf.dev/) is a language-neutral, platform-neutral, extensible mechanism for serializing structured data. It's used for efficiently defining data structures and serializing them into a compact binary format.
6664

6765
### Schema groups
6866

6967
Schema groups are logical groups of similar schemas that are organized according to your business criteria. A schema group holds:
7068

71-
* Multiple schema definitions
72-
* Multiple versions of a specific schema
73-
* Metadata regarding the schema type and compatibility for all schemas in the group
69+
* Multiple schema definitions.
70+
* Multiple versions of a specific schema.
71+
* Metadata regarding the schema type and compatibility for all schemas in the group.
7472

75-
You can think of a schema group as a subset of the schema registry that aligns with a particular application or organizational unit, with a separate authorization model. This extra security boundary ensures that metadata and trade secrets aren't leaked in the shared services model. It also allows application owners to manage schemas independent of other applications that share the same namespace.
73+
You can think of a schema group as a subset of the schema registry that aligns with a particular application or organizational unit, with a separate authorization model. This extra security boundary helps ensure that metadata and trade secrets aren't leaked in the shared services model. It also allows application owners to manage schemas independently of other applications that share the same namespace.
7674

7775
## Schema evolution
7876

79-
Schemas need to evolve with the business requirement of producers and consumers. Azure schema registries support schema evolution by introducing compatibility modes at the schema group level. When you create a schema group, you can specify the compatibility mode of the schemas that you include in that schema group. When you update a schema, the change needs to comply with the assigned compatibility mode in order for it to create a new version of the schema.
77+
Schemas need to evolve with the business requirement of producers and consumers. Azure schema registries support schema evolution by introducing compatibility modes at the schema group level. When you create a schema group, you can specify the compatibility mode of the schemas that you include in that schema group. When you update a schema, the change needs to comply with the assigned compatibility mode so that it can create a new version of the schema.
8078

81-
Schema evolution is only supported for Avro schema format only.
79+
Schema evolution is supported for Avro schema format only.
8280

8381
Azure Event Hubs schema registries are supported in the following compatibility modes.
8482

@@ -102,15 +100,15 @@ When the ``None`` compatibility mode is used, the schema registry doesn't do any
102100

103101
## Client SDKs
104102

105-
You can use one of the following libraries to include an Avro serializer. You can use Avro serializers to serialize and deserialize payloads containing schema registry schema identifiers and Avro-encoded data:
103+
You can use one of the following libraries to include an Avro serializer. You can use Avro serializers to serialize and deserialize payloads that contain schema registry schema identifiers and Avro-encoded data:
106104

107105
* **.NET**: [Microsoft.Azure.Data.SchemaRegistry.ApacheAvro](https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/schemaregistry/Microsoft.Azure.Data.SchemaRegistry.ApacheAvro)
108106
* **Java**: [azure-data-schemaregistry-avro](https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/schemaregistry/azure-data-schemaregistry-apacheavro)
109107
* **Python**: [azure-schemaregistry-avroserializer](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/schemaregistry/azure-schemaregistry-avroencoder/)
110108
* **JavaScript**: [@azure/schema-registry-avro](https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/schemaregistry/schema-registry-avro)
111-
* **Apache Kafka**: Run [Kafka-integrated Avro](https://github.com/Azure/azure-schema-registry-for-kafka/) serializers and deserializers backed by Azure Schema Registry. The Java client's Apache Kafka® client serializer for Azure schema registries can be used in any Apache Kafka scenario and with any Apache Kafka-based deployment or cloud service.
112-
* **Azure CLI**: For an example of adding a schema to a schema group by using CLI, see [Adding a schema to a schema group using CLI](https://github.com/Azure/azure-event-hubs/tree/master/samples/Management/CLI/AddschematoSchemaGroups).
113-
* **PowerShell**: For an example of adding a schema to a schema group by using PowerShell, see [Adding a schema to a schema group using PowerShell](https://github.com/Azure/azure-event-hubs/tree/master/samples/Management/PowerShell/AddingSchematoSchemagroups).
109+
* **Apache Kafka**: Run [Kafka-integrated Avro](https://github.com/Azure/azure-schema-registry-for-kafka/) serializers and deserializers backed by Azure schema registries. The Java client's Apache Kafka® client serializer for Azure schema registries can be used in any Apache Kafka scenario and with any Apache Kafka-based deployment or cloud service.
110+
* **Azure CLI**: For an example of adding a schema to a schema group by using the Azure CLI, see [Adding a schema to a schema group by using the Azure CLI](https://github.com/Azure/azure-event-hubs/tree/master/samples/Management/CLI/AddschematoSchemaGroups).
111+
* **PowerShell**: For an example of adding a schema to a schema group by using PowerShell, see [Adding a schema to a schema group by using PowerShell](https://github.com/Azure/azure-event-hubs/tree/master/samples/Management/PowerShell/AddingSchematoSchemagroups).
114112

115113
## Limits
116114

@@ -127,17 +125,17 @@ To access a schema registry programmatically, follow these steps:
127125
| ---- | ----------- |
128126
| Owner | Read, write, and delete schema registry groups and schemas |
129127
| Contributor | Read, write, and delete schema registry groups and schemas |
130-
| [Schema registry reader](../role-based-access-control/built-in-roles.md#schema-registry-reader-preview) | Read and list schema registry groups and schemas |
131-
| [Schema registry contributor](../role-based-access-control/built-in-roles.md#schema-registry-reader-preview) | Read, write, and delete schema registry groups and schemas |
128+
| [Schema Registry Reader](../role-based-access-control/built-in-roles.md#schema-registry-reader-preview) | Read and list schema registry groups and schemas |
129+
| [Schema Registry Contributor](../role-based-access-control/built-in-roles.md#schema-registry-reader-preview) | Read, write, and delete schema registry groups and schemas |
132130

133-
To learn how to create and register an application using the Azure portal, see [Register an application with Microsoft Entra ID](../active-directory/develop/quickstart-register-app.md). You need the client ID (application ID), tenant ID, and the secret to use in the code.
131+
To learn how to create and register an application by using the Azure portal, see [Register an application with Microsoft Entra ID](../active-directory/develop/quickstart-register-app.md). You need the client ID (application ID), the tenant ID, and the secret to use in the code.
134132

135-
## Next steps
133+
## Related content
136134

137-
* To learn how to create a schema registry by using the Azure portal, see [Create an Event Hubs schema registry using the Azure portal](create-schema-registry.md).
135+
* To learn how to create a schema registry by using the Azure portal, see [Create an Event Hubs schema registry by using the Azure portal](create-schema-registry.md).
138136
* See the following **schema registry Avro client library** samples.
139137
* [.NET](https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/schemaregistry/Microsoft.Azure.Data.SchemaRegistry.ApacheAvro/tests/Samples)
140138
* [Java](https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/schemaregistry/azure-data-schemaregistry-apacheavro/src/samples)
141139
* [JavaScript](https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/schemaregistry/schema-registry-avro/samples)
142140
* [Python](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/schemaregistry/azure-schemaregistry-avroencoder/samples)
143-
* [Kafka Avro Integration for Azure Schema Registry](https://github.com/Azure/azure-schema-registry-for-kafka/tree/master/csharp/avro/samples)
141+
* [Kafka Avro Integration for Azure schema registries](https://github.com/Azure/azure-schema-registry-for-kafka/tree/master/csharp/avro/samples)

0 commit comments

Comments
 (0)