Skip to content

Commit 8d97ceb

Browse files
Copilotswathipil
Copilot
and
swathipil
committed
Add CLIENT_DEVELOPER.md for azure-schemaregistry with comprehensive developer guidance
Co-authored-by: swathipil <[email protected]>
1 parent fade9db commit 8d97ceb

File tree

1 file changed

+247
-0
lines changed

1 file changed

+247
-0
lines changed

‎sdk/schemaregistry/azure-schemaregistry/CLIENT_DEVELOPER.md‎

Lines changed: 247 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,247 @@
1+
# Azure Schema Registry Client Library Developer Guide
2+
3+
This guide is intended for developers contributing to the Azure Schema Registry Python client library. It provides information on setting up your development environment, regenerating the client from TypeSpec, understanding the code structure, running tests, and contributing to the codebase.
4+
5+
## Setting Up Development Environment
6+
7+
### Prerequisites
8+
9+
- Python version [supported by the client library](https://github.com/Azure/azure-sdk-for-python/wiki/Azure-SDKs-Python-version-support-policy)
10+
- Git
11+
- pip and setuptools
12+
- Node.js and npm (for TypeSpec client generation)
13+
- Azure subscription to create Schema Registry resources
14+
15+
### Setup
16+
17+
1. Clone the repository:
18+
```bash
19+
git clone https://github.com/Azure/azure-sdk-for-python.git
20+
cd azure-sdk-for-python/sdk/schemaregistry/azure-schemaregistry
21+
```
22+
23+
2. Create a virtual environment:
24+
```bash
25+
# Linux/macOS
26+
python -m venv .venv && source .venv/bin/activate && pip install -r dev_requirements.txt
27+
28+
# Windows PowerShell
29+
python -m venv .venv; .\.venv\Scripts\Activate.ps1; pip install -r dev_requirements.txt
30+
```
31+
32+
3. Install the package in development mode:
33+
```bash
34+
pip install -e .
35+
```
36+
37+
## Regenerating the Client from TypeSpec
38+
39+
The Azure Schema Registry client is generated from TypeSpec definitions. The TypeSpec files are maintained in the [Azure REST API Specs repository](https://github.com/Azure/azure-rest-api-specs/tree/main/specification/schemaregistry/SchemaRegistry).
40+
41+
### Prerequisites for Code Generation
42+
43+
1. Install the TypeSpec client generator:
44+
```bash
45+
npm install -g @azure-tools/typespec-client-generator-cli
46+
```
47+
48+
2. Ensure you have the latest TypeSpec definitions by checking the [specification directory](https://github.com/Azure/azure-rest-api-specs/tree/main/specification/schemaregistry/SchemaRegistry).
49+
50+
### Regeneration Process
51+
52+
The client regeneration is controlled by the `tsp-location.yaml` file in the root of this package directory. This file specifies:
53+
- The TypeSpec specification repository (`Azure/azure-rest-api-specs`)
54+
- The specific commit hash to use
55+
- The directory path (`specification/schemaregistry/SchemaRegistry`)
56+
57+
To regenerate the client:
58+
59+
1. Update the commit hash in `tsp-location.yaml` if needed to point to the latest TypeSpec definitions
60+
2. Run the TypeSpec client generation tool:
61+
```bash
62+
tsp-client update
63+
```
64+
65+
3. Review the generated changes and ensure handwritten code in `_patch.py` files remains intact
66+
67+
### tsp-client Usage
68+
69+
The `tsp-client` tool will:
70+
- Download the TypeSpec definitions from the specified commit
71+
- Generate Python client code based on the TypeSpec specifications
72+
- Preserve handwritten customizations in `_patch.py` files
73+
- Update generated models, operations, and client classes
74+
75+
For more information on tsp-client usage, see the [tsp-client documentation](https://github.com/Azure/azure-sdk-tools/tree/main/tools/tsp-client/README.md).
76+
77+
## Code Structure
78+
79+
The Azure Schema Registry client library follows a hybrid approach with both generated and handwritten code:
80+
81+
### Generated Code
82+
83+
The following files are automatically generated from TypeSpec and should **not** be manually edited:
84+
- `_client.py` - Main client implementation
85+
- `_configuration.py` - Client configuration
86+
- `_model_base.py` - Base classes for models
87+
- `_operations/` - Generated operation classes
88+
- `_serialization.py` - Serialization utilities
89+
- `_vendor.py` - Vendored dependencies
90+
- `models/*.py` (except `_patch.py`) - Data models
91+
- `aio/_client.py` - Async client implementation
92+
- `aio/_configuration.py` - Async client configuration
93+
94+
### Handwritten Code
95+
96+
The following files contain handwritten customizations and extensions:
97+
98+
#### `_patch.py` Files
99+
- `azure/schemaregistry/_patch.py` - Sync client customizations
100+
- `azure/schemaregistry/aio/_patch.py` - Async client customizations
101+
- `azure/schemaregistry/models/_patch.py` - Model customizations
102+
103+
These files allow you to:
104+
- Add custom methods to generated classes
105+
- Override generated method implementations
106+
- Add custom models and enums
107+
- Extend functionality while preserving generated code
108+
109+
#### Encoder Implementation
110+
- `azure/schemaregistry/encoder/jsonencoder/` - Complete handwritten JSON Schema encoder implementation
111+
- `_json_encoder.py` - Main encoder/decoder logic
112+
- `_exceptions.py` - Custom exceptions
113+
- `__init__.py` - Public API
114+
115+
The encoder directory contains entirely handwritten code that provides schema-based encoding and decoding capabilities using the Schema Registry client.
116+
117+
### Code Customization Guidelines
118+
119+
When adding handwritten code:
120+
1. Use `_patch.py` files for extending generated classes
121+
2. Follow the existing patterns for method signatures and documentation
122+
3. Import required types from generated modules
123+
4. Use the `patch_sdk()` function to register customizations
124+
5. Ensure async and sync implementations remain consistent
125+
126+
## Related Libraries
127+
128+
### Azure Schema Registry Avro Encoder
129+
130+
The [azure-schemaregistry-avroencoder](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/schemaregistry/azure-schemaregistry-avroencoder) is an extension library that provides Avro-specific encoding and decoding capabilities. It builds on top of this Schema Registry client to provide:
131+
132+
- Avro schema-based serialization and deserialization
133+
- Integration with Apache Avro
134+
- Optimized performance for Avro payloads
135+
136+
For detailed usage and examples, see the [Avro Encoder README](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/schemaregistry/azure-schemaregistry-avroencoder/README.md).
137+
138+
## Running Tests
139+
140+
### Unit and Integration Tests
141+
142+
1. Set up test resources following the patterns in `conftest.py`
143+
2. Run tests using pytest:
144+
```bash
145+
# Run all tests
146+
pytest tests/
147+
148+
# Run specific test files
149+
pytest tests/test_schema_registry.py
150+
pytest tests/test_json_encoder.py
151+
152+
# Run async tests
153+
pytest tests/async_tests/
154+
```
155+
156+
### Live Tests
157+
158+
Live tests require an Azure Schema Registry namespace. Set the following environment variables:
159+
160+
```bash
161+
# Required for live tests
162+
AZURE_SUBSCRIPTION_ID=<your-subscription-id>
163+
SCHEMAREGISTRY_FULLY_QUALIFIED_NAMESPACE=<your-namespace>.servicebus.windows.net
164+
SCHEMAREGISTRY_GROUP=<your-schema-group>
165+
AZURE_TEST_RUN_LIVE=true
166+
167+
# Authentication (choose one)
168+
AZURE_TEST_USE_CLI_AUTH=true # Use Azure CLI authentication
169+
# OR
170+
AZURE_CLIENT_ID=<service-principal-id>
171+
AZURE_CLIENT_SECRET=<service-principal-secret>
172+
AZURE_TENANT_ID=<tenant-id>
173+
```
174+
175+
Run live tests:
176+
```bash
177+
pytest tests/ --live
178+
```
179+
180+
### Common Test Issues
181+
182+
- **Authentication errors**: Ensure Azure CLI is logged in (`az login`) or service principal credentials are correctly set
183+
- **Resource not found**: Verify that the Schema Registry namespace and group exist
184+
- **Permission errors**: Ensure the authentication principal has appropriate Schema Registry permissions
185+
186+
## Performance Testing
187+
188+
Performance tests are located in `tests/perfstress_tests/` and help measure client library performance under various conditions.
189+
190+
### Setup for Performance Tests
191+
192+
1. Install performance test dependencies:
193+
```bash
194+
pip install -r dev_requirements.txt
195+
pip install -e .
196+
```
197+
198+
2. Set environment variables:
199+
```bash
200+
SCHEMAREGISTRY_FULLY_QUALIFIED_NAMESPACE=<your-namespace>.servicebus.windows.net
201+
SCHEMAREGISTRY_GROUP=<your-schema-group>
202+
```
203+
204+
### Running Performance Tests
205+
206+
1. Navigate to the tests directory:
207+
```bash
208+
cd tests/
209+
```
210+
211+
2. List available performance tests:
212+
```bash
213+
perfstress
214+
```
215+
216+
3. Run specific performance tests:
217+
```bash
218+
# Example: Run schema registration test
219+
perfstress RegisterSchemaTest --duration=30 --parallel=5
220+
221+
# Example: Run schema retrieval test
222+
perfstress GetSchemaByIdTest --duration=30 --parallel=10 --num-schemas=100
223+
```
224+
225+
### Performance Test Options
226+
227+
Common options for all performance tests:
228+
- `--duration=30` - Number of seconds to run operations (default: 10)
229+
- `--iterations=1` - Number of test iterations (default: 1)
230+
- `--parallel=1` - Number of parallel operations (default: 1)
231+
- `--warm-up=5` - Warm-up time in seconds (default: 5)
232+
- `--sync` - Run synchronous tests instead of async (default: async)
233+
234+
Schema Registry specific options:
235+
- `--schema-size=150` - Size of each schema in bytes (default: 150)
236+
- `--num-schemas=10` - Number of schemas to use in tests (default: 10)
237+
238+
For detailed performance testing setup and resource requirements, see the [Performance Tests README](tests/perfstress_tests/README.md).
239+
240+
## Additional Resources
241+
242+
- [Azure Schema Registry Documentation](https://docs.microsoft.com/azure/event-hubs/schema-registry-overview)
243+
- [Azure SDK Design Guidelines](https://azure.github.io/azure-sdk/python_design.html)
244+
- [TypeSpec Documentation](https://typespec.io/)
245+
- [Schema Registry Samples](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/schemaregistry/azure-schemaregistry/samples)
246+
- [Azure Schema Registry REST API Reference](https://docs.microsoft.com/rest/api/schemaregistry/)
247+
- [Event Hubs and Schema Registry Quickstart](https://learn.microsoft.com/azure/event-hubs/create-schema-registry)

0 commit comments

Comments
 (0)