Skip to content

Commit 074c141

Browse files
Copilotswathipil
Copilot
and
swathipil
committed
Initial creation of CLIENT_DEVELOPER.md files from PR #41477
Co-authored-by: swathipil <[email protected]>
1 parent d194344 commit 074c141

File tree

2 files changed

+377
-0
lines changed

2 files changed

+377
-0
lines changed

‎sdk/eventhub/azure-eventhub/CLIENT_DEVELOPER.md‎

Lines changed: 186 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
# Azure Event Hubs Client Library Developer Guide
2+
3+
This guide is intended for developers contributing to the Azure Event Hubs Python client library. It provides information on setting up your development environment, running tests, and contributing to the codebase.
4+
5+
## Setting Up Development Environment
6+
7+
### Prerequisites
8+
9+
- Python version supported by the client library
10+
- Git
11+
- pip and setuptools
12+
- Azure subscription to create Event Hubs resources
13+
14+
### Setup
15+
16+
1. Clone the repository:
17+
```bash
18+
git clone https://github.com/Azure/azure-sdk-for-python.git
19+
cd azure-sdk-for-python/sdk/eventhub/azure-eventhub
20+
```
21+
22+
2. Create a virtual environment:
23+
```bash
24+
# Linux/macOS
25+
python -m venv .venv && source .venv/bin/activate && pip install -r dev_requirements.txt
26+
27+
# Windows PowerShell
28+
python -m venv .venv; .\.venv\Scripts\Activate.ps1; pip install -r dev_requirements.txt
29+
```
30+
31+
3. Install the package in development mode:
32+
```bash
33+
pip install -e .
34+
```
35+
36+
If you encounter import errors when running tests, use the standard installation:
37+
```bash
38+
pip install .
39+
```
40+
Note: You'll need to rerun this command after making changes to the package.
41+
42+
## Running Tests
43+
44+
### Unit Tests
45+
46+
Unit tests don't require any Azure resources:
47+
48+
```bash
49+
# Run all unit tests
50+
pytest tests/unittest
51+
52+
# Run specific unit test
53+
pytest tests/unittest/test_specific_file.py::test_specific_test
54+
```
55+
56+
### Live Tests
57+
58+
Live tests require an Azure Event Hubs namespace. Resources are created dynamically for live tests using fixtures in the `conftest.py` file.
59+
60+
1. Login to Azure:
61+
```bash
62+
az login
63+
```
64+
65+
2. Set required environment variables (or create a `.env` file):
66+
```
67+
EVENTHUB_RESOURCE_GROUP=<your-resource-group>
68+
EVENTHUB_NAMESPACE=<your-namespace> # Only the name, not the full URL
69+
EVENTHUB_NAME=<your-eventhub-name>
70+
```
71+
72+
3. Run tests:
73+
```bash
74+
# Run a specific test to check if your environment is set up correctly
75+
pytest tests/livetest/synctests/test_consumer_client.py::test_receive_partition
76+
77+
# Run all live tests
78+
pytest tests/livetest
79+
80+
# Run with output visible even if tests pass
81+
pytest -s tests/livetest/synctests/test_specific_file.py::test_specific_function
82+
```
83+
84+
### Common Test Issues
85+
86+
- **Authentication errors**: If you see "The access token is invalid" errors, run `az login` to refresh your authentication.
87+
- **Resource Not Found**: Ensure your environment variables are correctly set. For `EVENTHUB_NAMESPACE`, only use the name part (without the `.servicebus.windows.net` suffix).
88+
- **ImportError**: If you see errors like "cannot import name 'EventHubProducerClient' from 'azure.eventhub'", use `pip install .` instead of `pip install -e .`.
89+
- **PytestUnknownMarkWarning**: You can ignore warnings about "Unknown pytest.mark.livetest" as they're related to the Azure SDK test framework.
90+
- **asyncio_default_fixture_loop_scope warnings**: These warnings from pytest-asyncio can be ignored.
91+
92+
## Stress Testing
93+
94+
Stress tests help verify reliability and performance under load. They are designed to test the library under extreme conditions and for extended periods.
95+
96+
### Structure of Stress Test Files
97+
98+
Stress test files are typically organized in the `stress` directory with the following structure:
99+
100+
```
101+
stress/
102+
├── scenarios/ # Contains stress test scenario definitions
103+
├── scripts/ # Support scripts for stress test deployment
104+
│ └── dev_requirement.txt # Dependencies for the stress test
105+
└── templates/ # Kubernetes templates for test deployment
106+
└── testjob.yaml # Test job configuration
107+
```
108+
109+
### Prerequisites
110+
111+
- Docker Desktop
112+
- Kubernetes CLI (kubectl)
113+
- PowerShell
114+
- Helm
115+
116+
### Running Stress Tests
117+
118+
1. Ensure Docker Desktop is running with Kubernetes enabled.
119+
120+
2. Deploy stress tests from the library folder:
121+
```powershell
122+
..\..\..\eng\common\scripts\stress-testing\deploy-stress-tests.ps1 -Namespace <your-namespace>
123+
```
124+
125+
3. Common kubectl commands:
126+
```bash
127+
# Check on pods
128+
kubectl get pods -n <your-namespace>
129+
130+
# Delete namespace
131+
kubectl delete namespace <your-namespace>
132+
133+
# List all active namespaces
134+
helm list --all-namespaces
135+
136+
# Show logs in console
137+
kubectl logs -n <your-namespace> <pod-name>
138+
139+
# For init failure
140+
kubectl logs -n <your-namespace> <pod-name> -c init-azure-deployer
141+
```
142+
143+
4. Testing branch changes:
144+
- To run stress tests against changes in a specific branch, update the `stress/scripts/dev_requirement.txt` file to replace the library with:
145+
```
146+
git+<gh-repo-url>@<branch>#subdirectory=<path/to/sdk>&egg=<package-name>
147+
```
148+
149+
5. Monitoring performance:
150+
- Check the CPU/Memory/errors in the Production Dashboard specified in the Reliability Testing wiki
151+
- Add `--debug_level Debug` in `stress/templates/testjob.yaml` to output debug logs to the File Share (link available in Reliability Testing wiki)
152+
153+
For complete stress testing requirements and setup, refer to the [Reliability Testing documentation](https://dev.azure.com/azure-sdk/internal/_wiki/wikis/internal.wiki/463/Reliability-Testing).
154+
155+
## Performance Testing
156+
157+
The Performance framework helps measure performance metrics for the client library.
158+
159+
### Running Performance Tests Locally
160+
161+
1. Navigate to the performance test directory:
162+
```bash
163+
cd tests/perfstress
164+
```
165+
166+
2. Review the README file for specific test setup and instructions on running tests:
167+
```bash
168+
cat README.md
169+
```
170+
171+
### Running Performance Tests in Pipelines
172+
173+
- Performance test resources are deployed dynamically and specified in the service level directory's `perf-resources.bicep` file
174+
- Default parameters are in the service level directory in `perf.yml`, and baseline and source package versions are specified in `perf-tests.yml`
175+
176+
To run performance tests in a pipeline for a PR:
177+
1. Add a comment to the PR: `/azp run python - eventhub - perf`
178+
2. To run manually with custom parameters:
179+
- Find the perf pipelines in azure-sdk internal DevOps pipelines
180+
- Click Run and set the commit to: `refs/pull/<PR #>/merge`
181+
182+
## Additional Resources
183+
184+
- [Event Hubs Architecture](https://docs.microsoft.com/azure/event-hubs/event-hubs-about)
185+
- [Azure SDK Design Guidelines](https://azure.github.io/azure-sdk/python_design.html)
186+
- [Event Hubs Samples](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/eventhub/azure-eventhub/samples)

‎sdk/servicebus/azure-servicebus/CLIENT_DEVELOPER.md‎

Lines changed: 191 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,191 @@
1+
# Azure Service Bus Client Library Developer Guide
2+
3+
This guide is intended for developers contributing to the Azure Service Bus Python client library. It provides information on setting up your development environment, running tests, and contributing to the codebase.
4+
5+
## Setting Up Development Environment
6+
7+
### Prerequisites
8+
9+
- Python version supported by the client library
10+
- Git
11+
- pip and setuptools
12+
- Azure subscription to create Service Bus resources
13+
14+
### Setup
15+
16+
1. Clone the repository:
17+
```bash
18+
git clone https://github.com/Azure/azure-sdk-for-python.git
19+
cd azure-sdk-for-python/sdk/servicebus/azure-servicebus
20+
```
21+
22+
2. Create a virtual environment:
23+
```bash
24+
# Linux/macOS
25+
python -m venv .venv && source .venv/bin/activate && pip install -r dev_requirements.txt
26+
27+
# Windows PowerShell
28+
python -m venv .venv; .\.venv\Scripts\Activate.ps1; pip install -r dev_requirements.txt
29+
```
30+
31+
3. Install the package in development mode:
32+
```bash
33+
pip install -e .
34+
```
35+
36+
If you encounter import errors when running tests, use the standard installation:
37+
```bash
38+
pip install .
39+
```
40+
Note: You'll need to rerun this command after making changes to the package.
41+
42+
## Running Tests
43+
44+
### Unit Tests
45+
46+
Unit tests don't require any Azure resources:
47+
48+
```bash
49+
# Run all unit tests
50+
pytest tests/unittest
51+
52+
# Run specific unit test
53+
pytest tests/unittest/test_specific_file.py::test_specific_test
54+
```
55+
56+
### Live Tests
57+
58+
Live tests require an Azure Service Bus namespace. The test framework uses `ServiceBusPreparer` to create the necessary resources.
59+
60+
1. Login to Azure:
61+
```bash
62+
az login
63+
```
64+
65+
2. Set required environment variables (or create a `.env` file):
66+
```
67+
SERVICEBUS_CONNECTION_STRING=<your-connection-string>
68+
```
69+
70+
3. Run tests:
71+
```bash
72+
# Run all live tests
73+
pytest tests/livetest
74+
75+
# Run specific live test
76+
pytest tests/livetest/synctests/test_queue_client.py::test_specific_function
77+
78+
# Show output during test run
79+
pytest -s tests/livetest/synctests/test_queue_client.py::test_specific_function
80+
```
81+
82+
### Common Test Issues
83+
84+
- **Authentication errors**: If you see "The access token is invalid" errors, run `az login` to refresh your authentication.
85+
- **Resource Not Found**: Ensure your environment variables are correctly set and don't include unnecessary parts (e.g., don't include the `.servicebus.windows.net` suffix in namespace names).
86+
- **ImportError**: If you see errors importing library components, try reinstalling with `pip install .` instead of development mode.
87+
88+
## Stress Testing
89+
90+
Stress tests help verify reliability and performance under load. They are designed to test the library under extreme conditions and for extended periods.
91+
92+
### Structure of Stress Test Files
93+
94+
Stress test files are typically organized in the `stress` directory with the following structure:
95+
96+
```
97+
stress/
98+
├── scenarios/ # Contains stress test scenario definitions
99+
├── scripts/ # Support scripts for stress test deployment
100+
│ └── dev_requirement.txt # Dependencies for the stress test
101+
└── templates/ # Kubernetes templates for test deployment
102+
└── testjob.yaml # Test job configuration
103+
```
104+
105+
### Prerequisites
106+
107+
- Docker Desktop
108+
- Kubernetes CLI (kubectl)
109+
- PowerShell
110+
- Helm
111+
112+
### Running Stress Tests
113+
114+
1. Ensure Docker Desktop is running with Kubernetes enabled.
115+
116+
2. Deploy stress tests from the library folder:
117+
```powershell
118+
..\..\..\eng\common\scripts\stress-testing\deploy-stress-tests.ps1 -Namespace <your-namespace>
119+
```
120+
121+
3. Common kubectl commands:
122+
```bash
123+
# Check on pods
124+
kubectl get pods -n <your-namespace>
125+
126+
# Delete namespace
127+
kubectl delete namespace <your-namespace>
128+
129+
# List all active namespaces
130+
helm list --all-namespaces
131+
132+
# Show logs in console
133+
kubectl logs -n <your-namespace> <pod-name>
134+
135+
# For init failure
136+
kubectl logs -n <your-namespace> <pod-name> -c init-azure-deployer
137+
```
138+
139+
4. Testing branch changes:
140+
- To run stress tests against changes in a specific branch, update the `stress/scripts/dev_requirement.txt` file to replace the library with:
141+
```
142+
git+<gh-repo-url>@<branch>#subdirectory=<path/to/sdk>&egg=<package-name>
143+
```
144+
145+
5. Monitoring performance:
146+
- Check the CPU/Memory/errors in the Production Dashboard specified in the Reliability Testing wiki
147+
- Add `--debug_level Debug` in `stress/templates/testjob.yaml` to output debug logs to the File Share (link available in Reliability Testing wiki)
148+
149+
For complete stress testing requirements and setup, refer to the [Reliability Testing documentation](https://dev.azure.com/azure-sdk/internal/_wiki/wikis/internal.wiki/463/Reliability-Testing).
150+
151+
## Performance Testing
152+
153+
The Performance framework helps measure performance metrics for the client library.
154+
155+
### Running Performance Tests Locally
156+
157+
1. Navigate to the performance test directory:
158+
```bash
159+
cd tests/perfstress
160+
```
161+
162+
2. Review the README file for specific test setup:
163+
```bash
164+
cat README.md
165+
```
166+
167+
3. Run performance tests:
168+
```bash
169+
# Run with default options
170+
python servicebus_perf.py
171+
172+
# View available options
173+
python servicebus_perf.py --help
174+
```
175+
176+
### Running Performance Tests in Pipelines
177+
178+
- Performance test resources are deployed dynamically and specified in the service level directory's `perf-resources.bicep` file
179+
- Default parameters are in the service level directory in `perf.yml`, and baseline and source package versions are specified in `perf-tests.yml`
180+
181+
To run performance tests in a pipeline for a PR:
182+
1. Add a comment to the PR: `/azp run python - servicebus - perf`
183+
2. To run manually with custom parameters:
184+
- Find the perf pipelines in azure-sdk internal DevOps pipelines
185+
- Click Run and set the commit to: `refs/pull/<PR #>/merge`
186+
187+
## Additional Resources
188+
189+
- [Service Bus Architecture](https://docs.microsoft.com/azure/service-bus-messaging/service-bus-architecture)
190+
- [Azure SDK Design Guidelines](https://azure.github.io/azure-sdk/python_design.html)
191+
- [Service Bus Samples](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/servicebus/azure-servicebus/samples)

0 commit comments

Comments
 (0)