You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
update READMEs with new information and refs to java stress test doc (Azure#47323)
* update READMEs with new information and refs to java stress test doc
* remove section on logging, fix spelling errors
* add comment about internal links for stress test framekwork
Only the most frequently used commands are listed below. See [Deploying A Stress Test][deploy_stress_test] for more details.
@@ -34,7 +44,7 @@ List deployed packages:
34
44
helm list -n <stress test namespace>
35
45
```
36
46
37
-
the namespace usually matches your username.
47
+
The namespace usually matches your username and `java` for auto-deployed tests.
38
48
39
49
Get stress test pods and status:
40
50
@@ -70,28 +80,13 @@ helm uninstall <stress test name> -n <stress test namespace>
70
80
71
81
### Fault injection
72
82
83
+
For details on using the HTTP fault injector and how it is wired into the Java stress tests, see the [Java Stress Testing documentation](https://msazure.visualstudio.com/One/_wiki/wikis/One.wiki/697419/Java-Stress-Testing-Documentation) and the [HTTP fault-injector documentation][http-fault-injector].
84
+
73
85
## Running tests locally
74
86
75
87
You can also run stress tests locally with or without fault-injection.
76
-
To run test locally:
77
-
1. Build `azure-storage-blob-stress` jar file. For example, you can do it with:
@@ -127,36 +122,24 @@ To get a super-quick idea about test results, look for a log record that looks l
127
122
"succeeded":468,"failed":0}
128
123
```
129
124
130
-
After the stress test is deployed on the cluster, we can monitor the progress on the Application Insights resource inside the stress test resource group.
125
+
After the stress test is deployed on the cluster, we can monitor the progress in the Log Analytics Workspace resource inside the stress test resource group.
131
126
There are several dashboards within the stress test resource group that we can use to monitor the AKS pod and stress test status.
132
127
133
128
#### Stress Test Dashboard
134
129
135
-
General-purpose stress test dashboard is available at https://aka.ms/azsdk/stress/dashboard. It shows:
136
-
- Pod status events
137
-
- CPU and memory utilization of the stress test pods
138
-
- Container logs and events
139
-
140
-
Stress test dashboard does not know about local stress test runs.
- distributed tracing and dependency calls (HTTP, Azure SDK calls)
148
-
- exceptions and logs
149
-
- profiling in production
130
+
General-purpose stress test dashboard is available at [Stress Test Dashboard][stress-test-dashboard]. It shows:
150
131
151
-
Application Insights is useful to:
152
-
- monitor and compare throughput and latency across runs
153
-
- investigate issues and find bottlenecks
132
+
- Stress test pods per namespace
133
+
- Successful runs
134
+
- Duration
135
+
- Container status
136
+
- The ability to query details on each of the pods
154
137
155
-
Application Insights is available for local runs (as long as you provide `-javaagent` option and make sure connection string is configured).
138
+
Stress test dashboard does not know about local stress test runs.
156
139
157
-
##### Stress test workbook
140
+
##### Azure Monitor
158
141
159
-
Storage stress test workbook is available [here][storage-workbook] and allows to pick a specific run and see it's summary:
142
+
Storage stress test workbook is available [here][azure-monitor] and allows to pick a specific run and see its summary:
160
143
1. Key test parameters
161
144
2. Throughput and latency charts
162
145
3. Failed operations including their status and fault injected (if any)
@@ -167,9 +150,7 @@ The workbook relies on tests to emit:
167
150
- certain logs in a specific format produced by [TelemetryHelper][telemetry-helper] class
168
151
- certain spans produced in [BlobScenarioBase][blob-scenario-base] class.
169
152
- report cloud role name that follows `storage-{runId}` pattern.
170
-
Make sure to set `APPLICATIONINSIGHTS_ROLE_NAME` environment variable accordingly to make sure run appears on the dashboard.
171
-
172
-
_Note: some failures are expected and there is no clear 'success' criteria for the stress test_
153
+
Make sure to set `APPLICATIONINSIGHTS_ROLE_NAME` environment variable accordingly to make sure run appears on the dashboard.
173
154
174
155
Here are a few things that clearly indicate an issue:
175
156
- Content mismatch - they can be detected in following ways:
@@ -178,35 +159,13 @@ Here are a few things that clearly indicate an issue:
178
159
- Throughput is noticeably lower than for previous runs with the similar parameters. This would normally correlate with latency being higher than usual.
179
160
- CPU/memory consumption is noticeably higher than for previous runs with the similar parameters
180
161
181
-
### Logging
182
-
183
-
We use [logback.xml][logback_xml] to configure the logging. By default, the stress test run on cluster will output
184
-
`INFO` level log which you may adjust based on your needs.
185
-
186
-
The [storage workbook](#stress-test-workbook) needs `com.azure.storage.blob.stress` and `com.azure.storage.stress` to stay at the `INFO` level.
187
-
188
-
You may also control the verbosity of logs that go to Application Insights - see [Application Insights logging configuration][application-insights-logging] for more details.
189
-
190
-
See also [Logging in Azure SDK][logging-azure-sdk] for more details.
191
-
192
162
## Key concepts
193
163
194
164
### Project Structure
195
165
196
-
See [Layout][stress_test_layout] section for details.
166
+
See [Layout][stress_test_layout] section for details as described by the Azure SDK team.
197
167
198
-
Below is the current structure of project:
199
-
```
200
-
.
201
-
├── src/ # Test code
202
-
├── templates/ # A directory of helm templates that will generate Kubernetes manifest files.
203
-
├── Chart.yaml # A YAML file containing information about the helm chart and its dependencies
204
-
├── scenarios-matrix.yaml # A YAML file containing configuration and custom values for stress test(s)
205
-
├── Dockerfile # A Dockerfile for building the stress test image
206
-
├── stress-test-resources.bicep # An Azure Bicep for deploying stress test azure resources
207
-
├── pom.xml
208
-
└── README.md
209
-
```
168
+
To understand the project structure, you may also refer to the [Java Stress Testing Documentation](https://msazure.visualstudio.com/One/_wiki/wikis/One.wiki/697419/Java-Stress-Testing-Documentation?anchor=how-it-works).
0 commit comments