Skip to content

Commit d2bfad2

Browse files
author
“Kevin”
committed
Update Readme after refactor
1 parent 1066542 commit d2bfad2

File tree

1 file changed

+26
-16
lines changed
  • cmd/samples/advanced/autoscaling-monitoring

1 file changed

+26
-16
lines changed

cmd/samples/advanced/autoscaling-monitoring/README.md

Lines changed: 26 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33
This sample demonstrates three advanced Cadence worker features:
44

55
1. **Worker Poller Autoscaling** - Dynamic adjustment of worker poller goroutines based on workload
6-
2. **Prometheus Tally Reporter** - Metrics collection using Tally and Prometheus
7-
3. **HTTP Endpoint for Prometheus Scraping** - Exposing metrics for monitoring
6+
2. **Integrated Prometheus Metrics** - Real-time metrics collection using Tally with Prometheus reporter
7+
3. **Autoscaling Metrics** - Comprehensive autoscaling behavior metrics exposed via HTTP endpoint
88

99
## Features
1010

@@ -18,10 +18,11 @@ The worker uses `worker.NewV2` with `AutoScalerOptions` to enable true autoscali
1818
The worker automatically adjusts the number of poller goroutines between the min and max values based on the current workload.
1919

2020
### Prometheus Metrics
21-
The sample exposes standard Cadence metrics via Prometheus, including comprehensive autoscaling behavior metrics:
22-
- Standard worker performance metrics (poller counts, task processing rates)
23-
- Autoscaling-specific metrics (`cadence_concurrency_auto_scaler_*`)
24-
- All metrics automatically emitted by the Cadence Go client
21+
The sample uses Tally with Prometheus reporter to expose comprehensive metrics:
22+
- **Real-time autoscaling metrics** - Poller count changes, quota adjustments, wait times
23+
- **Worker performance metrics** - Task processing rates, poller utilization, queue depths
24+
- **Standard Cadence metrics** - All metrics automatically emitted by the Cadence Go client
25+
- **Sanitized metric names** - Prometheus-compatible metric names and labels
2526

2627
### Monitoring Dashboards
2728
When running the Cadence server locally with Grafana, you can access the client dashboards at:
@@ -41,7 +42,10 @@ When running the Cadence server locally with Grafana, you can access the client
4142
./bin/autoscaling-monitoring -m worker
4243
```
4344

44-
### 2. Start the Prometheus Server (Optional)
45+
The worker automatically exposes metrics at: http://127.0.0.1:8004/metrics
46+
47+
### 2. Access Metrics (Optional)
48+
For dedicated metrics server:
4549
```bash
4650
./bin/autoscaling-monitoring -m server
4751
```
@@ -100,8 +104,8 @@ The configuration values are used throughout the sample:
100104
3. **Activity Configuration** (`activities.go`):
101105
- `minProcessingTime`, `maxProcessingTime` → Activity processing time range
102106

103-
4. **Prometheus Configuration** (`prometheus_server.go`):
104-
- `listenAddress` HTTP server port
107+
4. **Prometheus Configuration** (integrated):
108+
- `listenAddress` → Metrics endpoint port (default: 127.0.0.1:8004)
105109

106110
### Default Configuration
107111

@@ -128,6 +132,9 @@ autoscaling:
128132

129133
### Metrics Endpoints
130134
- **Prometheus Metrics**: http://127.0.0.1:8004/metrics
135+
- Exposed automatically when running worker or server mode
136+
- Real-time autoscaling and worker performance metrics
137+
- Prometheus-compatible format with sanitized names
131138

132139
### Grafana Dashboard
133140
Access the Cadence client dashboard at: http://localhost:3000/d/dehkspwgabvuoc/cadence-client
@@ -144,6 +151,8 @@ Access the Cadence client dashboard at: http://localhost:3000/d/dehkspwgabvuoc/c
144151
- `cadence_worker_poller_count` - Number of active poller goroutines (key autoscaling indicator)
145152
- `cadence_concurrency_auto_scaler_poller_quota` - Current poller quota for autoscaling
146153
- `cadence_concurrency_auto_scaler_poller_wait_time` - Time pollers wait for tasks
154+
- `cadence_concurrency_auto_scaler_scale_up_count` - Number of scale-up events
155+
- `cadence_concurrency_auto_scaler_scale_down_count` - Number of scale-down events
147156

148157
## How It Works
149158

@@ -161,11 +170,11 @@ The worker uses `worker.NewV2` with `AutoScalerOptions` to:
161170
- Automatically adjust based on task queue depth and processing time
162171

163172
### Metrics Collection
164-
The sample exposes standard Cadence metrics via Prometheus:
165-
- Worker poller count and utilization
166-
- Decision and activity task processing rates
167-
- Task queue depth and processing times
168-
- All standard Cadence worker metrics via Tally
173+
The sample uses Tally with Prometheus reporter for comprehensive metrics:
174+
- **Real-time autoscaling metrics** - Poller count changes, quota adjustments, scale events
175+
- **Worker performance metrics** - Task processing rates, poller utilization, queue depths
176+
- **Standard Cadence metrics** - All metrics automatically emitted by the Cadence Go client
177+
- **Sanitized metric names** - Prometheus-compatible format with proper character replacement
169178

170179
## Production Considerations
171180

@@ -207,9 +216,10 @@ The sample exposes standard Cadence metrics via Prometheus:
207216
- Review default values if config file is not found
208217

209218
4. **Metrics Not Appearing**:
210-
- Verify Prometheus server is running
211-
- Check metrics endpoint is accessible
219+
- Verify worker is running (metrics are exposed automatically)
220+
- Check metrics endpoint is accessible: http://127.0.0.1:8004/metrics
212221
- Ensure Prometheus is configured to scrape the endpoint
222+
- Check for metric name sanitization issues
213223

214224
5. **Dashboard Not Loading**:
215225
- Verify Grafana is running

0 commit comments

Comments
 (0)