Skip to content

Commit 5b1efd7

Browse files
cap10morgancap10morgan
authored
Merge pull request #281 from HarperDB/describe-analytics
Add table describing known analytics
2 parents edea9da + 0bc9fab commit 5b1efd7

File tree

2 files changed

+134
-22
lines changed

2 files changed

+134
-22
lines changed

‎docs/technical-details/reference/analytics.md‎

Lines changed: 67 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -102,16 +102,72 @@ And a summary record looks like:
102102
}
103103
```
104104

105-
The following are general resource usage statistics that are tracked:
106-
107-
- `memory` - This includes RSS, heap, buffer and external data usage.
108-
- `utilization` - How much of the time the worker was processing requests.
109-
- `mqtt-connections` - The number of MQTT connections.
105+
# Standard Analytics Metrics
106+
107+
While applications can define their own metrics, Harper provides a set of standard metrics that are tracked for all services:
108+
109+
## HTTP
110+
111+
The following metrics are tracked for all HTTP requests:
112+
113+
| `metric` | `path` | `method` | `type` | Unit | Description |
114+
| ------------------ | ------------- | -------------- | ---------------------------------------------- | ------------ | ------------------------------------------------------- |
115+
| `duration` | resource path | request method | `cache-hit` or `cache-miss` if a caching table | milliseconds | Duration of request handler |
116+
| `duration` | route path | request method | fastify-route | milliseconds | |
117+
| `duration` | operation | | operation | milliseconds | |
118+
| `success` | resource path | request method | | % | |
119+
| `success` | route path | request method | fastify-route | % | |
120+
| `success` | operation | | operation | % | |
121+
| `bytes-sent` | resource path | request method | | bytes | |
122+
| `bytes-sent` | route path | request method | fastify-route | bytes | |
123+
| `bytes-sent` | operation | | operation | bytes | |
124+
| `transfer` | resource path | request method | operation | milliseconds | duration of transfer |
125+
| `transfer` | route path | request method | fastify-route | milliseconds | duration of transfer |
126+
| `transfer` | operation | | operation | milliseconds | duration of transfer |
127+
| `socket-routed` | | | | % | percentage of sockets that could be immediately routed |
128+
| `tls-handshake` | | | | milliseconds | |
129+
| `tls-reused` | | | | % | percentage of TLS that reuses sessions |
130+
| `cache-hit` | table name | | | % | The percentage of cache hits |
131+
| `cache-resolution` | table name | | | milliseconds | The duration of resolving requests for uncached entries |
132+
133+
The following are metrics for real-time MQTT connections:
134+
| `metric` | `path` | `method` | `type` | Unit | Description |
135+
|---|---|---|---|---|---|
136+
| `mqtt-connections` | | | | count | The number of open direct MQTT connections |
137+
| `ws-connections` | | | | count | number of open WS connections|
138+
| `connection` | `mqtt` | `connect` | | % | percentage of successful direct MQTT connections |
139+
| `connection` | `mqtt` | `disconnect` | | % | percentage of explicit direct MQTT disconnects |
140+
| `connection` | `ws` | `connect` | | % | percentage of successful WS connections |
141+
| `connection` | `ws` | `disconnect` | | % | percentage of explicit WS disconnects |
142+
| `bytes-sent` | topic | mqtt command | `mqtt` | bytes | The number of bytes sent for a given command and topic |
143+
144+
The following are metrics for replication:
145+
146+
| `metric` | `path` | `method` | `type` | Unit | Description |
147+
| ---------------- | ------------- | ------------- | --------- | ----- | ----------------------------------------------------- |
148+
| `bytes-sent` | node.database | `replication` | `egress` | bytes | The number of bytes sent for replication |
149+
| `bytes-sent` | node.database | `replication` | `blob` | bytes | The number of bytes sent for replication of blobs |
150+
| `bytes-received` | node.database | `replication` | `ingress` | bytes | The number of bytes received for replication |
151+
| `bytes-received` | node.database | `replication` | `blob` | bytes | The number of bytes received for replication of blobs |
110152

111-
The following types of information is tracked for each HTTP request:
112-
113-
- `success` - How many requests returned a successful response (20x response code). TTFB - Time to first byte in the response to the client.
114-
- `transfer` - Time to finish the transfer of the data to the client.
115-
- `bytes-sent` - How many bytes of data were sent to the client.
153+
The following are general resource usage statistics that are tracked:
116154

117-
Requests are categorized by operation name, for the operations API, by the resource (name) with the REST API, and by command for the MQTT interface.
155+
| `metric` | primary attribute(s) | other attribute(s) | Unit | Description |
156+
| ------------------------- | ------------------------------------------------------------------------------------------------ | ------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------- |
157+
| `database-size` | `size`, `used`, `free`, `audit` | `database` | bytes | The size of the database in bytes |
158+
| `main-thread-utilization` | `idle`, `active`, `taskQueueLatency`, `rss`, `heapTotal`, `heapUsed`, `external`, `arrayBuffers` | `time` | various | Main thread resource usage; including idle time, active time, task queue latency, RSS, heap, buffer and external memory usage |
159+
| `resource-usage` | | | various | [See breakout below](#resource-usage) |
160+
| `storage-volume` | `available`, `free`, `size` | `database` | bytes | The size of the storage volume in bytes |
161+
| `table-size` | `size` | `database`, `table` | bytes | The size of the table in bytes |
162+
| `utilization` | | | % | How much of the time the worker was processing requests |
163+
164+
<a id="resource-usage"></a>
165+
`resource-usage` metrics are everything returned by [node:process.resourceUsage()](https://nodejs.org/api/process.html#processresourceusage)[^1] plus the following additional metrics:
166+
167+
| `metric` | Unit | Description |
168+
| ---------------- | ---- | ----------------------------------------------------- |
169+
| `time` | ms | Current time when metric was recorded (Unix time) |
170+
| `period` | ms | Duration of the metric period |
171+
| `cpuUtilization` | % | CPU utilization percentage (user and system combined) |
172+
173+
[^1]: The `userCPUTime` and `systemCPUTime` metrics are converted to milliseconds to match the other time-related metrics.

‎versioned_docs/version-4.6/technical-details/reference/analytics.md‎

Lines changed: 67 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -102,16 +102,72 @@ And a summary record looks like:
102102
}
103103
```
104104

105-
The following are general resource usage statistics that are tracked:
106-
107-
- `memory` - This includes RSS, heap, buffer and external data usage.
108-
- `utilization` - How much of the time the worker was processing requests.
109-
- mqtt-connections - The number of MQTT connections.
105+
# Standard Analytics Metrics
106+
107+
While applications can define their own metrics, Harper provides a set of standard metrics that are tracked for all services:
108+
109+
## HTTP
110+
111+
The following metrics are tracked for all HTTP requests:
112+
113+
| `metric` | `path` | `method` | `type` | Unit | Description |
114+
| ------------------ | ------------- | -------------- | ---------------------------------------------- | ------------ | ------------------------------------------------------- |
115+
| `duration` | resource path | request method | `cache-hit` or `cache-miss` if a caching table | milliseconds | Duration of request handler |
116+
| `duration` | route path | request method | fastify-route | milliseconds | |
117+
| `duration` | operation | | operation | milliseconds | |
118+
| `success` | resource path | request method | | % | |
119+
| `success` | route path | request method | fastify-route | % | |
120+
| `success` | operation | | operation | % | |
121+
| `bytes-sent` | resource path | request method | | bytes | |
122+
| `bytes-sent` | route path | request method | fastify-route | bytes | |
123+
| `bytes-sent` | operation | | operation | bytes | |
124+
| `transfer` | resource path | request method | operation | milliseconds | duration of transfer |
125+
| `transfer` | route path | request method | fastify-route | milliseconds | duration of transfer |
126+
| `transfer` | operation | | operation | milliseconds | duration of transfer |
127+
| `socket-routed` | | | | % | percentage of sockets that could be immediately routed |
128+
| `tls-handshake` | | | | milliseconds | |
129+
| `tls-reused` | | | | % | percentage of TLS that reuses sessions |
130+
| `cache-hit` | table name | | | % | The percentage of cache hits |
131+
| `cache-resolution` | table name | | | milliseconds | The duration of resolving requests for uncached entries |
132+
133+
The following are metrics for real-time MQTT connections:
134+
| `metric` | `path` | `method` | `type` | Unit | Description |
135+
|--------------------|---------------|----------------|--------------------------------------------|--------------------------------------------------------|---------------------------------------------------------|
136+
| `mqtt-connections` | | | | count | The number of open direct MQTT connections |
137+
| `ws-connections` | | | | count | number of open WS connections|
138+
| `connection` | `mqtt` | `connect` | | % | percentage of successful direct MQTT connections |
139+
| `connection` | `mqtt` | `disconnect` | | % | percentage of explicit direct MQTT disconnects |
140+
| `connection` | `ws` | `connect` | | % | percentage of successful WS connections |
141+
| `connection` | `ws` | `disconnect` | | % | percentage of explicit WS disconnects |
142+
| `bytes-sent` | topic | mqtt command | `mqtt` | bytes | The number of bytes sent for a given command and topic |
143+
144+
The following are metrics for replication:
145+
146+
| `metric` | `path` | `method` | `type` | Unit | Description |
147+
| ---------------- | ------------- | ------------- | --------- | ----- | ----------------------------------------------------- |
148+
| `bytes-sent` | node.database | `replication` | `egress` | bytes | The number of bytes sent for replication |
149+
| `bytes-sent` | node.database | `replication` | `blob` | bytes | The number of bytes sent for replication of blobs |
150+
| `bytes-received` | node.database | `replication` | `ingress` | bytes | The number of bytes received for replication |
151+
| `bytes-received` | node.database | `replication` | `blob` | bytes | The number of bytes received for replication of blobs |
110152

111-
The following types of information is tracked for each HTTP request:
112-
113-
- `success` - How many requests returned a successful response (20x response code). TTFB - Time to first byte in the response to the client.
114-
- `transfer` - Time to finish the transfer of the data to the client.
115-
- bytes-sent - How many bytes of data were sent to the client.
153+
The following are general resource usage statistics that are tracked:
116154

117-
Requests are categorized by operation name, for the operations API, by the resource (name) with the REST API, and by command for the MQTT interface.
155+
| `metric` | primary attribute(s) | other attribute(s) | Unit | Description |
156+
| ------------------------- | ------------------------------------------------------------------------------------------------ | ------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------- |
157+
| `database-size` | `size`, `used`, `free`, `audit` | `database` | bytes | The size of the database in bytes |
158+
| `main-thread-utilization` | `idle`, `active`, `taskQueueLatency`, `rss`, `heapTotal`, `heapUsed`, `external`, `arrayBuffers` | `time` | various | Main thread resource usage; including idle time, active time, task queue latency, RSS, heap, buffer and external memory usage |
159+
| `resource-usage` | | | various | [See breakout below](#resource-usage) |
160+
| `storage-volume` | `available`, `free`, `size` | `database` | bytes | The size of the storage volume in bytes |
161+
| `table-size` | `size` | `database`, `table` | bytes | The size of the table in bytes |
162+
| `utilization` | | | % | How much of the time the worker was processing requests |
163+
164+
<a id="resource-usage"></a>
165+
`resource-usage` metrics are everything returned by [node:process.resourceUsage()](https://nodejs.org/api/process.html#processresourceusage)[^1] plus the following additional metrics:
166+
167+
| `metric` | Unit | Description |
168+
| ---------------- | ---- | ----------------------------------------------------- |
169+
| `time` | ms | Current time when metric was recorded (Unix time) |
170+
| `period` | ms | Duration of the metric period |
171+
| `cpuUtilization` | % | CPU utilization percentage (user and system combined) |
172+
173+
[^1]: The `userCPUTime` and `systemCPUTime` metrics are converted to milliseconds to match the other time-related metrics.

0 commit comments

Comments
 (0)