Skip to content

Commit a3828ac

Browse files
mythzmythz
committed
Add new Release Notes docs
1 parent 97f7279 commit a3828ac

File tree

9 files changed

+1698
-46
lines changed

9 files changed

+1698
-46
lines changed

‎MyApp/_pages/admin-ui-analytics.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Admin UI Analytics
2+
title: Admin UI Analytics for SQLite
33
---
44

55
Comprehensive API Analytics is available to all ServiceStack Apps configured with [SQLite Request Logging](/sqlite-request-logs).

‎MyApp/_pages/admin-ui-rdbms-analytics.md‎

Lines changed: 342 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,342 @@
1+
---
2+
title: Admin UI Analytics for RDBMS
3+
---
4+
5+
[ServiceStack v8.9](/releases/v8_09) restores parity to **PostgreSQL**, **SQL Server** & **MySQL** RDBMS's for our previous
6+
SQLite-only features with the new `DbRequestLogger` which is a drop-in replacement for
7+
[SQLite Request Logging](/sqlite-request-logs) for persisting API Request Logs to a RDBMS.
8+
9+
Whilst maintaining an archive of API Requests is nice, the real value of DB Request Logging is that it unlocks the
10+
comprehensive API Analytics and querying Logging available that was previously limited to SQLite Request Logs.
11+
12+
:::youtube kjLcm1llC5Y
13+
In Depth and Interactive API Analytics available to all ASP .NET Core ServiceStack Apps!
14+
:::
15+
16+
### Benefits of API Analytics
17+
18+
They provide deep and invaluable insight into your System API Usage, device distribution, its Users, API Keys and the
19+
IPs where most traffic generates:
20+
21+
- **Visibility:** Provides a clear, visual summary of complex log data, making it easier to understand API usage and performance at a glance.
22+
- **Performance Monitoring:** Helps track key metrics like request volume and response times to ensure APIs are meeting performance expectations.
23+
- **User Understanding:** Offers insights into how users (and bots) are interacting with the APIs (devices, browsers).
24+
- **Troubleshooting:** Aids in quickly identifying trends, anomalies, or specific endpoints related to issues.
25+
- **Resource Planning:** Understanding usage patterns helps in scaling infrastructure appropriately.
26+
- **Security Insight:** Identifying bot traffic and unusual request patterns can be an early indicator of security concerns.
27+
28+
### Interactive Analytics
29+
30+
Analytics are also interactive where you're able to drill down to monitor the activity of individual APIs, Users, API Keys
31+
and IPs which have further links back to the request logs which the summary analytics are derived from.
32+
33+
As they offer significant and valuable insights the `SqliteRequestLogger` is built into all ASP.NET Core
34+
IdentityAuth templates, to switch it over to use a RDBMS we recommend installing `db-identity` mix gist to
35+
also replace SQLite BackgroundJobs with the RDBMS `DatabaseJobFeature`:
36+
37+
:::sh
38+
x mix db-identity
39+
:::
40+
41+
Or if you just want to replace SQLite Request Logs with a RDBMS use:
42+
43+
:::sh
44+
x mix db-requestlogs
45+
:::
46+
47+
Or you can copy the [Modular Startup](/modular-startup) script below:
48+
49+
```csharp
50+
[assembly: HostingStartup(typeof(MyApp.ConfigureRequestLogs))]
51+
52+
namespace MyApp;
53+
54+
public class ConfigureRequestLogs : IHostingStartup
55+
{
56+
public void Configure(IWebHostBuilder builder) => builder
57+
.ConfigureServices((context, services) => {
58+
59+
services.AddPlugin(new RequestLogsFeature {
60+
RequestLogger = new DbRequestLogger {
61+
// NamedConnection = "<alternative db>"
62+
},
63+
EnableResponseTracking = true,
64+
EnableRequestBodyTracking = true,
65+
EnableErrorTracking = true
66+
});
67+
services.AddHostedService<RequestLogsHostedService>();
68+
69+
if (context.HostingEnvironment.IsDevelopment())
70+
{
71+
services.AddPlugin(new ProfilingFeature());
72+
}
73+
});
74+
}
75+
76+
public class RequestLogsHostedService(ILogger<RequestLogsHostedService> log, IRequestLogger requestLogger) : BackgroundService
77+
{
78+
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
79+
{
80+
using var timer = new PeriodicTimer(TimeSpan.FromSeconds(3));
81+
if (requestLogger is IRequireAnalytics logger)
82+
{
83+
while (!stoppingToken.IsCancellationRequested && await timer.WaitForNextTickAsync(stoppingToken))
84+
{
85+
await logger.TickAsync(log, stoppingToken);
86+
}
87+
}
88+
}
89+
}
90+
```
91+
92+
### RDBMS Provider
93+
94+
When using a remote RDBMS, network latency becomes a primary concern that any solution needs to be designed around,
95+
as such the API Request Logs are initially maintained in an in memory collection before being flushed to the database
96+
**every 3 seconds** — configurable in the `PeriodicTimer` interval above.
97+
98+
To reduce the number of round-trips to the database, the `DbRequestLogger` batches all pending logs into a single
99+
request using [OrmLite's Bulk Inserts](/ormlite/bulk-inserts) which is supported by all
100+
major RDBMS's.
101+
102+
### PostgreSQL Table Partitioning
103+
104+
PostgreSQL provides native support for table partitioning, allowing us to automatically create monthly partitions using
105+
`PARTITION BY RANGE` on the `CreatedDate` column. The `DbRequestLogger` automatically creates new monthly partitions
106+
as needed, maintaining the same logical separation as SQLite's monthly .db's while keeping everything within a single
107+
Postgres DB:
108+
109+
```sql
110+
CREATE TABLE "RequestLog" (
111+
-- columns...
112+
"CreatedDate" TIMESTAMP NOT NULL,
113+
PRIMARY KEY ("Id","CreatedDate")
114+
) PARTITION BY RANGE ("CreatedDate");
115+
116+
-- Monthly partitions are automatically created, e.g.:
117+
CREATE TABLE "RequestLog_2025_01" PARTITION OF "RequestLog"
118+
FOR VALUES FROM ('2025-01-01') TO ('2025-02-01');
119+
```
120+
121+
### SQLServer / MySQL - Manual Partition Management
122+
123+
For **SQL Server** and **MySQL**, monthly partitioned tables need to be created **out-of-band**
124+
(either manually or via cronjob scripts) since they don't support the same level of automatic
125+
partition management as PostgreSQL. However, this still works well in practice as because `RequestLog` is an
126+
**Append Only** table with all querying from the Admin UIs being filtered by its indexed `CreatedDate`
127+
in monthly viewable snapshots like it was with SQLite.
128+
129+
### Separate RequestLog Database
130+
131+
Or if preferred, you can maintain request logs in a **separate database** from your main application database.
132+
This separation keeps the write-heavy logging load off your primary database, allowing you to optimize
133+
each database independently for its specific workload patterns like maintaining different backup strategies
134+
for your critical application data vs. log history.
135+
136+
```csharp
137+
// Configure.Db.cs
138+
services.AddOrmLite(options => options.UsePostgres(connectionString))
139+
.AddPostgres("logs", logsConnectionString);
140+
141+
// Configure.RequestLogs.cs
142+
services.AddPlugin(new RequestLogsFeature {
143+
RequestLogger = new DbRequestLogger {
144+
NamedConnection = "logs"
145+
},
146+
//...
147+
});
148+
```
149+
150+
## Queryable Admin Logging UI
151+
152+
This will enable a more feature rich Request Logging Admin UI which utilizes the full queryability of the
153+
[AutoQueryGrid](/vue/autoquerygrid) component to filter, sort and export Request Logs.
154+
155+
[![](/img/pages/admin-ui/sqlitelogs.webp)](/img/pages/admin-ui/sqlitelogs.webp)
156+
157+
## Analytics Overview
158+
159+
Utilizing an `DbRequestLogger` also enables the **Analytics** Admin UI in the sidebar which initially
160+
displays the API Analytics Dashboard:
161+
162+
:::{.wideshot}
163+
[![](/img/pages/admin-ui/analytics-apis1.webp)](/img/pages/admin-ui/analytics-apis1.webp)
164+
:::
165+
166+
### Distribution Pie Charts
167+
168+
Lets you quickly understand the composition of your user base and traffic sources and the
169+
distribution of users across different web browsers, device types, and to identify the proportion of traffic coming from automated bots.
170+
171+
### Requests per day Line Chart
172+
173+
Lets you monitor API usage trends and performance over time. It tracks the total number of API requests and the average response
174+
time day-by-day. You can easily spot trends like peak usage hours/days, identify sudden spikes or drops in traffic,
175+
and correlate request volume with API performance which is crucial for capacity planning and performance troubleshooting.
176+
177+
### API tag groups Pie Chart
178+
179+
Lets you understand the usage patterns across different functional categories of your APIs.
180+
By grouping API requests based on assigned tags (like Security, Authentication, User Management, Tech, etc.), you get a
181+
high-level view of which *types* of functionalities are most frequently used or are generating the most load.
182+
183+
### API Requests Bar Chart
184+
185+
Lets you identify the most and least frequently used specific API endpoints which ranks individual API endpoints by
186+
the number of requests they receive. This helps pinpoint:
187+
188+
- **Critical Endpoints:** The most heavily used APIs that require robust performance and monitoring.
189+
- **Optimization Targets:** High-traffic endpoints that could benefit from performance optimization.
190+
- **Underutilized Endpoints:** APIs that might be candidates for deprecation or require promotion.
191+
- **Troubleshooting:** If performance issues arise (seen in the line chart), this helps narrow down which specific endpoint might be responsible.
192+
193+
:::{.wideshot}
194+
[![](/img/pages/admin-ui/analytics-apis2.webp)](/img/pages/admin-ui/analytics-apis2.webp)
195+
:::
196+
197+
### Total Duration Bar Chart
198+
199+
Identifies which API endpoints consume the most *cumulative processing time* over the selected period.
200+
Even if an API endpoint is relatively fast per call, if it's called extremely frequently, it can contribute significantly to overall server load.
201+
Optimizing these can lead to significant savings in server resources (CPU, memory).
202+
203+
### Average Duration Bar Chart
204+
205+
Pinpoints which API endpoints are the slowest on a *per-request* basis. APIs at the top of this list are prime candidates
206+
for performance investigation and optimization, as they represent potential user-facing slowness or system bottlenecks.
207+
208+
### Requests by Duration Ranges Histogram
209+
210+
Provides an overview of the performance distribution for *all* API requests.
211+
This chart shows how many requests fall into different speed buckets and helps you understand the overall responsiveness of your API system at a glance.
212+
213+
## Individual API Analytics
214+
215+
Clicking on an API's bar chart displays a dedicated, detailed view of a single API endpoint's behavior, isolating its performance
216+
and usage patterns from the overall system metrics offering immediate insight into the endpoint's traffic volume and reliability.
217+
218+
:::{.wideshot}
219+
[![](/img/pages/admin-ui/analytics-api.webp)](/img/pages/admin-ui/analytics-api.webp)
220+
:::
221+
222+
### Total Requests
223+
224+
Displays the total requests for an API during the selected month. It includes HTTP Status Breakdown which
225+
provide **direct access to the filtered request logs**. This is a major benefit for **rapid troubleshooting**, allowing
226+
you to instantly view the specific log entries corresponding to successful requests or particular error codes for this API.
227+
228+
### Last Request Information
229+
230+
Provides immediate context on the most recent activity for this endpoint with *when* the last request occurred,
231+
the source **IP address** and device information to help understand recent usage and check if the endpoint is still active,
232+
or quickly investigate the very last interaction if needed.
233+
234+
### Duration Summary Table (Total, Min, Max)
235+
236+
Quantifies the performance characteristics specifically for this endpoint with the cumulative (Total) processing load,
237+
the best-case performance (Min), and the worst-case performance (Max) which is useful for identifying performance outliers.
238+
239+
### Duration Requests Histogram
240+
241+
Visualizes the performance distribution for this API.
242+
243+
### Top Users Bar Chart
244+
245+
Identifies which authenticated users are most frequently calling this API and relies on this endpoint the most.
246+
This can be useful for identifying power users, potential API abuse by a specific user account, or understanding the impact of changes to this API on key users.
247+
248+
### Top IP Addresses Bar Chart
249+
250+
Shows which source IP addresses are generating the most traffic for this API.
251+
Useful for identifying high-volume clients, specific servers interacting with this endpoint, or potentially malicious IPs.
252+
253+
## Users
254+
255+
The **Users** tab will display the top 100 Users who make the most API Requests and lets you click on a Users bar chart
256+
to view their individual User analytics.
257+
258+
:::{.wideshot}
259+
[![](/img/pages/admin-ui/analytics-users.webp)](/img/pages/admin-ui/analytics-users.webp)
260+
:::
261+
262+
### Individual User Analytics
263+
264+
Provides a comprehensive view of a single user's complete interaction history and behavior across all APIs they've accessed,
265+
shifting the focus from API performance to user experience and activity.
266+
267+
:::{.wideshot}
268+
[![](/img/pages/admin-ui/analytics-user.webp)](/img/pages/admin-ui/analytics-user.webp)
269+
:::
270+
271+
### User Info & Total Requests
272+
273+
Identifies the user and quantifies their overall activity level. Clicking on their ID or Name will navigate to the Users Admin UI.
274+
It also shows their success/error rate via the clickable status code links. This helps gauge user engagement and baseline activity.
275+
276+
### Last Request Information
277+
278+
Offers a snapshot of the user's most recent interaction for immediate context.
279+
Knowing **when**, **what** API they called, from which **IP address**, using which **client** & **device** is valuable
280+
for support, identifying their last action or checking recent activity.
281+
282+
### HTTP Status Pie Chart
283+
284+
Visualizes the overall success and error rate specifically for this user's API requests.
285+
286+
### Performance & Request Body Summary Table
287+
288+
Quantifies the performance experienced by this user and the data they typically send.
289+
290+
### Duration Requests Histogram
291+
292+
Shows the distribution of response times for requests made by this user to help understand the typical performance this user experiences.
293+
294+
### Top APIs Bar Chart
295+
296+
Reveals which API endpoints this user interacts with most frequently and help understanding user behavior and which features they use most.
297+
298+
### Top IP Addresses Bar Chart
299+
300+
Identifies the primary network locations or devices the user connects from.
301+
302+
### User Admin UI Analytics
303+
304+
To assist in discoverability a snapshot of a Users Analytics is also visible in the Users Admin UI:
305+
306+
[![](/img/pages/admin-ui/analytics-user-adminui.webp)](/img/pages/admin-ui/analytics-user-adminui.webp)
307+
308+
Clicking on **View User Analytics** takes you to the Users Analytics page to access to the full Analytics features and navigation.
309+
310+
## API Keys
311+
312+
The **API Keys** tab will display the top 100 API Keys who make the most API Requests and lets you click on an API Key
313+
bar chart to view its individual API Key analytics.
314+
315+
:::{.wideshot}
316+
[![](/img/pages/admin-ui/analytics-apikeys.webp)](/img/pages/admin-ui/analytics-apikeys.webp)
317+
:::
318+
319+
### Individual API Key Analytics
320+
321+
Provides comprehensive API Key analytics Similar to User Analytics but limited to the API Usage of a single API Key:
322+
323+
:::{.wideshot}
324+
[![](/img/pages/admin-ui/analytics-apikey.webp)](/img/pages/admin-ui/analytics-apikey.webp)
325+
:::
326+
327+
## IPs
328+
329+
The **IP Addresses** tab will display the top 100 IPs that make the most API Requests. Click on an IP's
330+
bar chart to view its individual analytics made from that IP Address.
331+
332+
:::{.wideshot}
333+
[![](/img/pages/admin-ui/analytics-ips.webp)](/img/pages/admin-ui/analytics-ips.webp)
334+
:::
335+
336+
### Individual IP Analytics
337+
338+
Provides comprehensive IP Address analytics Similar to User Analytics but limited to the API Usage from a single IP Address:
339+
340+
:::{.wideshot}
341+
[![](/img/pages/admin-ui/analytics-ip.webp)](/img/pages/admin-ui/analytics-ip.webp)
342+
:::

0 commit comments

Comments
 (0)