Commit 139c1b7
metric: enhance mo-tool and optimize file service metrics (#23253)
### **User description**
## What type of PR is this?
- [ ] API-change
- [ ] BUG
- [x] Improvement
- [ ] Documentation
- [ ] Feature
- [ ] Test and CI
- [ ] Code Refactoring
## Which issue(s) this PR fixes:
issue #23246
## What this PR does / why we need it:
```shell
./mo-tool dashboard create --mode local --port 3001
./mo-tool dashboard list --mode cloud --port 3000
./mo-tool dashboard delete --mode local --port 3001 # Delete all dashboards in folder
./mo-tool dashboard delete-dashboard --uid <uid> # Delete single dashboard by UID
./mo-tool dashboard delete-folder --mode local --port 3001 # Delete entire folder and all dashboards
```
<img width="1028" height="314" alt="image"
src="https://github.com/user-attachments/assets/f7468043-ee48-45e3-b653-01dc09864b27"
/>
<img width="1465" height="283" alt="image"
src="https://github.com/user-attachments/assets/a64e23e0-06d5-438d-a51f-1840378f040d"
/>
<img width="1473" height="288" alt="image"
src="https://github.com/user-attachments/assets/7514a117-a5f8-457f-94d0-d049f9d0d7f2"
/>
___
### **PR Type**
Enhancement
___
### **Description**
- Add comprehensive dashboard management commands (list, delete,
delete-dashboard, delete-folder)
- Implement S3 connection pool tracking with active connection gauge
metric
- Add disk cache eviction and error tracking metrics
- Enhance HTTP trace metrics with connection reuse and failure rate
analysis
- Support configurable MaxConnsPerHost for S3 object storage connections
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["Dashboard Commands"] -->|create/list/delete| B["Grafana API"]
C["HTTP Client"] -->|track connections| D["S3ConnActiveGauge"]
E["Disk Cache"] -->|evict/error events| F["Cache Metrics"]
G["HTTP Trace"] -->|connection lifecycle| H["Connection Pool Metrics"]
I["Config"] -->|MaxConnsPerHost| C
```
<details><summary><h3>File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>8
files</summary><table>
<tr>
<td><strong>main.go</strong><dd><code>Refactor dashboard command into
subcommands with full CRUD operations</code></dd></td>
<td><a
href="https://github.com/matrixorigin/matrixone/pull/23253/files#diff-ee6866f9fb95784ecffcdd1a9dbac5e896f6a743467c18a07fdd5c5cda357e7c">+314/-3</a>
</td>
</tr>
<tr>
<td><strong>disk_cache.go</strong><dd><code>Add disk cache eviction and
error counter metrics</code>
</dd></td>
<td><a
href="https://github.com/matrixorigin/matrixone/pull/23253/files#diff-2775f0987c87dd3548487f327c4b4f0b8a93e6adbea6fb51b8c9aeaddb543985">+7/-0</a>
</td>
</tr>
<tr>
<td><strong>http_client.go</strong><dd><code>Implement active connection
tracking and periodic cleanup</code></dd></td>
<td><a
href="https://github.com/matrixorigin/matrixone/pull/23253/files#diff-2e7f85fbe5d0260fc0ce5c05baabf885a5c1a77ff1621db750c6bb87001cc719">+150/-4</a>
</td>
</tr>
<tr>
<td><strong>http_trace.go</strong><dd><code>Enhance HTTP trace with
connection reuse and failure tracking</code></dd></td>
<td><a
href="https://github.com/matrixorigin/matrixone/pull/23253/files#diff-849e1f1fbb030fb73508f10940ae8d74cef2191e7e28b732805ccf06e3b36eef">+50/-2</a>
</td>
</tr>
<tr>
<td><strong>grafana_dashboard.go</strong><dd><code>Implement dashboard
list, delete, and folder deletion operations</code></dd></td>
<td><a
href="https://github.com/matrixorigin/matrixone/pull/23253/files#diff-b11e4d02a1c8126d2afb936cff26c8d95feaedeaea7173bef747c3f645ade1df">+317/-0</a>
</td>
</tr>
<tr>
<td><strong>grafana_dashboard_fs.go</strong><dd><code>Add cache
evict/error row and enhance S3 connection metrics
</code><br><code>visualization</code></dd></td>
<td><a
href="https://github.com/matrixorigin/matrixone/pull/23253/files#diff-150a2d0e9e7884f246192255bceb432b8eff0f74015e938db8e21f9bf0b6d6d8">+197/-32</a></td>
</tr>
<tr>
<td><strong>fileservice.go</strong><dd><code>Define S3 active connection
and disk cache error metrics</code> </dd></td>
<td><a
href="https://github.com/matrixorigin/matrixone/pull/23253/files#diff-40c4045e01211017f7ffa2d3be743d46f32abff373c9fc5ec5e4a5fee0eb4ae6">+40/-0</a>
</td>
</tr>
<tr>
<td><strong>metrics.go</strong><dd><code>Register new connection pool
and disk cache metrics</code>
</dd></td>
<td><a
href="https://github.com/matrixorigin/matrixone/pull/23253/files#diff-97ea305cc79979fa7db0a8fd618613cdda1c9bda32e6aa91847d8a754de671f3">+4/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Configuration
changes</strong></td><td><details><summary>2 files</summary><table>
<tr>
<td><strong>object_storage_arguments.go</strong><dd><code>Add
MaxConnsPerHost configuration parameter for S3</code>
</dd></td>
<td><a
href="https://github.com/matrixorigin/matrixone/pull/23253/files#diff-8fbae320ed4d3771224b3ea411e82bdc12c00823f6a8ffd713bb75e5c26987c8">+6/-0</a>
</td>
</tr>
<tr>
<td><strong>Makefile</strong><dd><code>Add list and delete dashboard
make targets with documentation</code></dd></td>
<td><a
href="https://github.com/matrixorigin/matrixone/pull/23253/files#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52">+86/-8</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>DEV_README.md</strong><dd><code>Document dashboard list and
delete operations with examples</code></dd></td>
<td><a
href="https://github.com/matrixorigin/matrixone/pull/23253/files#diff-71ef0b6c2617dd81654dca538af01d544e9cdfdb92239bf702e26f874427ca91">+37/-4</a>
</td>
</tr>
</table></details></td></tr></tbody></table>
</details>
___
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>1 parent ccb29ae commit 139c1b7
File tree
11 files changed
+1205
-53
lines changed- cmd/mo-dashboard
- etc
- pkg
- fileservice
- util/metric/v2
- dashboard
11 files changed
+1205
-53
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
371 | 371 | | |
372 | 372 | | |
373 | 373 | | |
374 | | - | |
375 | | - | |
376 | | - | |
377 | | - | |
378 | | - | |
379 | | - | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
380 | 393 | | |
381 | 394 | | |
382 | 395 | | |
383 | 396 | | |
384 | 397 | | |
385 | 398 | | |
386 | 399 | | |
387 | | - | |
388 | | - | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
389 | 405 | | |
390 | 406 | | |
391 | 407 | | |
| |||
790 | 806 | | |
791 | 807 | | |
792 | 808 | | |
| 809 | + | |
| 810 | + | |
| 811 | + | |
| 812 | + | |
| 813 | + | |
| 814 | + | |
| 815 | + | |
| 816 | + | |
| 817 | + | |
| 818 | + | |
| 819 | + | |
| 820 | + | |
| 821 | + | |
| 822 | + | |
| 823 | + | |
| 824 | + | |
| 825 | + | |
| 826 | + | |
| 827 | + | |
| 828 | + | |
| 829 | + | |
| 830 | + | |
| 831 | + | |
| 832 | + | |
| 833 | + | |
| 834 | + | |
| 835 | + | |
| 836 | + | |
| 837 | + | |
| 838 | + | |
| 839 | + | |
| 840 | + | |
| 841 | + | |
| 842 | + | |
| 843 | + | |
| 844 | + | |
| 845 | + | |
| 846 | + | |
| 847 | + | |
| 848 | + | |
| 849 | + | |
| 850 | + | |
| 851 | + | |
| 852 | + | |
| 853 | + | |
| 854 | + | |
| 855 | + | |
| 856 | + | |
| 857 | + | |
| 858 | + | |
| 859 | + | |
| 860 | + | |
| 861 | + | |
| 862 | + | |
| 863 | + | |
| 864 | + | |
| 865 | + | |
| 866 | + | |
| 867 | + | |
| 868 | + | |
| 869 | + | |
| 870 | + | |
793 | 871 | | |
794 | 872 | | |
795 | 873 | | |
| |||
0 commit comments