Commit 5a074ea
authored
Improve datafusion-cli object store profiling summary display (#18085)
## Which issue does this PR close?
- part of #17207
## Rationale for this change
As suggested by @BlakeOrth in
#18045 (comment)
here is an attempt to improve the output of datafusion object store
trace profiling:
## What changes are included in this PR?
Update the output format when `\object_store_profiling summary` is set
Current format (on main, before this PR):
```sql
Summaries:
Get
count: 2
duration min: 0.024603s
duration max: 0.031946s
duration avg: 0.028274s
size min: 8 B
size max: 34322 B
size avg: 17165 B
size sum: 34330 B
```
New format (after this PR):
```sql
DataFusion CLI v50.2.0
> \object_store_profiling summary
ObjectStore Profile mode set to Summary
> select count(*) from 'https://datasets.clickhouse.com/hits_compatible/athena_partitioned/hits_1.parquet';
+----------+
| count(*) |
+----------+
| 1000000 |
+----------+
1 row(s) fetched.
Elapsed 6.754 seconds.
Object Store Profiling
Instrumented Object Store: instrument_mode: Summary, inner: HttpStore
Summaries:
+-----------+----------+-----------+-----------+-----------+-----------+-------+
| Operation | Metric | min | max | avg | sum | count |
+-----------+----------+-----------+-----------+-----------+-----------+-------+
| Get | duration | 0.031645s | 0.047780s | 0.039713s | 0.079425s | 2 |
| Get | size | 8 B | 34322 B | 17165 B | 34330 B | 2 |
+-----------+----------+-----------+-----------+-----------+-----------+-------+
```
## Are these changes tested?
Yes
## Are there any user-facing changes?
Nicer datafusion-cli output1 parent 1b001a1 commit 5a074ea
File tree
5 files changed
+278
-146
lines changed- datafusion-cli
- src
- object_storage
- tests
- snapshots
- docs/source/user-guide/cli
5 files changed
+278
-146
lines changed
0 commit comments