Skip to content

Improve the datafusion explain output#4358

Merged
jackkleeman merged 1 commit intomainfrom
jk/vzwlxmnmpvmn
Feb 7, 2026
Merged

Improve the datafusion explain output#4358
jackkleeman merged 1 commit intomainfrom
jk/vzwlxmnmpvmn

Conversation

@jackkleeman
Copy link
Contributor

@jackkleeman jackkleeman commented Feb 6, 2026

It makes it quite a bit easier to debug if you can see the filters and the projection

Also we can track metrics including io thread scan time for explain analyze

explain:

                                                           PartitionedExecutionPlan: scanner=RemotePartitionsScanner { manager:
                                                             RemoteScannerManager, table_name: "sys_invocation_status" }, partitions=1,
                                                             projection=[partition_key, uuid, modified_at], predicate=DynamicFilter [
                                                             empty ], schema=[partition_key:UInt64;N, uuid:FixedSizeBinary(16);N,
                                                             modified_at:Timestamp(ms, "+00:00");N]

explain tree:

                                             ┌─────────────┴─────────────┐
                                             │  PartitionedExecutionPlan │
                                             │    --------------------   │
                                             │       partitions: 1       │
                                             │                           │
                                             │         predicate:        │
                                             │  DynamicFilter [ empty ]  │
                                             │                           │
                                             │        projection:        │
                                             │   [partition_key, uuid,   │
                                             │        modified_at]       │
                                             │                           │
                                             │          scanner:         │
                                             │ RemotePartitionsScanner { │
                                             │          manager:         │
                                             │         RemoteScan        │
                                             │  nerManager, table_name:  │
                                             │  "sys_invocation_status"  │
                                             │              }            │
                                             └───────────────────────────┘

analyze:

                    PartitionedExecutionPlan: scanner=RemotePartitionsScanner { manager: RemoteScannerManager, table_name:
                    "sys_invocation_status" }, partitions=1, projection=[id, modified_at], predicate=true, metrics=[output_rows=1.00 K,
                    elapsed_compute=4.67ms, output_bytes=71.3 KB, output_batches=8]

@github-actions
Copy link

github-actions bot commented Feb 6, 2026

Test Results

 5 files   -   2   5 suites   - 2   3m 5s ⏱️ - 7m 45s
34 tests  -  15  34 ✅  -  15  0 💤 ±0  0 ❌ ±0 
52 runs   - 158  52 ✅  - 158  0 💤 ±0  0 ❌ ±0 

Results for commit de97bfa. ± Comparison against base commit ba4317f.

This pull request removes 49 and adds 34 tests. Note that renamed tests count towards both.
dev.restate.sdktesting.tests.CallOrdering ‑ ordering(boolean[], Client)[1]
dev.restate.sdktesting.tests.CallOrdering ‑ ordering(boolean[], Client)[2]
dev.restate.sdktesting.tests.CallOrdering ‑ ordering(boolean[], Client)[3]
dev.restate.sdktesting.tests.Cancellation ‑ cancelFromAdminAPI(BlockingOperation, Client, URI)[1]
dev.restate.sdktesting.tests.Cancellation ‑ cancelFromAdminAPI(BlockingOperation, Client, URI)[2]
dev.restate.sdktesting.tests.Cancellation ‑ cancelFromAdminAPI(BlockingOperation, Client, URI)[3]
dev.restate.sdktesting.tests.Cancellation ‑ cancelFromContext(BlockingOperation, Client)[1]
dev.restate.sdktesting.tests.Cancellation ‑ cancelFromContext(BlockingOperation, Client)[2]
dev.restate.sdktesting.tests.Cancellation ‑ cancelFromContext(BlockingOperation, Client)[3]
dev.restate.sdktesting.tests.Combinators ‑ awakeableOrTimeoutUsingAwaitAny(Client)
…
dev.restate.sdktesting.tests.AwakeableIngressEndpointTest ‑ completeWithFailure(Client)
dev.restate.sdktesting.tests.AwakeableIngressEndpointTest ‑ completeWithSuccess(Client)
dev.restate.sdktesting.tests.BackwardCompatibilityTest$NewVersion ‑ completeAwakeable(Client)
dev.restate.sdktesting.tests.BackwardCompatibilityTest$NewVersion ‑ completeRetryableOperation(Client)
dev.restate.sdktesting.tests.BackwardCompatibilityTest$NewVersion ‑ proxyCallShouldBeDone(Client)
dev.restate.sdktesting.tests.BackwardCompatibilityTest$NewVersion ‑ proxyOneWayCallShouldBeDone(Client)
dev.restate.sdktesting.tests.BackwardCompatibilityTest$OldVersion ‑ createAwakeable(Client)
dev.restate.sdktesting.tests.BackwardCompatibilityTest$OldVersion ‑ startOneWayProxyCall(Client)
dev.restate.sdktesting.tests.BackwardCompatibilityTest$OldVersion ‑ startProxyCall(Client)
dev.restate.sdktesting.tests.BackwardCompatibilityTest$OldVersion ‑ startRetryableOperation(Client)
…

♻️ This comment has been updated with latest results.

@jackkleeman jackkleeman marked this pull request as ready for review February 6, 2026 21:40
@jackkleeman jackkleeman merged commit 42114b8 into main Feb 7, 2026
31 checks passed
@jackkleeman jackkleeman deleted the jk/vzwlxmnmpvmn branch February 7, 2026 11:45
@github-actions github-actions bot locked and limited conversation to collaborators Feb 7, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants