Skip to content

Commit 46a28db

Browse files
authored
fix: Document enabling comet explain plan usage in Spark (4.0) (#1176)
1 parent b9ac78b commit 46a28db

File tree

1 file changed

+10
-0
lines changed

1 file changed

+10
-0
lines changed

docs/source/user-guide/tuning.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,3 +127,13 @@ Here is a guide to some of the native metrics.
127127
| `elapsed_compute` | Total time spent in this operator, fetching batches from a JVM iterator. |
128128
| `jvm_fetch_time` | Time spent in the JVM fetching input batches to be read by this `ScanExec` instance. |
129129
| `arrow_ffi_time` | Time spent using Arrow FFI to create Arrow batches from the memory addresses returned from the JVM. |
130+
131+
## Explain Plan
132+
### Extended Explain
133+
With Spark 4.0.0 and newer, Comet can provide extended explain plan information in the Spark UI. Currently this lists
134+
reasons why Comet may not have been enabled for specific operations.
135+
To enable this, in the Spark configuration, set the following:
136+
```shell
137+
-c spark.sql.extendedExplainProviders=org.apache.comet.ExtendedExplainInfo
138+
```
139+
This will add a section to the detailed plan displayed in the Spark SQL UI page.

0 commit comments

Comments
 (0)