[explain] new syntax (LIR-based, Postgres like) (#32262)

Introduces new default syntax for `EXPLAIN`, such that now (1) `EXPLAIN`
by default explains LIR plans, which have unambiguous interpretations
(unlike MIR plans), and (2) `EXPLAIN` shows information in a
Postgres-like syntax, and significantly less information than it used to
for `EXPLAIN PHYSICAL PLAN FOR` (i.e., for LIR).

You can still explain MIR plans with the old syntax using `EXPLAIN
OPTIMIZED PLAN FOR`.

You can still explain LIR plans with the old, very verbose syntax using
`EXPLAIN PHYSICAL PLAN AS VERBOSE TEXT FOR`.

Remaining TODOs:

- [x] Update docs for `mz_lir_mapping` to describe new operator names.
- [x] Write docs for new default syntax.
- [x] Write test for new default syntax.
- [x] Write changelog post.
MaterializeInc/www#1457

Some remaining questions should be resolved by follow-up PRs, per conversation with @ggevay.
MaterializeInc/database-issues#9375
MaterializeInc/database-issues#9376
MaterializeInc/database-issues#9377

### Motivation

  * This PR adds a known-desirable feature.
  #31643
  MaterializeInc/database-issues#8889

Author: mgree

doc/user/content/sql/explain-analyze.md

@@ -96,12 +96,13 @@ FOR ] -- The FOR keyword is required if the PLAN keyword is specified
 {{</tab>}}
 {{</tabs>}}
 
-Note that the `FOR` keyword is required if the `PLAN` keyword is present. In other words, the following three statements are equivalent:
+Note that the `FOR` keyword is required if the `PLAN` keyword is present. The following four statements are equivalent:
 
 ```mzsql
 EXPLAIN <explainee>;
 EXPLAIN PLAN FOR <explainee>;
-EXPLAIN OPTIMIZED PLAN FOR <explainee>;
+EXPLAIN PHYSICAL PLAN FOR <explainee>;
+EXPLAIN PHYSICAL PLAN AS TEXT FOR <explainee>;
 ```
 
 ### Explained object
@@ -138,8 +139,8 @@ Plan Stage | Description
 **RAW PLAN** | Display the raw plan; this is closest to the original SQL.
 **DECORRELATED PLAN** | Display the decorrelated but not-yet-optimized plan.
 **LOCALLY OPTIMIZED** | Display the locally optimized plan (before view inlining and access path selection). This is the final stage for regular `CREATE VIEW` optimization.
-**OPTIMIZED PLAN** | _(Default)_ Display the optimized plan.
-**PHYSICAL PLAN** | Display the physical plan; this is close but not identical to the operators shown in [`mz_introspection.mz_lir_mapping`](../../sql/system-catalog/mz_introspection/#mz_lir_mapping).
+**OPTIMIZED PLAN** | Display the optimized plan.
+**PHYSICAL PLAN** | _(Default)_ Display the physical plan; this corresponds to the operators shown in [`mz_introspection.mz_lir_mapping`](../../sql/system-catalog/mz_introspection/#mz_lir_mapping).
 
 ### Output modifiers
 
@@ -246,19 +247,38 @@ Materialize plans are directed, potentially cyclic, graphs of operators. Each op
 receives inputs from zero or more other operators and produces a single output.
 Sub-graphs where each output is consumed only once are rendered as tree-shaped fragments.
 Sub-graphs consumed more than once are represented as common table expressions (CTEs).
-In the example below, the CTE `l0` represents a linear sub-plan (a chain of `Get`,
-`Filter`, and `Project` operators) which is used in both inputs of a self-join.
+In the example below, the CTE `l0` represents a linear sub-plan (a chain of `Read` from the table `t`)
+which is used in both inputs of a self-join (`Differential Join`).
 
 ```text
-With
-  cte l0 =
-    Project (#0, #1)
-      Filter (#0 > #2)
-        ReadStorage materialize.public.t
-Return
-  Join on=(#1 = #2)
-    Get l0
-    Get l0
+> CREATE TABLE t(x INT NOT NULL, y INT NOT NULL);
+CREATE TABLE
+> EXPLAIN SELECT t1.x, t1.y
+          FROM (SELECT * FROM t WHERE x > y) AS t1,
+               (SELECT * FROM t where x > y) AS t2
+          WHERE t1.y = t2.y;
+                     Physical Plan
+--------------------------------------------------------
+ Explained Query:                                      +
+   →With                                               +
+     cte l0 =                                          +
+       →Read materialize.public.t                      +
+   →Return                                             +
+     →Differential Join %0 » %1                        +
+       Join stage %0: Lookup key #0{y} in %1           +
+       →Arrange                                        +
+         Keys: 1 arrangement available, plus raw stream+
+           Arrangement 0: #1{y}                        +
+         →Stream l0                                    +
+       →Arrange                                        +
+         Keys: 1 arrangement available, plus raw stream+
+           Arrangement 0: #0{y}                        +
+         →Read l0                                      +
+                                                       +
+ Source materialize.public.t                           +
+   filter=((#0{x} > #1{y}))                            +
+                                                       +
+ Target cluster: quickstart                            +
 ```
 
 Note that CTEs in optimized plans do not directly correspond to CTEs in your original SQL query: For example, CTEs might disappear due to inlining (i.e., when a CTE is used only once, its definition is copied to that usage site); new CTEs can appear due to the optimizer recognizing that a part of the query appears more than once (aka common subexpression elimination). Also, certain SQL-level concepts, such as outer joins or subqueries, do not have an explicit representation in optimized plans, and are instead expressed as a pattern of operators involving CTEs. CTE names are always `l0`, `l1`, `l2`, ..., and do not correspond to SQL-level CTE names.
@@ -270,7 +290,7 @@ Many operators need to refer to columns in their input. These are displayed like
 columns assigned to `Map` operators, it might be useful to request [the `arity` output modifier](#output-modifiers).
 
 Each operator can also be annotated with additional metadata. Details are shown
-by default in the `EXPLAIN PHYSICAL PLAN` output, but are hidden elsewhere. <a
+by default in the `EXPLAIN PHYSICAL PLAN` output (the default), but are hidden elsewhere. <a
 name="explain-with-join-implementations"></a>In `EXPLAIN OPTIMIZED
 PLAN`, details about the implementation in the `Join` operator can be requested
 with [the `join implementations` output modifier](#output-modifiers) (that is,
@@ -336,20 +356,28 @@ actually run).
 
 {{< tabs >}}
 
-{{< tab "In decorrelated and optimized plans (default EXPLAIN)" >}}
-{{< explain-plans/operator-table data="explain_plan_operators" planType="optimized" >}}
-{{< /tab >}}
-
 {{< tab "In fully optimized physical (LIR) plans" >}}
 {{< explain-plans/operator-table data="explain_plan_operators" planType="LIR" >}}
 {{< /tab >}}
 
+{{< tab "In decorrelated and optimized plans (default EXPLAIN)" >}}
+{{< explain-plans/operator-table data="explain_plan_operators" planType="optimized" >}}
+{{< /tab >}}
+
 {{< tab "In raw plans" >}}
 {{< explain-plans/operator-table data="explain_plan_operators" planType="raw" >}}
 {{< /tab >}}
 
 {{< /tabs >}}
 
+Operators are sometimes marked as `Fused ...`. We write this to mean that the operator is fused with its input. That is, if you see a `Fused X` operator above a `Y` operator:
+
+```
+→Fused X
+  →Y
+```
+
+Then the `X` and `Y` operators will be combined into a single, more efficient operator.
 
 ## Examples
 
@@ -489,6 +517,10 @@ EXPLAIN PHYSICAL PLAN FOR
 MATERIALIZED VIEW my_mat_view;
 ```
 
+## Debugging running dataflows
+
+The [`EXPLAIN ANALYZE`](/sql/explain-analyze/) statement will let you debug memory and cpu usage (optionally with information about worker skew) for existing indexes and materialized views in terms of their physical plan operators. It can also attribute [TopK hints](/transform-data/idiomatic-materialize-sql/top-k/#query-hints-1) to individual operators.
+
 ## Privileges
 
 The privileges required to execute this statement are:

doc/user/content/sql/explain-plan.md

@@ -279,10 +279,8 @@ A larger size cluster will provision more memory and CPU resources.
 
 ## Which part of my query runs slowly or uses a lot of memory?
 
-{{< public-preview />}}
-
 You can [`EXPLAIN`](/sql/explain-plan/) a query to see how it will be run as a
-dataflow. In particular, `EXPLAIN PHYSICAL PLAN` will show the concrete, fully
+dataflow. In particular, `EXPLAIN PHYSICAL PLAN` (the default) will show the concrete, fully
 optimized plan that Materialize will run. That plan is written in our "low-level
 intermediate representation" (LIR).