Skip to content

Commit 3432480

Browse files
AbacnAbacn
authored
Update website for roadmap and version compatibility (#36602)
* Update website for roadmap version compatibility * Update roadmap page adding Beam 3, removing deprecated roadmaps * Added version compatibility table for Java, Python, SQL * Document Calcite dialect-specific function for Beam SQL * fix typo
1 parent 5fe0d4c commit 3432480

File tree

7 files changed

+125
-14
lines changed

7 files changed

+125
-14
lines changed

‎website/www/site/content/en/documentation/dsls/sql/calcite/overview.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,3 +65,30 @@ The following table summarizes the Apache Calcite functions and operators suppor
6565
<tr><td><a href="https://calcite.apache.org/docs/reference.html#match_recognize">MATCH_RECOGNIZE</a></td><td>No</td></tr>
6666
<tr><td><a href="https://calcite.apache.org/docs/reference.html#ddl-extensions">DDL Extensions</a></td><td>See Beam SQL extension <a href="/documentation/dsls/sql/create-external-table/">CREATE EXTERNAL TABLE</a></td></tr>
6767
</table>
68+
69+
## Calcite Version Compatibility
70+
71+
Since Beam 2.17.0, Beam SQL uses a vendored Calcite that is pinned to a Apache Calcite version.
72+
73+
<table class="table table-bordered">
74+
<tr>
75+
<th>Calcite Version</th>
76+
<th>Supported Beam Versions</th>
77+
</tr>
78+
<tr>
79+
<td>1.40.0</td>
80+
<td>&ge; 2.68.0</td>
81+
</tr>
82+
<tr>
83+
<td>1.28.0</td>
84+
<td>2.35.0 - 2.67.0</td>
85+
</tr>
86+
<tr>
87+
<td>1.26.0</td>
88+
<td>2.34.0</td>
89+
</tr>
90+
<tr>
91+
<td>1.20.0</td>
92+
<td>2.16.0 - 2.33.0</td>
93+
</tr>
94+
</table>

‎website/www/site/content/en/documentation/dsls/sql/calcite/scalar-functions.md‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,14 @@ limitations under the License.
1919

2020
# Beam Calcite SQL scalar functions
2121

22-
This page documents the Apache Calcite functions supported by Beam Calcite SQL.
22+
This page documents the Apache Calcite functions supported by Beam Calcite SQL. The list is not exhausted.
23+
For a full list of Calcite builtin functions, please refer to [Apache Calcite reference](https://calcite.apache.org/docs/reference.html).
24+
Not all functions in Calcite documentations are supported.
25+
The support status depends on [Beam version](/documentation/dsls/sql/calcite/overview/#Calcite_Version_Compatibility) and support status on Calcite internals.
26+
27+
In addition to standard SQL scalar functions, Beam SQL supports Calcite's
28+
[dialect-specific](https://calcite.apache.org/docs/reference.html#dialect-specific-operators)
29+
functions by configuring pipeline option `--calciteConnectionProperties={"fun":"<value>"}` (since Apache Beam 2.67.0).
2330

2431
## Comparison functions and operators
2532

‎website/www/site/content/en/documentation/dsls/sql/zetasql/overview.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ limitations under the License.
1717
-->
1818
# Beam ZetaSQL overview
1919

20-
**Note:** Beam ZetaSQL has been deprecated ([details](https://github.com/apache/beam/issues/34423)). Please switch to use the default [Calcite SQL](/documentation/dsls/sql/calcite/overview) dialect.
20+
**Note:** ZetaSQL support has been removed in Beam 2.68.0 and newer versions ([details](https://github.com/apache/beam/issues/34423)). Please switch to use the default [Calcite SQL](/documentation/dsls/sql/calcite/overview) dialect.
2121

2222
Beam SQL supports a variant of the [ZetaSQL](https://github.com/google/zetasql) language. ZetaSQL is similar to the language in BigQuery's SQL framework. This Beam SQL dialect is especially useful in pipelines that [write to or read from BigQuery tables](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.html).
2323

‎website/www/site/content/en/documentation/sdks/java.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,3 +52,32 @@ In addition several [3rd party Java libraries](/documentation/sdks/java-thirdpar
5252
## Java multi-language pipelines quickstart
5353

5454
Apache Beam lets you combine transforms written in any supported SDK language and use them in one multi-language pipeline. To learn how to create a multi-language pipeline using the Java SDK, see the [Java multi-language pipelines quickstart](/documentation/sdks/java-multi-language-pipelines).
55+
56+
## Java Version Compatibility
57+
58+
<table class="table table-bordered">
59+
<tr>
60+
<th>Java Version</th>
61+
<th>Supported Beam Versions</th>
62+
</tr>
63+
<tr>
64+
<td>25</td>
65+
<td>&ge; 2.69.0</td>
66+
</tr>
67+
<tr>
68+
<td>21</td>
69+
<td>&ge; 2.52.0</td>
70+
</tr>
71+
<tr>
72+
<td>17</td>
73+
<td>&ge; 2.37.0</td>
74+
</tr>
75+
<tr>
76+
<td>11</td>
77+
<td>&ge; 2.29.0</td>
78+
</tr>
79+
<tr>
80+
<td>8</td>
81+
<td>2.x</td>
82+
</tr>
83+
</table>

‎website/www/site/content/en/documentation/sdks/java/euphoria.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@ For each of the assigned windows the extracted value is accumulated using a user
2929
a key/window pair.
3030
-->
3131

32+
**Note:** Beam Euphoria support has been deprecated ([details](https://github.com/apache/beam/issues/29451)) and may be removed in a future version.
33+
3234
## What is Euphoria
3335
Easy to use Java 8 API build on top of the Beam's Java SDK. API provides a [high-level abstraction](#operator-reference) of data transformations, with focus on the Java 8 language features (e.g. lambdas and streams). It is fully inter-operable with existing Beam SDK and convertible back and forth. It allows fast prototyping through use of (optional) [Kryo](https://github.com/EsotericSoftware/kryo) based coders, lambdas and high level operators and can be seamlessly integrated into existing Beam `Pipelines`.
3436

‎website/www/site/content/en/documentation/sdks/python.md‎

Lines changed: 50 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,4 +62,53 @@ Apache Beam lets you combine transforms written in any supported SDK language an
6262

6363
## Unrecoverable Errors in Beam Python
6464

65-
Some common errors can occur during worker start-up and prevent jobs from starting. To learn about these errors and how to troubleshoot them in the Python SDK, see [Unrecoverable Errors in Beam Python](/documentation/sdks/python-unrecoverable-errors).
65+
Some common errors can occur during worker start-up and prevent jobs from starting. To learn about these errors and how to troubleshoot them in the Python SDK, see [Unrecoverable Errors in Beam Python](/documentation/sdks/python-unrecoverable-errors).
66+
67+
## Python Version Compatibility
68+
69+
<table class="table table-bordered">
70+
<tr>
71+
<th>Python Version</th>
72+
<th>Supported Beam Versions</th>
73+
</tr>
74+
<tr>
75+
<td>3.13</td>
76+
<td>&ge; 2.69.0</td>
77+
</tr>
78+
<tr>
79+
<td>3.12</td>
80+
<td>&ge; 2.57.0</td>
81+
</tr>
82+
<tr>
83+
<td>3.11</td>
84+
<td>&ge; 2.47.0</td>
85+
</tr>
86+
<tr>
87+
<td>3.10</td>
88+
<td>&ge; 2.43.0</td>
89+
</tr>
90+
<tr>
91+
<td>3.9</td>
92+
<td>&ge; 2.37.0</td>
93+
</tr>
94+
<tr>
95+
<td>3.8</td>
96+
<td>2.23.0 - 2.60.0</td>
97+
</tr>
98+
<tr>
99+
<td>3.7</td>
100+
<td>2.12.0 - 2.48.0</td>
101+
</tr>
102+
<tr>
103+
<td>3.6</td>
104+
<td>2.12.0 - 2.38.0</td>
105+
</tr>
106+
<tr>
107+
<td>3.5</td>
108+
<td>2.11.0 - 2.24.0</td>
109+
</tr>
110+
<tr>
111+
<td>2.7</td>
112+
<td>&le 2.24.0</td>
113+
</tr>
114+
</table>

‎website/www/site/content/en/roadmap/_index.md‎

Lines changed: 8 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,10 @@ The major components of Beam each have their own roadmap which you can find
2727
via the menu.
2828
Below are some highlights for the project as a whole.
2929

30+
## Beam 3
31+
32+
Beam 3 is the planned first major version upgrade. See https://s.apache.org/beam3-milestones for details.
33+
3034
## Portability Framework
3135

3236
Portability is the primary Beam vision: running pipelines authored with _any SDK_
@@ -50,16 +54,15 @@ The Go SDK is not actively being developed beyond bugfixes due to lack of contri
5054

5155
## Python 3 support
5256

53-
As of Apache Beam 2.61.0, Python 3.8 support has been removed. We support python version from 3.9 uptil Python 3.12. Supporting Python 3.13 is in our roadmap.
57+
As of Apache Beam 2.69.0, we support python version from 3.9 uptil Python 3.13. Supporting Python 3.14 is in our roadmap.
5458

5559
See details on
5660
the [Python SDK's Roadmap](/roadmap/python-sdk/#python-3-support).
5761

58-
## Java 17 support
62+
## Java support
5963

60-
Java 17 is already supported and Java's next LTS (Long Term Support)
61-
version (21) is already on roadmap. See details on
62-
the [Java SDK's Roadmap](/roadmap/java-sdk).
64+
As of Beam 2.69.0, we support Java 8, 11, 17, 21, 25. Java 8 support is deprecated and scheduled for removal in Beam 3.0.0.
65+
See details on the [Java SDK's Roadmap](/roadmap/java-sdk).
6366

6467
## SQL
6568

@@ -76,9 +79,3 @@ Portable schemas enable compatibility between rows in Python and Java.
7679
A particularly interesting use case is the combination of SQL (implemented in Java)
7780
with the Python SDK via Beam's cross-language support.
7881
Learn more about portable schemas from this [presentation](https://s.apache.org/portable-schemas-seattle).
79-
80-
## Euphoria
81-
82-
Euphoria is Beam's newest API, offering a high-level, fluent style for
83-
Beam Java developers. See the [Euphoria API Roadmap](/roadmap/euphoria).
84-

0 commit comments

Comments
 (0)