Skip to content

Commit e041458

Browse files
aanejaaaneja
authored
JDK Upgrade RFC (#35)
RFC for upgrade to JDK 17
1 parent 8d4f04c commit e041458

File tree

2 files changed

+304
-0
lines changed

2 files changed

+304
-0
lines changed

‎RFC-0010-jdk-upgrade.md‎

Lines changed: 304 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,304 @@
1+
# **RFC 10 for Presto**
2+
3+
## JDK Upgrade for Presto
4+
5+
Proposers
6+
7+
* ZacBlanco
8+
* imjalpreet
9+
* aaneja
10+
11+
12+
## Summary
13+
14+
This RFC lists out the steps and process for upgrading the build and runtime JDK version used in Presto to a more recent LTS release like 17 and 21
15+
As of today (2025-01-24) the version we use is JDK 8
16+
17+
18+
## Background
19+
20+
The Java community has slowly but surely been moving to JDK versions beyond JDK 8
21+
22+
Some of our dependencies, e.g [Jetty](https://github.com/jetty/jetty.project/) have already moved to JDK 17, and declared prior versions to be out-of-support(See https://github.com/jetty/jetty.project/issues/10485. The latest supported version is Jetty 12)
23+
24+
Not using modern versions of our dependencies opens Presto installations to security risks through CVEs, which hurts adoption of Presto
25+
26+
See this [Prestocon 2024 presentation](https://www.youtube.com/watch?v=K6CjuPZtqW4&list=PLJVeO1NMmyqW_qoMMEyVq-wW8lBSxA7hP&index=10) for a deep-dive motivating the issue further (Slides [here](RFC-0010/PrestoJavaUpgradeLightningTalk.pdf))
27+
28+
29+
## Proposed Migration Plan
30+
31+
To migrate to newer versions of the JDK, we propose a phased approach -
32+
33+
| Phase | Time Window | Release Version | Runtime JDK Support | Compilation JDK Support |
34+
| --- | --- | --- | --- | --- |
35+
| Dual Support | Month N | X | JDK Y and Y+1 | JDK Y |
36+
| Dual Support & Deprecation | Month N + 2 | X+1 | JDK Y and Y+1 | JDK Y |
37+
| Stable JDK | Month N + 4 | X+2 | JDK Y+1 | JDK Y+1 |
38+
39+
_Note :
40+
JDK Y - Current JDK version (e.g. 8)
41+
JDK Y+1 - Next JDK version (e.g. 17)_
42+
43+
44+
The salient points -
45+
46+
- During the upgrade, the project will be built with two JDK versions , the current and the new target version (e.g JDK 8 and 17) simultaneously
47+
- We will make use of our CI pipelines to run the full breadth of unit, integration and end-to-end product tests on the newest JDK version that we intend to support. (e.g. in the case of 8 to 17 upgrade, we will run our CI on Java 17)
48+
- Users will be invited to test new builds with their production workloads to report any issues observed around correctness, performance and stability
49+
- As blocking issues shake out and get resolved, we will drop support for older JDK versions over one or two releases, making the new JDK version the default
50+
51+
52+
## Current Blockers
53+
We list the current blockers for the migration effort below
54+
55+
### Presto On Spark
56+
[Presto on Spark](https://github.com/prestodb/presto/issues/13856) (PoS) integrates Presto's SQL engine with Apache Spark's distributed computing capabilities, allowing Presto queries to leverage Spark's scalability and fault tolerance for large-scale batch processing workloads.
57+
58+
While the Spark project has been putting out releases supporting newer JDK versions, Presto's users may not have migrated to the latest Spark version
59+
60+
Here's is a summary of the max version that a Spark application can have, by Spark version (see [link](https://community.cloudera.com/t5/Community-Articles/Spark-and-Java-versions-Supportability-Matrix/ta-p/383669) for more details)
61+
62+
63+
| Spark Version | Supported Java Version(s) |
64+
| --- | --- |
65+
| 4.x (unreleased) | Java 8/11/17/21 |
66+
| 3.3.0 - 3.5.1 | Java 8/11/17 |
67+
| 3.0.0 - 3.2.4 | Java 8/11 |
68+
| 2.4.0 - 2.4.8 | Java 8 |
69+
70+
71+
This means, for example, if we upgrade the bytecode level of Presto to that of JDK 17, users would only be able to use PoS on Spark 3.3.0+ clusters
72+
73+
### Workaround
74+
The [presto-spark](https://github.com/prestodb/presto/tree/498784bb35feaf2787da865865ffe8ab3e8673ca/presto-spark) maven module has the below dependency graph -
75+
76+
<details>
77+
78+
```
79+
com.facebook.presto:presto-spark:jar:0.291-SNAPSHOT
80+
\- com.facebook.presto:presto-spark-base:jar:0.291-SNAPSHOT:runtime
81+
+- com.facebook.presto:presto-client:jar:0.291-SNAPSHOT:runtime
82+
| +- com.fasterxml.jackson.core:jackson-core:jar:2.15.4:runtime
83+
| +- com.fasterxml.jackson.core:jackson-databind:jar:2.15.4:runtime
84+
| +- com.facebook.airlift:security:jar:0.216:runtime
85+
| +- com.facebook.drift:drift-api:jar:1.40:runtime
86+
| +- com.google.auth:google-auth-library-oauth2-http:jar:0.12.0:runtime
87+
| | +- com.google.auth:google-auth-library-credentials:jar:0.12.0:runtime
88+
| | +- com.google.http-client:google-http-client:jar:1.27.0:runtime
89+
| | | \- org.apache.httpcomponents:httpclient:jar:4.5.5:runtime
90+
| | | +- org.apache.httpcomponents:httpcore:jar:4.4.9:runtime
91+
| | | \- commons-codec:commons-codec:jar:1.17.0:runtime
92+
| | \- com.google.http-client:google-http-client-jackson2:jar:1.27.0:runtime
93+
| +- com.squareup.okhttp3:okhttp:jar:3.9.0:runtime
94+
| | \- com.squareup.okio:okio:jar:1.13.0:runtime
95+
| \- com.squareup.okhttp3:okhttp-urlconnection:jar:3.9.0:runtime
96+
+- com.facebook.presto:presto-parser:jar:0.291-SNAPSHOT:runtime
97+
| \- org.antlr:antlr4-runtime:jar:4.7.1:runtime
98+
+- com.facebook.presto:presto-analyzer:jar:0.291-SNAPSHOT:runtime
99+
+- com.github.luben:zstd-jni:jar:1.5.2-3:runtime
100+
+- com.facebook.presto:presto-common:jar:0.291-SNAPSHOT:runtime
101+
+- com.facebook.presto:presto-spark-common:jar:0.291-SNAPSHOT:runtime
102+
+- com.facebook.presto:presto-spi:jar:0.291-SNAPSHOT:runtime
103+
+- com.facebook.presto:presto-main:jar:0.291-SNAPSHOT:runtime
104+
| +- com.esri.geometry:esri-geometry-api:jar:2.2.4:runtime
105+
| +- com.facebook.presto:presto-geospatial-toolkit:jar:0.291-SNAPSHOT:runtime
106+
| | \- org.locationtech.jts.io:jts-io-common:jar:1.19.0:runtime
107+
| | \- com.googlecode.json-simple:json-simple:jar:1.1.1:runtime
108+
| +- org.apache.commons:commons-math3:jar:3.6.1:runtime
109+
| +- com.facebook.presto:presto-bytecode:jar:0.291-SNAPSHOT:runtime
110+
| | +- org.ow2.asm:asm-tree:jar:9.2:runtime
111+
| | +- org.ow2.asm:asm-util:jar:9.2:runtime
112+
| | \- org.ow2.asm:asm-analysis:jar:9.2:runtime
113+
| +- io.airlift:aircompressor:jar:0.27:runtime
114+
| +- com.facebook.airlift:discovery:jar:0.216:runtime
115+
| +- com.facebook.airlift:event:jar:0.216:runtime
116+
| +- com.facebook.airlift:http-server:jar:0.216:runtime
117+
| | +- org.eclipse.jetty.http2:http2-server:jar:9.4.56.v20240826:runtime
118+
| | +- org.eclipse.jetty:jetty-server:jar:9.4.56.v20240826:runtime
119+
| | +- org.eclipse.jetty:jetty-servlet:jar:9.4.56.v20240826:runtime
120+
| | | \- org.eclipse.jetty:jetty-util-ajax:jar:9.4.56.v20240826:runtime
121+
| | +- org.eclipse.jetty:jetty-security:jar:9.4.56.v20240826:runtime
122+
| | \- org.eclipse.jetty:jetty-jmx:jar:9.4.56.v20240826:runtime
123+
| +- com.facebook.airlift:jaxrs:jar:0.216:runtime
124+
| | +- javax.xml.bind:jaxb-api:jar:2.3.1:runtime
125+
| | | \- javax.activation:javax.activation-api:jar:1.2.0:runtime
126+
| | +- org.glassfish.jersey.core:jersey-common:jar:2.26:runtime
127+
| | | \- org.glassfish.hk2:osgi-resource-locator:jar:1.0.1:runtime
128+
| | +- org.glassfish.jersey.core:jersey-server:jar:2.26:runtime
129+
| | | +- org.glassfish.jersey.core:jersey-client:jar:2.26:runtime
130+
| | | \- org.glassfish.jersey.media:jersey-media-jaxb:jar:2.26:runtime
131+
| | +- org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.26:runtime
132+
| | +- org.glassfish.jersey.containers:jersey-container-servlet:jar:2.26:runtime
133+
| | \- org.glassfish.jersey.inject:jersey-hk2:jar:2.26:runtime
134+
| | \- org.glassfish.hk2:hk2-locator:jar:2.5.0-b42:runtime
135+
| | +- org.glassfish.hk2:hk2-api:jar:2.5.0-b42:runtime
136+
| | +- org.glassfish.hk2:hk2-utils:jar:2.5.0-b42:runtime
137+
| | \- org.javassist:javassist:jar:3.22.0-GA:runtime
138+
| +- com.facebook.airlift:jmx:jar:0.216:runtime
139+
| | \- com.sun:tools:jar:1.8:system
140+
| +- com.facebook.airlift:jmx-http:jar:0.216:runtime
141+
| +- io.airlift.resolver:resolver:jar:1.4:runtime
142+
| | +- org.sonatype.aether:aether-spi:jar:1.13.1:runtime
143+
| | +- org.sonatype.aether:aether-impl:jar:1.13.1:runtime
144+
| | +- org.sonatype.aether:aether-util:jar:1.13.1:runtime
145+
| | +- org.sonatype.aether:aether-connector-file:jar:1.13.1:runtime
146+
| | +- org.sonatype.aether:aether-connector-asynchttpclient:jar:1.13.1:runtime
147+
| | | \- com.ning:async-http-client:jar:1.6.5:runtime
148+
| | +- io.netty:netty:jar:3.6.2.Final:runtime
149+
| | +- org.apache.maven:maven-core:jar:3.0.4:runtime
150+
| | | +- org.apache.maven:maven-settings:jar:3.0.4:runtime
151+
| | | +- org.apache.maven:maven-settings-builder:jar:3.0.4:runtime
152+
| | | +- org.apache.maven:maven-repository-metadata:jar:3.0.4:runtime
153+
| | | +- org.apache.maven:maven-plugin-api:jar:3.0.4:runtime
154+
| | | +- org.apache.maven:maven-model-builder:jar:3.0.4:runtime
155+
| | | +- org.codehaus.plexus:plexus-interpolation:jar:1.14:runtime
156+
| | | +- org.codehaus.plexus:plexus-utils:jar:2.0.6:runtime
157+
| | | +- org.codehaus.plexus:plexus-component-annotations:jar:1.5.5:runtime
158+
| | | \- org.sonatype.plexus:plexus-sec-dispatcher:jar:1.3:runtime
159+
| | +- org.apache.maven:maven-model:jar:3.0.4:runtime
160+
| | +- org.apache.maven:maven-artifact:jar:3.0.4:runtime
161+
| | +- org.apache.maven:maven-aether-provider:jar:3.0.4:runtime
162+
| | +- org.apache.maven:maven-embedder:jar:3.0.4:runtime
163+
| | | +- org.apache.maven:maven-compat:jar:3.0.4:runtime
164+
| | | | \- org.apache.maven.wagon:wagon-provider-api:jar:2.2:runtime
165+
| | | \- org.sonatype.plexus:plexus-cipher:jar:1.7:runtime
166+
| | +- org.codehaus.plexus:plexus-container-default:jar:1.5.5:runtime
167+
| | | \- org.apache.xbean:xbean-reflect:jar:3.4:runtime
168+
| | +- org.codehaus.plexus:plexus-classworlds:jar:2.4:runtime
169+
| | \- org.slf4j:slf4j-api:jar:1.7.32:runtime
170+
| +- com.facebook.airlift:trace-token:jar:0.216:runtime
171+
| +- io.airlift:joni:jar:2.1.5.3:runtime
172+
| +- com.facebook.drift:drift-server:jar:1.40:runtime
173+
| +- com.facebook.drift:drift-transport-netty:jar:1.40:runtime
174+
| | +- io.netty:netty-common:jar:4.1.115.Final:runtime
175+
| | +- io.netty:netty-buffer:jar:4.1.115.Final:runtime
176+
| | +- io.netty:netty-handler-proxy:jar:4.1.115.Final:runtime
177+
| | | +- io.netty:netty-codec-socks:jar:4.1.115.Final:runtime
178+
| | | \- io.netty:netty-codec-http:jar:4.1.115.Final:runtime
179+
| | +- io.netty:netty-handler:jar:4.1.115.Final:runtime
180+
| | | \- io.netty:netty-transport-native-unix-common:jar:4.1.115.Final:runtime
181+
| | +- io.netty:netty-transport-classes-epoll:jar:4.1.115.Final:runtime
182+
| | +- io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.115.Final:runtime
183+
| | \- io.netty:netty-codec:jar:4.1.115.Final:runtime
184+
| +- com.facebook.drift:drift-transport-spi:jar:1.40:runtime
185+
| +- com.facebook.drift:drift-client:jar:1.40:runtime
186+
| +- com.facebook.drift:drift-protocol:jar:1.40:runtime
187+
| +- com.facebook.drift:drift-codec:jar:1.40:runtime
188+
| | +- io.airlift:parameternames:jar:1.3:runtime
189+
| | \- com.facebook.airlift:bytecode:jar:1.3:runtime
190+
| +- com.facebook.drift:drift-codec-utils:jar:1.40:runtime
191+
| +- com.teradata:re2j-td:jar:1.4:runtime
192+
| +- com.facebook.airlift.discovery:discovery-server:jar:1.33:runtime
193+
| | +- com.facebook.airlift:jmx-http-rpc:jar:0.198:runtime
194+
| | +- org.iq80.leveldb:leveldb-api:jar:0.10:runtime
195+
| | \- org.iq80.leveldb:leveldb:jar:0.10:runtime
196+
| +- javax.servlet:javax.servlet-api:jar:3.1.0:runtime
197+
| +- javax.ws.rs:javax.ws.rs-api:jar:2.1:runtime
198+
| +- com.fasterxml.jackson.module:jackson-module-afterburner:jar:2.15.4:runtime
199+
| +- com.jayway.jsonpath:json-path:jar:2.9.0:runtime
200+
| | \- net.minidev:json-smart:jar:2.5.0:runtime
201+
| | \- net.minidev:accessors-smart:jar:2.5.0:runtime
202+
| +- org.sonatype.aether:aether-api:jar:1.13.1:runtime
203+
| +- org.ow2.asm:asm:jar:9.2:runtime
204+
| +- org.jgrapht:jgrapht-core:jar:1.3.1:runtime
205+
| | \- org.jheaps:jheaps:jar:0.10:runtime
206+
| +- org.apache.lucene:lucene-analyzers-common:jar:8.10.0:runtime
207+
| | \- org.apache.lucene:lucene-core:jar:8.10.0:runtime
208+
| +- org.locationtech.jts:jts-core:jar:1.19.0:runtime
209+
| +- io.jsonwebtoken:jjwt-api:jar:0.11.5:runtime
210+
| +- io.jsonwebtoken:jjwt-impl:jar:0.11.5:runtime
211+
| +- io.jsonwebtoken:jjwt-jackson:jar:0.11.5:runtime
212+
| +- org.apache.datasketches:datasketches-memory:jar:2.2.0:runtime
213+
| +- org.apache.datasketches:datasketches-java:jar:5.0.1:runtime
214+
| +- com.facebook.presto:presto-plugin-toolkit:jar:0.291-SNAPSHOT:runtime
215+
| +- io.netty:netty-transport:jar:4.1.115.Final:runtime
216+
| | \- io.netty:netty-resolver:jar:4.1.115.Final:runtime
217+
| \- com.facebook.presto:presto-ui:jar:0.291-SNAPSHOT:runtime
218+
+- com.facebook.presto:presto-expressions:jar:0.291-SNAPSHOT:runtime
219+
+- com.facebook.presto:presto-matching:jar:0.291-SNAPSHOT:runtime
220+
+- com.facebook.presto:presto-memory-context:jar:0.291-SNAPSHOT:runtime
221+
+- io.airlift:slice:jar:0.38:runtime
222+
+- io.airlift:units:jar:1.3:runtime
223+
+- com.facebook.airlift:stats:jar:0.216:runtime
224+
| \- org.hdrhistogram:HdrHistogram:jar:2.1.9:runtime
225+
+- com.facebook.airlift:json:jar:0.216:runtime
226+
| +- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:2.15.4:runtime
227+
| +- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.15.4:runtime
228+
| +- com.fasterxml.jackson.datatype:jackson-datatype-guava:jar:2.15.4:runtime
229+
| +- com.fasterxml.jackson.datatype:jackson-datatype-joda:jar:2.15.4:runtime
230+
| +- com.fasterxml.jackson.module:jackson-module-parameter-names:jar:2.15.4:runtime
231+
| \- com.fasterxml.jackson.dataformat:jackson-dataformat-smile:jar:2.15.4:runtime
232+
+- com.facebook.airlift:http-client:jar:0.216:runtime
233+
| +- ch.qos.logback:logback-core:jar:1.2.13:runtime
234+
| +- org.eclipse.jetty:jetty-client:jar:9.4.56.v20240826:runtime
235+
| +- org.eclipse.jetty:jetty-io:jar:9.4.56.v20240826:runtime
236+
| +- org.eclipse.jetty:jetty-util:jar:9.4.56.v20240826:runtime
237+
| +- org.eclipse.jetty:jetty-http:jar:9.4.56.v20240826:runtime
238+
| +- org.eclipse.jetty.http2:http2-client:jar:9.4.56.v20240826:runtime
239+
| | +- org.eclipse.jetty.http2:http2-common:jar:9.4.56.v20240826:runtime
240+
| | | \- org.eclipse.jetty.http2:http2-hpack:jar:9.4.56.v20240826:runtime
241+
| | \- org.eclipse.jetty:jetty-alpn-client:jar:9.4.56.v20240826:runtime
242+
| +- org.eclipse.jetty.http2:http2-http-client-transport:jar:9.4.56.v20240826:runtime
243+
| | \- org.eclipse.jetty:jetty-alpn-openjdk8-client:jar:9.4.56.v20240826:runtime
244+
| +- com.facebook.airlift:http-utils:jar:0.216:runtime
245+
| \- net.jodah:failsafe:jar:2.0.1:runtime
246+
+- com.fasterxml.jackson.core:jackson-annotations:jar:2.15.4:runtime
247+
+- com.google.guava:guava:jar:32.1.0-jre:runtime
248+
| +- com.google.guava:failureaccess:jar:1.0.1:runtime
249+
| +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:runtime
250+
| +- org.checkerframework:checker-qual:jar:3.37.0:runtime
251+
| +- com.google.errorprone:error_prone_annotations:jar:2.18.0:runtime
252+
| \- com.google.j2objc:j2objc-annotations:jar:2.8:runtime
253+
+- com.google.inject:guice:jar:4.2.2:runtime
254+
| \- aopalliance:aopalliance:jar:1.0:runtime
255+
+- javax.annotation:javax.annotation-api:jar:1.3.2:runtime
256+
+- javax.inject:javax.inject:jar:1:runtime
257+
+- com.facebook.airlift:concurrent:jar:0.216:runtime
258+
+- com.facebook.airlift:configuration:jar:0.216:runtime
259+
| +- org.apache.bval:bval-jsr:jar:2.0.0:runtime
260+
| \- cglib:cglib-nodep:jar:3.2.5:runtime
261+
+- com.facebook.airlift:bootstrap:jar:0.216:runtime
262+
+- com.facebook.airlift:node:jar:0.216:runtime
263+
+- com.facebook.airlift:log:jar:0.216:runtime
264+
+- com.facebook.airlift:log-manager:jar:0.216:runtime
265+
| +- org.slf4j:slf4j-jdk14:jar:1.7.32:runtime
266+
| +- org.slf4j:log4j-over-slf4j:jar:1.7.32:runtime
267+
| \- org.slf4j:jcl-over-slf4j:jar:1.7.32:runtime
268+
+- com.google.code.findbugs:jsr305:jar:3.0.2:runtime
269+
+- org.pcollections:pcollections:jar:2.1.2:runtime
270+
+- org.weakref:jmxutils:jar:1.19:runtime
271+
+- org.openjdk.jol:jol-core:jar:0.2:runtime
272+
+- joda-time:joda-time:jar:2.12.7:runtime
273+
+- javax.validation:validation-api:jar:2.0.1.Final:runtime
274+
+- it.unimi.dsi:fastutil:jar:8.5.2:runtime
275+
\- org.apache.commons:commons-text:jar:1.10.0:runtime
276+
\- org.apache.commons:commons-lang3:jar:3.12.0:runtime
277+
278+
```
279+
280+
</details>
281+
282+
By carefully limiting how and when a dependency is upgraded, we can ensure that Spark jobs using Presto are still runnable on older Spark clusters
283+
284+
For example, the Jetty dependency comes in via `presto-main`. If we can figure out a way to either
285+
286+
1. Remove this dependency or
287+
1. Replace it with an equivalent implementation,
288+
289+
we can migrate `presto-main` to a higher JDK version
290+
In this particular case, we could identify that code in PoS that uses Jetty's HttpClient can be deprecated. See [TODO issue link]() for more details on this
291+
292+
For the modules that *cannot be upgraded* to a higher bytecode level, we can refactor code to generate smaller modules that can be kept at lower bytecode levels.
293+
This can prove to be beneficial for other modules too, e.g `presto-client` can be kept at JVM 8 bytecode level to allow for a broader support for Java based clients
294+
295+
## Timeline for the project
296+
297+
A detailed timeline is out-of-scope for this RFC.
298+
We will track project milestones and timeline on https://github.com/orgs/prestodb/projects/31/
299+
300+
We have the below high-level timeline targets -
301+
- Migrate Presto-On-Spark modules to JDK 17 starting early 2026
302+
303+
_Note: Active discussions on timelines are happening on the slack channel `#jdk21-upgrade`. Please join the [PrestoDB Slack](https://communityinviter.com/apps/prestodb/prestodb) to participate_
304+

‎RFC-0010/PrestoJavaUpgradeLightningTalk.pdf‎

662 KB
Binary file not shown.

0 commit comments

Comments
 (0)