Skip to content

[FLINK-36646] Test different versions of the JDK in the Flink image #910

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed

[FLINK-36646] Test different versions of the JDK in the Flink image #910

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
88b3f06
[FLINK-36646] add tests of the various different Java versions as the…
SamBarker Nov 6, 2024
3d04158
[FLINK-36646] make the flink java versions wait for the smoke test of…
SamBarker Nov 6, 2024
85e803e
[FLINK-36646] conditionally append the java version to the image ref
SamBarker Nov 6, 2024
6e599dd
[FLINK-36646] Typo fix in job name
SamBarker Nov 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,15 +83,33 @@ jobs:
flink-version: "v1_20"
http-client: ${{ matrix.http-client }}
test: test_application_operations.sh
java_rutimes:
name: Java runtimes smoke test
needs: e2e_smoke_test
strategy:
matrix:
http-client: [ "okhttp" ]
java-version: [ "11", "17"]
flink-version:
- "v1_20"
- "v1_19"
- "v1_18"
uses: ./.github/workflows/e2e.yaml
with:
java-version: ${{ matrix.java-version }}
Copy link

@robobario robobario Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if it's a bit funky that we are using java-version to control both the JDK/JRE used to build/run the operator and the runtime of the flink image. Aren't they different dimensions of the matrix?

This would show that a JDK17 operator works with a JDK17 flink image, but not cover the current default operator runtime with a JDK17 flink image.

Copy link
Contributor Author

@SamBarker SamBarker Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I had debated making a flink-tag parameter but didn't as we would then have to manually encode the matrix.

This would show that a JDK17 operator works with a JDK17 flink image, but not cover the current default operator runtime with a JDK17 flink image.

I'm not convinced that the operator JDK really matters so don't think its particularly important to have a JDK11 operator deploying a JDK 17 Flink. I can go that direction if others see it as valuable.

Copy link

@robobario robobario Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah you're right we aren't likely to see some incompatibility arising from different JRE combinations. Maybe it would be better to hold the operator JDK constant and vary only the flink runtime. Just to make it clearer which thing is varied.

Copy link
Contributor

@tomncooper tomncooper Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I agree with @SamBarker. The issue here is actually not the interplay of JDK versions but rather the config supplied by the operator's logic not being compatible (or more accuratly enabling) Java 17 support. So that's what we need to test.

flink-version: ${{ matrix.flink-version }}
http-client: ${{ matrix.http-client }}
test: test_application_operations.sh
append-java-version: true
e2e_namespace_tests:
name: Alternative namespace tests
needs: e2e_smoke_test
strategy:
matrix:
flink-version:
- "v1_20"
- "v1_18"
- "v1_19"
- "v1_18"
- "v1_17"
- "v1_16"
mode:
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/e2e.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ on:
create-namespace:
type: boolean
default: false
append-java-version:
Copy link

@robobario robobario Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe this should be an optional flink-java-version and the matrix could contain the versions we want to test

type: boolean
default: false

jobs:
e2e_test:
Expand Down Expand Up @@ -78,6 +81,9 @@ jobs:
- name: Run Flink e2e tests
run: |
FLINK_IMAGE=$(sed --regexp-extended 's/v([0-9]+)_([0-9]+)/flink:\1.\2/g' <<< ${{ inputs.flink-version }} )
if [[ "${{ inputs.append-java-version }}" == "true" ]]; then
FLINK_IMAGE=${FLINK_IMAGE}-java${{ inputs.java-version }}
fi
echo FLINK_IMAGE=${FLINK_IMAGE}
sed -i "s/image: flink:.*/image: ${FLINK_IMAGE}/" e2e-tests/data/*.yaml
sed -i "s/flinkVersion: .*/flinkVersion: ${{ inputs.flink-version }}/" e2e-tests/data/*.yaml
Expand Down