Skip to content

Commit 07fb39d

Browse files
diegomarquezpdiegomarquezp
authored
chore: introduce .kokoro templates for GraalVM jobs (#3737)
Context: [we will introduce a new GraalVM job](https://docs.google.com/document/d/1bOeGtVFLsq5ts71If5pFXCvHIeNpbtBRvF6XQfavLZs/edit?tab=t.dcjxhf429j6t#bookmark=id.3s7d4dj247hr) and will leverage the hermetic build templates to propagate this change to some of the repositories. Each PR will be followed with ad-hoc changes such as sync-repo-settings (the cannot be templated). Demo PRs * googleapis/java-bigtable#2558 * googleapis/java-bigquerystorage#2937 * googleapis/java-storage#3029 * googleapis/java-datastore#1816 * There are manual changes in `build.sh`. We will add this file to the [ignored files in owlbot.py](https://github.com/googleapis/java-datastore/blob/8ceb62b5182e30b4f771d6c1b586a22fb084c9ac/owlbot.py#L48-L57) and will raise an issue to move this logic elsewhere (if feasible). * googleapis/java-firestore#2074 * googleapis/java-logging#1789 * googleapis/java-pubsub#2389 * googleapis/java-pubsublite#1839 * googleapis/java-spanner#3821
1 parent 087597e commit 07fb39d

File tree

9 files changed

+244
-17
lines changed

9 files changed

+244
-17
lines changed

‎.github/workflows/verify_library_generation.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,8 @@ jobs:
7878
scandir: 'hermetic_build'
7979
format: tty
8080
severity: error
81+
ignore_paths:
82+
.kokoro
8183
library-generation-lint-python:
8284
runs-on: ubuntu-22.04
8385
needs: should-run-library-generation-tests

‎hermetic_build/library_generation/owlbot/templates/java_library/.github/sync-repo-settings.yaml

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,8 +49,9 @@ branchProtectionRules:
4949
- "Kokoro - Test: Integration"
5050
- "cla/google"
5151
- "OwlBot Post Processor"
52-
- "Kokoro - Test: Java GraalVM Native Image"
53-
- "Kokoro - Test: Java 17 GraalVM Native Image"
52+
- "Kokoro - Test: Java GraalVM Native Image A"
53+
- "Kokoro - Test: Java GraalVM Native Image B"
54+
- "Kokoro - Test: Java GraalVM Native Image C"
5455
# List of explicit permissions to add (additive only)
5556
permissionRules:
5657
- team: yoshi-admins

‎hermetic_build/library_generation/owlbot/templates/java_library/.github/workflows/renovate_config_check.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ on:
77

88
jobs:
99
renovate_bot_config_validation:
10-
runs-on: ubuntu-22.04
10+
runs-on: ubuntu-24.04
1111

1212
steps:
1313
- name: Checkout code
@@ -16,7 +16,7 @@ jobs:
1616
- name: Set up Node.js
1717
uses: actions/setup-node@v4
1818
with:
19-
node-version: '20'
19+
node-version: '22'
2020

2121
- name: Install Renovate and Config Validator
2222
run: |

‎hermetic_build/library_generation/owlbot/templates/java_library/.github/workflows/update_generation_config.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ on:
2121
{% raw %}
2222
jobs:
2323
update-generation-config:
24-
runs-on: ubuntu-22.04
24+
runs-on: ubuntu-24.04
2525
env:
2626
# the branch into which the pull request is merged
2727
base_branch: main

‎hermetic_build/library_generation/owlbot/templates/java_library/.kokoro/build.sh

Lines changed: 120 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
#!/bin/bash
2-
3-
# Copyright 2018 Google LLC
2+
# Copyright 2025 Google LLC
43
#
54
# Licensed under the Apache License, Version 2.0 (the "License");
65
# you may not use this file except in compliance with the License.
@@ -16,13 +15,125 @@
1615

1716
set -eo pipefail
1817

19-
cd github/synthtool
18+
## Get the directory of the build script
19+
scriptDir=$(realpath $(dirname "${BASH_SOURCE[0]}"))
20+
## cd to the parent directory, i.e. the root of the git repo
21+
cd ${scriptDir}/..
22+
23+
# include common functions
24+
source ${scriptDir}/common.sh
25+
26+
# Print out Maven & Java version
27+
mvn -version
28+
echo ${JOB_TYPE}
29+
30+
# attempt to install 3 times with exponential backoff (starting with 10 seconds)
31+
retry_with_backoff 3 10 \
32+
mvn install -B -V -ntp \
33+
-DskipTests=true \
34+
-Dclirr.skip=true \
35+
-Denforcer.skip=true \
36+
-Dmaven.javadoc.skip=true \
37+
-Dgcloud.download.skip=true \
38+
-T 1C
39+
40+
# if GOOGLE_APPLICATION_CREDENTIALS is specified as a relative path, prepend Kokoro root directory onto it
41+
if [[ ! -z "${GOOGLE_APPLICATION_CREDENTIALS}" && "${GOOGLE_APPLICATION_CREDENTIALS}" != /* ]]; then
42+
export GOOGLE_APPLICATION_CREDENTIALS=$(realpath ${KOKORO_GFILE_DIR}/${GOOGLE_APPLICATION_CREDENTIALS})
43+
fi
44+
{% if metadata['repo']['api_shortname'] == 'storage' %}
45+
46+
export TEST_UNIVERSE_DOMAIN_CREDENTIAL=$(realpath ${KOKORO_GFILE_DIR}/secret_manager/client-library-test-universe-domain-credential)
47+
export TEST_UNIVERSE_DOMAIN=$(gcloud secrets versions access latest --project cloud-devrel-kokoro-resources --secret=client-library-test-universe-domain)
48+
export TEST_UNIVERSE_PROJECT_ID=$(gcloud secrets versions access latest --project cloud-devrel-kokoro-resources --secret=client-library-test-universe-project-id)
49+
export TEST_UNIVERSE_LOCATION=$(gcloud secrets versions access latest --project cloud-devrel-kokoro-resources --secret=client-library-test-universe-storage-location)
50+
{% endif %}
51+
52+
RETURN_CODE=0
53+
set +e
54+
55+
case ${JOB_TYPE} in
56+
test)
57+
echo "SUREFIRE_JVM_OPT: ${SUREFIRE_JVM_OPT}"
58+
mvn test -B -ntp -Dclirr.skip=true -Denforcer.skip=true ${SUREFIRE_JVM_OPT}
59+
RETURN_CODE=$?
60+
;;
61+
lint)
62+
mvn com.coveo:fmt-maven-plugin:check -B -ntp
63+
RETURN_CODE=$?
64+
;;
65+
javadoc)
66+
mvn javadoc:javadoc javadoc:test-javadoc -B -ntp
67+
RETURN_CODE=$?
68+
;;
69+
integration)
70+
mvn -B ${INTEGRATION_TEST_ARGS} \
71+
-ntp \
72+
-Penable-integration-tests \
73+
-DtrimStackTrace=false \
74+
-Dclirr.skip=true \
75+
-Denforcer.skip=true \
76+
-Dcheckstyle.skip=true \
77+
-DskipUnitTests=true \
78+
-fae \
79+
verify
80+
RETURN_CODE=$?
81+
;;
82+
graalvm)
83+
# Run Unit and Integration Tests with Native Image
84+
mvn -B ${INTEGRATION_TEST_ARGS} -ntp -Pnative test
85+
RETURN_CODE=$?
86+
;;
87+
samples)
88+
SAMPLES_DIR=samples
89+
# only run ITs in snapshot/ on presubmit PRs. run ITs in all 3 samples/ subdirectories otherwise.
90+
if [[ ! -z ${KOKORO_GITHUB_PULL_REQUEST_NUMBER} ]]
91+
then
92+
SAMPLES_DIR=samples/snapshot
93+
fi
94+
95+
if [[ -f ${SAMPLES_DIR}/pom.xml ]]
96+
then
97+
for FILE in ${KOKORO_GFILE_DIR}/secret_manager/*-samples-secrets; do
98+
[[ -f "$FILE" ]] || continue
99+
source "$FILE"
100+
done
101+
102+
pushd ${SAMPLES_DIR}
103+
mvn -B \
104+
-ntp \
105+
-DtrimStackTrace=false \
106+
-Dclirr.skip=true \
107+
-Denforcer.skip=true \
108+
-fae \
109+
verify
110+
RETURN_CODE=$?
111+
popd
112+
else
113+
echo "no sample pom.xml found - skipping sample tests"
114+
fi
115+
;;
116+
clirr)
117+
mvn -B -ntp -Denforcer.skip=true clirr:check
118+
RETURN_CODE=$?
119+
;;
120+
*)
121+
;;
122+
esac
123+
124+
if [ "${REPORT_COVERAGE}" == "true" ]
125+
then
126+
bash ${KOKORO_GFILE_DIR}/codecov.sh
127+
fi
20128

21-
# Disable buffering, so that the logs stream through.
22-
export PYTHONUNBUFFERED=1
129+
# fix output location of logs
130+
bash .kokoro/coerce_logs.sh
23131

24-
# Run tests
25-
nox -s lint test
132+
if [[ "${ENABLE_FLAKYBOT}" == "true" ]]
133+
then
134+
chmod +x ${KOKORO_GFILE_DIR}/linux_amd64/flakybot
135+
${KOKORO_GFILE_DIR}/linux_amd64/flakybot -repo=googleapis/java-{{ metadata['repo']['api_shortname'] }}
136+
fi
26137

27-
# remove all files, preventing kokoro from trying to sync them.
28-
rm -rf *
138+
echo "exiting with ${RETURN_CODE}"
139+
exit ${RETURN_CODE}

‎hermetic_build/library_generation/owlbot/templates/java_library/.kokoro/presubmit/graalvm-a.cfg

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Format: //devtools/kokoro/config/proto/build.proto
2+
3+
# Configure the docker image for kokoro-trampoline.
4+
env_vars: {
5+
key: "TRAMPOLINE_IMAGE"
6+
value: "gcr.io/cloud-devrel-public-resources/graalvm_sdk_platform_b:3.45.1"
7+
}
8+
9+
env_vars: {
10+
key: "JOB_TYPE"
11+
value: "graalvm"
12+
}
13+
14+
# TODO: remove this after we've migrated all tests and scripts
15+
env_vars: {
16+
key: "GCLOUD_PROJECT"
17+
value: "gcloud-devel"
18+
}
19+
20+
env_vars: {
21+
key: "GOOGLE_CLOUD_PROJECT"
22+
value: "gcloud-devel"
23+
}
24+
25+
env_vars: {
26+
key: "GOOGLE_APPLICATION_CREDENTIALS"
27+
value: "secret_manager/java-it-service-account"
28+
}
29+
30+
env_vars: {
31+
key: "SECRET_MANAGER_KEYS"
32+
value: "java-it-service-account"
33+
}
34+
35+
env_vars: {
36+
key: "IT_SERVICE_ACCOUNT_EMAIL"
37+
value: "[email protected]"
38+
}

‎hermetic_build/library_generation/owlbot/templates/java_library/.kokoro/presubmit/graalvm-b.cfg

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Format: //devtools/kokoro/config/proto/build.proto
2+
3+
# Configure the docker image for kokoro-trampoline.
4+
env_vars: {
5+
key: "TRAMPOLINE_IMAGE"
6+
value: "gcr.io/cloud-devrel-public-resources/graalvm_sdk_platform_b:3.45.1"
7+
}
8+
9+
env_vars: {
10+
key: "JOB_TYPE"
11+
value: "graalvm"
12+
}
13+
14+
# TODO: remove this after we've migrated all tests and scripts
15+
env_vars: {
16+
key: "GCLOUD_PROJECT"
17+
value: "gcloud-devel"
18+
}
19+
20+
env_vars: {
21+
key: "GOOGLE_CLOUD_PROJECT"
22+
value: "gcloud-devel"
23+
}
24+
25+
env_vars: {
26+
key: "GOOGLE_APPLICATION_CREDENTIALS"
27+
value: "secret_manager/java-it-service-account"
28+
}
29+
30+
env_vars: {
31+
key: "SECRET_MANAGER_KEYS"
32+
value: "java-it-service-account"
33+
}
34+
35+
env_vars: {
36+
key: "IT_SERVICE_ACCOUNT_EMAIL"
37+
value: "[email protected]"
38+
}

‎hermetic_build/library_generation/owlbot/templates/java_library/.kokoro/presubmit/graalvm-c.cfg

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Format: //devtools/kokoro/config/proto/build.proto
2+
3+
# Configure the docker image for kokoro-trampoline.
4+
env_vars: {
5+
key: "TRAMPOLINE_IMAGE"
6+
value: "gcr.io/cloud-devrel-public-resources/graalvm_sdk_platform_b:3.45.1"
7+
}
8+
9+
env_vars: {
10+
key: "JOB_TYPE"
11+
value: "graalvm"
12+
}
13+
14+
# TODO: remove this after we've migrated all tests and scripts
15+
env_vars: {
16+
key: "GCLOUD_PROJECT"
17+
value: "gcloud-devel"
18+
}
19+
20+
env_vars: {
21+
key: "GOOGLE_CLOUD_PROJECT"
22+
value: "gcloud-devel"
23+
}
24+
25+
env_vars: {
26+
key: "GOOGLE_APPLICATION_CREDENTIALS"
27+
value: "secret_manager/java-it-service-account"
28+
}
29+
30+
env_vars: {
31+
key: "SECRET_MANAGER_KEYS"
32+
value: "java-it-service-account"
33+
}
34+
35+
env_vars: {
36+
key: "IT_SERVICE_ACCOUNT_EMAIL"
37+
value: "[email protected]"
38+
}

‎hermetic_build/library_generation/setup.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,10 +25,9 @@
2525
"owlbot/templates/clirr/*.j2",
2626
"owlbot/templates/poms/*.j2",
2727
"owlbot/templates/java_library/.github/**/*",
28-
# TODO: uncomment this line after https://github.com/googleapis/sdk-platform-java/pull/3723
29-
# has been merged.
30-
# "owlbot/templates/java_library/.kokoro/**/*",
3128
"owlbot/templates/java_library/**/*",
29+
"owlbot/templates/java_library/.kokoro/build.sh",
30+
"owlbot/templates/java_library/.kokoro/presubmit/graalvm-*.cfg",
3231
],
3332
"synthtool": ["owlbot/synthtool/**/*"],
3433
},

0 commit comments

Comments
 (0)