Skip to content

Commit 6265cec

Browse files
derrickburnsclaude
andcommitted
feat: enhance cross-version persistence CI to test all model types
Enhanced persistence-cross job to test all 4 model types in matrix: - GeneralizedKMeansModel - KMedoidsModel - SoftKMeansModel - StreamingKMeansModel Each model type now tested across: - Scala 2.12 ↔ 2.13 (bidirectional) - Spark 3.4.0 ↔ 3.5.1 (bidirectional) This validates PersistenceLayoutV1's cross-version compatibility guarantee for all model types with their unique state (medoids, weights, soft params). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent 04a9ffc commit 6265cec

File tree

1 file changed

+22
-9
lines changed

1 file changed

+22
-9
lines changed

.github/workflows/ci.yml

Lines changed: 22 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -112,10 +112,19 @@ jobs:
112112
run: sbt ++${{ env.SCALA_213 }} "runMain examples.PersistenceRoundTrip load /tmp/test-model"
113113

114114
# Job 4: Cross-version persistence test (save in 2.12, load in 2.13, vice versa)
115+
# Tests all 4 model types: GeneralizedKMeans, KMedoids, SoftKMeans, StreamingKMeans
115116
persistence-cross:
116117
runs-on: ubuntu-latest
117118
needs: build
118-
name: Cross-version Persistence
119+
strategy:
120+
fail-fast: false
121+
matrix:
122+
algo:
123+
- { name: "GeneralizedKMeans", main: "examples.PersistenceRoundTrip" }
124+
- { name: "KMedoids", main: "examples.PersistenceRoundTripKMedoids" }
125+
- { name: "SoftKMeans", main: "examples.PersistenceRoundTripSoftKMeans" }
126+
- { name: "StreamingKMeans", main: "examples.PersistenceRoundTripStreamingKMeans" }
127+
name: Cross-version Persistence (${{ matrix.algo.name }})
119128
steps:
120129
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
121130
- uses: actions/setup-java@8df1039502a15bceb9433410b1a100fbe190c53b # v4.5.0
@@ -129,19 +138,23 @@ jobs:
129138
with:
130139
name: jars
131140
path: target
132-
- name: Save model with Scala 2.12
141+
- name: Test 2.12 → 2.13 (Scala version compatibility)
133142
run: |
134143
mkdir -p /tmp/persistence-test
135-
sbt ++${{ env.SCALA_212 }} "runMain examples.PersistenceRoundTrip save /tmp/persistence-test/model-212"
136-
- name: Load model with Scala 2.13
144+
sbt ++${{ env.SCALA_212 }} "runMain ${{ matrix.algo.main }} save /tmp/persistence-test/model-212"
145+
sbt ++${{ env.SCALA_213 }} "runMain ${{ matrix.algo.main }} load /tmp/persistence-test/model-212"
146+
- name: Test 2.13 → 2.12 (reverse Scala compatibility)
137147
run: |
138-
sbt ++${{ env.SCALA_213 }} "runMain examples.PersistenceRoundTrip load /tmp/persistence-test/model-212"
139-
- name: Save model with Scala 2.13
148+
sbt ++${{ env.SCALA_213 }} "runMain ${{ matrix.algo.main }} save /tmp/persistence-test/model-213"
149+
sbt ++${{ env.SCALA_212 }} "runMain ${{ matrix.algo.main }} load /tmp/persistence-test/model-213"
150+
- name: Test Spark 3.4 → 3.5 (Spark version compatibility)
140151
run: |
141-
sbt ++${{ env.SCALA_213 }} "runMain examples.PersistenceRoundTrip save /tmp/persistence-test/model-213"
142-
- name: Load model with Scala 2.12
152+
sbt ++${{ env.SCALA_213 }} -Dspark.version=3.4.0 "runMain ${{ matrix.algo.main }} save /tmp/persistence-test/model-spark34"
153+
sbt ++${{ env.SCALA_213 }} -Dspark.version=3.5.1 "runMain ${{ matrix.algo.main }} load /tmp/persistence-test/model-spark34"
154+
- name: Test Spark 3.5 → 3.4 (reverse Spark compatibility)
143155
run: |
144-
sbt ++${{ env.SCALA_212 }} "runMain examples.PersistenceRoundTrip load /tmp/persistence-test/model-213"
156+
sbt ++${{ env.SCALA_213 }} -Dspark.version=3.5.1 "runMain ${{ matrix.algo.main }} save /tmp/persistence-test/model-spark35"
157+
sbt ++${{ env.SCALA_213 }} -Dspark.version=3.4.0 "runMain ${{ matrix.algo.main }} load /tmp/persistence-test/model-spark35"
145158
146159
# Job 5: Performance sanity check (ensure no major regression)
147160
perf-sanity:

0 commit comments

Comments
 (0)