Enable MiMa in the new build #23910

hamzaremmal · 2025-09-12T11:46:07Z

[skip ci]

hamzaremmal · 2025-09-12T11:46:45Z

project/Build.scala

   *   - `3.(M-1).0` if `P = 0`
   */
-  val mimaPreviousDottyVersion = "3.7.0"
+  val mimaPreviousDottyVersion = "3.7.3" // for 3.8.0, we compare against 3.7.3


This is fine since we only compare backward compatibility, not forward.

Why was this change needed? Is it because of merging Scala 2.13 stdlib into this repo?
We should be really careful about this one. Let's at least store the forward compatibility issues in the gist for review. All of these should be documented here currently

I could call the fat jar 3.7.0 if that helps. I don't think it really matters if it's 3.7.3 or 3.7.0 for 3.8.0.

hamzaremmal · 2025-09-12T11:47:37Z

project/Build.scala

    Test / develocityBuildCacheClient := None,
    extraDevelocityCacheInputFiles := Seq.empty,
    extraDevelocityCacheInputFiles / outputFileStamper := FileStamper.Hash,
+    resolvers += ("Artifactory" at "https://repo.scala-lang.org/artifactory/fat-jar/"),


To be able to use MiMa, I created a fat jar that contains both Scala 3.7.3 and Scala 2.13.16 manually and pushed it to this resolver. Once we publish 3.8.0, we can tear down this system and compare against the artifacts from Maven Central.

Let's add TODO notes to related changes that would need to be adjusted (resolvers, mimaPreviousArtifacts, etc)

hamzaremmal · 2025-09-12T14:10:34Z

project/MiMaFilters.scala

+          ProblemFilters.exclude[DirectMissingMethodProblem]("scala.runtime.Tuple3Zipped.coll2$extension"),
+          ProblemFilters.exclude[DirectMissingMethodProblem]("scala.runtime.Tuple3Zipped.coll3$extension"),
+
+          // singleton case classes modules inherit AbstractFunction1??


Needs investigation

Do we have dedicated issue to track these? If no let's create one

hamzaremmal · 2025-09-12T15:06:04Z

MiMa for Scala.js stdlib has been dropped in this PR ands not replaced yet. It should be fine and we cannot add it yet since we still need to patch some files over (aka what Scala.js call library-aux). I removed it so that I don't block too much myself when I'm tearing down the old build.

hamzaremmal · 2025-09-12T15:08:52Z

project/MiMaFilters.scala

          ProblemFilters.exclude[FinalClassProblem]("scala.annotation.experimental"),
+
+          // ================================== STDLIB MIGRATION ==================================
+          // ============================ module classes are not final ============================


This category of filters will be dropped once we move to 3.8.1.

…ary-bootstrapped`

WojciechMazur · 2025-09-13T22:19:22Z

.github/workflows/stdlib.yaml

+  mima-scala-library-nonbootstrapped:
+    runs-on: ubuntu-latest
+    needs: scala-library-nonbootstrapped
+    steps:
+      - name: Git Checkout
+        uses: actions/checkout@v5
+
+      - name: Set up JDK 17
+        uses: actions/setup-java@v5
+        with:
+          distribution: 'temurin'
+          java-version: 17
+          cache: 'sbt'
+
+      - uses: sbt/setup-sbt@v1
+      - name: Report MiMa issues in `scala-library-nonbootstrapped`
+        run: ./project/scripts/sbt scala-library-nonbootstrapped/mimaReportBinaryIssues


I think we either don't use build cache of it's misconfigured becouse in the last run scala-library is built at least twice: once in scala-library-nonbootstrapped, again in mima-scala-library-nonbootstrapped and once in every project that depends on it.
Even though splitting it might be easier to manage with the current build setup it's wasting resources.
Ideally we'd like to build everything once, but it's hard to achieve. If so, let's at least minimise the overhead by making the mima checks the next step in scala-library-nonbootstrapped and other jobs.
The MiMa check itself takes only a few seconds, building artifact takes minutes.

I think we either don't use build cache
I disabled it earlier today. It is misconfigured and very unpredictable. The intention is to use the cache, but this is independent of this PR and the cache will be enabled back soon. Just need the time to investigate why it doesn't sometime invalidated some builds.

WojciechMazur · 2025-09-13T22:23:50Z

project/Build.scala

   *   - `3.(M-1).0` if `P = 0`
   */
-  val mimaPreviousDottyVersion = "3.7.0"
+  val mimaPreviousDottyVersion = "3.7.3" // for 3.8.0, we compare against 3.7.3


Why was this change needed? Is it because of merging Scala 2.13 stdlib into this repo?
We should be really careful about this one. Let's at least store the forward compatibility issues in the gist for review. All of these should be documented here currently

WojciechMazur · 2025-09-13T22:27:49Z

project/Build.scala

    Test / develocityBuildCacheClient := None,
    extraDevelocityCacheInputFiles := Seq.empty,
    extraDevelocityCacheInputFiles / outputFileStamper := FileStamper.Hash,
+    resolvers += ("Artifactory" at "https://repo.scala-lang.org/artifactory/fat-jar/"),


Let's add TODO notes to related changes that would need to be adjusted (resolvers, mimaPreviousArtifacts, etc)

WojciechMazur · 2025-09-13T22:36:47Z

project/MiMaFilters.scala

+          ProblemFilters.exclude[DirectMissingMethodProblem]("scala.runtime.Tuple3Zipped.coll2$extension"),
+          ProblemFilters.exclude[DirectMissingMethodProblem]("scala.runtime.Tuple3Zipped.coll3$extension"),
+
+          // singleton case classes modules inherit AbstractFunction1??


Do we have dedicated issue to track these? If no let's create one

WojciechMazur · 2025-09-13T22:40:27Z

project/MiMaFilters.scala

+          // TO INVESTIGATE: This constructor changed, but it is private... why complaining?
+          ProblemFilters.exclude[IncompatibleMethTypeProblem]("scala.collection.immutable.LazyList.this"),
+          // This one should be fine, public class inside private object
+          ProblemFilters.exclude[IncompatibleResultTypeProblem]("scala.collection.immutable.LazyList#LazyBuilder#DeferredState.eval"),
+
+          // MIX IN FORWARDERS ISSUE (SHOULD BE FIXED WHEN WE REMERGE THE PR)
+          ProblemFilters.exclude[NewMixinForwarderProblem]("scala.collection.StrictOptimizedSeqOps.prepended"),
+          ProblemFilters.exclude[NewMixinForwarderProblem]("scala.collection.StrictOptimizedSeqOps.appended"),
+          ProblemFilters.exclude[NewMixinForwarderProblem]("scala.collection.StrictOptimizedSeqOps.appendedAll"),
+          ProblemFilters.exclude[NewMixinForwarderProblem]("scala.collection.StrictOptimizedSeqOps.prependedAll"),
+          ProblemFilters.exclude[NewMixinForwarderProblem]("scala.collection.StrictOptimizedSeqOps.padTo"),
+          ProblemFilters.exclude[NewMixinForwarderProblem]("scala.collection.immutable.StrictOptimizedSeqOps.updated"),
+          ProblemFilters.exclude[NewMixinForwarderProblem]("scala.collection.immutable.StrictOptimizedSeqOps.patch"),
+
+          // NO IDEA FOR NOW :)
+          ProblemFilters.exclude[ReversedMissingMethodProblem]("scala.collection.mutable.ArrayDequeOps.scala$collection$mutable$ArrayDequeOps$$super$sliding"),


Same for these: Let's make an issue so we don't forget about these.
Even a single tracker issue with list of problem kinds might be good for start

I have my own todo list with this and many others things that still need to be done. I don't want to open one issue per thing as it's too long.

I get it, but also a list like this one scala-native/scala-native#3165 might be helpful to communicate progress with rest of the team and might help @Gedochao to plan or adjust the current dev cycle.

.github/workflows/ci.yaml

sjrd · 2025-09-15T11:55:56Z

project/MiMaFilters.scala

+          // These are inner objects/private classes and it is fine to not have the outer reference captured.
+          // The code that is emitted is correct within the compilation unit and does not escape it
+          ProblemFilters.exclude[DirectMissingMethodProblem]("scala.Enumeration#ValueOrdering.this"),
+          ProblemFilters.exclude[DirectMissingMethodProblem]("scala.collection.IterableOnceOps#Maximized.this"),
+          ProblemFilters.exclude[DirectMissingMethodProblem]("scala.collection.LinearSeqIterator#LazyCell.this"),
+          ProblemFilters.exclude[DirectMissingMethodProblem]("scala.collection.mutable.PriorityQueue#ResizableArrayAccess.this"),
+          ProblemFilters.exclude[DirectMissingMethodProblem]("scala.concurrent.BatchingExecutor#AbstractBatch.this"),
+          ProblemFilters.exclude[DirectMissingMethodProblem]("scala.concurrent.Channel#LinkedList.this"),
+          ProblemFilters.exclude[DirectMissingMethodProblem]("scala.io.Source#RelaxedPosition.this"),


Note to self: check those.

hamzaremmal commented Sep 12, 2025

View reviewed changes

hamzaremmal force-pushed the new-mima branch from 96dd6a2 to 98867de Compare September 12, 2025 14:18

hamzaremmal changed the title ~~Enable MiMa in the new build~~ Enable MiMa in the new build (JVM stdlib) Sep 12, 2025

hamzaremmal changed the title ~~Enable MiMa in the new build (JVM stdlib)~~ Enable MiMa in the new build Sep 12, 2025

hamzaremmal requested a review from WojciechMazur September 12, 2025 15:06

hamzaremmal assigned WojciechMazur Sep 12, 2025

hamzaremmal marked this pull request as ready for review September 12, 2025 15:06

hamzaremmal commented Sep 12, 2025

View reviewed changes

hamzaremmal force-pushed the new-mima branch 2 times, most recently from 3da880b to 53713ae Compare September 12, 2025 15:34

hamzaremmal added 10 commits September 13, 2025 20:31

chore: drop old MiMa

a52fbf1

chore: enable MiMa in scala-library-nonbootstrapped and `scala-libr…

16be37d

…ary-bootstrapped`

fix: add temporary filters for modules classes being final in the stdlib

f66baf4

chore: delete old filters from the stdlib

b6e36d8

chore: add CI jobs for MiMa in stdlib

df82900

chore: add filter for module def methods being final in Scala 3

b301b8c

chore: add filters for issue scala#22495

7b3fd49

chore: add all the remaining filters

40c58ee

chore: add MiMa checks for scala3-interfaces

101e655

chore: add MiMa checks for tasty-core

e274d0c

hamzaremmal force-pushed the new-mima branch from 53713ae to e274d0c Compare September 13, 2025 18:31

WojciechMazur reviewed Sep 13, 2025

View reviewed changes

WojciechMazur approved these changes Sep 15, 2025

View reviewed changes

hamzaremmal merged commit 7821e79 into scala:main Sep 15, 2025
48 checks passed

hamzaremmal deleted the new-mima branch September 15, 2025 10:49

sjrd reviewed Sep 15, 2025

View reviewed changes

Enable MiMa in the new build #23910

Enable MiMa in the new build #23910

Uh oh!

Conversation

hamzaremmal commented Sep 12, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hamzaremmal commented Sep 12, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!