Skip to content

Use the new stdlib when generating scaladoc #23744

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 14, 2025
Merged

Use the new stdlib when generating scaladoc #23744

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
82ebe49
chore: drop patching scaladoc for previously patched symbols
hamzaremmal Aug 14, 2025
1200b81
chore: update the build to use the new stdlib
hamzaremmal Aug 14, 2025
30e6c3c
chore: publish command before testing the commands
hamzaremmal Aug 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/scaladoc.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ jobs:
- name: Compile and test
run: |
./project/scripts/sbt scaladoc/test
./project/scripts/sbt dist/Universal/stage
./project/scripts/cmdScaladocTests

- name: Locally publish self
Expand Down
84 changes: 21 additions & 63 deletions project/Build.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3116,7 +3116,7 @@ object Build {
}.value,

generateScalaDocumentation := Def.inputTaskDyn {
val majorVersion = (LocalProject("scala3-library-bootstrapped") / scalaBinaryVersion).value
val majorVersion = (`scala-library-bootstrapped` / scalaBinaryVersion).value

val extraArgs = spaceDelimited("[<output-dir>] [--justAPI]").parsed
val outputDirOverride = extraArgs.headOption.fold(identity[GenerationConfig](_))(newDir => {
Expand Down Expand Up @@ -3733,25 +3733,12 @@ object ScaladocConfigs {

def dottyExternalMapping = ".*scala/.*::scaladoc3::https://dotty.epfl.ch/api/"
def javaExternalMapping = ".*java/.*::javadoc::https://docs.oracle.com/javase/8/docs/api/"
def scalaSrcLink(v: String, s: String) = s"${s}github://scala/scala/v$v#src/library"
def dottySrcLink(v: String, sourcesPrefix: String = "", outputPrefix: String = "") =
sys.env.get("GITHUB_SHA") match {
case Some(sha) =>
s"${sourcesPrefix}github://${sys.env("GITHUB_REPOSITORY")}/$sha$outputPrefix"
case None => s"${sourcesPrefix}github://scala/scala3/$v$outputPrefix"
def defaultSourceLinks(version: String) = {
def dottySrcLink(v: String) = sys.env.get("GITHUB_SHA") match {
case Some(sha) => s"github://scala/scala3/$sha"
case None => s"github://scala/scala3/$v"
}

def defaultSourceLinks(version: String = dottyNonBootstrappedVersion, refVersion: String = dottyVersion) = Def.task {
def stdLibVersion = stdlibVersion(NonBootstrapped)
def srcManaged(v: String, s: String) = s"out/bootstrap/scala2-library-bootstrapped/scala-$v/src_managed/main/$s-library-src"
SourceLinks(
List(
scalaSrcLink(stdLibVersion, srcManaged(version, "scala") + "="),
dottySrcLink(refVersion, "library/src=", "#library/src"),
dottySrcLink(refVersion),
"docs=github://scala/scala3/main#docs"
)
)
SourceLinks(List(dottySrcLink(version), "docs=github://scala/scala3/main#docs"))
}

lazy val DefaultGenerationSettings = Def.task {
Expand All @@ -3766,17 +3753,14 @@ object ScaladocConfigs {
def skipById = SkipById(List(
"scala.runtime.stdLibPatches",
"scala.runtime.MatchCase",
"dotty.tools.tasty",
"dotty.tools.tasty.util",
"dotty.tools.tasty.besteffort"
))
def projectFooter = ProjectFooter(s"Copyright (c) 2002-$currentYear, LAMP/EPFL")
def defaultTemplate = DefaultTemplate("static-site-main")
GenerationConfig(
List(),
ProjectVersion(projectVersion),
GenerateInkuire(true),
defaultSourceLinks().value,
defaultSourceLinks(version = dottyVersion),
skipByRegex,
skipById,
projectLogo,
Expand All @@ -3798,13 +3782,8 @@ object ScaladocConfigs {
)
}

lazy val DefaultGenerationConfig = Def.task {
def distLocation = (dist / Universal / stage).value
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The build failed because this had the side effect of being executed (which is obviously an sbt bug and why I removed it) but the CI workflow relies on this buggy effect.
I'm updating the CI workflow file to publish the commands locally before testing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just checked with @eed3si9n and this is a very very interesting case as it works as expected... since the sbt semantics are not the scala semantics. Still weird for me.

DefaultGenerationSettings.value
}

lazy val Scaladoc = Def.task {
DefaultGenerationConfig.value
DefaultGenerationSettings.value
.add(UseJavacp(true))
.add(ProjectName("scaladoc"))
.add(OutputDir("scaladoc/output/self"))
Expand All @@ -3815,7 +3794,7 @@ object ScaladocConfigs {

lazy val Testcases = Def.task {
val tastyRoots = (Test / Build.testcasesOutputDir).value
DefaultGenerationConfig.value
DefaultGenerationSettings.value
.add(UseJavacp(true))
.add(OutputDir("scaladoc/output/testcases"))
.add(ProjectName("scaladoc testcases"))
Expand All @@ -3831,56 +3810,35 @@ object ScaladocConfigs {
}

lazy val Scala3 = Def.task {
val dottyJars: Seq[java.io.File] = Seq(
(`scala2-library-bootstrapped`/Compile/products).value,
(`scala3-library-bootstrapped`/Compile/products).value,
(`scala3-interfaces`/Compile/products).value,
(`tasty-core-bootstrapped`/Compile/products).value,
).flatten

val roots = dottyJars.map(_.getAbsolutePath)

val managedSources =
(`scala2-library-bootstrapped`/Compile/sourceManaged).value / "scala-library-src"
val projectRoot = (ThisBuild/baseDirectory).value.toPath
val stdLibRoot = projectRoot.relativize(managedSources.toPath.normalize())
val docRootFile = stdLibRoot.resolve("rootdoc.txt")

val dottyManagesSources = (`scala3-library-bootstrapped`/Compile/baseDirectory).value

val tastyCoreSources = projectRoot.relativize((`tasty-core-bootstrapped`/Compile/scalaSource).value.toPath().normalize())

val dottyLibRoot = projectRoot.relativize(dottyManagesSources.toPath.normalize())
DefaultGenerationConfig.value
DefaultGenerationSettings.value
.add(ProjectName("Scala 3"))
.add(OutputDir(file("scaladoc/output/scala3").getAbsoluteFile.getAbsolutePath))
.add(Revision("main"))
.add(ExternalMappings(List(javaExternalMapping)))
.add(DocRootContent(docRootFile.toString))
.add(DocRootContent(((`scala-library-bootstrapped` / baseDirectory).value / "src" / "rootdoc.txt").toString))
.add(CommentSyntax(List(
s"${dottyLibRoot}=markdown",
s"${stdLibRoot}=wiki",
s"${tastyCoreSources}=markdown",
//s"${dottyLibRoot}=markdown",
//s"${stdLibRoot}=wiki",
Comment on lines +3820 to +3821
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, we cannot have both syntax available in the same sources... for now we just kept the default one until we find a better solution.

"wiki"
)))
.add(VersionsDictionaryUrl("https://scala-lang.org/api/versions.json"))
.add(DocumentSyntheticTypes(true))
.add(SnippetCompiler(List(
s"$dottyLibRoot/src/scala=compile",
s"$dottyLibRoot/src/scala/compiletime=compile",
s"$dottyLibRoot/src/scala/util=compile",
s"$dottyLibRoot/src/scala/util/control=compile"
)))
//.add(SnippetCompiler(List(
//s"$dottyLibRoot/src/scala=compile",
//s"$dottyLibRoot/src/scala/compiletime=compile",
//s"$dottyLibRoot/src/scala/util=compile",
//s"$dottyLibRoot/src/scala/util/control=compile"
//)))
Comment on lines +3826 to +3831
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We disable the snippet compiler for now too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When can we turn it back on?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we determine the set of packages we need to enable it for.

.add(SiteRoot("docs"))
.add(ApiSubdirectory(true))
.withTargets(roots)
.withTargets((`scala-library-bootstrapped` / Compile / products).value.map(_.getAbsolutePath))
}

def stableScala3(version: String) = Def.task {
val scalaLibrarySrc = s"out/bootstrap/scala2-library-bootstrapped/scala-$version-bin-SNAPSHOT-nonbootstrapped/src_managed"
val dottyLibrarySrc = "library/src"
Scala3.value
.add(defaultSourceLinks(version + "-bin-SNAPSHOT-nonbootstrapped", version).value)
.add(defaultSourceLinks(version = version))
.add(ProjectVersion(version))
.add(SnippetCompiler(
List(
Expand Down
27 changes: 1 addition & 26 deletions scaladoc/src/dotty/tools/scaladoc/tasty/ClassLikeSupport.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ trait ClassLikeSupport:
if summon[DocContext].args.generateInkuire then doInkuireStuff(classDef)

if signatureOnly then baseMember else baseMember.copy(
members = classDef.extractPatchedMembers.sortBy(m => (m.name, m.kind.name)),
members = classDef.extractMembers.sortBy(m => (m.name, m.kind.name)),
selfType = selfType,
companion = classDef.getCompanion
)
Expand Down Expand Up @@ -267,31 +267,6 @@ trait ClassLikeSupport:
inherited.flatMap(s => parseInheritedMember(c)(s))
}

/** Extracts members while taking Dotty logic for patching the stdlib into account. */
def extractPatchedMembers: Seq[Member] = {
val ownMembers = c.extractMembers
def extractPatchMembers(sym: Symbol) = {
// NOTE for some reason scala.language$.experimental$ class doesn't show up here, so we manually add the name
val ownMemberDRIs = ownMembers.iterator.map(_.name).toSet + "experimental$"
sym.tree.asInstanceOf[ClassDef]
.membersToDocument.filterNot(m => ownMemberDRIs.contains(m.symbol.name))
.flatMap(parseMember(c))
}
c.symbol.fullName match {
case "scala.Predef$" =>
ownMembers ++
extractPatchMembers(qctx.reflect.Symbol.requiredClass("scala.runtime.stdLibPatches.Predef$"))
case "scala.language$" =>
ownMembers ++
extractPatchMembers(qctx.reflect.Symbol.requiredModule("scala.runtime.stdLibPatches.language").moduleClass)
case "scala.language$.experimental$" =>
ownMembers ++
extractPatchMembers(qctx.reflect.Symbol.requiredModule("scala.runtime.stdLibPatches.language.experimental").moduleClass)
case _ => ownMembers
}

}

Comment on lines -270 to -294
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new stdlib doesn't patch anymore, we can remove the logic here.

def getTreeOfFirstParent: Option[Tree] =
c.getParentsAsTreeSymbolTuples.headOption.map(_._1)

Expand Down
2 changes: 1 addition & 1 deletion scaladoc/src/dotty/tools/scaladoc/tasty/TastyParser.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ case class ScaladocTastyInspector()(using ctx: DocContext) extends Inspector:
def mergeAnyRefAliasAndObject(parser: TastyParser) =
import parser.qctx.reflect._
val javaLangObjectDef = defn.ObjectClass.tree.asInstanceOf[ClassDef]
val objectMembers = parser.extractPatchedMembers(javaLangObjectDef)
val objectMembers = javaLangObjectDef.extractMembers
val aM = parser.parseTypeDef(
defn.AnyRefClass.tree.asInstanceOf[TypeDef],
defn.AnyClass.tree.asInstanceOf[ClassDef],
Expand Down
Loading