Separate signature and parameter documentation using swift-markdown #2292

a7medev · 2025-09-13T13:50:49Z

Description

Separate signature and parameter documentation by parsing the documentation returned by sourcekitd using swift-markdown and extracting the parameter documentation.

Alternatives considered

The parameter extraction implementation is almost ported from the implementation in the Swift compiler codebase. The problem with doing that in the Swift compiler codebase is that once you parse a the comment as markdown into a Document you cannot easily convert it back into markdown (we'd need to write our own markdown formatter). Besides, cmark doesn't handle Doxygen commands.

We considered using swift-docc but we faced some problems with it:

It's not easy to use. You need to retrieve a symbol graph (typically through a cursor info request) and use that along with other info to interact with DocC.
The result returned by DocC can't be directly converted to markdown, we'd need to provide our own DocC markdown renderer.

Implementing this using swift-markdown allows us to easily parse the comment, process it, convert it back to markdown and it also provides minimal parsing for Doxygen commands (we're only interested in \param) allowing us to use the same implementation for Clang-based declarations too.

Although this approach involves code duplication, it's works for our simple use case and we can probably improve it a bit in the future.

Dependencies

Add preceding newlines before Doxygen commands swift-markdown#237: Fixes a bug in swift-markdown where Doxygen commands are formatted without preceding newlines.
[IDE] Add internal parameter names in signature help swift#84272: Adds the parameter name to the signature's parameters allowing matching parameters with their documentation using their name.

a7medev · 2025-09-13T14:02:16Z

Sources/SwiftLanguageService/ParametersDocumentationExtractor.swift

+    let parametersPrefix = "parameters:"
+    let headingContent = headingText.string.trimmingCharacters(in: .whitespaces)
+
+    guard headingContent.lowercased().hasPrefix(parametersPrefix) else {


We check for a prefix instead of headingContent == parametersPrefix to match the behavior in Xcode.

a7medev · 2025-09-13T14:08:02Z

swiftlang/swift-markdown#237
swiftlang/swift#84272

@swift-ci please test

a7medev · 2025-09-13T16:57:51Z

swiftlang/swift-markdown#237
swiftlang/swift#84272

@swift-ci please test

a7medev · 2025-09-13T18:23:06Z

swiftlang/swift-markdown#237
swiftlang/swift#84272

@swift-ci please test Windows

a7medev · 2025-09-13T23:29:12Z

swiftlang/swift-markdown#237
swiftlang/swift#84272

@swift-ci please test

a7medev · 2025-09-13T23:35:02Z

swiftlang/swift-markdown#237
swiftlang/swift#84272

@swift-ci please test Windows

ahoppen

Very cool. I would have preferred to use docc here but maybe this is the simpler approach.

Do you know how hard it would be to get the symbol graph for a completion item, similar to how we get the documentation comment of a completion item. I’m just wondering that maybe it wouldn’t be all to hard to do so. But I haven’t tried.

Sources/SwiftLanguageService/ParametersDocumentationExtractor.swift

Sources/SwiftLanguageService/SignatureHelp.swift

Sources/SwiftLanguageService/ParametersDocumentationExtractor.swift

Tests/SourceKitLSPTests/SwiftSignatureHelpTests.swift

Sources/SwiftLanguageService/ParametersDocumentationExtractor.swift

a7medev · 2025-09-14T18:58:25Z

Do you know how hard it would be to get the symbol graph for a completion item, similar to how we get the documentation comment of a completion item. I’m just wondering that maybe it wouldn’t be all to hard to do so. But I haven’t tried.

@ahoppen That should be doable using a cursor info request provided the USR I think. There are other things that DocC needs (and I'm not entirely aware of what each of them is used for/represents). Besides, conversion from DocC response to markdown would require implementing a renderer within SourceKit-LSP (maybe that should be done in the long-term anyway but I'm not sure if it fits in the timeline here).

I can attempt to use DocC again and see how it goes though. Last time I tried it wasn't a very pleasant experience. 😅

QQ, tests are failing on Windows due to issues with CMakeLists.txt, is there any documentation for how the CMake configuration with Swift is structured and how that maps to the information in Package.swift? It'd also be great if I can test the build locally on macOS before running the Windows tests (which take too long), is that doable?

hamishknight · 2025-09-15T13:45:26Z

Sources/SwiftLanguageService/ParametersDocumentationExtractor.swift

+    let name = String(components[0]).trimmingCharacters(in: .whitespaces)
+    guard !name.isEmpty else {
+      return nil
+    }


What about raw identifiers e.g - `foo bar`: hello?

Added, can you please recheck? 🙏🏼

a7medev · 2025-09-17T20:11:46Z

@ahoppen @hamishknight I added the discussion for why we didn't use swift-docc, eliminated mutable state in ParametersDocumentationExtractor, added support for raw identifiers, and added more tests. Can you please recheck? 🙏🏼

a7medev · 2025-09-17T20:12:21Z

swiftlang/swift#84302
swiftlang/swift-markdown#237

@swift-ci please test

a7medev · 2025-09-18T10:53:31Z

swiftlang/swift#84302
swiftlang/swift-markdown#237

@swift-ci please test Windows

ahoppen · 2025-09-20T13:42:05Z

That should be doable using a cursor info request provided the USR I think. There are other things that DocC needs (and I'm not entirely aware of what each of them is used for/represents). Besides, conversion from DocC response to markdown would require implementing a renderer within SourceKit-LSP (maybe that should be done in the long-term anyway but I'm not sure if it fits in the timeline here).

Definitely out-of-scope for this PR, I would just like to explore future options so we can decide whether it is something we want to investigate in the future and file an actionable issue. Do you know if it would be possible to get the symbol graph of a completion item without having to go through a cursor info (@hamishknight, maybe you know)? Because if it was, I think the main thing to do would be to implement the symbol graph to Markdown renderer, which while being a bit of work might come in handy in the future anyway.

ahoppen

Thanks for adding the test cases and refining this. I have a few more comments on the implementation, the behavior looks great to me.

Sources/SwiftLanguageService/ParametersDocumentationExtractor.swift

Sources/SwiftLanguageService/SignatureHelp.swift

Tests/SourceKitLSPTests/ParametersDocumentationExtractorTests.swift

ahoppen

I’ve got a few more nitpicky comments, otherwise this looks great to me now.

Sources/SwiftLanguageService/ExtractParametersDocumentation.swift

Sources/SwiftLanguageService/ParametersDocumentationExtractor.swift

…tTextContent parameter

a7medev · 2025-09-25T07:53:11Z

swiftlang/swift-markdown#237
swiftlang/swift#84272

@swift-ci please test Windows

a7medev · 2025-09-25T17:46:44Z

swiftlang/swift-markdown#237
swiftlang/swift#84272

@swift-ci please test Windows

a7medev · 2025-09-25T17:54:45Z

swiftlang/swift#84302
swiftlang/swift-markdown#237

@swift-ci please test Windows

a7medev · 2025-09-26T10:01:32Z

@ahoppen @hamishknight @bnbarham The Windows build is still failing even after linking SwiftMarkdown in SwiftLanguageService's CMakeLists.txt and the modifications in swiftlang/swift#84302 producing the following error.

Building for production...
[0/28] Write sources
[18/28] Copying INPUTS
[19/28] Compiling CSKTestSupport CSKTestSupport.c
[20/28] Write swift-version-773C61ADED4095D8.txt
[21/28] Copying INPUTS
[23/30] Compiling CompletionScoringTestSupport RepeatableRandomNumberGenerator.swift
[24/32] Compiling CompletionScoringPerfTests CandidateBatchPerfTests.swift
[25/32] Compiling CompletionScoringTests BinaryCodingTests.swift
[26/32] Compiling SKTestSupport Assertions.swift
<unknown>:0: error: missing required module 'CAtomic'

error: fatalError
Error: Error: swift exited with code 1.
Invocation:
  swift test --scratch-path T:\x86_64-unknown-windows-msvc\SourceKitLSPTests --package-path C:\Users\swift-ci\jenkins\workspace\sourcekit-lsp-PR-windows\sourcekit-lsp -c release -debug-info-format none --parallel -Xcc -IC:\Users\swift-ci\jenkins\workspace\sourcekit-lsp-PR-windows\swift-corelibs-libdispatch -Xcc -IC:\Users\swift-ci\jenkins\workspace\sourcekit-lsp-PR-windows\swift-corelibs-libdispatch\src\BlocksRuntime -Xswiftc -IT:\5\lib\swift\host -Xswiftc -LT:\5\lib\swift\host -Xswiftc -IC:\Users\swift-ci\jenkins\workspace\sourcekit-lsp-PR-windows\cmark\src\include -Xswiftc -IC:\Users\swift-ci\jenkins\workspace\sourcekit-lsp-PR-windows\cmark\extensions\include -Xlinker -IC:\Users\swift-ci\jenkins\workspace\sourcekit-lsp-PR-windows\cmark\extensions\include -Xlinker T:\x86_64-unknown-windows-msvc\cmark-gfm-0.29.0.gfm.13\src\cmark-gfm.lib -Xlinker T:\x86_64-unknown-windows-msvc\cmark-gfm-0.29.0.gfm.13\extensions\cmark-gfm-extensions.lib -Xswiftc -IC:\Users\swift-ci\jenkins\workspace\sourcekit-lsp-PR-windows\swift-system\Sources\CSystem\include -Xswiftc -IT:\x86_64-unknown-windows-msvc\System\swift -Xlinker -LT:\x86_64-unknown-windows-msvc\System\lib -Xswiftc -IT:\x86_64-unknown-windows-msvc\ToolsSupportCore\swift -Xlinker -LT:\x86_64-unknown-windows-msvc\ToolsSupportCore\lib -Xswiftc -IC:\Users\swift-ci\jenkins\workspace\sourcekit-lsp-PR-windows\llbuild\products\libllbuild\include -Xswiftc -IT:\x86_64-unknown-windows-msvc\LLBuild\products\llbuildSwift -Xlinker -LT:\x86_64-unknown-windows-msvc\LLBuild\lib -Xswiftc -IT:\x86_64-unknown-windows-msvc\ArgumentParser\swift -Xlinker -LT:\x86_64-unknown-windows-msvc\ArgumentParser\lib -Xswiftc -IT:\x86_64-unknown-windows-msvc\Crypto\swift -Xlinker -LT:\x86_64-unknown-windows-msvc\Crypto\lib -Xlinker T:\x86_64-unknown-windows-msvc\Crypto\lib\libCCryptoBoringSSL.lib -Xswiftc -IT:\x86_64-unknown-windows-msvc\ASN1\swift -Xlinker -LT:\x86_64-unknown-windows-msvc\ASN1\lib -Xswiftc -IT:\x86_64-unknown-windows-msvc\PackageManager\swift -Xlinker -LT:\x86_64-unknown-windows-msvc\PackageManager\lib -Xswiftc -IC:\Users\swift-ci\jenkins\workspace\sourcekit-lsp-PR-windows\swift-markdown\Sources\CAtomic\inclde -Xlinker T:\x86_64-unknown-windows-msvc\Markdown\lib\CAtomic.lib -Xswiftc -IT:\x86_64-unknown-windows-msvc\Markdown\swift -Xlinker -LT:\x86_64-unknown-windows-msvc\Markdown\lib -Xswiftc -IT:\x86_64-unknown-windows-msvc\Format\swift -Xlinker -LT:\x86_64-unknown-windows-msvc\Format\lib -Xswiftc -IT:\x86_64-unknown-windows-msvc\IndexStoreDB\swift -Xlinker -LT:\x86_64-unknown-windows-msvc\IndexStoreDB\Sources\IndexStoreDB -Xlinker T:\x86_64-unknown-windows-msvc\IndexStoreDB\Sources\IndexStoreDB_CIndexStoreDB\CIndexStoreDB.lib -Xlinker T:\x86_64-unknown-windows-msvc\IndexStoreDB\Sources\IndexStoreDB_Core\Core.lib -Xlinker T:\x86_64-unknown-windows-msvc\IndexStoreDB\Sources\IndexStoreDB_Database\Database.lib -Xlinker T:\x86_64-unknown-windows-msvc\IndexStoreDB\Sources\IndexStoreDB_Index\Index.lib -Xlinker T:\x86_64-unknown-windows-msvc\IndexStoreDB\Sources\IndexStoreDB_LLVMSupport\LLVMSupport.lib -Xlinker T:\x86_64-unknown-windows-msvc\IndexStoreDB\Sources\IndexStoreDB_Support\Support.lib -Xlinker T:\x86_64-unknown-windows-msvc\LMDB\lib\CLMDB.lib -Xswiftc -IC:\Users\swift-ci\jenkins\workspace\sourcekit-lsp-PR-windows\sourcekit-lsp\Sources\CAtomics\include -Xswiftc -IC:\Users\swift-ci\jenkins\workspace\sourcekit-lsp-PR-windows\sourcekit-lsp\Sources\CSourcekitd\include -Xlinker T:\x86_64-unknown-windows-msvc\SourceKitLSP\lib\CSourcekitd.lib -Xswiftc -IC:\Users\swift-ci\jenkins\workspace\sourcekit-lsp-PR-windows\sourcekit-lsp\Sources\CCompletionScoring\include -Xswiftc -IT:\x86_64-unknown-windows-msvc\SourceKitLSP\swift -Xlinker -LT:\x86_64-unknown-windows-msvc\SourceKitLSP\lib

After some investigation, it can't find CAtomic (singular) which is a target defined in swift-markdown (not to be confused with SourceKit-LSP's CAtomics target). I checked swift-format's CMakeLists.txt files configuration for swift-markdown and it doesn't add any special handling for CAtomic so I'm not sure what should be done here.

Do you have any specific suggestions here? Thanks in advance.

hamishknight · 2025-09-26T12:37:20Z

Ah looks like there's a typo in build.ps1:

https://github.com/swiftlang/swift/blob/aa1f0009a1fc3ca9796807997fb2b92f066ebd49/utils/build.ps1#L3742

Should be include not inclde. Note also the error here is related to the SwiftPM build, not CMake build

a7medev · 2025-09-26T15:34:00Z

swiftlang/swift#84302
swiftlang/swift-markdown#237

@swift-ci please test Windows

a7medev · 2025-09-26T21:01:22Z

swiftlang/swift#84302
swiftlang/swift-markdown#237

@swift-ci please test macOS
@swift-ci please test Linux

a7medev · 2025-09-26T21:03:36Z

swiftlang/swift#84302
swiftlang/swift-markdown#237

@swift-ci please test macOS

a7medev requested review from ahoppen, bnbarham, hamishknight and rintaro as code owners September 13, 2025 13:50

a7medev mentioned this pull request Sep 13, 2025

[IDE] Add internal parameter names in signature help swiftlang/swift#84272

Merged

a7medev commented Sep 13, 2025

View reviewed changes

a7medev force-pushed the feat/signature-help-separate-param-doc branch from b94a4a3 to da27530 Compare September 13, 2025 16:57

Separate signature and parameter documentation using swift-markdown

0ac09ca

a7medev force-pushed the feat/signature-help-separate-param-doc branch from da27530 to 0ac09ca Compare September 13, 2025 23:28

ahoppen reviewed Sep 14, 2025

View reviewed changes

hamishknight reviewed Sep 15, 2025

View reviewed changes

Correctly add swift-markdown in cmake configuration

beb9e0c

a7medev mentioned this pull request Sep 15, 2025

Add swift-markdown to SourceKit-LSP Windows build swiftlang/swift#84302

Open

a7medev added 3 commits September 15, 2025 21:57

Refine ParametersDocumentationExtractor to avoid mutations

7a1d592

Test parameter extraction edge cases and match swift-docc

63e9704

Extract parameter documentation with raw identifiers

83e1522

a7medev requested review from hamishknight and ahoppen September 17, 2025 20:10

ahoppen reviewed Sep 20, 2025

View reviewed changes

a7medev requested a review from ahoppen September 20, 2025 20:32

ahoppen reviewed Sep 22, 2025

View reviewed changes

Make ParametersDocumentationExtractor members top-level & remove firs…

668da58

…tTextContent parameter

a7medev force-pushed the feat/signature-help-separate-param-doc branch from 6f583dd to 668da58 Compare September 25, 2025 07:23

a7medev requested a review from ahoppen September 25, 2025 07:26

Attempt to fix Windows build

992bedc

hamishknight approved these changes Sep 26, 2025

View reviewed changes

ahoppen approved these changes Sep 29, 2025

View reviewed changes

Separate signature and parameter documentation using swift-markdown #2292

Are you sure you want to change the base?

Separate signature and parameter documentation using swift-markdown #2292

Uh oh!

Conversation

a7medev commented Sep 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Alternatives considered

Dependencies

Uh oh!

a7medev Sep 13, 2025

Choose a reason for hiding this comment

Uh oh!

a7medev commented Sep 13, 2025

Uh oh!

a7medev commented Sep 13, 2025

Uh oh!

a7medev commented Sep 13, 2025

Uh oh!

a7medev commented Sep 13, 2025

Uh oh!

a7medev commented Sep 13, 2025

Uh oh!

ahoppen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

a7medev commented Sep 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hamishknight Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

a7medev Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

a7medev commented Sep 17, 2025

Uh oh!

a7medev commented Sep 17, 2025

Uh oh!

a7medev commented Sep 18, 2025

Uh oh!

ahoppen commented Sep 20, 2025

Uh oh!

ahoppen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ahoppen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

a7medev commented Sep 25, 2025

Uh oh!

a7medev commented Sep 25, 2025

Uh oh!

a7medev commented Sep 25, 2025

Uh oh!

a7medev commented Sep 26, 2025

Uh oh!

hamishknight commented Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

a7medev commented Sep 13, 2025 •

edited

Loading

a7medev commented Sep 14, 2025 •

edited

Loading

hamishknight commented Sep 26, 2025 •

edited

Loading