Add delimiterResolvers hook for custom delimiter handling

[bxff]

Summary

This adds a delimiterResolvers hook that lets extensions customize delimiter resolution before the standard CommonMark algorithm runs. It also makes InlineContext.parts public as it's required for resolvers to function.

The Problem

Extensions that need to handle delimiters differently from CommonMark have no clean way to do it. The current resolveMarkers function enforces a strict tree structure: find a closer, look backward for an opener, build a node. Unmatched openers become plain text.

For my use case, I needed to:

• Extend unmatched emphasis markers to the block end (so *hello becomes emphasis immediately)
• Handle overlapping delimiter ranges that can't be expressed as a tree

The only way to achieve this was monkey-patching resolveMarkers and accessing the @internal parts array, which is fragile and breaks with updates.

I've built an extension that demonstrates this use case: lezer-markdown-partial-emphasis. It creates "live" emphasis that appears as you type, using the new API.

The Solution

Add a delimiterResolvers array to MarkdownConfig. Each resolver receives the InlineContext and can inspect or modify cx.parts before standard resolution.

// In MarkdownConfig
delimiterResolvers?: readonly ((cx: InlineContext) => void)[]

// In resolveMarkers()
for (let resolver of this.parser.delimiterResolvers) resolver(this)

Resolvers can replace InlineDelimiter objects with Element objects or null them out. The standard algorithm skips already-resolved positions.

Making parts public is necessary for this API to work. Resolvers need to read delimiter positions, types, and side flags, then modify the array in place.

Why This Design

I considered alternatives but they all had issues:

• Per-delimiter resolve methods: The main loop is "closer-driven" and can't handle unmatched openers cleanly
• Post-process hooks: Inefficient and can't change text already parsed as literals
• Replace resolveMarkers entirely: Breaks composition if multiple extensions need customization

This approach follows the existing wrap pattern: an array of processors that compose without conflicts. It's opt-in with zero overhead when unused.

Testing

Added test/test-delimiter-resolvers.ts demonstrating the API. The test extension shows how to access and modify delimiters added by the built-in Emphasis parser (clearing asterisks while preserving underscores).

README Changes

Regenerated the README to include documentation for the new API. This also picked up docs from an earlier commit (4d5b25c) about block context nodes and close tokens that had been missed.

[marijnh]

I'm not happy about exporting a weird data structure like InlineContext.parts. I'm guessing you hand-edited the readme rather than using npm run build-readme, since the builddocs tool wouldn't use an unexported type (InlineDelimiter) in a signature.

In general, this feels like too messy and ad-hoc an API for me to commit to. If you can formulate it in some narrower way, for example by providing some way for unclosed delimiters to be auto-closed (rather than turned into plain text) somehow, that would probably be more attractive.

[bxff]

Thanks for the feedback! Let me clear up a couple things first.

The README was auto-generated with npm run build-readme. That inline type signature for parts is just builddocs inlining the unexported InlineDelimiter class structure, not hand-editing.

The core issue is overlapping emphasis ranges that can't be a tree. For *a **b* c**, I need both:

• Emphasis wrapping *a **b* (positions 0-7)
• StrongEmphasis wrapping **b* c** (positions 3-11)

Standard resolution picks the *...* first, leaving **... as plain text. A post-resolution hook can't fix this, the overlapping region is already locked in as literals. That's why I need pre-resolution access.

Auto-closing unclosed delimiters wouldn't help here either. That only extends unmatched openers to block end, but can't apply multiple styles to the same text region. The overlapping case needs both styles, not just extension.

Making this a built-in PartialEmphasis extension would dump 225 lines of complex, single-purpose logic on you to maintain. It solves one specific behavior and isn't generalizable.

I've implemented a tighter API that doesn't expose internals:

interface PreResolveContext {
  readonly delimiters: ReadonlyArray<{
    readonly type: DelimiterType
    readonly from: number
    readonly to: number
    readonly side: number  // 1=open, 2=close, 3=both
  }>
  readonly blockEnd: number
  markResolved(index: number): void
  addElement(element: Element): void
  elt(type: string, from: number, to: number, children?: readonly Element[]): Element
  slice(from: number, to: number): string
}

interface MarkdownConfig {
  preResolveDelimiters?: readonly ((ctx: PreResolveContext) => void)[]
}

Resolvers get an immutable delimiter list and explicit methods to mark resolved or add elements. No raw arrays, no class exposure. Zero overhead when unused.

Would this narrower approach work for you?

[bxff]

Hey @marijnh, just wanted to gently bump this when you have a moment. I know you're busy, no pressure at all.

[marijnh]

I have been looking back at this periodically trying to like it or to find an acceptable alternative for you, but I have been successful at neither so far.

[bxff]

Really appreciate that :)